Will Google Big Query Transform Big Data Analysis?Will Google Big Query Transform Big Data Analysis?

12 Hadoop Vendors To Watch In 2012

Google shared details Wednesday about Google Big Query, a cloud-based service that promises to bring the search giant's immense compute power and expertise with algorithms to bear on large data sets. The service is still in limited beta preview, but it promises to speed analysis of Google ad data while opening up ways to mash up and analyze huge data sets from external sources.

Google Big Query was described by Ju-Kay Kwek, product manager for Google Cloud Platform Team, as offering an array of SQL and graphical-user-interface-driven SQL analyses of tens of terabytes of data per customer, yet it doesn't require indexing or pre-caching. What's more, customers will get fine-grained analysis of all their data without summaries or aggregations.

"Fine-grained data is the key to the service because we don't know what questions customers are going to ask," said Kwek in an onstage interview at this week's GigaOm Structure Data conference in New York.

Some of Google's beta customers are uploading data to the service with batches and data streams and treating it as a cloud-based data warehouse, but Kwek said ad data would be the first priority, supporting a Google customer's need to understand massive global campaigns running in multiple languages.

[ Want more on big data in the cloud? Read Amazon DynamoDB: Big Data's Big Cloud Moment. ]

"When an advertiser wants to understand the ROI or effectiveness of a keyword campaign running across the globe, that's a big-data problem," Kwek said. "They're currently extracting data using the Adwords API, building sharded databases on-premises, doing all the indexing, and sometimes losing track of the questions they wanted to ask by the time they have the data available."

Thus, time to insight will be the biggest benefit of the service, Kwek said, with analyses taking a day or less, rather than days or weeks, when customers face extracting and structuring data on less robust and capable on-premises platforms.

A beta customer in the hospitality industry is using Google Big Query in a revenue-management application, mashing together ads data, reservations data, and property inventory information and serving up interactive dashboards using Google App Engine to tap into Google Big Query analyses.

"Nontechnical people can now access this data, so regional sales managers can now start their day looking at these interactive dashboards, and they base their conversations on the insight," Kwek said.

Another beta customer is We Are Cloud, a French business intelligence vendor that is using Big Query as a high-scale, backend data-management platform for its own query, analysis, and data-visualization capabilities. Racheal Delacour, We Are Cloud's CEO and founder, touted Big Query as an accessible and time-saving service that will free midsize and larger enterprises from concerns about running big-data platforms and the skills gap associated with that arms race. That's been the case for We Are Cloud, which now has 15 terabytes of data stored on Google Big Insights to support customer big-data analyses.

"One analyst can now complete a very large BI project," Delacour said, noting that her company is now delivering business dashboards based on one billion rows of data. "With powerful backends like Big Query and Hadoop [available in the cloud], it's only the beginning, and there will be lots of openings for innovators."

No launch date has been announced for Big Query, but it will find itself in good company with cloud-based big-data services already on the market, including Amazon Elastic MapReduce, IBM BigInsights services, Microsoft Azure Data Market, and 1010data. Google's prominence in ad serving, keyword advertising, and Web analytics will surely make related analyses of Google-sourced data the biggest play for Google Big Query.

The service is fully hosted, includes replication, and will use the same encryption and security capabilities and frameworks used for Google Docs and Gmail, and customers will be able to export their data as easily as they upload it, with no threat of vendor lock-in, Kwek said.

The pay-as-you go nature of the cloud makes ROI calculation seem easy. It's not. Also in the new, all-digital Cloud Calculations information supplement: Why infrastructure-as-a-service is a bad deal. (Free registration required.)