Cloud services provider Google Cloud has expanded its BigQuery data analysis service so that it can query data warehouse services from other cloud providers, namely Amazon Web Services and (in the near future) Microsoft Azure, the company announced this week for the kick-off of its Google Cloud Next ’20 virtual conference.
“We believe multicloud is the future,” said Debanjan Saha, Google Cloud general manager and vice president of engineering, data analytics, in a press conference. The company recognized that most of its customers had data spread out across multiple cloud service providers. This is problematic insofar that “It makes it difficult for people to run their analytics when their data is siloed across multiple clouds,” he said.
The service, called BigQuery Omni, will offer the ability to query data in using standard SQL queries across multiple clouds.
The BigQuery architecture decouples compute and storage, allowing the company’s Dremel query engine, by way of the Google Anthos service, to interrogate data in other cloud services, eliminating the need to migrate entire data sets to a single host. Google maintains a multitenant service running on multiple AWS regions to perform the work — so no data egress is necessary. For query execution, the data is moved temporarily from your data storage to the Anthos-driven BigQuery clusters. Supported formats include Avro, CSV, JSON, ORC, and Parquet. The results are returned to the user’s BigQuery UI, and can be exported back to data storage.
“For example, you can use BigQuery Omni to query Google Analytics 360 Ads data that are stored in Google Cloud, and also query logs data from your e-commerce platform and applications that are stored in AWS S3. Then, using Looker, you can build a dashboard that allows you to visualize your audience behavior and purchases alongside your advertising spend,” explained a Google blog post.
Currently, Google is testing BigQuery Omni private alpha, initially working with AWS S3 and Azure Data Lake . To try it out fill out this form, and to learn more check out this Google Cloud Next ‘20: OnAir session coming up next month: “Analytics in a multicloud world.”
Amazon Web Services is a sponsor of The New Stack.
Feature image by HowardWilks from Pixabay.
At this time, The New Stack does not allow comments directly on this website. We invite all readers who wish to discuss a story to visit us on Twitter or Facebook. We also welcome your news tips and feedback via email: [email protected].