Starburst: Governance Is Moving to More of a Feature Play
Starburst Data, at its Starburst Galaxy Launch Week digital event, announced the newest release of its Galaxy cloud data lake analytics platform. While a number of new features were announced, the one that sticks out is the public preview of Galaxy’s new governance layer, called Gravity. Starburst says that Galaxy provides unified data discovery, governance and data sharing. Combined with its underlying Trino engine’s ability to query any number of backend data sources, it’s clear that Starburst wants Galaxy to be the single point of contact for doing anything with data, including querying, analyzing and managing it.
Must read:
- Starburst Strengthens Data Governance Capabilities
- Starburst’s CEO Decries Big Data Lies; Touts Data Truths
Of course, a number of pure-play companies already vie for enterprise data governance and data catalog budget dollars and until recently, the assumption has been that such companies would partner with the data platform players for an integrated query and governance solution. But that seems to be changing. In the last few months, the data platform companies have been adding more of their own governance and catalog capabilities and there’s been consolidation in the standalone data governance space.
“The two sides of the data governance policy ‘barbell’ are policies as a statement of data management principle and intent, and data storage access control mechanisms. Historically, the two have not intersected, but now we see the emergence of vendors slowly, somewhat clumsily, trying to converge the two” said John Wills, Founder and Principal, Prentice Gate Advisors. Wills is also the former Field CTO for Alation a post he held after serving as VP, Customer Success at Collibra. His book The Goal is Autonomous Governance: Moving Data Governance into the Modern Age was published just recently.
Cloudera has had governance built right in since it launched the Cloudera Data Platform in 2019. Databricks launched a preview of its Unity Catalog component in 2021, getting it to general availability (GA) on AWS and Azure in August of last year and on Google Cloud in March. To buttress that offering, Databricks announced last month an agreement to acquire data governance player Okera, and just last week, Informatica announced its intention to acquire London-based data governance vendor Privitar. Microsoft has been pushing its own Purview data catalog and governance suite, as well as that service’s integration with both Azure Synapse Analytics and the newly announced Fabric end-to-end analytics platform.
Check out: Informatica Launches Freemium AI-Powered Integrator
Platforms Strike Back
What’s going on here? Data discovery, data trustworthiness and data privacy have all become extra-important in the age of AI. While the “brand name” companies in the space, like Collibra and Alation, likely won’t get gobbled up any time soon, it seems the platform players are no longer content to “outsource” catalog and governance capabilities to those vendors and their competitors. Moreover, the data platform contenders may feel the need to commoditize the governance players rather than being commoditized by them. If customers use catalogs as their go-to portal for data discovery, they could end up seeing their data lakehouses and data warehouses (and their providers) as peripheral. On the other hand, building governance into data platforms means customers can discover their data where they query it, allowing those platforms to remain the center of “gravity.”
Get the background:
This puts pressure on pure plays to differentiate, by supplying data catalog and governance capabilities that add significant value to, and outdo, the data platform providers’ counterparts, essentially showing them to be entry-level offerings. The data platform companies, meanwhile, need to supply the capabilities their customers really need, combined with the convenience of “in situ” functionality. They don’t have to be the best; they just have to be good enough.
Galaxy’s Governance Gravitas
So how well does Gravity do in that pursuit? Judge for yourself. Gravity adds new capabilities that fall into four categories:
- Attribute-based access controls (ABAC), including filters and column masks, that allow customers to define which roles can see what subset of data, and whether certain columns’ data should be fully visible, or obfuscated. Together with Galaxy’s existing role-based access controls (RBAC), customers can define comprehensive policies that are then applied to specific user roles.
- Automatic data cataloging that pulls in all relevant metadata, including tables, schemas, and descriptions, as soon as users connect Galaxy to a data source.
- A universal search facility that lets users find data assets that match specific criteria. Search results apply all applicable access controls, filters and column masks so that only the subset of data and assets users are authorized to see is in fact returned.
- Data product capabilities that allow specific Galaxy schemas (i.e. collections of tables, views and materialized views) to be promoted into documented, shared and supported data products. In addition, Gravity will allow new data products to be created from any federated query — right from the query editor — and will create the necessary underlying schema and data set on the fly. The schema promotion feature is generally available while the data product creation feature, like the rest of Gravity, is in public preview.
In addition to Gravity, Starburst is announcing two other new chunks of functionality in Galaxy:
- New Warp Speed query acceleration, powered by automatic data indexing and caching technology derived from Starburst’s 2022 acquisition of Varada.
- Cross-cloud capabilities that leverage Trino’s data virtualization prowess to let customers query data across multiple cloud providers as if it had been physically ingested into the central data lake.
Dispersion Begets Integration
The cycle of best-of-breed point solutions providing needed innovation, followed by larger providers simplifying things with more broadly-scoped/end-to-end solutions, has replicated itself all over the technology world. When the sheer number of specialist players is itself an issue, customers cry out for simplicity, and vendors try to provide more capabilities under one roof.
We are in that part of the cycle right now, and the cloud providers, enterprise software behemoths, and data platform players have the upper hand. They also have pressure on them, where data governance and catalog solutions are concerned. Starburst is responding to that pressure and is betting that Gravity will have some pull.