The architecture of the modern data lakehouse is shifting toward a model where metadata management is no longer tied to a single compute engine. As organizations move away from siloed environments, the need for a unified, interoperable metadata layer has become a primary requirement for scaling data operations.
It’s precisely this need that led Starburst to connect to Google Cloud’s BigLake as an ecosystem partner. With the announcement at Google Cloud Next and the transition from BigLake to Google Cloud’s cross-cloud Lakehouse, along with the introduction of the Lakehouse runtime catalog, we are pleased to take our work with Google Cloud to the next level.
Starburst at Google Next
The Starburst team recently attended Google Next and had a lot of fun talking to people about how Starburst and Google work together. Much of the talk was about interoperability, which is exciting.
Collectively, these moves represent a significant step and a deepening of our ongoing commitment to interoperability with different technologies. This evolution enables Starburst to align more closely with Google Cloud to deliver a high-performance environment for Apache Iceberg.
In this post, we’ll unpack these latest changes, reflect on the importance of interoperability, and discuss where this leaves us in the future.
Understanding the mechanics of managed metadata
Let’s start with the problem. What problem is being solved here? One traditional challenge of an open lakehouse is the administrative overhead of table maintenance. In a typical self-managed Iceberg environment, data teams are responsible for critical but labor-intensive tasks such as compaction, garbage collection, and manifest file tuning. When multiple engines access the same data, ensuring consistency across these operations becomes even more complex.
The Lakehouse runtime catalog addresses these challenges by offloading maintenance to the underlying Google Cloud infrastructure. This catalog implements the Apache Iceberg REST Catalog spec, which provides a standardized interface for metadata operations. By integrating Starburst Enterprise with this catalog, organizations can ensure that their Iceberg tables are automatically optimized for performance without manual intervention.
Interoperability in practice
In production, this integration enables a multi-engine strategy in which Starburst, BigQuery, and Spark each operate on a single copy of data. This is achieved through several key technical capabilities.
Unified Metadata Handshake
Because Starburst and BigQuery each use the same Lakehouse runtime catalog, schema changes, or data updates made in one engine are instantly visible to the other.
Advanced Storage Runtime
Starburst users can benefit from the performance enhancements provided by Google Cloud Storage infrastructure, including optimized table layouts and partitioning strategies.
Transactional Integrity Support
Support for multi-statement transactions ensures that complex ETL processes involving multiple tables remain consistent and atomic.
This level of interoperability means that a data engineer can choose how to taper their workloads. For example, they might choose to use Spark for large-scale ingestion while leveraging Starburst for high-concurrency federated queries. Similarly, they might opt to use Starburst’s data ingestion capabilities for certain workloads, all while the Lakehouse runtime manages the health of the underlying files.
Establishing a foundation for data access and data governance
The end result of this is better data access and data governance. Beyond performance, the alignment between Starburst and the Lakehouse runtime catalog provides a simplified pathway based on choice. Centralizing metadata management enables a more consistent application of security policies across an organization, all while decentralizing your infrastructure requirements. In effect, you choose which compute workloads operate on which infrastructure.
By using the Lakehouse runtime catalog as the source of truth, organizations can implement uniform access controls and track lineage across different compute environments. This shared governance model is essential for teams looking to move from experimental data projects to production-grade analytics and AI agents.
The future of compute is interoperable
Starburst’s collaboration with Google Cloud reflects a broader movement toward open, interoperable data systems. By focusing on the metadata layer, we are removing the friction that has historically prevented organizations from fully realizing the benefits of a data lakehouse.
Combining the performance of Starburst with the managed capabilities of Google Cloud’s Lakehouse provides a path forward for enterprises that prioritize flexibility and scale. This integrated approach ensures that your data architecture remains adaptable to whatever new tools or workloads the future may bring.
