When enterprise analytics lived inside a single data warehouse, governance felt straightforward because it was backstopped by centralization. A central administrator managed one catalog, one grant table, and one audit log. But as the number of data sources expanded to include cloud object stores, regional warehouses, and operational systems, that centralized model completely fractured in line with the proliferation of data sources.
How do you fix this? Attempting to fix this by copying distributed data into a new, single repository just to apply security policies creates massive pipeline delays and duplicate rules. Bringing all of those sources into one place typically isn’t an option, and that means that fixing it requires a different approach. The rise of production AI agents makes these delays unacceptable. An AI agent cannot wait for a multi-year migration project to complete before delivering an answer, nor can it operate safely if security rules are bypassed via a back-door connection to raw schemas.
Instead, modern data governance needs to operate at the access path. Rather than moving files to enforce policy, forward-looking platforms apply role-based access controls, attribute-based masking, and fine-grained row filtering dynamically at query execution time.
By wrapping distributed datasets into governed data products and managing them through a federated query engine, organizations can enforce unified security policies across both human analysts and AI agents without moving a single byte of data.
Key takeaways
- Most data is already distributed. Governance needs to match that reality.
- Enforce identity, row and column rules, and audit on the access path every query uses.
- Combine role-based access control for jobs with attribute-based policy for sensitive fields such as PII.
- Treat data products as the contract, including definitions, owners, and policies travel together.
- Roll out by domain. Classify, tag, attach policy, publish, then reuse the same pattern.
Governance broke when the platform stopped being one data warehouse
A single data warehouse made policy enforcement simple by default. That era is over. The reality of modern enterprises includes persistent data silos, multiple cloud platforms, and autonomous AI consumers that all require real-time answers to the exact same question of who can access a specific field in a specific query.
Answering that question by copying data into a central data source duplicates security rules and stalls engineering pipelines. AI workloads make those delays immediately visible, as autonomous agents cannot wait on multi-year data migration projects.
Furthermore, file-level permissions on a data lake do not equal true table governance. Storage bucket permissions can hide a file prefix, but they cannot mask a single column for one user while displaying it to another reading the exact same file. True governance requires fine-grained controls executed dynamically at query time.
Put governance on the access path
To understand why modern governance fails, you have to look at where security controls are applied. Infrastructure teams regularly encrypt cloud storage buckets and underlying disks at rest. Encryption is necessary for infrastructure security, but it is fundamentally insufficient for data governance. Storage layers operate at the file and folder level. An Amazon S3 bucket policy or file permission can grant or deny access to an entire dataset prefix, but it cannot inspect a SQL query, evaluate who is asking, or mask a single sensitive column while displaying the rest of the file.
True data governance must sit directly on the execution path that every SELECT statement travels. The grant that determines compliance is not the file-level permission checked at the storage layer, but the fine-grained rule evaluated by the query engine immediately before returning a row.
Federation and data access
By leveraging a federated query engine, platform teams establish a single, unified access layer across cloud object lakes, legacy warehouses, and operational databases without moving data. Because query federation accesses data directly in place, the query engine becomes the central point where you attach user identity, role-based access controls, attribute tags, row-level security filters, and audit logging.
This model is equally essential for open table formats like Apache Iceberg within a data lakehouse. While open formats bring warehouse-like schema capabilities to cloud object storage, file formats alone cannot enforce access rules. Governance must sit above the table abstraction layer so that column masking and row filtering execute identically whether a user queries a managed lakehouse table or a remote operational database.
Running governance in production requires moving beyond coarse, high-level permissions to enforce fine-grained controls directly on the query execution path. This framework relies on four fundamental pillars.
Identity authentication
First, start with identity resolution. Every consumer, whether a human analyst, an automated pipeline, or an autonomous AI agent, must authenticate through an identity provider that the query engine maps directly to authorized roles and attributes.
Access controls
Second, platform teams must combine role-based and attribute-based access controls. Role-based access control (RBAC) answers what a user or service is allowed to do, granting broad access to catalogs, schemas, or specific query jobs. RBAC alone is too coarse for sensitive data like personally identifiable information. Attribute-based access control (ABAC) answers what a specific field contains and who is allowed to view it. By attaching metadata tags such as PII or confidential to data columns, platform engineers can apply dynamic masking rules platform-wide without manually rewriting permission grants for thousands of individual tables.
Filtering
Third, the execution engine applies dynamic row filtering and column masking at query time. When a query executes, row filters automatically strip out records that the caller is unauthorized to view, such as filtering regional sales data by geographic clearance. Simultaneously, column masks replace sensitive string values with hashed or redacted placeholders in real time based on ABAC policies.
Audit logging
Fourth, platform engineers rely on centralized audit logging and lineage. Every query, filter execution, and column mask must be recorded in a centralized audit log to prove regulatory compliance. Data lineage sits directly alongside audit records, mapping how tagged columns flow into downstream data assets. Without clear lineage tracing, security rules quickly drift out of alignment whenever underlying database schemas change.
Enforcing these controls at access time ensures consistent security whether data sits in cloud object storage, an enterprise data warehouse, or an on-premises relational system. Starburst provides these exact RBAC, ABAC, dynamic masking, and audit capabilities across both fully managed cloud environments and self-managed deployments, giving engineering teams a single control plane across their entire distributed estate.
Data products as the unit of governance
Once identity resolution and ABAC policies live on the query path, the data product becomes the container that packages those controls into something reusable. A raw database table carries no usage contract, but a data product binds a dataset to an assigned domain owner, clear metric definitions, a certified grain, and the security rules that travel directly with it. Consumers subscribe to the certified product rather than fishing through ungoverned schemas.
Data products that carry access policy with them package RBAC, ABAC, PII masking rules, and lineage into a single governed asset. The federated query engine enforces this embedded contract automatically at query execution time, ensuring that analysts, notebooks, and AI models cannot bypass platform security logic.
Connecting governance directly to data products also guarantees data quality for AI workloads. Because an autonomous AI agent reading an unverified grain will deliver wrong answers with high confidence, certifying a product before widespread publication ensures both human analysts and agents query the exact same ground truth.
Platform teams should publish the minimum viable data product required to serve a specific domain, verifying that the security contract holds before scaling access to broader consumer groups.
Federation and the data lakehouse
Modern enterprise data architectures rely on a hybrid reality. Organizations maintain a data lakehouse to store primary analytical datasets in open formats like Apache Iceberg, alongside live operational databases and warehouses that cannot be easily migrated. Security policies must cover both environments equally. Query federation provides the unified access layer across these distributed systems, while the lakehouse serves as the storage layer for tables you choose to land.
In this paradigm, Starburst provides a single context layer for analytics and AI, connecting disparate storage platforms to shared business definitions and access paths. The operational rule is straightforward:
- Land data in the data lakehouse when long-term reuse and advanced table features justify it
- Query remaining operational data in place using federation
- Enforce the exact same identity checks, policy tags, row filters, and column masks across both.
A practical rollout sequence
The following represents a practical approach to implement the plan suggested above.
- Select a Single Domain. Begin with one specific business domain that has a clear owner and an active data access bottleneck.
- Classify and Tag. Map the underlying datasets, identify sensitive fields, and apply metadata tags to columns containing PII or confidential information.
- Attach Policies. Configure role grants and attribute-based masking rules against those tagged fields at the query engine layer.
- Publish the Data Product. Package the dataset, definitions, and security rules into a single certified data product.
- Connect Workloads. Point one human analytics workload and one AI agent or notebook directly at the published product.
- Review Audit Logs. Check the audit log during the first week to verify that the AI agent queried the certified product and that column masks fired correctly. If notebooks or scripts are still accessing raw schemas, close those unauthorized paths.
- Scale the Operating Model. Repeat this classification, tagging, and product contract sequence across adjacent domains using the exact same query engine to prevent policy fragmentation.
Avoid launching a massive, company-wide catalog initiative before governing a single domain. Broad cataloging programs often stall under their own administrative weight. A completed, fully governed domain establishes a proven operating model, whereas an unfinished enterprise catalog delivers no security value.
Next steps
Want to know more about governed data access across regions? Check out this webinar: Unlock Secure and Compliant Data Access and Sharing Across Regions.
FAQs
Where should governance be enforced if data stays in multiple systems?
Governance must be enforced on the central query and access layer that every consumer uses. Storage-level encryption remains necessary for infrastructure protection, but it cannot replace dynamic row filters, column masks, and audit logging executed on the SELECT path.
How do RBAC and ABAC work together on a data platform?
Role-based access control (RBAC) grants user roles permission to access broader catalogs, schemas, and data products. Attribute-based access control (ABAC) applies metadata tags to sensitive fields so that PII is masked dynamically based on user attributes, eliminating the need to write custom grants for individual tables.
How does file governance differ from table governance at query time?
File permissions can only hide or expose entire storage objects or folder prefixes. Table governance at query time evaluates SQL requests to mask specific columns or filter specific rows for one role while allowing another role to view the full dataset.
How do data products make governance usable for AI agents?
Data products bind business definitions, domain ownership, and security policies directly to a certified dataset. AI agents query the product under the exact same access rules as human analysts rather than scanning raw schemas through ungoverned connections.
What is a sensible first domain to govern end to end?
Select a single business domain with a clear owner, tag its sensitive fields, publish one governed data product, connect one human analyst alongside one AI notebook, and review the generated audit trail to verify policy execution before expanding across the enterprise.
