Share

Linkedin iconFacebook iconTwitter icon

More deployment options

Our CTO, Martin Traverso, and SVP of Engineering, Jitender Aswani, recently argued that AI agents are outgrowing the converged data platform. That changes the future of the lakehouse. Agents need governed access to data wherever it lives, and fast compute to match. A lakehouse alone can’t give them that. They’re right, and it’s only the first step.

Take the fraud agent from their post. It pulls a flagged transaction, checks the customer’s history, finds the open case, and joins all three. The answer comes back in seconds. The analyst who asked checks every step anyway.

That’s the story at most enterprises we talk to. Agents are everywhere. Autonomous agents are almost nowhere. This will continue until teams can defend what their agents do. That takes answering three questions:

  1. Did the agent properly interpret the data?
  2. Did the agent follow the right access patterns?
  3. Can we prove exactly what it touched and what it did?

This comes down to three core factors: meaning, access, and proof. These three things are holding back the progress of autonomous agents.

Accuracy isn’t enough without defensibility

Let’s look at an example. Say that an analyst is using an agent that is accurate. Now imagine an auditor asks the analyst to stand behind the answer. The agent works across PostgreSQL, Snowflake, and Salesforce.

How does this relate back to our three factors?

  1. Meaning. Which definition of “flagged” did the agent use? Fraud’s definition or Compliance’s definition? She can’t say.
  2. Access. Did it stay within what she’s allowed to see? Each system checked its own rules. None checked the whole task.
  3. Proof. What exactly did it read, and in what order? Three systems kept three logs. Nothing ties them together.

Defensibility stops at the platform’s edge

The core problem here is a tension between our three factors. Few enterprises run on one platform and the mix is only growing. Databricks and Snowflake sit side by side, with PostgreSQL, and Salesforce beside them. Multiple data sources are the norm.

The challenge is that each platform can only vouch for what happens inside it, and that impacts all 3 of our factors. The fraud agent crossed three sources in one task. Each can vouch for its own step. No one can vouch for the whole. 

What has to sit above

The reality is that agents cross boundaries, and they always will. We want them to, and need them to, but the unified lakehouse model wasn’t designed for this reality. To fix this enterprises need a layer above any single platform: the intelligence layer. 

Enterprises need a layer above any single platform: the intelligence layer. It holds what your business means. It controls who may see what. It records what every agent did, across every system it touches. That makes an answer defensible, not just accurate.

Build it once, above your data sources and platforms, and all three factors – meaning, access, and proof – get easier. 

Run the fraud agent again with that layer in place. “Flagged” means what fraud ops certified. Access is checked before the Salesforce read. One trace spans all three systems. When the auditor asks, the analyst can stand behind the answer.

What we believe

Agents earn autonomy when their answers are defensible. When this happens, people stop checking every step, and the return on AI starts to show. Earlier this year, we introduced the Enterprise Intelligence platform to solve exactly this problem.

We’ll show you how at Launch Point. Wednesday, November 18, at 12 p.m. ET. Register now.

Start for Free with Starburst Galaxy

Try our free trial today and see how you can improve your data performance.
Start Free