When it comes to AI, we all seem to know what’s missing from our workflows. The problem is that almost no one seems to know how to get it.
This is backed by data. A Harvard Business Review Analytic Services survey sponsored by Cloudera reported that only 7% of the enterprises it asked said their data is completely ready for AI.
The other 93% aren’t missing a better model. They’re missing context. Specifically, they’re missing business meaning not currently captured in any data source.
And that’s where the problem comes in. What is that missing context?
The answer lies in the delta between data and humans. In this article, we’ll discuss how this context has always existed, and how AI has just made it plainly (and painfully) visible. We’ll then discuss how context engineering is the real work of AI, and how to plan for and enable it across your company.
Finding out where your context is hiding
Everyone is suddenly talking about context, and with good reason. A lack of context is quickly becoming a bottleneck to AI success in production.
AI agents powered by large language models (LLMs) that seem to work well as a prototype fall apart when ordinary users get their hands on them. Suddenly, faced with real questions from real people with real problems, the agents return irrelevant, inaccurate, or even fictitious data.
What’s frustrating is that this context exists in your organization. The problem is that your AI agents can’t find it.
Once again, the data backs this up. A survey by Deloitte found that nearly half of organizations cite searchability (48%) and reusability (47%) of data as challenges to their AI automation strategy.
This hidden data exists in one of two places. The first can be reached; there just isn’t a path to it yet.
The second is the harder problem. It’s context that exists in a form AI agents can never reach.
The locked library
Imagine a library behind a locked door. It’s a treasure trove of knowledge. Many people, though, don’t know it exists. And those who stumble across it can’t get in.
These are your organization’s data silos. Everyone has them. The good news is, they’re accessible if you know where to look and how to get through that door. For data silos, this means using data connectors to ensure the data is centrally discoverable, queryable, and usable by other teams.
Like a long-lost library, the data in these data silos might need a little work before it’s usable. You have to dust off the books and put them in the correct order; in other words, clean and fix your data.
But the data’s there. Your AI agents just need a way to find it.
The unwritten book
Now, imagine trying to find a book that doesn’t exist yet. It exists only as an idea in someone’s head. We might speak about it as a possibility, but it doesn’t exist in an accessible state. It’s still all in the author’s head.
This is how institutional knowledge exists in every organization, distributed across the brains of your workforce.
In the past, most of us didn’t take care to write this institutional knowledge down. We didn’t need to. We knew we could trust a table because someone we trusted created it. Meanwhile, everyone calculated revenue by region using a spreadsheet that one specific employee managed. The mechanism for that management were never written down because they didn’t need to be.
This approach worked for decades, but it falls apart in the AI age because it isn’t a practical or scalable solution that works in the way that AI agents require.
Unfortunately, for all of us, the unwritten book, not the locked library, is the most common stumbling block in companies. That’s what’s making context so hard.
You don’t want AI to engineer its own context
The temptation is to point AI at the problem and find the magic prompt that says, “solve this.”
Why not? It works well for other problems. After all, it’s thanks to LLMs that we’re now able to unlock petabytes of knowledge previously unqueryable inside unstructured documents such as PDFs and regulatory filings.
But context is different. An AI model can only work with the meaning that already exists in a place it can reach. It can summarize what’s written in a derivative sense, but it can’t accurately create net new knowledge without any guidance. Context requires context requires context, and something needs to start that chain of context. Something original, something human.
We see this all the time. In practice, there are two things that happen when AI lacks context. Both are bad.
In the first scenario, an AI agent has competing information, with no disambiguating context to help it choose between them. This might be three different values for “revenue by region” in three separate data warehouses. In this case, the AI picks one arbitrarily that might not reflect current business practices.
In the second scenario, the information doesn’t exist. So the AI infers it from other information, or tries its best. But you don’t want it to try its best, you want it to get it right.
To avoid both of these scenarios, we need humans to draw out institutional knowledge from other humans and capture it somewhere. The book, in other words, must be written.
Capturing this meaning and making it accessible to AI agents is context engineering. It doesn’t involve focusing on AI output. It goes further upstream, supplying the input that AI needs to produce accurate results.
Bringing context into your semantic layer
If “ask the human” isn’t the solution, then what is? How should this context get stored and represented?
The raw storage format will take many forms, depending on the nature of the context itself. Ultimately, all of this context needs to serve as the basis of your semantic layer.
The semantic layer is the business-friendly translation layer that sits between your raw data and the applications or agents that consume it. Context serves as the building blocks of the semantic layer, creating a single source of truth for business meaning.
There are two important practices for providing context to the semantic layer.
Ensure data access
If you have data silos, the solution isn’t yet another migration to bring your data to your AI. Rather, it’s bringing your AI to the data.
Data access and data federation enable AI agents to find and use data where it currently lives. Using technologies such as Starburst that focus on federated data access, agents can use a central data catalog or service to query data warehouses, data lakehouses, and other sources across the enterprise. If a certain data set requires better performance or governance, the data team can eventually move it into a modern data lakehouse powered by Apache Iceberg.
Write your context down
Data access is a technological solution to an age-old process problem. It’s a problem that we at Starburst have spent years dedicating ourselves to solving.
Solving for the other problem, the unwritten book, is harder. It requires a rigorous upstream exercise in context engineering. This is less of a technical problem. It’s a human problem that requires:
- Identifying what’s missing. For example, what gaps in your context are your agents currently exposing?
- Identifying the relevant subject matter experts (SMEs) for a given area.
- Getting this context in a form easily accessible to AI agents that can then prepare it for storage in your semantic layer.
- Creating processes for capturing business context going forward, so it becomes a part of your AI agents from day one.
Context comes first
In the data world, we often hunger for technological solutions to all our problems. But context engineering isn’t just about technology, at least not alone.
Technology is necessary to solve the context issue (particularly data access). But it isn’t sufficient. You also need to put in the hard, human effort of changing the way people work. Maybe most interestingly, it’s a problem that technology made evident, but the solution requires more than technology itself.
But in the AI era, context is no longer something we can let linger in people’s heads. Every organization needs processes to capture, distill, and maintain the new context we’re creating every day. The companies that do this work will reap the benefits of world-class AI agents that users can trust.
