Context engineering is the practice of designing what information goes into a large language model so it can produce reliable, grounded answers. Think of it as wiring an AI system into your business the way an electrician wires a building. Every signal that reaches the model has been routed, filtered, and labeled on purpose. It sits between your data platform and the model’s prompt interface, and its goal is to make sure the model sees the right facts, in the right order, at the right freshness for the task.
Why context engineering matters now
For most of the past decade, enterprise AI meant a few well-scoped models running on curated training sets. That has changed. AI has pushed model use into customer support, sales, analytics, marketing, and security operations, and most of those use cases call into models that the company did not train. The question is no longer “do you have a model?” but “does the right information reach the model?”
That shift surfaces a familiar problem. A team builds an AI agent, hooks it up to a hosted model, and ships a demo that works on Tuesday. Two weeks later, finance asks why the assistant told a prospect that prices haven’t changed in six months. The answer is almost always the same. The model never saw the new price sheet. It lacked access to the context. Surveys from 2025 show that more than half of enterprise AI projects stall in production because the data feeding them is stale, fragmented, or governed poorly. The model is fine. The context is broken.
Why context engineering matters more in the AI era
That breakdown lines up with a broader trend. The center of gravity in data work is moving from BI dashboards to AI consumers. Dashboards tolerate slow refresh and approximate joins. AI agents do not. They pick the most plausible answer in the context window, full stop, and an out-of-date row becomes an authoritative-sounding wrong answer.
For example, if the retrieval step pulls a six-month-old version of a contract, the model will quote the six-month-old terms with full confidence. If the schema description in the prompt mislabels a column, the agent will issue SQL against the wrong field and return a number that looks reasonable but is wrong.
But it’s not all bad. This represents both a problem and an opportunity. With a clean, governed, queryable data foundation, the same model becomes a fluent analyst. It can reach across a customer’s billing history, support tickets, and product telemetry, blend them in a single answer, and cite each source. That kind of cross-domain reasoning is what every executive demo promises, and it is exactly what poor context engineering prevents.
This means that every team shipping AI features now has to do the engineering work that used to live in the BI layer, plus a lot more.
What context engineering enables
This is where context engineering shines. A working context layer powers more than just chatbots. The same primitives feed AI search, automated incident response, code-aware copilots, agent-to-agent workflows, and old-fashioned dashboards too. The team that gets context right once gets it across many applications, which is why most strong AI and analytics solutions treat the context layer as foundational rather than as a per-app concern.
Common patterns that benefit include:
- Conversational analytics over enterprise data, with answers grounded in current rows rather than yesterday’s extract.
- Retrieval-augmented documentation that returns one answer plus its source links, instead of three pages of search results.
- Decision agents that read live operational data, propose an action, and write back to the system of record. These look a lot like the building data applications pattern that platform teams already use for non-AI workloads.
- AI evaluations and red-team workflows that depend on reproducing the exact context an agent saw at the time of a decision.
That breadth is why context engineering is not a feature of one product. It is an architectural posture that touches storage, query, governance, and the prompt layer.
The technical challenges teams keep hitting
Context engineering is not without its challenges. It is, after all, something that needs to be engineered. This means that it needs to be intentional. It might look easy in a hackathon, but it can get hard in production.
Five problems show up almost everywhere.
Data freshness quickly reveals problems in your ETL pipeline
AI is unforgiving of ETL pipelines with data issues. Once an AI agent depends on the latest row, every overnight ETL job becomes a customer-facing dependency. A nightly extract-load that used to be fine for a Tuesday-morning report is now a 22-hour staleness window for an agent that answers customer questions in real time. Teams that started with an open data lakehouse architecture tend to get out of this trap fastest, because they can query source-of-truth tables directly instead of waiting on a copy.
Fragmented sources make AI less useful
Access to data is the lifeblood of AI, and nowhere is that more true than with contextual data. AI needs it, and it needs it wherever that context lives. The problem is that enterprise context rarely lives in one place. Customer data sits in a CRM, billing data in a warehouse, product telemetry in a lakehouse, support transcripts in object storage. Every retrieval-augmented call has to fan out across at least three of those, and every fan-out adds milliseconds and failure modes. A federated query layer that can read all of them in one statement collapses that latency budget into something an agent can actually use.
Data governance gaps compound at the prompt boundary
Most data governance was built for reports and people. AI agents read more rows, faster, and surface them to more people, including customers. Row-level security, column masks, and data-classification tags must travel with the data into the prompt, or the agent will leak something it should not.
One solution to this is data products. A clean data products approach helps here because it pushes the policy decisions to the producer, not the consumer, and the same controls protect every downstream agent automatically.
Schema drift quietly breaks every cached embedding
Every time an upstream team renames a column or splits a table, every embedding and every cached prompt template that referenced the old shape starts producing junk silently. The way AI Agents operate is critical here. If agents don’t have enough data, they infer an answer based on the best data available, and this can lead to errors. Treat schema as a contract, version it like an API, and you avoid the slow-motion regression. Open table formats give you a clean place to do this; teams comparing open table formats usually pick on this dimension as much as on performance.
Lineage complexity slows every postmortem
Data lineage is a key part of the solution, as it enables critical traceability. When a model produces a wrong answer, the on-call asks where each fact in the context came from. Without lineage that ties a sentence in the prompt back to a row in a table, every postmortem becomes archaeology. A federated query layer that already produces lineage for analytical queries gives most of this for free
How to get started with context engineering
The fastest teams treat context engineering as a real platform investment, not a retrieval-glue script. The order of operations matters.
Start with one well-scoped use case
Pick a single AI feature with clear input boundaries. For example, you might choose a support assistant for one product line, a sales-research agent for one segment, a code-review bot for one repository. Resist the urge to build a generic context platform first. The constraints from the first use case will tell you which sources matter and which can wait. The same scoping logic that makes data product solutions work in analytics applies here: small, clearly owned, well-documented.
Build retrieval on top of governed source-of-truth tables
Do not copy data into a vector store and call it the context. The vector store should hold pointers and embeddings; the answers should come from the same governed tables your BI runs on. That is exactly what an open data lakehouse architecture is designed for, and it keeps the AI layer aligned with the rest of the business.
Treat the prompt as code
Version-control prompts, retrievers, schema descriptions, and tool definitions together, in the same repository that ships your application. Diff them like code. Without that, you cannot tell whether last week’s regression came from a prompt change or a data change. The discipline looks a lot like the one teams already apply to ELT data processing pipelines, and the same review practices carry over.
Plan for evaluation from day one
Set aside a small, frozen evaluation set with known-correct answers. Run it on every prompt change and every schema change. The teams that skip this never know whether their context layer is improving. They only know whether the latest demo was lucky. Evaluation also gives you a way to argue about ROI later, which matters when an executive asks why context engineering is its own headcount line and not a side project, the same question they ask about why a dedicated analytics platform pays back.
Layer in lineage and observability
Once the use case works, instrument it. Log the rows that were retrieved, the schema version they came from, the prompt template that wrapped them, and the model output. When something goes wrong six weeks later, you will be able to reconstruct the answer instead of guessing. Tools that already produce lineage for analytical queries give you most of this without bespoke instrumentation.
Treat AI consumers as first-class data users
Add AI agents and pipelines to the same data catalog and access reviews you run for analysts. They are users. They have permissions, audit trails, and SLAs. The teams that get this right early avoid a lot of compliance pain later, and the work plugs into existing AI solutions patterns rather than creating a parallel governance regime.
Context engineering as an organizational capability
Success with context engineering looks less like a clever retrieval algorithm and more like a discipline. Organizations that treat it as a platform investment, not a feature, find that the same context layer powers their next ten AI applications. Their AI features stop drifting on their own, their agents stop hallucinating about pricing, and their data team stops being the bottleneck for every new model. That is not a model problem. It is a data infrastructure problem with a familiar shape.
Where Starburst fits
Starburst and the Icehouse architecture were designed for exactly this workload. Starburst gives AI applications a single, governed query surface across object storage, warehouses, and operational systems, so a retrieval call hits a lakehouse table the same way a BI query does, with the same row-level controls. Icehouse pairs Trino with Apache Iceberg to give AI teams an open, versioned source of truth that does not lock them into one vendor.
What customers are asking for next is tighter integration between the query layer, the semantic layer, and the AI runtime. This means automatic context retrieval from a natural-language prompt, agent-friendly schema descriptions, and built-in lineage that travels with every retrieved row.
If you are starting an AI initiative this quarter, book a Starburst demo to see how the same lakehouse that powers your analytics can become the context layer for your AI workloads.
