
Ask an AI agent the same question twice and you may get two different answers. That is what “probabilistic” means, and for a model it is a feature. It lets the agent weigh evidence and try another route when the first one fails.
The data underneath is a different story. When an agent asks how many active customers we had last quarter, the answer cannot change by run, by day, or by the hardware that computed it. Agents can be approximately right in how they reason. The facts they reason over have to be exactly right.
People often describe this as two worlds: deterministic software on CPUs and probabilistic AI on GPUs. I think the line is real, but it isn’t drawn at the chip layer. It is drawn at the application contract level. The agent can guess, the query engine can’t.
Why this matters more with agents
A person who sees a strange number stops and asks about it. An agent that gets a wrong number doesn’t stop. It acts on it, then does the same on the next task, and the next, thousands of times.
That raises the bar on the data layer. The more autonomy we give agents, the more we need data we can defend: the same query over the same data gives the same answer, with a record of what ran where. Speed matters too, because an agent’s task is a chain of queries and every slow link slows the agent. But fast and wrong is worse than slow and right.
Speed is the easy half of a GPU-native query engine
We built Torin, our GPU-native query engine, to run SQL on NVIDIA GPUs using cuDF. A GPU applies one instruction to thousands of values at once, so scans, filters, joins, and aggregations over large tables can finish sooner. That same parallelism is what makes exactness hard.
Take this filter:
WHERE l_quantity > 0 AND l_extendedprice / l_quantity > 1000
The first condition acts as a guard: it is there to protect the division from rows where l_quantity is zero. Queries written this way are common, and they run without error on the CPU engine today. A GPU that evaluates both sides of the AND over the whole column at once would divide by zero on those rows and fail a query the CPU engine finishes. An engine that fails queries the CPU engine finishes isn’t an upgrade.
So Torin doesn’t just evaluate everything everywhere. For each expression, it works out how the expression can fail and picks the cheapest strategy that is safe. When nothing can go wrong, it evaluates the whole column at once. When a guard is needed, it can blank out the inputs on rows the guard rules out, or copy only the rows that need the risky part into a smaller batch and put the results back afterward. The engineers who built it walk through the details in our engineering deep dive. What matters here is the rule behind them: Torin has to produce exactly the same results that execution without GPUs would.
One answer, two engines
Torin runs beside Starburst’s CPU engine, not instead of it. Trino still parses, optimizes, and schedules every query. Torin runs the operators it supports. Anything it doesn’t support yet continues on the CPU engine in the same query, and EXPLAIN ANALYZE shows what ran where.

The agent’s reasoning can vary from run to run. The rows it reasons over don’t.
That design is how we keep the contract: same SQL, same data, same governance, same answer, whichever engine does the work. By same answer, I mean the same rows a run without GPUs would return. A team can move a workload to the GPU and compare the results against the CPU run. They shouldn’t have to wonder whether the numbers changed.
NVIDIA, which contributed cuDF expertise to the work, frames the goal the same way. Todd Mostak, Senior Director of Analytics and Data Intelligence at NVIDIA, said Torin lets customers run their existing SQL on NVIDIA GPUs “without moving data, rebuilding workflows, or changing governance controls.”
Where Torin fits
Torin does its best work on heavy analytical queries: large scans, joins, and aggregations over Apache Iceberg, Hive, and Delta Lake tables in Parquet. Three kinds of workload stand out:
- Heavy analytics on big tables. Scan-, join-, and aggregation-heavy queries over large datasets, the work that keeps CPU engines busiest.
- Data preparation for AI. The aggregations, joins, and transformations that turn raw tables into model inputs are the same operations a GPU already speeds up in analytical SQL.
- Text and pattern matching at scale. LIKE and regular-expression filters over large text columns map well to a GPU’s parallelism.
It also suits teams that already run NVIDIA GPUs for AI, because the same hardware can serve analytics too. Each worker needs a supported NVIDIA GPU with at least 24 GB of memory, and 80 GB or more is recommended.
Some work stays on the CPU engine today: DISTINCT in a SELECT, user-defined functions, complex types such as ROW, MAP, and ARRAY, scans of anything other than Parquet tables, and data exchanges between workers. Those parts run on the CPU engine, in the same query.
What we’ve learned
A few lessons from building and tuning Torin that apply to anyone putting SQL on GPUs:
- Keep the data on the GPU. Moving data between host and GPU memory, and converting formats at each boundary, is expensive. When consecutive operators both run on the GPU, the data stays there. The biggest single jump in our TPC-H results came when joins moved onto the GPU, so scans, filters, joins, and aggregations could chain without leaving it. Each added operator removes another round trip.
- Feed it big, steady batches. Small files starve a GPU. Larger splits, composite splits for small files, and an SSD file-system cache on the workers all help. Single-node execution keeps intermediate data from moving in and out of GPU memory.
- Measure per query. Gains range from none to an order of magnitude. Early on, one TPC-H query ran slower on the GPU because an optimization the CPU engine relies on, dynamic row filtering, wasn’t implemented there yet. We fixed it.
EXPLAIN ANALYZEshows which parts of a plan ran on the GPU and why any part didn’t. - Give the GPU enough work. A GPU pays off when each query touches enough data to keep it busy. Small datasets and lightweight queries leave most of it idle.
The platform I want to build
Agents will push far more queries through the data layer, and some of that work will run best on the GPUs enterprises already own for AI. The platform that wins won’t make you choose between exact and fast, or between your data and your hardware. It will give you one SQL interface, one set of governance controls, and a choice of compute underneath.
The probabilistic part of your stack and the deterministic part now run on the same silicon. That makes the contract between them more important, not less.
Where Torin is today
Torin is in limited general availability for Starburst Enterprise. The full list of supported operations, data types, and requirements is in the GPU execution documentation, and launch benchmarks with their methodology are in the announcement.
Further reading: our engineering deep dive on lazy evaluation on GPUs, and the benchmark methodology and results in the June engineering blog.
If you run NVIDIA GPUs for AI and want to see what your SQL does on them, talk to us about Torin.

