
We’ve taken Iceberg and made it better, with LakeOps. Iceberg already gives you an open table format, multi engine access and a lot of flexibility. What it does not give you is tables that look after themselves.
What problem does this solve? With Iceberg, every write leaves something behind, causing new snapshot, a handful of small files, a few more manifests. Left alone, queries slow down and storage costs creep up. Most teams find out when a dashboard gets slow or the bill jumps, which is a rough way to learn that a table needed attention.
LakeOps is designed to solve this problem. Here’s a look at the LakeOps benefits at a glance.

The maintenance list never ends
Why are you doing this? If you run Iceberg in production, you already know the maintenance checklist runs long. A typical list looks something like the following.
| Task | Purpose |
| Compaction | Merge small files into right sized ones so scans do less work. |
| Snapshot expiration | Expire old snapshots so storage can actually be reclaimed. |
| Orphan file removal | Remove files in the table’s storage location that no table metadata references, such as leftovers from failed writes. |
| Manifest file rewriting | Consolidate manifests so query planning stays fast. |
| Delete file cleanup | Handle delete files, including dangling deletes that no longer apply to any data file. |
Can you run Trino or Spark jobs for this? Yes. For a handful of tables it works fine. The trouble starts when the handful turns into thousands, and that is where the real problems show up.
Where hand written maintenance breaks down
The jobs themselves are not the hard part. Everything around them is, and it causes the following problems.
It becomes a full time operation
Someone has to build, schedule, monitor and fix pipelines for thousands of jobs across thousands of tables. Every new table is another job to configure, and every failure is a ticket. What started as a few scripts quietly turns into a team’s worth of work.
You pay for compute on tables that did not need it
A scheduled job has no idea whether a table needs work. It runs anyway. So a big chunk of your maintenance compute goes to tables that were perfectly healthy, while the ones that really needed attention wait for their turn on the schedule.
Stability gets harder as you scale
Maintenance jobs commit to the same tables your writers commit to, and that means commit conflicts. Somebody’s job fails, gets retried and fails again. On top of that, at scale these jobs flood your catalog with listing and API calls, competing with the reads and writes you actually care about. Now maintenance is not just costing you money, it is putting pressure on the stability of the catalog itself.
You basically need to be an Iceberg expert
Getting all of the above right means knowing what to compact and when, how much to commit at once, and how hard it is okay to hit the catalog. Then you have to keep tuning it as tables grow and change. Maintenance often gets configured once and never revisited, even after the table outgrows the original setup, and doing it wrong at scale can put table correctness at risk, not just performance.
And you cannot see any of it
Engineering teams spend real money moving to Iceberg, and then have no tangible way to show what that investment bought them. You do not know a table is unhealthy until an internal customer is yelling at you about a slow dashboard or a slow query. By then it has usually been building for weeks.
The image below shows thousands of small files, stale snapshots and orphan files, with the worst offenders in red.

LakeOps takes Iceberg maintenance off your plate
LakeOps is an intelligent Iceberg execution engine. It runs on its own dedicated engine, keeps your tables optimized and does it hands off. Four things make it different from a pile of scripts.
Let’s look at five reasons why LakeOps helps with Iceberg table maintenance.
Reason 1 – Automated maintenance across every maintenance type
LakeOps covers compaction, snapshot expiration and manifest rewriting, dangling delete cleanup and orphan file removal. It checks your tables, plans the work, runs it and commits the results. No pipelines to build, no schedules to tune, no Spark or Trino maintenance jobs to own.
The image below depicts the latest run of every maintenance type, per table, with full maintenance history.

Reason 2 – Table state aware, so it only runs what a table needs
Before LakeOps does anything, it looks at the state of the table. Then it decides whether the table needs maintenance at all, and which types. Healthy tables are skipped. There is no blanket job touching everything on a timer.
This leads to two key outcomes:
Reduced operational overhead
Nothing to schedule, retune or babysit as tables grow and change.
Reduced compute
Work happens only where it is needed, not on tables that were already healthy.
The illustration below shows a fixed schedule job running on every table, compared with LakeOps running only on the tables that need work.

Reason 3 – Built for stability: fewer commit conflicts, lighter catalog load
This is the part that is easy to underestimate until you run maintenance at scale. Two things can go wrong, and they are related.
Your catalog
A scheduled job against a catalog like AWS Glue lists tables and makes a lot of API calls every time it runs. LakeOps keeps the discovered table list cached in memory instead of re listing the catalog every cycle, and uses batching and chunking when it commits to the catalog to cut the number of requests it makes. Maintenance stops competing with your reads and writes for catalog API bandwidth, and you are less likely to run into rate limits. It is about stability, not just cost.
The illustration below shows catalog API requests per maintenance cycle, with a cached table list and batched, chunked commits.

Your table writing
Iceberg uses optimistic concurrency. When a maintenance job and a writer commit to the same table at the same time, one of them can lose. LakeOps reduces those collisions in three ways:
Targeted jobs
Because it is table state aware, LakeOps only works on tables that need it, so there are fewer chances to collide with a writer in the first place.
File age aware compaction
Older data files are compacted more aggressively. Recent files, where writers are still active, are handled more gently. The data most likely to be in flight is left alone the longest.
Batched, rate limited commits
Results are grouped before they are committed, large tables are committed in controlled chunks, and LakeOps limits its own commit rate to ease contention on the catalog.
The illustration below shows file age aware compaction. Older data files (darker blocks) are compacted more aggressively, and recent files more gently.

Reason 4 – Visibility into the health of your lakehouse
Engineering teams spend a lot of money moving to Iceberg. Until now, quantifying the benefit meant waiting for somebody to complain. LakeOps gives you a proper observability layer for Iceberg:
Table level
Metadata metrics, physical table metrics, trend graphs and health metrics for every table, including size, record count, files compacted, storage reclaimed and errors from the last run.
Lakehouse level
One view across all your catalogs and tables, so you can see overall storage savings and performance without opening tables one at a time.
Health thresholds
A table gets flagged when a health metric crosses its threshold, so you hear about it before your internal customers do.
UI or MCP
Look at it in the interface or ask for it from your own agents.
The image below shows table health, file size, storage reclaimed and compaction activity for a single table.

Reason 5 – A dedicated engine, whoever writes the data
LakeOps runs on a dedicated engine built for Iceberg operations. On Galaxy it runs on serverless compute, so there is nothing to size. On Starburst Enterprise it runs on dedicated compute. Either way, it maintains tables regardless of who wrote them: Flink, Spark, dbt or Starburst. No migration required.
LakeOps vs hand written jobs, side by side
| Trino or Spark jobs | Starburst LakeOps | |
| Triggering | Fixed schedules and scripts | Driven by table state |
| Coverage | Whatever you script | Compaction, snapshot expiration, manifest rewriting, delete cleanup, orphan file removal |
| Compute | Runs on tables that did not need it | Runs only where a table needs work |
| Commit conflicts | Retries and manual tuning | Targeted jobs, file age aware compaction, batched and rate limited commits |
| Catalog load | Repeated listing and API calls | Cached table list, batched and chunked commits |
| Expertise | You tune what to run, when and how much | LakeOps decides |
| Observability | Build your own | Table and lakehouse health, in the UI or over MCP |
| Operations | You own the jobs, the tuning and the failures | Hands off |
Now Available in Starburst Galaxy and Starburst Enterprise
LakeOps is now available for both Starburst Galaxy and Starburst Enterprise. Let’s look at the two deployments side by side.
Starburst Galaxy
- Serverless
- Runs on serverless compute, nothing to size or manage
- Register your catalogs and LakeOps takes it from there
- Table and lakehouse observability in the UI or over MCP
Starburst Enterprise
- Self hosted
- Runs on dedicated compute in your own environment
- Catalogs registered through the Starburst control plane
- Table and lakehouse observability in the UI or over MCP
Stop difficult Iceberg maintenance jobs
Your Iceberg lakehouse shouldn’t be difficult to maintain. It should run on autopilot, and your team should be spending its time on the work that actually moves the business forward. Not babysitting compaction pipelines. LakeOps helps make that happen.
Want to know more about LakeOps? Watch the webinar.

