Zero Ops
Iceberg Pipelines

Automate your Iceberg Lakehouse in Starburst Galaxy with production-scale pipelines, ready in minutes to ingest real-time data without infrastructure complexity.

Let Starburst manage your Iceberg pipelines!

Start for Free with Starburst Galaxy Up to $500 in usage credits included

"We needed a solution that is scalable, wouldn't lock us into a specific vendor, and is cost efficient and easy to manage. with the Starburst partnership its lakehouse platform, we have unlocked data for all employees to focus on providing near real-time value, from 50 PB of streaming fata and growing, to our customers and spend less time on data management."

Brian Kidwell

CEO and Cofounder, Going

Read morechevron_right

Enable rapid data discovery and unification.

Starburst Galaxy delivers always-ready data, unified across sources and formats, for faster time to insight.

  • File Ingest ingests batch files from S3 into Iceberg tables with automatic schema inference and evolution.
  • Streaming Ingest brings real-time Kafka data into raw Iceberg tables with low-latency delivery, built-in backfill, and fault tolerance.

Activate shared insight through declarative, reliable pipelines.

Starburst Galaxy aligns data producers and data consumers with trustworthy, analysis + AI ready tables by default.

  • Raw Tables serve as structured landing zones for ingested data, enabling modular ownership across teams.
  • Live Tables define reusable, declarative transformations that create governed, analysis-ready Iceberg outputs.

Build trust in data with automated lifecycle management.

Starburst Galaxy builds reliability and compliance into the lakehouse through continuous, automated optimization.

Live Table Maintenance continuously optimizes performance and integrity through serverless:

  • Compaction
  • Statistics Collection
  • Vacuuming Expired Snapshots
  • Orphan File Removal

Real-Time Streaming Ingest

Ingest up to 100GB/second of Kafka topics, land in Apache Iceberg, transform, and govern to start querying within a minute.

Fully managed system

Avoid high operational overhead and fragility. Starburst Galaxy has no infrastructure to manage, error handling (DLQ), snapshot rollback, or schema alerts.

Lower Cost

Starburst Galaxy is up to 12x cheaper than alternatives, and pricing scales linearly alongside no idle infra costs.

Proven scale + smart backfill

Is your current system slowing time-to-insight and lagging during usage spikes? The Starburst Galaxy is proven at production scale, with parallel backfill, recency bias mode, and 1-minute ingestion-to-query.

No Complex Set-Up & Config

Starburst Galaxy offers point-and-click ingest, automatic compaction handling, and serverless mode. Get rid of multiple tools.

File Ingest

Continuously ingest files from cloud object storage into Iceberg tables - no pipelines to build, no infrastructure to manage.

Declarative, fault-tolerant ingest

Avoid manual batch ingest pipelines that are brittle and slow: File Ingests takes in JSON from S3 into raw Iceberg tables with schema inference, dedupe, and error handling.

Trust what lands

Eliminate errors and duplication through write-once behavior and schema validation for raw table hydration, making ingestion auditable and safe.

Zero infra to run

No orchestration and compute clusters to manage: Galaxy handles polling, ingestion, and Iceberg table creation automatically.

Flexible triggers

You don’t have to roll your own notification systems: Starburst Galaxy supports both S3 event-based and polling-based ingest for high confidence in file change detection.

Live Table Maintenance

A fully managed, serverless service that continuously optimizes live Iceberg tables with automatic compaction, snapshot expiration, orphaned file cleanup, and NDV statistics collection.

Keeps Iceberg fast

Prevent query performance from degrading over time: Starburst Galaxy auto-compacts small files, prunes metadata, and maintains column stats for faster joins and filters.

No ops required

Typical Iceberg maintenance is agonizing and manual: Starburst Galaxy’s fully managed optimization runs continuously without tuning, clusters, or jobs to schedule.

Built for trust

Data consumers can rely on your pipeline’s freshness and consistency: Starburst Galaxy automates retention, snapshot expiration, and cleanup so tables remain lean, queryable, and audit-ready.

Enterprise-Grade Data Security

Starburst Galaxy offers built-in security controls to protect all of your data, including support for enterprise-grade features like RBAC/ABAC, data encryption, and data masking.

  • Authorization
  • Access Management
  • Data and Network Security

A reliable partner for your production-level needs, no matter the scale.

99.5%

Uptime SLA

24/7

Expert Support

30

minute response time

AICPA SOC LogoISO 27001HIPAAGDPR

Explore Managed Iceberg Pipelines in Starburst