Choosing the Right Python Path for Starburst: From DataFrames to Production Pipelines

Python-first teams don’t want to bounce between SQL, notebooks, and orchestration tools just to get reliable pipelines into production. But “using Python with Starburst” isn’t one path, and picking the right one matters for performance, portability, and how the pipeline gets maintained.

Starburst supports a few different ways to work with Python: Ibis and PyStarburst as DataFrame APIs, the Trino Python client for direct SQL, and Python UDFs for custom logic. Python UDFs in Starburst Enterprise are currently a public preview and run in a sandboxed environment, so they are best suited for self-contained custom logic rather than code that depends on external files, network access, or arbitrary Python packages.

In this hands-on workshop, we’ll compare them, then build a real pipeline using the Ibis DataFrame API. 

You’ll learn how to:

• Know when to use Ibis, PyStarburst, the Trino Python client, or Python UDFs, including the constraints of Python UDFs
• Express transformations in Python with a DataFrame API while pushing execution to Starburst
• Turn exploratory Python work into a repeatable, maintainable pipeline