Schema evolution simply means the modification of tables as business rules and source systems are modified over time. Trino’s Iceberg connector supports different modifications to tables including the table name itself, column and partition changes.
Much like a database, you perform “alters” to Iceberg tables to modify their structure. Since Iceberg is just a table format which contains metadata about a table, modifying the table is rather trivial. For more details on Apache Iceberg fundamentals, see our comprehensive guide.
Table Changes
Rename a table
alter table customer_iceberg rename to customer_iceberg_new;
Note: this is a change to the table name in the metastore and no changes will be made in the storage. So, the location s3://<bucket>/customer_iceberg will remain the same.
Column Changes
Adding a column:
alter table customer_iceberg add column tier varchar(1);
Rename a column:
alter table customer_iceberg rename column address to fulladdress;
Partition Changes
Often times a table is initially partitioned by a column or set of columns only later it’s discovered this may not be optimal. With Iceberg, you can modify the partition columns at any time.
For example, initially this table is partitioned by month:
create table orders_iceberg
with (partitioning=ARRAY['month(orderdate)']) as
select * from tpch.sf1.orders;
After reviewing query a patterns, it’s determined that partitioning by day would perform better as a majority of queries are filter by certain days. A simple alter table statement as seen below will modify the partitioning on this table from month to day:
alter table orders_iceberg SET PROPERTIES partitioning = ARRAY['day(orderdate)'];
After new data is inserted into the table, you will see a change in the data directory where the table data is stored: This flexibility is a key advantage of modern open table formats.
Notice the orderdate_month is now orderdate_day. Note that queries that filter by day will partition prune at the partition day level but the existing monthly partitions will still need to be searched. If you would like to have the entire table partitioned by day then you could recreate the table using a CTAS (create table as) from the existing table. For enterprise-scale Iceberg management, consider Starburst Enterprise.
Example to create a new table partitioned by day from the existing table:
create table orders_iceberg_new with (partitioning=ARRAY['day(orderdate)']) (as select * from orders_iceberg)
Schema evolution in Trino’s Iceberg connector is very powerful and easy to use. These types of functions were not available in Hive and database veterans will be very happy to see them added to the data lake landscape.
Frequently Asked Questions
What is schema evolution in Apache Iceberg?
Schema evolution refers to the modification of tables as business rules and source systems change over time. Trino's Iceberg connector supports different modifications including table name changes, column changes, and partition changes.
Can I rename an Iceberg table without affecting the storage location?
Yes, renaming an Iceberg table only changes the table name in the metastore. No changes are made to the storage location, so the original storage path remains the same.
How do I change the partitioning scheme of an existing Iceberg table?
You can modify partition columns at any time using an ALTER TABLE statement with SET PROPERTIES. For example, you can change from monthly to daily partitioning using: alter table orders_iceberg SET PROPERTIES partitioning = ARRAY['day(orderdate)']. After new data is inserted, the partitioning scheme will reflect the change.
What happens to existing data when I change partition columns?
When you modify partitioning on an Iceberg table, existing partitions remain unchanged. New data will use the new partitioning scheme. Queries that filter by the new partition level will still need to search the existing partitions. If you want the entire table repartitioned, you can recreate it using a CTAS (create table as) statement from the existing table.
What column operations are supported in Iceberg schema evolution?
Trino's Iceberg connector supports adding new columns using ALTER TABLE ADD COLUMN and renaming existing columns using ALTER TABLE RENAME COLUMN. These operations are straightforward because Iceberg is a table format containing metadata about the table.
To learn more about migrating from Hive to modern table formats, explore our Hadoop modernization solutions.
