layers of ice in front of sunset

Iceberg Updates

  • Added support in Spark for storage-partitioned joins. Storage partitioned joins are like bucketed joins, but are generic and can use any matching partitioning. (Anton Okolnychyi)
  • Added Spark changelog readers, to make Iceberg tables readable as CDC streams. (Yufei Gu)
    Fixed an important NaN bug (#6517) (Russel Spitzer)
    Updated Arrow to automatically set tuning parameters that greatly affect performance. (Anton Okolnychyi)
  • Added a position deletes metadata table that will be used for delete file compaction (Szehon Ho)
    Extended branch commits to all remaining commit operations. (Namratha Keshavaprakash & Amogh Jahagirdar)
  • Added support for reading branches and tags using VERSION AS OF syntax in Spark (Jack Ye)
  • Added a catalog implementation so you can read Snowflake’s internal Iceberg tables from other engines (Dennis Huo)
  • Added support to write Avro GenericRecord to Iceberg tables in Flink (Steven Wu)
  • Added optimization to detect filters that are completely pushed down and skip evaluating them in Spark. (Anton Okolnychyi)

PyIceberg updates

Version 0.2.1 was released. This hotfix release fixes an issue that caused tables partitioned by date not to work. For more details, please refer to the PRs:

More information can be found on the project site and the installer can be found here

Iceberg in the industry

Blogs from the community

Iceberg in the news

Keep up to date on all things iceberg