blog

January 2023 - Iceberg Community News

blog-image

Iceberg Updates

  • Added support in Spark for storage-partitioned joins. Storage partitioned joins are like bucketed joins, but are generic and can use any matching partitioning. (Anton Okolnychyi)
  • Added Spark changelog readers, to make Iceberg tables readable as CDC streams. (Yufei Gu)
    Fixed an important NaN bug (#6517) (Russel Spitzer)
    Updated Arrow to automatically set tuning parameters that greatly affect performance. (Anton Okolnychyi)
  • Added a position deletes metadata table that will be used for delete file compaction (Szehon Ho)
    Extended branch commits to all remaining commit operations. (Namratha Keshavaprakash & Amogh Jahagirdar)
  • Added support for reading branches and tags using VERSION AS OF syntax in Spark (Jack Ye)
  • Added a catalog implementation so you can read Snowflake’s internal Iceberg tables from other engines (Dennis Huo)
  • Added support to write Avro GenericRecord to Iceberg tables in Flink (Steven Wu)
  • Added optimization to detect filters that are completely pushed down and skip evaluating them in Spark. (Anton Okolnychyi)

PyIceberg updates

Version 0.2.1 was released. This hotfix release fixes an issue that caused tables partitioned by date not to work. For more details, please refer to the PRs:

More information can be found on the project site and the installer can be found here

Iceberg in the industry

Blogs from the community

Iceberg in the news

Keep up to date on all things iceberg

Careers

Senior Software Engineer, OSS

Improve Apache Iceberg by building new capabilities for Tabular and the community

Senior Software Engineer, Product

Design services and using cloud infrastructure to build a resilient and scalable data platform

Senior UI Engineer

Design and implement Tabular’s user experience, where people will create, monitor, and manage their data platform

Developer Advocate

Build examples to solve real-world challenges, write tutorials that help developers succeed, and be a community liaison

Developer Experience Engineer

Build technical documentation and tutorials, assist in maintaining the release processes, and lower the time to dopamine (TTD) of developers using Apache Iceberg