November 2022 – Iceberg Community News

Catching up with all the exciting Iceberg community news from October and November 2022.

Iceberg Updates

  • Iceberg v1.1.0 released
  • Spark: Integration to read from branches and tags
  • Spark: added initial APIs to use Iceberg metadata for aggregate queries
  • Spark: Support for Spark 3.0 has been dropped
  • Flink: 1.16 support was added, 1.13 removed
  • Flink: Added sink options to override the compression properties of the table
  • Core: Added changelog table and new scan class
  • Core: Improved the performance of snapshot expiration
  • Core: Removed extra header read when opening a Manifest file.
  • Core: Increase inferred column metrics limit to 100
  • Core: Support performing merge appends and deletes files on branches
  • Scan planning now reported via REST, also contains additional metadata from Spark+Flink

PyIceberg Updates

PyIceberg has been in very active development on GitHub, with a few notable updates being:

  • Added support for AWS Glue
  • Added initial support for scan planning, and experimental support for reading Iceberg tables into PyArrow, Pandas, and DuckDB
    -Added SSL client certificate support to REST Client

For instructions on how to install PyIceberg, and to get quick access to the many packages available for various technologies, visit the website.

Iceberg In the Industry

New Blogs From the Community

Keep Up to Date on All Things Iceberg

Look out for new blog posts added to the Blogs page
See the community Contribute guide to learn how to start contributing to Iceberg
Join the Apache Iceberg workspace on Slack using the invite link
Subscribe to the Apache Iceberg mailing list