Catching up with all the exciting Iceberg community news from October and November 2022.
- Iceberg Updates
- PyIceberg Updates
- Iceberg In the Industry
- New Blogs From the Community
- Keep Up to Date on All Things Iceberg
Iceberg Updates
- Iceberg v1.1.0 released
- Spark: Integration to read from branches and tags
- Spark: added initial APIs to use Iceberg metadata for aggregate queries
- Spark: Support for Spark 3.0 has been dropped
- Flink: 1.16 support was added, 1.13 removed
- Flink: Added sink options to override the compression properties of the table
- Core: Added changelog table and new scan class
- Core: Improved the performance of snapshot expiration
- Core: Removed extra header read when opening a Manifest file.
- Core: Increase inferred column metrics limit to 100
- Core: Support performing merge appends and deletes files on branches
- Scan planning now reported via REST, also contains additional metadata from Spark+Flink
PyIceberg Updates
PyIceberg has been in very active development on GitHub, with a few notable updates being:
- Added support for AWS Glue
- Added initial support for scan planning, and experimental support for reading Iceberg tables into PyArrow, Pandas, and DuckDB
-Added SSL client certificate support to REST Client
For instructions on how to install PyIceberg, and to get quick access to the many packages available for various technologies, visit the website.
Iceberg In the Industry
- Amazon Athena enhanced Iceberg table operations and file format support
- Trino Summit Recap
- Apple Presentation at Trino Summit
- Trino Summit Summary with Iceberg
New Blogs From the Community
- Dremio – Compaction in Apache Iceberg: Fine-Tuning Your Iceberg Table’s Data Files
- Dremio – The Life of a Read Query for Apache Iceberg Tables
- Dremio – Puffins and Icebergs: Additional Stats for Apache Iceberg Tables
- Dremio – Apache Iceberg and the Right to Be Forgotten
- Dremio – Streaming Data into Apache Iceberg Tables Using AWS Kinesis and AWS Glue
- Tabular – Demonstrating PyIceberg
- Starburst – Introduction to Apache Iceberg In Trino
- Starburst – Iceberg Partitioning and Performance Optimizations in Trino
- Starburst – Apache Iceberg DML (update/delete/merge) & Maintenance in Trino
- Starburst – Apache Iceberg Schema Evolution in Trino
Keep Up to Date on All Things Iceberg
Look out for new blog posts added to the Blogs page
See the community Contribute guide to learn how to start contributing to Iceberg
Join the Apache Iceberg workspace on Slack using the invite link
Subscribe to the Apache Iceberg mailing list
