A Tabular newsletter revisiting the last month in Apache Iceberg
Apache Iceberg? Spread the word by giving it a 
on the apache/iceberg repo!
Project updates
Iceberg Java
- Added encryption support for Parquet and Avro data.
- Added Spark support for reading, dropping, and renaming views.
- Initial project setup for bringing the Kafka Connect Iceberg Sink Connector into the Apache Iceberg project.
- Added Spark support for controlling delete file granularity. Now delete files can be created at the partition or file level.
- Removed support for Spark 3.2.
- Parallelized file footer reads in add_files procedure.
- Updated the new docs in preparation for moving to /site and /docs directories in the main Iceberg repo. This will make it easier to contribute to and release the Iceberg docs.
PyIceberg, iceberg-go, and iceberg-rust
- Exciting progress in PyIceberg, including
- The news we’ve all been waiting for… Write support has been added to PyIceberg! This opens the door to tons of new use cases!
- Added commit support for the SQL catalog. This enables using a local catalog, making it much easier to run locally or in a notebook.
- Also added commit support for Hive and Glue.
- Name mapping support will allow the reading of parquet files without field IDs.
- An expression system was added to iceberg-rust, paving the way for more expressions to be added.
- The rust-iceberg project also gained basic file scan planning.
- The Rust API now has documentation available at https://rust.iceberg.apache.org.
Bergy Blogs
- Streaming Event Data to Iceberg with Kafka Connect. An introduction to the Iceberg Sink connector for Kafka Connect.
- Streaming CDC data from MySql to Apache Iceberg with Hive Metastore using Apache Flink.
- How not to use Apache Iceberg! Great advice for getting the most out of Iceberg.
- Migrate existing Amazon S3 data lake tables to Snowflake Unmanaged Iceberg tables using AWS Glue.
- Why You Should Use Apache Iceberg with PySpark.
- 2024 Lakehouse Format Rundown. Good analysis of the current state of table formats.
- Learning Apache Iceberg — storing the data to Minio S3. The third installment of a series on learning Iceberg.
Ecosystem Updates
- Breaking the Ice, Apache Iceberg Meetup in Tel Aviv, 11 March 2024.
- Netflix Creates Incremental Processing Solution Using Maestro and Apache Iceberg.
- Integrating Databricks with Snowflake-managed Iceberg Tables.
- Apache Iceberg vs Parquet – File Formats vs Table Formats.
- Use Amazon Athena with Spark SQL for your open-source transactional table formats.
- Used Iceberg tables in AWS Athena to generate daily snapshot data.
Vendor Updates
Iceberg Resources
Get Started with Apache Iceberg.
Get cookin’ with recipes from the Apache Iceberg Cookbook.
Learn more about Apache Iceberg on the official Apache site.
Watch and subscribe to the Iceberg YouTube Channel.
Read up on some community blog posts.
Contribute to Iceberg.
`SELECT * FROM you JOIN iceberg_community`.
Subscribe to the Apache Iceberg mailing list.
