Building an ingestion architecture for Apache Iceberg

As companies adopt Apache Iceberg as their table format, they quickly realize that part of the battle is building and maintaining the pipelines that continuously, reliably and efficiently populate these tables. They have to consider the variety of data sources (e.g. logs, IoT, clickstreams, transactions); batch, micro-batch and event streaming methods, ETL, ELT and EtLT patterns, and specifically what products to use.

In this webinar, Jason Reid, co-founder and head of product at Tabular, and Vicky Bukta, senior product manager at Tabular take the ingestion challenge head on.

Their talk will cover:

• The common hurdles you will encounter when building a data ingestion architecture for Iceberg
• How to evaluate trade offs across latency, reliability, total cost and architecture complexity
• The pros and cons of various design patterns and open source or vendor options
• A discussion of complex real-world use cases we’ve encountered
• A live demo of file, CDC and event ingestion in the Tabular storage engine