Product Insights at Scale: ZeroETL ingestion from PostgreSQL to Iceberg Lakehouse

Hands-On Workshop | February 28th | 10am PST / 1pm EST / 6pm GMT

Operational databases like PostgreSQL excel in handling transactional workloads, offering the speed and reliability essential for day-to-day operations. However, when it comes to analytical processing, the volume and complexity of queries can strain these systems, leading to degraded performance and higher operational costs.

Integrating PostgreSQL with an Iceberg Lakehouse represents a paradigm shift, enabling the mirroring of operational data into a scalable, versioned, highly efficient and cost-effective  shared storage. This integration facilitates complex queries, historical data analysis, and advanced data science capabilities on vast amounts of data without vendor lock-in or impacting the transactional database’s performance. 

The result is a harmonious balance between operational efficiency and analytical depth, empowering organizations to leverage their data in real time and at operational scale for comprehensive product insights.

 

What you will learn:

  • Real-Time Product Analytics: Understand the importance of up-to-the-minute analytics to the successful adoption and pace of innovation of your product. 
  • Open Lakehouse Architecture: Discuss how to design and construct a Lakehouse architecture on AWS, its components and integrations.
  • Visual vs. Declarative Development: Understand the trade-offs between visually authoring data movement jobs and developing custom ETL pipelines using SQL for finer control over data preparation and quality.
  • Data Contracts: Explore what data contracts are and how engineers can use them to automate and communicate pipeline and data model changes without breaking downstream systems.

 

What you will build

  • Simple pipeline to ingest application data from PostgreSQL to an Iceberg-based Lakehouse in Amazon S3. 
  • Use SQL to define a simple data contract for the final table output.
  • Use SQL to build a simple job that continuously transforms, models and loads raw, bronze, data into a gold table for your product team to analyze

 

Presented by:

Jason Hall
Senior Solution Architect

Save Your Spot

Templates

All Templates

Explore our expert-made templates & start with the right one for you.