Upsolver joins Qlik to deliver real-time data, Iceberg optimizations and cost savings under a single platform

From 5.5 hours to 39 seconds: adapting Iceberg querying to high-scale data

Jason Fine, Chief Architect @ Upsolver

Jason talked about Iceberg performance optimization, focusing on high-scale streaming data challenges. He introduced a more efficient method for managing upserts, crucial for streaming data, by optimizing query plans and reducing IO operations.

This approach reduced operation time from over five hours to thirty-nine seconds. He also discussed Upsolver’s contributions to open source projects Iceberg, Trino and Presto to avail benefits of this optimization across the board in real-time lakehouse architecture.

 

Templates

All Templates

Explore our expert-made templates & start with the right one for you.