On-demand webinar

How to Build & Manage Data Pipelines for Streaming Data on AWS

Harnessing streaming event data enables companies to observe and respond to interactions before it’s too late. Examples include clickstreams for website, digital product, e-commerce or digital campaign performance; IoT sensor data for predictive analytics in oil and gas, heavy machinery or telecoms, or log data for IT and security monitoring across all industries. 

In this session experts from AWS and Upsolver will cover the challenges of building a streaming-first data stack as opposed to batch-only or batch and streaming (lambda architecture).

Ori Rafael

CEO and Co-founder, Upsolver

Gitika Vijh

Sr. Solutions Architect, AWS

Sign up to attend

Thank you!

Thanks for registering. Click below to watch the webinar

Watch Now

Some of the challenges that will be covered include:

  • Supporting incremental processing without batch start and end times
  • Implementing stateful operations like large-scale joins
  • Circumventing append-only limitations model for streaming by using upserts
  • Handling increased pipeline orchestration complexity
  • Ensuring exactly-once consistency (no dups or loss) from stream to eyeballs
  • Addressing data lake performance challenges such as the “small files problem”

 

Powering data lakes for data-intensive companies