LIVE WEBINAR

How Proofpoint Built a Big Data Pipeline for Real-Time Network Traffic (with 1 Developer in 3 Weeks)

May 25, 10 AM PST | 1 PM EST | 7 PM CET

As workforces have gone remote, the network perimeter has become increasingly fragmented and software-defined. Maintaining continuous visibility into network security requires ingesting and processing real-time data from network systems and endpoints in order to identify potential threats and to provide useful analytics to network owners. 

Proofpoint is a leader in cloud network security that helps organizations protect themselves against these threats by using big data. Their cloud security service collects large volumes of data about customers’ networks in real-time: DNS, traffic, IP addresses, API access and more. Just one user visiting one website can produce hundreds of new data points. This tremendous amount of raw data must be made immediately available for ad hoc analytics, data science and customer-facing dashboards and reporting. 

Alon Horowitz

Sr. Director R&D, Proofpoint

Roy Hasson

Head of Product, Upsolver

Sign up to attend

Thank you!

Thanks for registering. You will get an email from Zoom with more details.

In this online talk Alon Horowitz from Proofpoint and Roy Hasson from Upsolver will discuss:

  • The challenges Proofpoint faced when it came to building their real-time analytics solution, such as ingestion and processing at scale, ensuring low latency from end-to-end, and combining real-time and historical data. 
  • The details of the real-time streaming architecture they built, including Kinesis, S3, Athena, Upsolver and Elasticsearch.
  • The speedy development process they enjoyed, which took only one developer without big data skills, three weeks to complete.
  • Lessons learned over the lifecycle, from development and implementation to ongoing operation under conditions of greatly increasing scale. 

 

Powering data lakes for data-intensive companies