Explore our expert-made templates & start with the right one for you.
Centene
Powering real-time analytics in Snowflake
CUSTOMER STORY
INDUSTRY:
Healthcare
USE CASE:
Providing business teams with better real-time access and management of their data in Snowflake
DATA:
Member data including claims, provider, authorizations, corporate operations, health plans and more
100
data ingestion pipelines and
45
active users in
2
months
1 sprint
to build new Snowflake ingestion pipelines
Download Centene’s presentation from Snowflake Summit 2023.
Centene Corporation’s purpose is to transform the health of the communities they serve one person at a time.
In order to create better health outcomes at lower costs, Centene provides access to high-quality healthcare, innovative programs and a wide range of health solutions that help families and individuals get well, stay well, and be well.
Centene brings a number of resources to the table. Clocking in at Fortune #26, Centene has 74,300 employees and covers all 50 U.S. states, serving 27.1 million members. With such a large organization and member base, Centene felt it was important that individual business units have direct access to relevant data so they could make better decisions and respond to a changing market without having to go through the bureaucracy of centralized teams for every request.
The Requirements
- Security – Centene has a strict cloud security model so it can protect member data. It’s important that all cloud solutions deploy in their VPC.
- Scale – With more than 27 million members and a vast number of data sources, Centene could only consider solutions that scale – up or down – according to their needs.
- Ease of use – Centene needed to open up data ingestion to Snowflake to multiple business units. Any data ingestion platform they considered needed to be low-case or no-code so they didn’t need to resource additional technical headcount in every unit.
The Goal
The data platform team’s goal was to ingest data from source systems and ship relevant data to domain-driven data stores, depending on different criteria and SLAs. Information in the source systems would be updated throughout the day.
As Centene moved from on-premises solutions to the cloud, they recognized an opportunity for each domain to play a bigger role in how they accessed and managed their data. Centene knew they needed:
- A platform that would allow different teams to self-serve their relevant data.
- An enablement team to set up processes and establish usage patterns, providing templates, documentation, and how-to guides on data flow patterns.
- A solution that would work for data platform engineers, domain data engineers, and data analysts without requiring additional training.
100
data ingestion pipelines and
45
active users in
2
months
1 sprint
to build new Snowflake ingestion pipelines
The Solution
- VPC deployment – One of the initial attributes of Upsolver that attracted Centene was that Upsolver would be deployed in their AWS VPC. Data is streamed through Kafka, processed by Upsolver, and pushed into Snowflake. Data from different sources are transformed and combined into larger, domain-informed data sets for downstream operations by business partners.
- Data observability – Operating at such massive scale, Centene is able to inspect data in Upsolver before it is ingested to Snowflake, so they can spot any data quality issues at the point of ingestion and fix them before they create issues in downstream analytics and applications.
- Self-service pipelines – In under 2 months, Centene had more than 45 active users on Upsolver, who had built more than 100 new data ingestion pipelines in that time. With their previous solution, it was taking too long to onboard and train data engineers because Centene’s domain-driven use cases are highly specialized.
Upsolver is like the “easy button” for Snowflake. We ingest data from our Kafka streams, process it as necessary for different use cases, and deliver it, all while observing how our schema and data are changing in real time.”
Alexander Adam, Cloud Platform Engineering Manager, Centene Why Centene Chose Upsolver
Upsolver simplifies workflows for Centene. In near real-time, Upsolver ingests and flattens JSON data into Snowflake while providing observability of schema drift.
- Upsolver is able to fan out pipelines to multiple Snowflake databases with different transformation requirements.
- Upsolver eliminated the development complexities of hand-coding AWS Glue and PySpark jobs.
- With Upsolver, users avoid rogue Glue jobs that may unpredictably drive up costs.
- Upsolver required minimal provisioning by leveraging Centene’s AWS account.
Upsolver emphasizes security, governance, and data quality. Data never leaves Centene’s cloud infrastructure. Centene builds monitoring based on the metadata within Upsolver’s system tables. Clusters auto-scale, significantly reducing the cost compared with other solutions, and Upsolver enables Centene to attribute cost based on each team’s usage.