Easily and affordably transform streaming data into live tables on top of cloud object storage. Query directly on the lake or continuously output tables to external analytics systems.
self-service data engineering
stateful operations on data streams at scale.
reduction in time spent on ETL work
reduction in cloud data warehouse costs
of data in motion with billions of join keys.
Efficient EC2 or Azure utilization via Spot instances.
Upsolver processes 10X more data in a given RAM footprint.
Use our Visual SQL IDE to build your pipeline, while we automate orchestration, compaction and other ugly plumbing.
Get live tables that can be queried on the lake, or easily output to other systems.
Tables are stored in an open format – no vendor lock-in.
Upsolver makes it easy to ingest data with native connectors to:
Regardless of the source, Upsolver continuously stores a raw copy of the data for lineage and historical replay, while generating and outputting consumption-ready tables based on your desired transformations
Upsolver creates query-ready live tables for analytics using:
In all cases, Upsolver allows you to select and configure the output visually.
Upsolver provides a visual SQL-based interface so that transformation pipelines can be built without coding and hundreds of configurations in Spark/Hadoop:
Cloud object storage has traditionally required an inordinate amount of manual configuration and tuning to deliver performance.
Upsolver automates the management and optimization of output tables so that data engineers only need to specify their transformations visually or in SQL, including:
Upsolver started as an internal project for accelerating our own analytics engineering work over AWS S3. We were frustrated by the convoluted and expensive process of building stateful streaming pipelines using Apache Spark, Airflow and Cassandra for large states.
So we built the world’s first fully decoupled key-value store called Lookup Tables which are computed using streaming SQL on multiple sources of streaming and batch data. It only stores data on cloud object storage and in-memory and it gets 10X more data in a given RAM footprint compared to Cassandra. Under the hood, Lookup Tables are based on a new file format and compression algorithm which combine efficient columnar compression with millisecond key-based queries.
Upsolver has everything you need to implement continuous, at-scale, production pipelines. You can monitor flows, manage pipeline changes methodically, and deploy in your Cloud account or ours.