Upsolver’s mission is to eliminate complex data engineering so organizations can easily build and affordably run advanced cloud analytics workloads, at scale and in real-time.
Upsolver is a closely-knit group of data engineers and infrastructure developers who are obsessed with making data pipelines so easy to create that any data practitioner could do it, even when dealing with streaming and complex data and big data scale. We built the first version of Upsolver for our own use, and enjoyed eating our own dog food so much that we decided to bring this value to every company doing analytics in the cloud. We now claim numerous companies as customers spanning industries as diverse as oil & gas to gaming, feeding both operational analytics and customer-facing data products.
What we lost in the transition to big data
In the early days of Upsolver we recognized that something important was lost in the transition to big data. Decades ago, databases were invented to abstract away the complexity of coding over raw file systems, letting data teams “speak” in schemas and SQL, which let them transform data more easily and iterate faster.
In the new millenium, big data processing platforms like MapReduce and Spark were adopted to slash costs dramatically as volumes and velocity exploded. However, they arrived without many of the abstractions that made databases so easy to use.
The responsibility to bridge this gap fell to the new data engineering profession. They had to not only code transformations, but specify the query execution and orchestrate all of the jobs that would make the transformation cost-effective and performant.
Back to the future
Upsolver’s goal is to re-invent the simplicity of a database on top of the tremendous economic and “power at scale” advantages of a cloud data lake. That is why we focused on using SQL as the lingua franca for designing pipelines, and why we automate all of the pipeline plumbing that used to be taken for granted in traditional databases, but was left behind with the advent of Hadoop and Spark.
By eliminating any compromise across the dimensions of power, simplicity and affordability, we are transforming data pipelines for cloud analytics workloads. Data engineers who use Upsolver become much more productive, delivering data pipelines that normally would have taken months, in hours. Plus, SQL-savvy data consumers such as data analysts and data scientists now can transform complex raw data, without having to spend all of their time on data engineering.
Realizing the potential of the cloud data lake
This will widen the aperture of analytics, unlocking data that heretofore has been difficult to prepare. System logs, clickstreams, IoT sensor feeds, and other continuous data sources will be blended with static and historical data to create query-ready data sets that feed reporting or ad hoc queries with unmatched data freshness, as well as customer-facing data products.
Over the longer term, Upsolver will empower the cloud data lake to be a key component of every company’s data stack, acting as the central hub that connects raw data to analytics systems from cloud data warehouses to ML systems, graph databases and beyond.
Yoni is a technologist specializing in big data and predictive analytics algorithms. He served in several senior technology roles, including as CTO of the data science division in the IDF’s elite technology intelligence unit. Yoni holds a BA in Mathematics and Computer Science which he started before he was 16.
Peter Wagner is a Founding Partner of Wing. He focuses on the Modern Enterprise, seeking out founders and opportunities that have the potential to transform or create explosive markets. Peter has led investments in dozens of early stage companies, many of which have gone on to complete IPO’s or successful acquisitions. Since leaving Accel in 2011 and founding Wing, Peter invests in and helps to build exciting early stage enterprise technology venture
Gadi has over 20 years of experience in the Hi-Tech industry, in management positions delivering products to global markets. Gadi sits on the board of several of JVP’s companies including Teridion, Iguaz.Io, Vicarius, Valid, Compira Labs, Copilot, Datarix, and Cyberfish. Gadi has an Executive MBA from Bar-Ilan University, Israel and a BA in Economics from Ruppin College, Israel.
Upsolver’s innovative streaming data technology is rapidly changing the big data landscape.