Make your databases instantly available for analytics with Upsolver.
Free forever Community Edition
With out-of-the-box support for all major cloud and on-premise MySQL or PostgreSQL deployments, Upsolver makes replicating your databases to your data lake a breeze.
On-prem and cloud-based transactional and operational databases are an important source of data needed for data science, machine learning and ad hoc analytics. The best way to maintain an up-to-date replica of these databases in your cloud data lake is to create a log-based CDC (Change Data Capture) pipeline that migrates an initial snapshot of the data and then monitors and streams changes from the source database log to keep the copy in sync with the original.
Upsolver’s CDC connector lets you continuously replicate any on-prem or cloud MySQL or PostgreSQL database into your AWS or Azure data lake in a few simple steps, and without writing code. Upsolver replicates a baseline snapshot of the source database and then keeps it up to date by monitoring the transaction log and replicating changes as they occur.
This data can be accessed by query engines, or output to external data stores such as your cloud data warehouse.
Configure the replication pipeline through Upsolver’s Visual SQL IDE, where you can select one, some or all tables to replicate. You can also apply filters, aggregations, joins and any other transformations you’d like to the incoming data.
Select the tables you want to move, then build transformation logic using our Visual SQL IDE which allows you to use drag-and-drop or edit SQL directly, with the two systems staying in sync.
Upsolver automates data lake engineering best practices, saving you the time and trouble of hand-coding and configuration of distributed systems.
By leveraging open standards in a data lake, you can choose to query data directly from tables in the data lake or distribute the prepared data out to the systems of your choice.
Build working solutions for stream and batch processing on your data lake in minutes.