Relational databases hold critical company information that can be combined with big data for analytics insights. However, you need to get this data into your data lake first, in a way that is efficient, timely, and doesn’t interfere with the operations of the source database.
You can use the Upsolver change data capture (CDC) source connector to create an always up-to-date copy of your database in your data lake. After the initial replication, the connector uses the source database transaction log to stream changes to the data lake without interfering with database operations. The replica is now available for use by a data lake query engine or further processing and output to your data warehouse or other analytics systems.
These short interactive exercises will illustrate how to easily connect your RDBMS to AWS Athena. We will query the Views in Athena as the CDC data is streaming in, thus providing a very simple solution to database replication into your data lake using Upsolver.