Incrementally streaming rdbms data to your data lake automagically
1 like662 views
The document outlines details about ApacheCon events in 2020 and discussions on streaming RDBMS data. It references various speakers and technologies, such as Apache Kudu and Flume, while providing links to additional resources related to big data, cloud storage, and data ingestion. The focus is on modern data processing techniques and architectures for handling large volumes of messages and data.
Incrementally streaming rdbms data to your data lake automagically
- 1. APACHECON @HOME Spt, 29th – Oct. 1st 2020
- 2. APACHECON NA Spt, 28th – Oct. 2nd 2020
- 3. APACHECON NA Spt, 28th – Oct. 2nd 2020
- 5. 5 Future of Data - Princeton @PaasDev https://www.meetup.com/futureofdata-princeton/ From Big Data to AI to Streaming to Containers to Cloud to Analytics to Cloud Storage to Fast Data to Machine Learning to Microservices to ...
- 6. 6 Speakers John Kuchmek Senior Solutions Engineer
- 7. 7 Speakers Tim Spann Principal DataFlow Field Engineer @PaasDev DZone Zone Leader and Big Data MVB Princeton NJ Future of Data Meetup https://github.com/tspannhw https://www.datainmotion.dev/
- 8. 8 JDBC Database to Apache Kudu / JDBC Database to HDFS and Hive
- 9. 9 Trillions of Messages to SQL Databases and Data Warehouses PROCESS DELIVER
- 11. 11 Reference Architecture Files to RDBMS
- 15. • Example • Flat files on an FTP server named by date • Downloads file • HTTP REST API endpoint • Invokes API and downloads data • Legacy/Remote DB • Performs SQL queries 1 5 DBCP Connection Pool to remote SQL Server ExecuteSQLRecord processor