Bridge to the Cloud: Using Apache Kafka to Migrate to AWS
2 likes1,018 views
The document outlines a strategy for migrating applications to Amazon Web Services (AWS) using Apache Kafka, emphasizing the need for a thoughtful approach to integrate complex legacy systems with cloud solutions. It discusses the steps to establish a cloud foundation, create data pipelines, and migrate applications while ensuring real-time, secure, and reliable data flow. Additionally, it highlights the benefits of using a managed service for Kafka on AWS and shares insights from a real-world transition experience.
Bridge to the Cloud: Using Apache Kafka to Migrate to AWS
- 1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. BRIDGE TO THE CLOUD: U SIN G A PA C H E K A F K A ® TO MIG R AT E TO AWS JAY KREPS C E O & C O - F O U N D E R , C O N F L U E N T N O V E M B E R 2 0 1 7
- 2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. CLOUD MIGRATION: A ONE TIME THING?
- 3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. In reality, we keep running • Our apps are complex, intricately tied together • Some, built to run on legacy, immovable systems
- 4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. We don’t want to just move – we want to build for the cloud
- 5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. MySQL App TeradataApp Oracle Elasticsearch App DC-1 Amazon DynamoDB Amazon S3 App Amazon Redshift AppApp Amazon RDS
- 6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon S3 Amazon DynamoDB Oracle App
- 7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. WHAT IS OUR BRIDGE? Real-time Secure Reliable
- 8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 1/3 of the Fortune 500
- 9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ESTABLISH YOUR FOUNDATION Amazon S3 Amazon DynamoDB Oracle 1 Deploy Kafka on-premise and Amazon Web Services (AWS) App Teradata
- 10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. ESTABLISH YOUR FOUNDATION Amazon S3 Amazon DynamoDB Oracle App Teradata 2 Create your pipeline and replicate your topics to your AWS cluster
- 11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. LET THE TRAFFIC FLOW Amazon S3 Amazon DynamoDB Oracle Topic: dc1clicks Topic: dc1users 3 Migrate app by app, database by database App App
- 12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. MySQL App TeradataApp Oracle Elasticsearch App DC-1 Amazon DynamoDB Amazon S3 App Amazon Redshift AppApp Amazon RDS eu-central ap-northeast DC-NY DC-LON
- 13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. MySQL App TeradataApp Oracle Elasticsearch App Amazon DynamoDB Amazon S3 App Amazon Redshift AppApp Amazon RDS eu-central ap-northeast DC-NY DC-LON DC-1
- 14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. BUILT FOR THE FUTURE ML
- 15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. WE CAN EVEN MANAGE IT FOR YOU Now Generally Available on AWS Fully-managed by the experts at Confluent 99.95% Uptime Guarantee
- 16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. OR HELP RUN YOUR OWN
- 17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Learn from Monsanto’s Data Platform Architect Transitioning to the cloud using a streaming data platform THANK YOU! Watch here – http://bit.ly/2zUDAKN