SlideShare a Scribd company logo

COM311 Migrating 600 Databases To AWS

M
Matt Houghton

The document outlines a presentation by Matt Houghton on the migration of 600 databases to AWS, emphasizing enhancements in speed, cost-effectiveness, and high availability. It covers various migration steps, automation strategies, and the evolving role of database administrators (DBAs) focusing on infrastructure as code and improved developer experiences. Additional topics include database tuning, monitoring, and the benefits of using AWS tools for successful migration.

Data & Analytics
Related topics:
Cloud Computing Insights
1 of 30
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Migrating 600 databases to AWS and making them better, faster, and cheaper C O M 3 1 1 Matt Houghton (he/him) Data Architect CDL
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL High availability
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Enterprise-grade database
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Hybrid cloud Primary Secondary Read-Only
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Move up the stack • Learn by failing • Undifferentiated heavy lifting • Focus on the business problem
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Migration steps • AWS Schema Conversion Tool (AWS SCT) • Change data capture (CDC) • Database objects • Database code • Engine-specific functions • Code/rewrite/rearchitect
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Change data capture
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Demo
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Event-driven -
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Migration monitoring
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Automation • Infrastructure as code • Parameterize • Switchover • Rollback • Reduce risk • Consistency • Speed of deployment
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Tuning • join_collapse_limit • from_collapse_limit • work_mem • pg_stat_statements • pg_prewarm • Long-running SQL • Tools • The organization
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL What do the DBAs do now? • Infrastructure as code • More automation • Config-driven deployments • Faster upgrades/fixes • Scale the business
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Developer experience • Self-service • Database clones • Privacy automation • Speed • Preapproved queries • Health checks • Database-as-a-service product • New skills
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon RDS Ready
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Connection pooling
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Application logging
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Data encryption
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Database failure and performance
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Open source
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Fast refresh materialized views
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Fast refresh materialized views – Walkthrough
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Fast refresh materialized views – Walkthrough
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Fast refresh materialized views – Walkthrough
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Fast refresh materialized views – Walkthrough
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Demo
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Final tip
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Thank you! Matt Houghton @mattdevdba Please complete the session survey

More Related Content

PDF
AWS UK User Group Migrating 600 Databases - February 2023.pdf
Matt Houghton
 
PDF
AWS Summit London 2023 - Migrating 600 Databases To AWS
Matt Houghton
 
PDF
re:Invent OPN306 AWS Lambda Powertools Lessons 10M downloads.pdf
Heitor Lessa
 
PDF
Best Practices for Using AWS Credits
TechSoup
 
PDF
Mainstream Serverless
Dhaval Nagar
 
PPTX
SEC209-S-143991-Session-Presentation.f500a51002fec4b9a0f60923b1455fb2272a1767...
jopivihine
 
PDF
2020-04-02 DevConf - How to migrate an existing application to serverless
Marcia Villalba
 
PDF
20200513 - CloudComputing UCU
Marcia Villalba
 
AWS UK User Group Migrating 600 Databases - February 2023.pdf
Matt Houghton
 
AWS Summit London 2023 - Migrating 600 Databases To AWS
Matt Houghton
 
re:Invent OPN306 AWS Lambda Powertools Lessons 10M downloads.pdf
Heitor Lessa
 
Best Practices for Using AWS Credits
TechSoup
 
Mainstream Serverless
Dhaval Nagar
 
SEC209-S-143991-Session-Presentation.f500a51002fec4b9a0f60923b1455fb2272a1767...
jopivihine
 
2020-04-02 DevConf - How to migrate an existing application to serverless
Marcia Villalba
 
20200513 - CloudComputing UCU
Marcia Villalba
 

Similar to COM311 Migrating 600 Databases To AWS (9)

PPTX
IoT enable smoker for great BBQ
Jimmy Dahlqvist
 
PDF
20201013 - Serverless Architecture Conference - How to migrate your existing ...
Marcia Villalba
 
PDF
Advanced Monitoring for Amazon RDS - AWS 4D Event Manchester 16th June 2023
Matt Houghton
 
PPTX
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
ScyllaDB
 
PDF
20200522 - How to migrate an existing app to serverless
Marcia Villalba
 
PPTX
2018 re:Invent - Safeguard the Integrity of Your Code for Fast and Secure Dep...
Martin Klie
 
PDF
Sederhanakan_integrasi_data_anda_dengan_AWS_Glue_handout.pdf
Jazzy44
 
PDF
C04 Driving understanding from Documents and unstructured data sources final.pdf
PhilipBasford
 
PDF
Introduction to Serverless computing and AWS Lambda - Floor28
Boaz Ziniman
 
IoT enable smoker for great BBQ
Jimmy Dahlqvist
 
20201013 - Serverless Architecture Conference - How to migrate your existing ...
Marcia Villalba
 
Advanced Monitoring for Amazon RDS - AWS 4D Event Manchester 16th June 2023
Matt Houghton
 
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
ScyllaDB
 
20200522 - How to migrate an existing app to serverless
Marcia Villalba
 
2018 re:Invent - Safeguard the Integrity of Your Code for Fast and Secure Dep...
Martin Klie
 
Sederhanakan_integrasi_data_anda_dengan_AWS_Glue_handout.pdf
Jazzy44
 
C04 Driving understanding from Documents and unstructured data sources final.pdf
PhilipBasford
 
Introduction to Serverless computing and AWS Lambda - Floor28
Boaz Ziniman
 
Ad

More from Matt Houghton (6)

PDF
AWS DATABASE USER GROUP - LAUNCH EVENT (LONDON) December 7, 2022 - COM311 Mi...
Matt Houghton
 
PDF
Qlik Partner OEM Webinar Townhall October 2022.pdf
Matt Houghton
 
PDF
Migrating 600 Databases To AWS - Better Faster Cheaper - AWS Community Summit...
Matt Houghton
 
PDF
Turning Transactions Into Insights and Triggers - Qlik Partner Event July 2022
Matt Houghton
 
PDF
Turning Transactions Into Insights and Triggers - Future of FinTech Conferenc...
Matt Houghton
 
PDF
Getting AWS Certified - Big Data Manchester Meetup August 2022
Matt Houghton
 
AWS DATABASE USER GROUP - LAUNCH EVENT (LONDON) December 7, 2022 - COM311 Mi...
Matt Houghton
 
Qlik Partner OEM Webinar Townhall October 2022.pdf
Matt Houghton
 
Migrating 600 Databases To AWS - Better Faster Cheaper - AWS Community Summit...
Matt Houghton
 
Turning Transactions Into Insights and Triggers - Qlik Partner Event July 2022
Matt Houghton
 
Turning Transactions Into Insights and Triggers - Future of FinTech Conferenc...
Matt Houghton
 
Getting AWS Certified - Big Data Manchester Meetup August 2022
Matt Houghton
 
Ad

Recently uploaded (20)

PDF
Lecture1 pattern recognition............
ZafriFarid
 
PPTX
Introduction to machine learning and Linear Models
ShivanshMehta17
 
PPTX
Acceptance and paychological effects of mandatory extra coach I classes.pptx
ZaheerAhmad228692
 
PDF
“Getting Started with Data Analytics Using R – Concepts, Tools & Case Studies”
Nusrat Gulbarga
 
PPTX
Introduction to Basics of Ethical Hacking and Penetration Testing -Unit No. 1...
AshutoshDeshmukh33
 
PDF
Business Analytics and business intelligence.pdf
khalidisah4750
 
PDF
Foundation of Data Science unit number two notes
sanguleumeshit
 
PPTX
Microsoft-Fabric-Unifying-Analytics-for-the-Modern-Enterprise Solution.pptx
Mahesh Reddy
 
PDF
TRAFFIC-MANAGEMENT-AND-ACCIDENT-INVESTIGATION-WITH-DRIVING-PDF-FILE.pdf
chouziyu14
 
PPTX
MODULE 8 - DISASTER risk PREPAREDNESS.pptx
AceAquino4
 
PPTX
AI Strategy room jwfjksfksfjsjsjsjsjfsjfsj
ankitpratapsingh28
 
PDF
.pdf is not working space design for the following data for the following dat...
jerinjoy242
 
PDF
BF and FI - Blockchain, fintech and Financial Innovation Lesson 2.pdf
JamesWilliams752225
 
PPTX
Computer network topology notes for revision
BatoolRawat
 
PPTX
Introduction to Firewall Analytics - Interfirewall and Transfirewall.pptx
kuthubussaman1
 
PPTX
01_intro xxxxxxxxxxfffffffffffaaaaaaaaaaafg
Vittaya Boon Suwansiri
 
PDF
Fluorescence-microscope_Botany_detailed content
dollydoll12
 
PDF
Recruitment and Placement PPT.pdfbjfibjdfbjfobj
Alka392097
 
PPTX
The THESIS FINAL-DEFENSE-PRESENTATION.pptx
hshakobe3
 
PPTX
advance b rammar.pptxfdgdfgdfsgdfgsdgfdfgdfgsdfgdfgdfg
rohullahansari5
 
Lecture1 pattern recognition............
ZafriFarid
 
Introduction to machine learning and Linear Models
ShivanshMehta17
 
Acceptance and paychological effects of mandatory extra coach I classes.pptx
ZaheerAhmad228692
 
“Getting Started with Data Analytics Using R – Concepts, Tools & Case Studies”
Nusrat Gulbarga
 
Introduction to Basics of Ethical Hacking and Penetration Testing -Unit No. 1...
AshutoshDeshmukh33
 
Business Analytics and business intelligence.pdf
khalidisah4750
 
Foundation of Data Science unit number two notes
sanguleumeshit
 
Microsoft-Fabric-Unifying-Analytics-for-the-Modern-Enterprise Solution.pptx
Mahesh Reddy
 
TRAFFIC-MANAGEMENT-AND-ACCIDENT-INVESTIGATION-WITH-DRIVING-PDF-FILE.pdf
chouziyu14
 
MODULE 8 - DISASTER risk PREPAREDNESS.pptx
AceAquino4
 
AI Strategy room jwfjksfksfjsjsjsjsjfsjfsj
ankitpratapsingh28
 
.pdf is not working space design for the following data for the following dat...
jerinjoy242
 
BF and FI - Blockchain, fintech and Financial Innovation Lesson 2.pdf
JamesWilliams752225
 
Computer network topology notes for revision
BatoolRawat
 
Introduction to Firewall Analytics - Interfirewall and Transfirewall.pptx
kuthubussaman1
 
01_intro xxxxxxxxxxfffffffffffaaaaaaaaaaafg
Vittaya Boon Suwansiri
 
Fluorescence-microscope_Botany_detailed content
dollydoll12
 
Recruitment and Placement PPT.pdfbjfibjdfbjfobj
Alka392097
 
The THESIS FINAL-DEFENSE-PRESENTATION.pptx
hshakobe3
 
advance b rammar.pptxfdgdfgdfsgdfgsdgfdfgdfgsdfgdfgdfg
rohullahansari5
 

COM311 Migrating 600 Databases To AWS

  • 1. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL
  • 2. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Migrating 600 databases to AWS and making them better, faster, and cheaper C O M 3 1 1 Matt Houghton (he/him) Data Architect CDL
  • 3. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL
  • 4. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL High availability
  • 5. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Enterprise-grade database
  • 6. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Hybrid cloud Primary Secondary Read-Only
  • 7. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Move up the stack • Learn by failing • Undifferentiated heavy lifting • Focus on the business problem
  • 8. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Migration steps • AWS Schema Conversion Tool (AWS SCT) • Change data capture (CDC) • Database objects • Database code • Engine-specific functions • Code/rewrite/rearchitect
  • 9. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Change data capture
  • 10. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Demo
  • 11. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Event-driven -
  • 12. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Migration monitoring
  • 13. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Automation • Infrastructure as code • Parameterize • Switchover • Rollback • Reduce risk • Consistency • Speed of deployment
  • 14. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Tuning • join_collapse_limit • from_collapse_limit • work_mem • pg_stat_statements • pg_prewarm • Long-running SQL • Tools • The organization
  • 15. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL What do the DBAs do now? • Infrastructure as code • More automation • Config-driven deployments • Faster upgrades/fixes • Scale the business
  • 16. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Developer experience • Self-service • Database clones • Privacy automation • Speed • Preapproved queries • Health checks • Database-as-a-service product • New skills
  • 17. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon RDS Ready
  • 18. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Connection pooling
  • 19. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Application logging
  • 20. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Data encryption
  • 21. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Database failure and performance
  • 22. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Open source
  • 23. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Fast refresh materialized views
  • 24. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Fast refresh materialized views – Walkthrough
  • 25. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Fast refresh materialized views – Walkthrough
  • 26. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Fast refresh materialized views – Walkthrough
  • 27. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Fast refresh materialized views – Walkthrough
  • 28. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Demo
  • 29. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Final tip
  • 30. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. CDL - EXTERNAL Thank you! Matt Houghton @mattdevdba Please complete the session survey