Amazon Aurora (Debanjan Saha) - AWS DB Day
1 like1,135 views
Amazon Aurora is a relational database service that is compatible with MySQL and PostgreSQL databases. It is fully managed by AWS and provides faster performance than MySQL databases at lower costs. Aurora provides high availability across three availability zones and automatic failover. It is easy to migrate existing MySQL databases to Aurora using AWS database migration services. Aurora is optimized for the cloud and leverages other AWS services like DynamoDB and S3 for storage. It has a simple pricing model based on the instance size and storage used.
![ALTER SYSTEM CRASH [{INSTANCE | DISPATCHER | NODE}]
ALTER SYSTEM SIMULATE percent_failure DISK failure_type IN
[DISK index | NODE index] FOR INTERVAL interval
ALTER SYSTEM SIMULATE percent_failure NETWORK failure_type
[TO {ALL | read_replica | availability_zone}] FOR INTERVAL interval
Simulate failures using SQL
To cause the failure of a component at the database node:
To simulate the failure of disks:
To simulate the failure of networking:](https://image.slidesharecdn.com/amazonaurora-160502083919/85/Amazon-Aurora-Debanjan-Saha-AWS-DB-Day-19-320.jpg)
Amazon Aurora (Debanjan Saha) - AWS DB Day
- 1. ยฉ 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Aurora Relational database reimagined for the cloud Debanjan Saha โ GM, Amazon Aurora Amazon Web Services April, 2016 Now available in Seoul region!
- 2. MySQL-compatible relational database Performance and availability of commercial databases Simplicity and cost-effectiveness of open source databases Delivered as a managed service What is Amazon Aurora?
- 3. Customers have been frustrated by the proprietary nature, high cost, and licensing terms of traditional, commercial-grade database providers. And while many companies have started moving toward more open engines like MySQL and Postgres, they often struggle to get the performance they need. Customers asked us if we could eliminate that inconvenient trade-off, and thatโs why we built Aurora. Jeff Bezos, Founder and CEO, Amazon.com Annual letter to the share holders, 2016
- 4. Re-imagined for the cloud Architected for the cloud โ e.g. moved the logging and storage layer into a multitenant, scale-out database-optimized storage service Leverages existing AWS services: Amazon EC2, Amazon VPC, Amazon DynamoDB, Amazon SWF, and Amazon S3 Maintain compatibility with MySQL โ customers can migrate their MySQL applications as-is, use all MySQL tools. Control PlaneData Plane Amazon DynamoDB Amazon SWF Amazon Route 53 Logging + Storage SQL Transactions Caching Amazon S3 1 2 3
- 5. Fastest growing service in AWS history Aurora customer adoption
- 6. Expedia: Online travel marketplace Migration from SQL Server ๏ง Real-time business intelligence and analytics on a growing corpus of online travel market place data. ๏ง Current SQL server based architecture is too expensive. Performance degrades as data volume grows. ๏ง Cassandra with Solr index requires large memory footprint and hundreds of nodes, adding cost. Aurora benefits: ๏ง Aurora meets scale and performance requirements with much lower cost. ๏ง 25,000 inserts/sec with peak up to 70,000. 30 ms average response time for write and 17 ms for read. Worldโs leading online travel company, with a portfolio that includes more than 150 travel sites in 70 countries.
- 7. Pearson Education: Publishing and testing Migration from MySQL ๏ง Pearsonโs applications enable student registration, instruction, and testing ๏ง Database reliability is critical when dealing with millions of studentsโ data. Data cannot be lost. Aurora benefits: ๏ง Allowed Pearson to stop self-managing their database while still achieving their performance and availability goals. ๏ง โNo more babysitting MySQL traditional async replicationโ A leading firm in educational publishing, testing, and certification, as well as in-class and online learning tools. Pearson operates in over 70 countries and serves millions of students.
- 8. Higher Performance, Lower Cost โข Fewer instances needed โข Smaller instances can be used โข No need to pre-provision storage โข No additional storage for read replicas Safe.com lowered their bill by 40% by switching from sharded MySQL to a single Aurora instance. Double Down Interactive (gaming) lowered their bill by 67% while also achieving better latencies (most queries ran faster) and lower CPU utilization.
- 9. Amazon Aurora is fast โฆ 5x faster than MySQL on SYSBENCH
- 10. WRITE PERFORMANCE READ PERFORMANCE MySQL SysBench results R3.8XL: 32 cores / 244 GB RAM 5X faster than RDS MySQL 5.6 & 5.7 Five times higher throughput than stock MySQL based on industry standard benchmarks. 0 25,000 50,000 75,000 100,000 125,000 150,000 0 100,000 200,000 300,000 400,000 500,000 600,000 700,000 Aurora MySQL 5.6 MySQL 5.7
- 11. WRITE PERFORMANCE READ PERFORMANCE Scaling with instance sizes Aurora scales with instance size for both read and write. Aurora MySQL 5.6 MySQL 5.7
- 12. Reproducing benchmark results https ://d0.a wsstat ic . com /product -m ark eting/Aurora /R DS_ Auro ra_Perf orm ance_Assessm ent_Benchm ark ing_v 1-2 .pdf AMAZON AURORA R3.8XLARGE R3.8XLARGE R3.8XLARGE R3.8XLARGE R3.8XLARGE โข Create an Amazon VPC (or use an existing one). โข Create four EC2 R3.8XL client instances to run the SysBench client. All four should be in the same AZ. โข Enable enhanced networking on your clients โข Tune your Linux settings (see whitepaper) โข Install Sysbench version 0.5 โข Launch a r3.8xlarge Amazon Aurora DB Instance in the same VPC and AZ as your clients โข Start your benchmark! 1 2 3 4 5 6 7
- 13. Beyond benchmarks If only real world applications saw benchmark performance POSSIBLE DISTORTIONS Real world requests contend with each other Real world metadata rarely fits in data dictionary cache Real world data rarely fits in buffer cache Real world production databases need to run with HA enabled
- 14. Do fewer IOs Minimize network packets Cache prior results Offload the database engine DO LESS WORK Process asynchronously Reduce latency path Use lock-free data structures Batch operations together BE MORE EFFICIENT How did we achieve this? DATABASES ARE ALL ABOUT I/O NETWORK-ATTACHED STORAGE IS ALL ABOUT PACKETS/SECOND HIGH-THROUGHPUT PROCESSING DOES NOT ALLOW CONTEXT SWITCHES
- 15. What about availability โPerformance only matters if your database is upโ
- 16. Six copies across three availability zones 4 out 6 write quorum; 3 out of 6 read quorum Peer-to-peer replication for repairs Volume striped across hundreds of storage nodes SQL Transaction AZ 1 AZ 2 AZ 3 Caching SQL Transaction AZ 1 AZ 2 AZ 3 Caching Read and write availabilityRead availability Fault-tolerant storage
- 17. Read replicas are failover targets Aurora cluster contains primary node and up to fifteen secondary nodes Failing database nodes are automatically detected and replaced Failing database processes are automatically detected and recycled Secondary nodes automatically promoted on persistent outage, no single point of failure Customer application may scale-out read traffic across secondary nodes AZ 1 AZ 3AZ 2 Primary Node Primary Node Primary Node Primary Node Primary Node Secondary Node Primary Node Primary Node Secondary Node ๏ง Customer specifiable fail-over order ๏ง Read balancing across read replicas
- 18. Faster failover App RunningFailure Detection DNS Propagation Recovery Recovery DB Failure MYSQL App Running Failure Detection DNS Propagation Recovery DB Failure AURORA WITH MARIADB DRIVER 1 5 - 2 0 s e c 3 - 2 0 s e c
- 19. ALTER SYSTEM CRASH [{INSTANCE | DISPATCHER | NODE}] ALTER SYSTEM SIMULATE percent_failure DISK failure_type IN [DISK index | NODE index] FOR INTERVAL interval ALTER SYSTEM SIMULATE percent_failure NETWORK failure_type [TO {ALL | read_replica | availability_zone}] FOR INTERVAL interval Simulate failures using SQL To cause the failure of a component at the database node: To simulate the failure of disks: To simulate the failure of networking:
- 20. Compatible with the MySQL ecosystem Use all your existing tools and applications
- 21. Well established MySQL ecosystem Business Intelligence Data Integration Query and Monitoring SI and Consulting Source: Amazon โWe ran our compatibility test suites against Amazon Aurora and everything just worked." - Dan Jewett, Vice President of Product Management at Tableau
- 22. Advanced monitoring 50+ system/OS metrics | sorted process list view | 1โ60 sec granularity alarms on specific metrics | egress to Amazon CloudWatch Logs | integration with third-party tools ALARM
- 23. Many 3rd party monitoring tools
- 24. Correlate Aurora metrics with metrics and events from the rest of your infrastructure Monitoring the whole stack
- 25. How much does it cost? Less expensive than MySQL 1/10 the cost of commercial databases
- 26. Cost of ownership: Aurora vs. MySQL MySQL configuration hourly cost Primary r3.8XL Standby r3.8XL Replica r3.8XL Replica R3.8XL Storage 6 TB / 10 K PIOP Storage 6 TB / 10 K PIOP Storage 6 TB / 5 K PIOP Storage 6 TB / 5 K PIOP $1.33/hr $1.33/hr $1.33/hr $1.33/hr $2,42/hr $2,42/hr $2,42/hr Instance cost: $5.32 / hr Storage cost: $8.30 / hr Total cost: $13.62 / hr $2,42/hr
- 27. Cost of ownership: Aurora vs. MySQL Aurora configuration hourly cost Instance cost: $4.86 / hr Storage cost: $4.43 / hr Total cost: $9.29 / hr Primary r3.8XL Replica r3.8XL Replica R3.8XL Storage / 6 TB $1.62 / hr $1.62 / hr $1.62 / hr $4.43 / hr *At a macro level Aurora saves over 50% in storage cost compared to RDS MySQL. 31.8% Savings ๏ง No idle standby instance ๏ง Single shared storage volume ๏ง No POIPs โ pay for use I/O ๏ง Reduction in overall IOP
- 28. Cost of ownership: Aurora vs. MySQL Further opportunity for saving Instance cost: $2.43 / hr Storage cost: $4.43 / hr Total cost: $6.86 / hrStorage IOPs assumptions: 1. Average IOPs is 50% of Max IOPs 2. 50% savings from shipping logs vs. full pages 49.6% Savings Primary r3.8XL Replica r3.8XL Replica r3.8XL Storage / 6TB $0.81 / hr $0.81 / hr $0.81 / hr $4.43 / hr r3.4XL r3.4XL r3.4XL ๏ง Use smaller instance size ๏ง Pay-as-you-go storage
- 29. Ready to move? We made it easy to migrate ..
- 30. 1. Establish baseline a. RDS MySQL to Aurora DB snapshot migration b. MySQL dump/import 2. Catch-up changes a. Binlog replication b. Tungsten replicator Simplify migration from RDS MySQL Application Users MySQL Aurora Network
- 31. Migration from EC2 & on-premise MySQL Data migration service โข Logical data replication from on-premise or EC2 โข Code & schema conversion across engines S3 integration โข Load partial datasets directly from / to S3 โข Ingest large database snapshots (>2TB) โข Snowball integration โข Ingest huge database snapshots (>10TB) โข Send us your data in a suitcase!
- 32. ๏ผ Move data to the same or different database engine ๏ผ Keep your apps running during the migration ๏ผ Start your first migration in 10 minutes or less ๏ผ Replicate within, to, or from Amazon EC2 or RDS AWS Database Migration Service Migration non-MySQL databases
- 33. Q&A