Netherlands Tech Tour 03 - MySQL Cluster
0 likes660 views
The document discusses MySQL Cluster, an in-memory database that provides real-time performance, scalability, and high availability. It describes how MySQL Cluster is used by major companies like PayPal, Big Fish, Alcatel-Lucent, and Playful Play to power applications that require fast data access, high scalability, and near 100% uptime. These companies chose MySQL Cluster because it can meet the demanding requirements for their mission-critical systems.
![NoSQL
API
for
Node.js
&
FKs
FKs
enforced
on
all
APIs:
{ message: 'Error',
sqlstate: '23000',
ndb_error: null,
cause:
{message: 'Foreign key constraint violated: No parent row found [255]',
sqlstate: '23000',
ndb_error:
{ message: 'Foreign key constraint violated: No parent row found',
code: 255,
classification: 'ConstraintViolation',
handler_error_code: 151,
status: 'PermanentError' },
cause: null } }
Copyright
2015,
oracle
and/or
its
affiliates.
All
rights
reserved
24](https://image.slidesharecdn.com/03-mysqlcluster7-150423033237-conversion-gate02/85/Netherlands-Tech-Tour-03-MySQL-Cluster-24-320.jpg)
![MySQL
Cluster
Manager
1.3.3
GA
“Unmanaged”
produc?on
Cluster
mcm> create cluster --import
mcm> import config [--dryrun]
mcm> import cluster[--dryrun]
Cluster
now
managed
by
MCM
Import
a
running
Cluster
into
MCM
Copyright
2015,
oracle
and/or
its
affiliates.
All
rights
reserved
40](https://image.slidesharecdn.com/03-mysqlcluster7-150423033237-conversion-gate02/85/Netherlands-Tech-Tour-03-MySQL-Cluster-40-320.jpg)
Netherlands Tech Tour 03 - MySQL Cluster
- 1. MySQL Cluster In-‐Memory Real-‐Time Performance, Web Scalability & 99.999% Availability Mark Swarbrick Principle Presales Consultant
- 2. Safe Harbor Statement The following is intended to outline our general product direc?on. It is intended for informa?on purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or func?onality, and should not be relied upon in making purchasing decisions. The development, release, and ?ming of any features or func?onality described for Oracle’s products remains at the sole discre?on of Oracle. Copyright 2015, oracle and/or its affiliates. All rights reserved 2
- 3. consumer employee business model around-the-clock availability mobile social personalized service Information any time, anywhere flawless engagement insight real-time fulfillment new markets
- 4. Focus on driving the business rather than on infrastructure Driving new Database Requirements In-‐Memory Real-‐Time Performance Extreme Read & Write Scalability Rock Solid Availability Rapid Service Innova?on Copyright 2015, oracle and/or its affiliates. All rights reserved 4
- 5. MySQL Cluster Overview • Memory op?mized tables with durability • Predictable Low-‐Latency, Bounded Access Time REAL-‐TIME • Auto-‐Sharding, Mul?-‐Master • ACID Compliant, OLTP + Real-‐Time Analy?cs HIGH SCALE, READS + WRITES • Shared nothing, no Single Point of Failure • Self Healing + On-‐Line Opera?ons 99.999% AVAILABILITY • Key/Value + Complex, Rela?onal Queries • SQL + Memcached + JavaScript + Java + HTTP/REST & C++ SQL + NoSQL • Open Source + Commercial Edi?ons • Commodity hardware + Management, Monitoring Tools LOW TCO Copyright 2015, oracle and/or its affiliates. All rights reserved 5
- 6. Who’s Using MySQL Cluster? Copyright 2015, oracle and/or its affiliates. All rights reserved 6
- 7. Company Overview PayPal allows any business or individual with an email address to securely, conveniently and cost-‐effec?vely send and receive payments online. ApplicaSon PayPal built a cloud-‐based globally-‐distributed database with 100 TB of user-‐related data based on MySQL Cluster. “Must NOT lose data” system, delivering 99,999% availability, transac?onal, with data available WW anywhere in < 1 Sec. Why MySQL Cluster? “You can achieve high performance and availability without giving up rela8onal models and read consistency.” Daniel Aus?n, Chief Architect, PayPal PayPal Copyright 2015, oracle and/or its affiliates. All rights reserved 7
- 8. Company Overview World's largest producer of casual games. Big Fish has distributed more than 2.5 billion games to customers in 150 countries. ApplicaSon Serve real-‐?me recommenda?ons to Big Fish customers, requiring high velocity data ingest, low latency access, online scalability, 99.999% availability and Enterprise SLAs & roadmap. Why MySQL Cluster CGE? "The MMS pla@orm is a strategic project within Big Fish. We couldn’t afford to take any chances and MySQL Cluster provided us with a proven and trusted solu8on to meet the demands of both our business and our users.“ Sean Chighizola, Sr. Director of Database Administra?on, Big Fish Big Fish Copyright 2015, oracle and/or its affiliates. All rights reserved 8
- 9. Company Overview Global telecommunica?ons equipment company, focuses on fixed, mobile, and converged networking hardware, IP technologies, somware, and services. ApplicaSon MySQL Cluster CGE is at the heart of Alcatel-‐Lucent’s Subscriber Data Manager to deliver converged IMS services to mobile and fixed line users. Why MySQL Cluster CGE? • Delivers the performance, scalability and availability required by this mission cri?cal real-‐?me applica?on • Real-‐?me read & write access for tens of millions of subscribers in a single system • Always-‐on service: no offline maintenance window for services users depend on (voice, SMS, email, Web, social media…) • Rapid delivery & low TCO Alcatel-‐Lucent 23/04/15 Copyright 2015, oracle and/or its affiliates. All rights reserved 9
- 10. Company Overview Playful Play created La?n America’s #1 Facebook game based on "El Chavo del 8“. ApplicaSon Playful Play selected MySQL Cluster CGE running on a public cloud to power their gaming plavorm, providing: • 45% improvement in performance • 99.999% up?me • 80% reduc?on in DBA overhead • Local language support, 24x7 Why MySQL Cluster CGE? “MySQL Cluster has allowed us to keep pace with the rapid success of the game. Being the number one Facebook game in La8n America is a big responsibility, and we rely on Oracle’s products and MySQL support for con8nued growth.” Ricardo Rocha, CEO, Playful Play Playful Play Copyright 2015, oracle and/or its affiliates. All rights reserved 10
- 11. MySQL ReplicaSon MySQL Fabric DRBD Windows/ Solaris/ Clusterware Clustering or Oracle VM MySQL Cluster MySQL HA Solu?ons 19th February 2015 9 9 . 9 9 9 % Copyright 2015, Oracle and/or its affiliates. All rights reserved 11
- 12. Oracle MySQL HA & Scaling Solu?ons MySQL ReplicaSon MySQL Fabric Oracle VM Template Oracle Clusterware Solaris Cluster Windows Cluster DRBD MySQL Cluster App Auto-‐Failover ✖ ✔ ✔ ✔ ✔ ✔ ✔ ✔ Data Layer Auto-‐Failover ✖ ✔ ✔ ✔ ✔ ✔ ✔ ✔ Zero Data Loss MySQL 5.7 MySQL 5.7 ✔ ✔ ✔ ✔ ✔ ✔ Plavorm Support All All Linux Linux Solaris Windows Linux All Clustering Mode Master + Slaves Master + Slaves AcSve/ Passive AcSve/ Passive AcSve/ Passive AcSve/ Passive AcSve/ Passive MulS-‐ Master Failover Time N/A Secs Secs + Secs + Secs + Secs + Secs + < 1 Sec Scale-‐out Reads ✔ ✖ ✖ ✖ ✖ ✖ ✔ Cross-‐shard opera?ons N/A ✖ N/A N/A N/A N/A N/A ✔ Transparent rou?ng ✖ For HA ✔ ✔ ✔ ✔ ✔ ✔ Shared Nothing ✔ ✔ ✖ ✖ ✖ ✖ ✔ ✔ Storage Engine InnoDB+ InnoDB+ InnoDB+ InnoDB+ InnoDB+ InnoDB+ InnoDB+ NDB Single Vendor Support ✔ ✔ ✔ ✔ ✔ ✖ ✔ ✔ 23/04/15 Copyright 2015, oracle and/or its affiliates. All rights reserved 12
- 13. MySQL Cluster Architecture MySQL Cluster Data Nodes Clients Applica?on Layer Data Layer Copyright 2015, oracle and/or its affiliates. All rights reserved 13
- 14. MySQL Cluster Scaling MySQL Cluster Data Nodes Clients Applica?on Layer Data Layer Copyright 2015, oracle and/or its affiliates. All rights reserved 14
- 15. On-‐line Opera?ons • Scale the cluster (add & remove nodes on-‐line) • Repar??on tables • Upgrade / patch servers & OS • Upgrade / patch MySQL Cluster • Back-‐Up • Evolve the schema on-‐line, in real-‐?me Copyright 2015, oracle and/or its affiliates. All rights reserved 15
- 16. MySQL Cluster Auto-‐Installer • Fast configura?on • Auto-‐discovery • Workload op?mized • Repeatable best prac?ces Specify Workload Auto-‐ Discover Define Topology Deploy Copyright 2015, oracle and/or its affiliates. All rights reserved 16
- 17. Copyright 2015, oracle and/or its affiliates. All rights reserved 17
- 18. Copyright 2015, oracle and/or its affiliates. All rights reserved 18
- 19. NoSQL Access to MySQL Cluster data Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps JPA Cluster JPA PHP Perl Python Ruby JDBC Cluster J JS Apache Memcached MySQL JNI Node.JS mod_ndb ndb_eng NDB API (C++) MySQL Cluster Data Nodes
- 20. Schema-‐Free apps • Rapid applica?on evolu?on – New types of data constantly added – No ?me to get schema extended – Missing skills to extend schema – Ini?ally roll out to just a few users – Constantly adding to live system Copyright 2015, oracle and/or its affiliates. All rights reserved 20
- 21. Cluster & Memcached – Schema-‐Free <town:maidenhead,SL6> key value <town:maidenhead,SL6> key value Key Value town:maidenhead SL6 generic table Applica?on view SQL view Copyright 2015, oracle and/or its affiliates. All rights reserved 21
- 22. Cluster & Memcached -‐ Configured Schema <town:maidenhead,SL6> prefix key value <town:maidenhead,SL6> key value Prefix Table Key-‐col Val-‐col policy town: map.zip town code cluster Config tables town ... code ... maidenhead ... SL6 ... map.zip Applica?on view SQL view Copyright 2015, oracle and/or its affiliates. All rights reserved 22
- 23. Node.js NoSQL API • Na?ve JavaScript access to MySQL Cluster – End-‐to-‐End JavaScript: browser to the app & DB – Storing and retrieving JavaScript objects directly in MySQL Cluster – Eliminate SQL transforma?on • Implemented as a module for node.js – Integrates Cluster API library within the web app • Couple high performance, distributed apps, with high performance distributed database • Op?onally routes through MySQL Server – Use with InnoDB V8 JavaScript Engine MySQL Cluster Node.js Module MySQL Cluster Data Nodes Clients Copyright 2015, oracle and/or its affiliates. All rights reserved 23
- 24. NoSQL API for Node.js & FKs FKs enforced on all APIs: { message: 'Error', sqlstate: '23000', ndb_error: null, cause: {message: 'Foreign key constraint violated: No parent row found [255]', sqlstate: '23000', ndb_error: { message: 'Foreign key constraint violated: No parent row found', code: 255, classification: 'ConstraintViolation', handler_error_code: 151, status: 'PermanentError' }, cause: null } } Copyright 2015, oracle and/or its affiliates. All rights reserved 24
- 25. MySQL Cluster 7.4 GA Copyright 2015, oracle and/or its affiliates. All rights reserved 25
- 26. MySQL Cluster 7.4 GA • 200 Million NoSQL Reads/Sec • 2.5M SQL Ops/Sec • 50% Faster Reads • 40% Faster Mixed Performance • Ac?ve-‐Ac?ve Geographic Redundancy • Conflict Detec?on/ Resolu?on Ac?ve-‐Ac?ve • 5X Faster Maintenance Ops • Detailed Repor?ng Management Copyright 2015, oracle and/or its affiliates. All rights reserved 26
- 27. • Memory op?mized tables – Durable – Mix with disk-‐based tables • Massively concurrent OLTP • Distributed Joins for analy?cs • Parallel table scans for non-‐indexed searches • MySQL Cluster 7.4 FlexAsych – 200M NoSQL Reads/Second 27 MySQL Cluster 7.4 NoSQL Performance 200 Million NoSQL Reads/Second Copyright 2015, oracle and/or its affiliates. All rights reserved !"!!!! !50,000,000!! !100,000,000!! !150,000,000!! !200,000,000!! !250,000,000!! 2! 4! 6! 8! 10! 12! 14! 16! 18! 20! 22! 24! 26! 28! 30! 32! Reads&per&second& Data&Nodes& FlexAsync&Reads&
- 28. • Memory op?mized tables – Durable – Mix with disk-‐based tables • Massively concurrent OLTP • Distributed Joins for analy?cs • Parallel table scans for non-‐indexed searches • MySQL Cluster 7.4 DBT2 BM – 2.5M SQL Statements/Second 23/04/15 28 MySQL Cluster 7.4 SQL Performance 2.5M SQL Statements/Second Copyright 2015, oracle and/or its affiliates. All rights reserved !"!!!! !500,000!! !1,000,000!! !1,500,000!! !2,000,000!! !2,500,000!! !3,000,000!! 2! 4! 6! 8! 10! 12! 14! 16! SQL$Statements/sec$ Data$Nodes$ DBT2$SQL$Statements$per$Second$
- 29. Performance Enhancements MySQL Cluster 7.4 50% Read-‐Only Increase Copyright 2015, oracle and/or its affiliates. All rights reserved 29 0" 1000" 2000" 3000" 4000" 5000" 6000" 7000" 8000" 9000" 64" 128" 192" 256" 320" 384" 448" 512" Transac'ons)per)second) Threads) Sysbench)R/W) 7.4" 7.3" 7.2" 0" 1000" 2000" 3000" 4000" 5000" 6000" 7000" 8000" 9000" 10000" 64" 128" 192" 256" 320" 384" 448" 512" Transac'on)per)second) Threads) Sysbench)RO) 7.4" 7.3" 7.2" 40% Read/Write Increase
- 30. • Asynchronous replica?on between MySQL Clusters • Ac?ve-‐Ac?ve – Update anywhere – Conflict detec?on • Applica?on no?fied through excep?on tables • Can opt to have conflicts resolved automa?cally – Auto-‐conflict-‐resolu?on • Conflic?ng transac?on and dependent ones are rolled-‐back • No changes to applica?on schema Ac?ve-‐Ac?ve Geo-‐Replica?on Copyright 2015, oracle and/or its affiliates. All rights reserved 30
- 31. MySQL Cluster 7.4 Restart Improvements • Dura?on of long-‐running maintenance ac?vi?es dominated by Data Node restart ?mes • MySQL Cluster 7.4 = 5.5X faster restarts • Benefits both nodal and rolling restarts – Upgrades, add-‐node,… • Benefits both SQL and NoSQL APIs • Benefits both ”manual” and MySQL Cluster Manager opera?ons • Achieve 5x as much during a single maintenance window Make Data Node Restarts Fast! Copyright 2015, oracle and/or its affiliates. All rights reserved 31
- 32. MySQL Cluster 7.4 – Enhanced memory repor?ng • ndbinfo.memory_per_fragment memory usage informa?on for each fragment replica, for each table and index • Allocated memory and how much of that is actually in use. • Exposes – Fragmenta?on of fixed and var-‐sized fragment pages – Accurate Data and Index Memory use – Comparison of Primary and Backup fragment usage – Par??oning effec?veness Copyright 2015, oracle and/or its affiliates. All rights reserved 32
- 33. MySQL Cluster 7.4 – Enhanced ac?vity repor?ng • ndbinfo.operations_per_fragment ac?vity counters for each fragment replica, for each table and index • PK & scan access – requests, bytes, rows… • Exposes – How traffic maps to tables and indices – Query execu?on, use of indexes etc. – LDM and node imbalances – Hotspots and scan overloads Copyright 2015, oracle and/or its affiliates. All rights reserved 33
- 34. When to Consider MySQL Cluster l Scalability demands l Sharding for write performance? l Latency demands l Cost of each millisecond? l Up?me requirements l Cost per minute of down?me? l Failure versus maintenance? l Applica?on agility l Developer languages and frameworks? l SQL or NoSQL? Copyright 2015, oracle and/or its affiliates. All rights reserved 34
- 35. General Usage Considera?ons • MySQL Cluster is designed for – Short transac?ons – Many parallel transac?ons • U?lize simple access pa~erns for high running transac?ons – Use efficient scans and batching interfaces – AQL gives huge performance in JOIN opera?ons – MySQL Cluster 7.4 speeds up table scans • Storage engine configurable for each table…InnoDB or NDB MySQL Cluster Evalua?on Guide h~p://mysql.com/why-‐mysql/white-‐papers/mysql_cluster_eval_guide.php Copyright 2015, oracle and/or its affiliates. All rights reserved 35
- 36. MySQL Cluster Carrier Grade Edi?on (CGE) MySQL Cluster CGE MySQL Cluster Manager MySQL Enterprise Scalability MySQL Enterprise Audit MySQL Enterprise Security Oracle Premier Life?me Support Oracle Product Cer?fica?ons MySQL Enterprise Mornitor MySQL Workbench Copyright 2015, oracle and/or its affiliates. All rights reserved 36
- 37. Enhancing DevOps Agility, Reducing Down?me Automated Management • Start / Stop node or whole cluster • On-‐Line Scaling • On-‐Line Reconfigura?on • On-‐Line Upgrades • On-‐Line Backup & Restore • Import Running Cluster Self-‐Healing • Node monitoring • Auto-‐recovery extended to SQL + mgmt nodes HA Opera?ons • Cluster-‐wide configura?on consistency • Persistent configura?ons • HA Agents Copyright 2015, oracle and/or its affiliates. All rights reserved 37
- 38. Without MySQL Cluster Manager • 1 x preliminary check of cluster state • 8 x ssh commands per server • 8 x per-‐process stop commands • 4 x scp of config files (2 x mgmd & 2 x mysqld) • 8 x per-‐process start commands • 8 x checks for started and re-‐joined processes • 8 x process comple?on verifica?ons • 1 x verify comple?on of the whole cluster. • Excludes manual edi?ng of each configura?on file. • Total: 46 commands – 2.5 hours of a~ended opera?on With MySQL Cluster Manager mcm> upgrade cluster --package=7.4 mycluster; • Total: 1 Command -‐ – Una~ended Opera?on Somware Upgrade – example of benefits of MCM? Copyright 2015, oracle and/or its affiliates. All rights reserved 38
- 39. MCM: Upgrade Cluster mcm> upgrade cluster --package=7.4 mycluster; Copyright 2015, oracle and/or its affiliates. All rights reserved 39
- 40. MySQL Cluster Manager 1.3.3 GA “Unmanaged” produc?on Cluster mcm> create cluster --import mcm> import config [--dryrun] mcm> import cluster[--dryrun] Cluster now managed by MCM Import a running Cluster into MCM Copyright 2015, oracle and/or its affiliates. All rights reserved 40
- 41. Oracle Premier Support for MySQL • Straight from the Source • Largest Team of MySQL Experts • Backed by MySQL Developers • Forward Compatible Hot Fixes • MySQL Maintenance Releases • MySQL Support in 29 Languages • 24/7/365 • Unlimited Incidents • Knowledge Base • MySQL Consultative Support Rely on The Experts -‐ Get Unique Benefits "The MySQL support service has been essen?al in helping us with troubleshoo?ng and providing recommenda?ons for the produc?on cluster, Thanks." -‐-‐ Carlos Morales – Playfulplay.com Only From Oracle Copyright 2015, oracle and/or its affiliates. All rights reserved 41
- 42. Evaluate MySQL Cluster CGE • Navigate to http://edelivery.oracle.com/ and step through (selecting “MySQL Database” as the Product Pack) • Select MySQL Cluster Manager 30-‐Day Trial Copyright 2015, oracle and/or its affiliates. All rights reserved 42
- 43. Next Steps Learn More • www.mysql.com/cluster • Authen?c MySQL Curriculum: h~p://oracle.com/educa?on/mysql Try it Out • dev.mysql.com/downloads/cluster/ Let us know what you think • clusterdb.com • @clusterdb • forums.mysql.com/list.php?25 Copyright 2015, oracle and/or its affiliates. All rights reserved 43