The mysqlnd replication and load balancing plugin
- 1. Ulf Wendel, Oracle Web scale made easy (just kidding...) The mysqlnd replication and load balancing plugin
- 2. MySQL scale-out solutions Part 1 - Proven and designed for web-scale
- 3. The speaker says... MySQL offers a variety of scale-out solutions. Which are they, when to use and how to use from an application developers point of view?
- 4. Developer dreams Day 1 (Launch) – web site visitors The first customer Systemload: 0.1 100%
- 5. The speaker says... Let's take a PHP developer centric look at a high-growth web site. The web site is build on top of the industrial leading LAMP platform: Linux, MySQL, Apache and PHP. The day before the launch, the developer dreams the dream of his options or shares rising in value: 2x, 5x, 42x ...
- 6. Developer dreams Day 2 (Growth) – web site visitors The first customer Twitter follower Facebook friend A slashdot reader A TV moderator Systemload: 0.5
- 7. The speaker says... ... a wonderful dream. The enterprise becomes popular, IPO is on the horizon. There are many reasons for growth. As a developer you should know about the MySQL scale-out solutions to be prepared. According to a MySQL Online Quickpoll from 2009 a majority of the MySQL users is using MySQL Replication.
- 8. Developer dreams Day 3 (Success) – scale out needed 408 Request Timeout 503 Service Unavailable 1040 SQLSTATE: 08004 (ER_CON_COUNT_ERROR) - Too many connections Systemload: 42
- 9. The speaker says... Dear developer, help is on the way. The PHP mysqlnd replication and load balancing plugin ( PECL/mysqlnd_ms ) is a drop-in solution for your PHP MySQL web application adding MySQL scale-out support. For basic scale-out setups no code changes are required. As of PHP 5.3 all three PHP MySQL APIs ( mysql , mysqli , PDO_MySQL ) can be compiled to use the MySQL native driver for PHP ( mysqlnd ) library. Mysqlnd is shipped with PHP as of version 5.3. It is a default as of PHP 5.4.
- 10. Scale out means clustering Scale-out horizontally TCO: Commodity hardware
- 11. TCO: Proven Open Source solutions MySQL Server 1 MySQL Server 2 MySQL Server n
- 12. The speaker says... Scale out, not up. Use a cluster of commodity servers. Scale horizontally building upon proven Open Source solutions. Try MySQL Replication, MySQL Cluster or a custom MySQL scale-out solution.
- 13. Asynchronous read scale-out Master: writes, Slaves: reads
- 15. Other use cases: backup, HA, OLTP/warehousing MySQL Replication Writes Reads Master Slave Slave Slave
- 16. The speaker says... MySQL Replication is used for read scale-out. Writes are send to a MySQL master, reads are send to a MySQL slave. The master sends updates to the slaves in an asynchronous way. Slaves may not have the latest changes. MySQL Replication can also be used for: Backup – perfom blocking backup on slave
- 17. High Availablility – for example, remote slaves
- 18. Warehousing – OLTP reporting on the slaves
- 19. Real-time transactional write scale-out 99,999% availability
- 20. Auto sharding, shared nothing architecture
- 21. Web use cases: session server, tele communication MySQL Cluster SQL Node SQL Node Cluster Data Node Cluster Data Node Cluster Data Node
- 22. The speaker says... MySQL Cluster is used for real-time transactional read and write scale-out. Data is auto-partitioned on cluster data nodes resulting in 99.999% availability. The cluster is accessed through MySQL Application Cluster Nodes. The most basic MySQL Application Cluster Node is a SQL Node – a MySQL server (mysqld). If MySQL Cluster is new to you, think of MySQL Cluster as a storage engine for the network. Like InnoDB and MyISAM are storage engines for disks.
- 23. For example, Partitioning Horizontal: rows in different tables
- 24. Vertical: columns in additional tables
- 25. ... sharding Custom Cluster Shard 1 ID % 2 = 0 Node 3 Cats Node 4 Dogs Shard 2 ID % 2 = 1
- 26. The speaker says... Special requirements may make it necessary to build a custom cluster. For example, your application may require a sharding solution.
- 27. On using any database cluster Part 2 - Required application changes
- 28. Tasks: Choose servers to execute statement
- 29. Load balance within candidates
- 31. Automatic fail over for High Availability Application using any DB cluster MySQL Server 1 MySQL Server 2 MySQL Server n
- 32. The speaker says... All PHP applications talking to a cluster of MySQL database servers are facing the same tasks. Replacing a single database with a database cluster means changing the 1:1 relation between the application and the database into a 1:n relation . 1:n puts an additional task on the application: find the right n, find the right server to talk to.
- 33. The plugin for all of you MySQL Server 1 MySQL Server 2 MySQL Server n
- 34. The speaker says... PECL/mysqlnd_ms is a plugin for the MySQL native driver for PHP (mysqlnd) library . The mysqlnd library is part of PHP as of version 5.3. As of 5.4 mysqlnd is a default. All three PHP MySQL APIs (mysql, mysqli, PDO_MySQL) can be compiled to use the mysqlnd library, thus all existing PHP MySQL application are supported by the plugin. From an applications point of view a mysqlnd plugin can act as a transparent proxy. Depending on plugin an use case, no code changes are needed. A plugin is a drop-in solution.
- 35. Plugin focus: MySQL Replication Part 3 - Tasks, solutions and limitations
- 36. Using MySQL Replication Statement redirection Read/write split (optional: master on write)
- 37. Replication filter (table based parititioning)
- 38. Custom: callback Load Balancing Round robin, random, random once
- 39. Custom: callback Connection Pooling Lazy connections
- 40. Failover
- 41. The speaker says... The plugin is not limited to but optimized for MySQL Replication. The common clustering support feature set focusses on MySQL Replication. For example, statement redirection covers read/write splitting and MySQL Replication filter support. All read accesses are executed on the slave servers, all writes are performed on the master. MySQL Replication supports table based partitioning, replicating selected tables to selected slaves only. The plugin can be made aware of this.
- 42. Speaking code /* Connection user handle represents a connection pool */ $mysqli = new mysqli( "hostname" , $user, $pw, $db, $port); /* Read/Write split, Load Balancing */ $res = $mysqli->query("SELECT 'Slave speaking' FROM DUAL"); var_dump($res->fetch_all(MYSQLI_ASSOC)); { "hostname" :{ "master":{ "host":"localhost" }, "slave":{ "slave_0":{"host":"192.168.2.27"} }, } }
- 43. The speaker says... No code changes for basic use cases. Install the plugin following standard PHP PECL procedures. Create a configuration file. Enable the plugin and set its configuration file in your PHP configuration using mysqlnd_ms.enable=1 and mysqlnd_ms.ini_file=config.ini . In the plugin configuration create a section with the name of the host your application connects to, for example „hostname“. List master and slave servers. Any connection to „hostname“ is now handled by the plugin. Using any API.
- 44. Some code changes /* Connection user handle represents a connection pool */ $mysqli = new mysqli("hostname", $user, $pw, $db, $port); / * R/W split: no SELECT, run on master */ $mysqli->query("SET @myrole='Master'); /* R/W split: overruled by SQL hint */ $sql = sprintf("/*%s*/%s", MYSQLND_MS_MASTER_SWITCH, "SELECT @myrole AS _role"); $mysqli->query($sql); var_dump($res->fetch_all(MYSQLI_ASSOC));
- 45. The speaker says... Non-trivial use cases require support by the application. In other words: existing applications may need updates, new applications need to be designed appropriately. In the example a SQL hint is used to force executio of a SELECT statement on a MySQL Replication master server. There are SQL hints for running a statement on the master, on the slave and on the last used server. Alternative: new APIs call, no win compared to SQL hint.
- 46. Why and when code changes $mysqli = new mysqli("hostname", $user, $pw, $db, $port); $mysqli->query("SET @myrole='Master'); $mysqli->query("SELECT myrole AS _role"); Without plugin With plugin
- 47. The speaker says... An application not designed for use with a cluster assumes a connection handle always points to the same server (1:1), having the same state. When using the plugin a connection handle represents a pool of connections (1:n), potentially having different states. Connection switches may occur whenever a statement is executed. After the switch, the connection handle may point to a connection with a different state. Applications need to take care and hint the plugin.
- 48. A trouble maker in detail Connection State Connection settings, e.g. charsets
- 50. Temporary tables
- 51. Table locks
- 52. Session system variables and user variables
- 53. Prepared statements (API and SQL PREPARE)
- 55. Locks acquired with GET_LOCK
- 56. Units of work from an application point of view
- 57. The speaker says... No client library, no client proxy can monitor all state changing actions. Applications in need for a certain connection state need to control the plugins connection switches using : SQL hints MYSQLND_MS_SLAVE_SWITCH
- 59. MYSQLND_MS_LAST_USED_SWITCH Callback to replace built-in server selection logic
- 60. Trouble maker transaction Unit of work Part of the connection state
- 61. Application to prevent connection switches Workaround PHP 5.3: SQL hints must be used
- 62. PHP 5.4: API calls monitored, switch auto-disabled Replication cluster tip Always run on master, only master has latest data!
- 63. use trx_stickiness = master
- 64. The speaker says... A transaction is part of the connection state. For the duration of a transaction, server switches must be disabled. Transactions can be controlled by SQL statements and API calls. The plugin cannot monitor all of them with reasonable effort. Thus, you must hint the plugin. Disable load balancing by adding SQL hints (MYSQLND_MS_LAST_USED_SWITCH), you install your own load balancer logic (callback). As of PHP 5.4, the plugin monitors API calls to auto-disable switching.
- 65. Slave request read-only: statement begins with SELECT
- 66. read-only: statement begins with slave SQL hint
- 67. custom: callback picks slave for statement Redirection: Read/Write split Writes Reads (SELECT, SQL hint) Master Slave Slave Slave
- 68. The speaker says... MySQL Replication is for read scale-out. Read-only statements shall go to slaves, writes to the master. The plugin automatically redirects a statement to a slave, if the statement begins with SELECT or the MYSQLND_MS_SLAVE_SWITCH SQL hint. All other statements are considered as write operations and executed on the master. By default SHOW or CALL will be run on the master. It is a design principle of the plugin that all automatic decisions can be overruled. A callback can be installed to pick the server for running a statement on, if needed.
- 69. Optimization: use master only after write MySQL Replication is asynchronous
- 70. MySQL slaves can lag behind
- 71. Most applications need current data after write Redirection: Master on write Writes, Reads after write Reads before write Master Slave Slave
- 72. The speaker says... The master_on_write configuration option is an optimization of the R/W split. MySQL Replication slaves can lag behind the master because MySQL Replication is asynchronous. When using master_on_write all reads made after a write are executed on the master for the duration of the web request. Applications that perform a write often need to rely on fresh reads afterwards. Think of a web shop creating an order, writing to the master. If reading again from the order table, within the same web request, the application should access the master to make sure the previously written is available.
- 73. MySQL Replication filter support Slaves replicate different databases/tables Redirection: Partitioning Writes Reads (Filter) DB_Report DB_Shop DB_Report DB_Internal DB_Shop DB_Shop Master Slave Slave Slave
- 74. The speaker says... MySQL Replication supports partitioning: slaves can be configured to replicate only selected databases and tables using MySQL Replication filter. Filter help to reduce the amount of internal replication messages. Less frequently accessed data can be replicated to only few slaves. In future versions, plugin can be made aware of the filtering rules for appropriate statement redirection. Every statement is analyzed to find a slave for reading. If no match, the plugin redirects the statement to the master.
- 75. Built-in or user-defined through callback Round robin: iterate over slaves for each read
- 76. Random: pick random slave for each read
- 77. Random once: pick random slave for all reads Load Balancing Writes Load Balancing of Reads Master Slave Slave Slave
- 78. The speaker says... MySQL Replication is about read scale-out. The plugin helps distributing the load over the read-only slaves. The best and default load balancing stategy is the one with the lowest impact on the connection state: random once. Minimize switches for the life-span of your PHP script. Pick a random slave during startup and use it for all reads. The life-span of PHP is the of a web request. It is short. Two web requests will end up on two random slaves. Load balancing adaptive to server health can be achieved through a user-defined callback.
- 79. Caution: connection state change Default: failover=disabled, raise exception
- 80. Optional: failover=master, automatic Pooling: connection failover Writes Reads Master Slave Slave Slave
- 81. The speaker says... Automatic and transparent failover is impossible with stateful connections. The plugin does not and cannot know the full state of a connection. Parts of the state are only known to the server. By default the plugin leaves it to the application to handle a connection failure. Like without the plugin, it is up to the application to decide what to do. Applications using stateless connections can enable automatic failover to the master.
- 82. Do not open connection before needed Delay open until statement execution
- 83. Reduce number of connections, reduce load Pooling: lazy connections Writes Reads Master Slave Slave Slave
- 84. The speaker says... By default, to reduce the connection pool size of every PHP web request, connections are not opened before executing a statement. Imagine you have 50 concurrent web requests, you configured four slaves and, you are using random once. If not using lazy connections, the plugin opens 50 x 4 = 200 slave connections but uses at max 50 of them (random once). Every connection occupies resources – ever seen a PHP application opening connections before using? There are many badly written ones...
- 85. See also...
- 86. Documentation and resources php.net/mysqlnd_ms Installation, Quickstart, Concepts, Reference php.net/mysqlnd Introduction, Changes, C plugin API
- 87. THE END Credits: Andrey Hristov, Johannes Schlüter - Contact: ulf.wendel@oracle.com