Mysqlnd Query Cache
- 2. <Insert Picture Here> MySQL Client Side Caching Johannes Schlüter Twitter: @phperror MySQL Engineering – Connectors and Client Connectivity
- 3. # pecl install mysqlnd_qc-beta
- 4. Gracias por vuestra atención!
- 5. mysqlnd Server API (SAPI) CGI CLI Embed ISAPI NSAPI phttpd thttpd ... Zend Engine PHP Runtime PHP Extensions bcmath mysql mysqli mysqlnd pdo pdo_mysql xml ...
- 6. PHP 5.3 and mysqlndPHP PHPMemory PHPStreams Infrastructure mysqlnd – MySQL native driver for PHP MySQL Server ext/mysql mysqli PDO_mysql … PHP Module (Extension) API
- 7. Building PHP with mysqlnd • ./configure --with-mysql=mysqlnd --with-mysqli=msqlnd --with-pdo-mysql=mysqlnd • Default on Windows and some distributions
- 8. mysqlnd vs. libmysql MySQL Server MySQL Server mysqlnd libmysql PHP PHP PHP Memory libmysql Memory PHP Memory PHP Memory copy copy usedirectly copy
- 9. mysqlnd Statistics • Around 150 statistic values collected • mysqli_get_client_stats (), mysqli_get_connection_stats()
- 10. Asynchronous Queries /* Do something */ PHP Script MySQL query result query poll result $conn = new MySQLi(...); $conn->query( "SELECT * FROM t WHERE ....", MYSQLI_ASYNC); /* Process query results */ mysqli_poll($links, $errors, $reject, 1);
- 12. Sharding foreach ($all_links as $link) $link->query("SELECT 'test' ", MYSQLI_ASYNC); $processed = 0; do { $links = $all_links; if (!mysqli_poll($links, $errors, $reject, 1)) continue; /* TIMEOUT */ foreach ($links as $link) { if ($result = $link->reap_async_query()) { print_r($result->fetch_row()); mysqli_free_result($result); $processed++; } } } while ($processed < count($all_links));
- 13. mysqlnd plugins Plugin Hook mysqlnd Query mysqli::query()mysql_query() PDO::query() Wire Protocol Plugin Hook Network
- 14. Drupal, Symphony, phpMyFAQ, phpMyAdmin, Oxid, ... ext/mysql, ext/mysqli, ext/PDO_MYSQL MySQL Server mysqlnd Load Balancing Monitoring Performance mysqlnd plugin
- 15. Query Cache Plugin vs. MySQL Proxy Application client Application client Application client MySQL Proxy PHP client PHP client PHP client mysqlnd pluginmysqlnd plugin mysqlnd plugin MySQL Server MySQL Server
- 16. Experimental Extensions • By Oracle: – mysqlnd_sip – mysqlnd_mc – mysqlnd_ms – mysqlnd_pscache • By Community: – mysqlnd_uh (David Soria Parra / Mayflower GmbH) • http://pecl.php.net/
- 17. The Database Is The Bottleneck
- 18. Caching!
- 19. “Traditional” Caches • MySQL Server Cache – MySQL Query Cache • Application-Level Cache – PEAR::Cache, Zend_Cache, …
- 20. MySQL Query Cache (Server-Side) ✔ Integrated with the MySQL Server ✔ No application changes ✔ Automatic invalidation – table based ✗ Needs network communication with server ✗ Needs server resources (memory, CPU)
- 21. Application-Level Caches ✔ Can cache higher level structures – Complete rendered HTML blocks ✔ Multiple backends – Can be shared over multiple servers (via memcache etc.) ✗ No automatic invalidation ✗ Requires application changes
- 22. <Insert Picture Here> Introducing: mysqlnd Client Side Cache
- 23. # pecl install mysqlnd_qc-beta Andrey Hristov Ulf Wendel
- 24. mysqlnd Query Cache PHP mysql / mysqli / PDO_mysql mysqlnd Cache Backend Query Cache MySQL Server SELECT a FROM t No! SELECT a FROM t ResultResult Local Memory, APC, Memcache, Custom Handler
- 25. Key Properties • Transparent – PHP Extension hooking into mysqlnd • Works with ext/mysql, mysqli, pdo_mysql • Pluggable storage handler – By default: local memory, APC, memcache, SQLite – PHP Userspace • Invalidation via TTL – No automatic invalidation by server – Custom handlers may use custom invalidation logic
- 26. Transparent? $mysqli = new mysqli($host, $user, $pw, $db); $sql = “SELECT SLEEP(10) FROM table”; $start = microtime(true); $res = $mysqli->query($sql); $res = $mysqli->query($sql); $end = microtime(true); echo $end - $start; → 20.019539117813
- 27. Transparent? $mysqli = new mysqli($host, $user, $pw, $db); $sql = sprintf(“/*%s*/SELECT SLEEP(10) FROM table”, MYSQLND_QC_ENABLE_SWITCH); $start = microtime(true); $res = $mysqli->query($sql); $res = $mysqli->query($sql); $end = microtime(true); echo $end - $start; → 10.142804088593
- 28. Transparent! mysqlnd_qc.cache_by_default = 1 !Usually you should NOT do this!
- 29. SQL Hints • MYSQLND_QC_ENABLE_SWITCH – qc=on • MYSQLND_QC_DISABLE_SWITCH – qc=off • MYSQLND_QC_TTL_SWITCH – qc_ttl=
- 30. Storage Handlers ● Storage – Scope: request, process, machine, multi-machine – Location: distance to cache – Replacement strategy – Slam defense strategy ● Decide what to cache – is_select() - detect SQL hints ● Extended statistics – Storage statistics, traces, timings
- 31. Cache Expiry Client 1 MySQL Client 2...n Cache Hit MySQL Client 1 Client 2...n
- 32. Slam Defense Client 1 MySQL Client 2...n Expired MySQL Client 1 Client 2...n Cache Hit Refresh
- 33. Core Statistics • Collected by: mysqlnd_qc core – php.net/manual/en/function.mysqlnd_qc_get_core_stats.php • Scope: process – Process: mysqlnd_qc_get_core_stats() – Aggregated values from all PHP MySQL APIs • Contents: wide range – Cache usage and efficiency – Network related – Timings
- 34. Query Statistics and Backtraces • Collected by: mysqlnd_qc core – php.net/manual/en/function.mysqlnd_qc_get_query_trace_log .php • Scope: process – mysqlnd_qc_get_query_trace_log() – mysqlnd_qc_get_normalized_query_trace_log() • Contents – Origin - backtrace – Timings
- 35. Storage Handler Statistics • Collected by: storage handler – php.net/manual/en/function..mysqlnd_qc_get_cache_info.php • Scope: cache entry – Depends on storage handler scope – Aggregated values from all PHP MySQL APIs • Contents: none or assorted – Depends on storage handler support – APC: timings, hit ratio, result set size – Default: APC plus result set meta data
- 36. User-Defined Storage Handler • Procedural – php.net/manual/en/function.mysqlnd_qc_set_user_handlers.php – Callback functions • Object oriented – slideshare.net/nixnutz/mysqlnd-query-cache-plugin-userdefined- storage-handler – Interface mysqlnd_qc_handler • Extending mysqlnd_qc_handler_default
- 37. <?php /* auto_prepend_file = /path/to/prepend.php */ class my_qc extends mysqlnd_qc_handler_default { public function is_select($query) { if (preg_match("@from employees where@ism", $query)) { /* cache query for mysqlnd_qc.ttl seconds */ return true; } return parent::is_select($query); } } $qc = new my_qc(); mysqlnd_qc_change_handler($qc); ?>
- 38. Resources • php.net/mysqlnd_qc – Installation, Examples, Functions • php.net/mysqlnd – Introduction, Changes, C plugin API • slideshare.net/nixnutz/presentations – QC Basics – QC User defined storage handler – QC Statistics – QC Benchmark impressions
- 40. The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.