New features in Performance Schema 5.7 in action
4 likes621 views
The document discusses new features in Performance Schema 5.7, including improved instrumentation for locks, memory usage, stored routines, prepared statements, and variables. It provides examples of using Performance Schema tables like METADATA_LOCKS, TABLE_HANDLES, and prepared_statements_instances to diagnose issues like locks preventing DDL statements from completing and inconsistently timed stored procedure executions. Practices are suggested to identify memory usage and optimize prepared statement performance.
![• Variables
• Status variables
• global_status
• session_status
• status_by_[account|host|thread|user]
Variables instrumentation
37](https://image.slidesharecdn.com/pstutorial-150928121117-lva1-app6891/85/New-features-in-Performance-Schema-5-7-in-action-43-320.jpg)
![• Same information which is in
• SHOW [GLOBAL] STATUS
• I_S.GLOBAL_VARIABLES (removed in 5.7)
• I_S.SESSION_VARIABLES (removed in 5.7)
• Helps to watch session variables changes
Global and session variables
38](https://image.slidesharecdn.com/pstutorial-150928121117-lva1-app6891/85/New-features-in-Performance-Schema-5-7-in-action-44-320.jpg)
![• Same information which is in
• SHOW [GLOBAL] STATUS
• I_S.GLOBAL_STATUS (removed in 5.7)
• I_S.SESSION_STATUS (removed in 5.7)
Status variables
39](https://image.slidesharecdn.com/pstutorial-150928121117-lva1-app6891/85/New-features-in-Performance-Schema-5-7-in-action-45-320.jpg)
New features in Performance Schema 5.7 in action
- 1. New features in Performance Schema 5.7 in action September, 21, 2015 | M¨ovenpick Hotel | Amsterdam Sveta Smirnova, Nickolay Ihalainen, Vlad Lesin
- 2. •Performance Schema Configuration for our Tutorial •Locks diagnostic: MDL, table, index •Memory usage •Stored routines instrumentation •Prepared Statements •Variables •Replication: new tables, slave diagnostic, GTID Table of Contents 2
- 3. Performance Schema Configuration for our Tutorial 3
- 4. • 87 tables • 1006 instruments • 42 variables Performance Schema in version 5.7 4
- 5. • ON by default • Only global, thread, statements and transactions instrumentation enabled • All other consumers are disabled Performance Schema defaults 5
- 6. • Memory allocated on demand • You don’t need to limit size of tables anymore • Sys schema included into standard MySQL distribution • You can turn statistics on or off for particular host and/or user • Size of SQL_DIGEST is tunable Configuraiton improvements in 5.7 6
- 7. • We will turn required instrumentation ON for each example separately • We will use pattern update performance_schema.setup_consumers set enabled=’yes’ where name like ’OUR_REQUIREMENT_%’; update performance_schema.setup_instruments set enabled=’yes’, timed=’yes’ where name like ’OUR_REQUIREMENT_%’; • Be careful! • They are memory and CPU intensive • Do not turn them all ON until needed Prepare 7
- 9. • Table METADATA_LOCKS • Which thread is waiting for a lock • Which thread holds the lock • Not only for talbes: GLOBAL, SCHEMA, TABLE, FUNCTION, PROCEDURE, EVENT, COMMIT, USER LEVEL LOCK, TABLESPACE MDL 9
- 10. mysql> select processlist_id, object_type, lock_type, lock_status, source -> from metadata_locks join threads on (owner_thread_id=thread_id) -> where object_schema=’employees’ and object_name=’titles’G *************************** 1. row *************************** processlist_id: 4 object_type: TABLE lock_type: EXCLUSIVE lock_status: PENDING – waits source: mdl.cc:3263 *************************** 2. row *************************** processlist_id: 5 object_type: TABLE lock_type: SHARED_READ lock_status: GRANTED – holds source: sql_parse.cc:5707 METADATA_LOCKS: example 10
- 11. • Start vm • Login: ubuntu • Password: ubuntu • Run load ./test1.sh CALL help_task()G CALL help_solve()G CALL task_prepare(); • What prevents ALTER from finishing? MDL: practice 11
- 12. • Table TABLE_HANDLES • Not only locks, but also information about open tables • FLUSH TABLES removes data from this table Table locks 12
- 13. mysql1> select count(*) from employees where first_name like ’Svet%’; • While sleeping, check what is going on in parallel connection: mysql2> select * from table_handlesG *************************** 1. row *************************** OBJECT_TYPE: TABLE OBJECT_SCHEMA: employees OBJECT_NAME: employees OBJECT_INSTANCE_BEGIN: 140544885988272 OWNER_THREAD_ID: 23 OWNER_EVENT_ID: 818320 INTERNAL_LOCK: NULL EXTERNAL_LOCK: READ EXTERNAL – Table lock! 1 row in set (0.00 sec) Table locks: example 13
- 14. mysql1> select count(*), sleep(10) from employees where emp_no=10001; • In parallel connection: mysql2> select * from table_handlesG *************************** 1. row *************************** OBJECT_TYPE: TABLE OBJECT_SCHEMA: employees OBJECT_NAME: employees OBJECT_INSTANCE_BEGIN: 140544885988272 OWNER_THREAD_ID: 23 OWNER_EVENT_ID: 1011419 INTERNAL_LOCK: NULL EXTERNAL_LOCK: NULL – Now everything is good: index access 1 row in set (0.00 sec) Table locks: example 14
- 15. • Run load ./tables.sh CALL help_task()G CALL help_solve()G • Why so many threads are waiting for a lock? • Fix the issue • We can also examine table cache content Table Handles: practice 15
- 17. • You could not diagnose where memory gone before version 5.7 • Buffers? • Temporary tables? • Internal structures which are out of user control? • There is no leak, simply OS did not show memory as freed yet? Why this is our favorite improvement? 17
- 18. • free • top • vmstat • Investigation • There was no way to know how exactly memory was allocated Diagnostic tools before 5.7 18
- 19. • free $free total used free shared buffers cached Mem: 16149184 6223916 9925268 317536 1048 3655160 -/+ buffers/cache: 2567708 13581476 Swap: 2110460 0 2110460 • top • vmstat • Investigation Diagnostic tools before 5.7 18
- 20. • free • top $top Tasks: 295 total, 3 running, 292 sleeping, 0 stopped, 0 zombie %Cpu(s): 3.0 us, 0.8 sy, 0.1 ni, 95.4 id, 0.8 wa, 0.0 hi, 0.0 si, 0.0 st KiB Mem: 16149184 total, 6231688 used, 9917496 free, 1048 buffers KiB Swap: 2110460 total, 0 used, 2110460 free. 3670752 cached Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1914 mysql 20 0 670m 95m 1296 S 0.7 1.2 2:42.14 mysqld • vmstat • Investigation Diagnostic tools before 5.7 18
- 21. • free • top • vmstat $vmstat -t 5 3 procs –––––––––––memory––––––––––– –––swap–– –––––io–––– –system–– ––––––cpu... r b swpd free buff cache si so bi bo in cs us sy id wa... 2 0 0 9923160 1048 3662724 0 0 168 86 167 674 3 1 87... 0 0 0 9923252 1048 3662904 0 0 30 122 1168 5264 3 1 96... 0 0 0 9922864 1048 3663120 0 0 25 128 1191 5342 2 1 96... • Investigation Diagnostic tools before 5.7 18
- 22. • free • top • vmstat • Investigation • Total size of buffers • Number of temporary tables • Number of parallel connections Diagnostic tools before 5.7 18
- 23. mysql> select thread_id tid, user, current_allocated ca, total_allocated -> from sys.memory_by_thread_by_current_bytes; +–––––+–––––––––––––––––––––––––+–––––––––––––+–––––––––––––––––+ | tid | user | ca | total_allocated | +–––––+–––––––––––––––––––––––––+–––––––––––––+–––––––––––––––––+ | 1 | sql/main | 2.53 GiB | 2.69 GiB | | 150 | root@127.0.0.1 | 4.06 MiB | 32.17 MiB | | 146 | sql/slave_sql | 1.31 MiB | 1.44 MiB | | 145 | sql/slave_io | 1.08 MiB | 2.79 MiB | ... | 60 | innodb/io_read_thread | 0 bytes | 384 bytes | | 139 | innodb/srv_purge_thread | -328 bytes | 754.21 KiB | | 69 | innodb/io_write_thread | -1008 bytes | 34.28 KiB | | 68 | innodb/io_write_thread | -1440 bytes | 298.05 KiB | | 74 | innodb/io_write_thread | -1656 bytes | 103.55 KiB | | 4 | innodb/io_log_thread | -2880 bytes | 132.38 KiB | | 72 | innodb/io_write_thread | -7632 bytes | 1.10 MiB | +–––––+–––––––––––––––––––––––––+–––––––––––––+–––––––––––––––––+ 145 rows in set (2.65 sec) Memory diagnostic in 5.7 19
- 24. mysql> select * from sys.memory_by_thread_by_current_bytes -> order by current_allocated descG *************************** 1. row *************************** thread_id: 152 user: lj@127.0.0.1 current_count_used: 325 current_allocated: 36.00 GiB current_avg_alloc: 113.43 MiB current_max_alloc: 36.00 GiB total_allocated: 37.95 GiB ... • Finding connections, using too much memory, now is matter of seconds! Threads Statistics 20
- 25. • memory_summary_by_account_by_event_name • memory_summary_by_host_by_event_name • memory_summary_by_thread_by_event_name • memory_summary_by_user_by_event_name • memory_summary_global_by_event_name • sys schema also includes information about user name RAW Performance Schema tables 21
- 26. • NAME@HOST - regular user • System users • sql/main • innodb/* • ... • Data comes from table THREADS Users in sys.memory_* tables 22
- 27. • Run load ./test2.sh CALL help_task()G CALL help_solve()G CALL task_prepare(); • How much memory uses SysBench load? • To identify how much RAM used by whole server run select * from sys.memory_global_total; Memory usage: practice 23
- 29. mysql> select * from setup_instruments where name like ’statement/sp%’; +––––––––––––––––––––––––––––––––+–––––––––+–––––––+ | NAME | ENABLED | TIMED | +––––––––––––––––––––––––––––––––+–––––––––+–––––––+ ... | statement/sp/stmt | YES | YES | | statement/sp/hreturn | YES | YES | | statement/sp/set | YES | YES | | statement/sp/cpush | YES | YES | | statement/sp/set_trigger_field | YES | YES | | statement/sp/cpop | YES | YES | | statement/sp/jump | YES | YES | | statement/sp/copen | YES | YES | | statement/sp/jump_if_not | YES | YES | | statement/sp/cclose | YES | YES | | statement/sp/freturn | YES | YES | | statement/sp/cfetch | YES | YES | | statement/sp/hpush_jump | YES | YES | | statement/sp/error | YES | YES | | statement/sp/hpop | YES | YES | | statement/sp/set_case_expr | YES | YES | ... +––––––––––––––––––––––––––––-+–––––––––+–––––––+ 16 rows in set (0.00 sec) New instruments 25
- 30. • What happens inside the routine • Queries, called from the routine • statement/sp/stmt Stored routines instrumentation 26
- 31. • We will use this procedure CREATE DEFINER=‘root‘@‘localhost‘ PROCEDURE ‘sp_test‘(val int) BEGIN DECLARE CONTINUE HANDLER FOR 1364, 1048, 1366 BEGIN INSERT IGNORE INTO t1 VALUES(’Some string’); GET STACKED DIAGNOSTICS CONDITION 1 @stacked_state = RETURNED_SQLSTATE; GET STACKED DIAGNOSTICS CONDITION 1 @stacked_msg = MESSAGE_TEXT; END; INSERT INTO t1 VALUES(val); END • When HANDLER called? Stored routines: example 27
- 32. mysql> call sp_test(1); Query OK, 1 row affected (0.07 sec) mysql> select thread_id, event_name, sql_text from events_statements_history -> where event_name like ’statement/sp%’; +–––––––––––+–––––––––––––––––––––––––+––––––––––––––––––––––––––––+ | thread_id | event_name | sql_text | +–––––––––––+–––––––––––––––––––––––––+––––––––––––––––––––––––––––+ | 24 | statement/sp/hpush_jump | NULL | | 24 | statement/sp/stmt | INSERT INTO t1 VALUES(val) | | 24 | statement/sp/hpop | NULL | +–––––––––––+–––––––––––––––––––––––––+––––––––––––––––––––––––––––+ 3 rows in set (0.00 sec) Correct value 28
- 33. mysql> call sp_test(NULL); Query OK, 1 row affected (0.07 sec) mysql> select thread_id, event_name, sql_text from events_statements_history -> where event_name like ’statement/sp%’; +–––––––––––+–––––––––––––––––––––––––+––––––––––––––––––––––––––––––––––––––––––-+ | thread_id | event_name | sql_text | +–––––––––––+–––––––––––––––––––––––––+––––––––––––––––––––––––––––––––––––––––––-+ | 24 | statement/sp/hpush_jump | NULL | | 24 | statement/sp/stmt | INSERT INTO t1 VALUES(val) | | 24 | statement/sp/stmt | INSERT IGNORE INTO t1 VALUES(’Some str... | | 24 | statement/sp/stmt | GET STACKED DIAGNOSTICS CONDITION 1 @s... | | 24 | statement/sp/stmt | GET STACKED DIAGNOSTICS CONDITION 1 @s... | | 24 | statement/sp/hreturn | NULL | | 24 | statement/sp/hpop | NULL | +–––––––––––+–––––––––––––––––––––––––+––––––––––––––––––––––––––––––––––––––––––-+ 7 rows in set (0.00 sec) HANDLER call 29
- 34. • Run load ./crazy_timing.sh CALL help_task()G CALL help_solve()G CALL task_prepare(); • Why procedure takes different time each run? • For nicer output set pager to less: mysql> P less Stored routines: practice 30
- 36. • Contains current prepared statements • Statistics by • Which thread owns the statement • How many times executed • Optimizer statistics, similar to events_statements_* Table prepared_statements_instances 32
- 37. mysql1> prepare stmt from ’select count(*) from employees where hire_date > ?’; Query OK, 0 rows affected (0.00 sec) Statement prepared mysql1> set @hd=’1995-01-01’; Query OK, 0 rows affected (0.00 sec) mysql1> execute stmt using @hd; +––––––––––+ | count(*) | +––––––––––+ | 34004 | +––––––––––+ 1 row in set (1.44 sec) • Try EXECUTE with different variable values Example: prepared statement 33
- 38. mysql2> select statement_name, sql_text, owner_thread_id, count_reprepare, -> count_execute, sum_timer_execute from prepared_statements_instancesG *************************** 1. row *************************** statement_name: stmt sql_text: select count(*) from employees where hire_date > ? owner_thread_id: 22 count_reprepare: 0 count_execute: 3 sum_timer_execute: 4156561368000 1 row in set (0.00 sec) mysql1> drop prepare stmt; Query OK, 0 rows affected (0.00 sec) mysql2> select * from prepared_statements_instancesG Empty set (0.00 sec) Example: diagnosis 34
- 39. • Run load ./prepared.sh CALL help_task()G CALL help_solve()G • How effectively prepared statement is executing? • If not improve situation, without touching the statement • Run commands PREPARE stmt FROM ’SELECT COUNT(*) FROM sbtest1 WHERE pad=?’; SET @p = ’foo; EXECUTE stmt USING @p; - Run few times Prepared statements: practice 35
- 41. • Variables • Status variables • show_compatibility_56 = 0 Variables instrumentation 37
- 42. • Variables • global_variables • session_variables • user_variables_by_thread • variables_by_thread • Status variables Variables instrumentation 37
- 43. • Variables • Status variables • global_status • session_status • status_by_[account|host|thread|user] Variables instrumentation 37
- 44. • Same information which is in • SHOW [GLOBAL] STATUS • I_S.GLOBAL_VARIABLES (removed in 5.7) • I_S.SESSION_VARIABLES (removed in 5.7) • Helps to watch session variables changes Global and session variables 38
- 45. • Same information which is in • SHOW [GLOBAL] STATUS • I_S.GLOBAL_STATUS (removed in 5.7) • I_S.SESSION_STATUS (removed in 5.7) Status variables 39
- 46. mysql> SELECT ss.variable_name, ss.variable_value FROM session_status ss LEFT JOIN global_status gs USING(variable_name) WHERE ss.variable_value != gs.variable_value OR gs.variable_value IS NULL AND ss.variable_value>0; +––––––––––––––––––––––––––––+––––––––––––––––+ | variable_name | variable_value | +––––––––––––––––––––––––––––+––––––––––––––––+ | Bytes_sent | 197774 | | Handler_commit | 0 | | Handler_external_lock | 44 | | Handler_read_first | 3 | | Handler_read_key | 523 | | Handler_read_next | 0 | | Handler_read_rnd_next | 7241 | | Opened_table_definitions | 0 | | Opened_tables | 92 | | Table_open_cache_hits | 21 | | Table_open_cache_misses | 92 | ... Status variables 40
- 47. • variables_by_thread • status_by_ • account • host • thread • user Possible to group 41
- 48. • variables_by_thread mysql> select * from variables_by_thread where variable_name=’tx_isolation’; +–––––––––––+–––––––––––––––+–––––––––––––––––+ | THREAD_ID | VARIABLE_NAME | VARIABLE_VALUE | +–––––––––––+–––––––––––––––+–––––––––––––––––+ | 71 | tx_isolation | REPEATABLE-READ | | 83 | tx_isolation | REPEATABLE-READ | | 84 | tx_isolation | SERIALIZABLE | +–––––––––––+–––––––––––––––+–––––––––––––––––+ 3 rows in set, 3 warnings (0.00 sec) • status_by_ Possible to group 41
- 49. • variables_by_thread • status_by_ mysql> select * from status_by_thread where variable_name=’Handler_write’; +–––––––––––+–––––––––––––––+––––––––––––––––+ | THREAD_ID | VARIABLE_NAME | VARIABLE_VALUE | +–––––––––––+–––––––––––––––+––––––––––––––––+ | 71 | Handler_write | 94 | | 83 | Handler_write | 477 | – Most writes | 84 | Handler_write | 101 | +–––––––––––+–––––––––––––––+––––––––––––––––+ 3 rows in set (0.00 sec) Possible to group 41
- 50. • Grouped by connection • Sometimes can help to find tricky bugs with persistent connections mysql> select * from user_variables_by_thread; +–––––––––––+–––––––––––––––+––––––––––––––––+ | THREAD_ID | VARIABLE_NAME | VARIABLE_VALUE | +–––––––––––+–––––––––––––––+––––––––––––––––+ | 71 | baz | boo | | 84 | foo | bar | +–––––––––––+–––––––––––––––+––––––––––––––––+ 2 rows in set (0.00 sec) User variables 42
- 51. • Run load ./variables.sh CALL help_task()G CALL help_solve()G CALL task_prepare(); • Watch progress of INSERT command, running by stored routine, and find out how many rows were already inserted. • Note what there is parallel load, caused by SysBench. We are not interested in its statistics. Variables: practice 43
- 52. Replication 44
- 53. • Data from SHOW SLAVE STATUS available in replication_* tables • Support of Replication Channels (Multi-threaded slave) • More instruments for GTID Major improvements 45
- 54. • No need to parse SHOW output • Configuration • IO thread • SQL thread SLAVE STATUS 46
- 55. • No need to parse SHOW output • Configuration • replication_connection_configuration • replication_applier_configuration • IO thread • SQL thread SLAVE STATUS 46
- 56. • No need to parse SHOW output • Configuration • IO thread • replication_connection_status • SQL thread SLAVE STATUS 46
- 57. • No need to parse SHOW output • Configuration • IO thread • SQL thread • replication_applier_status • replication_applier_status_by_coordinator • replication_applier_status_by_worker - MTS only SLAVE STATUS 46
- 58. • Configuation mysql> select * from replication_connection_configuration -> join replication_applier_configuration using(channel_name)G *************************** 1. row *************************** CHANNEL_NAME: HOST: 127.0.0.1 PORT: 13000 USER: root NETWORK_INTERFACE: AUTO_POSITION: 1 SSL_ALLOWED: NO SSL_CA_FILE: ... CONNECTION_RETRY_INTERVAL: 60 CONNECTION_RETRY_COUNT: 10 HEARTBEAT_INTERVAL: 60.000 CHANNEL_NAME: DESIRED_DELAY: 0 1 row in set (0.00 sec) SLAVE STATUS 47
- 59. • State of IO Thread mysql> select * from replication_connection_statusG *************************** 1. row *************************** CHANNEL_NAME: GROUP_NAME: SOURCE_UUID: d0753e78-14ec-11e5-b3fb-28b2bd7442fd THREAD_ID: 21 SERVICE_STATE: ON COUNT_RECEIVED_HEARTBEATS: 17 LAST_HEARTBEAT_TIMESTAMP: 2015-06-17 15:49:08 RECEIVED_TRANSACTION_SET: LAST_ERROR_NUMBER: 0 LAST_ERROR_MESSAGE: LAST_ERROR_TIMESTAMP: 0000-00-00 00:00:00 1 row in set (0.00 sec) SLAVE STATUS 48
- 60. • State of SQL Thread mysql> select * from replication_applier_status -> join replication_applier_status_by_coordinator using(channel_name)G *************************** 1. row *************************** CHANNEL_NAME: SERVICE_STATE: ON REMAINING_DELAY: NULL COUNT_TRANSACTIONS_RETRIES: 0 THREAD_ID: 22 SERVICE_STATE: ON LAST_ERROR_NUMBER: 0 LAST_ERROR_MESSAGE: LAST_ERROR_TIMESTAMP: 0000-00-00 00:00:00 1 row in set (0.00 sec) SLAVE STATUS 49
- 61. mysql> select * from replication_applier_status_by_workerG *************************** 1. row *************************** CHANNEL_NAME: WORKER_ID: 1 THREAD_ID: 25 SERVICE_STATE: ON LAST_SEEN_TRANSACTION: LAST_ERROR_NUMBER: 0 LAST_ERROR_MESSAGE: LAST_ERROR_TIMESTAMP: 0000-00-00 00:00:00 *************************** 2. row *************************** CHANNEL_NAME: WORKER_ID: 2 THREAD_ID: 26 SERVICE_STATE: ON LAST_SEEN_TRANSACTION: d0753e78-14ec-11e5-b3fb-28b2bd7442fd:770 LAST_ERROR_NUMBER: 0 LAST_ERROR_MESSAGE: LAST_ERROR_TIMESTAMP: 0000-00-00 00:00:00 Multi-threaded slave diagnostics 50
- 62. • RECEIVED_TRANSACTION_SET in table replication_connection_status • LAST_SEEN_TRANSACTION in table replication_applier_status_by_worker • New instruments • memory • wait • stage GTID diagnostics 51
- 63. • Blog of developers team • Blog of Mark Leith: author of sys schema • Official reference manual More informaiton 52
- 64. ??? Place for your questions 53