Efficient Monitoring & Tuning of Dynamic SQL in DB2 for z/OS by Namik Hrle -IBM Fellow
5 likes3,912 views
The document discusses efficient monitoring and tuning of dynamic SQL in IBM's DB2 for z/OS, with a focus on improvements introduced in DB2 10. It highlights features like caching mechanisms, explain support for dynamic SQL, and various monitoring indicators designed to enhance performance analysis. Additionally, it outlines how to control virtual storage consumption related to dynamic statement storage and adjustments for optimized usage.
Efficient Monitoring & Tuning of Dynamic SQL in DB2 for z/OS by Namik Hrle -IBM Fellow
- 1. #IDUG Efficient Monitoring & Tuning of Dynamic SQL Namik Hrle IBM April 16, 2014 | Platform: DB2 for z/OS
- 2. 2 #IDUG © Copyright IBM Corporation 2012. All rights reserved. U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS” WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. IN ADDITION, THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE. IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, NOR SHALL HAVE THE EFFECT OF, CREATING ANY WARRANTIES OR REPRESENTATIONS FROM IBM (OR ITS SUPPLIERS OR LICENSORS), OR ALTERING THE TERMS AND CONDITIONS OF ANY AGREEMENT OR LICENSE GOVERNING THE USE OF IBM PRODUCTS AND/OR SOFTWARE. IBM, the IBM logo, ibm.com, DB2 and DB2 for z/OS are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml Other company, product, or service names may be trademarks or service marks of others. Disclaimer
- 4. 4 #IDUG program A program B COMMIT EXECUTE S EDM Statement Pool thread A thread B EXECUTE S sqlcode=0 no effect on prepared statements sqlcode=0 EXECUTE S sqlcode=0 EXECUTE S sqlcode=0 2 3 1 4 5 sqlcode=0 PREPARE S prepared statementprepared statement SS SKDSSKDS SS sqlcode=0 full prepare PREPARE S prepared statementprepared statement SS sqlcode=0 short prepare avoided prepare CACHEDYN=YES and KEEPDYNAMIC=YES
- 5. 5 #IDUG Monitoring – Global statements cache – Local statements cache – Specific cached statements Systems tuning Applications tuning Agenda
- 6. 6 #IDUG Prepare requests QXPREP + QXSTIPRP 100005 Not found in cache QXSTNFND 5005 Full prepares QISEDSI 5003 Found in cache QXSTFND 95000 Short prepares QISEDSG - QISEDSI 95000 Global cache hit ratio QISEDSG - QISEDSI QISEDSG 95% Monitoring Global Statements Cache
- 7. 7 #IDUG Implicit prepares QXSTIPRP 20000 Avoided prepares QXSTNPRP 200000 Statements invalidated QXSTDEXP+QXSTDINV 8010 MAXKEEPD exceeded QXSTDEXP 8000 DROP/ALTER/REVOKE/RUNSTATS QXSTDINV 10 Local cache hit ratio QXSTNPRP QXSTNPRP+QXSTIPRP 91% Monitoring Local Statements Cache
- 8. 8 #IDUG EXPLAIN support – STMTCACHE ALL – STMTCACHE STMTID statement-id – STMTCACHE STMTTOKEN statement-token – Special register: CURRENT EXPLAIN MODE (1) DB2 Trace support – IFCID 316 – externalizes execution statistics – IFCID 317 – externalizes full statement text – IFCID 318 – controls type collected statistics (1) – available as of DB2 10 Monitoring Cached Statements
- 9. 9 #IDUG Explains a set of cached statements Populates table DSN_STATEMENT_CACHE_TABLE – Qualified by current SQLID value One row for each cached statement – All statements included if EXPLAIN executed by SYSADM or SQLADM – Otherwise only statements with matching authorization Reports the statement’s execution statistics – Analogous to IFCIDs 316 and 317 – Statistics controlled and reset by means of IFCID 318 – Details documented later in the presentation Member specific – Need to be executed on all members EXPLAIN STMTCACHE ALL
- 10. 10 #IDUG Explains a specific statement identified by its id – Can be either integer constant or host variable – It is assigned by DB2 at prepare time and it uniquely identifies cached statement – It can be retrieved via IFI call from IFCIDs 316 and 124 • also available in IFCIDs 172, 196, 337 – Value reported in PLAN_TABLE’s QUERYNO column EXPLAIN populates following tables: – PLAN_TABLE, DSN_STATEMNT_TABLE, DSN_FUNCTION_TABLE • Identifies actual access path details for specified statement providing it is still in the cache – DSN_STATEMENT_CACHE_TABLE • Reports execution statistics for the specified statement EXPLAIN STMTCACHE STMTID
- 11. 11 #IDUG Explains a specific statement identified by its token – Can be either string constant or host variable • Up to 240 characters – It is assigned by application at prepare time • RRSAF SET_ID function for local applications • sqlseti API for remote applications – Value reported in PLAN_TABLE’s STMTTOKEN column EXPLAIN populates following tables: – PLAN_TABLE, DSN_STATEMNT_TABLE, DSN_FUNCTION_TABLE • Identifies actual access path details for specified statement providing it is still in the cache – DSN_STATEMENT_CACHE_TABLE • Reports execution statistics for the specified statement. – EXPLAIN STMTCACHE STMTTOKEN
- 12. 12 #IDUG Prior to DB2 10, performance monitoring and tuning of dynamic SQL statements was not ideally suited for application programmers – There is no equivalent of EXPLAIN bind option that is provided for static SQL • Each monitored statements need to be changed (adding STMTTOKEN) • Alternatively, time consuming and complex analysis of performance traces or DSN_STATEMENT_CACHE_TABLE is needed – Statement could be purged from the cache before having a chance to retrieve its performance characteristics DB2 10 addresses these challenges – Application programmers can turn on/off collection of access path details and performance data for dynamic SQL statements • SET CURRENT EXPLAIN MODE = NO | YES | EXPLAIN • JCC connection property: currentExplainMode • ODBC/CLI: – keyword DB2Explain in db2cli.ini file, or – Setting SQL_ATTR_DB2EXPLAIN with SQLSetConnectAttr() function – For YES and EXPLAIN, access path details are written in standard explain tables • This also applies to CACHEDYN=NO and REOPT(ALWAYS) cases – For YES, performance data are written in DSN_STATEMENTS_CACHE_TABLE Special Register CURRENT EXPLAIN MODE
- 13. 13 #IDUG PLAN_TABLE DSN_STATEMENT_ CACHE_TABLE QUERYNO STMT_ID • The key correlation value. • DB2 generates the value when the statement got inserted into cache. COLLID COLLID ● One of the constants: ➔ DSNDYNAMICSQLCACHE ➔ DSNEXPLAINMODEYES (1) (2) ➔ DSNEXPLAINMODEEXPLAIN (1) (2) STMT_TOKEN STMT_TOKEN • Optionally present – only if specified by the application BIND_TIME CACHED_TS (2) • The time when the statement got inserted into the cache. TIMESTAMP or EXPLAIN_TIME (1) EXPLAIN_TS (2) • The time when EXPLAIN STMTCACHE was executed • The same statement can have multiple entries in PLAN_TABLE and DSN_STATEMENT_CACHE_TABLE but wih different TIMESTAMP i.e. EXPLAIN_TIME and EXPLAIN_TS values, respectively. • As of DB2 10 TIMESTAMP is deprecated; use EXPLAIN_TIME instead (1) Available as of DB2 10 (2) For DSN_STATEMENT_CACHE_TABLE entries created by EXPLAIN STMTCACHE ALL, use CACHED_TS=BIND_TIME For DSN_STATEMENT_CACHE_TABLE entries created by EXPLAIN MODE=YES|EXPLAIN, use EXPLAIN_TS=BIND_TIME Correlating EXPLAIN Tables
- 14. 14 #IDUG IFCID 316 – Externalises statement-level statistics – Prior to DB2 10, the statistics available for statements that are found in the dynamic statements cache at retrieval time • Retrieved via IFI READS request • Require active Monitor Trace Class 1 – In DB2 10 the statistics is also available for statements that were removed from the dynamic statements cache • These are retrieved via IFI READA request or by post-processing SMF/GTF data • Require active Monitor trace Class 29 • Side remark: IFCID 400 as an analogue to IFCID 316 is introduced for static SQL statements IFCID 317 – Complements IFCID 316 – Provides full statement text for a give statement ID For detailed usage information see DB2 documentation library, e.g. 'Managing Performance' manual in DB2 10 DB2 Trace Support
- 15. 15 #IDUG IFCID 318 – This is a switch that controls what gets externalised in IFCID 316 – Performance Trace user defined Class, e.g. 30 • DB2 10 adds Monitor trace Class 29 – Accounting Class 3 required for suspension timers – Statistics reset at performance class stop – Side remark: IFCID 401 as an analogue to IFCID 318 is introduced for static SQL statements. Included in Monitor trace Class 29 Techniques to limit the number of reported statements – Prior to DB2 10, the result set can be limited to the statements that: • exceeded a given threshold, e.g. number of executions, number of getpages, … • belong to a top list by a given performance indicator, e.g. elapsed time, CPU, time, … • match a given statement ID (single statement retrieval) – As of DB2 10 the result set can be also limited to the statements that: • were executed for a given end-user ID, transaction/application ID, workstation name • refer to a specific table • were prepared after specific time • were re-executed after specific time – DB2 Trace Support
- 16. 16 #IDUG DSN_STATEMENT_CACHE_TABLE IFCID Statement ID (generated by DB2) STMT_ID QW0316ID Statement token (provided by application) STMT_TOKEN QW0316UI Collection ID COLLID Package/DBRM that performed PREPARE PROGRAM_NAME QW0316X1 Precompiler line number for PREPARE statement LINES QW0316LX Transaction name QW0316X2 Primary authorization ID who did initial prepare PRIMAUTH QW0316X3 Current SQLID of user who did initial prepare CURSQLID QW0316X4 CURRENT SCHEMA special register content SCHEMA QW0316SC Data sharing member name (1) GROUP_MEMBER QW0316MBR Table qualifier QW0316QD Table name QW0316TD First 60 characters of statement text QW0316ST Full statement text STMT_TEXT QW0317TX Literal replacement ('&') indication (1) LITERAL_REPL QW0317LR (1) – available as of DB2 10 Statement Level Indicators: Correlation
- 17. 17 #IDUG DSN_STATEMENT_CACHE_TABLE IFCID Timestamp of initial prepare CACHED_TS QW0316TM Timestamp of statistics last update QW0316UT Timestamp of last IFCID 318 activation STAT_TS QW0316TS Status: valid or reason for invalidation (LRU,RUNSTATS,REVOKE,DROP/ALTER) INV_LRU, INV_RUNSTATS INV_REVOKE, INV_DROPALT QW0316FL Value of QUALIFIER bind option BIND_QUALIFIER QW0316X5 Value of ISOLATION bind option BIND_ISO QW0316X6 Value of CURRENTDATA bind option BIND_CDATA QW0316X7 Value of DYNAMICRULES bind option BIND_DYNRL QW0316X8 Value of CURRENT DEGREE special register BIND_DEGRE QW0316X9 Value of CURRENT RULES special register BIND_SQLRL QW0316XA Number of REBINDs due to REOPT(AUTO) BIND_RA_TOT Value of REOPT option (1) BIND_RO_TYPE Value of CURRENT PRECISION special register QW0316XB Cursor WITH HOLD identification BIND_CHOLD QW0316XC (1) – available as of DB2 10 Statement Level Indicators: Status
- 18. 18 #IDUG DSN_STATEMENT_CACHE_TABLE IFCID Locally cached copies currently used (1) USERS QW0316US Locally cached copies – total (1) COPIES QW0316CP Statement executions (1) STAT_EXEC or STAT_EXECB(2) QW0316NE Synchronous reads (1) STAT_SYNR or STAT_SYNRB(2) QW0316NB Synchronous writes (1) STAT_WRIT or STAT_WRITB(2) QW0316NW Getpages (1) STAT_GPAG or STAT_GPAGB(2) QW0316NG Rows examined (1) STAT_EROW or STAT_EROWB(2) QW0316NR Rows processed (1) STAT_PROW or STAT_PROWB(2) QW0316NP Sorts performed (1) STAT_SORT or STAT_SORTB(2) QW0316NS Index scans performed (1) STAT_INDX or STAT_INDXB(2) QW0316NI Tablespace scans performed (1) STAT_RSCN or STAT_RSCNB(2) QW0316NT RID List failures: no storage (1) STAT_RIDSTOR or STAT_RIDSTORB(2) QW0316RS RID List failures: limit exceeded (1) STAT_RIDLIMIT or STAT_RIDLIMITB(2) QW0316RT Parallel groups created (1) STAT_PGRP or STAT_PGRPB(2) QW0316NL (1) – available only if IFCID 318 is activated (2) – as of DB2 10 use columns with suffix 'B' (BIGINT) Statement Level Indicators: Statistics
- 19. 19 #IDUG DSN_STATEMENT_CACHE_TABLE IFCID Elapsed time (1) STAT_ELAP QW0316AE CPU time (1) STAT_CPU QW0316CT Wait time for synchronous I/O (1) (2) STAT_SUS_SYNIO QW0316W1 Wait time for locks (1) (2) (3) STAT_SUS_LOCK QW0316W2 Wait time for synchronous execution unit switch (1) (2) STAT_SUS_SWIT QW0316W3 Wait time for global locks (1) (2) STAT_SUS_GLCK QW0316W4 Wait time for other agents reads (1) (2) STAT_SUS_OTHR QW0316W5 Wait time for other agents writes (1) (2) STAT_SUS_OTHW QW0316W6 Wait time for latches (1) (2) (4) STAT_SUS_LATCH QW0316W7 Wait time for page latches (1) (2) (4) STAT_SUS_PLATCH QW0316W8 Wait time for drain locks (1) (2) (4) STAT_SUS_DRAIN QW0316W9 Wait time for releasing of claims (1) (2) (4) STAT_SUS_CLAIM QW0316WA Wait time for log writes (1) (2) (4) STAT_SUS_LOG QW0316WB (1) – available only if IFCID 318 is activated (2) – available only Accounting Class 3 is activated (3) – priot to DB2 10 the timer includes latches as well (4) – available as of DB2 10 Statement Level Indicators: Timers
- 20. 20 #IDUG DB2 10 adds an indicator whether a statement is dynamic or static to the following trace events: Event IFCID SQL statement end 58 SQL statement text at bind/prepare 63 and 350 Currently executing SQL statement 124 Deadlock details 172 Timeout details 196 Lock escalation details 337 The statement type indicator is also added to the applicable messages. Statement Type Indicators
- 21. 21 #IDUG Monitoring Systems tuning – EDM storage – Thread storage – Controlling virtual storage consumption – IBM DB2 Analytics Accelerator considerations Applications tuning Agenda
- 22. 22 #IDUG The dynamic statements are cached in the EDM Statement Pool which belongs to EDM storage. EDM Statement Pool – Skeleton dynamic statements (SKDS) – zparm EDMSTMTC – As of DB2 V8 above-the-bar EDM DBD Pool – Database Descriptors (DBDs) – zparm EDMDBDC – As of DB2 V8 above-the-bar EDM Skeleton Pool – Skeleton Cursor (SKCT) and Package Tables (SKPT) – zparm EDM_SKELETON_POOL – As of DB2 9 above-the-bar EDM RDS Pool – Cursor (CT) and Package Tables (PT) – zparm EDMPOOL – As of DB2 9 partially above-the-bar – As of DB2 10 entirely above-the-bar DBM1 2GBRDS Pool Skeleton Pool DBD Pool Statement Pool RDS Pool Skeleton Pool DBD Pool Statement Pool V9V9 V10V10 EDM Statement Pool
- 23. 23 #IDUG EDM Statement Pool resides entirely above-the-bar – Therefore, not critical as a virtual storage constraint – Of course, do not over-commit real storage Use zparm EDMSTMTC to specify the EDM Statement Pool size – Can be updated online For applications that rely on dynamic SQL allocate a large pool – E.g. In SAP case, start with 300MB – Adjust upwards in increments of 20MB if the global cache hit ratio keeps consistently below 95% – Beware of the effects of multiple authorization IDs • Result in separate cache entries even for identical statements Keep zparm EDMBFIT at its default: NO – Otherwise you can expect large increase in class 24 latches EDM Statement Pool Considerations
- 24. 24 #IDUG Prior to DB2 10, dynamic statements caching can consume lots of DBM1 virtual storage below the 2GB bar, especially with – Local caching, i.e. KEEPDYNAMIC(YES) • A large portion (up to 50%) of virtual storage that holds locally cached statements (particularly INSERTs) moved above-the-bar in V9 – Long running threads • Even worse with long units of recovery Selected knobs to control thread virtual storage utilization – KEEPDYNAMIC bind parameter – MAXKEEPD zparm – CACHEDYN_FREELOCAL zparm – CONTSTOR zparm – MINSTOR zparm Keep an eye on DB2 service – Informational APARs: II04309 and II10817 DB2 10 brings radical improvements in this space – Runtime structure moved to agent local storage above the bar Thread Virtual Storage
- 25. 25 #IDUG KEEPDYNAMIC(YES) turns on so called local statements caching – Results in reduced number of explicit prepares – Preserves statement text across commits – Automatically closes withheld cursors in DRDA at SQLCODE=+100 • reduces message traffic and locks duration – It is incompatible with REOPT(ALWAYS), but compatible with REOPT(ONCE) and REOPT(AUTO) – DDF server threads that execute KEEPDYNAMIC(YES) packages remain active (CMTSTAT=INACTIVE is ignored) • Active DDF server threads are subject to idle thread timeouts • Although CMTSTAT=INACTIVE is ignored, it’s needed for transaction-level accounting data and changes WLM enclaves at transaction boundaries – Prior to V10 it puts lots of pressure on thread virtual storage Recommendation: – Set KEEPDYNAMIC to NO, i.e. do not use local statements caching unless you fully understand ramifications or the application (e.g. SAP) requires it KEEPDYNAMIC Considerations
- 26. 26 #IDUG MAXKEEPD limits number of cached executables It enables prepare avoidance at cost of virtual storage It is checked at system level, but enforced at thread level – FIFO method regarding prepare operation Online zparm – range 0 - 65535, default 5000 – DB2 10 opens opportunity to be more relaxed with this parameter, subject to appropriate availability of real storage MAXKEEPD = 0 disables keeping statement executables across commits – Statements text is still preserved – Reasons for setting MAXKEEPD to 0 and KEEPDYNAMIC to YES: • Reduced number of explicit prepares • Cutting accounting records at commit • WLM flexibility • Reduced CPU utilization associated with package allocation MAXKEEPD Considerations
- 27. 27 #IDUG CONTSTOR – Default is NO in V8 and V9, but changed to YES in V10 – Updateable online – If set to YES, DB2 releases unused thread storage at commit instead of deallocation • The check is done after certain number of commits – Recommendation: CONTSTOR = YES MINSTOR – Default keeps changing • V8: NO • V9: YES • V10: NO – Updateable online – If set to YES, DB2 regularly reduces thread’s virtual storage footprint – Recommendation: • V8, V9: MINSTOR=YES • V10: MINSTOR=NO. CONTSTOR and MINSTOR zparms
- 28. 28 #IDUG Introduced in DB2 V8, APAR PK21861 – New serviceability functionality added by PM24696 Updateable online In case of below-the-bar virtual storage stress conditions, DB2 frees cached statement and releases associated virtual storage upon statement’s completion, i.e. does not wait for a commit to do that. If the statement is re-referenced after being freed, DB2 performs implicit prepare, so the process is transparent to the applications – Unless there are concurrent DROPs or ALTERs affecting associated objects, in which case -204 could be returned to the application executing the freed statement Applies only to statements bound with KEEPDYNAMIC(YES) Possible values – 0 : Feature is disabled. Default value in DB2 V8. – 1 : Feature is enabled. Default value in DB2 9 and 10 – 2, 3, etc are serviceability values CACHEDYN_FREELOCAL zparm
- 29. 29 #IDUG IBM DB2 Analytics Accelerator Considerations ● The queries that qualify for routing to the accelerator are not stored in the Dynamic Statement Cache ➔ Every prepare is a full prepare ● Not a concern because of the relative cost of prepare vs. query execution ➔ IBM DB2 Analytics Accelerator Studio monitoring capabilities compensate for the lack of statement-level execution statistics provided for the cached statements ● Queries that do not qualify for routing to the accelerator are stored in the Dynamic Statement Cache ➔ There is no practically measurable overhead for short prepares, i.e. the queries that have already been stored in the cache ➔ The cache is searched prior to the extensive checking whether the query qualifies for routing to the accelerator ➔ There are two cases when the cached entries need to be explicitly invalidated in order to give the optimizer chance to consider the accelerator DSN_PROFILE_ATRIBUTES table is updated to set new values for the accelerator relevant heuristics parameters (SMALLTABLE_THRESHOLD, CHECK_RESULTSET, RESULTSET_THRESHOLD) A new table has been defined, loaded and enabled for acceleration ➔ In both cases, the most efficient way to invalidate the relevant cached entries is to run RUNSTATS REPORT NO UPDATE NONE for the corresponding tablespaces
- 30. 30 #IDUG IBM DB2 Analytics Accelerator Considerations ● There are only few cases where the cost of short prepares is slightly higher than without IDAA ➔ Table not enabled (but defined or loaded) ➔ Table does not reside in the same accelerator as the rest of the tables referenced by the query ➔ A column has been added to the DB2 table after it has been defined and loaded to IDAA • For each of these you can have simple, self-explanatory tuning actions • The full prepares that are somewhat more expensive, but only if there is at least one IDAA started and the special register CURRENT ACCELERATION is set to ENABLE or ENABLE with FAILBACK ➔ Assuming that your system is connected to a started IDAA, make sure to set the special register to NONE (which is the default) if: ♦ There are no queries that should be considered for acceleration ♦ There are lots of full prepares for queries that do not qualify for acceleration. In this case set the special register explicitly to ENABLE or ENABLE with FAILBACK for the queries you want to accelerate and set it back to NONE for other queries.
- 31. 31 #IDUG Monitoring Systems tuning Applications tuning – Parameter markers considerations – Reoptimization options Agenda
- 32. 32 #IDUG Use dynamic SQL statements – SELECT, UPDATE, INSERT, DELETE – local and distributed Understand statements matching criteria – Statements must be identical character-for-character • including imbedded blanks – as of DB2 10 the trailing blanks are removed from the matching criteria • SELECT list considerations – Prior to DB2 10 caching was of limited value unless using parameter markers • Major enhancement in DB2 10 with CONCENTRATE STATEMENTS option – Authorization IDs must be identical – Selected bind options such as ISOLATION and CURRENTDATA must be identical – Selected special registers such as CURRENT DEGREE must be identical If using local caching do not explicitly re-prepare statements after commits – Still provide for possible sqlcode -514 or -518 No caching for REOPT(ALWAYS) Writing Cache Friendly Applications
- 33. 33 #IDUG Use of literals typically results in dynamic statements cache inefficiency – A short (cost effective) prepare is possible only if the new statement matches a cached statement character-for-character – New copies of statements that could have shared already cached statement 'thrash' dynamic statements cache – Usage of literals cannot be always avoided, e.g. for some CLI applications, Ruby on Rails, PHP DB2 10 introduces option to reuse a cached, previously prepared statement irrespective of literal values – CONCENTRATE STATEMENTS WITH LITERALS • New option for ATTRIBUTES on PREPARE • New JCC driver's datasource/connection property enableLiteralReplacement='Yes' • New keyword LITERALREPLACEMENT in the ODBC initialization file • CONCENTRATE STATEMENTS OFF (default) causes pre-V10 behaviour – Before a prepared statement is cached each literal is replaced by a single ‘&’ • Additionally, DB2 removes the trailing blacks that follow the statement • ‘&’s are shown instead of literals in instrumentation (e.g. IFCID 317) – Subsequent prepares of the same statement with different literals can result in short prepares • The exact match has a precedence over matching with ‘&’ • The literals must be ‘reusable’ in the prepare context • Mixture of parameter markers ‘?’ and literals results in the pre-V10 behaviour – Monitoring support in DSN_STATEMENT_CACHE_TABLE (column LITERAL_REPL) and IFCID 316: values ‘R’, ‘D’ or ‘ ‘ Concentrating Cached Statements in DB2 10
- 34. 34 #IDUG Dynamic statements cache efficiency is reduced by using different subsets of columns in otherwise identical INSERT, UPDATE and MERGE statements – Prior to V10 the applications had to prepare separate statements for each combination of columns to be inserted, updated or merged – typical case for data entry/update applications – The reason is a lack of mechanism to indicate that no value is provided for the target column DB2 10 introduces new extended indicator variables – The extended indicator variable is activated via: • PREPARE attributes WITH / WITHOUT EXTENDED INDICATORS • BIND parameter EXTENDEDINDICATOR = YES / NO – A value of -5 means that target column is to be set to its default value – A value of -7 means that target column is to be treated as if it had not been specified – Example: stmt1 = 'UPDATE T1 SET C1=?' PREPARE S1 FROM stmt1 stmt2 = 'UPDATE T1 SET C2=?' PREPARE S2 FROM stmt2 EXECUTE S1 USING :inputvar1 EXECUTE S2 USING :inputvar2 stmt = 'UPDATE T1 SET C1=?, C2=?' PREPARE S1 ATTRIBUTES WITH EXTENDED INDICATORS FROM stmt IND1 = -7 IND2 = -7 EXECUTE S1 USING :inputvar1, :IND2 EXECUTE S1 USING :IND1, :inputvar2 prior to V10 in V10 All-Columns INSERT, UPDATE, MERGE in DB2 10
- 35. 35 #IDUG When parameter markers are used for dynamic SQL, the DB2 Optimizer cannot take advantage of columns values distribution. A simple example is: For a low card(C1) this most often results in tablespace scan which is very bad if the cursor is opened with a non-existing value of C1. The same negative effect exist whenever there is a skew in column values distribution. REOPT(ALWAYS) is often too strong medicine with significant downsides in performance, concurrency and monitoring. PREPARE ... SELECT ... FROM T WHERE C1 = ? Parameter marker Optimizer Challanges
- 36. 36 #IDUG Bind option REOPT(ONCE) results in following: – Delaying optimization until the first execute • At that time the parameter marker values are known • DB2 optimizer can use the values and take full advantage of their frequency distribution – Fixing the access plan into the statements cache • Consequently take full advantage of caching – Statement-level hints (given via Access Path Repository) allow different REOPT settings for different statements within the same package In effect only for dynamic SQL and if global caching is activated Use the feature if the first set of values is more representative than the application agnostic set of default values which must be assumed by DB2 otherwise Traditional EXPLAIN is not suitable – See the Monitoring section in this presentation REOPT (ONCE)
- 37. 37 #IDUG REOPT(AUTO) addresses REOPT(ONCE) exposure which occurs when: – The first set of parameter marker values is not representative regarding DB2 optimization Bind option REOPT(AUTO) instructs DB2 to automatically determine if a new access path is needed to further optimize performance. In effect only for dynamic SQL and if global caching is activated Use the feature if the parameter marker values vary across statement executions so much that occasional reprepare is needed, but still REOPT(ALWAYS) would be an overkill. Traditional EXPLAIN is not suitable – See the Monitoring section in this presentation DSN_STATEMENT_CACHE_TABLE includes two new columns to support REOPT(AUTO): – BIND_RO_TYPE – The current specification of REOPT for the statement: • N – REOPT(NONE) • 1 – REOPT(ONCE) • A – REOPT(AUTO) • 0 – The current plan is deemed optimal and there is no need for REOPT(AUTO) – BIND_RA_TOT – Total number of REBINDs executed due to REOPT(AUTO) REOPT (AUTO)
- 38. 38 #IDUG DB2 for z/OS Product Documentation – DB2 10 for z/OS: Managing Performance, SC19-2978 – DB2 9 for z/OS: Performance Monitoring and Tuning Guide, SC18-9851 DB2 for z/OS and OS/390 : Squeezing the Most Out of Dynamic SQL - Redbook SG24-6418 DB2 for z/OS: Considerations on Small and Large Packages - Redpaper REDP-4424 Dynamic Statements Cache: Getting the Most Out of It by Namik Hrle - IDUG Solutions Journal Volume 8 Number 2 Memory Management for Dynamic SQL by Thomas Baumann - IDUG NA 2008 References
- 39. #IDUG Please fill out your session evaluation before leaving! Namik Hrle IBM hrle@de.ibm.com Title: Efficient Monitoring & Tuning of Dynamic SQL