SQL Optimization With Trace Data And Dbms Xplan V6

SQL Optimization (Tuning) with 10046 Trace Data and DBMS_XPLAN Hotsos Enterprises, Ltd. Grapevine, Texas Oracle. Performance. Now. [email_address]

Agenda Goal of SQL tuning or optimization Approach for SQL tuning Historical An alternative approach Concepts 10046 Trace Data and STAT lines in Trace Data STATISTICS_LEVEL DBMS_XPLAN Method R Method R Applied to STAT lines in conjunction with DBMS_XPLAN Caveats Why not use LIO instead of response time LIO versus PIO Examples Next steps Q&A

About the Speaker Mahesh Vallampati Career A Senior Sales Consulting Manager at Hotsos for over a year Prior to that, a Director of IT DBA Services at Eagle Global Logistics for 2 years Prior to that, at Oracle for more than 9 years in Consulting, Sales and Sales Consulting Education Master’s in Electrical Engineering from Texas A&M University Other Presented Papers at OOW and EOUG and several user groups Published articles in Oracle Magazine

Goals of SQL Tuning or Optimization

Goal of SQL Tuning or Optimization SQL Optimization Goals Improve or Sustain the response time for a SQL statement Consume the least amount of system and database resources for that SQL statement which is usually Logical I/O’s (LIO’s) Meet or exceed the response time expectation of the user using the SQL statement Know where most of time was spent in the processing of the SQL statement Evaluate alternate options to reduce the time

A Historical Perspective SQL Tuning usually has involved the following steps Trial and Error Make changes until you hit upon the solution; usually indexes or statistics Guessing Guess which part of the SQL statement is slow “Look at the Data Model” Study the data model Use Materialized Views or Temporary Tables or Rebuild Tables and Indexes Pre-build or Rebuild the tables or views and query against them Try to rewrite the SQL Make efforts to re-write the SQL

An Alternative Approach: An Actual Method for SQL Optimization What exactly happens when Oracle executes SQL? The kernel decomposes the SQL into row source operations SQL optimization method consists of four steps Which row source operation of the SQL Statement is taking too long? What part of the SQL Statement is driving it? Why is it taking a long time? What needs to be done to address it? This paper focuses on the first two steps The third and fourth steps are usually easy to figure out once the first two are known

Benefits of the Method The Oracle Database has instrumentation and diagnostic capabilities built in that provide this information The method is learnable and repeatable

10046 Trace Data and STAT Lines

Event 10046 instructs the Oracle kernel to emit a sequential record of what it does with your time. Event 10046: enable SQL statement timing More accurate: “enable database and system call timing” Sequential record logged to a trace file Database calls Oracle “timed events,” if you ask for them (waits => true) Placeholder-value bindings, if you ask for them (binds => true) Execution plans for SQL statements (called STAT lines)

STAT Lines STAT lines in the 10046 trace file reveals the execution plan that was chosen by the Oracle query optimizer STAT lines are individual row source operations that make up the execution plan STAT line examples STAT #7 id=9 cnt=3 pid=4 pos=2 obj=19 op='TABLE ACCESS BY INDEX ROWID IND$ (cr=8 pr=1 pw=0 time=9528 us)' STAT #7 id=5 cnt=8 pid=4 pos=1 obj=0 op='NESTED LOOPS (cr=696 pr=0 pw=0 time=13664 us)'

STAT Lines and STATISTICS_LEVEL Using the timing statistics from the STAT lines in trace files is straightforward. Time values are available even if STATISTICS_LEVEL is not set to “all” but are not accurate Prerequisites STATISTICS_LEVEL=all Will set "_rowsource_execution_statistics"=true Or the time values don’t roll up properly

STATISTICS_LEVEL Introduced in 9i R2 Controls all major statistics collections and advisories in the database Database parameter: instance-wide or session level Valid values for STATISTICS_LEVEL BASIC TYPICAL ALL

STATISTICS_LEVEL V$STATISTICS_LEVEL (Described in this view) STATISTICS_LEVEL should not be set to “all” at the instance level This will cause some stability issues Instead, set it at the session level for the SQL statement you are trying to optimize

STATISTICS_LEVEL BASIC No advisories or statistics are collected. TYPICAL Buffer cache advisory MTTR advisory Shared pool sizing advisory Segment level statistics PGA target advisory Timed statistics and more in 10g ALL All of TYPICAL, plus the following Timed operating system statistics Row source execution statistics

Script to Gather Row Source Execution Statistics alter session set timed_statistics = true; alter session set max_dump_file_size = unlimited; alter session set tracefile_identifier = ‘Query1_10046’; alter session set statistics_level=all; alter session set events '10046 trace name context forever, level 12'; or dbms_support in 9i or dbms_monitor in 10g Plug SQL statement that needs to be tuned here select object_type, count(*) from dba_objects where owner=‘SCOTT‘ group by object_type; Disconnect --(instead of disabling trace) exit; Instead of disabling trace, just disconnect Turning off trace before cursor close withholds the cursor’s execution plan information

Real Life STAT Lines STAT #6 id=1 cnt=12 pid=0 pos=1 obj=0 op='HASH GROUP BY (cr=1049 pr=10 pw=0 time=141768 us)' STAT #6 id=2 cnt=435 pid=1 pos=1 obj=2371 op='VIEW DBA_OBJECTS (cr=1049 pr=10 pw=0 time=139205 us)' STAT #6 id=3 cnt=435 pid=2 pos=1 obj=0 op='UNION-ALL (cr=1049 pr=10 pw=0 time=136579 us)' STAT #6 id=4 cnt=435 pid=3 pos=1 obj=0 op='FILTER (cr=1038 pr=7 pw=0 time=83601 us)' STAT #6 id=5 cnt=465 pid=4 pos=1 obj=0 op='NESTED LOOPS (cr=706 pr=6 pw=0 time=45592 us)' STAT #6 id=6 cnt=1 pid=5 pos=1 obj=22 op='TABLE ACCESS BY INDEX ROWID USER$ (cr=2 pr=0 pw=0 time=48 us)' STAT #6 id=7 cnt=1 pid=6 pos=1 obj=44 op='INDEX UNIQUE SCAN I_USER1 (cr=1 pr=0 pw=0 time=23 us)' STAT #6 id=8 cnt=465 pid=5 pos=2 obj=18 op='TABLE ACCESS FULL OBJ$ (cr=704 pr=6 pw=0 time=42730 us)' STAT #6 id=9 cnt=159 pid=4 pos=2 obj=19 op='TABLE ACCESS BY INDEX ROWID IND$ (cr=332 pr=1 pw=0 time=33601 us)' STAT #6 id=10 cnt=165 pid=9 pos=1 obj=39 op='INDEX UNIQUE SCAN I_IND1 (cr=167 pr=1 pw=0 time=29671 us)' STAT #6 id=11 cnt=0 pid=3 pos=2 obj=0 op='NESTED LOOPS (cr=3 pr=1 pw=0 time=12987 us)' STAT #6 id=12 cnt=1 pid=11 pos=1 obj=22 op='TABLE ACCESS BY INDEX ROWID USER$ (cr=2 pr=0 pw=0 time=50 us)' STAT #6 id=13 cnt=1 pid=12 pos=1 obj=44 op='INDEX UNIQUE SCAN I_USER1 (cr=1 pr=0 pw=0 time=20 us)' STAT #6 id=14 cnt=0 pid=11 pos=2 obj=107 op='INDEX RANGE SCAN I_LINK1 (cr=1 pr=1 pw=0 time=12917 us)'

STAT Lines Description STAT #6 id=9 cnt=159 pid=4 pos=2 obj=19 op='TABLE ACCESS BY INDEX ROWID IND$ (cr=332 pr=1 pw=0 time=33601 us)' Cursor Number The unique id of the row source operation within the STAT line set Number of rows returned by this row source operation Parent “id” for this row source operation Position of child within parent object_id of the row source operation (object_id from dba_objects). Set to 0 for non-object row source operations like nested loops etc. Name of the row source operation Consistent Reads Physical Reads (r in 9i) Physical Writes (w in 9i) Elapsed duration in microseconds including descendants

11g STAT Lines Enhancement Following Added to the STAT lines in 11g Cost  Cost Column in V$SQL_PLAN Card  Cardinality Column in V$SQL_PLAN Size  Bytes column in v$SQL_PLAN

Hotsos way of Looking at STAT lines “Profile of Execution Plan” Blue Rectangle – Cumulative Duration (time) in STAT lines Green Rectangle – Duration of Individual STAT lines calculated by profiler software op (obj) (cnt) (cr) (pw) (pr) (time) (calculated)

Recap 10046 trace files contain STAT Lines STAT lines contain execution plans for SQL statements STAT lines are nothing but row source operations of real execution plans STAT lines contain cumulative duration of row source operations including descendants For these durations to roll up properly, STATISTICS_LEVEL should be set to ALL Disabling trace or exiting from a session may suppress STAT lines, instead just disconnect Timing of individual row source operations should be carefully calculated to avoid over or undercounting The Hotsos Profiler displays the STAT lines in an easy to read and understand format and calculates the individual durations of row source operations

DBMS_XPLAN Introduced in 9iR2 Easy way of viewing the output of the EXPLAIN PLAN command in several, predefined formats 10g has some enhanced functionality We will use the DISPLAY function of DBMS_XPLAN available in 9iR2 and 10g

DBMS_XPLAN Ensure that you are using the right version of the PLAN_TABLE DBMS_XPLAN Version incompatibilities may cause some issues Where is plan table and DBMS_XPLAN? Plan Table - $ORACLE_HOME/rdbms/admin/utlxplan.sql DBMS_XPLAN - $ORACLE_HOME/rdbms/admin/dbmsxpln.sql

Using DBMS_XPLAN EXPLAIN PLAN SET STATEMENT_ID = 'abc' FOR select object_type, count(1) from dba_objects where owner= 'SCOTT' group by object_type;

DBMS_XPLAN.DISPLAY Options Format parameter for this function choices are BASIC – Just displays the minimum information in the plan TYPICAL – Displays the relevant information in the plan and predicate information (PX information if applicable) SERIAL – Like typical, but no parallel execution information even if applicable ALL – All of typical including projections, alias, etc.

DBMS_XPLAN. DISPLAY Examples SELECT * FROM TABLE(dbms_xplan.display('PLAN_TABLE','abc','BASIC')); SELECT * FROM TABLE(dbms_xplan.display('PLAN_TABLE','abc','TYPICAL')); SELECT * FROM TABLE(dbms_xplan.display('PLAN_TABLE','abc','ALL')); Note this is different from the STATISTICS_LEVEL parameter These are display formats for DBMS_XPLAN

Recap DBMS_XPLAN has been around since 9.2 It is an elegant way of looking at EXPLAIN PLAN outputs The “TYPICAL” option of DBMS_XPLAN.DISPLAY is most useful because it ties predicate information (access and filters) back to the predicted row source operation lines in the explained plan

Method R It’s simply the way human’s optimize Identify the important task. Measure its response time (R). In detail. Optimize R in the most economically efficient way. Repeat until system is economically optimal. Page 20 –Optimizing Oracle Performance – Cary Millsap and Jeff Holt, O’Reilly Books

Method R Tells you where to start Tells you when to stop So you can avoid “Compulsive Tuning Disorder”

Method R Applied to STAT lines and EXPLAIN PLAN

EXPLAIN PLAN and STAT Lines (Execution Plan) Side by Side

First Question Do the execution plan row source operation steps in the STAT lines match the EXPLAIN PLAN row source operation steps? If not, why? Statistics issue? Skewed data? Session optimizer settings? Bind variable type mismatch? Schema change? Optimizer bug?

Apply Method R to this Profiler View of STAT Lines Which row source operations are consuming most of the response time? Which access predicate is driving it? Why? Is this row source operation the most efficient way to do the job? e.g., NESTED LOOP versus HASH JOIN or vice versa Why are so many blocks being scanned to get a small subset of rows? If the index is being used and if it is doing a range scan over a large set of blocks, is that the right index to be used? Can an alternate index help or are we better off doing a full table scan? Is there a big difference in the actual rows in the execution plan versus the expected rows from the explain plan? What are the deepest and slowest row source operations?

Method R view of STAT lines and EXPLAIN PLAN Notice wide difference in row count estimated by the plan and the actual

Recap Tying back STAT lines in trace files ( actual execution plan) to the EXPLAIN PLAN output ( predicted execution plan) using DBMS_XPLAN should be the first step in SQL optimization Applying Method R to this view helps us find out which row source operations are driving most of the response time and the corresponding predicates driving the response time can be obtained from DBMS_XPLAN Large discrepancies in the actual rows returned in the execution plan and the estimated rows in the execution plan will point to what the problem is

Why not use LIO’s instead of timing of the row source operation?

LIO’s as a driving approach Some performance experts have recommended using the LIO count in the trace file as a framework for driving optimization activities While this is not a bad strategy, it is not reliable We have seen examples where high LIO counts in Row Source Operations were not the most expensive row source operations from a timing perspective Hash joins are an example where there are no LIO’s for the Hash Operation of two child row source operations but can be an expensive row source operations

A note of caution: LIO Versus PIO

A Note of Caution (LIO Versus PIO) In the lines that are highlighted, what % of the time was due to physical I/O and not logical I/Os?

LIO Versus PIO White paper by Cary Millsap “ Why you should focus on LIOs instead of PIOs ” Available on hotsos.com Library Cary’s white paper clearly documents why Focus on response time keeping logical I/Os in mind Even after eliminating physical I/Os there is still an opportunity to make the SQL statement run faster A focus on LIO reduction automatically drives PIO reduction http://www.hotsos.com/e-library/abstract.php?id =7

LIO Versus PIO SQL statements consume LIO which drives PIO consumption PIO is a secondary, derived effect When the query is run the first time, depending on the data cached in the database, the PIO may dominate the components of the response time When you run the query the second time, the data should be cached and you will see a better picture of the inefficient row source operations triggered by LIO’s.

LIO Versus PIO To get the right profile of the execution plan from a LIO perspective, we need to do the following. Execute the query once This will parse the query The buffer pool will cache the data Execute the query again This time the STAT lines show the effect of LIO on the SQL execution response time

Real World Example – SQL Statement SELECT iu.ship_product_id AS part_number, iu.merge_id AS fc_code, oa.country_code, COUNT (1) quantity FROM shipping.shippable sh, shipping.order_address oa, shipping.item_unit iu WHERE sh.manifest_date > TRUNC (SYSTIMESTAMP AT TIME ZONE 'GMT')- (7 * 13) AND oa.order_id = sh.order_id AND oa.addr_type_code = 'S' AND iu.order_id = oa.order_id AND iu.merge_id IS NOT NULL GROUP BY iu.ship_product_id, iu.merge_id, oa.country_code

Profiler View of Execution Plan Line 2 and Line 8 are the expensive row source operations Notice the calculation of the individual duration of the row source operations

SQL Statement – Access Predicates Highlighted SELECT iu.ship_product_id AS part_number, iu.merge_id AS fc_code, oa.country_code, COUNT (1) quantity FROM shipping.shippable sh, shipping.order_address oa, shipping.item_unit iu WHERE sh.manifest_date > TRUNC (SYSTIMESTAMP AT TIME ZONE 'GMT')- (7 * 13) AND oa.order_id = sh.order_id AND oa.addr_type_code = 'S' AND iu.order_id = oa.order_id AND iu.merge_id IS NOT NULL GROUP BY iu.ship_product_id, iu.merge_id, oa.country_code

Recap Run the query once to parse the query and warm up the buffer pool with the data blocks Trace it the second time with STATISTICS_LEVEL=ALL STAT lines in trace files contain timing information of row source operations. These timings includes descendants of the row source operations Calculate the individual row source operations timings using the Profiler software Obtain the EXPLAIN PLAN for the SQL statement using DBMS_XPLAN Put them side-by-side and see which access predicates are driving the row source operations that are taking the most time

Next Steps If you have a SQL statement that you think can be made to run faster Upload a zip file containing the following 10046 trace file for SQL statement Output of DBMS_XPLAN To the Profiler Portal at https://secure.hotsos.com/secure/profiler/fman.php Send an e-mail to [email_address] We will schedule a time to review it with you Education We recommend the Optimizing Oracle SQL Intensive Class https://portal.hotsos.com/education/OPINT

Q & Q U E S T I O N S A N S W E R S

SQL Optimization With Trace Data And Dbms Xplan V6

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to SQL Optimization With Trace Data And Dbms Xplan V6 (20)

More from Mahesh Vallampati (20)

Recently uploaded (20)

SQL Optimization With Trace Data And Dbms Xplan V6

Editor's Notes