The Cost Based Optimiser in 11gR2

www.sagecomputing.com.au
penny@sagecomputing.com.au
the Cost Based Optimiser in
11g Rel 2
Penny Cookson
SAGE Computing Services
Customised Oracle Training Workshops and Consulting

Penny Cookson
Managing Director and Principal Consultant
Working with Oracle products since 1987
Oracle Magazine Educator of the Year 2004

Agenda
Extended Statistics
Kernel trace granularity - oradebug
Cardinality Feedback
Adaptive Cursors
New behaviours
Hints in 11g
Getting rid of your hints
Assessing plan changes on upgrade
Results cache

Multi Column Statistics
Gather statistics on combinations of columns
Improves plans for complex dependency patterns
(and this is persistent)
SELECT count(b.comments)
FROM train.events_large e, train.bookings_large b
WHERE e.org_id = 2264
AND e.event_no = b.event_no
AND e.comments = 'TEST'

Multi Column Data Patterns
SELECT count(*)
FROM train.events_large e
SELECT count(*)
SELECT count(e.comments)
WHERE e.comments = 'TEST'
SELECT count(e.comments)
WHERE e.comments = 'TEST‘
AND e.org_id = 2264

With Individual Statistics
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 2 5.00 22.52 35341 36337 0 1
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 4 5.00 22.52 35341 36337 0 1
Misses in library cache during parse: 1
Optimizer mode: ALL_ROWS
Parsing user id: 82
Rows Row Source Operation
------- ---------------------------------------------------
1 SORT AGGREGATE (cr=36337 pr=35341 pw=35341 time=0 us)
175 HASH JOIN (cr=36337 pr=35341 pw=35341 time=13 us cost=9966 size=33365083 card=1076293)
4 TABLE ACCESS FULL EVENTS_LARGE (cr=991 pr=0 pw=0 time=3 us cost=274 size=319770 card=18810)
5767168 TABLE ACCESS FULL BOOKINGS_LARGE (cr=35346 pr=35341 pw=35341 time=225583 us cost=9665
size=80740352 card=5767168)
Elapsed times include waiting on following events:
Event waited on Times Max. Wait Total Waited
---------------------------------------- Waited ---------- ------------
SQL*Net message to client 2 0.00 0.00
direct path read 314 0.38 15.88
resmgr:cpu quantum 11 0.14 0.27
SQL*Net message from client 2 0.04 0.04
********************************************************************************

Auto Created Column Statistics
BEGIN
dbms_stats.gather_table_stats(ownname=>'TRAI1N',
tabname=>'EVENTS_LARGE', cascade=>TRUE,
method_opt => 'FOR ALL COLUMNS SIZE AUTO',
no_invalidate=>FALSE);
END;
SELECT column_name, histogram, num_buckets
FROM user_tab_col_statistics
WHERE table_name = 'EVENTS_LARGE'

Auto Created Column Statistics
SELECT extension_name, extension
FROM user_stat_extensions
WHERE table_name='EVENTS_LARGE‘
So same access path used
********************************************************************************
------- ---------------------------------------------------
5767168 TABLE ACCESS FULL BOOKINGS_LARGE (cr=35346 pr=35341 pw=35341 time=225583 us cost=9665
size=80740352 card=5767168)

Manually Create Multi Column Statistics
BEGIN
dbms_stats.gather_table_stats(ownname=>'TRAIN',
tabname=>'EVENTS_LARGE', cascade=>TRUE,
method_opt => 'FOR COLUMNS (org_id, comments) SIZE
SKEWONLY', no_invalidate=>FALSE);
END;
OR
dbms_stats.create_extended_stats('TRAIN1','EVENTS_LARGE','(OR
G_ID,COMMENTS)'); --(Then gather stats)
SELECT extension_name, extension
FROM user_stat_extensions
WHERE table_name='EVENTS_LARGE'

Manually Create Multi Column Statistics
SELECT column_name, histogram, num_buckets
FROM user_tab_col_statistics
WHERE table_name = 'EVENTS_LARGE'

********************************************************************************
FROM train.events_large e, train.bookings_large b
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 2 0.01 0.01 0 1175 0 1
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 4 0.01 0.01 0 1175 0 1
------- ---------------------------------------------------
175 NESTED LOOPS (cr=1175 pr=0 pw=0 time=71 us)
175 NESTED LOOPS (cr=1001 pr=0 pw=0 time=39 us cost=1129 size=26815 card=865)
175 INDEX RANGE SCAN BK_EVT2 (cr=10 pr=0 pw=0 time=10 us cost=2 size=0 card=57)(object id 69868)
175 TABLE ACCESS BY INDEX ROWID BOOKINGS_LARGE (cr=174 pr=0 pw=0 time=0 us cost=57 size=798 card=57)
---------------------------------------- Waited ---------- ------------
********************************************************************************

********************************************************************************
FROM train.organisations o, train.events_large e, train.bookings_large b,
train.resources r
WHERE o.org_id = e.org_id
AND b.resource_code = r.code
AND o.name = 'Australian Medical Systems'
AND r.description = 'Buffet Lunch'
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 2 4.64 21.66 35341 36351 0 1
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 4 4.64 21.66 35341 36351 0 1
Parsing user id: 82

********************************************************************************
------- ---------------------------------------------------
25 HASH JOIN (cr=36351 pr=35341 pw=35341 time=1966789 us cost=9972 size=6555490
card=79945)
1 MERGE JOIN CARTESIAN (cr=14 pr=0 pw=0 time=0 us cost=6 size=46 card=1)
1 TABLE ACCESS FULL ORGANISATIONS (cr=7 pr=0 pw=0 time=0 us cost=3 size=26
card=1)
1 BUFFER SORT (cr=7 pr=0 pw=0 time=0 us cost=3 size=20 card=1)
1 TABLE ACCESS FULL RESOURCES (cr=7 pr=0 pw=0 time=0 us cost=3 size=20
card=1)
75243 TABLE ACCESS FULL EVENTS_LARGE (cr=991 pr=0 pw=0 time=1020 us cost=274
size=1282616 card=75448)
5767168 TABLE ACCESS FULL BOOKINGS_LARGE (cr=35346 pr=35341 pw=35341 time=350849 us
cost=9665 size=109576192 card=5767168)
---------------------------------------- Waited ---------- ------------
********************************************************************************

Multi Column Statistics - Summary
Manually create column groups and statistics on them
Complex joins may still need SQL Profiles
Use for dependent column such as Country, State
Use DBMS_STATS.SEED_COL_USAGE to gather column
usage stats
Use DBMS_STATS.REPORT_COL_USAGE to report on col
usage stats

Gathering Statistics - Preferences

Setting Statistics Gathering Defaults
Obsolete:
GET_PARAM
SET_PARAM Procedure
Use:
GET_PREFS Function
SET_TABLE_PREFS (set for individual tables)
SET_GLOBAL_PREFS – for new objects

Table Prefs
PROCEDURE SET_TABLE_PREFS
Argument Name Type In/Out Default?
------------------------------ ----------------------- ------ --------
OWNNAME VARCHAR2 IN
TABNAME VARCHAR2 IN
PNAME VARCHAR2 IN
PVALUE VARCHAR2 IN
PNAME
CASCADE
DEGREE
ESTIMATE_PERCENT
METHOD_OPT
NO_INVALIDATE
GRANULARITY
PUBLISH
INCREMENTAL (only scan partitions that have been changed)
STALE_PERCENT

Setting Statistics Gathering Defaults

Gathering and Publishing
Set preferences to PUBLISH = ‘FALSE’
Gather Statistics
user_tab_pending_stats
user_ind_pending_stats
user_col_pending_stats
Run test case
Alter session set
optimizer_pending_statistics
= TRUE
Run test case Worse
Better
dbms_stats.
delete_pending_stats
dbms_stats.
publish_pending_stats
user_tab_statistics
user_ind_statistics
user_tab_col_statistics

SELECT obj#, TO_CHAR(savtime,'dd/mm/yyyy') save_time,
rowcnt, blkcnt, avgrln, samplesize, analyzetime
FROM wri$_optstat_tab_history
ORDER BY savtime desc

Never publish any statistics until you
are happy they work better

Trace Granularity
Oracle 11g:
Improved Oracle diagnostic event infrastructure
ORADEBUG DOC

Tracing SQL Executions
ALTER SESSION SET EVENTS
'sql_trace[SQL: 324dr9k932njj]
plan_stat=all_executions,wait=true,bind=true';
FROM events_large e, bookings_large b
Execute this twice and format the trace file
ora11_ora_5852_324dr9k932njj.rtf.

We have been able to find information about this with
dbms_xplan since 10g:
Real cardinality v expected cardinality?

SELECT /*+ GATHER_PLAN_STATISTICS */
count(b.comments)
SELECT
DBMS_XPLAN.DISPLAY_CURSOR
(FORMAT=>'ALLSTATS LAST')
FROM dual;

SYS.DBMS_XPLAN_TYPE('SQL_ID 2yxvxc5r9z6tv, child number 0')
SYS.DBMS_XPLAN_TYPE('-------------------------------------')
SYS.DBMS_XPLAN_TYPE('SELECT /*+ GATHER_PLAN_STATISTICS */ count(b.comments) FROM ')
SYS.DBMS_XPLAN_TYPE('events_large e, bookings_large b WHERE e.org_id = 2264 AND e.event_no ')
SYS.DBMS_XPLAN_TYPE('= b.event_no AND e.comments = 'TEST'')
SYS.DBMS_XPLAN_TYPE(' ')
SYS.DBMS_XPLAN_TYPE('Plan hash value: 3269294976')
SYS.DBMS_XPLAN_TYPE('----------------------------------------------------------------------------------------------------------
--------------------------')
SYS.DBMS_XPLAN_TYPE('| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers | Reads |
OMem | 1Mem | Used-Mem |')
--------------------------')
SYS.DBMS_XPLAN_TYPE('| 0 | SELECT STATEMENT | | 1 | | 1 |00:00:02.23 | 36451 | 741 |
| | |')
SYS.DBMS_XPLAN_TYPE('| 1 | SORT AGGREGATE | | 1 | 1 | 1 |00:00:02.23 | 36451 | 741 |
| | |')
SYS.DBMS_XPLAN_TYPE('|* 2 | HASH JOIN | | 1 | 398K| 232 |00:00:13.89 | 36451 | 741 |
1452K| 1452K| 630K (0)|')
SYS.DBMS_XPLAN_TYPE('|* 3 | TABLE ACCESS FULL| EVENTS_LARGE | 1 | 6967 | 4 |00:00:00.01 | 992 | 0 |
| | |')
SYS.DBMS_XPLAN_TYPE('| 4 | TABLE ACCESS FULL| BOOKINGS_LARGE | 1 | 5767K| 5767K|00:00:01.64 | 35459 | 741 |
| | |')
--------------------------')
SYS.DBMS_XPLAN_TYPE('Predicate Information (identified by operation id):')
SYS.DBMS_XPLAN_TYPE('---------------------------------------------------')
SYS.DBMS_XPLAN_TYPE(' 2 - access("E"."EVENT_NO"="B"."EVENT_NO")')
SYS.DBMS_XPLAN_TYPE(' 3 - filter(("E"."ORG_ID"=2264 AND "E"."COMMENTS"='TEST'))')

Cursor Cache
Execute the SQL
Cursor Cache
Parse the SQL
Transform
Estimate cost
Generate
execution plan

If it has to do this it has got it wrong once
and
Its not persistent so it will keep getting it wrong once

Create your statistics so Oracle does not need to do this
dbms_stats.create_extended_stats
('TRAIN1','EVENTS_LARGE','(ORG_ID,COMMENTS)');

Set optimizer_dynamic_sampling so Oracle does not need
to do this
SELECT /*+ dynamic_sampling (4) */ count(b.comments)

No use across multiple tables
FROM organisations o, events_large e,
bookings_large b

Use optimizer_scaling to correct multi table estimates
SELECT /*+ opt_estimate(JOIN,("E","O"),
SCALE_ROWS= 0.000001) */ count(b.comments)
FROM organisations o, events_large e,
bookings_large b

Was it used?

Oracle asking Did I get it right?
Need stable data patterns
Can use where poor stats and no dynamic sampling
Designed to correct mistakes
Not for bind variables
Only once per plan
Not persistent

A Characteristic Problem
I haven’t changed anything
Its really slow this morning
I did the same thing yesterday and it was fine
Actually its OK now
No its not
Thank you so much you’ve fixed it (I haven’t done anything)

Bind Peeking + Adaptive Cursors
SELECT resource_code, count(*)
FROM bookings_large
GROUP BY resource_code;
Majority value
Minority value

Bind Peeking
Look at the value of the bind variable when the statement is
parsed
Plan is based on that value
BEGIN :v3 := ‘PC1'; END;
SELECT COUNT(quantity)
FROM bookings_large
WHERE resource_code = :v3;
BEGIN :v3 := ‘BRLG'; END;
SELECT COUNT(quantity)
FROM bookings_large
WHERE resource_code = :v3;
SAME PLAN
STILL INDEX
for 34% of rows
INDEX
for <1% of rows

SELECT COUNT(l.quantity) FROM train.bookings_large l
WHERE resource_code = :v1
SELECT sql_id FROM v$sqlarea
WHERE sql_text = 'SELECT COUNT(l.quantity)
FROM train.bookings_large l WHERE resource_code = :v1'

BEGIN :v1 := 'PC1'; END;
SELECT sql_id, child_number, is_bind_sensitive, is_bind_aware
FROM v$sql
WHERE sql_id ='95jktg3mza0qm';
IS_BIND_SENSITIVE - may need to change the plan
- set on first execution of the
statement
IS_BIND_AWARE - do need to change the plan

SELECT child_number, bind_set_hash_value, peeked, executions,
rows_processed, buffer_gets
FROM v$sql_cs_statistics
WHERE sql_id ='95jktg3mza0qm'

------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.01 0.01 0 0 0 0
Fetch 2 0.12 0.43 119 6635 0 1
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 4 0.14 0.44 119 6635 0 1
Misses in library cache during execute: 1
Parsing user id: 82
------- ---------------------------------------------------
48510 TABLE ACCESS BY INDEX ROWID BOOKINGS_LARGE (cr=6635 pr=119 pw=119 time=2870 us
cost=1977 size=316008 card=39501)
48510 INDEX RANGE SCAN BK_RES2 (cr=104 pr=0 pw=0 time=527 us cost=90 size=0
card=39501)(object id 69870)
********************************************************************************
---------------------------------------- Waited ---------- ------------
********************************************************************************

BEGIN :v1 := ‘BRLG'; END;
SELECT sql_id, child_number, is_bind_sensitive, is_bind_aware
FROM v$sql
WHERE sql_id ='95jktg3mza0qm';

------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 2 6.29 78.82 34335 39594 0 1
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 4 6.29 78.82 34335 39594 0 1
Parsing user id: 82
------- ---------------------------------------------------
1991051 TABLE ACCESS BY INDEX ROWID BOOKINGS_LARGE (cr=39594 pr=34335 pw=34335 time=626800
us cost=1977 size=316008 card=39501)
1991051 INDEX RANGE SCAN BK_RES2 (cr=4437 pr=4436 pw=4436 time=61616 us cost=90 size=0
card=39501)(object id 69870)
---------------------------------------- Waited ---------- ------------
db file sequential read 34335 0.29 72.08

------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 2 1.23 21.71 35341 35346 0 1
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 4 1.23 21.72 35341 35346 0 1
Misses in library cache during execute: 1
Parsing user id: 82
------- ---------------------------------------------------
1991051 TABLE ACCESS FULL BOOKINGS_LARGE (cr=35346 pr=35341 pw=35341 time=414975 us
cost=9652 size=15700664 card=1962583)
---------------------------------------- Waited ---------- ------------
********************************************************************************

Bind Peeking + Adaptive Cursors in 11g
Statements with bind variables (+histograms) are bind sensitive
The first time you execute a statement with different selectivity it
uses the original plan
The second time it changes the plan and become bind aware
New values will use a plan for the appropriate selectivity range

Adaptive Cursors Views
V$SQL
V$SQL_CS_SELECTIVITY

Adaptive Cursors Views
V$SQL_CS_HISTOGRAM
V$SQL_CS_STATISTICS

PLSQL has soft parse avoidance
Implicit cursor
Explicit cursor
Native dynamic SQL
Ref cursor
Session_cached_cursors = 0
Ref cursors
No default adaptive cursor functionality in a session
Default adaptive cursor functionality in a session
Once another session executes the statement it will adapt

Adaptive Cursors Functionality
Bind variable with Equality and Histogram
Not for range conditions
Bind variable with Equality and Histogram
Range conditions
Does not support LIKE
/*+ BIND_AWARE */
Adaptive Cursors 11.1.0.6
Adaptive Cursors 11.1.0.7 and 11.2

So where are we now?
Our own code – works properly first time with hint
in SQL and PL/SQL
Software Packages – will still get it wrong once
and won’t use adaptive cursors for PL/SQL
at all within one session
unless we set session cached cursors =0
or ref cursors

The Cost Based Optimiser in 11gR2

xxxxxxxxxxxxxxxxxxxxx
Maybe you should buy an
Exadata box

So Much Promise
And then the
disappointment

Bind Peeking + Adaptive Cursors Summary
Your code
Packages
Minimise statement invalidations
Consider running the statement with minority and majority value on start up
Use the BIND_AWARE hint (SQL and PL/SQL) for skewed data
Use ref cursors (if hints are not allowed)
PL/SQL – set session_cached_cursors = 0 (temporarily)

Adaptive Cursors What we Really Need
Once a statement has been bind aware it knows next time
its parsed
Adaptive Cursors Persistence

Try to get rid of your hints
_optimizer_ignore_hints
Create additional statistics
Set _optimizer_ignore_hints to TRUE
Test the app
If it runs OK remove your hints

68 New Hints
SELECT name, inverse, sql_feature, version
FROM v$sql_hint
WHERE version like '11%'
ORDER BY version desc, name
New Hints

Much More Query Transformation
SELECT e.event_no, e.start_date, sum(cost) totcost
WHERE e.event_no = b.event_no
GROUP BY e.event_no, e.start_date
alter session set tracefile_identifier = Penny
alter session set events '10053 trace name context forever'
explain plan for
SELECT e.event_no, e.start_date, sum(cost) totcost
GROUP BY e.event_no, e.start_date;
alter session set events '10053 trace name context off'

10G – JOIN before the GROUP BY

11G – GROUP BY before the JOIN

New Hints
MONITOR
NO_MONITOR
select /*+ MONITOR */
count(comments) from bookings ;
select DBMS_SQLTUNE.REPORT_SQL_MONITOR(
session_id=>sys_context('userenv','sid'),
report_level=>'ALL') as report
from dual;

Check the differences in plans
Upgrading

Results Cache
SGA
populate invalidate
Most recently
used
SQL/PL/SQL
SQL Query
Results Cache
Results Cache
PL/SQL Function
Results Cache
Most recently used result sets
Buffer Cache Library Cache
Most recently
data

Query Results Cache
RESULT_CACHE_MODE MANUAL
FORCE
/*+RESULT_CACHE */
/*+ NO_RESULT_CACHE */
SELECT /*+ RESULT_CACHE */
count(b.comments)
FROM train1.events_large e, train1.bookings_large b
WHERE e.org_id = :v1
AND e.comments = :v2;

PL/SQL Function Results Cache
CREATE OR REPLACE FUNCTION quantity_booked
(p_resource_code in resources.code%TYPE,p_event_date in date)
RETURN NUMBER
RESULT_CACHE
IS
v_total_booked number := 0;
BEGIN
SELECT sum(b.quantity)
INTO v_total_booked
FROM bookings b, events e
AND p_event_date between e.start_date and e.end_date
AND b.resource_code = p_resource_code;
RETURN (v_total_booked);
END;

Monitoring the Results Cache
SELECT * FROM v$result_cache_memory
SELECT * FROM v$result_cache_objects
SELECT * FROM v$result_cache_statistics
SELECT * FROM v$result_cache_dependency
RESULT_CACHE_MAX_SIZE
RESULT_CACHE_MAX_RESULT
Explain Plan

CBO - Instability
CBO
?access
path
decision
System parameters
Session parameters
Optimiser StatisticsIndexes
SQL Profile
Software version
System statistics
DBA playing around

CBO - Instability
– Leave it all alone
– SQL Plan Management
– Store plan baseline
– Plans not used till accepted
– Manually accept or
– Allow Oracle to evolve plans
Solution 1
Solution 2

SQL Plan Management
Manual capture
DBMS_SPM.LOAD_PLANS_FROM_SQLSET
DBMS_SPM.LOAD_PLANS_FROM_CURSOR_CACHE
Auto capture of repeatable statements
OPTIMIZER_CAPTURE_SQL_PLAN_BASELINE = TRUE
Baseline =
(Stored and Accepted plans)
SQL Management Base
New Plan
identified during
execution
Stored not accepted
SQL Tuning Advisor identifies new
plan – SQL*Profile accepted
Auto accept of new plan
(if it performs better)
DBMS_SPM.EVOLVE_SQL_PLAN_BASELINE
Manual load/accept of new plan
DBMS_SPM.LOAD_PLANS_FROM_SQLSET
DBMS_SPM.LOAD_PLANS_FROM_CURSOR_CACHE

Questions?

The Cost Based Optimiser in 11gR2

More Related Content

What's hot (20)

Viewers also liked (9)

Similar to The Cost Based Optimiser in 11gR2 (20)

More from Sage Computing Services (13)

Recently uploaded (20)

The Cost Based Optimiser in 11gR2