![SQL Plan Management | Mechanism
• Phase 1 – Baseline Capture
• Set OPTIMIZER_CAPTURE_SQL_PLAN_BASELINES=TRUE
• If OPTIMIZER_CAPTURE_SQL_PLAN_BASELINES=TRUE then all found plans will be put into the baseline
Parse Execute
SQL
GB
HJ
...HJ
SQL Plan acceptable
2X
GB
HJ
...HJ
Plan Baseline
SQL MANAGEMENT BASE
Residing in SYSAUX TS.
Will occupy max. 10% of SYSAUX.
Weekly job will delete plans
not used since 53 weeks [default].
Before Oracle 12c a massive number of hints get stored –
since Oracle 12c entire plans will be kept
SQL Profiles
Initial plan
will be accepted
if repeated and
will be added to the
SQL Plan Baseline
Plan History](https://image.slidesharecdn.com/27143699342350655448webinar3performance-200702101941/85/Performance-Stability-Tips-and-Tricks-and-Underscores-32-320.jpg)
![Cool Features | Inc Stats Enhancement
• Significantly smaller synopses with new algorithm
• Preference name: APPROXIMATE_NDV_ALGORITHM
• 'REPEAT OR HYPERLOGLOG' [Default]
• If synopses of old format exists, continue with the "old" algorithm
• If no or new synopses exist, use the "new" algorithm
• 'ADAPTIVE SAMPLING'
• Force to use pre-12.2 adaptive sampling algorithm – delete existing 12.2 synopses and regather - "old behavior"
• 'HYPERLOGLOG'
• Force to use the new algorithm
• Lab example:
• 8TB table with 84 partitions
• → Size in Oracle 12.1.0.2: 167 GB
• → Size in Oracle 12.2.0.1: 7 GB
NEW IN
12.2](https://image.slidesharecdn.com/27143699342350655448webinar3performance-200702101941/85/Performance-Stability-Tips-and-Tricks-and-Underscores-69-320.jpg)
Performance Stability, Tips and Tricks and Underscores
- 1. Upgrade to Oracle Database 19c Performance Stability, Tips and Tricks and Underscores Mike Dietrich Master Product Manager Database Upgrade and Migration Roy Swonger Vice President Database Upgrade, Utilities and Patching Virtual Classroom Series
- 2. Vice President Database Upgrade, Utilities & Patching Roy F. Swonger @RoyFSwonger
- 3. Master Product Manager Database Upgrade and Migrations Mike Dietrich https://MikeDietrichDE.com @MikeDietrichDE mikedietrich
- 5. Content Overview Migration to Multitenant 4 Ensure Performance Stability 3 Upgrade to Oracle Database 19c 2Release and Patching Strategy 1
- 6. Your key to successful Database Upgrades 1 2 3 4 Download and install Oracle 19c Download and install newest RU Download and use newest AutoUpgrade eDelivery MOS: 2521164.1 MOS: 2485457.1 SPM, STA, RAT for Performance Stability
- 7. Upgrade 19c | Speed it up • Check when dictionary stats have been gathered the last time • Refresh stats a day before the upgrade column OPERATION format a40 set linesize 200 select to_char(max(END_TIME),'DD-MON-YY hh24:mi') LATEST, OPERATION from DBA_OPTSTAT_OPERATIONS where OPERATION in ('gather_dictionary_stats', 'gather_fixed_objects_stats') group by operation;
- 8. Upgrade 19c | Make it run faster … • Stats not gathered on several clustered indexes (bug 25286819) • But wait … autoupgrade.jar does this for you already! exec dbms_stats.gather_schema_stats('SYS'); exec dbms_stats.gather_index_stats('SYS','I_OBJ#'); exec dbms_stats.gather_index_stats('SYS','I_FILE#_BLOCK#'); exec dbms_stats.gather_index_stats('SYS','I_TS#'); exec dbms_stats.gather_index_stats('SYS','I_USER#'); exec dbms_stats.gather_index_stats('SYS','I_TOID_VERSION#'); exec dbms_stats.gather_index_stats('SYS','I_MLOG#'); exec dbms_stats.gather_index_stats('SYS','I_RG#');
- 9. Upgrade 19c | Effect of having good stats • The larger the dictionary, the bigger the effect • This example has been done with one of the tiny Hands-On Lab databases Duration Reduction No dictionary and fixed objects stats 15 min 55 sec Gathered dictionary and fixed objects stats 14 min 10 sec 11 % Gathered schema and cluster index stats 13 min 41 sec 3.4 % to previous Total downtime saved 2 min 14 sec 14 % overall
- 10. Effect of Having Good Statistics | EBS • Upgrade duration for Oracle E-Business Suite Duration Reduction No dictionary and fixed objects stats 10 hrs 56 min 52 sec Gathered dictionary and fixed objects stats 52 min 42 sec 93 % Gathered schema and cluster index stats 52 min 25 sec 0.5 % to previous Total downtime saved 10 hrs 4 min 14 sec 93.5 % overall
- 11. Effect of Having Good Statistics • Stale/no stats: 9:59:23.87 • Good stats: 0:00:02.33 ID OPERATION OPTIONS OBJECT_NAME ----- -------------------- -------------------- ----------- 0 UPDATE STATEMENT 1 UPDATE DEPENDENCY$ 2 FILTER 3 TABLE ACCESS FULL DEPENDENCY$ 4 INDEX FULL SCAN I_OBJ2 5 INDEX FULL SCAN I_OBJ2 6 TABLE ACCESS BY INDEX ROWID BATCHED OBJ$ 7 INDEX RANGE SCAN I_OBJ1 8 TABLE ACCESS BY INDEX ROWID BATCH OBJ$ 9 INDEX RANGE SCAN I_OBJ1 ID OPERATION OPTIONS OBJECT_NAME ----- -------------------- -------------------- -------------- 0 UPDATE STATEMENT 1 UPDATE DEPENDENCY$ 2 FILTER 3 TABLE ACCESS FULL DEPENDENCY$ 4 INDEX RANGE SCAN I_OBJ1 5 INDEX RANGE SCAN I_OBJ1 6 TABLE ACCESS BY INDEX ROWID BATCHED OBJ$ 7 INDEX RANGE SCAN I_OBJ1 8 TABLE ACCESS BY INDEX ROWID BATCH OBJ$ 9 INDEX RANGE SCAN I_OBJ1
- 12. AutoUpgrade The recommended way to upgrade databases PhotobyJoshuaSortinoonUnsplash
- 13. Download | Get the latest autoupgrade.jar MOS Note: 2485457.1
- 14. Upgrade | AutoUpgrade upg> Job Console 11.2.0.4 12.1.0.2 12.2.0.1 18c autoupgrade.jar MOS Note: 2485457.1 2MB config.cfg autoupgrade.jar 12.2.0.1 18.5.0 19c 19c
- 15. Performance Best Practices A prescription to ensure performance stability
- 16. Parameter Recommendations General guidelines: • The fewer parameters you have in your spfile, the better • Remove outdated parameters, underscores, events ... • Unless directed by applications (e.g. E-Business Suite, Siebel, SAP etc.) to set them • For EBS: Interoperability notes
- 17. Things to do right after upgrade Configure statistics history retention period • Check space usage: • Check retention • Default: 31 days • Adjust setting • Example: 10 days SQL> select SPACE_USAGE_KBYTES/1024 MB from V$SYSAUX_OCCUPANTS where OCCUPANT_NAME='SM/OPTSTAT'; SQL> select DBMS_STATS.GET_STATS_HISTORY_RETENTION from DUAL; SQL> exec DBMS_STATS.ALTER_STATS_HISTORY_RETENTION(10);
- 18. Help me - I have an upgrade problem … In >95%of all cases PhotobynikkomacaspaconUnsplash “ ” an "upgrade problem" in reality is a performance issue after upgrade, or not a database upgrade topic. There is exactly one way to mitigate the risk: TESTING!
- 19. The entire upgrade project of our 300 Oracle databases at La Mobilier is running very well. When we catch issues, we fix them before going live. Paolo Kreth Group Manager Databases La Mobiliere “ ”
- 20. Testing | Typical Mistakes • Only 10% of real data used • Data sets artificially created • Tests done on a laptop • No testing tools used • Data from 6 months ago • No statistics refreshed • Testing?? Waste of time! • Real experts fix it after go-live … PhotobyElijahO'DonnellonUnsplash
- 21. Best Practice | COMPATIBLE vs Optimizer • COMPATIBLE and OPTIMIZER_FEATURES_ENABLE • Fully independent from each other • Set COMPATIBLE to the default of the release, e.g. 19.0.0 • Change OPTIMIZER_FEATURES_ENABLE only when you have to • Avoid it if possible • This is not a Swiss Army knife! • You will turn off a lot of great features
- 22. Testing | Completeness versus Costs Costs Completeness low high 0%100% 80% - 90% Costslow high Completeness 0%100%
- 23. Testing Tools | Workflow Collect execution plans before upgrade Compare AWR Snapshots Verify them with SQL Performance Analyzer Regressed plans? SQL Plan Management SQL Tuning Advisor Verify functionality and performance with Database Replay
- 24. Testing Tools | Collect Plans on Production STS Capturing from cursor cache and AWR into an STS does not require an extra license for EE anymore: Find it under: "SQL Tuning Sets" at: https://docs.oracle.com/database/121/DBLIC/editions.htm#DBLIC109
- 25. Testing Tools | Comparison Collect execution plans before upgrade Compare AWR Snapshots Verify them with SQL Performance Analyzer Regressed plans? SQL Plan Management SQL Tuning Advisor Verify functionality and performance with Database Replay
- 26. Testing Tools | AWR Diff Reports Upgrade / Migrate 11.2.0.4 19.7.0 Snapshot Interval Compare AWR Diff Report: awrddrpt.sql Snapshot Interval
- 27. Testing Tools | SQL Performance Analyzer Collect execution plans before upgrade Compare AWR Snapshots Verify them with SQL Performance Analyzer Regressed plans? SQL Plan Management SQL Tuning Advisor Verify functionality and performance with Database Replay
- 28. Testing Tools | SQL Performance Analyzer SQL Performance Analyzer STS Capture SQLs into a STS Load STS STS Evaluate, compare and tune – over and over again! 11.2.0.4 19.7.0 Upgrade / Migrate
- 29. Testing Tools | SPA - Regressed Report
- 30. Testing Tools | SQL Plan Management Collect execution plans before upgrade Compare AWR Snapshots Verify them with SQL Performance Analyzer Regressed plans? SQL Plan Management Verify functionality and performance with Database Replay
- 31. Testing Tools | Situation after change • Challenging to "freeze" execution plans Parse Execute Plan acceptable Plan possibly not acceptable Conditions change: New statistics, parameter change, upgrade GB HJ ...HJ GB NL ...NL SQL SQL ExecuteParse
- 32. SQL Plan Management | Mechanism • Phase 1 – Baseline Capture • Set OPTIMIZER_CAPTURE_SQL_PLAN_BASELINES=TRUE • If OPTIMIZER_CAPTURE_SQL_PLAN_BASELINES=TRUE then all found plans will be put into the baseline Parse Execute SQL GB HJ ...HJ SQL Plan acceptable 2X GB HJ ...HJ Plan Baseline SQL MANAGEMENT BASE Residing in SYSAUX TS. Will occupy max. 10% of SYSAUX. Weekly job will delete plans not used since 53 weeks [default]. Before Oracle 12c a massive number of hints get stored – since Oracle 12c entire plans will be kept SQL Profiles Initial plan will be accepted if repeated and will be added to the SQL Plan Baseline Plan History
- 33. SQL Plan Management | Mechanism • Phase 2 – Selection • Same statement parsed again but a different plan will be created GB HJ ...HJ Plan Baseline Plan History Parse Execute SQL GB NL ...NL Plan acceptable • OPTIMIZER_USE_SQL_PLAN_BASELINES=TRUE • OPTIMIZER_CAPTURE_SQL_PLAN_BASELINES=FALSE • Both switched to their defaults • Annotation: If OCSPB=TRUE then this plan would have been added to the baseline New plan will be added to the Plan History but it won‘t be used unless it has been verified GB NL ...NL
- 34. SQL Plan Management | Mechanism • Phase 3 – Evolution • Since Oracle 12.1.0.2: GB HJ ...HJ Plan Baseline Plan History GB NL ...NL GB NL HJHJ GB HJ ...HJ Plan Baseline Plan History GB NL ...NL GB NL HJHJ Equal or better plans can be added to the SQL Plan Baseline Optimizer Worse plans will be kept in the Plan History and will get purged New SYS_AUTO_SPM_EVOLVE_TASK job as part of the Automatic SQL Tuning Task Report: DBMS_SPM.REPORT_AUTO_EVOLVE_TASK Manual: DBMS_SPM.CREATE_EVOLVE_TASK
- 35. SQL> select PARAMETER_NAME, PARAMETER_VALUE from DBA_SQL_MANAGEMENT_CONFIG; SQL Plan Management | Settings • Configure SQL Plan Management (SPM) • Check settings: • Change retention: • Default: 53 weeks • Change space consumption: • Default: 10% of SYSAUX • Plans will be stored in a LOB • Sources to load plans from: SQL> exec DBMS_SPM.LOAD_PLANS_FROM_AWR (begin_snap=>29, end_snap=>69); SQL> exec DBMS_SPM.CONFIGURE('plan_retention_weeks',5); SQL> exec DBMS_SPM.CONFIGURE('space_budget_percent',5);
- 36. SQL Plan Management | Plan Stability GB HJ ...HJ Plan Baseline Plan History STS STSExport/Import for Migrations Upgrade Plans will be takenfrom STS and addedto the SQL Plan Baseline DBMS_SPM.LOAD_PLANS_FROM_SQLSET
- 37. Testing Tools | SQL Tuning Advisor Collect execution plans before upgrade Compare AWR Snapshots Verify them with SQL Performance Analyzer Regressed plans? SQL Plan Management SQL Tuning Advisor Verify functionality and performance with Database Replay
- 38. SQL Tuning Advisor | SQL Profiles • SQL Profiles provide more information to the optimizer • Part of the Tuning Pack • FORCE_MATCH=TRUE • Don't change the application code • Persistent, transportable (MOS Note:751068.1), can be tested without risk • Example:
- 39. PhotobyTjSedisaonUnsplash SQL Tuning Advisor: Example 1. Problem to be solved 2. Select candidate statement to tune 3. Get SQL statement details 4. Create and run SQL Tuning task 5. View report results
- 40. SQL Tuning Example 1. Problem to be solved
- 41. SQL Tuning Example 2. Select candidate statement to tune • Generate an AWR Report
- 42. SQL Tuning Example 3. Get SQL Statement Details
- 43. SQL Tuning Example 4. Create and run SQL Tuning task VARIABLE stmt_task VARCHAR2(64); EXEC :stmt_task := DBMS_SQLTUNE.CREATE_TUNING_TASK(sql_id => 'f4k19gvr3nu38'); EXEC DBMS_SQLTUNE.EXECUTE_TUNING_TASK(:stmt_task); SET LONG 10000 SET LONGCHUNKSIZE 10000 SET LINESIZE 100 SPOOL sql_tune_f4k19gvr3nu38.txt SELECT DBMS_SQLTUNE.REPORT_TUNING_TASK( :stmt_task )FROM DUAL; SPOOL OFF;
- 44. SQL Tuning Example 5. View report results
- 45. SQL Tuning Example 5. View report results
- 46. SQL Tuning Example 5. View report results
- 47. SQL Profile - evaluation • SQL Profiles should be evaluated before making them available to every user: • Now evaluate the statement's profile in a limited user context • If verification went fine, make it accessible to everybody exec :p_name:=dbms_sqltune.accept_sql_profile task_name=>‘TASK_21944',name=>'XT_PROFILE', category=>'TEST_ENV', FORCE_MATCH=>TRUE) alter session set SQLTUNE_CATEGORY='TEST_ENV'; exec dbms_sqltune.alter_sql_profile (name=>'XT_PROFILE', attribute_name=>'CATEGORY',value=>'DEFAULT')
- 48. SQL Tuning Example 5. View report results
- 49. SQL Tuning Example 5. View report results
- 50. SQL Tuning Example 6. Act on findings • Follow 5 statistics recommendations to gather stats on 5 tables • Result: 20x improvement!
- 51. Testing Tools | Database Replay Collect execution plans before upgrade Compare AWR Snapshots Verify them with SQL Performance Analyzer Regressed plans? SQL Plan Management SQL Tuning Advisor Verify functionality and performance with Database Replay
- 52. Testing Tools | Database Replay Database Replay Capture Workload Replay Workload Compare – then restore and run again with changes 11.2.0.4 19.7.0 Migrate Preprocess Workload
- 54. Time for a break
- 55. The "hidden" gems Fix it when you upgrade PhotobyandréspilborghsonUnsplash
- 56. Behavior Change 19c | Automatic SPM • SQL Plan Management: "Automatic" and enabled on Exadata
- 57. Behavior Change 19c | Automatic SPM • SQL Plan Management: "Automatic" and enabled on Exadata • If you'd like to revert to 12.1 or 12.2/18c behavior: • https://mikedietrichde.com/2019/06/03/automatic-sql-plan-management-in-oracle-database-19c/
- 58. Statistics Advisor Since Oracle 12.2.0.1, the Statistics Advisor is active • Some customers reported high growth of entries • Active on Standard Edition (SE2) as well • https://mikedietrichde.com/2017/08/22/oracle-optimizer-statistics-advisor-in-oracle-database-12-2-0-1/ • If you desire to turn it off: • BEGIN dbms_advisor.set_task_parameter ('AUTO_STATS_ADVISOR_TASK','_AUTO_MMON_INTERVAL',9999999999); dbms_advisor.set_task_parameter ('AUTO_STATS_ADVISOR_TASK','_AUTO_STATS_INTERVAL',9999999999); END; /
- 59. Automatic SQL Tuning Set In Oracle 19.7.0, a Automatic SQL Tuning Set gets populated • Some customers reported high growth and consumption in SYSAUX • See: https://mikedietrichde.com/2020/05/28/do-you-love-unexpected-surprises-sys_auto_sts-in-oracle-19-7-0/ • If you want to disable it: • BEGIN DBMS_AUTO_TASK_ADMIN.DISABLE(client_name=>'Auto STS Capture Task', operation=>NULL, window_name=>NULL); END; / • Task is not enabled by default from Oracle 19.8.0 on
- 60. Oracle 19c | _cursor_obsolete_threshold • MOS Note: 2431353.1 High Version Counts for SQL statements (>1024) post upgrade To 12.2 and above causing database slow performance • Since Oracle 12.2.0.1, the old default of 1024 has been raised to 8192 • This may lead to various drastic performance issues • Recommendation: • Set it to the old default to avoid mutex concurrency and other issues: _cursor_obsolete_threshold=1024
- 61. Oracle 19c | deferred_segment_creation • MOS Note: 1216282.1 All sorts of issues and bugs - from corruption to mutex contention to customer scripts not operating correctly anymore • Recommendation: Set it to deferred_segment_creation=false unless you have a valid reason
- 62. Oracle 19c | Adaptive Features • Recommendation: Set optimizer_adaptive_statistics explicitly in your SPFILE • See: • https://mikedietrichde.com/2018/01/18/additional-info-adaptive-features-fixes-oracle-12-1-0-2/ • MOS Note: 2187449.1 - Recommendations for Adaptive Features in Oracle Database 12.1 Oracle 12.1 optimizer_adaptive_features = TRUE Oracle 12.2 Adaptive Statistics optimizer_adaptive_statistics Default: FALSE Adaptive Plans optimizer_adaptive_plans Default: TRUE
- 63. Oracle 19c | _sql_plan_directive_mgmt_control • MOS Note: 2209560.1 - How To Disable SQL Plan Directive (SPD) • In order to fully disable SQL Plan Directives, you need to set: • _sql_plan_directive_mgmt_control=0 • Otherwise the database collects SPDs in the background, but won't use it • Having optimizer_adaptive_statistics=false - which is the default - disables only the usage of SPDs but not their creation
- 64. "Hidden" Features | Extended VARCHAR2 • Feature Introduced in Oracle 12.1.0.2 • Allow 32kB support for (N)VARCHAR2 • Allows length >4000 byte ALTER SYSTEM set MAX_STRING_SIZE=EXTENDED scope=SPFILE; CREATE TABLE Applicants (id NUMBER GENERATED AS IDENTITY, first_name varchar2(30), last_name varchar2(30), application date, CV varchar2(32767) ); SHUTDOWN IMMEDIATE STARTUP UPGRADE @?/rdbms/admin/utl32k.sql
- 65. "Hidden" Features | Extended VARCHAR2 • Fine print: • Oracle Forms and Reports don't support Extended VARCHAR2 • MOS Note: 2221740.1 ORA-24920 In Oracle Forms 12.2.x when using MAX_STRING_SIZE=EXTENDED • Above 3964 byte, data will be stored in an inline SecureFile LOB • See: http://www.ludovicocaldara.net/dba/extended-data-types-storage/ • Be aware of performance impact • See: https://blog.dbi-services.com/12c-extended-datatypes-better-than-clob/
- 66. Cool Features | Incremental Statistics • Introduced in Oracle 11g • DWH, Partitioning • Synopses on disk can consume hundreds of GB • Since Oracle Database 12.2.0.1: New algorithm for synopses can drastically reduce space consumption
- 67. Cool Features | Incremental Statistics Sales Table May 22nd 2019 May 23rd 2019 May 18th 2019 May 19th 2019 May 20th 2019 May 21st 2019 May 24rd 2019 Global Statistic Global stats generated by aggregating the original partition synopsis with the new one A new partition is added to the table & data is loaded Global stats generated by aggregating partition level statistics and synopsis Retrieve synopsis for each of the other partitions from SYSAUX Gather partition statistics for new partition Partition level stats are gathered & synopsis created S1 S2 S3 S4 S5 S6 S7 SYSAUX Tablespace
- 68. Cool Features | Inc Stats Enhancement • Enable that "changed" partitions won't be eligible for new stats generation • Adjust the stale percentage if needed SQL> exec DBMS_STATS.SET_DATABASE_PREFS('INCREMENTAL_STALENESS','USE_STALE_PERCENT'); SQL> exec DBMS_STATS.SET_DATABASE_PREFS('STALE_PERCENT','12'); NEW IN 12.1
- 69. Cool Features | Inc Stats Enhancement • Significantly smaller synopses with new algorithm • Preference name: APPROXIMATE_NDV_ALGORITHM • 'REPEAT OR HYPERLOGLOG' [Default] • If synopses of old format exists, continue with the "old" algorithm • If no or new synopses exist, use the "new" algorithm • 'ADAPTIVE SAMPLING' • Force to use pre-12.2 adaptive sampling algorithm – delete existing 12.2 synopses and regather - "old behavior" • 'HYPERLOGLOG' • Force to use the new algorithm • Lab example: • 8TB table with 84 partitions • → Size in Oracle 12.1.0.2: 167 GB • → Size in Oracle 12.2.0.1: 7 GB NEW IN 12.2
- 70. Cool Features | Inc Stats Enhancement • Preference: HYPERLOGLOG • Replace "old" with "new" - Recommended approach • Preference: REPEAT OR HYPERLOGLOG • Coexistence of "old" and "new" - Issues seen at customers with stats gathering SALESSYSAUX Tablespace P2 P1 has a synopsis in pre-12.2 format New synopsis for P1 in compact 12.2 format Partitioned Table P1 exec dbms_stats.set_table_prefs(owner, table, 'approximate_ndv_algorithm', 'hyperloglog'); exec dbms_stats.set_table_prefs(owner, table, 'incremental_staleness', 'NULL') SALESSYSAUX Tablespace P2 P1 has a synopsis in pre-12.2 format Partitioned Table P1 exec dbms_stats.set_table_prefs(owner, table, 'approximate_ndv_algorithm', 'hyperloglog'); exec dbms_stats.set_table_prefs(owner, table, 'incremental_staleness', 'allow_mixed_format') Synopsis forP2 in compact12.2 format NEW IN 12.2
- 71. Copyright © 2020 Oracle and/or its affiliates. Now RELAX ... Stay Calm ... And open an SR with Oracle Support in case of real trouble
- 72. DEMO
- 76. Thank you!
- 77. Upcoming Webinars Webinar 4: Thursday, July 2 at 11:00h CEST Migration to Oracle Multitenant https://tinyurl.com/upgrade19c