Myth busters - performance tuning 102 2008
- 1. 4 June 2008 MythBusters - performance tuning 102 Paul Guerin Bachelor of Engineering (Computer) Oracle DBA Certified Professional DBA 10+ years Employers: Origin Energy, Bluescope Steel, BHP Billiton
- 2. Why is application performance important? Directly: End users not waiting as long for data retrieval. More productive use of time while in the office. Increased likelyhood of meeting a tight deadline – especially if technical issues arise. Less frustration that may lead to comments along the line of “This system is xxxx!!!” Greater job satisfaction in current role? Indirectly: Improves employee performance. Increased value to the organisation. Less employee churn? faster application performance = increased user satisfaction
- 3. Topics Fundamentals + Cost Based Optimiser Column constraints + indexes SQL tuning + subqueries Non-SQL performance issues, including undo storage Processing: serial + parallel Data migrations
- 4. Fundamentals of application performance Minimise the database workload Minimise logical I/O (which in turn reduces physical I/O). Minimise unnecessary sorting. Minimise the result set. Balance the database workload Execute jobs outside normal business hours (including materialised views). Parallelise the database workload Not an option where high concurrency exists.
- 5. Cost Based Optimiser The Cost Based Optimiser (CBO) is a program in the database that predicts the cost of various execution plans and selects the one with the lowest estimated. Estimated cost = logical I/O + physical I/O + external database I/O Note: Cost is in terms of database blocks, not rows. To help predict the lowest cost execution plan, the CBO needs to refer to: Statistics for the table, columns, indexes, and system. Any constraints on the table columns. Rewriting the SQL influences the prediction that the CBO makes.
- 6. Cost Based Optimiser statistics Myth: Old statistics always lead to poor performance. Fact - Old statistics often remain accurate. Fact – A single insert usually makes the statistics obsolete, so most all database statistics are obsolete anyway. Myth: Recent statistics always lead to improved performance. Fact - New statistics are often not more accurate than the old ones if an insert has just taken place.
- 7. Cost Based Optimiser statistics Myth: Gathering CBO stats after each DML statement is the best practice. Myth: Taking statistics can’t hurt. Fact – Cached cursors are invalidated and must be validated again before reuse. Increased overhead. Fact – Increased contention for the same object blocks. Increased latching. Fact - Most of the time recent statistics don’t change the execution plan. Fact - Gathering statistics can become more CPU intensive than the actual SQL statement. Fact - Oracle does not automatically gather statistics for the reasons above. Increased overhead and latching leads to higher contention for CPU time.
- 8. Cost Based Optimiser statistics Myth: Poor performance always caused by the database. Fact – Network issues can be the cause (e.g. PSPD migration to new host)
- 9. Column constraints Column constraints are used to enforce data integrity. Some column constraints are associated with: an index. e.g primary key, unique key. another table. e.g. foreign key. Creating column constraints alone can result in reduced logical I/O and data sorting - resulting in better performance. However the sole presence of column constraints may not result in satisfactory performance. The creation of additional indexes may be necessary.
- 10. Indexes If column constraints alone do not result in satisfactory performance then create an index on the: Foreign key column: important if deleting or updating the primary key of the parent table. Predicate of a WHERE clause. ORDER BY or GROUP BY clause providing the index is in the same sort order. MIN() or MAX() function: index can be asc or desc (key can be anywhere in concatenated index). Single column index ideally for predicates separated by OR operators. Concatenated column index ideally for predicates separated by AND operators. Myth: An index access is always faster than a full-table scan. Fact - For large result sets a full-table scan is often faster. Fact - A table that features a poor clustering factor with respect to its indexes will perform better with a full-table scan.
- 11. SQL tuning tips Beware of statements that implicitly sort: DISTINCT, GROUP BY (<=9i), UNION (prefer UNION ALL) Use ROWNUM to retrieve only the top rows of a sort: SELECT * FROM (SELECT … FROM … ORDER BY …) WHERE rownum<=5; Write SQL that may be able to use an index: =, >, <, LIKE, IN Trailing wildcard character (%). Conditions that compare columns with constants. Rewrite SQL that will never be able to use an index: Functions, including implicit type conversions. Leading wildcard character (%). NULL, NOT, <>, !=, ||. Use >-9.99*POWER(10,125) instead of IS NOT NULL. Myth: Tuning an SQL statement only benefits that statement. Fact – All statements that execute concurrently also benefit due to less CPU contention.
- 12. SQL tuning tips – EXISTS or IN subqueries A subquery (i.e. semi-join) returns one row in the first table for which at least one match is found, whereas an inner-join returns every match between tables. Many different types of subqueries: Correlated (IN, EXISTS): A correlated subquery is where the inner query references a column in the outer query. The inner query is evaluated for each row processed in the outer query. e.g. SELECT * FROM hr.dept WHERE EXISTS ( SELECT NULL FROM hr.employees WHERE employees.department_id = dept.deptno ); Uncorrelated (IN, EXISTS): e.g. SELECT * FROM hr.dept WHERE deptno IN ( SELECT department_id FROM hr.employees ); General guidelines for EXISTS and IN subqueries: Choose an IN where the inner query is more selective. Choose an EXISTS where outer query is more selective.
- 13. SQL tuning tips – joins or subqueries The CBO will automatically attempt to unnest a correlated or uncorrelated subquery into a join if it thinks indexes will improve efficiency. Every subquery can be rewritten as a join. Not necessarily every join can be rewritten as a subquery. General guidelines for joins and subqueries: Use a subquery when no columns from the subquery table(s) are referenced. Use a join when you need to return columns from more than one table.
- 14. SQL tuning tips – multitable transactions Specify WITH CHECK OPTION to indicate that Oracle prohibits any changes to the table or view that would produce rows that are not included in the subquery. UPDATE/DELETE/INSERT INTO (select … from … with check option ) VALUES (…); Use the MERGE statement to select rows from one table for update or insertion into another table. The decision whether to update or insert into the target table is based on a condition in the ON clause. MERGE INTO <table1> USING <select from table2 subquery> ON <join condition for table1 and table2> WHEN MATCHED THEN <update clause> WHEN NOT MATCHED THEN <insert clause>;
- 15. Non-SQL performance issues There are a number of other performance inhibitors not related to the SQL statement: Hardware performance: e.g. CPU speed on one host may be slower than another. Non-cached data (performance is faster after the first execution) restart of database. small data buffer cache. e.g. execution plan cost is not proportional to elapsed time. Database parameter differences between environments. Undo storage: killing a session reads rollback information. data consistency.
- 16. Non-SQL performance issues Row locks (lock on a row in a table). Memory latching (lock on a block in memory). more overhead -> more CPU utilisation. Object inefficiency: non-deallocation of extents, index inefficiency. Sort spills over from memory to disk: Global Memory Bound will vary if other sessions are sorting/hashing. Temporary tablespace may need to extend. CPU contention between environments on the same host: riskr, ntst, ntsd, ndr, gtsd, gtst, gtsu, gtsx, ntsrt, ntsrx, ntsm Forced redo logging to nullify the Append hint on inserting. e.g. NTS, not NTSRP.
- 17. Non-SQL performance issues - UNDO Every time data is changed a representation of the old data is stored in the UNDO tablespace (also referred to as rollback). Allows concurrency: reads don’t block writes, writes don’t block reads. Overhead required to reconstruct data to the past. Once the transaction is committed the representation of the old data is disregarded. If the transaction is explicitly or implicitly rolledback (killed or deadlocked) then the UNDO tablespace is read to reconstruct the old data. A long running transaction (e.g. large query, refresh of materialised view) requires lots of UNDO space. The UNDO functionality is unique to the Oracle database
- 18. Processing Serial processing (default mode) One serial process accessing a single object at a time. No contention for the same data. No blocking occurs because no session references the same data at the same time. Regarded as best practice – focus on performing efficiently via SQL tuning.
- 19. Processing Serial processing (default mode) One serial process accessing a single object at a time. No contention for the same data. No blocking occurs because no session references the same data at the same time. Regarded as best practice – focus on performing efficiently via SQL tuning. Parallel processing Multiple parallel processes accessing a single object. No contention for the same data because the work is divided among the processes. Parallel processing is not suitable for high concurrency applications. e.g. NETSDEV No blocking occurs because each parallel process references different data at the same time.
- 20. Processing Serial processing (default mode) One serial process accessing a single object at a time. No contention for the same data. No blocking occurs because no session references the same data at the same time. Regarded as best practice – focus on performing efficiently via SQL tuning. Parallel processing Multiple parallel processes accessing a single object. No contention for the same data because the work is divided among the processes. Parallel processing is not suitable for high concurrency applications. e.g. NETSDEV No blocking occurs because each parallel process references different data at the same time. Multi-session serial processing via the application Attempt to simulate parallel processing using serial processes. Used instead of SQL tuning? Each session needs data consistent to when execution started. More overhead. Blocking occurs because each session attempts to reference the same data at the same time. Therefore latch contention occurs = more overhead = more CPU utilisation. Regarded as not best practice but acceptable for one-off and incidental processing.
- 21. Processing Myth: High CPU utilisation always means lots of real work is being performed. Fact - Maybe, but more likely high CPU indicates high overhead due to spinning on a latch. Fact - Latch spinning results due to contention between sessions for the same block from an object. e.g. index root node.
- 22. Data migrations Distributed database links: Smaller the data size the better (but without metadata). Can’t make the data consistent to the time required (unless read-only). e.g. To obtain all the data at midday, start the migration before this time, but the consistency is from the start time.
- 23. Data migrations Distributed database links: Smaller the data size the better (but without metadata). Can’t make the data consistent to the time required (unless read-only). e.g. To obtain all the data at midday, start the migration before this time, but the consistency is from the start time. Export and import utilities: Smaller the data size the better (with or without metadata). Can’t make the data consistent to the time required (unless read-only).
- 24. Data migrations Distributed database links: Smaller the data size the better (but without metadata). Can’t make the data consistent to the time required (unless read-only). e.g. To obtain all the data at midday, start the migration before this time, but the consistency is from the start time. Export and import utilities: Smaller the data size the better (with or without metadata). Can’t make the data consistent to the time required (unless read-only). RMAN duplicates: Small or large data sizes with metadata. Data consistent from a desired time – doesn’t need to be read-only. e.g. month-end snapshot. Example All the data may be required at mid-day today, but could be consistent from mid-day, yesterday, or last week. The production data is still available for reading and writing.
- 25. Use column constraints. e.g. primary key, foreign key, not null, etc. Statistics are not the “silver bullet” for all performance problems. Prefer to process jobs serially for the same table. Consult the DBAs Summary:
- 26. Analytic functions? Partitions and pruning? More fundamental SQL tuning? More subqueries? Using export & import utilities? Target a smaller audience? Currently accepting requests Possible topics for future forums: