Performance By Design
Download as PPTX, PDF5 likes632 views
This document discusses principles for designing systems and databases for high performance. It emphasizes that: 1. Performance requirements should be specified upfront and the architecture should support meeting those requirements. 2. The database design impacts performance, so logical and physical designs like normalization, indexing, partitioning, and caching should be considered. 3. Application architecture also affects performance through techniques like minimizing network traffic and SQL parsing overhead.
Performance By Design
- 1. Performance by DesignGuy HarrisonDirector, R&D Melbournewww.guyharrison.net
- 4. Save the red-shirt Toad!The Red-shirt Toad is NOT expendable!
- 5. Core messageDesign limits performanceArchitecture maps requirements to designMake sure performance requirements are specifiedMake sure architecture allows for performanceMake sure performance requirements are realized
- 6. Elements of Performance by Design
- 7. Methodology
- 8. High performance can mean different thingsSpeed: response time
- 11. Not usually easy to change architectures
- 12. Poorly defined requirements lead to this:
- 14. Twitter growth
- 15. “Twitter is, fundamentally, a messaging system. Twitter was not architected as a messaging system, however. For expediency's sake, Twitter was built with technologies and practices that are more appropriate to a content management system.”
- 16. Patterns of database performanceHard to distinguish patterns at low levels
- 17. Validating performance can’t wait...User adoption and growthUI Layer (HTML, JavaScript, Ajax) Middleware layer (J2EE)SQLsDatabase (Tables, views, partitions, etc)
- 19. Database Design
- 20. Normalize, but not too far!
- 21. Other logical design thoughtsArtificial keysGenerally more efficient than long composite keysNull valuesNot a good idea if you intend to search for “unknown” or “incomplete” valuesNull should not mean somethingBut beneficial as long as you don’t need to look for them. Data typesConstraints on precision can sometimes reduce row lengthsVariable length strings usually betterCarefully consider CLOBs vs long VARCHARs
- 22. Logical to Physical: Subtypes“Customers are people too”
- 23. Indexing, clustering and weird table typesLots’ of options:B*-Tree indexBitmap indexHash clusterIndex ClusterNested tableIndex Organized TableMost often useful:B*-Tree (concatenated) indexesBitmap indexesHash Clusters
- 25. Concatenated index effectivenessSELECT cust_idFROM sh.customers cWHERE cust_first_name = 'Connor'AND cust_last_name = 'Bishop'AND cust_year_of_birth = 1976;
- 26. Concatenated indexing guidleinesCreate a concatenated index for columns from a table that appear together in the WHERE clause.If columns sometimes appear on their own in a WHERE clause, place them at the start of the index.The more selective a column is, the more useful it will be at the leading end of the index (better single key lookups)But indexes compress better when the leading columns are less selective. (better scans) Index skip scans can make use of an index even if the leading columns are not specified, but it’s a poor second choice to a “normal” index range scan.
- 27. Bitmap indexes
- 28. Bitmap indexes
- 30. Bitmap join performance SELECT SUM (amount_sold)FROM customers JOIN sales s USING (cust_id) WHERE cust_email='flint.jeffreys@company2.com';
- 31. Index overhead
- 32. Hash ClusterCluster key determines physical location on diskSingle IO lookup by cluster keyMisconfiguration leads to overflow or sparse tables
- 33. Hash Cluster vs B-tree index
- 34. Hash cluster table scan
- 35. Denormalization and partitioningRepeating groups – VARRAYS, nested tablesSummary tables – Materialized Views, Result cacheHorizontal partitioning – Oracle Partition Option In-line aggregations – Dimensions Derived columns – Virtual columnsVertical partitioning Replicated columns - triggers
- 36. Summary tablesAggregate queries on big tables often the most expensivePre-computing them makes a lot of senseBalance accuracy with overhead Aggregate QueryMV on COMMITManual SummaryResult set cacheMV stale tolerated AccuracyEfficiency
- 38. Physical storage optionsLOB StoragePCTFREECompression Block size Partitioning
- 40. Application Architecture and implementation
- 41. The best SQL is no SQL Avoid asking for the same data twice.
- 42. 11g client side cache CLIENT_RESULT_CACHE_SIZE: this is the amount of memory each client program will dedicate to the cache.Use RESULT_CACHE hint or (11GR2) table propertyOptionally set the CLIENT_RESULT_CACHE_LAG
- 43. Parse overheadIt’s easy enough in most programming languages to create a unique SQL for every query:
- 44. Bind variables are preferred
- 46. Identifying similar SQLsSee force_matching.sql at www.guyharrison.net
- 47. Transaction design Optimistic vs. Pessimistic
- 48. Using ORA_ROWSCNSetting ROWDEPENDENCIES will reduce false fails
- 49. Network – stored procedures
- 51. Array processing - Fetch
- 52. Network overhead – Array processing
- 55. Array Insert – PL/SQL
- 56. Thank you
- 57. Geek quiz stuff:High probability answers (keep standing if):Know what Alice and Wally have in commonYou know the next number in this series 3 . 1 4 Know what “M” is in E=MC2
- 58. Know (or can work out) your age in hexHave an opinion about of ST vs SW If you know who Leonard McCoy is Think there is an important distinction between Nerd and Geek Can quote Monty Python …. Other than dead parrot?You’ve ever watched Jerry Springer
- 59. There are more networked devices in your house than people, pets and carsKnow the names of two of Thomas the tank engines friendsKnow the names of any of Angelina and Brad’s babiesLow probability answers: (sit down if you):Have a twitter account # Azure is your new favourite color
- 60. You’ve ever played ZorkYou have a favourite Dr Who companion Your favourite is Sarah Jane Know your age in binary (or can work it out in your head) You are proficient in some form of assembler
- 61. # You are proficient in some for or English There is a rubicks cube in your house Have your own domainHave ever been to AzerothWho is Know who said “Dude I am not your nemesis”
- 62. Worn a star trek or star wars costumePlayed a game that uses a non-six sided diceGet email on my phone – before getting out of bedCalculator watchBinary time piece Was on the internet prior to the WWW
- 63. # Met my current partner on lineKnow the next thing in this sequence: Hydrogen, Helium, Lithuim, Berilium, ….Know what a Gigaquads in a megaquad is
- 64. Saw a sci-fi movie more than twice at the movies=========================================================You cleaned up at home before going to work
Editor's Notes
- #6: TelsavsMasseratiLatency vs throughputVolume?Economics?
- #7: MethodsRequirementsMeasurementPrototypeBenchmarkData modelOptimize physical model to queryDenormalizeIndexing, clustering, partitionApplicationMinimize database accessOptimize database access
- #9: http://www.steveschmidtracing.com/our-team-5.html
- #12: Generally not easy to change architectures.....
- #21: Normal form is the best starting point for an efficient database design. However, don’t go overboard in eliminating redundancy. A normalized data model is one in which any data redundancy has been eliminatedand in which data and relationships are uniquely identifiable by primaryand foreign keys. Although the normalized data model is rarely the final destinationfrom a performance point of view, the normalized data model is almost alwaysthe best starting point. Indeed, failing to normalize your logical model isfrequently a recipe for a poorly performing physical model.Relational theory provides for various levels of normal form, some of whichare of academic interest only. Third normal form is the most commonly adoptednormalization level, and it has the following characteristics:❏ All data in an entity (table) is dependent on the primary key.❏ There should be no repeating groups of attributes (columns).❏ No data in an entity is dependent on only part of the key.❏ No data in an entity is dependent on any nonkey attribute.These characteristics are often remembered through the adage “the key, thewhole key, and nothing but the key.”
- #23: Subtypes categorize or partition a logical entity and help to classify thetypes of information that is within the entity. A subtype usually has a set of attributesthat are held in common with the parent entity (the super-type) andother attributes that are not shared with the super-type or other subtypes. Figure4-1 shows how a PEOPLE entity could be split into subtypes of CUSTOMERSand EMPLOYEES.When translating entity subtypes into tables, we have the following options:❏ Create tables for the super-type and for each subtype. The super-type tablecontains only columns that are common to both subtypes.❏ Create a table for the super-type only. Attributes from all subtypes becomecolumns in this super-table. Typically, columns from subtype attributes willbe nullable, and a category column indicates the subtype in which a row belongs.❏ Create separate tables for each subtype without creating a table for thesuper-type. Attributes from the super-type are duplicated in each table.The three solutions result in very different performance outcomes. In particular,creating tables for the super-type and each subtype is likely to reduce performancein most circumstances, except where only the super-type is subject to afull table scan. Table 4-1 compares the performance of each of the three solutionsfor common database operations.
- #50: Remember, you don’t always have control over the network – in particular client side code may sometimes be located anywhere
- #51: The further the code is (in network terms) from the database, the more the network effects will magnify. You can’t get any closer to the database than being inde the database as PL/SQL