SlideShare a Scribd company logo
UNDERSTANDING INDICES
Richard Douglas
• Sales Engineer, SQL Sentry
• Social Media: @SQLRich, LinkedIn,
• Blog Info: http://blogs.sqlsentry.com/author/RichardDouglas/;
http://SQL.RichardDouglas.co.uk
Tuning blog: http://www.sqlperformance.com/
E-mail ebooks@sqlsentry.com for free copies of our $10 e-books:
“CAN’TYOUJUST PUT AN
INDEXON IT?”
FIXING THE PROBLEM
WHAT ABOUT EUROPE?
YOU NEED THE BIG PICTURE
It’s all about DWI knowledge:
INDEXING STRATEGIES
D W I
a
t
a
o
r
k
l
o
a
d
n
t
e
r
n
a
l
s
YOUR HOST
• Richard Douglas
• Sales Engineer at SQL Sentry
• Blog:
http://SQL.RichardDouglas.co.uk
• Twitter: @SQLRich
• Email: RDouglas@SQLSentry.com
• Slides:
http://www.Slideshare.net/SQLRich
AGENDA
• Pre requisites
• Understanding indexes and their architecture
• Utilising SQL Server DMO’s
• Index golden rules
• The importance of maintenance
Understanding the SQL Server Data Page
PRE REQUISITES
• A data page is 8KB (8192
bytes)
• Page header is 96 bytes
leaving 8096 bytes for data
• Slot array stores the logical
order
Page header
1 - Cooper
1
2 - Hofstadter
2
3 - Koothrappali
3
4 - Kripke
54
5 - Wolowitz
5
INDEX ARCHITECTURE
Heaps
Clustered Indexes
Nonclustered Indexes
Nonclustered Columnstore Indexes
HEAP STRUCTURE
CLUSTERED INDEX STRUCTURE
INDEX EXAMPLE
1 Douglas
2 Kline
3 Bertrand
4 Martin
5 White
6 Fallen
7 Edwards
8 Ozar
9 Randal
10 Trip
11 Cunningham
12 Woody
13 La Rock
14 Locke
6993 Saxby
6994 Fryer
6995 Sabin
6996 Langford
6997 Bolton
6998 Fairweather
6999 Whittles
1 ,1, 100
8, 1, 101
You
Get
The
Idea
File 1 Page 100 File 1 Page 101 File 1 Page 404 File 1 Page 911 File 1 Page 999
Nothing to
see here,
move along
please
Please help
my query
performance
is poor
1 ,1, 100
Other
records
6993, 1, 999
6993, 1, 999
dbo.Customer
Clustered primary
key on an identity.
Lowest
key
NONCLUSTERED INDEX STRUCTURE
COLUMNSTORE PAGES
INDEX OPTIONS
Fill factor
Pad Index
Filtered index (WHERE)
Include
Data Compression
Online
HIDDEN DANGERS
Versioning
Uniqifiers
VARCHAR
NULLS
INTRODUCING INDEX DMO’S
• Admin
o Sys.dm_db_index_physical_stats
• Usage
o Sys.dm_db_index_usage_stats
o Sys.dm_db_index_operational_stats
• Missing
o Sys.dm_db_missing_index_details
o Sys.dm_db_missing_index_columns
o Sys.dm_db_missing_index_groups
o Sys.dm_db_missing_index_group_stats
MISSING INDEX LIMITATIONS
• Not intended to fine tune an indexing configuration.
• Only gathers 500 missing index groups.
• Optimal column orders for the index are not specified.
• In some scenarios it has been known to only return information on included columns.
• Returns only raw information about columns.
• Does not suggest filtered indexes.
• It can return different costs for the same missing index group that appears multiple times in XML Showplans.
• Multiple similar indexes may be returned.
• Only considers non-trivial query plans.
Bad example; Surname, Firstname, Middle Initial
GOLDEN RULES FOR CLUSTERED INDEXES
• Narrow
• Static
• Progressive
• Unique
• Fixed width
• Not Null
Impacts nonclustered indexes
Fragmentation
Space impact
This example could have 13 bytes of overhead alone;
• 4 byte uniquifier
• 2 byte variable offset + 6 bytes for variable length fields
• 1 byte for NULL values and NULL bitmap
GOLDEN RULES FOR NONCLUSTERED INDEXES
• Have an optimal clustered key
• Not narrow
• Reduce unnecessary overhead
o Fixed Width
o Not null
• Consolidate
• Index foreign keys
Page split scenario
THE IMPORTANCE OF MAINTENANCE
Page header
Data Row 1
Data Row 2
Data Row 3
Data Row 4
321
Data Row 5
5
Page header
Data Row 4
Data Row 5
54
Page header
Data Row 6
Data Row 7
Data Row 8
Data Row 10
109876
Data Row 9
Page pointers
Two main types
FRAGMENTATION
• Logical / External
“When the next logical page is
not the next physical page”
o Significantly reduces scan
performance
o Can cause extra IO
• Physical / Internal
“When page density is sub
optimal”
o Increased storage space
o More room required in the buffer pool
o More IO’s to read in the data
o More logging, bigger backups
FIXING FRAGMENTATION
• Rebuild Indexes
• Reorganize Indexes
• Options
o Maintenance plan
o Custom/community scripts
o Third party tools
• Guideline(sys.dm_db_Index_Physical_Stats provides these figures)
o 0 – 5% No action
o 5 – 30% Reorganize
o 30% + Rebuild
MAINTENANCE BEST PRACTICES
• Auto Close
• Auto Shrink
• Statistics
• Recovery models
If you value your job, don’t
turn the top two on.
SUMMARY
Architecture
Options
Querying
Design
Fragmentation
Maintenance
Understanding indices
THANK YOU!
• Slides will be available at http://Slideshare.net/SQLRich
• More information at:
o SQLSkills, et al
• E-mail ebooks@sqlsentry.com for free copies of our e-books:
o Just tell them where you met me
• My contact info for other questions:
o RDouglas@SQLSentry.com
o Twitter: @SQLRich
o Blogs: http://SQL.RichardDouglas.co.uk /
http://http://blogs.sqlsentry.com/author/RichardDouglas

More Related Content

PPTX
Wait Watchers ; Gain SQL Performance Increases Fast!
PPTX
Natural Born Killers, Performance issues to avoid
PPTX
Investigate TempDB Like Sherlock Holmes
PPTX
Understanding the Transaction Log, Your Key to Unlocking Greater Throughput
PPTX
Who wants to be a DBA? Roles and Responsibilities
PPTX
Reduce latency and boost sql server io performance
PPTX
Ultimate Free SQL Server Toolkit
PPTX
Products.intro.forum version
Wait Watchers ; Gain SQL Performance Increases Fast!
Natural Born Killers, Performance issues to avoid
Investigate TempDB Like Sherlock Holmes
Understanding the Transaction Log, Your Key to Unlocking Greater Throughput
Who wants to be a DBA? Roles and Responsibilities
Reduce latency and boost sql server io performance
Ultimate Free SQL Server Toolkit
Products.intro.forum version

What's hot (20)

PPTX
Diving into sql server 2016
PDF
KoprowskiT - SQLBITS X - 2am a disaster just began
PPTX
Ten query tuning techniques every SQL Server programmer should know
PPTX
SQL Server & SQL Azure Temporal Tables - V2
PPTX
SQL Server 2016 Temporal Tables
PPTX
SSIS Monitoring Deep Dive
PPTX
LVOUG meetup #2 - Forcing SQL Execution Plan Instability
PPTX
Enhancements that will make your sql database roar sp1 edition sql bits 2017
PPTX
SQL Server It Just Runs Faster
PDF
Splitgraph: AHL talk
PPTX
Dynamic SQL: How to Build Fast Multi-Parameter Stored Procedures
PPTX
SQL Explore 2012: P&T Part 3
PDF
PLSSUG - Troubleshoot SQL Server performance problems like a Microsoft Engineer
PPTX
Sql server 2016 it just runs faster sql bits 2017 edition
PPTX
How to Make SQL Server Go Faster
PPTX
Sq lite presentation
PPTX
Azure SQL & SQL Server 2016 JSON
PDF
SQL in the Hybrid World
PPTX
Experience sql server on l inux and docker
PPTX
How to Think Like the SQL Server Engine
Diving into sql server 2016
KoprowskiT - SQLBITS X - 2am a disaster just began
Ten query tuning techniques every SQL Server programmer should know
SQL Server & SQL Azure Temporal Tables - V2
SQL Server 2016 Temporal Tables
SSIS Monitoring Deep Dive
LVOUG meetup #2 - Forcing SQL Execution Plan Instability
Enhancements that will make your sql database roar sp1 edition sql bits 2017
SQL Server It Just Runs Faster
Splitgraph: AHL talk
Dynamic SQL: How to Build Fast Multi-Parameter Stored Procedures
SQL Explore 2012: P&T Part 3
PLSSUG - Troubleshoot SQL Server performance problems like a Microsoft Engineer
Sql server 2016 it just runs faster sql bits 2017 edition
How to Make SQL Server Go Faster
Sq lite presentation
Azure SQL & SQL Server 2016 JSON
SQL in the Hybrid World
Experience sql server on l inux and docker
How to Think Like the SQL Server Engine
Ad

Viewers also liked (14)

PDF
Sql Saturday 111 Atlanta applied enterprise semantic mining
PDF
Applied Semantic Search with Microsoft SQL Server
PDF
Secrets of Enterprise Data Mining 201310
PPTX
SQL Server - Full text search
PPTX
FileTable and Semantic Search in SQL Server 2012
KEY
Sql 2012 development and programming
PPT
Effective Usage of SQL Server 2005 Database Mirroring
PDF
SQL Server Performance Tuning Baseline
PPT
Sql Server Performance Tuning
PDF
SQL Server - Querying and Managing XML Data
PPTX
Always on in SQL Server 2012
PPT
File Upload
PPTX
What's new in SQL Server 2016
PPTX
Implementing Full Text in SQL Server
Sql Saturday 111 Atlanta applied enterprise semantic mining
Applied Semantic Search with Microsoft SQL Server
Secrets of Enterprise Data Mining 201310
SQL Server - Full text search
FileTable and Semantic Search in SQL Server 2012
Sql 2012 development and programming
Effective Usage of SQL Server 2005 Database Mirroring
SQL Server Performance Tuning Baseline
Sql Server Performance Tuning
SQL Server - Querying and Managing XML Data
Always on in SQL Server 2012
File Upload
What's new in SQL Server 2016
Implementing Full Text in SQL Server
Ad

Similar to Understanding indices (20)

PDF
Building better SQL Server Databases
PPTX
Top 10 tips for Oracle performance (Updated April 2015)
PDF
Scylla Summit 2022: Migrating SQL Schemas for ScyllaDB: Data Modeling Best Pr...
PPTX
Data+Modelling.pptx
PPTX
Data+Modelling.pptx
PDF
MongoDB Europe 2016 - The Rise of the Data Lake
PPTX
SQL Server 2019 Master Data Service
PPTX
SQL Server Tips & Tricks
PPTX
Government and Education Webinar: SQL Server—Indexing for Performance
PPT
Deep Dive: Oracle WebCenter Content Tips and Traps!
PPTX
Webinar: Scaling MongoDB
PDF
3 DATABASE MANAGEMENT SYSTEMS DESIGN PRINCIPLES.pdf
PPTX
How to Survive as a Data Architect in a Polyglot Database World
PPT
Data Warehouse Logical Design using Mysql
PPTX
Index the obvious and not so obvious
PDF
Statistics and Indexes Internals
PPTX
SQL To NoSQL - Top 6 Questions Before Making The Move
PDF
Framing the Argument: How to Scale Faster with NoSQL
PPTX
7 Database Mistakes YOU Are Making -- Linuxfest Northwest 2019
PDF
10 things to avoid in data model 09242010
Building better SQL Server Databases
Top 10 tips for Oracle performance (Updated April 2015)
Scylla Summit 2022: Migrating SQL Schemas for ScyllaDB: Data Modeling Best Pr...
Data+Modelling.pptx
Data+Modelling.pptx
MongoDB Europe 2016 - The Rise of the Data Lake
SQL Server 2019 Master Data Service
SQL Server Tips & Tricks
Government and Education Webinar: SQL Server—Indexing for Performance
Deep Dive: Oracle WebCenter Content Tips and Traps!
Webinar: Scaling MongoDB
3 DATABASE MANAGEMENT SYSTEMS DESIGN PRINCIPLES.pdf
How to Survive as a Data Architect in a Polyglot Database World
Data Warehouse Logical Design using Mysql
Index the obvious and not so obvious
Statistics and Indexes Internals
SQL To NoSQL - Top 6 Questions Before Making The Move
Framing the Argument: How to Scale Faster with NoSQL
7 Database Mistakes YOU Are Making -- Linuxfest Northwest 2019
10 things to avoid in data model 09242010

Recently uploaded (20)

PPTX
Cloud computing and distributed systems.
PDF
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
PDF
Diabetes mellitus diagnosis method based random forest with bat algorithm
DOCX
The AUB Centre for AI in Media Proposal.docx
PDF
Spectral efficient network and resource selection model in 5G networks
PDF
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
PPTX
Digital-Transformation-Roadmap-for-Companies.pptx
PDF
NewMind AI Weekly Chronicles - August'25 Week I
PDF
cuic standard and advanced reporting.pdf
PDF
Encapsulation_ Review paper, used for researhc scholars
PDF
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
PDF
Bridging biosciences and deep learning for revolutionary discoveries: a compr...
PPTX
Big Data Technologies - Introduction.pptx
PDF
Reach Out and Touch Someone: Haptics and Empathic Computing
PDF
Review of recent advances in non-invasive hemoglobin estimation
PDF
Empathic Computing: Creating Shared Understanding
PPTX
PA Analog/Digital System: The Backbone of Modern Surveillance and Communication
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
PDF
Mobile App Security Testing_ A Comprehensive Guide.pdf
PPTX
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
Cloud computing and distributed systems.
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
Diabetes mellitus diagnosis method based random forest with bat algorithm
The AUB Centre for AI in Media Proposal.docx
Spectral efficient network and resource selection model in 5G networks
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
Digital-Transformation-Roadmap-for-Companies.pptx
NewMind AI Weekly Chronicles - August'25 Week I
cuic standard and advanced reporting.pdf
Encapsulation_ Review paper, used for researhc scholars
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
Bridging biosciences and deep learning for revolutionary discoveries: a compr...
Big Data Technologies - Introduction.pptx
Reach Out and Touch Someone: Haptics and Empathic Computing
Review of recent advances in non-invasive hemoglobin estimation
Empathic Computing: Creating Shared Understanding
PA Analog/Digital System: The Backbone of Modern Surveillance and Communication
Dropbox Q2 2025 Financial Results & Investor Presentation
Mobile App Security Testing_ A Comprehensive Guide.pdf
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...

Understanding indices

  • 1. UNDERSTANDING INDICES Richard Douglas • Sales Engineer, SQL Sentry • Social Media: @SQLRich, LinkedIn, • Blog Info: http://blogs.sqlsentry.com/author/RichardDouglas/; http://SQL.RichardDouglas.co.uk
  • 2. Tuning blog: http://www.sqlperformance.com/ E-mail ebooks@sqlsentry.com for free copies of our $10 e-books:
  • 6. YOU NEED THE BIG PICTURE
  • 7. It’s all about DWI knowledge: INDEXING STRATEGIES D W I a t a o r k l o a d n t e r n a l s
  • 8. YOUR HOST • Richard Douglas • Sales Engineer at SQL Sentry • Blog: http://SQL.RichardDouglas.co.uk • Twitter: @SQLRich • Email: RDouglas@SQLSentry.com • Slides: http://www.Slideshare.net/SQLRich
  • 9. AGENDA • Pre requisites • Understanding indexes and their architecture • Utilising SQL Server DMO’s • Index golden rules • The importance of maintenance
  • 10. Understanding the SQL Server Data Page PRE REQUISITES • A data page is 8KB (8192 bytes) • Page header is 96 bytes leaving 8096 bytes for data • Slot array stores the logical order Page header 1 - Cooper 1 2 - Hofstadter 2 3 - Koothrappali 3 4 - Kripke 54 5 - Wolowitz 5
  • 11. INDEX ARCHITECTURE Heaps Clustered Indexes Nonclustered Indexes Nonclustered Columnstore Indexes
  • 14. INDEX EXAMPLE 1 Douglas 2 Kline 3 Bertrand 4 Martin 5 White 6 Fallen 7 Edwards 8 Ozar 9 Randal 10 Trip 11 Cunningham 12 Woody 13 La Rock 14 Locke 6993 Saxby 6994 Fryer 6995 Sabin 6996 Langford 6997 Bolton 6998 Fairweather 6999 Whittles 1 ,1, 100 8, 1, 101 You Get The Idea File 1 Page 100 File 1 Page 101 File 1 Page 404 File 1 Page 911 File 1 Page 999 Nothing to see here, move along please Please help my query performance is poor 1 ,1, 100 Other records 6993, 1, 999 6993, 1, 999 dbo.Customer Clustered primary key on an identity. Lowest key
  • 17. INDEX OPTIONS Fill factor Pad Index Filtered index (WHERE) Include Data Compression Online
  • 19. INTRODUCING INDEX DMO’S • Admin o Sys.dm_db_index_physical_stats • Usage o Sys.dm_db_index_usage_stats o Sys.dm_db_index_operational_stats • Missing o Sys.dm_db_missing_index_details o Sys.dm_db_missing_index_columns o Sys.dm_db_missing_index_groups o Sys.dm_db_missing_index_group_stats
  • 20. MISSING INDEX LIMITATIONS • Not intended to fine tune an indexing configuration. • Only gathers 500 missing index groups. • Optimal column orders for the index are not specified. • In some scenarios it has been known to only return information on included columns. • Returns only raw information about columns. • Does not suggest filtered indexes. • It can return different costs for the same missing index group that appears multiple times in XML Showplans. • Multiple similar indexes may be returned. • Only considers non-trivial query plans.
  • 21. Bad example; Surname, Firstname, Middle Initial GOLDEN RULES FOR CLUSTERED INDEXES • Narrow • Static • Progressive • Unique • Fixed width • Not Null Impacts nonclustered indexes Fragmentation Space impact This example could have 13 bytes of overhead alone; • 4 byte uniquifier • 2 byte variable offset + 6 bytes for variable length fields • 1 byte for NULL values and NULL bitmap
  • 22. GOLDEN RULES FOR NONCLUSTERED INDEXES • Have an optimal clustered key • Not narrow • Reduce unnecessary overhead o Fixed Width o Not null • Consolidate • Index foreign keys
  • 23. Page split scenario THE IMPORTANCE OF MAINTENANCE Page header Data Row 1 Data Row 2 Data Row 3 Data Row 4 321 Data Row 5 5 Page header Data Row 4 Data Row 5 54 Page header Data Row 6 Data Row 7 Data Row 8 Data Row 10 109876 Data Row 9 Page pointers
  • 24. Two main types FRAGMENTATION • Logical / External “When the next logical page is not the next physical page” o Significantly reduces scan performance o Can cause extra IO • Physical / Internal “When page density is sub optimal” o Increased storage space o More room required in the buffer pool o More IO’s to read in the data o More logging, bigger backups
  • 25. FIXING FRAGMENTATION • Rebuild Indexes • Reorganize Indexes • Options o Maintenance plan o Custom/community scripts o Third party tools • Guideline(sys.dm_db_Index_Physical_Stats provides these figures) o 0 – 5% No action o 5 – 30% Reorganize o 30% + Rebuild
  • 26. MAINTENANCE BEST PRACTICES • Auto Close • Auto Shrink • Statistics • Recovery models If you value your job, don’t turn the top two on.
  • 29. THANK YOU! • Slides will be available at http://Slideshare.net/SQLRich • More information at: o SQLSkills, et al • E-mail ebooks@sqlsentry.com for free copies of our e-books: o Just tell them where you met me • My contact info for other questions: o RDouglas@SQLSentry.com o Twitter: @SQLRich o Blogs: http://SQL.RichardDouglas.co.uk / http://http://blogs.sqlsentry.com/author/RichardDouglas