Clustered Columnstore Introduction
3 likes1,158 views
The document provides an introduction to clustered columnstore indexes, their principles, history, and development in SQL Server from 2012 to 2014. It highlights the differences between rowstore and columnstore, the benefits of batch mode processing, and the limitations of these technologies. It also mentions significant improvements in SQL Server 2014, such as the introduction of clustered columnstores, enhanced batch mode capabilities, and better data management features.
Clustered Columnstore Introduction
- 1. Clustered Columnstore - Introduction Niko Neugebauer
- 2. Sponsors
- 3. Niko Neugebauer Microsoft Data Platform Professional OH22 (http://www.oh22.net) 15+ years in IT SQL Server MVP Founder of 3 Portuguese PASS Chapters Blog: http://www.nikoport.com Twitter: @NikoNeugebauer LinkedIn: http://pt.linkedin.com/in/webcaravela Email: info@webcaravela.com
- 4. Explore Everything PASS Has to Offer Free SQL Server and BI Web Events Free 1-day Training Events Regional Event Local User Groups Around the World Free Online Technical Training This is Community Business Analytics Training Session Recordings PASS Newsletter
- 5. Today’s program: Principles of the Columnstore Indexes & Batch Mode History Columnstore in SQL Server 2012 Clustered Columnstore in SQL Server 2014 Batch Mode improvements for SQL Server 2014
- 6. Principles of the Columnstore Indexes: Using a-kind-of-in-memory™ engine X-Velocity aka Vertipaq (PowerPivot, SSAS Tabular) Verytically separated, extremely compressed & tuned for processing large volumes of data
- 7. Principles of the Columnstore Indexes topics: Clustered & Nonclustered Indexes RowStore vs ColumnStore Columnstore Index creation sequence Not-suitable for OLTP, unless you are processing whole table and not altering data
- 9. RowStore vs ColumnStore Row Store Index: 1, Jody, Philips, 43.04; 2, Mark, Johnson, 37.08; … ID First Name Last Name Salary 1 Jody Philips 43.03 2 Mark Johnson 37.08 3 Matt Markensen 16.81 4 Gail Lindberg 24.90 Column Store Index: 1, 2, 3, 4; Jody, Mark, Matt, Gail; …
- 11. Columnstore Index creation sequence: 1. Run-length compression 2. Dictionary enconding 3. Binary compression
- 13. Dictionary enconding ID Mark Andre John Mark John Andre John Mark ID Mark 1 Andre 2 John 3 ID 1 2 3 1 3 2 3 1
- 14. Binary Compression Super-secret compression turning data into LOB’s. Vertipac aka xVelocity
- 15. Row Groups ~ 1.000.000 Rows are grouped in Segments, called Row Groups Data is Encoded & Compressed Data is Unsorted Stored as LOB’s When executed in batch mode the Segments are eliminated
- 16. Principles of the Batch mode: Vector processing 1 row vs ~1000 rows Progammers, its like passing Array vs 1 param Think about it as if it would be a Factory processing vs Manual Processing (19th vs 18th Century)
- 17. History of the Columnstore Concept exists since 1969 (or since the beginning of Database era) Sybase IQ (since 1990s) PDW v1 had the first implementation for Microsoft Databases SQL Server 2012 introduced Columnstore Besides SQL Server there are Vertica, C-Store, Volt, Terradata, … Michael Stonebraker’s vision – In 10 years there will be virtually no Rowstore for Datawarehousing
- 18. Columnstore in SQL Server 2012 Only Nonclustered (but supporting multiple indedexs) Non-Updateable (and no schema-modification) Very limited data types (Precision 18 and no LOB’s) Batch mode functions only in certain conditions (no OUTER JOIN, UNION ALL, EXIST, IN, Scalar Aggregates, Distinct Aggregates)
- 19. Columnstore in SQL Server 2014 Clustered & Nonclustered Clustered Columnstore are updateable & Schema can be modified Large range of data types (except LOB) Better Memory estimation & consumption Batch mode is significantly improved NEW! Archival Compression
- 20. Columnstore in SQL Server 2014 Delta-Stores (open & close) Deleted Bitmap Delete & Update work as a DELETE + INSERT Schema modification Constraints are supported (Non-Unique)
- 21. Important Limitations in SQL Server 2014 No NonClustered Indexes for Clustered Columnstore No ONLINE operations Not every operator in Execution plans is supporting Batch mode.
- 22. Batch Mode Improvements in SQL Server 2014 Mixed Mode – Row & Batch mode can co-exist OUTER JOIN, UNION ALL, EXIST, IN, Scalar Aggregates, Distinct Aggregates Some TempDB operations for Columnstore Indexes are running in Batch mode.
- 23. Demo Time
- 24. Links • http://www.nikoport.com/2013/07/05/clustered- columnstore-indexes-part-1-intro/ – My series of blog posts on Clustered Columnstore Indexes • http://rusanu.com/2013/06/11/sql-server-clustered- columnstore-indexes-at-teched-2013/ – Remus Rusanu Introduction for Clustered Columnstore • http://research.microsoft.com/pubs/193599/Apollo3%20 -%20Sigmod%202013%20-%20final.pdf – White Paper on the Clustered Columnstore
- 26. Sponsors