SQL Server 2008 Fast Track Data Warehouse
3 likes903 views
The presentation discusses SQL Server 2008 Fast Track Data Warehouse 2.0, highlighting the differences between Data Warehousing (DW) and Online Transaction Processing (OLTP) systems. It emphasizes the need for balanced architecture to optimize performance by aligning server and storage configurations, focusing on scan-centric workloads. Fast Track provides reference architectures and best practices to accelerate data warehouse deployments, aiming for lower total cost of ownership (TCO) and improved performance using standard hardware.
SQL Server 2008 Fast Track Data Warehouse
- 1. SQL Server 2008 Fast Track Data Warehouse 2.0 Speaker: Phil Hummel of WinWire Technologies Presentation developed by: Bruce Campbell Western Region Data Warehouse Specialist, Microsoft Silicon Valley SQL Server User Group February 16, 2009 Mark Ginnebaugh, User Group Leader, mark@designmind.com
- 2. Agenda • DW vs. OLTP • Balanced Architecture Approach for DW Fast Track Defined • Fast Track Reference Architectures • Next Steps
- 3. Microsoft DW & BI Stack DELIVERY PerformancePoint Services END USER TOOLS BI & DW PLATFORM (RDBMS, ETL, OLAP, Reporting)
- 4. DW versus OLTP Database Database • Designed for analytical operations: • Designed for operational requirements: Strategic focus Tactical focus • Optimized for bulk load and large, complex, • Optimized for transactions: “single row” unpredictable queries entry and retrieval • Fewer concurrent users relative to OLTP • Thousands of concurrent users Storage • Primary focus on Read operations Storage • Optimized for disk scan over seek • Emphasizes transactional operations performance • Storage optimization focused on disk • Optimized for disk seek over scan scan rate (MB/s) operations • Storage optimization focused on I/O operations/s (IOPs)
- 5. Sequential I/O Sequential I/O Random I/O • Scans on large data stores are • OLTP usually random-read centric. usually read with sequential read Discrete lookups benefit from index patterns and not random read optimization and random read patterns capability. • Scalable, predictable performance • Not as predictable & scalable for data warehousing • Requires 1/3 or fewer drives to match server I/O consumption • Requires large number of drives to capability. match server I/O consumption capability. All databases contain both scans and seeks among with other types of reads and writes, DW workload indicate that the vast majority of reads are sequential – not all
- 6. Some SQL Data Warehouses today Big SAN Big 64-core Server Connected together What’s wrong with this picture?
- 7. Answer: system out of balance • This server can consume 16 GB/Sec of IO, but the SAN can only deliver 2 GB/Sec – Even when the SAN is dedicated to the SQL Data Warehouse, which it often isn’t – Lots of disks for Random IOPS BUT – Limited controllers Limited IO bandwidth • System is typically IO bound • Queries are slow Result: significant investment, not delivering performance
- 8. The Alternative: A Balanced System • Design a server + storage configuration that can deliver all the IO bandwidth that CPUs can consume when executing a SQL Relational DW workload • Avoid sharing storage devices among servers • Avoid overinvesting in disk drives – Focus on scan performance, not IOPS • Layout and manage data to maximize range scan performance and minimize fragmentation
- 9. Potential Performance Bottlenecks DISK DISK SQL SERVER CPU CORES A FC SWITCH FC SERVER WINDOWS A CACHE HBA B LUN CACHE A STORAGE A B CONTROLLER B DISK DISK FC A HBA B B LUN CPU Feed Rate SQL Server HBA Port Rate Switch Port Rate SP Port Rate LUN Read Rate Disk Feed Rate Read Ahead Rate
- 10. SQL Server Fast Track Data Warehouse Solution to help customers and partners accelerate their data warehouse deployments • A method for designing a cost-effective, balanced system for Data Warehouse workloads • Reference hardware configurations developed in conjunction with hardware partners using this method • Best practices for data layout, loading and management Relational Database Only – Not SSAS, IS, RS
- 11. Fast Track Data Warehouse Components Software: • SQL Server 2008 Enterprise • Windows Server 2008 Configuration guidelines: • Physical table structures • Indexes • Compression • SQL Server settings • Windows Server settings • Loading Hardware: • Tight specifications for servers, storage and networking • ‘Per core’ building block
- 12. Fast Track Scope Supporting Systems BI Data Storage Systems Presentation Layer Systems Integration Analysis Services Services ETL Cubes Presentation Data Presentation Data Web Analytic Tools Data Path Reporting Services SharePoint Services Dedicated SAN, Microsoft Office SharePoint Storage Array Data Warehouse PerformancePoint Data Staging, Excel Services Bulk Loading Reference Architecture Scope (dashed)
- 13. Two SQL DW Infrastructure Options: SQL Classic DW or Fast Track SQL DW SQL Classic DW Fast Track SQL DW Architecture Architecture Architecture modeled after DW Appliances Leverages Shared SAN Teradata, DATAllegro..etc “ Appliance Like” Uses Dedicated SAN arrays and Network Enterprise Shared Shared Network Dedicated SAN Storage Bandwidth Network Dedicated SAN Bandwidth SQL 2008 Data Warehouse SAN Arrays 1:4 cpu cores SMP Server 8 Data Disk / Array – 4 Raid 1 Pairs Simultaneous SQL Server Reads 2 Log and 1 Hot Spare EMC AX4 – HP MSA2312 IBM 3400 OLTP Applications SQL Fast Track DW supports “Scan Centric” DW workloads that are index light
- 14. Optimizing storage layout for scan intensive workloads • LUN configuration is based on RAID GP01 RAID GP02 RAID GP05 RAID1 pairs S P 01 02 03 04 09 10 – Optimal for scan type access patterns LUN1 LUN3 LUN0 • Striping across storage is A LUN2 LUN4 (Logs) HOT SPARE RAID GP03 RAID GP04 accomplished via SQL Server data S files P 05 06 07 08 LUN5 LUN7 • Observed throughput for a single B LUN6 LUN8 RAID pair >= 130 MB/s
- 15. Storage Layout Implications for SQL Server LUN 1 LUN 2 LUN 3 LUN16 Permanent FG Permanant_DB Permanent_1.ndf Permanent_2.ndf Permanent_3.ndf Permanent_16.ndf Stage FG Database Stage Stage_1.ndf Stage_2.ndf Stage_3.ndf Stage_16.ndf Local Drive 1 TempDB TempDB.mdf (25GB) TempDB_02.ndf (25GB) TempDB_03ndf (25GB) TempDB_16.ndf (25GB) Log LUN 1 Permanent DB Log Stage DB Log
- 16. Creating Sequential Data Layout • Goal: Align logical and physical ordering of data within a Filegroup • Two primary ways Fast Track optimizes allocation for Sequential Scan – Minimize Fragmentation – Manage Load processing
- 17. Maximum Consumption Rate Theoretical throughput for IO stack • Using a 2x quad-core server as a building Maximum theoretical throughput for IO stack components sized for an 8 CPU core Fast Track system block / starting point (assumes 200 MB/s per core) • Ensure that the per- core data consumption rate can be delivered by all 500 MB/s 300 MB/s 300 MB/s elements of the IO 300 MB/s stack MCR 1.6 GB/s 500 MB/s 300 MB/s Fiber Switch Storage Enclosure • Sticker on the new Windows Server OS car: “Miles Per 300 MB/s Gallon” CPU Socket HBA Min 500 MB/s 300 MB/s (4 Core) 300 MB/s 2 Min CPU GB/s 2 Socket HBA 500 MB/s 300 MB/s GB/s (4 Core) Server Storage Enclosure
- 18. Scaling the IO stack Storage Processor RAID-1 RAID-1 CPU CPU Fiber Storage Processor RAID-1 RAID-1 RAID-1 Socket Socket Storage Enclosure (4 Core) (4 Core) Switch Storage Processor RAID-1 CPU CPU RAID-1 RAID-1 Socket Socket Storage Processor RAID-1 (4 Core) (4 Core) RAID-1 Storage Enclosure CPU CPU Storage Processor RAID-1 RAID-1 Socket Socket RAID-1 (4 Core) (4 Core) Storage Processor RAID-1 RAID-1 Storage Enclosure CPU CPU Socket Socket Storage Processor RAID-1 (4 Core) (4 Core) RAID-1 RAID-1 Storage Processor RAID-1 RAID-1 Storage Enclosure HBA Storage Processor RAID-1 HBA RAID-1 RAID-1 RAID-1 Storage Processor RAID-1 Storage Enclosure HBA Storage Processor RAID-1 HBA RAID-1 RAID-1 Storage Processor RAID-1 RAID-1 Storage Enclosure HBA Storage Processor RAID-1 HBA RAID-1 RAID-1 Storage Processor RAID-1 RAID-1 Storage Enclosure HBA Server HBA Storage Processor RAID-1 RAID-1 RAID-1 Storage Processor RAID-1 RAID-1 Storage Enclosure
- 19. Fast Track Data Warehouse Reference Configurations CPU Initial Max Server CPU SAN Data Drive Count Cores Capacity* Capacity** HP Proliant (2) AMD Opteron Istanbul 12 (3) HP MSA2312fc (24) 300GB 15k SAS 6TB 12TB DL 385 G6 six core 2.6 GHz HP Proliant (2) Intel Xeon® 5500 Series 8 (2) HP MSA2312 (16) 300GB 15k SAS 4TB 8TB DL 380 G6 Quad core HP Proliant (4) AMD Opteron Instanbul 24 (6) HP MSA2312fc (48) 300GB 15k SAS 12TB 24TB DL 585 G6 six core 2.6 GHz HP Proliant (4) Intel Xeon® 7400 Series six 24 (6) HP MSA2312 (48) 300GB 15k SAS 12TB 24TB DL 580 G5 core HP Proliant (8) AMD Opteron Istanbul 48 (12) HP MSA2312 (96) 300GB 15k SAS 24TB 48TB DL 785 G6 six core 2.8 GHz Dell PowerEdge (2) Intel Xeon Nehalem quad 8 (2) EMC AX4 (16) 300GB 15k FC 4TB 8TB R710 core 2.66 GHz Dell Power Edge (4) Intel Xeon Dunnington 24 (6) EMC AX4 (48) 300GB 15k FC 12TB 24TB R900 six core 2.67GHz IBM X3650 M2 (2) Intel Xeon Nehalem quad 8 (2) IBM DS3400 (16) 200GB 15K FC 4TB 8TB core 2.67 GHx IBM X3850 M2 (4) Intel Xeon Dunnington six 24 (6) IBM DS3400 (24) 300GB 15k FC 12TB 24TB core 2.67 GHz IBM X3950 M2 (8) Intel Xeon Nehalem four 32 (8) IBM DS3400 (32) 300GB 15k SAS 16TB 32TB core 2.13 GHz Bull Novascale (2) Intel Xeon Nehalem quad 8 (2) EMC AX4 (16) 300GB 15k FC 4TB 8TB R460 E2 core 2.66 GHz Bull Novascale (4) Intel Xeon Dunnington 24 (6) EMC AX4 (48) 300GB 15k FC 12TB 24TB R480 E1 six core 2.67GHz * Core-balanced compressed capacity based on 300GB 15k SAS not including hot spares and log drives. Assumes 25% (of raw disk space) allocated for Temp DB. ** Represents storage array fully populated with 300GB15k SAS and use of 2.5:1 compression ratio. This includes the addition of one storage expansion tray per enclosure. 30% of this storage should be reserved for DBA operations
- 20. SQL Server Fast Track Data Warehouse 2.0 for HP – now on G6 Platform Five AMD and Intel based Reference configurations available for HP: AMD Based Reference Architectures 2 Processor Configuration – Server: HP ProLiant DL385 G6 with 2 6-core AMD Opteron CPUs – Storage server: MSA Storage – Scalability: 4 – 12 TB 4 Processor Configuration – Server: HP ProLiant DL 585 G6 with 4 6-core AMD Opteron CPUs – Storage server: MSA Storage – Scalability: 12 – 24 TB 8 processor Configuration – Server: HP ProLiant DL 785 G6 with 8 6-core AMD Opteron CPUs – Storage server: MSA Storage – Scalability: 24 – 48TB
- 21. SQL Server Fast Track Data Warehouse 2.0 for HP – now on G6 Platform Intel Based Reference Architectures 2 Processor Configuration – Server: HP ProLiant DL380 G6 with 2 4-core Intel Xeon® 5500 Series CPUs – Storage server: MSA Storage – Scalability: 4 – 8 TB 4 Processor Configuration – Server: HP ProLiant DL 580 G5 with 4 6-core Intel Xeon® 7400 Series CPUs – Storage server: MSA Storage – Scalability: 12 – 24 TB
- 22. New Fast Track Data Warehouse 2.0 for IBM Three Reference configurations available for IBM: 2 Processor Configuration – Server: IBM System x3650 M2 with 2 Quad-core Intel Xeon CPUs – Storage server: IBM System Storage DS3400 – Scalability: 4 – 8 TB 4 Processor Configuration – Server: IBM System x3850 M2 with 4 6-core Intel Xeon CPUs – Storage server: IBM System Storage DS3400 – Scalability: 12 – 24 TB 8 processor Configuration – Server: IBM System x3950 M2 with 8 Quad-core Intel Xeon CPUs – Storage server: IBM System Storage DS3400 – Scalability: 16 – 32TB
- 23. SQL Server Fast Track Data Warehouse 2.0 for DELL Two Reference configurations available for DELL: 2 Processor Configuration – Server: Dell Power Edge R710 with 2 Quad-core Intel Xeon processors – 8 CPU Cores – 32GB Memory – Storage server: EMC CLARiiON AX4 – Scalability: 4 – 8 TB 4 Processor Configuration – Server: Dell Power Edge R900 with 4 6-core Intel Xeon processors – 24 CPU Cores – 96 GB Memory – Storage server: EMC CLARiiON AX4 – Scalability: 12 – 24 TB
- 24. SQL Server Fast Track Data Warehouse for BULL Two Reference configurations available for BULL: 2 Processor Configuration – Server: Bull Novascale R460 E2 with 2 Quad-core Intel Xeon processors – Storage server: EMC CLARiiON AX4 – Scalability: 4 – 8 TB 4 Processor Configuration – Server: Bull Novascale R480 E1 with 4 6-core Intel Xeon processors – Storage server: EMC CLARiiON AX4 – Scalability: 12 – 24 TB • Also included in the Rack: – SQL Server Analysis Services – SQL Server Reporting Services – SQL Server Integration Services – HA Server – Administration Server (with Management Studio, Backup Server
- 25. Fast Track Data Warehouse Benefits • Lower TCO – Minimizes risk of overspending on un-balanced hardware configurations – Commodity Hardware • Choice – HW platform – Implementation vendor • Reduced Risk – Validated by Microsoft – Encapsulates best practices – Known performance & scalability
- 26. Summary Faster time to solution High scale: up to 48TB Low TCO with better price performance; industry standard hardware Fast Track Data offers customers Better performance out of the box and predictable performance Reduced risk through balanced hardware & Best practices Integration with Madison Hub & Spoke Architecture Twelve reference architectures from HP, Dell, Bull, EMC and SQL Server Fast Track Data IBM Warehouse has 2 components System Integrators with industry solution templates – Avanade, HP, Hitachi, Cognizant and EMC
- 27. Next Steps • Proof Steps – Quick Start DW Roadmap Service – Architectural Design Session – Madison Technology Preview (MTP) – Review Madison, SQL Server Classic or Fast Track DW HW/SW configurations and pricing
- 28. © 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.