Exchange 2010 storage improvements
Download as PPTX, PDF0 likes1,630 views
The document outlines significant storage improvements in Microsoft Exchange Server 2010, focusing on enhancements in I/O performance and storage flexibility. Key features include reduced I/O operations per second (IOPS), increased mailbox sizes, and the introduction of JBOD storage options to lower costs. The improvements allow for better handling of large mailboxes and high item counts, ultimately enhancing overall performance and disaster recovery capabilities.
Exchange 2010 storage improvements
- 1. Exchange 2010 Storage ImprovementsNathan Winters – Exchange MVP
- 2. AgendaA Brief History of Exchange StorageThe new ethosFeature Deep DiveSummary
- 3. History
- 4. HistoryESE/JET BlueIOPS – Random IO applicationWhy? – Small Expensive drives1.6GB disk $400 in 1996SCSI 2GB and 4GB 100 IOPSSingle Instance StorageClustering with Shared StorageBackup an issueSingle Point of Failure32 bit Not enough RAMRam limited number of users per server
- 5. History - Exchange 2007Big improvements in Exchange Server 2007Reduce storage input/output (I/O) (70%)Use large amounts of memory (64 bit)Increased page size (4 kilobyte (KB) -> 8 KB)Lower storage costsSupport large mailboxes (> 1 gigabyte (GB))Provide fast search (CI)Continuous replication (log shipping)High Availability (HA) + fast recoveryEliminate single points of failure5
- 6. New Ethos
- 7. Email UsageRadicati seeing 165 mails per day growing to 230 over next couple of yearsUsers used to large free storage25GB5GB 3 years of mailTriage once per year to archiveNot once per day!Mail available through all clientsCached Mode/Performance issuesHigh Item counts – 5000, 20000, 100000
- 8. Disk TechnologyCurrently 2TBMoving to 8TBRandom IO not getting quicker 15K RPM, 10K RPM, 7.2K RPMDensity is getting better so can read more data in the same timeFlash – SSD – Didn’t take that betOptimised for spinning media for E14Expensive – so use as Cache in SAN
- 9. Exchange Server 2010 Storage VisionIO ReductionSequential IOSATA/Tier 2 Disk OptimizationLarge, Fast, Low-cost MailboxesStorage Design FlexibilityRAID-less Storage (JBOD)9
- 10. Exchange Server 2010 HA Storage Design Flexibility10DAS (SAS) DAS (SATA) HA = Shared Storage Clustering+1.0 IOPS/Mailbox3.5” 15K 146GB FC DisksRAID10 for DB & LogsDedicated SpindlesMulti-path (HBA’s, FC Switches, SAN array controllers)Backup = Streaming off active Fast Recovery = Hardware VSS (Snapshots/Clones)HA = CCR.33 IOPS/Mailbox2.5” 146GB 10K SAS DisksRAID5 for DBRAID10 for LogsSAS Array Controller (/w BBU)Backup = VSS SnapshotFast Recovery = CCRHA = DAG (2 DB copies).11 IOPS/Mailbox3.5” 2TB 7.2K SATA/SAS DisksRAID10 for DB & LogsSAS Array Controller (/w BBU)Backup = Optional/VSSFast Recovery = Database FailoverHA = DAG (3+ DB copies).11 IOPS/Mailbox3.5” 2TB 7.2K SATA/SAS Disks1 DB = 1 DiskBackup = Optional/VSSFast Recovery = Database FailoverSANJBOD (SATA)More options to reduce storage cost
- 11. JBOD/RAID-less Storage: Now An OptionJBOD : 1 disk = 1 database (with logs)Requires Exchange Server 2010 High Availability (3+ DB Copies)Annual Disk Failure Rate (AFR) = 5% 11
- 12. Exchange Server 2010 HASimplified mailbox High Availability and disaster recovery with new unified platformNew YorkSan JoseMailbox ServerMailbox ServerMailbox ServerReplicate databases to remote datacenterDB1DB1DB1Recover quickly from disk and database failuresDB2DB2DB2DB3DB3DB3DB4DB4DB4DB5DB5DB5Evolution of continuous replication technology (database mobility)Easier than traditional clustering to deploy and manageAllows each database to have 16 replicated copiesProvides full redundancy of Exchange roles on as few as two servers12
- 13. Deep Dive
- 14. Exchange 2010 FeaturesMove to Sequential IOChange Table structureLazy ViewPage size 32KBDatabase Compression (LVC)Read/Write CoalescingDatabase ContiguityCache CompressionStorage Groups GoneSingle Point of Failure GoneOptimised for huge mailboxes
- 15. Random vs. Sequential Disk IORandom IODisk head has to move to process subsequent IOHead movement = High IO latencySeek Latency limits IO (IOPS)Sequential IODisk head does not move to process subsequent IOStationary head = low IO latencyDisk RPM speed limits I/O per second (IOPS)Disk Head7.2K SATA Disk (20ms Latency)Random = 50 IOPSSequential = +300 IOPS15
- 16. IO Reduction: Store Table ArchitecturePer DatabasePer FolderExchange Server 2007Secondary Indexes used for ViewsPer DatabasePer MailboxPer ViewExchange Server 2010New store schema = no more single instance storage within a database16
- 17. Exchange 2007M1M2M1M3M2Nickel & Dime ApproachMany, random, IOs (1 per update)TimeDB I/OM1 arrivesM2 arrivesM1 flaggedM3 arrivesM2 deletedUser uses OWA/Outlook Online and switches to this viewExchange 2010M1M2M1M3M2Pay to Play ApproachFewer, sequential, IOs (1 per view)Store Schema Changes: Lazy View Updates
- 18. IO Reduction: Database Page Size Increased to 32 KBExchange Server 2007 DB Read 20 KB MessageDBCacheDisk3 Read IO’s8 KB PagesExchange Server 2010 DB Read 20 KB MessageDBCacheDisk1 Read IO32 KB Pages18
- 19. Mitigate DB Space Growth: Database CompressionProblem:Store Schema change, space hints, B+Tree Defrag and 32 KB page size combine to increase DB file size by 20%Solution: Growth is 100% mitigated by Database CompressionTargeted compression for message headers and text/html bodies (7bit/Express)DB Space AnalysisDB File Size ComparisonMsg Views32KB Pages1 Database, 750 x 250MB mailboxesRTF = RTF Compressed, Mix = 77% HTML, 15% RTF, 8% TextAvg. Message size = ~50KB19
- 20. IO Reduction: Read IO Gap CoalescingExchange Server 2007 DB Read BehaviorDBCacheDisk3 Read IO’sExchange Server 2010 DB Read BehaviorDBCacheDisk1 Read IO20
- 21. IO Reduction: Maintain Contiguity Over TimeNew Database Maintenance Architecture:Database B+Tree Defragmentation (aka OLD2):Background/throttled process that maintains space and contiguity of database tables21
- 22. IO Reduction: Database Contiguity ResultsExchange Server 2007 Message Header Table (aka MFT)DB Page NumbersFRAGMENTEDRandom deletes at the tailExchange Server 2010 Message Header Table (aka MsgHeader)CONTIGUOUS*Production/Dogfood database analysisBlue = contiguous (good)Red = fragmented (bad)22
- 23. Summary
- 24. Exchange IO Trend+90% Reduction!24
- 25. Putting It All Together: Mailboxes/DiskExchange Server 2010 storage improvements cannot be quantified in IOPS reductions alone+4X Mailboxes/Disk!+500125250 MB Mailbox Size, 3MB DB Cache/user, 12 x 7.2k SATA disks (DB/Logs on same spindles), Loadgen Outlook 2007 Online Very Heavy Profile, measured at <20ms RPC Average latency25
- 26. SummaryExchange Server 2010 store has…Reduced DB IOPS by +70%...again!Optimized for large mailboxes (+10 GB) and 100K item countsOptimized for large/slow/low-cost disks (SATA/Tier2)Made JBOD/RAID-less storage a viable optionEnables unmatched storage flexibility to push storage Capex costs downProvides many more backup/DR options26
Editor's Notes
- #11: Currently 2TBMoving to 8TBRandom IO not getting quicker 15K RPM, 10K RPM, 7.2K RPMDensity is getting better so can read more data in the same timeFlash – SSD – Didn’t take that betOptimised for spinning media for E14Expensive – so use as Cache in SAN
- #12: JBOD – i.e. 1 disk per database and log setRAID less – disks will fail3 + copies
- #15: 2007 roughly the same at Exchange 4.0One database and then a couple of really large tablesMessage table and attachments – all messages per databaseMessage folder table per mailbox which does all the viewsThis gives the benefit of single instance storage – one copy in the message table with pointers from the message folder tableRandom IO!2010 schema is changed massivelyOnly one table per database Now data specific to mailbox so data can be kept sequential for quick retrieval from the same area of diskNow message view table instead of secondary indexes
- #16: Really important in reduction of IOUpdate the view when user views!
- #17: Page size is smallest section of IOBigger page means less small IO for a single message read2007 random layout of data on disk means 3 Ios201020K message – pull the same message – get the message header and body on one pageThis makes a huge difference to IOPage size will be fine for handling large messages – 12-15K is mean size of message currently
- #18: As you add larger page sizes, and lay things out for sequential IO DB grows by 20%.Same as OST grew in SP2 for office 2007Now compress message headers and text/hmtl bodies – limited to this for speedCan bring database back to same size as 2007 or even less if bulk of HMTL messagesNow have many more tables and fewer bigger pagesThere is also Cache compressionSo when you pull a 32KB page – the smallest element of Exchange data but that page only holds 16KB of data the free space will be compresses so that only 16KB of cache is used.
- #19: Can do coalescing when pages are not next to each other2007 needs 3 ios to get pages off disk – random IO2010 bring all five up – a stream of IOThen evict the middle pages
- #20: Cleanup was done using online maintenance and defragThis has changes and cleanup is done when tombstone or dumpster cleanup happens – Page 0 happens automatically as because it occurs when the write is being done anyway, there is no additional IO2003 and 2007 are great for compaction2007 SP1 changed this slightly to reduce IO during the maintenance window.2010 this has changed a lot – it is done at run time as space is seenContiguity has never been a concern until now - compaction has always not worried about continuity to make it small2010 makes trade offs on size as we’ve mentioned to ensure contiguity – analysis happens continuouslyDB check summing
- #21: Utility MSFT built to track contiguity of the new DBThis is showing a Message folder table of the inbox on 2007This is massively Random2010 is contiguous as pages are laid out sequentially so that reading a huge folder full of 10000 items is quick and easy!