Data Center Survey: How Many IOPS Is Enough. Compares to site capacity and system latency.
- 1. 1 OBJECTIVE ANALYSIS – Semiconductor Market Research How Many IOPS is Enough? Tom Coughlin Coughlin Associate & Jim Handy Objective Analysis OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Outline • The Survey • Application Distribution and Attributes • More Survey Results: IOPS, Capacity and Latency • Developing tiers of storage for enterprise (and client) applications • Implications/Projections • Authors & Sources 2
- 2. 2 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Outline • The Survey • Application Distribution and Attributes • More Survey Results: IOPS, Capacity and Latency • Developing tiers of storage for enterprise (and client) applications • Implications/Projections • Authors and Sources 3 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Our Survey • Five-minute survey asked end users – IOPS needs – Capacity – Latency – System bottleneck IOPS – Primary application • Nearly 200 respondents • Report analyzes and interprets the results in depth 4
- 3. 3 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Outline • The Survey • Application Distribution and Attributes • More Survey Results: IOPS, Capacity and Latency • Developing tiers of storage for enterprise (and client) applications • Implications/Projections • Authors and Sources 5 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Applications Breakdown 6 Archiving and backup Video Creation or Distribution Cloud storage or services OLTP Scientific or Engineering Databases Mail server and mail storage
- 4. 4 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Application Types • Cloud Storage/Service-Virtualization • Databases • On-Line Transaction Processing (OLTP) • Video Creation and Distribution • Science & Engineering • Exchange Servers 7 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Cloud Storage/Services & Virtualization • The “IO Blender” – Many streams – Scrambled I/O – Highly random • Suits SSDs better than HDDs for rapid access • Many VM and VDI systems using flash cache to meet demand speed needs Image courtesy of Waring Corp. 8
- 5. 5 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Databases • Large data sets • Random traffic • High I/O load • Early SSD adopter (and before that used DRAM-based SSDs) • Some users load their entire DB on flash memory 9 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com OLTP (On-Line Transaction Processing) • Verified writes – Write/read back – Doubles I/O load • No room for errors • Speed is imperative – Delays lose customers 10
- 6. 6 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Video Creation or Distribution • Large data sets • Multiple video streams – Randomizes access • High bandwidth required • Expensive talent – Don’t want them sitting around waiting Image courtesy of the US Library of Congress 11 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Recording Media Share in Professional Video Cameras 2012 Digital Storage for Media and Entertainment Report, Coughlin Associates 12
- 7. 7 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Science & Engineering • Complex problems – Genome sequencing – CAD/CAM – Natural Resources – Nuclear modeling • Large data sets • Expensive talent – Don’t want them sitting around waiting Image courtesy of Wikimedia Commons 13 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Exchange Server • Multiple tasks – e-mail – Scheduling/calendars – Data storage • Scads of users • e-mail chaos – Multiple mailboxes – Asynchronous sends & receives – Spam & virus filters Image courtesy of Dell Computer 14
- 8. 8 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Outline • The Survey • Application Distribution and Attributes • More Survey Results: IOPS, Capacity and Latency • Developing tiers of storage for enterprise (and client) applications • Implications/Projections • Authors and Sources 15 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Let’s Look At the Overall Results Report breaks analysis down by application – That’s 54 charts – Too much for this presentation! • We’ll just look at Database
- 9. 9 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com IOPS Required 0% 10% 20% 30% 40%Respondents 10 100 1K 10K 100K 1M 10M IOPS OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Storage Capacity Needed 0% 5% 10% 15% 20% 25% Respondents 1G B 10G B 50G B 100G B500G B 1TB 5TB 10TB 50TB 100TB Capacity
- 10. 10 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com How Do The Two Compare To Each Other? OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com IOPS vs. Capacity 1 10 100 1,000 10,000 100,000 1.E+00 1.E+01 1.E+02 1.E+03 1.E+04 1.E+05 1.E+06 1.E+07 IOPS Capacity(GB) 20
- 11. 11 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com System Bottleneck IOPS What is the Fastest Storage Your System Can Use? 0% 10% 20% 30% 40%Respondents 10 100 1K 10K 100K 1M 10M IOPS OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Minimum Latency Requirement 0% 10% 20% 30% 40% Respondents 10ns 1µs 100µs 10ms 1 sec Latency
- 12. 12 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Outline • The Survey • Application Distribution and Attributes • More Survey Results: IOPS, Capacity and Latency • Developing tiers of storage for enterprise (and client) applications • Implications/Projections • Authors and Sources 23 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com DRAMs Now 6,000x HDD Speed! From : HDDs and Flash Memory: A Marriage of Convenience 24
- 13. 13 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Memory and Storage Price vs. Bandwidth 1.E+00 1.E+01 1.E+02 1.E+03 1.E+04 1.E+05 1.E+06 1.E+07 1.E-01 1.E+00 1.E+01 1.E+02 1.E+03 1.E+04 1.E+05 1.E+06 Price per Gigabyte Bandwidth(MB/s) Tape HDD NAND DRAM L3 L2 L1 10 1 10 0 10 2 10 3 10 4 10 5 10 6 10 7 10 1 10 2 10 3 10 4 10 5 10 6 10 0 10 -1 From Objective Analysis: Are Hybrid Drives Finally Coming of Age? 25 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Price/GB Roughly Follows IOPS $0.01 $0.10 $1.00 $10.00 $100.00 $1,000.00 0 200 400 600 800 1,000 1,200 IOPS $/GB 26
- 14. 14 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Device IOPS by Form Factor 102 103 104 105 106 HDD SATA SAS/FC 2-Hop 1-Hop 27 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Outline • The Survey • Application Distribution and Attributes • More Survey Results: IOPS, Capacity and Latency • Developing tiers of storage for enterprise (and client) applications • Implications/Projections • Authors and Sources 28
- 15. 15 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Implications/Projections (1) • SSDs adoption will increase – Usually more IOPS is better • Fast storage is changing – From short-stroked HDDs to SSDs • HDDs becoming a tier behind SSDs • Other system elements will become the bottleneck – Network, software, servers… 29 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Implications/Projections (2) • Users will focus more attention on IOPS – Understanding will be greater than it is today • Higher IOPS will support data/content growth – This means more storage • SSD • HDD • Even tape 30
- 16. 16 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Report Compiles Survey Results • An analysis of complete survey results by application • Report, published July 2014, can be purchased for immediate download at www.Objective-Analysis.com. • Orders can also be processed through Coughlin Assocaites at: http://www.tomcoughlin.com/techpapers.htm. • You can contact Coughlin Associates by calling Tom at 408-871-8808, or e-mailing: Tom@TomCoughlin.com. 31 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Analysts 32 Tom Coughlin, President, Coughlin Associates is a highly-respected storage analyst and consultant with over 30 years in the data storage industry in engineering and management at high profile companies. Jim Handy is a widely recognized semiconductor analyst, has over 30 years in the electronics industry. His background includes marketing and design positions at market-leading suppliers.Jim Handy Objective Analysis Thomas Coughlin Coughlin Associates
- 17. 17 OBJECTIVE ANALYSIS – www.OBJECTIVE-ANALYSIS.com Sources • How Many IOPS do You Really Need? • 2012 Digital Storage for Media and Entertainment, Coughlin Associates: (www.tomcoughlin.com/techpapers) • HDDs and Flash Memory: A Marriage of Convenience, Coughlin Associates and Objective Analysis, 2011 (www.tomcoughlin.com/techpapers) • Two may be Better than One: Why HDD and Flash Belong • SNIA SSSI White Paper, Coughlin/Handy 2010 • Are Hybrid Drives Finally coming of Age?, Objective Analysis, 2010 33