VMworld 2014: Databases in a Virtualized World
2 likes290 views
The document discusses challenges related to application and server sprawl, emphasizing the importance of optimizing database architectures for performance and cost. It contrasts RAM-based and storage-based approaches, highlighting issues such as I/O wait times and latency. Modern architectures are suggested to improve efficiency by leveraging fast storage solutions while optimizing resource allocation.
VMworld 2014: Databases in a Virtualized World
- 1. 1 © Violin Memory, Inc. 2014 CONFIDENTIAL Violin Memory, Inc. 2014 Databases in the Virtualized World By Matt Henderson Director, Microsoft Solutions
- 2. 2 © Violin Memory, Inc. 2014 CONFIDENTIAL Application Sprawl Application Sprawl •CRM •Billing •ERP •Sales •Engineering •Monitoring •Manufacturing Real World •3rdParty apps •Old versions
- 3. 3 © Violin Memory, Inc. 2014 CONFIDENTIAL Server Sprawl
- 4. 4 © Violin Memory, Inc. 2014 CONFIDENTIAL Databases: 31 flavors •Databases •MySQL •Oracle •MSFT SQL •Hadoop •Workloads •BI / DW •OLTP •Resources •RAM •CPU •Storage
- 5. © Violin Memory, Inc. 2014 CONFIDENTIAL 5 Dynamic & Shifting Workloads Unique and changing • Per Server • Per Application • Per Day/Hour • Week day versus weekend • Adhoc reporting • Administration Designed for Peak • Not optimal
- 6. 6 © Violin Memory, Inc. 2014 CONFIDENTIAL Databases Persistent data storage Data access engine Application logic processor
- 7. 7 © Violin Memory, Inc. 2014 CONFIDENTIAL Many Databases –Many Data Sets •Not all data can fit in memory •Tieringworks only if patterns well defined •Caching is for reads •Access patterns change •Dedicating resources is inefficient & wasteful
- 8. 8 © Violin Memory, Inc. 2014 CONFIDENTIAL RAM versus Storage Architectural Concepts RAM based: •Invest in memory tier. Dedicate resources to applications. Over allocate to ensure consistent performance. “Distributed” model as performance is reliant on local resources. Storage based •Invest in high performance storage. Reduce memory to minimal amounts. Data needed will almost always come from storage. “Central” model as performance is reliant on shared storage.
- 9. 9 © Violin Memory, Inc. 2014 CONFIDENTIAL RAM versus Storage Memory Challenges •Hard cap in OS (~2TB) •Size limitations per server •More expensive $ per GB •Disruptive to upgrade •Not shareable (dedicated) Database Buffer / Virtualization •Buffers don’t deallocate •Buffers don’t dedup •Inefficient to virtualize Storage Challenges •Disk massively slower than memory •Complex to setup •Needs frequent tuning •Over provisioning is key New Technology Needed •Bridge gap in speed •Easier to manage
- 10. 10 © Violin Memory, Inc. 2014 CONFIDENTIAL Visualizing I/O Wait •Applications wait for I/O to return = I/O Wait •Time slices missed while waiting are lost •Faster CPUs and more cores only increase lost time slices between I/O returns •Solve actual bottleneck, don’t shoot from the hip → Your applications now spend more time doingand less time waiting CPU Cyclewith Disk I/O Wait I/O Wait 20% Work 80% Wait CPU Cyclewith Violin 5% Wait 95% Work
- 11. 11 © Violin Memory, Inc. 2014 CONFIDENTIAL Optimizing Architecture –Performance & Cost •CPU’s are faster than hard drives •Adding cores does not correct issue •8 cores @ 10% = 1 core @ 80% •40% utilization = 60% over paying
- 12. © Violin Memory, Inc. 2014 CONFIDENTIAL 12 Visualizing latency - I/O Wait Infrastructure I/O Wait HDD Storage Many milli-second latency PowerEdge M1000e 9 10 1 2 11 12 3 4 13 14 5 6 15 16 7 8 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 PowerEdge M1000e 9 10 1 2 11 12 3 4 13 14 5 6 15 16 7 8 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 I/O Wait SSD Storage Few milli-second latency SSD SSD SSD …. Or SSD SSD SSD …. Controllers Cache Flash Storage Micro-second latency Fully optimal system PowerEdge M1000e 9 10 1 2 11 12 3 4 13 14 5 6 15 16 7 8 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0
- 13. 13 © Violin Memory, Inc. 2014 CONFIDENTIAL Legacy Architecture Model •Large batches to SSDs •SSDs in disk chassis = bottlenecks •Stranded space and speed 64k64k64k
- 14. © Violin Memory, Inc. 2014 CONFIDENTIAL 14 Modern Architecture Model • Small batches to custom cards • Data spread evenly • 100% usable space • 100% performance on every I/O 4k 4k 4k 4k 4k 4k 4k 4k 4k 4k 4k 4k 4k 4k 4k 4k 64k 4k data + 1k parity 5k Data Data Data Data Parity 1k + 1k + 1k + 1k + 1k = 5k VIMM VIMM VIMM VIMM VIMM flash flash flash flash flash flash flash flash controller 1k bits bits VIMM Parallel and distributed for lowest latency
- 15. 15 © Violin Memory, Inc. 2014 CONFIDENTIAL RAM versus Storage Take Aways: •Architectures should be optimal across all tiers •I/O latency is currency of performance •Centralized performance, through fast storage = always works •Licensing is the highest expense, optimize for CPU usage •Violin performance optimizes IT spend.