SUSE: Software Defined Storage
1 like1,082 views
1) Ceph is an open source distributed storage system that provides scalable, fault-tolerant storage and manages petabytes of data across clusters of commodity hardware. 2) It uses Object Storage Daemons (OSDs) that serve storage objects and replicate data across peers for redundancy. Multiple OSDs can be grouped in monitor nodes that track cluster state. 3) Ceph offers self-healing capabilities through redundancy and allows data to be placed close to applications for performance. It provides APIs and integration with clouds for flexible, software-defined storage.
![8
From 10,000 Meters
[1] As per 2014 OpenStack user survey
• Open Source Distributed Storage solution
• Most popular choice of distributed storage for
OpenStack
[1]
• Lots of goodies
‒ Distributed Object Storage
‒ Redundancy
‒ Efficient Scale-Out
‒ Extensible
‒ Can be built on commodity hardware](https://image.slidesharecdn.com/suseolafkirchsoftwaredefinedstorage-150709084826-lva1-app6892/85/SUSE-Software-Defined-Storage-8-320.jpg)
SUSE: Software Defined Storage
- 1. Software Defined Storage Powered by Ceph Olaf Kirch Director SUSE Linux Enterprise, SUSE R&D okir@suse.com
- 2. 2 Current Enterprise Data Storage Market More data to store •business needs •more data driven processes •more applications •e-commerce More data to store •business needs •more data driven processes •more applications •e-commerce Bigger data to store •richer media types •presentations, images, video Bigger data to store •richer media types •presentations, images, video For longer •regulations / compliance needs •business intelligence needs For longer •regulations / compliance needs •business intelligence needs 2000 2014
- 3. 3 While you think about Storage... • ... can you also make it ‒ more scalable ‒ more fault tolerant ‒ more flexible? • Sure! For here or to go?
- 4. 4 Today's Storage Arrays • Limits: ‒ Tightly controlled environment ‒ Limited scalability ‒ Few options ‒ Only certain approved drives ‒ Constrained number of disk slots ‒ Few memory variations ‒ Only very few networking choices ‒ Typically fixed controller and CPU • Benefits: ‒ Reasonably easy to understand ‒ Long-term experience and “gut instincts” ‒ Somewhat deterministic behavior and pricing
- 5. 5 What about Better File Systems? • Layered on top of your block storage, providing ‒ scalability ‒ fault tolerance • Most of the time, it's an either-or decision ‒ separation of data and metadata (pNFS, glusterfs) ‒ clustering (ocfs2, gfs2) ‒ ... they require special drivers in your favorite OS ‒ ... and they all want to talk to a storage array • Recent evolution ‒ HDFS (underlying FS of Hadoop)
- 8. 8 From 10,000 Meters [1] As per 2014 OpenStack user survey • Open Source Distributed Storage solution • Most popular choice of distributed storage for OpenStack [1] • Lots of goodies ‒ Distributed Object Storage ‒ Redundancy ‒ Efficient Scale-Out ‒ Extensible ‒ Can be built on commodity hardware
- 10. 10 Not for the Faint of Heart • At the core of Ceph is a PhD Thesis ‒ http://ceph.com/papers/weil-crush-sc06.pdf • Goals: ‒ no bottlenecks ‒ no single point of failure
- 11. 11 Several Ingredients • Distributed ‒ Coarse grained partitioning of storage supports policy based mapping (don't put all copies of my data in one rack) ‒ Topology map and Rules allow clients to “compute” the exact location of any storage object • Redundancy ‒ Achieved by data replication • Flexibility ‒ Multiple Storage “Pools” can be defined with different parameters
- 13. 13 For a Moment, Zooming to Atom Level FS Disk OSD Object Storage Daemon File System (btrfs, xfs) Physical Disk ● OSDs serve storage objects to clients ● Peer to perform replication and recovery
- 14. 14 Put Several of These in One Node FS Disk OSD FS Disk OSD FS Disk OSD FS Disk OSD FS Disk OSD FS Disk OSD
- 15. 15 Mix In a Few Monitor Nodes M • Monitors are the brain cells of the cluster ‒ Cluster Membership ‒ Consensus for Distributed Decision Making • Do not serve stored objects to clients
- 16. 16 Voilà, a Small RADOS Cluster M M M
- 17. 17 Linux Host RADOS Block Device M M M M krbd librados
- 18. 18 RADOS Block Device • Disk images are striped across (parts of) the cluster • Supports ‒ Snapshot and rollback ‒ COW cloning ‒ Thin provisioning
- 19. 19 RADOS Block Device: Placement M M M M
- 20. 20 Ceph in Action: Reading Data M M M M Reads can be serviced by any of the replicas (parallel reads improve thruput)
- 21. 21 Ceph in Action: Writing M M M M Writes go to one OSD, which then propagates the changes to other replicas
- 22. Self-Healing
- 23. 23 Self-Healing M M M M Monitors detect a dead OSD
- 24. 24 Self-Healing M M M M Monitors allocate other OSDs and update mapping
- 26. 26 Self-Healing M M M M Future writes update the new replica
- 27. Why am I telling you this?
- 28. 28 Because we think Ceph is Great!
- 29. 29 By 2018, open-source storage will gain 20% of the market share, up from less than 1% in 2013 (Gartner)
- 30. 30 Enterprise Data Capacity Utilization (Percent) 50-60% of Enterprise Data 20-25% 15-20% 1-3% Tier 0 Ultra High Performance Tier 1 High-value, OLTP, Revenue Generating Tier 2 Backup/Recovery, Reference Data, Bulk Data Tier 3 Object, Archive, Compliance Archive, Long-term Retention Source: Horison Information Strategies - Fred Moore
- 31. 31 SUSE Enterprise Storage Market LOW FUNCTIONALITY HIGH FUNCTIONALITY Object Storage Archive Storage Data Backup Video Audio Big Data Data Analytics OLTP CRM ERP Email HPC Compliance ArchiveCAPACITY OPTIMIZED PERFORMANCE OPTIMIZED DR Target Initial Target MarketInitial Target Market Bulk Storage VM-Aware
- 32. In Closing
- 33. 33 Summary: Why Ceph? • Can be scaled arbitrarily ‒ No central bottleneck “master” servers • Low operational cost ‒ Automation ‒ Commodity hardware • Can move data close to application • Redundancy through data replication ‒ Self-healing ‒ No need for RAID • APIs and cloud integration for self-service ‒ Software defined storage
- 34. Thank you. 34 For questions: okir@suse.com Please visit us as www.suse.com
- 36. Corporate Headquarters Maxfeldstrasse 5 90409 Nuremberg Germany +49 911 740 53 0 (Worldwide) www.suse.com Join us on: www.opensuse.org 36
- 37. Unpublished Work of SUSE. All Rights Reserved. This work is an unpublished work and contains confidential, proprietary and trade secret information of SUSE. Access to this work is restricted to SUSE employees who have a need to know to perform tasks within the scope of their assignments. No part of this work may be practiced, performed, copied, distributed, revised, modified, translated, abridged, condensed, expanded, collected, or adapted without the prior written consent of SUSE. Any use or exploitation of this work without authorization could subject the perpetrator to criminal and civil liability. General Disclaimer This document is not to be construed as a promise by any participating company to develop, deliver, or market a product. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. SUSE makes no representations or warranties with respect to the contents of this document, and specifically disclaims any express or implied warranties of merchantability or fitness for any particular purpose. The development, release, and timing of features or functionality described for SUSE products remains at the sole discretion of SUSE. Further, SUSE reserves the right to revise this document and to make changes to its content, at any time, without obligation to notify any person or entity of such revisions or changes. All SUSE marks referenced in this presentation are trademarks or registered trademarks of Novell, Inc. in the United States and other countries. All third-party trademarks are the property of their respective owners.