Red Hat Gluster Storage - Direction, Roadmap and Use-Cases
2 likes3,849 views
The document outlines the roadmap and future direction of Red Hat Gluster Storage, detailing its features, integration, and use cases for enterprise environments. It highlights key developments in storage technology, including self-managing features, support for big data analytics, and scalable file storage solutions tailored for various workloads. Additionally, it discusses future trends and planned enhancements in storage management and integration with modern IT infrastructures.
Red Hat Gluster Storage - Direction, Roadmap and Use-Cases
- 1. RED HAT GLUSTER STORAGE: DIRECTION, ROADMAP AND USE-CASES Sayandab Saha Product Management, Red Hat Storage
- 2. A Quick Look Back The Present What's Next: The Future Gluster Upstream Roadmap Red Hat Gluster Storage Integration Roadmap AGENDA
- 3. THE RED HAT STORAGE PORTFOLIO Ceph management OPENSOURCE SOFTWARE Gluster management Ceph data services Gluster data services STANDARD HARDWARE Share-nothing, scale-out architecture provides durability and adapts to changing demands Self-managing and self-healing features reduce operational overhead Standards-based interfaces and full APIs ease integration with applications and systems Supported by the experts at Red Hat
- 4. Nimble file storage for petabyte-scale workloads ● Machine analytics with Splunk ● Big data analytics with Hadoop TARGET USE CASES Enterprise File Sharing ● Media Streaming ● Active Archives Analytics Enterprise Virtualization Purpose-built as a scale-out file store with a straightforward architecture suitable for public, private, and hybrid cloud Simple to install and configure, with a minimal hardware footprint Offers mature NFS, SMB and HDFS interfaces for enterprise use Customer Highlight: Intuit Intuit uses Red Hat Gluster Storage to provide flexible, cost-effective storage for their industry- leading financial offerings. Rich Media & Archival RED HAT GLUSTER STORAGE
- 5. ANALYTICS Big Data analytics with Hadoop CLOUD INFRASTRUCTURE RICH MEDIA AND ARCHIVAL SYNC AND SHARE ENTERPRISE VIRTUALIZATION Machine data analytics with Splunk Virtual machine storage with OpenStack Object storage for tenant applications Cost-effective storage for rich media streaming Active archives File sync and share with ownCloud Storage for conventional virtualization with RHEV FOCUSED SET OF USE CASES
- 6. A QUICK LOOK BACK
- 7. Red Hat Storage Server 2.0 (GA June 2012) ● 6 updates released ● Features: VM image store, performance & stability ● EOL-ed on June 2014 Active/Active cluster-on-cluster ● 6 updates released. Planned EOL October 2015. ● Features: Quota, Geo-Rep, management console, SMB 2.0 LOOKING BACK
- 8. Launched September 2014 Key Features ● Volume snapshots for disk based backup Management ● Monitoring using Nagios ● SNMP Support ● Rolling upgrade support, CDN delivery Hadoop Plug-in for HortonWorks Data Platform 2.0.6 Scale ● 60 drives per server, 128 nodes per cluster RHGS 3.0 (DENALI) PREVIOUS MAJOR RELEASE
- 9. BETWEEN 3.0 AND TODAY 3.0 3.0.1 3.0.2 3.0.3 3.0.4 3.1 Sept 2014 Oct 2014 Nov 2014 Jan 2015 Mar 2015 Summer 2015 “Denali” release Bug fixes RHEL 6.6 support HDP 2.1 Tez Hbase RDMA USS IceHouse rebase for Swift 3-way replication + JBOD Small-file performance “Everglades” release
- 10. THE PRESENT
- 11. Key Features ● Erasure Coding, Tiering, Bit-Rot Detection Protocols ● Active/Active NFSv4 ● SMB 3 (protocol negotiation, in-flight encryption, server-side copy) Red Hat Gluster Storage Console ● Device Management, Geo-Rep, Snapshot, Dashboard, Snapshot Scheduling Security ● SSL based network encryption ● SELinux Enforcing Mode Performance ● Rebalance performance enhancement (100% improvement) RHGS 3.1 (EVERGLADES)
- 12. Data protection without using RAID & replication Break data into smaller fragments, store and recover from a smaller number of fragments New type of volumes: Dispersed, dist- dispersed Initial supported configurations: 8+3, 8+4 & 4+2 configuration Algorithm used is REED-Solomon ERASURE CODING
- 13. SEQUENTIAL IO PERFORMANCE WITH ERASURE CODING
- 14. HOT TIER COLD TIER GLUSTER VOLUME TRUSTED STORAGE POOL TIERING Automated data movement between hot & cold tiers Movement based on access frequency ● Hot tiers could be SSDs, cold tiers are normal disks Attach & detach a tier to and from an existing Gluster volume All I/Os forwarded to hot tier Cache misses promote data to hot tier
- 15. BIT ROT DETECTION Protection against “silent data corruption” Two fundamental procedures ● Signing using SHA256 ● Scanning/scrubbing for rot Lazy checksum maintenance ● (not inline to data path) Checksum calculation when a file is “stable” Alert/log on mismatched checksums Scanning mode is admin selectable to control impact
- 16. NFSv4 ACLs PseudoFS support Security ● Kerberos authentication using RPCSEC_GSS, krb5/i/p ● Kerberos authentication using RPCSEC_GSS, spkm3 Active/Active cluster-on-cluster ● With up to 16 active-active NFS heads ● Gluster storage pool scales out as usual Ability to add and delete RHS volume exports to nfs-ganesha at run-time Delegations to be supported in an update release ACTIVE/ACTIVE NFSv4
- 17. NFS HEAD NFS HEAD NFS HEAD NFS HEAD GANESHA CLUSTER USING PACEMAKER & COROSYNC TRUSTED STORAGE POOL GLUSTER VOLUME ACTIVE/ACTIVE NFSv4
- 18. WHAT'S NEXT – THE FUTURE
- 19. TRADITIONAL STORAGE NEXT GENERATION STORAGE Manual provisioning of LUNs and volumes with some degree of automation Self-service provisioning by lines of businesses and application developers Static selection of storage platforms based on application needs Catalog based storage service offerings with metering & charge-back Scale-up with some scale-out Expand, Shrink and scale on demand Little to no flexibility in selecting optimum storage back-end for workloads Policy based storage back-end selection STORAGE TRENDS: MODERN IT INFRASTRUCTURES
- 20. Key elements for modern storage infrastructure (Manila, containers, hyper-converged) ● Consumption Model (“File As A Service” or “NAS on Demand”) ● Dynamic provisioning, healing, tuning & balancing ● Security & multi-tenancy ● Cloud scale & stability at scale ● Performance: performant storage back-end for a wide variety of workloads ● Advanced data services: tiering, compression, de-duplication APPLIED TO GLUSTER
- 22. Gluster 4.0 will be our technology base for the next five years or so ● Based on 3.x experience Design must be based on estimates of where we’ll be in 2021 Higher node counts and more complex networks Heterogeneous storage ● e.g. NVMe for performance, SMR for capacity New workloads and usage models ● Hyper-convergence ● Containers ● “XYX as a service” and multi-tenancy GLUSTERFS 4.0 CONTEXT
- 23. Declarative and constraint-based ● Not “this brick and this brick and this brick” ● More like “this big, replicated this many times, these features” ● We figure out which combinations match user requirements Overlapping replica/stripe/erasure sets ● Ease requirement to add bricks in multiples ● Better load distribution during and after failures Multiple replication levels (and types) within one volume More sophisticated tiering, rack- or security-aware placement, etc. FLEXIBLE STORAGE MANAGEMENT
- 24. More scalable membership protocol Stronger consistency for configuration data Improved modularity ● Most-changed code in 3.x ● Increasing complexity and merge conflicts slow down the entire project ● Plugin approach allows independent development of new features Prerequisite for other 4.0 features GLUSTERD CHANGES
- 25. Client-side caching ● Fully consistent via “upcall” mechanism Third-party copy ● Already part of NFS and SMB protocols Multiple networks and Quality of Service ● Leverage faster private networks e.g. for replication ● Isolate internal traffic ● Protect tenants from each other PERFORMANCE ENHANCEMENTS
- 26. Theme: Storage/File as a Service Use-Cases: Storage for containers, OpenStack Manila Technology Enablers: ● Dynamic Provisioning ● At-rest Encryption ● Inode-quotas ● Cloud Scale & Stability at Scale ● Performant back-end for diverse workloads ● Autonomous operations ● Multi-tenancy FUTURE FOCUS AREAS
- 27. RHGS 3.2 (Fundy) H1-CY2016 Baseline ● GlusterFS 3.8, RHEL 6, RHEL 7 Management ● Dynamic provisioning of volumes Key Features ● Inode quotas Protocols ● SMB 3.0 (advanced features) ● Multi-channel support Performance ● Rebalance ● Self-heal Security ● At-rest encryption Baseline ● GlusterFS 4, RHEL 7 Key Features ● Compression, Deduplication Core Infrastructure ● Next gen replication ● Highly scalable control plane ● DHTv2 Protocols ● pNFS Performance ● QoS ● Client side caching Management ● USM, Gluster ReST API RHGS 4 (Gir) (In Planning) RED HAT GLUSTER STORAGE ROADMAP
- 28. RED HAT GLUSTER STROAGE INTEGRATION ROADMAP
- 29. Tech preview level support for RHELOSP 7 (Kilo) ● Create/delete/rename/list share ● Create/delete snapshots ● Allow/deny access to shares ● OSP Director integration planned for September release (ver 1.1) Full support expected in RHELOSP 8 (Liberty) ● Create/delete share dynamically ● Create share from snapshot ● Exploring integration with Barbican for managing certificates OPENSTACK MANILA
- 30. 3-key storage use-cases ● Persistent data store for state-full containers ● Container image registries ● Storage for live container images (local storage) Focused on “Persistent data store for containers” use-case ● Containerized applications mount Gluster as their data store ● NFS or GlusterFS native client integration in kubernetes Key attributes that makes Gluster interesting ● Not impacted my mount-storm ● Built in HA CONTAINERS