Cloudian hyper store
- 1. Cloudian HyperStore Smart Data Storage @scale Frank Turner, Cloudian Sr Solution Architect August 11, 2016
- 2. Amazon S3 – Simple Storage Service Object storage platform introduced in 2006 (US) and in Europe from 2007 Data is stored as binary objects, or blobls, referenced by object name Massively scalable infrastructure – 2 trillion objects stored in 2013 with 1.1 million requests per second – capacity doubling each year Data stored and accessed using web-based HTTP protocols (APIs) ReST, SOAP and BitTorrent (SOAP being discontinued) API access through native HTTP or SDKs (Ruby, Java, .NET, PHP) Data protection is managed by AWS – updates are eventually consistent across an availability zone (asynchronous protection) 2 Cloudian provides an on-premise OBS solution that is 100% S3 compatible
- 3. Proof 2 : Run Ceph S3 test. We provide Bug level compatibility Proof 1 : Ask customer to try DragonDisk, S3 Brower or other industry standard apps Proof 3 : Developers can use Amazon’s developer SDK & point to us Cloudian’s 100% S3 Guaranteed
- 4. Delivering an onsite Standards-based Cloud Every node provides Storage & S3 API access Start Small & Grow on Demand 3 Nodes 5000+ Nodes Always On Non-Disruptive Upgrades Survives multiple failures Multiple protection strategies Non-Disruptive Add Node Peer-to-peer Architecture Elastic and Geo-dispersed Distributed Everything
- 5. Multiple options for Protecting data Multi Tenancy, QoS, Billing Hybrid Cloud Smart Support Built for Scale 100% S3 Flexible Deployment Options Object Lifecycle Management Storage Optimizations One system for File & Objects Cloudian Features and Benefits…
- 6. Data Protection Multi DC One System with both Replication & Erasure Coding Local Aware Data Placement Flexible Consistency Levels Sync or Async IO Acknowledgements Configurable Data Protection & Consistency
- 7. Hybrid Cloud/Tiering Archive R&DContent Depot HyperStore Software Defined Storage Another Hyperstore Cluster Amazon S3 Amazon Glacier S3 Compatible Service ProviderFirewall • Migrate Data to Cheaper Tiers (based on policies) • Global Namespace (stubs kept on premise always) • 256bit encryption –(into public cloud) Eg Tier R&D data after 1 yr
- 8. Cross Region Replication & Restore Target Usecases Secure Vault – Log all changes to separate Cluster Hyperstore Backup – Backup data & Restore to any point time DR – Protection from Primary Cluster Loss (eg Disasters) v2 RESTORE Objects from Past To New Bucket (eg Jan 1) Photo.jpg ALL new PUTs are replicated to Destination Cluster v1 v2 v3 v1 v2 v3 Source Cluster Destination Cluster Photo.jpg Dec1 Jan1 Feb1 DELETEs has NO Affect on Destination Cluster v1 v2 v3 v1 v2 v3 Delete Marker Jan9 Supported Deployments Hyperstore 6 -> Hyperstore6 Hyperstore 6 ->
- 9. HyperStore Smart Support HyperStore Telemetrics Global Analytics Proactive Resolution Real Time Monitoring Cloudian Smart Support
- 10. HyperStore Use Cases 10 Media Content Store Data Analytics Private Cloud File Distribution & Sharing Backup Archive
Editor's Notes
- #12: Founded in 2011, built on deep IP from telco messaging 1) over 10,000 MPS systems, storing billions of messages in multiple DCs Architected it as Telco mission critical, features like QoS, dynamic protection 2) 3 bets: Cassandra – best multi-DC, most mileage on the code S3 – applications are key, SDKs are key, storage itself is not interesting. Native, no gateways, every node has S3; Hybrid – public storage will be free. Some data will always be kept locally for speed or privacy. All storage must be hybrid or it will be too expensive.