High Performance Object Storage in 30 Minutes with Supermicro and MinIO
- 1. Siva Yarramaneni Solution Engineering, Supermicro Eco Willson Engineer, MinIO Matt Sarrel Technical Marketing, MinIO Bala Kyaw Sr. System Engineer, Supermicro Dec 2021 High Performance Object Storage in 30 Minutes with MinIO
- 2. Building Clouds With Supermicro Solutions, building Cloud is very simple !!!
- 3. Can you give me a total solution (Hardware + Software + Service) ??? How to build 1. Cloud Computing 2. Cloud Storage Can you help me build compute solutions such as 1. OpenStack 2. Kubernetes Containers What kind of storage I need for these solutions? Is object storage my best choice?
- 4. Supermicro Solutions Cloud / Virtualization Software Defined Storage Big Data Other Solutions Analytics AI / ML Operating Systems
- 5. High Performance Object Storage in 30 Minutes with Supermicro and MinIO
- 6. Agenda 1. Meet the Software: MinIO object storage 2. Meet the Hardware: Supermicro Cloud DC 3. MinIO + Supermicro Cloud DC = Extreme Performance 4. Demonstration 5. Summary
- 7. What is MinIO MinIO is a high performance, Kubernetes-native object store. It is designed for large-scale data infrastructure. It was built from scratch to be cloud native. It has become the storage standard for hybrid cloud & multi-cloud architectures. TENANT 1 TENANT 2 TENANT n Object Storage CPU NETWORK DRIVE APPLICATIONS
- 8. Guiding Principles MinIO is focused on performance. MinIO’s benchmark’s have established us as the fastest object store in existence. Kubernetes-native. Born in the cloud with cloud native DNA. MinIO is designed for simplicity. Simplicity scales. Performant Cloud Native Simple
- 9. A Global Growth Story
- 10. Every Imaginable Enterprise Feature + Then Some S3 COMPATIBLE Amazon S3 API is the de facto standard for object storage. MinIO implements Amazon S3 v2/v4 API + S3 Select FULL INTERACTION SUITE MinIO offers a developer-driven command line interface in addition to an IT-friendly graphical user interface. LAMBDA COMPUTE MinIO server triggers Lambda functions through event notification service. OCR, audit compliance are good examples of lambda computing. ERASURE CODE & BITROT PROTECTION You may lose up to half the number of drives and still recover from it. Data protection code is accelerated using SIMD instructions on x64 and ARM CPUs. ACTIVE ACTIVE REPLICATION A resilient and scalable architecture that can withstand a complete DC failure. MONITORING API driven capabilities with a suite of web hooks to your favorite apps. ACCESS MANAGEMENT MinIO Server integrates with Identity Providers such as WSO2, Keycloak, Okta, Ping Identity to allow applications or users to authenticate and use Object Storage. ENCRYPTION & TAMPER-PROOF MinIO provides confidentiality, integrity and authenticity assurances for encrypted data with negligible performance overhead. IMMUTABILITY Complete suite of object locking capabilities is Cohasset certified - required by financial services and healthcare customers. KUBERNETES-NATIVE MinIO was built natively for RESTful APIs - no connectors needed. More than 62% of MinIO instances are containerized with 43% of those managed via Kubernetes.
- 11. CloudDC Product Highlights X12 CloudDC Optimized Server • Security is top priority • TPM 1.2/2.0, signed firmware, Silicon Root of Trust • Secure Boot, System Erase • Run time FW protection, FIPS Compliance, Trusted Execution Environment Secure • Compact system design makes no waste of internal space • < 650mm chassis depth • Fully utilized system resource with 12 NVMe, 4 PCIe 4.0 x16 + 2 PCIe 4.0 x8 expansion Compact • All hybrid drive bays (NVMe/SAS/SATA) + Flexible internal storage options (dual M.2 / SATADOM) • Up to 6 standard PCIe 4.0 expansion slots • Up to 2 FHFL DW GPU or 6 LP GPU Ultimate Flexibility • Tool-less mechanical design for rapid deployment • Hot-swap storage and PWS for easy maintenance. • IPMI, serial port and service tag for easy management Efficient • Optimized thermal design • High efficiency Platinum and Titanium level PWS (AC/DC) • Reduced waste with bulk packaging and customizable accessories We Keep it Green • Balanced architecture between CPUs and optimized for scalable compute, database, GPU, tiered storage and I/O intensive applications • Cost and performance optimized down to component level • Support OCP 3.0 standard Application Ready
- 12. CloudDC 1U Compact Compute Server System Memory Power Management Processors Drives I/O Security • 1U Rackmount • Chassis Depth 23.5” • Tool-less Design • 16 DIMM slots • DDR4-3200 RDIMM / LRDIMM • Redundant 860W Platinum Level (94%) AC/DC Power Supplies Open Industry Standard IPMI, Redfish APIs, Rack Scale Management • Dual Socket • Up to 270W TDP • 10x 2.5-inch Hybrid Drive Bays (NVMe / SAS / SATA) • 2x NVMe M.2 • 1x SATADOM • 2x PCIe 4.0 x16 FHHL • 2x AIOM (OCP 3.0 NIC) • 1x Dedicated IPMI • 1x VGA • 1x Serial • 4x USB (2x Rear, 2x Header) • TPM 1.2/2.0 • Signed firmware • Silicon Root of Trust • Secure Boot • System Erase PRODUCT FEATURES 1U 16 860W 2S SYS-120C-TN10R This server offers great scalability and flexibility, dual AIOM slots plus 10 full hybrid drive bays ensures a solution to most applications. OCP compatible networking adds on extra flexibility on hardware configuration and also helps bring down the TCO. Designed to have great serviceability with tool-less brackets and trays, redundant power supplies and hot-swap drives also help make maintenance easier and quicker. M.2 and SATADOM option can host OS installation with optional RAID feature. 10 PCIe 4.0 x16 - CPU 2 PCIe 4.0 x16 - CPU 1 AIOM - CPU 2 AIOM - CPU 1
- 13. CloudDC 2U I/O Optimized Server System Memory Power Management Processors Drives I/O Security • 2U Rackmount • Chassis Depth 25.5” • Tool-less Design • 16 DIMM slots • DDR4-3200 RDIMM / LRDIMM Redundant 1200W Titanium Level (96%) AC/DC Power Supplies Open Industry Standard IPMI, Redfish APIs, Rack Scale Management • Dual Socket • Up to 270W TDP • 12x 3.5-inch Front Drive Bays (NVMe/SAS/SATA) • 2x 2.5-inch Rear Drive Bays • 2x NVMe M.2 • 1x SATADOM • 4x PCIe 4.0 x16 FHHL • 2x PCIe 4.0 x8 FHHL (optional) • 2x AIOM (OCP 3.0 NIC) • 1x Dedicated IPMI • 1x VGA, 1x Serial • 4x USB (2x Rear, 2x Header) • TPM 1.2/2.0 • Signed firmware • Silicon Root of Trust • Secure Boot • System Erase PRODUCT FEATURES 2U 16 1200W 2S SYS-620C-TN12R The CloudDC 2U optimized server offers ultimate flexibility on I/O and storage. Go beyond the limit with 4x PCIe 4.0 x16, 2x PCIe 4.0 x8 FH slots for add on features like NIC, storage controller and GPU. 12x fully hybrid front drive bays with PCIe 4.0 NVMe support makes SYS-620C-TN12R suitable for high density, high performance applications. 14 AIOM - CPU 2 AIOM - CPU 1 PCIe 4.0 x16 FHHL - CPU 2 *PCIe 4.0 x16 FHFL - CPU 2 *PCIe 4.0 x8 FHHL - CPU 2 PCIe 4.0 x16 FHHL - CPU 1 *PCIe 4.0 x8 FHHL – CPU 1 *PCIe 4.0 x16 FHFL - CPU 1 * Configurable to x16 or two 2 x8
- 14. High Performance
- 15. What Makes MinIO Fast SINGLE LAYER We are a single layer, object only. Multiple layers cause latency, complexity. SIMD ACCELERATION By writing the core parts of MinIO in assembly language (SIMD extensions) we are hyperfast on commodity HW. COMBINATION OF GO + GOASM Delivering faster than C performance by combining GO + Assembly Language and targeting them to the task. NO METADATA DATABASE By writing object and metadata together you make all operations single and atomic. Multiple steps for other vendors.
- 16. 16 Supermicro Cloud DC Benchmark Results 4 Node NVMe 100 Gbe Network Read/GET Write/PUT Distributed 42.57 GB/s 24.69 GB/s Distributed/Encrypted 42.54 GB/s 25.96 GB/s Distributed/TLS 42.35 GB/s 24.42 GB/s Distributed/TLS/Encr. 42.41 GB/s 23.88 GB/s 8 Node NVMe 100 Gbe Network Read/GET Write/PUT Distributed 46.54 GB/s 34.4 GB/s Distributed/Encrypted 46.4 GB/s 34.6 GB/s 32 Node NVMe 100 Gbe Network Read/GET Write/PUT Distributed 183.2 GB/s 171.3 GB/s Distributed/Encrypted 162 GB/s 114.7 GB/s Source: https://blog.min.io/scaling-minio-more-hardware-for-higher- scale/ Comparative results Source: https://blog.min.io/supermicro-cloud-dc-benchmark/
- 17. https://github.com/minio/warp Open source S3 performance benchmark tool developed and maintained by MinIO. Automated and coordinated testing of one or more hosts with one or more clients. Can be configured to run with different size objects, number of threads, TLS, encryption to represent your workload.
- 18. WARP S3 Benchmark on Cloud DC with NVMe
- 19. Video Demo
- 20. SuperMicro and MinIO: A High- Performance Partnership
- 21. 7 Enterprise Cloud / Web Storage / HCI AI / ML / HPC Form Factor Optimized All-Flash NVMe Top- Loading Scale Storage Max I/O Rack-mount Multi-node SuperStorage GPU SuperServers WI O DCO Blade Edge Computing MicroBlad e™ JBO F NF1/EDS FF 1U 20 Bay TwinPr o2® BigTwi n™ MicroCloud FatTwi n™ SuperBlade® GPU SuperBlade® 1U 4GPU 10U 16GPU 2U 6GPU 4U 20 single- width GPUs Workload Optimized Industry’s strongest server/storage product line Cloud and Edge infrastructure building blocks
- 22. Q&A
- 23. DISCLAIMER Super Micro Computer, Inc. may make changes to specifications and product descriptions at any time, without notice. The information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions and typographical errors. Any performance tests and ratings are measured using systems that reflect the approximate performance of Super Micro Computer, Inc. products as measured by those tests. Any differences in software or hardware configuration may affect actual performance, and Super Micro Computer, Inc. does not control the design or implementation of third party benchmarks or websites referenced in this document. The information contained herein is subject to change and may be rendered inaccurate for many reasons, including but not limited to any changes in product and/or roadmap, component and hardware revision changes, new model and/or product releases, software changes, firmware changes, or the like. Super Micro Computer, Inc. assumes no obligation to update or otherwise correct or revise this information. SUPER MICRO COMPUTER, INC. MAKES NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE CONTENTS HEREOF AND ASSUMES NO RESPONSIBILITY FOR ANY INACCURACIES, ERRORS OR OMISSIONS THAT MAY APPEAR IN THIS INFORMATION. SUPER MICRO COMPUTER, INC. SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE. IN NO EVENT WILL SUPER MICRO COMPUTER, INC. BE LIABLE TO ANY PERSON FOR ANY DIRECT, INDIRECT, SPECIAL OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE USE OF ANY INFORMATION CONTAINED HEREIN, EVEN IF SUPER MICRO COMPUTER, Inc. IS EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. ATTRIBUTION © 2020 Super Micro Computer, Inc. All rights reserved.