IBM general parallel file system - introduction
7 likes12,387 views
The document provides information about IBM's General Parallel File System (GPFS) 3.5 and introduces the GPFS Storage Server (GSS). It summarizes that GPFS is a scalable high-performance file management system that can scale from 1 to 8192 nodes. The GSS is a new storage solution using IBM servers and JBOD storage to provide high capacity and performance storage in a scalable building block approach. The GSS has no storage controllers and provides a single integrated storage solution built on GPFS software.
IBM general parallel file system - introduction
- 1. IBM General Parallel File System (GPFS™) 3.5 and GSS Introduction Karl Hansen, Nordic HPC/Technical Computing Sales Manager IBM Systems & Technology Group A New Era in Technical Computing: Powerful. Comprehensive. Intuitive. IBM Confidential
- 2. Technical Computing: Powerful. Comprehensive. Intuitive The IBM General Parallel File SystemTM (GPFSTM) Shipping since 1998 Extreme Scalability File system 263 files per file system Maximum file system size: 299 bytes Production 19PB file system Number of nodes 1 to 8192 Proven Reliability Manageability No special nodes Add/remove nodes and storage on the fly Rolling upgrades Administer from any node Data replication Snapshots Integrated tiered storage Storage pools Quotas Policy-Driven automation Clustered NFS SNMP monitoring TSM / HPSS (DMAPI) File system journaling © 2012 IBM Corporation
- 3. Technical Computing: Powerful. Comprehensive. Intuitive IBM General Parallel File System (GPFS) IBM General Parallel File System (GPFS) is a scalable highperformance file management infrastructure for AIX®, Linux® and Windows™ systems. 3 A highly available cluster architecture Concurrent shared disk access to a global namespace Capabilities for high performance parallel workloads © 2012 IBM Corporation
- 4. Technical Computing: Powerful. Comprehensive. Intuitive File data infrastructure optimization Databases GPFS enables: A global namespace across platforms High performance common storage Eliminating copies of data Improved storage utilization Simplified file management 4 Connections SAN TCP/IP InfiniBand File servers Management Centralized Monitoring Automated File Mgmt Availability Data Migration Replication Backup Backup and archive Application servers © 2012 IBM Corporation
- 5. Technical Computing: Powerful. Comprehensive. Intuitive How is GPFS different? Massive namespace support GPFS SAN Centrally deployed, managed, backed up and grown. Seamless capacity and performance scaling All features are included. All software features: snapshots, replication and multi-site connectivity are included in the GPFS license. With no license keys except for client and server to add on, you get all of the features up front. 5 © 2012 IBM Corporation
- 6. Technical Computing: Powerful. Comprehensive. Intuitive Network-based block input output Application data access on network attached nodes is exactly the same as a SAN attached node. GPFS transparently sends the block level IO request over a TCP/IP network. NSD clients LAN NSD servers GPFS SAN SAN Why? Enable virtually seamless multi-site operations Reduce costs for data administration Provide flexibility of file system access Establish highly scalable and reliable data storage Future protection by supporting mixed technologies 6 © 2012 IBM Corporation
- 7. Technical Computing: Powerful. Comprehensive. Intuitive IBM General Parallel File System (GPFS™) – History & Evolution GPFS 2.1-2.3 First called GPFS HPC GPFS General File Serving Standards Portable operating system interface (POSIX) semantics -Large block Directory and small file perf Data management 1998 Linux® Clusters (Multiple architectures) IBM AIX® Loose Clusters GPFS 3.3 GPFS 3.4 GPFS 3.5 HPC Virtual Tape Server (VTS) GPFS 3.1-3.2 Information lifecycle management (ILM) Restricted admin functions Enhanced Windows cluster support - Homogenous Windows server Caching via Active File Management (AFM) Research Visualization Digital media Seismic Weather Life sciences 32 bit /64 bit Inter-op (IBM AIX & Linux) GPFS Multicluster Storage pools File sets Policy engine Ease of administration Multiplenetworks/ RDMA Distributed token management GPFS over wide area networks (WAN) Large scale clusters thousands of nodes 2002 Windows 2008 Improved installation New license model Improved snapshot and backup Improved ILM policy engine GSS - GPFS Storage Server Performance and scaling improvements GPFS File Placement Optimizer (FPO) Enhanced migration and diagnostics support NFS v4 support 2005 Multiple NSD servers Small file performance 2006 2009 2010 2012 © 2012 IBM Corporation
- 8. Technical Computing: Powerful. Comprehensive. Intuitive A Disruptive HPC Play - GPFS Storage Server (GSS) At a Glance The New High Capacity, High PerformanceStorage Solution New Storage Solution fulfilled exclusively through IBM Intelligent Cluster High capacity, High performance, High Value offering Product importance Single, integrated, fully supported IBM solution Built to leverage a strong GPFS software market High capacity, scalable building-block approach - performance and capacity increases as you add multiple building blocks Cost competitive Extreme data integrity and reduced latency with faster rebuild times 8 © 2012 IBM Corporation
- 9. Technical Computing: Powerful. Comprehensive. Intuitive GPFS Storage Server - Product Description GSS is a new storage solution fulfilled exclusively through the IBM Intelligent Cluster Two x3650 servers combined with either four or six JBODs Two models: GSS 24 and GSS 26 – GSS 24 (Entry): 4 JBODs – starts at nearly 500TB of storage space – GSS 26 (Main): 6 JBODs – starts at over 700TB of storage space Data striped across all disks 2 and 3 TB options 10GbE or FDR Infiniband interconnects, or both! Scalable Building Block approach to HPC Storage - performance and capacity increase as you add multiple building blocks Complete Storage Solution with no Storage controllers GSS 24 Model De-Clustered RAID Techniques Built on GPFS software Industry standard components – Leverages standard components including x3650s, NetApp JBODs, LSI SAS cards and lots of HDDs, and Intelligent Cluster fulfillment as a single, integrated, fully supported IBM © 2012 IBM Corporation
- 10. Technical Computing: Powerful. Comprehensive. Intuitive IBM System x GPFS Storage Server provides a comprehensive storage solution with a scalable building block approach Storage solution includes Data Servers, Disk (2TB or 3TB NL-SAS, SSD), Software, InfiniBand / Ethernet with no Storage Controllers x3650 M4 Server JBOD Disk Enclosure GSS 24: Light and Fast GSS 26: HPC Workhorse High-Density HPC Option 2 3650 servers + 4 JBOD 20U rack 10 GB/Sec 2 3650 servers + 6 JBOD Enclosures, 28U 12 GB/sec 6 3650 servers + 18 JBOD 2 - 42U Standard Racks 36 GB/sec © 2012 IBM Corporation
- 11. Technical Computing: Powerful. Comprehensive. Intuitive Why you should care… Affordably Scalable Building Block approach to HPC Storage Performance and capacity increases as you add multiple buidling blocks Start Small and Scale via incremental additions Add capacity AND bandwidth Cost competitive Extreme data integrity and reduced latency Built on GPFS Fewer parts means lower cost Leverages System x servers and Commercial JBODs • Fast rebuild times and industry-leading performance • Better sustained performance • Industry-leading throughput using efficient DeClustered RAID Techniques • The Infrastructure for Global Technical Computing Data Management Single, integrated, fully supported IBM solution Fully integrated, fully supported Complete Storage Solution with no Storage controllers Easy to order through Intelligent Cluster 11 © 2012 IBM Corporation
- 12. Technical Computing: Powerful. Comprehensive. Intuitive A data management portfolio for Technical Computing Focus: Managed Building Block Focus: Ease of Use Reliability Focus: Raw Raw Performance I/O Bandwidth IBM Data Management Leadership Government High GPFS Petroleum Storage Server End Research Media/Ent. Financial SONAS Bio/Life Science CAE Services Higher End University DCS3700 DCS3700+ Direct Attached Smaller (DS3500 + V3700) Installations IBM Tape, Tivoli Storage Manager, and HPSS © 2012 IBM Corporation
- 13. Technical Computing: Powerful. Comprehensive. Intuitive What makes this different Clients Clients FDR IB 10 GbE File/Data Servers NSD File Server 1 x3650 NSD File Server 2 Custom Dedicated Disk Controllers JBOD Disk Enclosures NSD File Server 1 GPFS Native RAID Migrate RAID and Disk Management to Standard File Servers! NSD File Server 2 GPFS Native RAID JBOD Disk Enclosures © 2012 IBM Corporation
- 14. Technical Computing: Powerful. Comprehensive. Intuitive System x Smarter Systems for a Smarter Planet 14 © 2012 IBM Corporation
- 15. Technical Computing: Powerful. Comprehensive. Intuitive For more information ibm.com/systems/software/gpfs Email gpfs@us.ibm.com or contact your IBM Representative © 2012 IBM Corporation