DDN EXA 5 - Innovation at Scale
Download as PPTX, PDF1 like1,319 views
The document discusses growing data and performance pressures in high performance computing (HPC) and artificial intelligence (AI) workloads. It summarizes several customer use cases and how they are addressing these pressures through solutions like DDN's EXAScaler parallel file system, which provides comprehensive flash management, security, data protection, user and workload monitoring, and hybrid storage capabilities to support the increasing demands of HPC and AI workloads.
DDN EXA 5 - Innovation at Scale
- 1. DDN Confidential DDN ©2019 DataDirect Networks, Inc.
- 2. DDN Confidential DDN ©2019 DataDirect Networks, Inc. Data Volumes and I/O Pressures are Compounding Quantum Monte Carlo LHC Plasma Physics SKA LSST Nanopore 100+TB/s 200TB/day 100TB/s 1GB/s CCTV AD data collection
- 3. DDN Confidential DDN ©2019 DataDirect Networks, Inc. AI-HPC Simulation Big Data Deep Learning Materials Science APS Data Analysis Drug Response Prediction Cosmology HEP Data Analysis Scientific Image Classification Molecular Dynamics LSST Data Analysis Scientific Text Understanding Nuclear Reaction Modelling SKA Data Analytics Materials Property Design Combustion Metagenome Analysis Gravitational Lens Detection Quantum Computer Simulation Graph analysis Feature Detection in 3D Climate Modelling Virtual compound library Street Scene Analysis Power Grid Neuroscience data analysis Organism Design Discrete Event Simulation Genome Pipelines State Space Prediction Fusion Reactor Simulation Persistent Learning Transportation Networks Hyperspectral patterns Traditional HPC Large Scale Numerical Simulation Scalable Data Analytics Deep Learning
- 4. DDN ©2019 DataDirect Networks, Inc. XXII Computer Vision and AI for Smart Retail Flow management. People counting. Product counting. Autonomous check-out. Staff access control. One system, endless data. ► Intelligence built on a full network of cameras ► Object identification and classification in real-time ► Skeletal and facial human recognition ► One system, multiple uses, in real-time: • Store Analytics – flow management to optimize resources • Frictionless Shopping – seamless in-store experience to allow staff to focus on customers • People Safety – create safer environments and decrease human risk • Supply Chain Control – enhance quality and productivity Item 07482
- 5. DDN ©2019 DataDirect Networks, Inc. ToMMo Architecture for Genomics Acceleration Tokyo Medical Megabank Organization Accelerating genomic medicine research from a more robust Biobank by providing wider access and better performance to an expanded storage infrastructure. ► Shared computational resources made up of a 300 compute node cluster, 3 shared memory compute nodes, and 3 NVIDIA DGX-1 systems ► Mixed Infiniband and 40/10 Gbps networks to provide capabilities for different needs ► Analysis system (16PB) ► Public data system (7PB EXA) ► 2 x Information distribution systems (4PB EXA, 2PB GRID) ► DGX-1 systems are using EXA to run Parabricks genomics analysis at a much accelerated rate DDN is expected to continue to proactively propose solutions leveraging the newest technology to accelerate research. Compared to when the system was initially implemented, the organization carries heavier burden of social responsibility. Though the system’s original intended use was for analysis, it has become a system upon which many researchers have built their projects. GRID 16PB Analysis 7PB Public Data 6PB Info Distribution Diverse Compute Resources CPU GPU
- 6. DDN ©2019 DataDirect Networks, Inc. TACC Architecture and NVMe acceleration ► The primary computing system (Dell/EMC, Intel, Mellanox HDR) focussed on high precision performance ► Initial configuration of the system will have 8,008 available compute nodes. ► second subsystem focused on single precision streaming-memory computing. ► multiple storage systems ► interfaces to cloud and archive systems ► application nodes for hosting virtual servers Scratch/Work Disk based Filesystem 50PB, 240GB/s NVMe Acceleration 3PB, 1.2TB/s Capability: LNET routers, fs scanning, user and fs monitoring, project quota, data management, security, end-to-end data protection Mixed double precision cpu and single precision compute computational capability that makes it possible for investigators to tackle much larger and more complex research challenges across a wide spectrum of domains.
- 7. DDN ©2019 DataDirect Networks, Inc. So where are the Pressures? ► Data Volumes going up (HPC and IA) ► Flash Pricing Going down.. Still some market instabilities, But HDD prices going down too! ► Performance requirements going up (of course) • Ingest (AI (new scenarios) and HPC ) • Mixed IO (HPC and AI) • Single client IO and mmap() IO (AI) ► Uptimes and User Management requirements up (HPC and AI) ► Feature Set going up, Access and Protocols (primary driver from AI) ► Protocol access requirements going up (AI) ► Security going up - Encryption, External KEY Mgmt (general trend) ► MultiCloud demand rising (mainly AI, some HPC)
- 8. DDN Confidential DDN ©2019 DataDirect Networks, Inc. EXA5 Capability for the AI-HPC DataCenter EXTREME EDGE AI Move, ManageImport, Preprocess Tune, DeployInfer, PredictClassify, Tag Train, Score Management, Oversight, Governance HA, Uptime Backup, Archive, Sync Scalable Filesystem Security NFS CIFS HDFS GPU/Container Integration DATA MANAGEMENT Workflows Grand Challenge Big Data Multi-Physics HPC Mesh RefinementNoSQL INFRASTRUCTURE GPU CPU GPUs & Novel Processors CPU Broader Range of Credible CPUsEDR, HDR, Ethernet and More! IPU Cloud, Hybrid Cloud, MultiCloud User & Service Management More access protocols Strong Data Protection Comprehensive Security Full Spectrum Performance User and Workload management Complete Data Management Multiple Cloud options
- 9. DDN Confidential DDN ©2019 DataDirect Networks, Inc. POSIX NFS SMB HDFS S3 NVMe, SSD, HDD, TAPE, S3 EXA 5 Comprehensive Security Full Spectrum Performance Complete Data Management Innovation at Scale Google Cloud and DDN have developed an in-cloud file system suitable for HPC: DDN’s EXAScaler, a parallel file system designed to handle high concurrency access patterns to shared data sets. * Strong Data Protection User & Workload Management
- 10. DDN Confidential DDN ©2019 DataDirect Networks, Inc. EXA5 optimizations for AI Workloads ► DDN AI200 with 20 x 1.6TB PCI NVMe devices • 32x clients, 2x 12 CPU cores, 128GB memory ► EXA5 37% higher 4K IOPS than 2.10.7 • 70% of peak IOPS efficiency vs. RAM-only workload ► EXA5 is better than 2.10.7 at every IO size
- 11. DDN Confidential DDN ©2019 DataDirect Networks, Inc. COMPREHENSIVE FLASH MANAGEMENT ► Complete flash management requires integration with the Flash capability to manage garbage collection cycles ► EXA5 integrations with DDN’s SFA architecture to deliver unmap operations from the filesystems allowing total control over NAND Flash Devices ► Manage Flash Performance Behavior with EXA5 FSTRIM
- 12. DDN Confidential DDN ©2019 DataDirect Networks, Inc. Managing AI Workloads EXA5 MASS STORAGE INGEST Stream Engine Spark CLASSIFICATION MODEL DEVELOPMENT Ingest Clean Train Validate Infer Access Pattern Sequential Sequential or Random Random Sequential Sequential Access Type Write Read / Write Read Read Read Concurrency Varies by number of sources High High High Varies by use case Size of File Metadata small; Files vary by source Vary by source Small access within large files Varies Varies by use case
- 13. DDN Confidential DDN ©2019 DataDirect Networks, Inc. Comprehensive User & Workload Monitoring ► Out of the box access to your filesystem or user’s file distribution profile ► How many files, size, statistics ► Real-time access to granular properties of live workloads (jobstats) ► Simpler diagnostics of client issues and networking challenges. ► Easy identification of problem jobs (high metadata, dominating throughput, etc) File Size Distribution User 1 Metadata User 1 Throughput FileCountOpsGB/s
- 14. DDN Confidential DDN ©2019 DataDirect Networks, Inc. All Round Security, Multi-Tenancy, Audit and Encryption Group A Group C ► Managing Multiple Tenants in complex security environments ► Partition your users strictly at a sub filesystem ► Full FIPs/KMIP support for formal certification of encryption ► Access Control ► Audit Facility - records of all EXAScaler accesses using efficient Lustre changelog • Record file system namespace & metadata events in secure log • Record Failed access attempts Group B AUDIT
- 15. DDN Confidential DDN ©2019 DataDirect Networks, Inc. EXA5 NFS and CIFS Data Services Consistency and locking semantics with Linux/Unix and Windows clients Export Lustre namespace through CIFS and NFS gateway Authentication methods for UNIX passwords, LDAP and Microsoft AD High Availability Horizontal scalability • Formal, Qualified and Tested NAS Solutions for DDN EXAScaler • Performance Class NFS and CIFS for scalable Filesystem export
- 16. DDN Confidential DDN ©2019 DataDirect Networks, Inc. EXA5 END TO END DATA PROTECTION End-to-EndProtection APPLICATION/USERSPACE EXA5 CLIENT NETWORK EXA5 SERVER Integrated, end-to-end data protection using T10PI/DIX Standard Fully Transparent to Applications Supported by EXAScaler from the client through to Drives Minimal Performance Impact ► Integrated, end-to-end data protection using T10PI/DIX Standard ► Fully Transparent to Applications ► Supported by EXAScaler from the client through to Drives ► Minimal Performance Impact T10-PI
- 17. DDN ©2019 DataDirect Networks, Inc. ►Fast and efficient data management, built-in. ►Manage your most active data into scale-out Flash ►Automatic control of free space on Flash ►Quickly respond to changing demand ►Efficient namespace scanning ►Built in To EXA5 - No external servers EXA5 Native Scanning Engine
- 18. DDN Confidential DDN ©2019 DataDirect Networks, Inc. Hybrid at Scale Mixed I/O Mixed file sizes Stratagem manages Hot data automatically tiering between flash and HDD Small file data automatically goes into metadata Fewer RPCs Lower latencies ► Flexible Small File Acceleration brings Simple, Powerful Application of Flash and Flash optimized IO to Scalable Hybrid Storage Systems ► Expand the domain of file sizes accelerated simply from a few KB to Multiple MB ScaleOutScaleOut
- 19. DDN Confidential DDN ©2019 DataDirect Networks, Inc. POSIX NFS SMB HDFS S3 NVMe, SSD, HDD, TAPE, S3 EXA 5 Comprehensive Security Full Spectrum Performance Complete Data Management Innovation at Scale Google Cloud and DDN have developed an in-cloud file system suitable for HPC: DDN’s EXAScaler, a parallel file system designed to handle high concurrency access patterns to shared data sets. * • Data on Metadata • File Level Redundancy • Hot Pools • Scale Out Metadata • NVMe Optimizations • Stratagem Scanner • Automatic Tiering • DataFlow Integration • Multi-Tenancy • Audit • Kerberos • Access Control Strong Data Protection • DeClustered RAID • End-to-End data integrity • Partial Rebuilds • Enclosure Redundancy
- 20. DDN Confidential DDN ©2019 DataDirect Networks, Inc. Innovation, Stability, Partnership
Editor's Notes
- #18: DDN Enables and Accelerates Deep Learning DL is about growing autonomous capability Learning from a very large amounts of data In many ways, data is new source code Collect and access Success of DL development is access to large data sets The size, diversity and quality of data set critical Easy data discovery Store data in single location, accessible everywhere Presented unified interface, interoperable, standard No modification to application or container