CSCfi Computing Services 12/2014
Download as PPTX, PDF1 like989 views
The document discusses CSC's high performance computing capabilities and developments from 2012-2014. It summarizes that CSC upgraded its Sisu and Taito systems in 2014 with new Intel Haswell CPUs, increasing cores by 50% and reducing energy usage. The upgrades boosted Sisu's performance to 1700 TFlops, making it the 37th most powerful system in the world. CSC now provides a total of 2.54 PFlops of computing power and is the most powerful academic computing facility in the Nordic countries.
CSCfi Computing Services 12/2014
- 1. CSC Computing Services Olli-Pekka Lehto Development Manager Computing Platforms olli-pekka.lehto@csc.fi @ople December 11th 2014
- 2. CSC Computing Capacity 1989–2014 2
- 3. CSC Computing Capacity 2012-2014 244 240 1700 180 600 3000 2500 2000 1500 1000 500 0 2012 2013 2014 Bull Taito Sisu Vuori Louhi Phase 1 installed Louhi retired Phase 2 installed Vuori retired 3,4x 5,6x 19,2x
- 4. Total performance: 2.54 PFlop/s CSC is the most powerful academic computing facility in the Nordics
- 5. CSC Computing Services Performance Capacity Accelerated Cloud Hosting Sisu Massive parallelism Fast interconnect Taito General use Large memory >100 applications Taito extension Visualization Special codes Nvidia GPU Intel Xeon Phi cPouta Build your own Openstack IaaS Kajaani Espoo Efficient and secure datacenters Virtual and physical servers Fast parallel storage Backup Archiving Storage Services
- 6. CSC Computing Services Performance Capacity Accelerated Cloud Hosting Sisu 40512 cores 1700 TFlops Taito 18880 cores 600 TFlops Taito extension 76 Nvidia K40 GPU 90 Intel Xeon Phi 7120X 240 TFlops cPouta Dynamically provisioned from Taito Kajaani Espoo >4PB, ~ 100GB/s Storage Services
- 7. New in 2014: Xeon Haswell E5 CPUs Intel Xeon E5-2690v3 2,6GHz – 12 cores/CPU (+50%) – AVX2 instructions (2x max flops/GHz) – DDR4 memory – “Energy-to-solution” at best 1/3 vs. Sandy Bridge We are one of earliest adopters – Sisu upgraded 7/2014 – Taito upgraded 12/2014
- 8. Sisu Cray XC päivitys 8 new cabinets Haswell CPUs More memory per node >7x performance ~2x energy consumption 40512 cores 108TB memory 680kW 15384kg
- 9. Sisu – Cray XC40 Designed for computationally intensive tasks 40512 cores, 64GB RAM / node #37 on the Nov 14 Top500 -list Aries-interconnect with 7TB/s bisection BW Cray development tools Comprehensive set of scalable applications
- 10. Cray XC blade 4 dual CPU nodes (96 cores) 64GB RAM per node Aries Router Net Power (500GB/s switching capability)
- 11. Blade in XC Rack 48 blades 384 CPUs 4608 cores
- 13. Aries Interconnect Topology All-to-all network between groups 2 dimensional all-to-all network in a group Source: Robert Alverson, Cray Hot Interconnects 2012 keynote Optical uplinks to inter-group net CSC presentation 13
- 14. Aries Bisection Bandwidth 9 000 000 x 1,75 x Average European consumer IP traffic in 2013 OR 1080p Netflix streams ~7 TB/s =
- 15. Taito HP cluster for general use – 576 dual-CPU 8 core E5-2670 (Sandy Bridge) 64GB RAM per node – 400 dual-CPU 12 core E5-2690v3 (Haswell) 128GB RAM per node New HP Apollo 6000 chassis and blades – Big memory nodes 10 x 256GB ; 2 x 1,5TB 56Gbit/s FDR InfiniBand –interconnect Large selection of applications taito-shell for instant interactive use
- 16. Taito Extension Bull DLC 715 – Direct warm-water cooling Special processors for computing – 72 Nvidia Tesla K40 GPGPU – 90 Intel Xeon Phi 7120X High-performance and energy efficient – Porting and optimization of applications needed GPUs can be used for visualization – For example by using VirtualGL Connected to the Taito cluster
- 17. Taito Extension
- 18. Energy-efficiency of Systems 3.5 3 2.5 2 1.5 1 0.5 0 Vuori Sisu P1 Taito P1 Sisu P2 Taito P2 Bull GFlop/W
- 19. Taito ja Sisu user interfaces Unix shell & X-forwarding NoMachine NX –virtual desktop (NX Web-client in beta testing) Scientists’ User Interface https://sui.csc.fi
- 20. cPouta Cloud IaaS cloud service for HPC Use cases – Running HTTP and other servers – Non-CentOS Linux or Windows OS needed – Superuser access needed – Agile service development (“DevOps”) Tuned for HPC – Provisioned from Taito nodes – Powerful CPUs, interconnect, storage Simple web interface, CLI, REST API https://research.csc.fi/pouta-iaas-cloud
- 21. Creating an Instance in cPouta
- 22. Creating a Virtual Volume in cPouta
- 23. Storage Services HPC Storage (~4PB, ~100GB/s) – Lustre parallel filesystem – DDN SFA10k and SFA12k storage arrays – Capacity and performance scalable Cloud storage (in 2015) – Ceph filesystem – Software-defined storage (SDS) – Block and object storage Archive (iRODS), tape backup
- 24. Kajaani Datacenter 30MW power PUE 1.05-1.2 3000m2 floorspace 99% free cooling
- 25. Modular Datacenter (MDC): Easily Expandable, Highly Efficient
- 26. Interior view of the MDC
- 27. Cooling Technologies Water-air hybrid (Sisu) Air (Taito) Direct warm-water (Bull Taito extension) PUE: 1,2 PUE: 1,05 PUE: 1,02
- 28. Why CSC? High-performance, latest technologies Secure environment (ISO27001:2013) Finnish, non-profit organization Ecologically sustainable infrastructure Competitive and simple pricing model Excellent network connectivity Everything under one roof – Cloud, traditional HPC, visualization, accelerators – Various storage solutions, EUDAT, RDA – Consulting, training, porting, optimization
- 29. ISO27001:2013 certification Reflects our commitment to security – Risk management – Leadership – Technical solutions and documentation – Continual improvement – Recovery planning – Security as part of company culture Covers nearly all ICT services and datacenters – Cloud services to be certified in early 2015
- 30. Leveraging Best-of-breed Open Source Software Deployment & Clustering Cloud Operating Systems Monitoring Storage Queuing Logstash Graphite
- 31. Near future development cPouta improvements, including: – Oversubscribed instances (WWW-servers etc) – Docker-support – ISO27001:2013 certification New ePouta service – Productization of Biomedinfra – Secure computing for organizations – No direct visibility to Internet (VPN / OPN) – Possible to extend your local resources seamlessly Services for data-intensive computing – Hadoop/MapReduce optimized systems – SSD storage
- 33. Backup slides Intro to HPC architecture
- 34. Supercomputers in Olden Days
- 35. Supercomputers Today Commodity technologies – Server clusters – Linux – Ethernet, InfiniBand – x86 Proprietary solutions in very high-end – BlueGene, Cray, NEC Cloud services on the rise – Especially for modest compute needs Use is constantly spreading to new fields – Skilled people needed!
- 36. Basic Supercomputer Architecture Edustasolmu Frontend nodes Interconnect network Compute nodes Storage servers Storage system Internet Management network Hallintasolmut Management nodes Management nodes §
Editor's Notes
- #25: Kajaani datacenter is housed in an old paper warehouse There is plenty of local hydroelectric power (30MW) and the climate is nice and cool for free air cooling nearly year-round