Available HPC resources at CSUC
Download as PPTX, PDF0 likes210 views
The document summarizes available HPC resources at CSUC, including hardware facilities, the working environment, development tools, and how to access services. The main systems are Canigó with 384 cores and 33 TFlops peak performance, and Pirineus II with 2,688 cores and 284 TFlops. Resources are managed by Slurm and available partitions include standard, GPU, and Intel KNL nodes. Users can access resources through RES projects or by purchasing compute units.
Available HPC resources at CSUC
- 1. Available HPC resources at CSUC Adrián Macía 17 / 03 / 2021
- 2. Summary Who are we? Scientific computing at CSUC Hardware facilities Working environment Development environment How to access our services?
- 3. Summary Who are we? Scientific computing at CSUC Hardware facilities Working environment Development environment How to access our services?
- 4. What is the CSUC?
- 5. What is the CSUC? Institutions in the consortium Associated institutions
- 6. What we do? Scientific computing Communications IT Infrastructures Procurements Scientific documentation management Joint purchases Electronic administration
- 7. Summary Who are we? Scientific computing at CSUC Hardware facilities Working environment Development environment How to access our services?
- 8. Science before scientific computing Theory Experiments
- 9. New paradigm: scientific computing Science needs to solve problems that, otherwise, can not be solved Development of new theoretical and technological tools Problem resolution that drives to new questions
- 10. But... what is Scientific Computing?
- 11. Science progress after Scientific Computing Theory Experiment Simulations
- 12. New and powerful tools change the scientific process
- 13. Usage examples: Engineering simulations Aerodynamics of a plane Vibrations in structures Thermal simulation of lighting systems Thermal distribution in a brake disc
- 14. Usage examples: Simulations in life sciences Interaction between SARS-CoV-2 spike protein and different surfaces Prediction of protein structures using Artificial Intelligence
- 15. Usage examples: simulations in material science Emergent structures in ultracold materials Graphene electronic structure Adsorbed molecules in surfaces
- 16. Main applications per knowledge area Chemistry and Materials Science Life and Health Sciences Mathematics, Physics and Engineering Astronomy and Earth Sciences
- 17. Software available In the following link you can find a detailed list of the software installed: https://confluence.csuc.cat/display/HPC KB/Installed+software If you don't find your application ask for it to the support team and we will be happy to install it for you or help you in the installation process
- 18. National Competence Center in HPC • EuroCC is an H2020 European project that want to establish a network of National competence center (NCC) in HPC, HPDA and AI in each country involved on the project https://www.eurocc-project.eu/ • The aim of the project is to promote the usage of scientific computing, mainly for SME's, but also in academia and public administration • We are participating in the Spanish NCC with other 7 institutions that also provide computing services. https://eurocc-spain.res.es/
- 19. NCC in HPC: who are we?
- 20. Demography of the service: users 47 research projects from 22 different institutions are using our HPC service. These projects are distributed in: • 11 Large HPC projects (> 500.000 UC) • 4 Medium HPC project (250.000 UC) • 11 Small HPC projects (100.000 UC) • 2 XSmall HPC projects (40.000 UC) • 19 RES projects
- 21. Demography of the service: jobs (I) Jobs per # cores
- 22. Demography of the service: jobs (II) CPUtime per number of cores
- 23. Demography of the service: jobs (III) % Jobs vs Memory/core
- 24. Top 10 apps per usage (2020)
- 25. Top 10 apps per usage (2020)
- 26. Wait time of the jobs % Jobs vs wait time
- 27. Wait time vs Job core count
- 28. Summary Who are we? Scientific computing at CSUC Hardware facilities Working environment Development environment How to access our services?
- 29. Hardware facilities Canigo(2018) Bull Sequana X800 384 cores Intel SP Platinum 6148 9 TB RAM memory 33,18 Tflop/s Pirineus II(2018) Bull Sequana X550 2688 cores Intel SP Platinum 6148 4 nodes with 2 GPU + 4 Intel KNL nodes 283,66 TFlop/s More info: https://confluence.csuc.cat/display/HPCKB/Machine+specifications
- 30. Canigó Shared memory machines (2 nodes) 33.18 Tflop/s peak performance (16,59 per node) 384 cores (8 cpus Intel SP Platinum 8168 per node) Frequency of 2,7 GHz 4,6 TB main memory per node 20 TB disk storage
- 31. 4 nodes with 2 x GPGPU • 48 cores (2x Intel SP Platinum 8168, 2.7 GHz) • 192 GB main memory • 4.7 Tflop/s per GPGPU 4 Intel KNL nodes • 1 x Xeon-Phi 7250 (68 cores @ 1.5 GHz, 4 hw threads) • 384 GB main memory per node • 3.5 Tflop/s per node Pirineus II
- 32. Standard nodes (44 nodes) • 48 cores (2x Intel SP Platinum 6148, 2.7 GHz) • 192 GB main memory (4 GB/core) • 4 TB disk storage per node High memory nodes (6 nodes) • 48 cores (2x Intel SP Platinum 6148, 2.7 GHz) • 384 GB main memory (8 GB/core) • 4 TB disk storage per node Pirineus II
- 33. High performace scratch system High performance storage available based on BeeGFS 180 TB total space available Very high read / write speed Infiniband HDR direct connection (100 Gbps) between the BeeGFS cluster and the compute nodes.
- 34. HPC Service infrastructure at CSUC Canigó Pirineus II
- 35. Summary of HW infrastructure Canigó Pirineus II TOTAL Cores 384 2 688 3 072 Total Rpeak (TFlop/s) 33.18 283.66 317 Power consumption (kW) 5.24 32.80 38 Efficiency (Tflop/s/kW) 6.33 8.65 8.34
- 36. Evolution of the performance of HPC at CSUC 10 x
- 37. Summary Who are we? Scientific computing at CSUC Hardware facilities Working environment Development environment How to access our services?
- 38. Working environment The working environment is shared between all the users of the service. Each machine is managed by GNU/Linux operating system (Red Had). Computational resources are managed by the Slurm Workload manager. Compilers and development tools availble: Intel, GNU and PGI
- 39. Batch manager: Slurm Slurm manages the available resources in order to have an optimal distribution between all the jobs in the system Slurm assign different priority to each job depending on a lot of factors … more on this after the coffee!
- 40. Storage units (*) There is a limit per project depending on the project category. Group I: 200 GB, group II 100 GB, group III 50 GB, group IV 25 GB Name Variable Availability Quota Time limit Backu p /home/$USER $HOME Global 25- 200 GB (*) Unlimited Yes /scratch/$USER/ − Global 1 TB 30 days No /scratch/$USER/tmp/$J OBID $SCRATCH / $SHAREDSCRAT CH Global 1 TB 7 days No /tmp/$USER/$JOBID $SCRATCH / $LOCALSCRATC H Local node − Job execution No
- 41. Choosing your architecture: HPC partitions // queues We have 5 partitions available for the users: std, std-fat, gpu, knl, mem working on standard, standard fat, gpu, knl or shared memory nodes. Each user can use any of them (except RES users that are restricted to their own partitions) depending on their needs … more on this later...
- 42. Do you need help? http://hpc.csuc.cat
- 43. Documentation: HPC Knowledge Base http://hpc.csuc.cat > Documentation
- 44. Problems or requests? Service Desk http://hpc.csuc.cat > Support
- 45. Summary Who are we? Scientific computing at CSUC Hardware facilities Working environment Development environment How to access our services?
- 46. Development tools @ CSUC HPC Compilers available for the users: • Intel compilers • PGI compilers • GNU compilers MPI libraries: • Open MPI • Intel MPI • MPICH • MVAPICH
- 47. Development tools @ CSUC HPC Intel Advisor, VTune, ITAC, Inspector Scalasca Mathematical libraries: • Intel MKL • Lapack • Scalapack • FFTW If you need anything that is not installed let us know
- 48. Summary Who are we? Scientific computing at CSUC Hardware facilities Working environment Development environment How to access our services?
- 49. How to access to our services? If you are not granted with a RES project or you are not interested in applying for it you can still work with us. More info in https://www.csuc.cat/ca/supercomputacio/soll icitud-d-us
- 50. HPC Service price Academic project¹ Initial block - Group I: 500.000 UC 8.333,33 € - Group II: 250.000 UC 5.555,55 € - Group III: 100.000 UC 3.333,33 € - Group IV: 50.000 UC 1.666,66€ Additional 50.000 UC block - When you have paid for 500.000 UC 280 €/block - When you have paid for 250.000 UC 665 €/block - When you have paid for 100.000 UC 945 €/block - When you have paid for 40.000 UC 1.835 €/block DGR discount for catalan academic groups -10 %
- 51. Accounting HPC resources There are some considerations concerning the accounting of HPC resources: If you want to use the gpu partition you need to allocate a full socket (24 cores) at minimum. This is imposed by the fact that we don't want two different jobs sharing the same GPU If you want to use the KNL nodes you need to allocate the full node (68 cores). Same reason that the previous case Each partition has an associated default memory per core. If you need more than that you should ask for it and the system will assign more cores (with its associated memory) for your job.
- 52. Access through RES project You can apply for a RES (red española de supercomputación) project asking to work at CSUC (in pirineus II or canigo). More information about this on https://www.res.es/es/acceso-a-la-res
- 53. Questions?
- 54. MOLTES GRÀCIES http://hpc.csuc.cat Cristian Gomollón Adrián Macía Ricard de la Vega Ismael Fernàndez Víctor Pérez