Migration of a computation cluster to Debian
0 likes911 views
The document discusses the migration of a computation cluster at the Institut de Génétique et Microbiologie to a Debian-based system. It outlines the existing hardware and software, the reasons for the migration, and the new virtualization and storage solutions implemented. The migration aimed to enhance resource management and scientific computing capabilities for researchers.
Migration of a computation cluster to Debian
- 1. Migration of a computation cluster to Debian Julien Cristau julien.cristau@logilab.fr November 25th, 2012 Mini-Debconf Paris 2012
- 2. Outline Context Hardware Existing software Solution 0/5
- 3. Context Institut de G´n´tique et Microbiologie (IGM): e e joint lab, CNRS & Universit´ Paris Sud e cluster part of the eBio platform, providing resources for scientists Logilab: software services python specialist, Debian-based scientific computing / simulation 1/5
- 4. Hardware Existing: 10 compute nodes 1 frontend 1 separate assembly node (lots of RAM and storage) 24-port 1Gb ethernet switch New: 2 hosts for virtualization and storage 2 more storage bays 2/5
- 5. Existing software Based on RHEL5 Sun Grid Engine (SGE) batch scheduler No shared storage No central management 3/5
- 6. Solution KVM host WEB Galaxy Compute Node Ganglia/Munin Compute Node Master Public network Condor/OAR/SGE Compute Node Provision. DHCP/tFTP/PXE/FAI Compute Node Salt Storage NFS ZFS Compute Node* ADMIN for BLAST assembly SAS SAN SAS SAS SAS SAS CLUSTER 4/5
- 7. Solution (2) One storage node using ZFS on Linux VM hosts for all services: provisioning, monitoring, frontend (submission host), databases, web server, . . . Nodes reinstalled with FAI (http://www.fai-project.org/) Configuration/deployment with Salt (http://www.saltstack.org/) 5/5