SlideShare a Scribd company logo
Beowulf Cluster, building cheap powerful hpc
By Akash Sahoo
Beowulf
www.top500.org
Speed
•16 Pentium Ppro
200Mhz each
•2GB Memory
•1.25 Glops
•130 Mflops individual
speed
•Made by Caltech in ‘96
FLOPS (or flops or flop/s, for floating-point operations
per second) is a measure of a computer's performance,
especially in fields of scientific calculations. It is
instruction per second
i7-920 has 40Gflops
In 2007 a Microwulf Cluster (less nodes) with just 4
nodes gave a performance of 26.25 Gflops (Wikipedia)
Why Speed
Computer simulations are far cheaper and faster than
physical experiments
Computers can solve a much wider range of problems
than specific laboratory equipment can
Computational approaches are only limited by
computer speed and memory capacity, while physical
experiments have many practical constraints
Uses: Predictive Modeling and Simulations,
Engineering Design and Automation, Military,
Visualization
Why Beowulf
Commodity Hardware, No need to throw old PCs !!
Ease of addition of nodes
Work if nodes fail
VVVV Cheap (Supercomputer costs million)
World Fastest is a cheap Beowulf Cluster
Can be used in Homes, educational Institutes
Students will study and write parallel, optimized code
Simulation Weather, BLAST (comparing primary
biological sequence information), BIONIC (Cancer),
distributed Matlab, DNA, Image processing, Web server,
own software
Software can run on one/many node as specified
INDIA has outdated supercomputer - PARAM
Parallel Computing
Shared Memory NFS
Message Passing
Efficiency
Latency
Topology
Admin and maintainace
Redundancy
Workload, Unbalancing, Arch
Energy, cost, scalable
Building It
Install Linux in each node (MPICH2), Can be build
on Windows (PVM)
SSH Equivalence
SSH Hostkeys – RSA, DSA
Installation of PDSH
Setting NFS
Creation of MPI users
Installation of compat-g77,gcc
MPI install, LINPACK Benchmark
OSCAR, ROCKS, SCARY- maintainance
Mpdboot –n 3
Starting cluster
Mpdtrace
Know which
nodes are
running
Mpicc
Compiling
prog
Mpiexec –n
1 ./prog
Execute a
sample prog
on 1 node
Same prog on 2
nodes
Same prog on 3
nodes
Improvements
Add fibre optic cable
Dual Lan card
Powerful Gateway, Router
Faster computers – Cost High, Performance High
Keep in mind-Total memory, Memory bandwidth,
Storage bandwidth, System interconnect latency,
The effect of interconnect topologies such as meshes,
cubes, and fat trees on application design
SSD Harddisk, RAID, Graphics Card
Notes
Gave a performance boost of > 200% on 3 nodes
Originality : Security both RSA, DSA. PDSH makes
run commands in all nodes at once
Commercialization: giving manuals on prog, software,
setting cluster, maitainace
Proj done in 3rd
year
Patents
 Dynamically allocated cluster system
 Charles Evert Prael  ;  (Patent No: US5093912)
The invention provides a system and method for managing clusters of parallel processors 
for use by groups and individuals requiring supercomputer level computational power.
Given Scheduling algo PSB, job queue… 28 claims mostly on scheduling
 Method and apparatus for securing and managing cluster
David Bruce Kumhyr 2002
Reference
Beowulf Cluster Computing with Linux (2001)- 
Thomas Sterling
White papers by HP - Considerations in Specifying 
Beowulf Clusters 2000
MPI: The Complete Reference , Ewing Lusk 
?????????
THANK YOU

More Related Content

PPTX
«Training Deep Learning Models on Multi-GPUs Systems», Dmitry Spodarets.
PDF
Get Lower Latency and Higher Throughput for Java Applications
PDF
Exploring the Performance Impact of Virtualization on an HPC Cloud
PPTX
Sharing High-Performance Interconnects Across Multiple Virtual Machines
PPTX
GPU Programming with CUDA
PDF
Scaling Apache Pulsar to 10 Petabytes/Day
PDF
Where Did All These Cycles Go?
PDF
HPC Cloud: Clouds on supercomputers for HPC
«Training Deep Learning Models on Multi-GPUs Systems», Dmitry Spodarets.
Get Lower Latency and Higher Throughput for Java Applications
Exploring the Performance Impact of Virtualization on an HPC Cloud
Sharing High-Performance Interconnects Across Multiple Virtual Machines
GPU Programming with CUDA
Scaling Apache Pulsar to 10 Petabytes/Day
Where Did All These Cycles Go?
HPC Cloud: Clouds on supercomputers for HPC

What's hot (18)

PDF
Bruno Silva - eMedLab: Merging HPC and Cloud for Biomedical Research
PDF
Kernel Recipes 2017 - What's new in the world of storage for Linux - Jens Axboe
PDF
Rust, Wright's Law, and the Future of Low-Latency Systems
PDF
Stig Telfer - OpenStack and the Software-Defined SuperComputer
PDF
Crimson: Ceph for the Age of NVMe and Persistent Memory
PDF
DB Latency Using DRAM + PMem in App Direct & Memory Modes
PDF
RISC-V on Edge: Porting EVE and Alpine Linux to RISC-V
PDF
Kernel Recipes 2017 - 20 years of Linux Virtual Memory - Andrea Arcangeli
PDF
Vanquishing Latency Outliers in the Lightbits LightOS Software Defined Storag...
PDF
Ceph Day Netherlands - Ceph Management and Monitoring with openATTIC 3.x
PDF
Let’s Fix Logging Once and for All
PDF
Unikraft: Fast, Specialized Unikernels the Easy Way
PDF
RADOS improvements and roadmap - Greg Farnum, Josh Durgin, Kefu Chai
PDF
PDF
Data Structures for High Resolution, Real-time Telemetry at Scale
PDF
Utilizing AMD GPUs: Tuning, programming models, and roadmap
PDF
Expectations for optical network from the viewpoint of system software research
PDF
Achieving the Ultimate Performance with KVM
Bruno Silva - eMedLab: Merging HPC and Cloud for Biomedical Research
Kernel Recipes 2017 - What's new in the world of storage for Linux - Jens Axboe
Rust, Wright's Law, and the Future of Low-Latency Systems
Stig Telfer - OpenStack and the Software-Defined SuperComputer
Crimson: Ceph for the Age of NVMe and Persistent Memory
DB Latency Using DRAM + PMem in App Direct & Memory Modes
RISC-V on Edge: Porting EVE and Alpine Linux to RISC-V
Kernel Recipes 2017 - 20 years of Linux Virtual Memory - Andrea Arcangeli
Vanquishing Latency Outliers in the Lightbits LightOS Software Defined Storag...
Ceph Day Netherlands - Ceph Management and Monitoring with openATTIC 3.x
Let’s Fix Logging Once and for All
Unikraft: Fast, Specialized Unikernels the Easy Way
RADOS improvements and roadmap - Greg Farnum, Josh Durgin, Kefu Chai
Data Structures for High Resolution, Real-time Telemetry at Scale
Utilizing AMD GPUs: Tuning, programming models, and roadmap
Expectations for optical network from the viewpoint of system software research
Achieving the Ultimate Performance with KVM
Ad

Similar to Dad i want a supercomputer on my next (20)

PPT
Parallel_and_Cluster_Computing.ppt
PPTX
Cluster computer
PPTX
Copy of Parallel_and_Cluster_Computing.pptx
PPT
NWU and HPC
PDF
Beowulf Cluster Computing With Windows Illustrated Edition Thomas Sterling
PDF
Beowulf Cluster Computing with Windows Thomas Sterling all chapter instant do...
PPT
CLUSTER COMPUTING
PPTX
Clustering
PDF
Beowulf Cluster Computing With Linux Scientific And Engineering Computation T...
PDF
Building A Linux Cluster Using Raspberry PI #2!
PDF
Beowulf Cluster Computing With Linux 1st Thomas Sterling Editor
PDF
Building SuperComputers @ Home
PDF
Cluster Setup Manual Using Ubuntu and MPICH
PPT
Clusters (Distributed computing)
PPTX
Clustering by AKASHMSHAH
PPTX
Cluster Technique used in Advanced Computer Architecture.pptx
PDF
Big Data Everywhere Chicago: High Performance Computing - Contributions Towar...
PPTX
Beowulf cluster
PDF
Computer_Clustering_Technologies
Parallel_and_Cluster_Computing.ppt
Cluster computer
Copy of Parallel_and_Cluster_Computing.pptx
NWU and HPC
Beowulf Cluster Computing With Windows Illustrated Edition Thomas Sterling
Beowulf Cluster Computing with Windows Thomas Sterling all chapter instant do...
CLUSTER COMPUTING
Clustering
Beowulf Cluster Computing With Linux Scientific And Engineering Computation T...
Building A Linux Cluster Using Raspberry PI #2!
Beowulf Cluster Computing With Linux 1st Thomas Sterling Editor
Building SuperComputers @ Home
Cluster Setup Manual Using Ubuntu and MPICH
Clusters (Distributed computing)
Clustering by AKASHMSHAH
Cluster Technique used in Advanced Computer Architecture.pptx
Big Data Everywhere Chicago: High Performance Computing - Contributions Towar...
Beowulf cluster
Computer_Clustering_Technologies
Ad

Recently uploaded (20)

PPTX
DPT-MAY24.pptx for review and ucploading
PDF
Manager Resume for R, CL & Applying Online.pdf
PPT
Gsisgdkddkvdgjsjdvdbdbdbdghjkhgcvvkkfcxxfg
PDF
esg-supply-chain-webinar-nov2018hkhkkh.pdf
DOCX
How to Become a Criminal Profiler or Behavioural Analyst.docx
PPTX
A slide for students with the advantagea
PPTX
chapter 3_bem.pptxKLJLKJLKJLKJKJKLJKJKJKHJH
PPTX
Autonomic_Nervous_SystemM_Drugs_PPT.pptx
PDF
Blue-Modern-Elegant-Presentation (1).pdf
DOCX
mcsp232projectguidelinesjan2023 (1).docx
PPTX
PMP (Project Management Professional) course prepares individuals
PDF
Josh Gao Strength to Strength Book Summary
PPTX
AREAS OF SPECIALIZATION AND CAREER OPPORTUNITIES FOR COMMUNICATORS AND JOURNA...
PPTX
1751884730-Visual Basic -Unitj CS B.pptx
PPTX
Nervous_System_Drugs_PPT.pptxXXXXXXXXXXXXXXXXX
PPTX
E-Commerce____Intermediate_Presentation.pptx
PDF
313302 DBMS UNIT 1 PPT for diploma Computer Eng Unit 2
PDF
Prostaglandin E2.pdf orthoodontics op kharbanda
PPTX
FINAL PPT.pptx cfyufuyfuyuy8ioyoiuvy ituyc utdfm v
PPTX
Job-opportunities lecture about it skills
DPT-MAY24.pptx for review and ucploading
Manager Resume for R, CL & Applying Online.pdf
Gsisgdkddkvdgjsjdvdbdbdbdghjkhgcvvkkfcxxfg
esg-supply-chain-webinar-nov2018hkhkkh.pdf
How to Become a Criminal Profiler or Behavioural Analyst.docx
A slide for students with the advantagea
chapter 3_bem.pptxKLJLKJLKJLKJKJKLJKJKJKHJH
Autonomic_Nervous_SystemM_Drugs_PPT.pptx
Blue-Modern-Elegant-Presentation (1).pdf
mcsp232projectguidelinesjan2023 (1).docx
PMP (Project Management Professional) course prepares individuals
Josh Gao Strength to Strength Book Summary
AREAS OF SPECIALIZATION AND CAREER OPPORTUNITIES FOR COMMUNICATORS AND JOURNA...
1751884730-Visual Basic -Unitj CS B.pptx
Nervous_System_Drugs_PPT.pptxXXXXXXXXXXXXXXXXX
E-Commerce____Intermediate_Presentation.pptx
313302 DBMS UNIT 1 PPT for diploma Computer Eng Unit 2
Prostaglandin E2.pdf orthoodontics op kharbanda
FINAL PPT.pptx cfyufuyfuyuy8ioyoiuvy ituyc utdfm v
Job-opportunities lecture about it skills

Dad i want a supercomputer on my next

  • 1. Beowulf Cluster, building cheap powerful hpc By Akash Sahoo
  • 3. Speed •16 Pentium Ppro 200Mhz each •2GB Memory •1.25 Glops •130 Mflops individual speed •Made by Caltech in ‘96 FLOPS (or flops or flop/s, for floating-point operations per second) is a measure of a computer's performance, especially in fields of scientific calculations. It is instruction per second i7-920 has 40Gflops In 2007 a Microwulf Cluster (less nodes) with just 4 nodes gave a performance of 26.25 Gflops (Wikipedia)
  • 4. Why Speed Computer simulations are far cheaper and faster than physical experiments Computers can solve a much wider range of problems than specific laboratory equipment can Computational approaches are only limited by computer speed and memory capacity, while physical experiments have many practical constraints Uses: Predictive Modeling and Simulations, Engineering Design and Automation, Military, Visualization
  • 5. Why Beowulf Commodity Hardware, No need to throw old PCs !! Ease of addition of nodes Work if nodes fail VVVV Cheap (Supercomputer costs million) World Fastest is a cheap Beowulf Cluster Can be used in Homes, educational Institutes Students will study and write parallel, optimized code Simulation Weather, BLAST (comparing primary biological sequence information), BIONIC (Cancer), distributed Matlab, DNA, Image processing, Web server, own software Software can run on one/many node as specified INDIA has outdated supercomputer - PARAM
  • 6. Parallel Computing Shared Memory NFS Message Passing Efficiency Latency Topology Admin and maintainace Redundancy Workload, Unbalancing, Arch Energy, cost, scalable
  • 7. Building It Install Linux in each node (MPICH2), Can be build on Windows (PVM) SSH Equivalence SSH Hostkeys – RSA, DSA Installation of PDSH Setting NFS Creation of MPI users Installation of compat-g77,gcc MPI install, LINPACK Benchmark OSCAR, ROCKS, SCARY- maintainance
  • 8. Mpdboot –n 3 Starting cluster Mpdtrace Know which nodes are running
  • 10. Same prog on 2 nodes Same prog on 3 nodes
  • 11. Improvements Add fibre optic cable Dual Lan card Powerful Gateway, Router Faster computers – Cost High, Performance High Keep in mind-Total memory, Memory bandwidth, Storage bandwidth, System interconnect latency, The effect of interconnect topologies such as meshes, cubes, and fat trees on application design SSD Harddisk, RAID, Graphics Card
  • 12. Notes Gave a performance boost of > 200% on 3 nodes Originality : Security both RSA, DSA. PDSH makes run commands in all nodes at once Commercialization: giving manuals on prog, software, setting cluster, maitainace Proj done in 3rd year
  • 13. Patents  Dynamically allocated cluster system  Charles Evert Prael  ;  (Patent No: US5093912) The invention provides a system and method for managing clusters of parallel processors  for use by groups and individuals requiring supercomputer level computational power. Given Scheduling algo PSB, job queue… 28 claims mostly on scheduling  Method and apparatus for securing and managing cluster David Bruce Kumhyr 2002