Clouds in High Energy
1 like883 views
This document discusses the use of cloud computing in high energy physics (particle physics). It describes how clouds are being used to preserve long-term software and data from particle physics experiments. Clouds are also being used to provide distributed computing for exceptional computing demands. Private clouds have been enabled using the existing high-level trigger farms of the ATLAS and CMS experiments during periods when the accelerators are idle. Both private and commercial clouds are playing an important role in high energy physics.
Clouds in High Energy
- 1. Clouds in High Energy Physics Randall Sobie Ins$tute of Par$cle Physics University of Victoria
- 2. High Energy Physics (Par$cle Physics) The area of physics that studies the fundamental par$cles of nature and their interac$ons. Accelerators Underground labs Orbi;ng labs SLAC Linear Accelerator Sudbury Solar Neutrino Detector Alpha Magne$c Spectrometer Space Sta$on Randall Sobie IPP/UVictoria 2
- 3. Is the Higgs boson the source of mass of our fundamental par;cles? Randall Sobie IPP/UVictoria 3
- 4. Why is the universe made of maCer and not equal amounts of maCer/an;maCer? Randall Sobie IPP/UVictoria 4
- 5. What is origin of Dark MaCer and Dark Energy? We do not know the T composi$on of 95% of the universe Temperature of the universe WMAP satellite Randall Sobie IPP/UVictoria 5
- 6. Understanding our World Randall Sobie IPP/UVictoria 6
- 7. Understanding our World Randall Sobie IPP/UVictoria 7
- 8. Large Hadron Collider at CERN in Geneva 27 km in circumference 100 m underground Probes distances of 10-‐20 m Randall Sobie IPP/UVictoria 8
- 9. Large Hadron Collider at CERN in Geneva 27 km in circumference 100 m underground Probes distances of 10-‐20 m Randall Sobie IPP/UVictoria 9
- 10. ATLAS Detector 44 m wide 25 m diameter Randall Sobie IPP/UVictoria 10
- 11. ATLAS Detector 44 m wide 25 m diameter Randall Sobie IPP/UVictoria 11
- 12. ATLAS Detector 44 m wide 25 m diameter Randall Sobie IPP/UVictoria 12
- 13. WLCG Compu;ng Grid CERN Tier 0 40 million collisions per second 10 Tier1 sites 60+ Tier2 sites 140 PB data 100,000 collisions selected 200 events per second Randall Sobie IPP/UVictoria 13
- 14. Higgs Discovery in 2012 The ATLAS and CMS experiments see evidence for a Higgs-‐like par$cle Picture shows an event where the Higgs candidate decays to 4 electron-‐like par$cles Randall Sobie IPP/UVictoria 14
- 15. Search for Higgs decays to 4 “leptons” (electrons or muons) Number of candidates (ver$cal axis) Mass of the candidates (horizontal axis) We observe an excess of candidates with a mass of 125 proton-‐masses Also observed in the CMS experiment Randall Sobie IPP/UVictoria 15
- 16. July 4, 2012 Randall Sobie IPP/UVictoria 16
- 17. Clouds in High Energy Physics Long-‐term preserva$on of Distributed cloud compu$ng so^ware and data of HEP using HEP and non-‐HEP experiments clouds U$lize special compu$ng Use commercial clouds for resources a_ached to the excep$onal compu$ng detectors demands Simplify the management of heterogeneous in-‐house resources Randall Sobie IPP/UVictoria 17
- 18. Using Clouds for Data Preserva$on /+012'3"01#4'562%$7% 8+%)"9%':4#; !<==<'>'!?!?8 SLAC Linear Accelerator BaBar Detector BaBar experiment stopped recording electron-‐positron collisions in 2008 Electron-‐positron collision Randall Sobie IPP/UVictoria 18 !"#$%&'()*+',--.
- 19. Using Clouds for Data Preserva$on /+012'3"01#4'562%$7% 8+%)"9%':4#; Term Data Access system Long !<==<'>'!?!?8 Sta$c SL5 VMs SLAC Linear Accelerator BaBar SW and data Full access to BaBar Detector Transparent to the user In opera$on since 2010 BaBar experiment stopped recording h_p://today.slac.stanford.edu/feature/2010/babar-‐prototype-‐ data-‐servers.asp electron-‐positron collisions in 2008 Electron-‐positron collision Randall Sobie IPP/UVictoria 19 !"#$%&'()*+',--.
- 20. ATLAS/CMS High Level Trigger Clouds HLT Farms 40 million collisions per second These systems are used in real-‐$me when there is colliding beams The aim is to use the resources during the idle periods for other purposes 100,000 collisions selected Enabled as private OpenStack clouds 50,000 cores See talk by Toni Perez Wednesday 1100 ATLAS, CMS, LHCb 200 events per second Randall Sobie IPP/UVictoria 20
- 21. Private and commercial clouds Ibex @ CERN (J. van Eldik, T. Bell, B.Moreira) OpenStack cloud with 5000 cores Provide batch services and cloud services HEP using Amazon, Google, Rackspace and others Star Experiment at RHIC (Brookhaven NL) Belle Experiment at KEK (Japan) ATLAS Experiment Commercial clouds used for excep1onal low I/O demands Challenges: iden1ty management, API compa1bility, VM configura1on and network connec1vity Costs are higher than our private resources Randall Sobie IPP/UVictoria 21
- 22. Distributed cloud compu$ng Grid of Clouds Seamlessly use mul$ple, heterogeneous IaaS clouds for batch workloads Sky Compu;ng K. Keahey etal Use dedicated HEP and non-‐HEP opportunis$c resources Independent of the IaaS cloud type Removes any applica$on requirements from remote site Support mul$ple projects Randall Sobie IPP/UVictoria 22
- 23. Distributed cloud overview Cloud Scheduler HTCondor JobQueue Discover User-‐Job UserJob User VM IaaS Clouds Randall Sobie IPP/UVictoria 23
- 24. Distributed cloud overview Cloud Scheduler HTCondor JobQueue Boot User-‐VM on a cloud UserJob User VM IaaS Clouds Randall Sobie IPP/UVictoria 24
- 25. Distributed cloud overview Cloud Scheduler HTCondor JobQueue UserJob User VM VM registers with HTCondor IaaS Clouds Randall Sobie IPP/UVictoria 25
- 26. Distributed cloud overview Cloud Scheduler HTCondor JobQueue UserJob Dispatches UserJob User to VM VM IaaS Remote VM image repository (Nimbus clouds) Clouds Upload VMs to OpenStack clouds Remote data repositories for all clouds Randall Sobie IPP/UVictoria 26
- 27. Distributed cloud status Nimbus Victoria(3) O_awa FG Chicago FG SanDiego FG Florida Opera;onal since Nov 2011 Approximately 250K jobs Amazon EC2 (Astronomy 500K jobs) OpenStack Google GCE Melbourne CERN Using 10 clouds for ATLAS jobs Edmonton 500-‐1000 simultaneous jobs Oxford 12 hour jobs ATLAS jobs submi_ed from CERN Randall Sobie IPP/UVictoria 27
- 28. Distributed cloud status April 9 2013 8 clouds Nimbus 100-‐120 8-‐core VMs Victoria(3) O_awa FG Chicago FG SanDiego FG Florida Opera;onal since Nov 2011 Melbourne Approximately 250K j-‐15 2013 April 8 obs Amazon EC2 CERN-‐-‐Ibex (Astronomy 500K jobs) 8-‐core VMs 100-‐150 OpenStack Google GCE Melbourne CANARIE CERN WG-‐Victoria Using 10 clouds for ATLAS jobs Edmonton NRC-‐O_awa 500-‐1000 simultaneous jobs ç Oxford Victoria 12 hour jobs FG-‐San Diego FG-‐Chicago Victoria ATLAS jobs submi_ed from CERN Randall Sobie IPP/UVictoria 28
- 29. Technology innova$on for new science We see ourselves as integrators rather than developers of cloud technology The OpenStack developer community can help us Common authen$ca$on Centralized VM image storage Consistent meta-‐data Unique cloud names Simplify the integraNon of OpenStack clouds for us Randall Sobie IPP/UVictoria 29
- 30. Summary Understand our Universe Studying the Higgs Search for Dark Ma_er Difference between an$ma_er/ma_er High Energy Physics Impact on society Compu;ng CERN is the birthplace of the WWW Large distributed systems Medical physics Global research network Technology innova$on Novel use of compu$ng technologies HQP Randall Sobie IPP/UVictoria 30
- 31. Acknowledgements Contact informa$on: rsobie@uvic.ca Web sites: h_p://rjs.phys.uvic.ca/ h_p://heprc.phys.uvic.ca/home/ Randall Sobie IPP/UVictoria 31