HTCondor flocking between two clouds
- 1. Flocking HTCondor Between Two Clouds Guy Tel-Zur tel-zur@computer.org March 2013
- 2. Goal • Submit HTCondor jobs from Amazon EC2 HTCondor pool and flock to another HTCondor pool on Rackspace. • Both pools use Windows Server 2008
- 4. Outline • Install Windows Server 2008 with HTCondor on Rackspace (Pool 1) • Install Windows Serve 2008 with HTCondor on EC2 (Pool 2) • Modify the condor_config file FLOCK_FROM and FLOCK_TO specifying the IP addresses of the Central Managers. • Verify the Flocking works
- 6. Open terminal from here
- 7. The server is up
- 8. In the next slides we show HTCondor Installation steps on the first node
- 17. Verify that condor service is running
- 19. Before first submission store your credentials
- 20. A simple test program
- 21. Now we repeat the above steps and install a HTCondor node on AWS EC2
- 26. The new machine is ready to be launched
- 27. EC2 dashboard
- 28. Connecting with remote desktop
- 30. The test program works also on EC2
- 31. Add Flocking • Edit condor_config • At both ends: – Set the IP addresses of the condor master nodes in the flock_from and flock_to statements – Save and restart condor • That’s it !!!
- 32. Check Flocking at both ends Each pool “sees” the other pool NOTE! In this demo for simplicity firewalls were disabled. This is not recommended in real production runs!
- 33. Each pool can see the other pool
- 35. Reference • HTCondor: http://research.cs.wisc.edu/htcondor/ • Rackspace: http://www.rackspace.com/ • AWS: http://aws.amazon.com/