Code Upload Challenge on EvalAI
- 1. Hosting Reinforcement Learning Challenges on EvalAI Deepesh Pathak Deshraj Yadav Kartik Verma Rishabh Bharadwaj Rishabh Jain
- 2. Project Goal Build feature that will help users to host Reinforcement Learning Challenges easily on EvalAI and evaluate the agents at Scale.
- 3. The Flow
- 4. Submission 1. Participant submits agent image using EvalAI-CLI, it will Poll API for various details 2. The Agent image will be pushed to a registry 3. The EvalAI CLI will send submission details to API
- 5. Manage Submissions 1. The Submission details received at API will be pushed to Challenge Specific Queue. 2. RL Submission worker will be polling this queue and will get details of new submissions.
- 6. RL Worker Using the Submission details, RL Worker will deploy the logical entity of environment and agent containers on Kubernetes. Deployment Logs from rl-worker.py Polling the API for new submissions Processing Submission and deploying
- 7. Deployment
- 8. Evaluation 1. Multi Container Policy 2. gRPC for interaction between Agent and Environment.
- 9. Demo 1. A pole is attached by an un-actuated joint to a cart, which moves along a frictionless track. 2. The system is controlled by applying a force of +1 or -1 to the cart. The pendulum starts upright, and the goal is to prevent it from falling over. 3. A reward of +1 is provided for every timestep that the pole remains upright.
- 10. Plan for Phase 3 1. Hosting challenges that have environments and agents coupled tightly. e.g. Habitat Challenge 2. Resource Optimization using GPU sharing
- 11. What I learnt?
- 12. Next steps after GSoC Keep Contributing and help other contributors Improve EvalAI
- 13. Thank You!
Editor's Notes
- #3: What is the problem? Other Class of problems not solved by evalai Host RL Types of
- #4: What is Env, and what is agent and how they are entities? Challenge creator and participant.
- #5: Explain
- #9: The Project can be used for Multi Container Policy as well as Single Container Policy. The advantage of Multi Container Policy is that we decouple things that are not related making the approach more generalized. Environment and Agent: Code on https://github.com/vkartik97/rl-grpc Images on https://hub.docker.com/r/vkartik97/agent and https://hub.docker.com/r/vkartik97/env
- #11: 1 - example: Habitat
- #12: GPU Sharing