Playing Atari with Deep Reinforcement Learning
- 1. DeepMind Technologies Playing Atari with Deep Reinforcement Learning
- 2. Contents Abstract Introduction Background Deep Reinforcement Learning Experiments Conclusion
- 3. Abstract First deep learning model using reinforcement learning • Successfully learn control policies directly • From high-dimensional sensory input (pixels) CNN model trained on a variant of Q-learning • Input: raw pixel • Output: a value function estimating future reward Applied seven Atari 2600 games (no adjustment) • Outperforms ALL previous approaches on six games • Surpasses a human expert on three games
- 4. Introduction Learning directly from high-dimensional sensory input is Long-standing challenges of RL Most successful RL relies on hand-crafted features Recent advances in deep learning extract high-level features from raw sensory data • Breakthroughs in computer vision/Speech recognition Neural architectures on supervised/unsupervised learning • Convolutional networks • Multilayer perceptrons • Restricted Boltzmann machines • Recurrent Neural nets Problems to solve : Motivation
- 5. Introduction Most DL requires hand labeled training data • RL must learn from a scalar reward signal • Reward signal is often sparse, noisy, and delayed • Delay between actions and resulting rewards can be thousand time steps CNN with a variant Q-learning • Most DL assumes data samples are independent • RL encounters sequences of highly correlated states Experience replay Remaining challenges
- 6. Introduction
- 9. Background Major Components of an RL Agent
- 11. Background On-policy learning vs Off-policy learning
- 12. Background On-policy learning vs Off-policy learning http://www.modulabs.co.kr/RL_library/2621
- 17. Background Model
- 18. Deep Reinforcement Learning Deep Q-Networks : Experience Replay
- 19. Deep Reinforcement Learning Deep Q-Networks : Experience Replay
- 20. Deep Reinforcement Learning Model Architecture
- 21. Experiments
- 22. Experiments
- 23. Experiments
- 24. Conclusion