Why Supercomputing matters to Deep Learning (DLAI D3L2 2017 UPC Deep Learning for Artificial Intelligence)
1 like822 views
Jordi Torres discusses the significance of supercomputing in enhancing deep learning and artificial intelligence (AI). He highlights advancements in AI areas such as speech recognition, natural language processing, and computer vision that are revolutionizing technologies like robotics and self-driving cars. The document emphasizes the necessity of powerful computing resources, including GPUs, and the growing accessibility of cloud-based AI services to support deep learning innovations.
Why Supercomputing matters to Deep Learning (DLAI D3L2 2017 UPC Deep Learning for Artificial Intelligence)
- 1. JORDI TORRES Why Supercomputing matters to Deep Learning Barcelona - UPC, October 2017 JORDI TORRES
- 2. Artificial Intelligence is changing our life
- 3. Quantum leaps in the quality of a wide range of everyday technologies thanks to Artificial Intelligence
- 4. Credits: https://www.yahoo.com/tech/battle-of-the-voice-assistants-siri-cortana-211625975.html We are increasingly interacting with “our” computers by just talking to them JORDI TORRES Speech Recognition
- 5. JORDI TORRES Google Translatenow renders spoken sentences in one language into spoken sentences in another, for 32 pairs of languages and offers text translation for 100+ languages. Natural Language Processing
- 6. JORDI TORRES Google Translatenow renders spoken sentences in one language into spoken sentences in another, for 32 pairs of languages and offers text translation for 100+ languages. Natural Language Processing
- 7. JORDI TORRES Computer Vision Now our computers can recognize images and generate descriptionsfor photos in seconds.
- 8. All these three areas are crucial to unleashing improvements in robotics, drones, self-driving cars, etc. Source: http://edition.cnn.com/2013/05/16/tech/innovation/robot-bartender-mit-google-makr-shakr/ All these three areas are crucial to unleashing improvements in robotics, drones, self-driving cars, etc. JORDI TORRES
- 9. Source: http://axisphilly.org/article/military-drones-philadelphia-base-control/ All these three areas are crucial to unleashing improvements in robotics, drones, self-driving cars, etc. JORDI TORRES
- 10. Source: http://fortune.com/2016/04/23/china-self-driving-cars/ All these three areas are crucial to unleashing improvements in robotics, drones, self-driving cars, etc. JORDI TORRES
- 11. image source: http://acuvate.com/blog/22-experts-predict-artificial-intelligence-impact-enterprise-workplace AI is at the heart of today’s technological innovation.
- 12. JORDI TORRES Many of these breakthroughs have been made possible by a family of AI known as Neural Networks
- 13. JORDI TORRES Neural networks, also known as a Deep Learning, enables a computer to learn from observational data Although the greatest impacts of deep learning may be obtained when it is integrated into the whole toolbox of other AI techniques
- 14. Artificial Intelligence and Neural Networks, are not new concepts!
- 15. http://www.independent.co.uk/news/obituaries/john-mccarthy-computer-scientist-known-as-the-father-of-ai-6255307.html John McCarthy coined the term Artificial Intelligence in the 1950s .
- 16. Source: http://www.enzyklopaedie-der-wirtschaftsinformatik.de/wi-enzyklopaedie/Members/wilex4/Rosen-2.jpg In 1958 Frank Rosenblatt built a prototype neural net, which he called the Perceptron
- 17. So why did Deep Learning only take off few years ago?
- 18. JORDI TORRES Source:http://www.economist.com/node/15579717 One of the key drivers: The data deluge
- 19. JORDI TORRES Source:http://www.economist.com/node/15579717 One of the key drivers: The data deluge Thanks to the advent of Big Data AI models can be “trained” by exposing them to large data sets that were previously unavailable.
- 20. JORDI TORRES source: cs231n.stanford.edu/slides/2017/cs231n_2017_lecture15.pdf Training DL neural nets has an insatiable demand for Computing
- 21. Thanks to advances in Computer Architecture, nowadays we can solve problems that would have been intractable some years ago.
- 22. Credits:http://www.ithistory.org/sites/default/files/hardware/facom230-50.jpg FACOM 230 – Fujitsu Instructions per second: few Mips * (M = 1.000.000) Processors : 1 1982
- 23. 2012 MARENOSTRUM III - IBM Instructions per second: 1.000.000.000 MFlops Processors : 6046 (48448 cores)
- 24. 2012 MARENOSTRUM III - IBM Instructions per second: 1.000.000.000 MFlops Processors : 6046 (48448 cores) only 1.000.000.000 times faster
- 25. CPU improvements! Until then, the increase in computational power every decade of “my” computer, was mainly thanks to CPU
- 26. CPU improvements! Since then, the increase in computational power for Deep Learning has not only been from CPU improvements . . . Until then, the increase in computational power every decade of “my” computer, was mainly thanks to CPU
- 27. JORDI TORRES SOURCE:https://www.hpcwire.com/2016/11/23/nvidia-sees-bright-future-ai-supercomputing/?eid=330373742&bid=1597894 but also from the realization that GPUs (NVIDIA) were 20 to 50 times more efficient than traditional CPUs.
- 28. Deep Learning requires computer architecture advancements AI specific processors Optimized libraries and kernels Fast tightly coupled network interfaces Dense computer hardware
- 29. COMPUTING POWER is the real enabler!
- 30. What if I do not have this hardware?
- 31. Now we are entering into an era of computation democratization for companies !
- 32. And what is “my/your” computer like now?
- 33. And what is “my/your” computer like now? Source: http://www.google.com/about/datacenters/gallery/images
- 34. Source: http://www.google.com/about/datacenters/gallery/images And what is “my/your” computer like now? The Cloud
- 39. For those (experts) who want to develop their own software, cloud services like Amazon Web Services provide GPU-driven deep-learning computation services
- 40. And Google ...
- 41. And all major cloud platforms... Microsoft Azure IBM Cloud Aliyun Cirrascale NIMBIX Outscale . . . Cogeco Peer 1 Penguin Computing RapidSwitch Rescale SkyScale SoftLayer . . .
- 42. And for “less expert” people, various companies are providing a working scalable implementation of ML/AI algorithms as a Service (AI-as-a-Service) Source: https://twitter.com/smolix/status/804005781381128192Source: http://www.kdnuggets.com/2015/11/machine-learning -apis-data-science.html
- 43. An open-source world for the Deep Learning community
- 44. Many open-source DL software have greased the innovation process
- 46. In this course: we will consider the 3 frameworks with steepest gradient frameworks with more slope TensorFlow Keras Pytorch
- 47. and no less important, an open-publication ethic, whereby many researchers publish their results immediately on a database without awaiting peer-review approval.
- 48. JORDI TORRES 16 Next lectures: (in lab sessions) 1. High-level neural networks API: Keras 2. The most popular middleware: TensorFlow 3. Why DL researchers are beginning to embrace PyTorch