![15
The many faces of deep learning
Figure 1: The Transformer - model architecture.
The Transformer follows this overall architecture using stacked self-attention and p
connected layers for both the encoder and decoder, shown in the left and right hal
respectively.
3.1 Encoder and Decoder Stacks
Encoder: The encoder is composed of a stack of N = 6 identical layers. Each
sub-layers. The first is a multi-head self-attention mechanism, and the second is a s
wise fully connected feed-forward network. We employ a residual connection [11]
Scaled Dot-Product Attention Multi-Head Attention
Scaled Dot-Product Attention Multi-Head Attention
Attention
Is
All
You
Need,
A.Vaswani
et
all.,
2017,
https://doi.org/10.48550/arXiv.1706.03762
© DAIN Studios 2024
Deep Learning Model for Diagnosing Cardiac Sarcoidosis, DAIN Studios 2023
LSTM Architecture:
Multimodal 3D CNN architecture:
Transformers architecture:](https://image.slidesharecdn.com/hugogavert-deeplearning-domachinesdream2024-02-13-240802081201-ed68020d/85/Do-Machines-Dream-Reviewing-how-Deep-Learning-and-Human-Brain-Research-was-linked-and-what-kinds-of-links-there-are-today-14-320.jpg)
Do Machines Dream? - Reviewing how Deep Learning and Human Brain Research was linked and what kinds of links there are today
- 1. 1 © DAIN Studios 2024
- 2. 2 Hugo Gävert Chief AI and Data Officer DAIN Studios • HUT (Aalto) Information Sciences lab • AI & data science for 20+ years • Previously: Xtract, Nokia, Sanoma, OP Financial Group © DAIN Studios 2024
- 3. 3 Data & AI Consultancy From Strategy to Execution 70+ Experts Data Strategists Data Scientists & Engineers BI Developers Broad Experience 80+ Clients 20 Industries 7 Countries 3 Studios Helsinki - Berlin - Munich
- 4. 4 TIME, March 25, 1996 • Garry Kasparov had just lost 2 games to Deep Blue in February 1996. He did still win this 1st match with Deep Blue 4 – 2. • Comparison with human thought • Deep Blue playing chess: • Brute force calculations • Game databases • Limited strategic thinking: positional scoring and rules • Philosophical perspectives on machine thinking • Machine sentience, consciousness
- 5. 5 Artificial Narrow Intelligence (ANI) An AI that focuses on specific, predetermined tasks with limited to no utility beyond the environment it was specifically built for. Artificial General Intelligence (AGI) An AI with human-like intelligence, being capable adapting to new situations through insight, learning, and reasoning beyond what it was originally trained for. Artificial Superintelligence (ASI) An AI that is vastly superior to humans in every domain. Capabilities beyond what we can comprehend. ChatGPT © DAIN Studios 2024 We are here! What Is Artificial Intelligence?
- 6. 7 Artificial General Intelligence (AGI) An AI with human-like intelligence, being capable adapting to new situations through insight, learning, and reasoning beyond what it was originally trained for. © DAIN Studios 2024 What Is Artificial Intelligence? Artificial General Intelligence should • Be capable of performing intellectual tasks like human • Communicate in human like manner • Be capable of using tools • Be able to solve problems across diverse domains, adapt • Learn new things, develop its own goals and learning strategies
- 7. 8 Early AI Research • Influenced by brain research • Artificial Neural Networks • Perceptron • Backpropagation algorithm • Self-Organizing Maps © DAIN Studios 2024 © DAIN Studios 2024
- 8. 9 Artificial neural networks mimicking human brains © DAIN Studios 2024 Cat Dog Horse … …
- 9. 10 Kohonen’s Self-Organizing Map (SOM) • Organizing high-dimensional data • Maps and clusters high-dimensional data into simpler map • Maintains topological relationship of points – similar data points are positioned closely on the map • Easy visualization à OpenStax Collegederivative work: Popadius - This file was derived from: 1421 Sensory Homunculus.jpg:, CC BY 3.0, https://commons.wikimedia.org/w/index.php?curid=88916983 • Brain-inspired training • The training algorithm was inspired by how the cerebral cortex processes and organizes sensory information • In the training phase, SOM expands to cover all the points of the input space keeping similar things together Classic example: World Poverty Map World Bank data 1992, 39 features Samuel Kaski, 1997, URL: http://www.cis.hut.fi/research/som-research/worldmap.html © DAIN Studios 2024
- 10. 11 AlexNet started the deep learning renaissance • In 2012, AlexNet wins the ImageNet challenge by a large marging, marking a paradigm shift in image recognition technology • Convolutional Neural Networks, already in 1998 • Deep Belief Networks, 2006 • Generative Adversial Networks, 2014 © DAIN Studios 2024
- 11. 12 Andrew Ng, 2016 DeepLearning.AI, Landing AI, AI Fund Co-Founder of Coursera Stanford Professor
- 12. 13 Deep learning research had diverged from brain research • Research of the deep learning algorithms shifted • Computing power • Big data • Open source • Commercial interests • This led to a fast iteration of new algorithms and ideas © DAIN Studios 2024
- 13. 14 • Open-Source frameworks, like TensorFlow and PyTorch democratized the tools for creating complex models • Open Models, like Google Inception v3, enabled rapid adaptation across various domains Stanford: Dermatologist-level Classification of Skin Cancer using Deep Neural Networks, 2017 Open-source and open models empowering research © DAIN Studios 2024
- 14. 15 The many faces of deep learning Figure 1: The Transformer - model architecture. The Transformer follows this overall architecture using stacked self-attention and p connected layers for both the encoder and decoder, shown in the left and right hal respectively. 3.1 Encoder and Decoder Stacks Encoder: The encoder is composed of a stack of N = 6 identical layers. Each sub-layers. The first is a multi-head self-attention mechanism, and the second is a s wise fully connected feed-forward network. We employ a residual connection [11] Scaled Dot-Product Attention Multi-Head Attention Scaled Dot-Product Attention Multi-Head Attention Attention Is All You Need, A.Vaswani et all., 2017, https://doi.org/10.48550/arXiv.1706.03762 © DAIN Studios 2024 Deep Learning Model for Diagnosing Cardiac Sarcoidosis, DAIN Studios 2023 LSTM Architecture: Multimodal 3D CNN architecture: Transformers architecture:
- 15. 16 Is ChatGPT an AGI? Emergent capabilities • Question answering, summarizing without losing the essence • Reasoning and inference, solving logic puzzles • Generalizing and adapting (zero shot learning) • Instruction following and task completion, code generation • Creative writing Some comparisons: • Brain has 100-600 trillion synapses • Neurons’ firing rate is about 200Hz • Brain uses 20W
- 16. 17 Learning From Dreaming? Some functions of dreaming • Emotional processing • Memory consolidation • Cognitive development • Mental preparation • Self-awareness Dreams: • Recurring themes and situations • Fluid and dynamic: environment, characters and identities, locations, time change • Non-linear scenes and events, lack of a clear narrative, timeline àThese ideas are being taken into use in ML training (for example in RL simulations) © DAIN Studios 2023 © DAIN Studios 2024
- 17. 18 Key takeaways • Human brain was the original model • AI Research / Neural Networks was originally largely inspired by the human brain research • Deep Learning breakthroughs & technological advancements lead the AI research into different paths • Similarities still exist • Hierarchical nature of models • Training through simulations / dreams • AGI through whole brain emulation • Need for explainability • We don’t know how LLMs get their emergent capabilities – strong need to understand (XAI) © DAIN Studios 2024
- 18. Helsinki Salomonkatu 17 A (Autotalo) 00100 Helsinki, Finland Berlin Erkelenzdamm 7 10999 Berlin, Germany info@dainstudios.com