![Initialisation and Loss Function
Zero initialisation ☹
W1 = np.zeros((layers_dims[l],
layers_dims[l - 1]))
Random initialisation 😐
W1 =
np.random.randn(layers_di
ms[l], layers_dims[l - 1])
* 10
Mean Squared Error ☹
1/m.sum(Y_hat, Y)**2](https://image.slidesharecdn.com/ai-apracticalapproach-171215194224/85/Ai-A-Practical-Approach-7-320.jpg)
![Initialisation and Loss Function
Xavier Glorot 😄
W1 = np.random.randn(layers[l], layers[l - 1]) *
np.sqrt(2 / (layers[l - 1] + layers[l]))
L2 regularisation ☺
(ƛ/2.m).sum(W**2)
Adam
Exponentially Weighted Averages
Vt = βVt-1 + (1 - β)𝛳t
RMSProp
Cross Entropy 😄
1/m.sum(Y.log(a)+(1-Y).log(1-a))](https://image.slidesharecdn.com/ai-apracticalapproach-171215194224/85/Ai-A-Practical-Approach-9-320.jpg)
![Initialisation and Cost Function
He 😄
W1 = np.random.randn(layers[l],
layers[l - 1]) * np.sqrt(2 / layers[l - 1])
Dropout 😄
❌
✅](https://image.slidesharecdn.com/ai-apracticalapproach-171215194224/85/Ai-A-Practical-Approach-11-320.jpg)
Ai - A Practical Approach
- 1. Artificial Intelligence A Practical Approach
- 2. Who am I? Software & Machine Learning Engineer; City.AI Ambassador; IBM Watson AI XPRIZE contestant; Kaggler; Guest attendee at AI for Good Global Summit at the UN; X-Men geek; family man and father of 5 (3 kids and 2 cats). @wilderrodrigues
- 3. Machine Learning without a Ph.D.
- 4. What is it About? Source: World Wide Web Foundation, June 2017
- 5. Points to Focus On Number and Types of Layers; Initialisation; Regularisation; Cost Functions; Optimisation; Data Augmentation.
- 6. Shallow Neural Networks Random Initialisation Mean Squared Error Stochastic Gradient Descent Sigmoid
- 7. Initialisation and Loss Function Zero initialisation ☹ W1 = np.zeros((layers_dims[l], layers_dims[l - 1])) Random initialisation 😐 W1 = np.random.randn(layers_di ms[l], layers_dims[l - 1]) * 10 Mean Squared Error ☹ 1/m.sum(Y_hat, Y)**2
- 8. Xavier Glorot L2 regularisation Cross Entropy Adam Tanh Intermediate Neural Networks
- 9. Initialisation and Loss Function Xavier Glorot 😄 W1 = np.random.randn(layers[l], layers[l - 1]) * np.sqrt(2 / (layers[l - 1] + layers[l])) L2 regularisation ☺ (ƛ/2.m).sum(W**2) Adam Exponentially Weighted Averages Vt = βVt-1 + (1 - β)𝛳t RMSProp Cross Entropy 😄 1/m.sum(Y.log(a)+(1-Y).log(1-a))
- 10. Deep Neural Networks He Cross Entropy Dropout Adam ReLU
- 11. Initialisation and Cost Function He 😄 W1 = np.random.randn(layers[l], layers[l - 1]) * np.sqrt(2 / layers[l - 1]) Dropout 😄 ❌ ✅
- 12. Convolutional Networks Xavier Glorot Layers: Convolutional Layers Max Pooling Layers Fully Connected Layers Cross Entropy Dropout Adam ReLU
- 13. Convolutional Networks * = 6x6x3 3x3x16 4x4x16 4x4x16 2x2x16 2x2x16 * =
- 14. Residual Networks Xavier Glorot Layers: Convolutional Layers Bottleneck Layers Max Pooling Layers Fully Connected Layers Cross Entropy Dropout Adam ReLU
- 15. Residual Networks using Inception 5x5 Same 32 28x28x192 28x28x32 28x28x192 * 5x5x32 = 120m 1x1 16 1x1x192 28x28x192 28x28x16 5x5 Same 32 28x28x32 28x28x16 * 192 = 2.4m 28x28x32 * 5x5x16 = 10m
- 16. Capsule Networks The is no pose (translational and rotational) relationship between simpler features. Successive convolutional or max pooling layers to reduce spacial size.
- 17. Capsule Networks
- 18. Resources and References https://github.com/ekholabs/DLinK Machine Learning: Andrew Ng, Stanford University, Coursera. Neural Networks for Machine Learning: Geoffrey Hinton, University of Toronto, Coursera. Computational Neuroscience: Rajesh Rao & Adrienne Fairhall, Washington University, Coursera. Neural Networks and Deep Learning: Andrew Ng, DeepLearning.ai, Coursera. Structuring Machine Learning Projects: Andrew Ng, DeepLearning.ai, Coursera. Improving Deep Neural Networks with Hyperparameter Tuning, Regularisation and Optimisation: Andrew Ng, DeepLearning.ai, Coursera. Convolutional Neural Networks: Andrew Ng, DeepLearning.ai, Coursera. Calculus I: Jim Fowler, Ohio State University, Coursera.
- 19. Thank you!