Deep learning: Modeling high-level face features through deep networks
3 likes1,044 views
The document discusses the use of deep learning, particularly convolutional networks, for modeling high-level facial features with low error rates and reduced learning time. It highlights various techniques and adaptive methods used to improve performance in image classification, referencing results from multiple models, including Clarifai's attempts on original training data. The document also includes references to relevant literature in the field of neural networks and image recognition.
![Biological motivation
[1]](https://image.slidesharecdn.com/deeplearning-modelinghigh-levelfacefeaturesthroughdeepnetworks-140709100725-phpapp02/85/Deep-learning-Modeling-high-level-face-features-through-deep-networks-3-320.jpg)
![Learning
● Lower learning time
● Distributed networks
● Feature extraction
● Deepness with low
error rates
[2]](https://image.slidesharecdn.com/deeplearning-modelinghigh-levelfacefeaturesthroughdeepnetworks-140709100725-phpapp02/85/Deep-learning-Modeling-high-level-face-features-through-deep-networks-5-320.jpg)
![Layer n - Primitive shapes
[3]](https://image.slidesharecdn.com/deeplearning-modelinghigh-levelfacefeaturesthroughdeepnetworks-140709100725-phpapp02/85/Deep-learning-Modeling-high-level-face-features-through-deep-networks-7-320.jpg)
![Layer n+1 - Complex shapes
[3]](https://image.slidesharecdn.com/deeplearning-modelinghigh-levelfacefeaturesthroughdeepnetworks-140709100725-phpapp02/85/Deep-learning-Modeling-high-level-face-features-through-deep-networks-8-320.jpg)
![Performance ILSVRC2013
Team Comments Error rate
Clarifai
Average of multiple models on original
training data.
0.11743
Clarifai
Another attempt at multiple models on
original training data.
0.1215
Clarifai Single model trained on original data. 0.12535
NUS
adaptive non-parametric rectification of
all outputs from CNNs and refined
PASCAL VOC12 winning solution, with
further retraining on the validation set.
0.12953
NUS
adaptive non-parametric rectification of
all outputs from CNNs and refined
PASCAL VOC12 winning solution.
0.13303
[4]](https://image.slidesharecdn.com/deeplearning-modelinghigh-levelfacefeaturesthroughdeepnetworks-140709100725-phpapp02/85/Deep-learning-Modeling-high-level-face-features-through-deep-networks-9-320.jpg)
Deep learning: Modeling high-level face features through deep networks
- 1. Deep learning Modeling high-level face features through deep networks
- 2. Explaining deep learning to my mommy
- 4. Scale ● Low costs ○ ≈ $ 4000/m (100ECUs e 10TB) ● Elastic clusters ● Network vs. Disk ● GPUs ● Data o Variability o Volume o Easy acquisition
- 5. Learning ● Lower learning time ● Distributed networks ● Feature extraction ● Deepness with low error rates [2]
- 7. Layer n - Primitive shapes [3]
- 8. Layer n+1 - Complex shapes [3]
- 9. Performance ILSVRC2013 Team Comments Error rate Clarifai Average of multiple models on original training data. 0.11743 Clarifai Another attempt at multiple models on original training data. 0.1215 Clarifai Single model trained on original data. 0.12535 NUS adaptive non-parametric rectification of all outputs from CNNs and refined PASCAL VOC12 winning solution, with further retraining on the validation set. 0.12953 NUS adaptive non-parametric rectification of all outputs from CNNs and refined PASCAL VOC12 winning solution. 0.13303 [4]
- 11. Clarifai results
- 13. Clarifai results
- 14. References 1. Wong, Rachel et al. (2005): "Circuits of vertebrate retina". Nature Magazine, Volume 9 Number 1. 2. Huang G.B., Lee H., Miller E. L. (2012): “Learning Hierarchical Representations for Face Verification with Convolutional Deep Belief Networks” 3. Ng A. et al (2012): “Emergence of Object-Selective Features in Unsupervised Feature Learning.” 4. Image-net.org, (2014). ImageNet. Available at: http://www.image-net.org/.
Editor's Notes
- #3: Invariance theory: Objects are recognized by structural information or individual parts. Rotation and completion are made by invariant brain models.
- #4: Especialização das células Células horizontais -> Reconhecer formas difusas Células bipolares -> Contraste e luz Células amácrinas -> Resolução (ajuste de foco) Células ganglionares -> Separação e caracteríticas
- #5: m1.medium HDD - 120MB/s SSD - 600MB/s GPUs +100x rápidos que CPUs (para operações matemáticas)
- #6: Representações automáticas = Aprendizado não-supervisionado de características Redes neurais menos direcionadas = Aprendem características gerais
- #7: Convolução resulta em aproveitamento de pesos.