Deep Learning
Dr. Baljit Singh Khehra
Professor
CSE Department
Baba Banda Singh Bahadur Engineering College
Fatehgarh Sahib-140407, Punjab, India
Convolution


 

M1 N1
N1
f * g (x, y)
f * g (x) 
 
0

   f (,)g(x , y  )
 0 0
 f (,)g(x , y  )dd
f ()g(x )
 f ()g(x )d
2D(continuous, discrete) :
1D(continuous, discrete) : Input
Kernel
Output is
sometimes called
Feature map
Convolution
32
3
32x32x3 image
width
height
32
depth
Digital Color Image
32x32x3 image
5x5x3 filter
32
Convolve the filter with the image
i.e. “slide over the image spatially,
computing dot products”
32
3
Convolutions: More detail
32
3
Convolution Layer
32x32x3 image
5x5x3 filter
32
1 number:
the result of taking a dot product between the
filter and a small 5x5x3 chunk of the image
(i.e. 5*5*3 =75-dimensional dot product + bias)
Convolutions: More detail
32
3
activation mapConvolution Layer
32x32x3 image
5x5x3 filter
32
1
28
28
convolve (slide) over all
spatial locations
Convolutions: More detail
32
3 6
28
For example, if we had 6 5x5 filters, we’ll get 6 separate activation
maps:
activation maps
32
28
Convolution Layer
We stack these up to get a “new image” of size28x28x6!
Size of Image after Convolution
 Input Image of Size M×N denoted as
 If Number of Filter: NF applied on input image, then Activation Maps will be NF
 Size of each filter is m×n
 Filters are denoted as
 Let stride be s and padding be p
 Then, size of each activation map
),(),(),(
:),(
yxByxGyxR
yxf
fff
),(,..........),,(),,( 21 yxgyxgyxg NF
),(),(),(
.....................................
),(),(),(
),(),(),(
222
111
kjhkjhkjh
kjhkjhkjh
kjhkjhkjh
b
NF
g
NF
r
NF
bgr
bgr
















1
2
1
2
s
npN
s
mpM
Example
 Input Image of Size M×N=32×32 denoted as
 If Number of Filter: NF=6 applied on input image, then Activation Maps will be
NF=6
 Size of each filter is m×n=5×5
 Filters are denoted as
 Let stride be s=1 and padding be p=0
 Then, size of each activation map will be 28×28
),(),(),(
:),(
yxByxGyxR
yxf
fff
),(,..........),,(),,( 621 yxgyxgyxg
),(),(),(
.....................................
),(),(),(
),(),(),(
666
222
111
kjhkjhkjh
kjhkjhkjh
kjhkjhkjh
bgr
bgr
bgr












1
1
50*232
1
1
50*232
Input Image: 32x32x3
10 5x5 filters with stride (s)= 1,
pad (p)=2
Output volume size: ?
Another Example
Input volume: 32x32x3
10 5x5 filters with stride 1, pad 2
Output volume size:
[(32+2*2-5)/1+1]×[(32+2*2-5)/1+1] = 32×32
spatially, so
32x32x10
Output
• 7
• 7x7 input (spatially)
assume 3x3 filter
• 7
A closer look at spatial dimensions:
Convolutions: More detail
• 7
• 7x7 input
(spatially)
assume 3x3
filter
• 7
A closer look at spatial dimensions:
Convolutions: More detail
• 7
• 7x7 input (spatially)
assume 3x3 filter
• 7
A closer look at spatial dimensions:
Convolutions: More detail
• 7
• 7x7 input
(spatially)
assume 3x3
filter
• 7
A closer look at spatial dimensions:
Convolutions: More detail
7
• • 7x7 input (spatially)
assume 3x3 filter
7
A closer look at spatial dimensions:
Convolutions: More detail
[(7+2*0-3)/1+1]×[(7+2*0-3)/1+1] = 5×5 Output
7x7 input (spatially)
assume 3x3 filter
applied with stride
2
7
7
A closer look at spatial dimensions:
Convolutions: More detail
7x7 input (spatially)
assume 3x3 filter
applied with stride
2
7
7
A closer look at spatial dimensions:
Convolutions: More detail
7x7 input (spatially)
assume 3x3 filter
applied with stride
2
=> 3x3 output!
7
7
A closer look at spatial dimensions:
Convolutions: More detail
[(7+2*0-3)/2+1]×[(7+2*0-3)/2+1] = 3×3 Output
7x7 input (spatially)
assume 3x3 filter
applied with stride
3?
7
7
A closer look at spatial dimensions:
Convolutions: More detail
[(7+2*0-3)/3+1]×[(7+2*0-3)/3+1] = [4/3+1]×[4/3+1] =2.33×2.33
7x7 input (spatially)
assume 3x3 filter
applied with stride
3?
7
7
A closer look at spatial dimensions:
doesn’t fit!
cannot apply 3x3 filter on
7x7 input with stride 3.
Convolutions: More detail
In practice: Common to zero pad the border
0 0 0 0 0 0
0
0
0
0
e.g. input 7x7
3x3 filter, applied with stride 1
pad with 1 pixel border => what is theoutput?
Convolutions: More detail
[(7+2*1-3)/1+1]×[(7+2*1-3)/1+1] = 7×7 Output
In practice: Common to zero pad the border
e.g. input 7x7
3x3 filter, applied with stride 2
pad with 1 pixel border => what is theoutput?
0 0 0 0 0 0
0
0
0
0
Convolutions: More detail
[(7+2*1-3)/2+1]×[(7+2*1-3)/2+1] = 4×4 Output
In practice: Common to zero pad the border
0 0 0 0 0 0
0
0
0
0
Convolutions: More detail
e.g. input 7x7
3x3 filter, applied with stride 3
pad with 1 pixel border => what is theoutput?
[(7+2*1-3)/3+1]×[(7+2*1-3)/3+1] = 3×3 Output
Preview: ConvNet is a sequence of Convolution Layers, interspersedwith
activation functions
32
32
3
28
28
6
CONV,
ReLU
e.g. 6
5x5x3
filters
Convolutions: More detail
RELU Activation Function






0
00
)(
zifz
zif
zR
Preview: ConvNet is a sequence of Convolutional Layers, interspersed with activation
functions
32
32
3
CONV,
ReLU
e.g. 6
5x5x3
filters
28
28
6
CONV,
ReLU
e.g. 10
5x5x6
filters
POOL
Andrej Karpathy
….
10
24
24
Convolutions: More detail
Pooling layer
 makes the representations smaller and more manageable
 operates over each activation map independently:
1 1 2 4
5 6 7 8
3 2 1 0
1 2 3 4
Single depth slice
x
y
max pool with 2x2 filters
and stride 2
6 8
3 4
MAX POOLING
[(CONVRELU)*NPOOL]*MFC
N: up to 5
M is Large
FC: Contains neurons that connect to the
entire input volume, as in ordinary Neural
Networks
General Architecture of CNNs
Example to recognize Car from Car, truck,
airplane, ship and horse
Deep learning

More Related Content

TXT
Efnsjdnfsuies
PDF
Blur Filter - Hanpo
PPTX
Gaussian Image Blurring in CUDA C++
PDF
Assignment4
PPTX
Constraint propagation
PDF
Bresenham Line Drawing Algorithm
PDF
Tutor 41 Big Numbers
Efnsjdnfsuies
Blur Filter - Hanpo
Gaussian Image Blurring in CUDA C++
Assignment4
Constraint propagation
Bresenham Line Drawing Algorithm
Tutor 41 Big Numbers

What's hot (20)

PDF
Image Processing 2
PPTX
Lec02 03 rasterization
PDF
Ch 06
DOCX
Dda algo notes
PDF
Distributed Machine Learning with Apache Mahout
PPT
PPT
16083116
PDF
Ch 02
PDF
Computer graphics 2
PPTX
Computer Graphics - Introduction in Brief By: Prof. Manisha Waghmare- Butkar
PPTX
Explanation on Tensorflow example -Deep mnist for expert
PPTX
DDA algorithm
PDF
PPTX
Product rule
PPTX
Digit recognizer by convolutional neural network
PPT
Line drawing algorithm and antialiasing techniques
PPT
Intro to scan conversion
PDF
PMHMathematicaSample
PDF
Bresenham derivation
PDF
Image Processing 2
Lec02 03 rasterization
Ch 06
Dda algo notes
Distributed Machine Learning with Apache Mahout
16083116
Ch 02
Computer graphics 2
Computer Graphics - Introduction in Brief By: Prof. Manisha Waghmare- Butkar
Explanation on Tensorflow example -Deep mnist for expert
DDA algorithm
Product rule
Digit recognizer by convolutional neural network
Line drawing algorithm and antialiasing techniques
Intro to scan conversion
PMHMathematicaSample
Bresenham derivation
Ad

Similar to Deep learning (20)

PDF
Introduction to Convolutional Neural Networks
PPT
Adv.TopicsAICNN.ppt
PDF
"Deep Learning" Chap.6 Convolutional Neural Net
PPTX
Convolutional Neural Network Architecture
PPTX
CNN_AH.pptx
PPTX
CNN_AH.pptx
PDF
Introduction to Applied Machine Learning
PDF
Lecture 2.A: Convolutional Networks - Full Stack Deep Learning - Spring 2021
PDF
Overview of Convolutional Neural Networks
PPT
Introduction to Deep-Learning-CNN Arch.ppt
PPTX
Introduction to convolutional networks .pptx
PDF
Applying your Convolutional Neural Networks
PPTX
Machine Learning - Introduction to Convolutional Neural Networks
PDF
Practical Deep Learning Using Tensor Flow - Sandeep Kath
PPTX
Convolutional neural network in deep learning
PPT
Deep Learning approach in Machine learning
PDF
Deep Learning: CNN and RNN Architectures pdf
PPTX
Deep Learning Unit Two Power Point Presentation
PPTX
Deep-Learning-2017-Lecture5CNN.pptx
PDF
Deep Learning - STM 6
Introduction to Convolutional Neural Networks
Adv.TopicsAICNN.ppt
"Deep Learning" Chap.6 Convolutional Neural Net
Convolutional Neural Network Architecture
CNN_AH.pptx
CNN_AH.pptx
Introduction to Applied Machine Learning
Lecture 2.A: Convolutional Networks - Full Stack Deep Learning - Spring 2021
Overview of Convolutional Neural Networks
Introduction to Deep-Learning-CNN Arch.ppt
Introduction to convolutional networks .pptx
Applying your Convolutional Neural Networks
Machine Learning - Introduction to Convolutional Neural Networks
Practical Deep Learning Using Tensor Flow - Sandeep Kath
Convolutional neural network in deep learning
Deep Learning approach in Machine learning
Deep Learning: CNN and RNN Architectures pdf
Deep Learning Unit Two Power Point Presentation
Deep-Learning-2017-Lecture5CNN.pptx
Deep Learning - STM 6
Ad

More from DrBaljitSinghKhehra (6)

PPT
Ga for shortest_path
PPT
Back propagation
PPT
Artificial Neural Networks-Supervised Learning Models
PPT
Artificial Neural Networks-Supervised Learning Models
PPT
Artificial Neural Networks-Supervised Learning Models
Ga for shortest_path
Back propagation
Artificial Neural Networks-Supervised Learning Models
Artificial Neural Networks-Supervised Learning Models
Artificial Neural Networks-Supervised Learning Models

Recently uploaded (20)

PPTX
TG Hospitality workshop Vietnam (1).pptx
PPTX
Phylogeny and disease transmission of Dipteran Fly (ppt).pptx
PPTX
Phylogeny and disease transmission of Dipteran Fly (ppt).pptx
PPTX
WEB_DEVELOPMENTGJMFGHJMGJMFJM FGJMFGHMNF
PDF
Unnecessary information is required for the
DOC
EVC毕业证学历认证,北密歇根大学毕业证留学硕士毕业证
PPTX
Ulangan Harian_TEOREMA PYTHAGORAS_8.pptx
PPTX
Unit 8#Concept of teaching and learning.pptx
PPTX
CASEWORK Pointers presentation Field instruction I
PPTX
Pharmaceutical industry and drugdevelopment.pptx
PDF
Public speaking for kids in India - LearnifyU
PDF
Yoken Capital Network Presentation Slide
PPTX
Lesson 1 (Digital Media) - Multimedia.pptx
PPTX
Analytics in Human Resource Management FY
PPTX
power point presentation ofDracena species.pptx
PPTX
Shizophrnia ppt for clinical psychology students of AS
PPTX
HOW TO HANDLE THE STAGE FOR ACADEMIA AND OTHERS.pptx
PPT
Comm.-100W-Writing-a-Convincing-Editorial-slides.ppt
PPT
Lessons from Presentation Zen_ how to craft your story visually
PPTX
Phrases and phrasal verb for a small step.
TG Hospitality workshop Vietnam (1).pptx
Phylogeny and disease transmission of Dipteran Fly (ppt).pptx
Phylogeny and disease transmission of Dipteran Fly (ppt).pptx
WEB_DEVELOPMENTGJMFGHJMGJMFJM FGJMFGHMNF
Unnecessary information is required for the
EVC毕业证学历认证,北密歇根大学毕业证留学硕士毕业证
Ulangan Harian_TEOREMA PYTHAGORAS_8.pptx
Unit 8#Concept of teaching and learning.pptx
CASEWORK Pointers presentation Field instruction I
Pharmaceutical industry and drugdevelopment.pptx
Public speaking for kids in India - LearnifyU
Yoken Capital Network Presentation Slide
Lesson 1 (Digital Media) - Multimedia.pptx
Analytics in Human Resource Management FY
power point presentation ofDracena species.pptx
Shizophrnia ppt for clinical psychology students of AS
HOW TO HANDLE THE STAGE FOR ACADEMIA AND OTHERS.pptx
Comm.-100W-Writing-a-Convincing-Editorial-slides.ppt
Lessons from Presentation Zen_ how to craft your story visually
Phrases and phrasal verb for a small step.

Deep learning

  • 1. Deep Learning Dr. Baljit Singh Khehra Professor CSE Department Baba Banda Singh Bahadur Engineering College Fatehgarh Sahib-140407, Punjab, India
  • 2. Convolution      M1 N1 N1 f * g (x, y) f * g (x)    0     f (,)g(x , y  )  0 0  f (,)g(x , y  )dd f ()g(x )  f ()g(x )d 2D(continuous, discrete) : 1D(continuous, discrete) : Input Kernel Output is sometimes called Feature map
  • 5. 32x32x3 image 5x5x3 filter 32 Convolve the filter with the image i.e. “slide over the image spatially, computing dot products” 32 3 Convolutions: More detail
  • 6. 32 3 Convolution Layer 32x32x3 image 5x5x3 filter 32 1 number: the result of taking a dot product between the filter and a small 5x5x3 chunk of the image (i.e. 5*5*3 =75-dimensional dot product + bias) Convolutions: More detail
  • 7. 32 3 activation mapConvolution Layer 32x32x3 image 5x5x3 filter 32 1 28 28 convolve (slide) over all spatial locations Convolutions: More detail
  • 8. 32 3 6 28 For example, if we had 6 5x5 filters, we’ll get 6 separate activation maps: activation maps 32 28 Convolution Layer We stack these up to get a “new image” of size28x28x6!
  • 9. Size of Image after Convolution  Input Image of Size M×N denoted as  If Number of Filter: NF applied on input image, then Activation Maps will be NF  Size of each filter is m×n  Filters are denoted as  Let stride be s and padding be p  Then, size of each activation map ),(),(),( :),( yxByxGyxR yxf fff ),(,..........),,(),,( 21 yxgyxgyxg NF ),(),(),( ..................................... ),(),(),( ),(),(),( 222 111 kjhkjhkjh kjhkjhkjh kjhkjhkjh b NF g NF r NF bgr bgr                 1 2 1 2 s npN s mpM
  • 10. Example  Input Image of Size M×N=32×32 denoted as  If Number of Filter: NF=6 applied on input image, then Activation Maps will be NF=6  Size of each filter is m×n=5×5  Filters are denoted as  Let stride be s=1 and padding be p=0  Then, size of each activation map will be 28×28 ),(),(),( :),( yxByxGyxR yxf fff ),(,..........),,(),,( 621 yxgyxgyxg ),(),(),( ..................................... ),(),(),( ),(),(),( 666 222 111 kjhkjhkjh kjhkjhkjh kjhkjhkjh bgr bgr bgr             1 1 50*232 1 1 50*232
  • 11. Input Image: 32x32x3 10 5x5 filters with stride (s)= 1, pad (p)=2 Output volume size: ? Another Example
  • 12. Input volume: 32x32x3 10 5x5 filters with stride 1, pad 2 Output volume size: [(32+2*2-5)/1+1]×[(32+2*2-5)/1+1] = 32×32 spatially, so 32x32x10 Output
  • 13. • 7 • 7x7 input (spatially) assume 3x3 filter • 7 A closer look at spatial dimensions: Convolutions: More detail
  • 14. • 7 • 7x7 input (spatially) assume 3x3 filter • 7 A closer look at spatial dimensions: Convolutions: More detail
  • 15. • 7 • 7x7 input (spatially) assume 3x3 filter • 7 A closer look at spatial dimensions: Convolutions: More detail
  • 16. • 7 • 7x7 input (spatially) assume 3x3 filter • 7 A closer look at spatial dimensions: Convolutions: More detail
  • 17. 7 • • 7x7 input (spatially) assume 3x3 filter 7 A closer look at spatial dimensions: Convolutions: More detail [(7+2*0-3)/1+1]×[(7+2*0-3)/1+1] = 5×5 Output
  • 18. 7x7 input (spatially) assume 3x3 filter applied with stride 2 7 7 A closer look at spatial dimensions: Convolutions: More detail
  • 19. 7x7 input (spatially) assume 3x3 filter applied with stride 2 7 7 A closer look at spatial dimensions: Convolutions: More detail
  • 20. 7x7 input (spatially) assume 3x3 filter applied with stride 2 => 3x3 output! 7 7 A closer look at spatial dimensions: Convolutions: More detail [(7+2*0-3)/2+1]×[(7+2*0-3)/2+1] = 3×3 Output
  • 21. 7x7 input (spatially) assume 3x3 filter applied with stride 3? 7 7 A closer look at spatial dimensions: Convolutions: More detail [(7+2*0-3)/3+1]×[(7+2*0-3)/3+1] = [4/3+1]×[4/3+1] =2.33×2.33
  • 22. 7x7 input (spatially) assume 3x3 filter applied with stride 3? 7 7 A closer look at spatial dimensions: doesn’t fit! cannot apply 3x3 filter on 7x7 input with stride 3. Convolutions: More detail
  • 23. In practice: Common to zero pad the border 0 0 0 0 0 0 0 0 0 0 e.g. input 7x7 3x3 filter, applied with stride 1 pad with 1 pixel border => what is theoutput? Convolutions: More detail [(7+2*1-3)/1+1]×[(7+2*1-3)/1+1] = 7×7 Output
  • 24. In practice: Common to zero pad the border e.g. input 7x7 3x3 filter, applied with stride 2 pad with 1 pixel border => what is theoutput? 0 0 0 0 0 0 0 0 0 0 Convolutions: More detail [(7+2*1-3)/2+1]×[(7+2*1-3)/2+1] = 4×4 Output
  • 25. In practice: Common to zero pad the border 0 0 0 0 0 0 0 0 0 0 Convolutions: More detail e.g. input 7x7 3x3 filter, applied with stride 3 pad with 1 pixel border => what is theoutput? [(7+2*1-3)/3+1]×[(7+2*1-3)/3+1] = 3×3 Output
  • 26. Preview: ConvNet is a sequence of Convolution Layers, interspersedwith activation functions 32 32 3 28 28 6 CONV, ReLU e.g. 6 5x5x3 filters Convolutions: More detail
  • 28. Preview: ConvNet is a sequence of Convolutional Layers, interspersed with activation functions 32 32 3 CONV, ReLU e.g. 6 5x5x3 filters 28 28 6 CONV, ReLU e.g. 10 5x5x6 filters POOL Andrej Karpathy …. 10 24 24 Convolutions: More detail
  • 29. Pooling layer  makes the representations smaller and more manageable  operates over each activation map independently:
  • 30. 1 1 2 4 5 6 7 8 3 2 1 0 1 2 3 4 Single depth slice x y max pool with 2x2 filters and stride 2 6 8 3 4 MAX POOLING
  • 31. [(CONVRELU)*NPOOL]*MFC N: up to 5 M is Large FC: Contains neurons that connect to the entire input volume, as in ordinary Neural Networks General Architecture of CNNs
  • 32. Example to recognize Car from Car, truck, airplane, ship and horse