SlideShare a Scribd company logo

Les net

Download as PPTX, PDF
H
heedaeKwon

hi

Engineering
1 of 12
Download to read offline
1
2
3
4
5
6
7
8
9
10
11
12
Deep Residual Learning for Image Recognition Feb 21, 2021 HeeDae Kwon
Contents • 1. INTRODUCTION • 2. Related Work • 3. Residual Learning • 4. Experiments
Introduction The deeper the depth , the better the result. Q : really? Problem : degradation (vanishing/exploding gradient) Solution : Residual learning
Related Works • Residual Representations => VLAD image recognition • Shortcut Connection => Multi-layer perceptrons
Residual Learning 기존의 networks H(x) Mapping F(x) = H(x) - x H(x)에근사 Residual mapping F(x) 를 optimize 으로 쉬워진 난이도
Residual Learning
Achitecture Plain network 1. 동일한 output feature map size에 대해, layer는 동일한 수의 filter를 갖는다. 2. feature map size가 절반 인 경우, layer 당의 time complexity를 보전하기 위해 filter의 수를 2배로 한다 Residual network 1.zero entry를 추가로 padding하여 dimension matching 후 identity mapping을 수행한다. (별도의 parameter가 추가되지 않음) 2. y = F(x, {Wi}) + Wsx의 projection shortcut을 dimension matching에 사용한다.
Implementaion • 각각의 conv layer와 activation 사이에는 batch normalization을 사용하며, He initialization 기법으 로 weight를 초기화하여 모든 plain/residual nets을 학습한다. • batch normalization에 근거해 dropout을 사용하지 않는다 • learning rate는 0.1에서 시작하여, error plateau 상 태마다 rate를 10으로 나누어 적용하며, decay는 0.0001, momentum은 0.9로 한 SGD를 사용했다. • mini-batch size는 256로 했으며, iteration은 총 600K회 수행된다.
Experiments Plain 18 > Plain 34 ResNet 18 < ResNet 34 Plain 18 < ResNet 18,34
Experiments ResNet - 152 ResNet -152 ResNet
Experiments
감사합니다

More Related Content

PPTX
Recurrent Neural Net의 이론과 설명
홍배 김
 
PPTX
Survey on Monocular Depth Estimation
범준 김
 
PPTX
03.12 cnn backpropagation
Dea-hwan Ki
 
PDF
Deep Learning & Convolutional Neural Network
agdatalab
 
PPTX
Deep learning seminar_snu_161031
Jinwon Lee
 
PPTX
Visualizing data using t-SNE
홍배 김
 
PDF
FCN to DeepLab.v3+
Whi Kwon
 
PDF
[Paper] eXplainable ai(xai) in computer vision
Susang Kim
 
Recurrent Neural Net의 이론과 설명
홍배 김
 
Survey on Monocular Depth Estimation
범준 김
 
03.12 cnn backpropagation
Dea-hwan Ki
 
Deep Learning & Convolutional Neural Network
agdatalab
 
Deep learning seminar_snu_161031
Jinwon Lee
 
Visualizing data using t-SNE
홍배 김
 
FCN to DeepLab.v3+
Whi Kwon
 
[Paper] eXplainable ai(xai) in computer vision
Susang Kim
 

What's hot (20)

PDF
[기초개념] Graph Convolutional Network (GCN)
Donghyeon Kim
 
PPTX
Dsh data sensitive hashing for high dimensional k-nn search
WooSung Choi
 
PDF
Deep learning super resolution
NAVER Engineering
 
PDF
Attention is all you need 설명
Junho Lee
 
PPTX
Convolutional Neural Networks
Sanghoon Yoon
 
PPTX
Data Visualization and t-SNE
Hyeongmin Lee
 
PDF
Lecture 3: Unsupervised Learning
Sang Jun Lee
 
PDF
Lecture 4: Neural Networks I
Sang Jun Lee
 
PPTX
MNIST for ML beginners
홍배 김
 
PDF
deep encoder, shallow decoder reevaluating non-autoregressive machine transl...
taeseon ryu
 
PPTX
Convolutional neural networks
HyunjinBae3
 
PPTX
Learning to remember rare events
홍배 김
 
PPTX
Murpy's Machine Learing: 10. Directed Graphical Model
Jungkyu Lee
 
PPTX
Graph Neural Network (한국어)
Jungwon Kim
 
PDF
CNN Introduction
pko89403
 
PPTX
Convolution 종류 설명
홍배 김
 
PDF
Detecting fake jpeg images
NAVER Engineering
 
PDF
[PR12] image super resolution using deep convolutional networks
Taegyun Jeon
 
PPTX
AlexNet, VGG, GoogleNet, Resnet
Jungwon Kim
 
PDF
여러 컨볼루션 레이어 테크닉과 경량화 기법들
Dongyi Kim
 
[기초개념] Graph Convolutional Network (GCN)
Donghyeon Kim
 
Dsh data sensitive hashing for high dimensional k-nn search
WooSung Choi
 
Deep learning super resolution
NAVER Engineering
 
Attention is all you need 설명
Junho Lee
 
Convolutional Neural Networks
Sanghoon Yoon
 
Data Visualization and t-SNE
Hyeongmin Lee
 
Lecture 3: Unsupervised Learning
Sang Jun Lee
 
Lecture 4: Neural Networks I
Sang Jun Lee
 
MNIST for ML beginners
홍배 김
 
deep encoder, shallow decoder reevaluating non-autoregressive machine transl...
taeseon ryu
 
Convolutional neural networks
HyunjinBae3
 
Learning to remember rare events
홍배 김
 
Murpy's Machine Learing: 10. Directed Graphical Model
Jungkyu Lee
 
Graph Neural Network (한국어)
Jungwon Kim
 
CNN Introduction
pko89403
 
Convolution 종류 설명
홍배 김
 
Detecting fake jpeg images
NAVER Engineering
 
[PR12] image super resolution using deep convolutional networks
Taegyun Jeon
 
AlexNet, VGG, GoogleNet, Resnet
Jungwon Kim
 
여러 컨볼루션 레이어 테크닉과 경량화 기법들
Dongyi Kim
 
Ad

Similar to Les net (20)

PPTX
Review EDSR
Woojin Jeong
 
PDF
History of Vision AI
Tae Young Lee
 
PDF
[컴퓨터비전과 인공지능] 8. 합성곱 신경망 아키텍처 4 - ResNet
jdo
 
PDF
밑바닥부터 시작하는딥러닝 8장
Sunggon Song
 
PPTX
Review Wide Resnet
Woojin Jeong
 
PPTX
Image net classification with deep convolutional neural networks
Korea, Sejong University.
 
PDF
Understanding deep learning requires rethinking generalization (2017) 2 2(2)
정훈 서
 
PDF
Alexnet paper review
오 혜린
 
PDF
LeNet & GoogLeNet
Institute of Agricultural Machinery, NARO
 
PPTX
Review Dense net
Woojin Jeong
 
PPTX
Image Deep Learning 실무적용
Youngjae Kim
 
PDF
Invertible residual networks Review
수철 박
 
PPTX
Unsupervised learning for real-world super-resolution review presentation
Seoung-Ho Choi
 
PDF
Dl from scratch(8)
Park Seong Hyeon
 
PPTX
Dense sparse-dense training for dnn and Other Models
Dong Heon Cho
 
PDF
AnoGAN을 이용한 철강 소재 결함 검출 AI
HYEJINLIM10
 
PDF
Deep neural networks cnn rnn_ae_some practical techniques
Kang Pilsung
 
PDF
Summary in recent advances in deep learning for object detection
창기 문
 
PDF
Summary in recent advances in deep learning for object detection
창기 문
 
PDF
델켐매거진5호
kr_delcam
 
Review EDSR
Woojin Jeong
 
History of Vision AI
Tae Young Lee
 
[컴퓨터비전과 인공지능] 8. 합성곱 신경망 아키텍처 4 - ResNet
jdo
 
밑바닥부터 시작하는딥러닝 8장
Sunggon Song
 
Review Wide Resnet
Woojin Jeong
 
Image net classification with deep convolutional neural networks
Korea, Sejong University.
 
Understanding deep learning requires rethinking generalization (2017) 2 2(2)
정훈 서
 
Alexnet paper review
오 혜린
 
LeNet & GoogLeNet
Institute of Agricultural Machinery, NARO
 
Review Dense net
Woojin Jeong
 
Image Deep Learning 실무적용
Youngjae Kim
 
Invertible residual networks Review
수철 박
 
Unsupervised learning for real-world super-resolution review presentation
Seoung-Ho Choi
 
Dl from scratch(8)
Park Seong Hyeon
 
Dense sparse-dense training for dnn and Other Models
Dong Heon Cho
 
AnoGAN을 이용한 철강 소재 결함 검출 AI
HYEJINLIM10
 
Deep neural networks cnn rnn_ae_some practical techniques
Kang Pilsung
 
Summary in recent advances in deep learning for object detection
창기 문
 
Summary in recent advances in deep learning for object detection
창기 문
 
델켐매거진5호
kr_delcam
 
Ad

More from heedaeKwon (18)

PPTX
Generative adversarial nets
heedaeKwon
 
PPTX
Generating sequences with recurrent neural networks
heedaeKwon
 
PPTX
Fully convolutional networks for semantic segmentation
heedaeKwon
 
PPTX
Feature pyramid networks for object detection
heedaeKwon
 
PPTX
Attention is all you need
heedaeKwon
 
PPTX
Se net
heedaeKwon
 
PPTX
Perceptual losses for real time style transfer and super-resolution
heedaeKwon
 
PPTX
Localisation network
heedaeKwon
 
PPTX
Learning to learn by gradient descent by gradient descent
heedaeKwon
 
PPTX
Grad cam visual explanations from deep networks via gradient-based localizati...
heedaeKwon
 
PPTX
Goog lenet
heedaeKwon
 
PPTX
Learning deep features for discriminative localization
heedaeKwon
 
PPTX
Image net classification with deep convolutional neural networks
heedaeKwon
 
PPTX
Show, attend and tell
heedaeKwon
 
PPTX
Vgg
heedaeKwon
 
PPTX
A neural image caption generator
heedaeKwon
 
PPTX
A.i
heedaeKwon
 
PPTX
Ai basic
heedaeKwon
 
Generative adversarial nets
heedaeKwon
 
Generating sequences with recurrent neural networks
heedaeKwon
 
Fully convolutional networks for semantic segmentation
heedaeKwon
 
Feature pyramid networks for object detection
heedaeKwon
 
Attention is all you need
heedaeKwon
 
Se net
heedaeKwon
 
Perceptual losses for real time style transfer and super-resolution
heedaeKwon
 
Localisation network
heedaeKwon
 
Learning to learn by gradient descent by gradient descent
heedaeKwon
 
Grad cam visual explanations from deep networks via gradient-based localizati...
heedaeKwon
 
Goog lenet
heedaeKwon
 
Learning deep features for discriminative localization
heedaeKwon
 
Image net classification with deep convolutional neural networks
heedaeKwon
 
Show, attend and tell
heedaeKwon
 
Vgg
heedaeKwon
 
A neural image caption generator
heedaeKwon
 
A.i
heedaeKwon
 
Ai basic
heedaeKwon
 

Les net

  • 1. Deep Residual Learning for Image Recognition Feb 21, 2021 HeeDae Kwon
  • 2. Contents • 1. INTRODUCTION • 2. Related Work • 3. Residual Learning • 4. Experiments
  • 3. Introduction The deeper the depth , the better the result. Q : really? Problem : degradation (vanishing/exploding gradient) Solution : Residual learning
  • 4. Related Works • Residual Representations => VLAD image recognition • Shortcut Connection => Multi-layer perceptrons
  • 5. Residual Learning 기존의 networks H(x) Mapping F(x) = H(x) - x H(x)에근사 Residual mapping F(x) 를 optimize 으로 쉬워진 난이도
  • 7. Achitecture Plain network 1. 동일한 output feature map size에 대해, layer는 동일한 수의 filter를 갖는다. 2. feature map size가 절반 인 경우, layer 당의 time complexity를 보전하기 위해 filter의 수를 2배로 한다 Residual network 1.zero entry를 추가로 padding하여 dimension matching 후 identity mapping을 수행한다. (별도의 parameter가 추가되지 않음) 2. y = F(x, {Wi}) + Wsx의 projection shortcut을 dimension matching에 사용한다.
  • 8. Implementaion • 각각의 conv layer와 activation 사이에는 batch normalization을 사용하며, He initialization 기법으 로 weight를 초기화하여 모든 plain/residual nets을 학습한다. • batch normalization에 근거해 dropout을 사용하지 않는다 • learning rate는 0.1에서 시작하여, error plateau 상 태마다 rate를 10으로 나누어 적용하며, decay는 0.0001, momentum은 0.9로 한 SGD를 사용했다. • mini-batch size는 256로 했으며, iteration은 총 600K회 수행된다.
  • 9. Experiments Plain 18 > Plain 34 ResNet 18 < ResNet 34 Plain 18 < ResNet 18,34
  • 10. Experiments ResNet - 152 ResNet -152 ResNet