Deep learning image recognition for autonomous driving(classification, object detection, segmentation)
- 1. 딥러닝을 통한 자율주행의 인지 June. 30, 2019 Dongyul Lee
- 2. • 자율주행 인지의 목적 • 자율주행의 인지 대상 • Classification • 탐지 • 의미 분할 Contents - 컨볼루션 심층 신경망을 이용한 탐지/의미분할 2
- 3. • Path planning • 경로제어, 회피제어, (depth estimation) • Static object detection for slam 자율주행 인지의 목적 3 https://taylor.raack.info/2018/01/autonomous-vehicle-technology-path-planning/ https://www.researchgate.net/figure/The-proposed-SLAM-aware-object-recognition-system-is-able-to-robustly-localize-and_fig1_319770107
- 4. 자율주행의 인지 대상 4 • Cityscape – 27(+3 void)개의 클래스에 대한 dataset 제공 • 5k fine +20k coarse annotation frames • Mapillary • BDD100k • Apolloscape https://www.cityscapes-dataset.com/
- 5. • Cityscape • Mapillary • 100 object categories, 60 of those instance-specific • 25k annotation images (+데이터의 질이 좋음) • Global geographic reach (다양한지역에서 촬영) • High variability in weather conditions and capturing times • Broad range of camera sensors, varying focal length, image aspect ratios, and different types of camera noise • Different capturing viewpoints (from road, sidewalks, and off-road) • BDD100k • Apolloscape 자율주행의 인지 대상 5 https://blog.mapillary.com/product/2017/05/03/mapillary-vistas-dataset.html
- 6. • Cityscape • Mapillary • BDD100k • Object의 경우 100k database 제공 (2D bounding box 제공) • Mapillary와 같이 다양한 속성의 분포를 갖는 data 제공 • +다양한 lane category제공 (제어를 위해 instance 구분됨) • Apolloscape 자율주행의 인지 대상 6 https://arxiv.org/pdf/1805.04687.pdf
- 7. • Cityscape • Mapillary • BDD100k • Apolloscape • resolution 3384 x 2710. • About 200K image frames with corresponding pixel-level annotations • 25 different labels covered by five groups. • Including lidar object dataset for 3D bounding box • 110 000+ frames high quality pixel-level annotations • For HD-map 자율주행의 인지 대상 7 http://apolloscape.auto/
- 8. • History 컨볼루션 심층 신경망 (CNN) 8 Illustrationof LeCun et al. 1998 from CS231n 2017 Lecture 1
- 9. • Convolution (Dilated convolution) • Non-linear Activations (ReLu, ReLu6, …) • Batch normalization (Group Normalization, Instance Normalization, …) • Pooling (max pooling, average pooling, …) • Skip connection • Loss (L1, L2, log loss, …) • => 이들의 조합 + FC (fully connected layers) Classification 9 http://cs231n.stanford.edu/slides/2019/cs231n_2019_lecture05.pdf
- 10. • Convolution (Dilated convolution) • Non-linear Activations (ReLu, ReLu6, …) • Batch normalization (Group Normalization, Instance Normalization, …) • Pooling (max pooling, average pooling, …) • Skip connection • Loss (L1, L2, log loss, …) • => 이들의 조합 + FC (fully connected layers) Classification CNN Architectures 10 http://cs231n.stanford.edu/slides/2019/cs231n_2019_lecture05.pdf
- 11. • 탐지와 의미 분할을 위한 특징(feature) 인코딩 모듈 • 그럼 어떠한 인코딩 모듈을 사용해야 하는가? • 디코딩 모듈(탐지, 의미분할)과 잘 맞는 네트워크 적용 • 동작환경(모바일 vs 서버), 요구사항 (시간, 파워, 성능)에 따라 최적의 효용을 갖는 네트워크 사용 Classification 의 역할 11 http://cs231n.stanford.edu/slides/2019/cs231n_2019_lecture09.pdf
- 12. • 이미지 내에 타겟 물체가 어느 위치에 있는지를 class 정보와 함께 찾는 것 • 위치 (x,y,w,h): (x, y) – box의 좌상단, (w, h) – box의 폭과 높이 • class 정보 : classification 탐지 12 https://towardsdatascience.com/understanding-ssd-multibox-real-time-object-detection-in-deep-learning- 495ef744fab
- 13. 탐지 13 https://towardsdatascience.com/understanding-ssd-multibox-real-time-object-detection-in-deep-learning- 495ef744fab • 어떻게 찾는가? • CNN을 통해 (w,h,c)차원의 feature로 변환 후 최적의 box를 proposal
- 14. • 어떻게 CNN을 설계하는가? • Classification의 FC layer를 제외한 layer를 인코딩(백본) 레이어로 사용 탐지 14 http://cs231n.stanford.edu/slides/2019/cs231n_2019_lecture09.pdf
- 15. 탐지 딥러닝 흐름 15 https://hoya012.github.io/blog/Tutorials-of-Object-Detection-Using-Deep-Learning-what-is-object-detection/
- 16. 탐지 딥러닝 흐름 16 https://slideslive.com/38917182/an-overview-of-googles-work-on-automl-and-future-directions
- 17. • COCO dataset leaderboard를 참조 탐지 딥러닝 흐름 17 https://hoya012.github.io/blog/Tutorials-of-Object-Detection-Using-Deep-Learning-what-is-object-detection/
- 18. • AutoML의 등장 (ICML 2019) 탐지 딥러닝 흐름 18 https://slideslive.com/38917182/an-overview-of-googles-work-on-automl-and-future-directions 기존 NASNet
- 19. • Pixel-wise classification • Semantic segmentation • Instance segmentation • Panoptic segmentation (= Semantic segmentation + Instance segmentation) 의미 분할 19 http://cs231n.stanford.edu/slides/2019/cs231n_2019_lecture12.pdf
- 26. Thank you 26