Artem Melnytskyi "Friendly Сo-pilot as a Practical AI Application"
- 1. Friendly co-piloting as a practical deep learning and AI application
- 2. History
- 3. Autopilot, is it there? 46 Corporations Working On Autonomous Vehicles
- 8. But what if we have ideal AI pilot?
- 12. What about sober adequate driver?
- 14. How can we help drivers? ●Watch the driver ●Alert about distraction or mis-condition ●Watch the road and road objects ●Alert about any potential danger
- 15. How can we help drivers? https://www.pyimagesearch.com/2017/05/08/drowsiness-detection-opencv/
- 16. And for all this things need hardware...
- 17. LIDAR ~7k $ GPS ~46$ GPS ~449$ US sensors ~14$ IR sensor ~26$ TOTAL ~10k $ Ok. Google you won
- 18. Musk has called LIDAR “a crutch” for the self-driving industry and has defended Tesla’s strategy of achieving “full autonomy” using only cameras, radar, and ultrasonic sensors.
- 19. Ok… but we need big team? Or some collaboration for working on it ? Yes…?
- 20. Ok… but we need big team? Or some collaboration for working on it ? Yes…?
- 21. This guy don't think so. George Francis Hotz (born October 2, 1989)
- 22. So we want something like this Comma AI
- 23. Level 0 as you guess – our level...
- 24. What we will need? Object recognition Road sign recognition Road recognition Decision maker Pretty ugly UI (well done by default)
- 25. First three point its about recognition ●Traditional CV or DL ? From this point we will not back to CV methods. And
- 26. Several facts about NN
- 27. Several facts about NN
- 28. Several facts about NN
- 30. Choosing NN
- 32. YOLOv3 train/integration plans ●1. Build darknet project ( in manual we trust … ) ●2. Create config file ●2. Create train dataset ●3 Train model ●4 Test model ●5. Integrate darknet library, create wrapper. ●5 Integrate trained model.
- 33. Build darknet project ( in manual we trust … ) In INTERNET you can find about 10 git repo with darknet sources.... My advice, just remember name AlexAB
- 34. Create config file yolo9000.cfg ●Classes=9418 ●Filters=28269 ●Train time ~1 week yolo-tiny.cfg ●Classes=20 ●Filters=128 ●Train time ~ 20 min yolov3- tiny_road_signs.cfg ●Classes=183 ●Filters=564 yolov3-tiny_road_xxx.cfg ●Classes=10 ●Filters=30
- 36. Object marking
- 37. Road signs data Who could have guessed that Ukrainian road signs are not universal ... Who could have guessed that there are NO universal road signs ...
- 38. And the problem is more serious than I thought.
- 39. And all I have to do is make the base myself Variant A ●Download video trips to Ukraine from YouTube. ●Split into frames, and mark in manual. ●30 seconds per frame. Total: 5,400,000 seconds. ●This is only 187.5 days. Variant B Generate base. … and pray to the gods of deep learning. I need around 1k for each road sing images… So I need ~180k images
- 40. Lets do it!
- 41. Take most important signs Mark one of them
- 42. ●21 lines. And we have mark all of them
- 43. Need more data ... ●Imgaug lib, and another 66 lines of code
- 44. Backgrounds ●Cut out every 10th frame ●Cut out every 40th frame (played ugly chanson) ●Cut out every 10th frame And we have around 2k backgrounds
- 45. 92 lines, And king of cycles! And there are 162k like this.
- 46. Yolo3 train
- 47. Test
- 49. Is it cross-platform? ●QT (Android, linux , windows,AGL, QNX, IOS) ●Darknet (Android, Linux, AGL ,QNX, Windows, IOS )
- 50. Road detection SegNet – commercial, very slow, too big
- 51. Road detection Autoencoder neural networks
- 52. Road detection Autoencoder neural networks
- 54. Road detection Autoencoder neural networks
- 55. Road detection Autoencoder neural networks
- 56. Road detection Manually marked image
- 58. Road detection Recognized road shape
- 59. Road detection Autoencoder neural networks
- 60. Road detection Autoencoder training: ●NN Architecture (layers, dimensions, ...) ●Dataset preparation
- 61. Road detection ●NN Architecture: ●320 x 180 grayscale input/output ●19 layers (12 encoder, 7 decoder) ●~ 250k weights to train
- 62. Road detection ●Dataset preparation: MANY images :)
- 63. Autoencoder training ●Data marking by hand, semi-automatic, automatic Qt-based tool Python scripts Time...
- 64. Autoencoder training ●Data frames, no data frames, augmentation
- 65. Autoencoder training … and more frames ...
- 66. Autoencoder training … and MORE frames, about 37800 total
- 67. Autoencoder training ●NVidia Volta on Google Colab for free :)
- 68. Convolution kernels visualization Somewhere in VGG-16 internal layers
- 71. Convolution kernels visualization Somewhere in VGG-16 internal layers
- 74. And here is our result https://www.youtube.com/watch?v=g9OujeQM6uY&t=121s
- 75. Neuromonks Thank you, friends: - Stanislav Kusovskiy - Taras Hetman - Alexander Artemchuk - Volodymyr Bahrii - Evgenii Fastovets
- 76. Questions?