“An Introduction to Semantic Segmentation,” a Presentation from Au-Zone Technologies
0 likes51 views
The document provides an introduction to image segmentation, detailing its importance in computer vision through various approaches: semantic, instance, and panoptic segmentation. Practical applications include areas such as autonomous vehicles, medical imaging, and agriculture, with emphasis on accuracy metrics and computational requirements. The document concludes by highlighting the distinctions between segmentation types and resources for further learning.
“An Introduction to Semantic Segmentation,” a Presentation from Au-Zone Technologies
- 1. Introduction to Semantic Segmentation Sébastien Taylor V.P. of Research & Development Au-Zone Technologies
- 2. • Introduction to segmentation • Practical examples and applications • Various types of segmentation • Accuracy metrics • Computational requirements • Resources Outline 2 © 2024 Au-Zone Technologies
- 3. • Image Segmentation is a process that subdivides an image into its constituent parts or objects. • Key task in computer vision and image processing • It can be formulated as a pixel classification problem with three different approaches (semantic, instance and panoptic) Introduction to Segmentation 3 © 2024 Au-Zone Technologies
- 4. Image Segmentation vs. Object Detection 4 © 2024 Au-Zone Technologies
- 5. • Autonomous vehicles • Smart agriculture • Drones and aerial imaging • Medical image diagnosis • Image editing • Dataset augmentation Practical Examples 5 © 2024 Au-Zone Technologies
- 6. • Semantic segmentation: produces a contextual description of the “stuff” in the image. Classes are isolated but not objects within the same class. We don’t have access to a single object. • Instance segmentation: produces a better description that can list objects as individual instances of “things” but lower generalization on the environment and background “stuff”. • Panoptic segmentation: Combines semantic and instance segmentation. We have access to the environmental context but also to the individual objects. So, we see both “stuff” and “things”. Instance, Semantic and Panoptic Segmentation 6 Image Semantic Segmentation Instance Segmentation Panoptic Segmentation © 2024 Au-Zone Technologies
- 7. Image Segmentation Using Deep Learning 7 • Deconstruction: Feature extraction (backbone, encoder) • Reconstruction: Upsampler (decoder) Deconstruction Reconstruction © 2024 Au-Zone Technologies
- 8. Deep Learning Segmentation Architecture 8 © 2023 Au-Zone Technologies Encoder Decoder
- 9. • 1-hot encoding, just like classification • Score applied to each pixel • Class with highest score sets the pixel Semantic Segmentation Output 9 © 2024 Au-Zone Technologies
- 10. Instance Segmentation – Naïve 10 Detections • Additional model output for computing bounding boxes • Same as SSD, YOLO, etc… • Boxes are post-processed to re- colour masks in order to distinguish instances. • Overlapping instances will be poorly segmented because of box limitations. © 2024 Au-Zone Technologies
- 11. • Additional model output computes per-instance mask predictions. • Learns to separate objects in each mask which are then fused with semantic mask. • Handles overlapping instances. Instance Segmentation – Proto Masks 11 © 2024 Au-Zone Technologies
- 12. • Extension of detection models. Inherently instance based. • Instead of predicting boxes for objects, the model predicts masks. Instance Segmentation – Box Masks 12 © 2024 Au-Zone Technologies
- 13. Fusing semantic and instance segmentation to detect “things” and “stuff” Panoptic Segmentation 13 © 2024 Au-Zone Technologies
- 14. • Label masks • Object polygons • Very high annotation effort • “Segment Anything Model” has been a game changer for annotation effort Dataset Types 14 © 2024 Au-Zone Technologies
- 15. • Similar IoU concept as detection • Panoptic Quality “PQ” is a new metric and applied, in part, to all segmentation challenges • PQ metrics for “things” and “stuffs” categories • COCO metrics “Panoptic Evaluation” Accuracy Metrics 15 © 2024 Au-Zone Technologies
- 16. • Same backbone as detection • Segmentation head incurs ~20% overhead • Post-processing demands • Instance and panoptic incur additional overhead Computational Requirements 16 © 2024 Au-Zone Technologies
- 17. • Semantic segmentation is a technique that enables us to isolate different objects in an image along their contours. • Improves on detection models for objects with more complex shapes. • It can be considered an image classification task at a pixel level. Conclusions 17 © 2024 Au-Zone Technologies
- 18. • Semantic segmentation classifies all pixels in an image by their class. • Instance segmentation refines the semantic masks to separate each object instance. • Panoptic segmentation fuses semantic and instance segmentation into a single unified model with knowledge of “things” and “stuff”. Conclusions 18 © 2024 Au-Zone Technologies
- 19. • Datasets • https://cocodataset.org/ • https://www.cityscapes-dataset.com/ • https://ai.facebook.com/datasets/segment-anything/ • Models • https://towardsdatascience.com/u-net-explained-understanding-its-image-segmentation- architecture-56e4842e313a • https://learnopencv.com/yolov5-instance-segmentation/ • https://segment-anything.com/ Resources 19 © 2024 Au-Zone Technologies