SlideShare a Scribd company logo
SCOPS: Self-Supervised Co-Part
Segmentation
UC Merced, NVIDIA
Wei-Chih Hung, Varun Jampani, Sifei Liu, Pavlo Molchanov, Ming-Hsuan Yang, and
Jan Kautz
Scops self supervised co-part segmentation
The problem
• Fully supervised
• Costly for the annotation work
• Hard to generalize to unseen categories
• Self-supervised
• Learn part segmentations that are semantically consistent across different
object instances, given only an image collection of the same object category.
• Class agnostic
Contributions
• Geometric concentration
• Geometric Concentration Loss
• Robustness to variations
• Equivariance Loss
• Semantic consistency
• Semantic Consistency Loss
• Objects as union of parts
• Saliency Constraint
Overall Framework
• Backbone: DeepLab-V2(ResNet50)
• Output: 𝑅 = 𝐹(𝐼; 𝜃𝑓) ∈ [0,1] 𝐾+1 ∗𝐻∗𝑊
• K is the number of parts
Geometric Concentration Loss
• Observation:
• Pixels belonging to the same object part are spatially concentrated or form a
connected component.
• Minimize the variance of spatial probability distribution
• The part center for a part k along axis u
Equivariance Loss
• Random spatial Transform Ts
• Random appearance perturbation
Ta
Transformed
part center
Semantic Consistency Loss
Pretrained
on ILSVRC
Avoid the bias towards
background

More Related Content

PDF
Deep learning for person re-identification
PDF
Cross-domain complementary learning with synthetic data for multi-person part...
PDF
Step zhedong
PPTX
Visual saliency
PDF
Image Synthesis From Reconfigurable Layout and Style
PPTX
Polysemous Visual-Semantic Embedding for Cross-Modal Retrieval
PPTX
Weijian image retrieval
PPTX
Video object detection
Deep learning for person re-identification
Cross-domain complementary learning with synthetic data for multi-person part...
Step zhedong
Visual saliency
Image Synthesis From Reconfigurable Layout and Style
Polysemous Visual-Semantic Embedding for Cross-Modal Retrieval
Weijian image retrieval
Video object detection

More from 哲东 郑 (20)

PDF
Center nets
PPTX
C2 ae open set recognition
PPTX
Sota semantic segmentation
PPTX
Deep randomized embedding
PPTX
Semantic Image Synthesis with Spatially-Adaptive Normalization
PPTX
Instance level facial attributes transfer with geometry-aware flow
PPTX
Learning to adapt structured output space for semantic
PPTX
Unsupervised Learning of Object Landmarks through Conditional Image Generation
PPTX
Graph based global reasoning networks
PPTX
Style gan
PDF
Vi2vi
PPTX
Variational Discriminator Bottleneck
PPTX
GNorm and Rethinking pre training-ruijie
PPTX
Smoothed manifold
PPTX
Controllable image to-video translation
PPTX
Comparator networks
PPTX
Swwae ruijie
PDF
UNIT -Deng
PDF
Pancreas Segmentation
PPTX
Dense pose
Center nets
C2 ae open set recognition
Sota semantic segmentation
Deep randomized embedding
Semantic Image Synthesis with Spatially-Adaptive Normalization
Instance level facial attributes transfer with geometry-aware flow
Learning to adapt structured output space for semantic
Unsupervised Learning of Object Landmarks through Conditional Image Generation
Graph based global reasoning networks
Style gan
Vi2vi
Variational Discriminator Bottleneck
GNorm and Rethinking pre training-ruijie
Smoothed manifold
Controllable image to-video translation
Comparator networks
Swwae ruijie
UNIT -Deng
Pancreas Segmentation
Dense pose
Ad

Recently uploaded (20)

PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
PDF
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
PDF
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
PDF
Review of recent advances in non-invasive hemoglobin estimation
PDF
Diabetes mellitus diagnosis method based random forest with bat algorithm
PPTX
Digital-Transformation-Roadmap-for-Companies.pptx
PDF
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
PPT
“AI and Expert System Decision Support & Business Intelligence Systems”
PPTX
20250228 LYD VKU AI Blended-Learning.pptx
PPTX
ACSFv1EN-58255 AWS Academy Cloud Security Foundations.pptx
PPTX
Cloud computing and distributed systems.
PDF
Spectral efficient network and resource selection model in 5G networks
PPTX
Programs and apps: productivity, graphics, security and other tools
PDF
Building Integrated photovoltaic BIPV_UPV.pdf
PDF
Machine learning based COVID-19 study performance prediction
PDF
Optimiser vos workloads AI/ML sur Amazon EC2 et AWS Graviton
PDF
Encapsulation_ Review paper, used for researhc scholars
PDF
The Rise and Fall of 3GPP – Time for a Sabbatical?
PDF
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
PDF
Encapsulation theory and applications.pdf
Dropbox Q2 2025 Financial Results & Investor Presentation
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Review of recent advances in non-invasive hemoglobin estimation
Diabetes mellitus diagnosis method based random forest with bat algorithm
Digital-Transformation-Roadmap-for-Companies.pptx
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
“AI and Expert System Decision Support & Business Intelligence Systems”
20250228 LYD VKU AI Blended-Learning.pptx
ACSFv1EN-58255 AWS Academy Cloud Security Foundations.pptx
Cloud computing and distributed systems.
Spectral efficient network and resource selection model in 5G networks
Programs and apps: productivity, graphics, security and other tools
Building Integrated photovoltaic BIPV_UPV.pdf
Machine learning based COVID-19 study performance prediction
Optimiser vos workloads AI/ML sur Amazon EC2 et AWS Graviton
Encapsulation_ Review paper, used for researhc scholars
The Rise and Fall of 3GPP – Time for a Sabbatical?
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
Encapsulation theory and applications.pdf
Ad

Scops self supervised co-part segmentation

  • 1. SCOPS: Self-Supervised Co-Part Segmentation UC Merced, NVIDIA Wei-Chih Hung, Varun Jampani, Sifei Liu, Pavlo Molchanov, Ming-Hsuan Yang, and Jan Kautz
  • 3. The problem • Fully supervised • Costly for the annotation work • Hard to generalize to unseen categories • Self-supervised • Learn part segmentations that are semantically consistent across different object instances, given only an image collection of the same object category. • Class agnostic
  • 4. Contributions • Geometric concentration • Geometric Concentration Loss • Robustness to variations • Equivariance Loss • Semantic consistency • Semantic Consistency Loss • Objects as union of parts • Saliency Constraint
  • 5. Overall Framework • Backbone: DeepLab-V2(ResNet50) • Output: 𝑅 = 𝐹(𝐼; 𝜃𝑓) ∈ [0,1] 𝐾+1 ∗𝐻∗𝑊 • K is the number of parts
  • 6. Geometric Concentration Loss • Observation: • Pixels belonging to the same object part are spatially concentrated or form a connected component. • Minimize the variance of spatial probability distribution • The part center for a part k along axis u
  • 7. Equivariance Loss • Random spatial Transform Ts • Random appearance perturbation Ta Transformed part center
  • 8. Semantic Consistency Loss Pretrained on ILSVRC Avoid the bias towards background