Iccv2009 recognition and learning object categories p1 c03 - 3d object models
- 1. 3D object categorizationCourtesy of Prof. SilvioSavarese(U. Michigan, Ann-Arbor)
- 3. King Arthur and the knights of the Round Table.
- 4. 3D Object Categorization Weber et al. ‘00
- 5. Schneiderman et al. ’01
- 6. Capel et al ’02
- 7. Johnson & Herbert ‘99
- 8. Bronstein et al, ‘03
- 9. Ruiz-Correa et al. ’03,
- 10. Funkhouser et al ’03
- 11. Bart et al ‘04
- 12. Chiu et al. ‘07
- 13. Hoiem, et al., ’07
- 14. Yan, et al. ’07
- 15. Thomas et al. ‘06
- 16. Kushal, et al., ’07
- 17. Savarese et al, 07, 083D Object CategorizationChallenges- how to model 3D shape variability?- How to model texture (appearance) variability?- How to link texture (appearance) across views?
- 18. 3D Object CategorizationMixture of 2D single view models Weber et al. ‘00
- 19. Schneiderman et al. ’01
- 20. Bart et al. ‘04Full 3D modelsBronstein et al, ‘03
- 21. Ruiz-Correa et al. ’03,
- 22. Funkhouser et al ’03
- 23. Capel et al ’02
- 24. Johnson & Herbert ‘99Multi-view models Thomas et al. ‘06
- 25. Savarese et al, 07, 08
- 26. Chiu et al. ‘07
- 27. Hoiem, et al., ’07
- 28. Yan, et al. ’07
- 29. Kushal, et al., ’07
- 30. Liebelt et al 08
- 31. Sun, Su, Savarese et al 09a, 09bOverview Single 3D object recognition
- 32. Single view object categorization
- 33. 3D object categorizationSingle 3D object recognitionZhang et al ’95
- 34. Schmid & Mohr, ‘96
- 35. Schiele & Crowley, ’96
- 36. Lowe, ‘99
- 37. Jacob & Barsi, ‘99
- 38. Edelman et al. ’91
- 39. Ullman & Barsi, ’91
- 40. Rothwell ‘92
- 41. Linderberg, ’94
- 42. Murase & Nayar ‘94
- 43. Rothganger et al., ‘04
- 44. Ferrari et al, ’05
- 45. Brown & Lowe ’05
- 46. Snavely et al ’06
- 47. Yin & Collins, ‘07
- 48. Ballard, ‘81
- 49. Grimson & L.-Perez, ‘87
- 50. Lowe, ’87Difference of Gaussian(DOG): used in Lowe 99, Brown et al ‘05Courtesy of D. LoweHarris-Laplace: used inRothganger et al. ‘06Laplacian: used inLazebnik et al. ‘04Courtesy of Rothganger et al
- 51. Object representation: Collection of patches in 3DRothganger et al. ’06x,y,z +h,v + descriptorCourtesy of Rothganger et al
- 52. Model learning Rothganger et al. ‘03 ’06Build a 3D model: N images of object from N different view points
- 53. Match key points between consecutive views[ create sample set]Use affine structure from motion to compute 3D location and orientation + camera locations [RANSAC]
- 55. Use bundle adjustment to refine model
- 56. Upgrade model to Euclidean assuming zero skew and square pixelsRecognition [Rothganger et al. ‘03 ’06]1. Find matches between model and test image features
- 57. Recognition [Rothganger et al. ‘03 ’06]1. Find matches between model and test image features2. Generate hypothesis: Compute transformation M from N matches (N=2; affine camera; affine key points)3. Model verification Use M to project other matched 3D model features into test image
- 58. Compute residual = D(projections, measurements)Recognition [Rothganger et al. ‘03 ’06]Goal: Estimate (fit) the best M in presence of outliers
- 59. 3D Object CategorizationMixture of 2D single view models Weber et al. ‘00
- 60. Schneiderman et al. ’01
- 61. Bart et al. ‘04Full 3D modelsBronstein et al, ‘03
- 62. Ruiz-Correa et al. ’03,
- 63. Funkhouser et al ’03
- 64. Capel et al ’02
- 65. Johnson & Herbert ‘99Multi-view models Thomas et al. ‘06
- 66. Savarese et al, 07, 08
- 67. Chiu et al. ‘07
- 68. Hoiem, et al., ’07
- 69. Yan, et al. ’07
- 70. Kushal, et al., ’07
- 71. Liebelt et al 08
- 72. Sun et al 083D modelinstance3D modelinstanceFull 3D modelsBronstein et al, ‘03
- 73. Ruiz-Correa et al. ’03,
- 74. Funkhouser et al ’03
- 75. Kazhdan et al.03
- 76. Osada et al ‘02
- 77. Capel et al ’02
- 78. Johnson & Herbert ’99
- 79. Amberg et al ‘083D category model…A 3D model category is built from a collection of 3D range data or CAD models
- 80. Shape distributionsOsada et al 02Spherical harmonicsKazhdan et al. 03
- 81. 3D modelinstance3D modelinstanceFull 3D modelsBronstein et al, ‘03
- 82. Ruiz-Correa et al. ’03,
- 83. Funkhouser et al ’03
- 84. Kazhdan et al.03
- 85. Osada et al ‘02
- 86. Capel et al ’02
- 87. Johnson & Herbert ’99
- 88. Amberg et al ‘083D category model…A 3D model category is built from a collection of 3D range data or CAD models Build a 3d model is expensive
- 89. Difficult to incorporate appearance information- Need to cope with 3D alignment (orientation, scale, etc…)
- 90. 3D Object CategorizationMixture of 2D single view models Weber et al. ‘00
- 91. Schneiderman et al. ’01
- 92. Bart et al. ‘04Full 3D modelsBronstein et al, ‘03
- 93. Ruiz-Correa et al. ’03,
- 94. Funkhouser et al ’03
- 95. Capel et al ’02
- 96. Johnson & Herbert ‘99Multi-view models Thomas et al. ‘06
- 97. Savarese et al, 07, 08
- 98. Chiu et al. ‘07
- 99. Hoiem, et al., ’07
- 100. Yan, et al. ’07
- 101. Kushal, et al., ’07
- 102. Liebelt et al 08
- 103. Sun et al 08Multi-view models……3D Category model…Sparse set of interest points or parts of the objects are linked across views.
- 104. Multi-view models by rough 3d shapesYan, et al. ’07
- 105. Multi-view models by rough 3d shapesHoiem, et al., ’07
- 106. Multi-view models by ISM representations[Thomas et al. ’06] ‘Courtesy of Thomas et al. 06
- 107. Multi-view models by ISM representations[Thomas et al. ’06] ……Multi-view model…Sparse set of interest points or parts of the objects are linked across views.
- 108. A unified framework for 3D objectdetection, pose classification, pose synthesis Savarese, Fei-Fei, ICCV 07Savarese, Fei-Fei, ECCV 08Sun, Su, Savarese, Fei-Fei, CVPR 09Su, Sun, Fei-Fei, Savarese, ICCV 09 Canonical parts captures diagnostic appearance information
- 109. 2d ½ structure linking parts via weak geometryCanonical parts If physical part is planar, canonical part is stable point on the manifold
- 110. Canonical part can be computed from connected component of partsconnected component of parts
- 111. Canonical parts connected component of parts
- 112. Linkage structure
- 113. Object RecognitionQuery imagemodelAlgorithm1. Find hypotheses of canonical parts consistent with a given pose
- 114. Object RecognitionQuery imagemodelAlgorithm1. Find hypotheses of canonical parts consistent with a given pose2. Infer position and pose of other canonical parts
- 115. Object RecognitionQuery imagemodelAlgorithm1. Find hypotheses of canonical parts consistent with a given pose2. Infer position and pose of other canonical parts
- 116. Object RecognitionQuery imagemodelAlgorithm1. Find hypotheses of canonical parts consistent with a given pose2. Infer position and pose of other canonical parts3. Optimize over E, G ands to find best combination of hypothesis error
- 117. A unified framework for 3D objectdetection, pose classification, pose synthesis Savarese, Fei-Fei, ICCV 07Savarese, Fei-Fei, ECCV 08Sun, Su, Savarese, Fei-Fei, CVPR 09Su, Sun, Fei-Fei, Savarese, ICCV 09Probabilistic generative part-based model
- 118. Dense Multi-view representation on the viewing sphereA unified framework for 3D objectdetection, pose classification, pose synthesis Savarese, Fei-Fei, ICCV 07Savarese, Fei-Fei, ECCV 08Sun, Su, Savarese, Fei-Fei, CVPR 09Su, Sun, Fei-Fei, Savarese, ICCV 0910:50am Thursday, Oct 1, Oral session 9H. Su*, M. Sun*, L. Fei-Fei, S. Savarese, Learning a Dense Multi-view Representation for Detection, Viewpoint Classification and Synthesis of Object Categories.
- 119. A unified framework for 3D objectdetection, pose classification, pose synthesis Su, Sun, Fei-Fei, Savarese, ICCV 09
- 120. A unified framework for 3D objectdetection, pose classification, pose synthesis Su, Sun, Fei-Fei, Savarese, ICCV 09
- 121. A unified framework for 3D objectdetection, pose classification, pose synthesis Su, Sun, Fei-Fei, Savarese, ICCV 09
- 122. A unified framework for 3D objectdetection, pose classification, pose synthesis Su, Sun, Fei-Fei, Savarese, ICCV 09
- 123. A unified framework for 3D objectdetection, pose classification, pose synthesis Su, Sun, Fei-Fei, Savarese, ICCV 09
- 124. A unified framework for 3D objectdetection, pose classification, pose synthesis View point