SlideShare a Scribd company logo
1Challenge the future
Point Cloud Segmentation &
Surface Reconstruction
An overview of methods
Ir. Pirouz Nourian
PhD candidate & Instructor, chair of Design Informatics, since 2010
MSc in Architecture 2009
BSc in Control Engineering 2005
MSc Geomatics, GEO1004, Directed by Dr. Sisi Zlatanova
2Challenge the future
Topics
A very short introduction to segmentation and surface reconstruction
• Big Picture, Problem Statement
• Goal is to make a [simple] Brep, with few faces
• General Approaches to this problem (surface
reconstruction):
• Volumetric, Implicit, using voxels and iso-surfaces
• Alpha Shapes (Ball-Pivoting)
• Delaunay or Voronoi Based?
• Poisson?
• General Approaches to this problem (surface
reconstruction):
3Challenge the future
Topics
A very short introduction to segmentation and surface reconstruction
• Big Picture, Problem Statement
• Goal is to make a [simple] Brep, with few faces
Images courtesy of Ajith Sampath
?
4Challenge the future
A few approaches to surface reconstruction
• Voxels  Field of Signed-Distance Iso-Surface
• Ball-Pivoting (alpha shapes) triangulation
• Planar Segments Neighbourhood Matrix Edge List Simple Mesh
• Poisson Surface Reconstruction
• Implicit Function
• [ Hoppe et al. 92 ]
• Volumetric Reconstruction
• [ Curless and Levoy 96 ]
• Alpha Shapes
• [ Edelsbrunner and Mucke 94 ]
• 3D Voronoi-Based Reconstruction
• [ Amenta , Bern & Kamvysselis 98 ]
5Challenge the future
Segmentation for Detecting Features
Points belonging to a set of surfaces considered as Brep faces
Images courtesy of Dr. Shi Pu, Faculty of Feo Information Science and Earth Observations, University of Twente (ITC)
6Challenge the future
Segmentation for Detecting Features
Points belonging to a set of surfaces considered as Brep faces
3D reconstruction from AHN pointcloud: Carl Chen, Rusne Sileryte, Kaixuan Zhou; Ed. Pirouz Nourian, Sisi Zlatanova
7Challenge the future
Segmentation for Detecting Features
Points belonging to a set of surfaces considered as Brep faces
Images courtesy of Ajith Sampath
8Challenge the future
Segmentation for Detecting Features
Points belonging to a set of surfaces considered as Brep faces
Images courtesy of Ajith Sampath
Building Reconstruction
 The distance between two planar segments (P & Q) in a roof is defined as:
 A neighborhood Matrix is then generated. The matrix shows all mutuallyintersecting
planes.
 Any two intersecting planes are selected,and all planes thatintersectboth of them are
enumerated,and solved.
 For instance Planes {1,4,10} and {1,4,13} are solved to get the breakline (A-B)as shown.
9Challenge the future
Problem
• Estimated Normal Vectors For Each Point
• Estimated Curvature For Each Point
Preliminaries…
• Estimate Normal Vectors For Each Point Using the Covariance Matrix
• (Fit a Plane to a Bunch of Points)
• Cross Product of the Two Eigen Vectors Corresponding to the Two Largest Eigen Values
10Challenge the future
Covariance Matrix Computation
Check the attached code for the latest version!
Dim Neighbors = Pts.FindAll(Function(V) V.distanceto(point) < D)
Dim Centroid As New Point3d
For Each neighbor As point3d In Neighbors
Centroid = Centroid + neighbor
Next
For k As Integer=0 To Neighbors.count - 1
Dim CiCBar As New Matrix(3, Neighbors.count)
Dim Diff As point3d = Neighbors(k) - Centroid
CiCBar(0, k) = Diff.X
CiCBar(1, k) = Diff.y
CiCBar(2, k) = Diff.z
Dim CiCBarOld As New Matrix(3, Neighbors.count)
CiCBarOld = CiCBar.Duplicate()
CiCBar.Transpose()
CovM = CiCBarOld * CiCBar ‘Why is this a matrix of correct size?
Next
CovM.Scale(1 / Neighbors.count)
CovMatrices.Add(CovM)
11Challenge the future
Efficient Eigen Value Computation
• Find the roots of this characteristic function (why? & how?)
• Using a Trigonometric Method
• (The following code is my version of it, test it first on a cubic function)
A special case in which a 3x3 Matrix is dealt with
Function CubicRoots(a, b, c, d)
Dim t As Double
Dim p,q As Double
p = (3 * a * c - b ^ 2) / (3 * a ^ 2)
If p < 0 Then
q = (2 * b ^ 3 - 9 * a * b * c + 27 * a ^ 2 * d) / (27 * a ^ 3)
Dim roots As New list(Of Double)
For k As Integer = 0 To 2
t = 2 * Math.Sqrt(-p / 3) * Math.Cos((1 / 3) * Math.Acos(((3 * q) / (2 * p)) * Math.Sqrt(-3 / p)) - k * ((2 * Math.PI) / 3))
Dim x As Double = t - b / (3 * a)
roots.add(x)
Next
Return roots
Else
Return Nothing
End If
End Function
12Challenge the future
Ready for Segmentation!
We do an iterative segmentation called Region Growing
Mesh Segmentation in Action
low angle threshold
high angle threshold
13Challenge the future
Mesh Segmentation
• Connected Faces
• Start Making Regions out of Neighbours of Similar Aspects
(normal vectors/orientations)
• Recursively Grow Regions with an Angle Tolerance (how?)
How to detect faces of a Brep given a Mesh?
Define Faces=M.Faces
Define Regions As New list(Of List(Of Integer))
For i in range M.Faces.Count
If clusters.count = 0 Or There is no region containing(i) Then
Define region As New list(Of Integer)
region.Append(i)
For j in range adjacentfaces(of i)
If isSimilar(M.FaceNormals(i), M.FaceNormals(j), t) Then region.Append(j)
End For
region = RecursivelyGrowRegion(M, region, angle threshold)
Regions.Append(region)
End If
End For
14Challenge the future
Point Cloud Segmentation
Region Growing Segmentation Pseudocode
15Challenge the future
A few approaches to surface reconstruction
• 3d Hough Transform [Vosselman et al.]
• Implicit Function [ Hoppe et al. 92 ]
• Alpha Shapes [ Edelsbrunner and Mucke 94 ]
• Many more:
Berger, Matthew, et al. "State of the art in surface reconstruction from point clouds." EUROGRAPHICS star reports. Vol. 1. No. 1. 2014.
(e.g. Ball-Pivoting algorithm) The space generated by point pairs that can
be touched by an empty disc of radius alpha.
Using Voxels  Field of Signed-Distance Iso-Surface
16Challenge the future
2D Hough Transform
17Challenge the future
2d Hough
Transform
b = -ax + y
18Challenge the future
2d Hough
Transform
R=x cos θ+ y sin θ
19Challenge the future
3d Hough transform
c = -ax -by + z
R = x cos θ cos Φ + y sin θ cos Φ+ z sin Φ
20Challenge the future
20
Restricted
3d Hough Transform
Plane should contain
(0,0,0) → z = ax + by
b = -ax/y + z/y (y != 0)
21Challenge the future
21
22Challenge the future
22
23Challenge the future
23
24Challenge the future
24
Hough
Filtering
25Challenge the future
25
Final
result
26Challenge the future
Questions?
Thank you!
p.nourian@tudelft.nl

More Related Content

PPT
Two dimensional geometric transformations
PPTX
Computer Graphics - Hidden Line Removal Algorithm
PDF
Curves and fractals b spline and bezier
PDF
Hidden Surface Removal using Z-buffer
PPTX
Polygon mesh
PDF
Deep learning for 3 d point clouds presentation
PPTX
Computer Graphics: Visible surface detection methods
Two dimensional geometric transformations
Computer Graphics - Hidden Line Removal Algorithm
Curves and fractals b spline and bezier
Hidden Surface Removal using Z-buffer
Polygon mesh
Deep learning for 3 d point clouds presentation
Computer Graphics: Visible surface detection methods

What's hot (20)

PPTX
Hermit curves &amp; beizer curves
PPT
hidden surface elimination using z buffer algorithm
PPTX
Computer animation Computer Graphics
PPTX
Computer graphics presentation
PPT
PPTX
Image Acquisition and Representation
PDF
3D Reconstruction from Multiple uncalibrated 2D Images of an Object
PDF
Curves and surfaces
PPTX
Chapter 3 multiview drawings
PPTX
An Introduction to Laser Scanning - Part 1: How does it all work?
PPT
Polygon filling
PDF
Lecture 06 geometric transformations and image registration
PPTX
Computer graphics
PPT
Engineering Drawing: Chapter 01 introduction
PPT
Nurbs (1)
PPT
Shading
PDF
3D Transformation
PPTX
Area filling algo
PPT
leliso hobicho
PPT
Engineering Drawing & Ortographic projection
Hermit curves &amp; beizer curves
hidden surface elimination using z buffer algorithm
Computer animation Computer Graphics
Computer graphics presentation
Image Acquisition and Representation
3D Reconstruction from Multiple uncalibrated 2D Images of an Object
Curves and surfaces
Chapter 3 multiview drawings
An Introduction to Laser Scanning - Part 1: How does it all work?
Polygon filling
Lecture 06 geometric transformations and image registration
Computer graphics
Engineering Drawing: Chapter 01 introduction
Nurbs (1)
Shading
3D Transformation
Area filling algo
leliso hobicho
Engineering Drawing & Ortographic projection
Ad

Viewers also liked (20)

PDF
Point Cloud Processing: Estimating Normal Vectors and Curvature Indicators us...
PDF
On NURBS Geometry Representation in 3D modelling
PDF
Preliminaries of Analytic Geometry and Linear Algebra 3D modelling
PDF
Intro computational design_mega2016_1_with_recommendedplugins
PDF
Tudelft stramien 16_9_on_optimization
PDF
Polygon Mesh Representation
PDF
Indoor Point Cloud Processing
PDF
Point cloud modeling
PPTX
Transforming Rasters and Point Clouds
PDF
Surface reconstruction from point clouds using optimal transportation
PPTX
Surface reconstruction using point cloud
DOC
Dissertation_Full
PDF
Semantic 3D City Models with CityGML
PDF
3D Creations & Models update 25/07/2015
PDF
Lecture 02 yasutaka furukawa - 3 d reconstruction with priors
PDF
"High-resolution 3D Reconstruction on a Mobile Processor," a Presentation fro...
PDF
Structure and Motion - 3D Reconstruction of Cameras and Structure
PDF
3d Printing Overview
PDF
3D data acquisition and archaeological documentation, Alberto Sanchez, France...
PPTX
Tutorial Virtual Educa 2009 "Herramientas y estrategias de aprendizaje en ent...
Point Cloud Processing: Estimating Normal Vectors and Curvature Indicators us...
On NURBS Geometry Representation in 3D modelling
Preliminaries of Analytic Geometry and Linear Algebra 3D modelling
Intro computational design_mega2016_1_with_recommendedplugins
Tudelft stramien 16_9_on_optimization
Polygon Mesh Representation
Indoor Point Cloud Processing
Point cloud modeling
Transforming Rasters and Point Clouds
Surface reconstruction from point clouds using optimal transportation
Surface reconstruction using point cloud
Dissertation_Full
Semantic 3D City Models with CityGML
3D Creations & Models update 25/07/2015
Lecture 02 yasutaka furukawa - 3 d reconstruction with priors
"High-resolution 3D Reconstruction on a Mobile Processor," a Presentation fro...
Structure and Motion - 3D Reconstruction of Cameras and Structure
3d Printing Overview
3D data acquisition and archaeological documentation, Alberto Sanchez, France...
Tutorial Virtual Educa 2009 "Herramientas y estrategias de aprendizaje en ent...
Ad

Similar to Point Cloud Segmentation for 3D Reconstruction (20)

PPT
Build Your Own 3D Scanner: Surface Reconstruction
PDF
testpang
PPTX
Electrical Engineering Assignment Help
PPT
SolidModeling.ppt
PPTX
Unit 2-ME8691 & COMPUTER AIDED DESIGN AND MANUFACTURING
PPT
Surface models
PPT
CS 354 Transformation, Clipping, and Culling
PDF
Module- IV (CO-IV) Fundamental of solid design, parametric space of a solid, ...
PDF
Robot Motion Planning Introduction to Mobile Robotics.pdf
PDF
Module- III (CO-III) Techniques of surface modelling, plane surface, cylindri...
PDF
Algorithmic Techniques for Parametric Model Recovery
PPT
CS 354 More Graphics Pipeline
PDF
Theories and Engineering Technics of 2D-to-3D Back-Projection Problem
PPTX
Computer Graphics
PDF
International Journal of Engineering Research and Development
PDF
Topology-conform segmented volume meshing of volume images (Oct 2012)
PPTX
Image segmentation
PPT
Miniproject final group 14
PPT
Geometric modeling111431635 geometric-modeling-glad (1)
Build Your Own 3D Scanner: Surface Reconstruction
testpang
Electrical Engineering Assignment Help
SolidModeling.ppt
Unit 2-ME8691 & COMPUTER AIDED DESIGN AND MANUFACTURING
Surface models
CS 354 Transformation, Clipping, and Culling
Module- IV (CO-IV) Fundamental of solid design, parametric space of a solid, ...
Robot Motion Planning Introduction to Mobile Robotics.pdf
Module- III (CO-III) Techniques of surface modelling, plane surface, cylindri...
Algorithmic Techniques for Parametric Model Recovery
CS 354 More Graphics Pipeline
Theories and Engineering Technics of 2D-to-3D Back-Projection Problem
Computer Graphics
International Journal of Engineering Research and Development
Topology-conform segmented volume meshing of volume images (Oct 2012)
Image segmentation
Miniproject final group 14
Geometric modeling111431635 geometric-modeling-glad (1)

More from Pirouz Nourian (8)

PDF
Geo1004 lecture 1_topology&amp;topological_datamodels_final
PDF
Ar1 twf030 lecture3.1: Design Optimization
PDF
Ar1 twf030 lecture2.2
PDF
Ar1 twf030 lecture1.1
PDF
Ar1 twf030 lecture1.2
PDF
Ar1 twf030 lecture2.1: Geometry and Topology in Computational Design
PPTX
Mesh final pzn_geo1004_2015_f3_2017
PDF
Syntactic space syntax4generativedesign
Geo1004 lecture 1_topology&amp;topological_datamodels_final
Ar1 twf030 lecture3.1: Design Optimization
Ar1 twf030 lecture2.2
Ar1 twf030 lecture1.1
Ar1 twf030 lecture1.2
Ar1 twf030 lecture2.1: Geometry and Topology in Computational Design
Mesh final pzn_geo1004_2015_f3_2017
Syntactic space syntax4generativedesign

Recently uploaded (20)

PPT
CRASH COURSE IN ALTERNATIVE PLUMBING CLASS
PDF
composite construction of structures.pdf
PPTX
CARTOGRAPHY AND GEOINFORMATION VISUALIZATION chapter1 NPTE (2).pptx
PPTX
CYBER-CRIMES AND SECURITY A guide to understanding
PPTX
KTU 2019 -S7-MCN 401 MODULE 2-VINAY.pptx
PPTX
UNIT-1 - COAL BASED THERMAL POWER PLANTS
PPTX
Foundation to blockchain - A guide to Blockchain Tech
PPTX
M Tech Sem 1 Civil Engineering Environmental Sciences.pptx
DOCX
573137875-Attendance-Management-System-original
PDF
Automation-in-Manufacturing-Chapter-Introduction.pdf
PPT
Project quality management in manufacturing
PPTX
OOP with Java - Java Introduction (Basics)
PPTX
FINAL REVIEW FOR COPD DIANOSIS FOR PULMONARY DISEASE.pptx
PPTX
CH1 Production IntroductoryConcepts.pptx
PDF
BMEC211 - INTRODUCTION TO MECHATRONICS-1.pdf
DOCX
ASol_English-Language-Literature-Set-1-27-02-2023-converted.docx
PPTX
Construction Project Organization Group 2.pptx
PPTX
Internet of Things (IOT) - A guide to understanding
PDF
SM_6th-Sem__Cse_Internet-of-Things.pdf IOT
PDF
Evaluating the Democratization of the Turkish Armed Forces from a Normative P...
CRASH COURSE IN ALTERNATIVE PLUMBING CLASS
composite construction of structures.pdf
CARTOGRAPHY AND GEOINFORMATION VISUALIZATION chapter1 NPTE (2).pptx
CYBER-CRIMES AND SECURITY A guide to understanding
KTU 2019 -S7-MCN 401 MODULE 2-VINAY.pptx
UNIT-1 - COAL BASED THERMAL POWER PLANTS
Foundation to blockchain - A guide to Blockchain Tech
M Tech Sem 1 Civil Engineering Environmental Sciences.pptx
573137875-Attendance-Management-System-original
Automation-in-Manufacturing-Chapter-Introduction.pdf
Project quality management in manufacturing
OOP with Java - Java Introduction (Basics)
FINAL REVIEW FOR COPD DIANOSIS FOR PULMONARY DISEASE.pptx
CH1 Production IntroductoryConcepts.pptx
BMEC211 - INTRODUCTION TO MECHATRONICS-1.pdf
ASol_English-Language-Literature-Set-1-27-02-2023-converted.docx
Construction Project Organization Group 2.pptx
Internet of Things (IOT) - A guide to understanding
SM_6th-Sem__Cse_Internet-of-Things.pdf IOT
Evaluating the Democratization of the Turkish Armed Forces from a Normative P...

Point Cloud Segmentation for 3D Reconstruction

  • 1. 1Challenge the future Point Cloud Segmentation & Surface Reconstruction An overview of methods Ir. Pirouz Nourian PhD candidate & Instructor, chair of Design Informatics, since 2010 MSc in Architecture 2009 BSc in Control Engineering 2005 MSc Geomatics, GEO1004, Directed by Dr. Sisi Zlatanova
  • 2. 2Challenge the future Topics A very short introduction to segmentation and surface reconstruction • Big Picture, Problem Statement • Goal is to make a [simple] Brep, with few faces • General Approaches to this problem (surface reconstruction): • Volumetric, Implicit, using voxels and iso-surfaces • Alpha Shapes (Ball-Pivoting) • Delaunay or Voronoi Based? • Poisson? • General Approaches to this problem (surface reconstruction):
  • 3. 3Challenge the future Topics A very short introduction to segmentation and surface reconstruction • Big Picture, Problem Statement • Goal is to make a [simple] Brep, with few faces Images courtesy of Ajith Sampath ?
  • 4. 4Challenge the future A few approaches to surface reconstruction • Voxels  Field of Signed-Distance Iso-Surface • Ball-Pivoting (alpha shapes) triangulation • Planar Segments Neighbourhood Matrix Edge List Simple Mesh • Poisson Surface Reconstruction • Implicit Function • [ Hoppe et al. 92 ] • Volumetric Reconstruction • [ Curless and Levoy 96 ] • Alpha Shapes • [ Edelsbrunner and Mucke 94 ] • 3D Voronoi-Based Reconstruction • [ Amenta , Bern & Kamvysselis 98 ]
  • 5. 5Challenge the future Segmentation for Detecting Features Points belonging to a set of surfaces considered as Brep faces Images courtesy of Dr. Shi Pu, Faculty of Feo Information Science and Earth Observations, University of Twente (ITC)
  • 6. 6Challenge the future Segmentation for Detecting Features Points belonging to a set of surfaces considered as Brep faces 3D reconstruction from AHN pointcloud: Carl Chen, Rusne Sileryte, Kaixuan Zhou; Ed. Pirouz Nourian, Sisi Zlatanova
  • 7. 7Challenge the future Segmentation for Detecting Features Points belonging to a set of surfaces considered as Brep faces Images courtesy of Ajith Sampath
  • 8. 8Challenge the future Segmentation for Detecting Features Points belonging to a set of surfaces considered as Brep faces Images courtesy of Ajith Sampath Building Reconstruction  The distance between two planar segments (P & Q) in a roof is defined as:  A neighborhood Matrix is then generated. The matrix shows all mutuallyintersecting planes.  Any two intersecting planes are selected,and all planes thatintersectboth of them are enumerated,and solved.  For instance Planes {1,4,10} and {1,4,13} are solved to get the breakline (A-B)as shown.
  • 9. 9Challenge the future Problem • Estimated Normal Vectors For Each Point • Estimated Curvature For Each Point Preliminaries… • Estimate Normal Vectors For Each Point Using the Covariance Matrix • (Fit a Plane to a Bunch of Points) • Cross Product of the Two Eigen Vectors Corresponding to the Two Largest Eigen Values
  • 10. 10Challenge the future Covariance Matrix Computation Check the attached code for the latest version! Dim Neighbors = Pts.FindAll(Function(V) V.distanceto(point) < D) Dim Centroid As New Point3d For Each neighbor As point3d In Neighbors Centroid = Centroid + neighbor Next For k As Integer=0 To Neighbors.count - 1 Dim CiCBar As New Matrix(3, Neighbors.count) Dim Diff As point3d = Neighbors(k) - Centroid CiCBar(0, k) = Diff.X CiCBar(1, k) = Diff.y CiCBar(2, k) = Diff.z Dim CiCBarOld As New Matrix(3, Neighbors.count) CiCBarOld = CiCBar.Duplicate() CiCBar.Transpose() CovM = CiCBarOld * CiCBar ‘Why is this a matrix of correct size? Next CovM.Scale(1 / Neighbors.count) CovMatrices.Add(CovM)
  • 11. 11Challenge the future Efficient Eigen Value Computation • Find the roots of this characteristic function (why? & how?) • Using a Trigonometric Method • (The following code is my version of it, test it first on a cubic function) A special case in which a 3x3 Matrix is dealt with Function CubicRoots(a, b, c, d) Dim t As Double Dim p,q As Double p = (3 * a * c - b ^ 2) / (3 * a ^ 2) If p < 0 Then q = (2 * b ^ 3 - 9 * a * b * c + 27 * a ^ 2 * d) / (27 * a ^ 3) Dim roots As New list(Of Double) For k As Integer = 0 To 2 t = 2 * Math.Sqrt(-p / 3) * Math.Cos((1 / 3) * Math.Acos(((3 * q) / (2 * p)) * Math.Sqrt(-3 / p)) - k * ((2 * Math.PI) / 3)) Dim x As Double = t - b / (3 * a) roots.add(x) Next Return roots Else Return Nothing End If End Function
  • 12. 12Challenge the future Ready for Segmentation! We do an iterative segmentation called Region Growing Mesh Segmentation in Action low angle threshold high angle threshold
  • 13. 13Challenge the future Mesh Segmentation • Connected Faces • Start Making Regions out of Neighbours of Similar Aspects (normal vectors/orientations) • Recursively Grow Regions with an Angle Tolerance (how?) How to detect faces of a Brep given a Mesh? Define Faces=M.Faces Define Regions As New list(Of List(Of Integer)) For i in range M.Faces.Count If clusters.count = 0 Or There is no region containing(i) Then Define region As New list(Of Integer) region.Append(i) For j in range adjacentfaces(of i) If isSimilar(M.FaceNormals(i), M.FaceNormals(j), t) Then region.Append(j) End For region = RecursivelyGrowRegion(M, region, angle threshold) Regions.Append(region) End If End For
  • 14. 14Challenge the future Point Cloud Segmentation Region Growing Segmentation Pseudocode
  • 15. 15Challenge the future A few approaches to surface reconstruction • 3d Hough Transform [Vosselman et al.] • Implicit Function [ Hoppe et al. 92 ] • Alpha Shapes [ Edelsbrunner and Mucke 94 ] • Many more: Berger, Matthew, et al. "State of the art in surface reconstruction from point clouds." EUROGRAPHICS star reports. Vol. 1. No. 1. 2014. (e.g. Ball-Pivoting algorithm) The space generated by point pairs that can be touched by an empty disc of radius alpha. Using Voxels  Field of Signed-Distance Iso-Surface
  • 16. 16Challenge the future 2D Hough Transform
  • 17. 17Challenge the future 2d Hough Transform b = -ax + y
  • 18. 18Challenge the future 2d Hough Transform R=x cos θ+ y sin θ
  • 19. 19Challenge the future 3d Hough transform c = -ax -by + z R = x cos θ cos Φ + y sin θ cos Φ+ z sin Φ
  • 20. 20Challenge the future 20 Restricted 3d Hough Transform Plane should contain (0,0,0) → z = ax + by b = -ax/y + z/y (y != 0)
  • 26. 26Challenge the future Questions? Thank you! p.nourian@tudelft.nl