Making sense from 3D Point Clouds

Deep$Learning$
on
3D$Point$Clouds
SK#Reddy#
Chief#Product#Officer#AI
skreddy99
skreddy99

Confidential2
ImageNet(Large(Scale(Visual(Recognition(Challenge((ILSVRC)(winners

Confidential3 https://arxiv.org/pdf/1808.01462.pdf

Confidential5
O"CNN%&2017
https://arxiv.org/pdf/1712.01537.pdf
Left:<the<original<3D<shape.<Middle:<the<voxelized 3D<
shape.<Right:<the<octree<representation<with<normals
sampled<at<the<finest<leaf<octants
2D<quadtree<illustration<of<octree<structure

Confidential6
OctNet&'2017
Network=Architecture=Semantic=3D=Point=Cloud=
Labeling
Hybrid=GridKOctree=Data=Structure.=This=example=illustrates=a=
hybrid=gridKoctree=consisting=of=8=shallow=octrees=indicated=by=
different=colors.=Using=2=shallow=octrees=in=each=dimension=
with=a=maximum=depth=of=3=leads=to=a=total=resolution=of========
voxels
Bit=Representation.=Shallow=octrees=can=be=efficiently=encoded=
using=bitKstrings.=Here,=the=bitKstring=1=01010000=00000000=
01010000=00000000=01010000=0...=defines=the=octree=in=(a).=The=
corresponding=tree=is=shown=in=(b).=The=color=of=the=voxels=
corresponds=to=the=split=level

Confidential7
PointNet()2017

Confidential8
Deep$Kd'Network.$2017
A<kd>tree<built<on<the<point<cloud<of<eight<points<(left),<
and<the<associated<Kd>network<built<for<classification<
(right)
The<architecture<for<parts<segmentation<(individual<point<classification)<
for<the<point<cloud

Confidential9
PointNet++)*2017
(a) MultiAscaleCgroupingC(MSG)FC
(b) MultiresolutionCgroupingC(MRG)
MNISTCdigitCclassification

Confidential10
SPLATNet)*2018*
Bilateral*Convolution*Layer Left:<splatting<the<input<points<(orange)<onto<
the<lattice<corners<(black)C<Middle:<The<extent<of<a<filter<on<the<lattice<with<a<s<=<2<
neighborhood<(white<circles),<for<reference<we<show<a<Gaussian<filter,<with<its<
values<color<coded.<The<general<case<has<a<free<scalar/vector<parameter<per<
circle.<Right:<The<result<of<the<convolution<at<the<lattice<corners<(black)<is<projected<
back<to<the<output<points<(blue).<

Confidential11
MRTNet'(2018

Confidential12
SqueezeSeg:.CNNs.with.Recurrent.CRF.for.Real8Time.Road8
Object.Segmentation.from.3D.LiDAR.Point.CloudE.2017
Conditional.Random.Field.(CRF).as.an.RNN.layer

Confidential13
PointSeg-.Sep.2018
The.pipeline.of.PointSeg..Raw.LiDAR.point.cloud.is.projected.to.a.multichannel.range.
image..After.the.network.processing,.the.predicted.pointLwise.label.will.be.projected.back.
to.the.original.space.
Source.code:.https://github.com/ywangeq/PointSeg

Confidential14
LMNet:.Real0time.Multiclass.Object.Detection.on.CPU.using.3D.LiDAR@.2018
IMPLEMENTED.DILATED.LAYERS

Confidential15
DeLS%3D:.Deep.Localization.and.
Segmentation.with.a.3D.Semantic.Map;.2018

Confidential16
Spherical CNN for/3D/Point/Clouds5/2018
Classification/performance/on/ModelNetszz

Confidential17
Example(result(of(our(FCPN(on(semantic(voxel(labeling(and(captioning(on(an(Tango(3D(
reconstruction(/(point(cloud:((a)(3D(reconstruction((not(used),((b)(Input(point(cloud((c)(Output(
semantic(voxel(prediction.(
Visualization(of(the(semantic(segmentation(on((a)(a(depth(image,((b)(a(2.4m×2.4m×2.4m(partial(
reconstruction((c)(an(entire(reconstruction(of(a(hotel(suite.(Please(note(that(each(of(these(outputs(
are(predicted(in(a(single(shot(from(the(same(network(trained(with(2.4m(× 2.4m(× 2.4m(volumes(
Fully%Convolutional-Point-Network-Architecture Aug=2018=
(Source=code:=Not=yetD=https://github.com/drethage/fullyGconvolutionalGpointGnetwork)

Confidential18
Data$sets
• ScanNet:2Indoor2Scenes
• ShapeNet:23D2shapes
• Princeton2ModelNet
• KITTI2(Code2available2for2some)
• Pascal3D+2(….in2the2wild)
• Pascal2VOC
• SUNCG2(room2models)
• SceneNet (RGBMD2room2models)
• CV2Lab2datasets2with2Code2
(http://cvgl.stanford.edu/resources.html)
• Collective2Activity2Dataset2(old)
• Monocular2Multiview2Object2Tracking2with2
3D2Aspect2Parts2(???)
• Stanford22DM3DMSemantics2Dataset2(2DM
3DMS)2(**)
• Stanford2Drone2Dataset
• ObjectNet3D
List$of$datasets:
1.2http://clickdamage.com/sourcecode/cv_datasets.php
Other$Data$sets
• TOSCA2(nonMrigid2dataset)
• FAUST2(highMres2human2scans)
• SUN3D
• NYC3DCars
• LabelMe3D2(old2dataset)
• CVLab (MultiMview2car2dataset)
• MIT2Street2Scenes
• Daimler2Pedestrian2Datasets
• Caltech2Pedestrian2Detection2Benchmark
• Robust2MultiMPerson2Tracking2from2Mobile2Platforms
• FlyingThings3D2(Synthetic)
• BU4DFE23D2

Confidential19
PVNet:"A"Joint"Network"of"Point"Cloud"and"Multi6View"for"
3D"Shape"Recognition"(Aug"2018)

FlowNet3D:;Scene;Flow;in;3D;PCs(June;2018);

Confidential21
YOLO3D Aug02018
Sample0of0the0output0shown0in03D0and0projected0on0the0top0view0map0

Confidential22
PointFlowNet 3D.Scene.Flow.Estimation.from.PCs.(Sep.2018)

Confidential23
Point&Cloud&Library Modules

Confidential24
https://github.com/kzampog/cilantro;
Cilantro
library;for;point;cloud;data;processing;
Bundled;with:;
• nanoflann (for;fast;kdAtree;queries);
• Spectra;(ARPACKAinspired;library;for;large;
scale;eigen;decompositions;
• Qhull (for;convex;hull;and;halfA space;
intersection;computations);
• tinyply (for;PLY;format;geometry;I/O);
• External;dependencies:;
• Eigen;(linear;algebra;library);
• Pangolin;(lightweight;OpenGL;viewport;
manager;and;video;I/O;abstraction;library)

Confidential25
Thank&you
SK&Reddy
Chief&Product&Officer&AI
skreddy99
skreddy99

Making sense from 3D Point Clouds

More Related Content

Similar to Making sense from 3D Point Clouds (6)

More from SK Reddy (20)

Recently uploaded (20)

Making sense from 3D Point Clouds