Out-of-Core Construction of Sparse Voxel Octrees
3 likes5,407 views
The document discusses an out-of-core method for constructing sparse voxel octrees (SVOs) from large polygon meshes, emphasizing the need for efficient data streaming due to memory constraints. It outlines a multi-step pipeline that includes voxelization, partitioning, and SVO construction, while utilizing Morton ordering for efficient memory handling. The results demonstrate the algorithm's ability to process large models quickly and efficiently, comparable to in-core methods, with future work suggested on integrating GPU methods.
Out-of-Core Construction of Sparse Voxel Octrees
- 1. High Performance Graphics 2013 - 8/21/13 Out-Of-Core Construction of Sparse Voxel Octrees Jeroen Baert, Ares Lagae & Philip Dutré Computer Graphics Group, KU Leuven, Belgium
- 2. Out-Of-Core Construction of Sparse Voxel Octrees 2 / 34 Voxel-related research ● Voxel Ray Casting – Gigavoxels (Crassin, 2009-...) – Efficient SVO's (Laine, Karras, 2010) ● Voxel Cone Tracing – Indirect Illumination (Crassin, 2011) ● Voxel-based Visibility – Voxelized Shadow Volumes (Wyman, 2013, later today!) ● ... Crassin2009 Laine/Karras2010 Crassin2011
- 3. Out-Of-Core Construction of Sparse Voxel Octrees 3 / 34 Why voxels? ● Regular structure ● Hierarchical representation in Sparse Voxel Octrees (SVO's) – Level of Detail / Filtering ● Generic representation for geometry and appearance – In a single data structure CreativeCommons–Wikipedia
- 4. Out-Of-Core Construction of Sparse Voxel Octrees 4 / 34 Polygon mesh to SVO ● We want large, highly detailed SVO scenes ● Where do we find content? ● Let's voxelize massive polygon meshes – Majority of current content pipelines is polygon-based
- 5. Out-Of-Core Construction of Sparse Voxel Octrees 5 / 34 What do we want? ● Algorithm requirements: – Need an out-of-core method ● Because polygon mesh & intermediary structures could be >> system memory – Data should be streamed in/out ● from disk / network / other process – Ideally: out-of-core as fast as in-core Algorithm Polygon Mesh Sparse Voxel Octree
- 6. Out-Of-Core Construction of Sparse Voxel Octrees 6 / 34 Pipeline construction (1) ● Voxelization step – Polygon mesh → Voxel grid ● Followed by SVO construction step – Voxel grid → Sparse Voxel Octree Voxelization Polygon Mesh Sparse Voxel Octree SVO Construction
- 7. Out-Of-Core Construction of Sparse Voxel Octrees 7 / 34 Pipeline construction (2) ● Key insight: – If voxel grid is Morton-ordered – SVO construction can be done out-of-core ● Logarithmic in memory usage ~ octree size ● In a streaming manner – So voxelization step should deliver ordered voxels Voxelization Polygon Mesh Sparse Voxel Octree SVO Construction Morton Ordered Grid
- 8. Out-Of-Core Construction of Sparse Voxel Octrees 8 / 34 Pipeline construction (3) ● High-resolution 3D voxel grid may be >> system memory – So partitioning step (into subgrids) is needed – Seperate triangle streams for each subgrid Voxelization Polygon Mesh Sparse Voxel Octree SVO Construction Morton Ordered Grid Partitioning Mesh Parts
- 9. Out-Of-Core Construction of Sparse Voxel Octrees 9 / 34 Final Pipeline ● Now, every step in detail ... Voxelization Polygon Mesh Sparse Voxel Octree SVO Construction Morton Ordered Grid Partitioning Mesh Parts
- 10. Out-Of-Core Construction of Sparse Voxel Octrees 10 / 34 Morton order / Z-order ● Linearization of n-dimensional grid – Post-order depth-first traversal of 2 n -tree ● Space-filling curve, Z-shaped
- 11. Out-Of-Core Construction of Sparse Voxel Octrees 11 / 34 Morton order / Z-order ● Hierarchical in nature ● Cell at position (x,y) → Morton code – Efficiently computed – (x,y,z) = (5,9,1) → (0101,1001,0001) → 010001000111 → 1095th cell along Z-curve
- 12. Out-Of-Core Construction of Sparse Voxel Octrees 12 / 34 Partitioning subprocess ● Partitioning (1 linear pass) – Into power-of-2 subgrids until it fits in memory – Subgrids temporarily stored on disk – Subgrids correspond to contiguous range in Morton order ● If we voxelize subgrids in Morton order, output will be Morton-ordered Voxelization Polygon Mesh Sparse Voxel Octree SVO Construction Morton Ordered Grid Partitioning Mesh Parts
- 13. Out-Of-Core Construction of Sparse Voxel Octrees 13 / 34 Voxelization subprocess (1) ● Voxelize each subgrid in Morton order – Input: Subgrid triangle stream ● Each triangle voxelized independently – Output: Morton codes of non-empty cells ● Typically, majority of grid is empty Voxelization Polygon Mesh Sparse Voxel Octree SVO Construction Morton Ordered Grid Partitioning Mesh Parts
- 14. Out-Of-Core Construction of Sparse Voxel Octrees 14 / 34 Voxelization subprocess (2) ● We use a simple voxelization method – But any method that works one triangle at a time will do 111011010100 101111010111 ... HuangetAl. Morton codes of non-empty cells
- 15. Out-Of-Core Construction of Sparse Voxel Octrees 15 / 34 Out-of-core SVO Construction ● Input: Morton-ordered voxel grid ● Output: SVO nodes + referenced dataevel Morton code Voxelization Polygon Mesh Sparse Voxel Octree SVO Construction Morton Ordered Grid Partitioning Mesh Parts
- 16. Out-Of-Core Construction of Sparse Voxel Octrees 16 / 34 SVO Construction algorithm in 2D 0 0 3... 1 4 7... 2 8 11... 3 12 15... ● Required: queues of 2 d nodes / octree level – Ex: 20483 grid → 11 * 8 octree nodes-l Octree representationSVO Builder queues Input position In Memory / On-disk Level 0 Level 1 Level 2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
- 17. Out-Of-Core Construction of Sparse Voxel Octrees 17 / 34 SVO Construction algorithm in 2D ● Read Morton codes 0 → 3 (+ voxel data) – Store them in level 2 queue – Level 2 queue = full 0 3 Octree representationSVO Builder queues 0 1 2 3 Level 0 Level 1 Level 2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
- 18. Out-Of-Core Construction of Sparse Voxel Octrees 18 / 34 SVO Construction algorithm in 2D ● Create internal parent node – With level 1 Morton code 0 – Store parent-child relations – Write non-empty level 2 nodes to disk+clear level 2 Octree representationSVO Builder queues 0 1 2 3 0 0 0 3 Level 0 Level 1 Level 2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
- 19. Out-Of-Core Construction of Sparse Voxel Octrees 19 / 34 SVO Construction algorithm in 2D ● Read Morton codes 4 → 7 (+ voxel data) – Store them in level 2 queue Octree representationSVO Builder queues 4 5 6 7 0 0 0 3 4 7... Level 0 Level 1 Level 2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
- 20. Out-Of-Core Construction of Sparse Voxel Octrees 20 / 34 SVO Construction algorithm in 2D ● Create internal parent node – With level 1 Morton code 1 – Store parent-child relations (there are none) – Write non-empty level 2 nodes to disk+clear level 2 Octree representationSVO Builder queues 4 5 6 7 0 1 0 0 3... 1 Level 0 Level 1 Level 2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 4 7...
- 21. Out-Of-Core Construction of Sparse Voxel Octrees 21 / 34 SVO Construction algorithm in 2D ● Same for Morton codes 8 → 11 Octree representationSVO Builder queues 0 1 0 0 3 1 2 9 2 Level 0 Level 1 Level 2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
- 22. Out-Of-Core Construction of Sparse Voxel Octrees 22 / 34 SVO Construction algorithm in 2D ● Same for Morton codes 12 → 15 Octree representationSVO Builder queues 0 1 2 3 3 12 15... Level 0 Level 1 Level 2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 0 0 3 1 2 9
- 23. Out-Of-Core Construction of Sparse Voxel Octrees 23 / 34 SVO Construction algorithm in 2D ● Now level 1 is full – Create parent node (root node) – Store parent-child relations – Write non-empty level 1 nodes to disk+clear level 1 Octree representationSVO Builder queues 0 1 2 3 R R ... ... Level 0 Level 1 Level 2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 3 12 15... 0 0 3 2 9 1
- 24. Out-Of-Core Construction of Sparse Voxel Octrees 24 / 34 SVO Construction: optimization ● Lots of processing time for empty nodes – Sparseness = typical for high-res voxelized meshes ● Insight for optimization – Pushing back 2 d empty nodes in a queue at level n = Pushing back 1 empty node at level n-1 n-1 n ... SVO Builder queues
- 25. Out-Of-Core Construction of Sparse Voxel Octrees 25 / 34 SVO Construction: optimization ● Implementation details in paper ● Optimization exploits sparseness of voxelized meshes ● Speedup: two orders of magnitude – Building SVO from grid: ● David: 471 vs 0.55 seconds ● San Miguel: 453 vs 1.69 seconds
- 26. Out-Of-Core Construction of Sparse Voxel Octrees 26 / 34 Results: Tests ● Resolution: 2048 3 ● Memory limits – 8 Gb (in-core) – 1 Gb (out-of-core) – 128 Mb (out-of-core) ● Models – David (8.25 M polys) – San Marco (7.88 M polys) – XYZRGB Dragon (7.2 M polys)
- 27. Out-Of-Core Construction of Sparse Voxel Octrees 27 / 34 Results: Out-Of-Core performance ● Out-Of-Core method = ~ as fast as In-Core – Even when available memory is 1/64
- 28. Out-Of-Core Construction of Sparse Voxel Octrees 28 / 34 Results: Time breakdown ● Partitioning speedup from skipping empty space
- 29. Out-Of-Core Construction of Sparse Voxel Octrees 29 / 34 Results: Extremely large models ● 4096 3 – In-core: 64 Gb ● Atlas model – 17.42 Gb, 507 M tris – < 11 min at 1 Gb ● St. Matthew model – 13.1 Gb, 372 M tris – < 9 min at 1 Gb
- 30. Out-Of-Core Construction of Sparse Voxel Octrees 30 / 34 Results: SVO Construction ● SVO output stream – Good locality of reference ● Nonempty siblings on same level always stored next to each other – Nodes separated from data itself (separation hot/cold data) ● Using data pointers + offsets as reference
- 31. Out-Of-Core Construction of Sparse Voxel Octrees 31 / 34 Appearance ● Pipeline: binary voxelization ● Extend with appearance data? – Interpolate vertex attributes (color, normals, tex) – Propagate appearance data upwards ● Global data access ↔ Out-Of-Core algorithms – Multi-pass approach DecreasingSVOlevel
- 32. Out-Of-Core Construction of Sparse Voxel Octrees 32 / 34 Conclusion ● Voxelization and SVO construction algorithm – Out-of-core as fast as in-core – Support for extremely large meshes ● Future work – Combine with GPU method to speed up voxelization – Handle global appearance data
- 33. Out-Of-Core Construction of Sparse Voxel Octrees 33 / 34 Thanks! ● Source code / binaries will be available at project page ● Contact – jeroen.baert @ cs.kuleuven.be – @jbaert ● Acknowledgements: – Jeroen Baert funded by Agency for Innovation by Science and Technology in Flanders (IWT), Ares Lagae is a Postdoctoral Fellow of the Research Foundation – Flanders (FWO), David / Atlas / St. Matthew models : Digital Michelangelo project, San Miguel model : Guillermo M. Leal Llaguno (Evolucien VIsual)
- 34. Out-Of-Core Construction of Sparse Voxel Octrees 34 / 34