Modern OpenGL scientific visualization
3 likes2,306 views
This document discusses using shader-based rendering for modern and interactive scientific visualization in Python. It outlines limitations of existing Python visualization libraries like Matplotlib, and how shader programming in OpenGL can address these through higher quality rendering of text, dashed lines, image filters, grids and other primitives. The approach improves rendering quality and speeds through GPU processing while maintaining an easy to use Python interface. Several open source projects aim to integrate these techniques into interactive visualization libraries.
![A Bit of Context
The Python Scientific Stack
→ Python, modern computing script language
→ IPython, an advanced Python shell
→ Numpy, powerful numerical arrays objects.
→ Scipy, high-level data processing routines.
→ Matplotlib, 2-D visualization plots
Python
Matplotlib
NumPy
SciPy
IPython
IP[y]:
SymPy
Σ ∫dx
[1
00
1]
SciKits
Versatile, beautiful but... slow !
Matplotlib is a python plotting library, primarily for 2-D plotting, but with some 3-D support,
which produces publication-quality figures in a variety of hardcopy formats and interactive
environments across platforms.
→ Antigrain geometry, High Fidelity 2D Graphics (www.antigrain.com)
→ Ten Simple Rules for Better Figures, Nicolas P. Rougier, Michael Droettboom, Philip E.
Bourne PLoS Computational Biology, Vol. 10, No. 9.](https://image.slidesharecdn.com/talkcopy-161117204354/85/Modern-OpenGL-scientific-visualization-2-320.jpg)
Modern OpenGL scientific visualization
- 1. MODERN & INTERACTIVE SCIENTIFIC VISUALIZATION USING SHADER BASED RENDERING Nicolas P. Rougier Luke Campagola - Almar Klein - Cyrille Rossant - Eric Larson X.Org Developer Conference 2014 Bordeaux, October 8–10
- 2. A Bit of Context The Python Scientific Stack → Python, modern computing script language → IPython, an advanced Python shell → Numpy, powerful numerical arrays objects. → Scipy, high-level data processing routines. → Matplotlib, 2-D visualization plots Python Matplotlib NumPy SciPy IPython IP[y]: SymPy Σ ∫dx [1 00 1] SciKits Versatile, beautiful but... slow ! Matplotlib is a python plotting library, primarily for 2-D plotting, but with some 3-D support, which produces publication-quality figures in a variety of hardcopy formats and interactive environments across platforms. → Antigrain geometry, High Fidelity 2D Graphics (www.antigrain.com) → Ten Simple Rules for Better Figures, Nicolas P. Rougier, Michael Droettboom, Philip E. Bourne PLoS Computational Biology, Vol. 10, No. 9.
- 3. What about OpenGL ? Powerful, fast but... ugly ! • No decent anti-aliasing • Only two image filters • No native text handling • No markers, no arrows • No paths, no curves Proprietary solution → Mark Kilgard and Jeff Bolz, GPU-accelerated Path Rendering, ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia 2012), vol. 31, Num. 6, (2012). → OpenVG API, Standard for Vector Graphics Acceleration, Khronos group. But this can also be fixed freely !
- 4. Python/OpenGL frameworks Rendering framework • Pyglet www.pyglet.org • PyOpenGL pyopengl.sourceforge.net • Nodebox for OpenGL www.cityinabottle.org/nodebox • PyProcessing code.google.com/p/pyprocessing Visualization framework • mayavi 2 (Enthought) github.com/enthought/mayavi • VTK (Kitware) www.vtk.org • galry (Cyrille Rossant) rossant.github.io/galry/ • visvis (Almar Klein) code.google.com/p/visvis/ • glumpy (Nicolas Rougier) code.google.com/p/glumpy/ • pyqtgraph (Luke Campagnola) www.pyqtgraph.org
- 5. OpenGL history ES 1.0 ES 1.1 ES 3.0ES 2.0 1992 1.0 1997 1.1 1998 1.2 2001 1.3 2002 1.4 2003 1.5 2004 2.0 2006 2.1 2008 3.0 2009 3.2 2010 4.1 2011 4.2 2012 4.3 1993 1994 1995 1996 1999 2000 2005 2007 3.1 4.0 3.3 Fixed pipeline (no shaders) Programmable pipeline (vertex/fragment/geometry shaders) Legacy OpenGL Core Profile (deprecation model) Doom (1993) Rage (2011)
- 6. OpenGL 4.2 pipeline overview (could have been worse...) Around 2000 constants and 1000 functions.
- 7. OpenGL ES 2.0 pipeline overview (openglinsights.com) Around 350 constants and 150 functions.
- 8. Pipeline overview Data centered CPU(python) GPU(shaders) Raw Data Baked Data Transformed Data Rendered Data Critical parts are the baking process and the transfer to GPU memory.
- 9. Baking process Ideal case: no baking Interpolation, colorization, leveling, gridding, scaling, lighting, aliasing, rendering entirely done on GPU. Hard case: baking depends on transformation Transparency implies lot of CPU processing (sorting) or multi-pass rendering.
- 10. Where do we start ? Scalable Vector Graphics (SVG) 2 Text Paths Basic shapes Painting: Filling, Stroking and Marker Symbols Clipping, Masking and Compositing Filter Effects ...
- 11. Text rendering Different techniques Bitmap, stroke, texture, sdf, vector... → Nicolas P. Rougier, Higher Quality 2D Text Rendering, Journal of Computer Graphics Techniques (JCGT), vol. 2, no. 1, 50-64, 2013.
- 12. Higher quality text rendering Vertical vs Horizontal hinting No hinting Native hinting Auto hinting Vertical hinting → Maxim Shemarev, Texts Rasterization Exposures, An attempt to improve text rasterization algorithms, 2007 Implementation (github.com/rougier/freetype-gl) • Subpixel positioning & kerning • Per pixel gamma correction • Signed Distance Fields
- 13. Dashed stroked polyline GL line width (fixed pipeline) • Limited in thickness • No control over joins and caps • Deprecated & ugly GL Stipple (fixed pipeline) • Limited in pattern • No control over dash caps • Deprecated & ugly
- 14. Higher quality dashed stroked polyline Shader based approach A new method for rendering arbitrary dash patterns along any continuous polyline (smooth or broken). The proposed method does not tessellate individual dash patterns and allows for fast and accurate rendering of any user-defined dash pattern and caps → Nicolas P. Rougier, Shader-Based Antialiased, Dashed, Stroked Polylines, Journal of Computer Graphics Techniques (JCGT), vol. 2, no. 2, 105–121, 2013.
- 15. Image interpolation & filters OpenGL offers only nearest and linear filters while much more are needed for scientific visualization (Hanning, Hamming, Hermite, Kaiser, Quadric, Bicubic, CatRom, Mitchell, Spline16, Spline36, Gaussian, Bessel, Sinc, Lanczos, Blackman, etc.) → Kevin Bjorke, High-Quality Filtering in GPU gems 2 : programming techniques for high-performance graphics and general-purpose computation / edited by Matt Pharr ; Randima Fernando (2007).
- 16. Grids, markers and arrows Point based approach A new method for drawing grids, markers, and arrows using implicit functions such that it is possible draw pixel-perfect antialiased objects with very good performances. → Nicolas P. Rougier, Shader Based Antialiased 2D Grids, Markers, and Arrows, Journal of Computer Graphics Techniques (JCGT), to appear, 2014.
- 17. Still lot of problems ahead... ...but work is in progress Shader composition How to define a shader format that allow easy composition/templating ? Level of details How to set automatic level of details ? Very big data How to render data that doesn’t even fit into GPU memory ? Complex data transformation How to handle user-supplied exotic transformation ? From DesktopGL to WebGL How to render in browser from a python session ? ...
- 18. Conclusion We do not have to (always) trade quality for speed 10,000 pts - 403 FPS 100,000 pts - 140 FPS 1,000,000 pts - 40 FPS 10,000,000 pts - 1.5 FPS AntiGrain Geometry (matplotlib agg backend) OpenGL AntiGrain (using dedicated shaders) AntiGrain Geometry (matplotlib agg backend) OpenGL AntiGrain (using dedicated shaders)
- 19. Informations & links The code is spread in several projects but should be soon integrated in the master vispy project. Projects page • vispy.org • vispy.org/gallery.html • glumpy.github.io • glumpy.github.io/gallery.html Code repositories • github.com/vispy/vispy • github.com/glumpy/glumpy • github.com/rougier/gl-agg • github.com/rougier/freetype-gl Demo pages • Markers: https://www.shadertoy.com/view/XsXXDX • Arrows: https://www.shadertoy.com/view/ldlSWj • Transverse Mercator grid: https://www.shadertoy.com/view/lsSXzm • Cartesian grid: https://www.shadertoy.com/view/MdSXRm • Polar grid: https://www.shadertoy.com/view/MsBSRm