Getting more out of Matplotlib with GR

Getting more out of Matplotlib with GR
August 26th – 30th, 2015
Cambridge, UK | EuroSciPy 2015 | Josef Heinen | @josef_heinen
MemberoftheHelmholtzAssociation
http://goo.gl/sKh7uD

EuroSciPy 2015 j.heinen@fz-juelich.de | @josef_heinen
Scientiﬁc visualization tools for Python
Core package:
matplotlib — de-facto standard (“workhorse”) 
➟ Browser solutions: Bokeh, plot.ly
Other packages:
Mayavi (mlab) — powerful, but overhead fromVTK
ggplot, chaco — statistical, 2D graphics
VTK — versatile, but difﬁcult to learn
Vispy, Glumpy, OpenGL — fast, but low-level APIs
2
2D 3D
Bokeh
Plotly
ggplot
chaco

Problems so far — Crux of the matter
3
Quality
Interop
Speed
✓ separated 2D and 3D world —
missing interop
✓ due to conceptual limitations, most
packages can’t visualize continuous
data streams
✓ speed up often at the cost of device
speciﬁc code

EuroSciPy 2015 j.heinen@fz-juelich.de | @josef_heinen 4
Our approach …
Scripting Layer
pyplot (backend / artist wrappers)
Artist Layer
ﬁgure, plot, axes, primitives
Backend Layer
user interfaces, hardcopy devices
✓ achieve more graphics performance
by using GR as a Matplotlib backend
✓ extend Matplotlib’s capabilities by
combining the power of Matplotlib
and GR / GR3
✓ avoid extra APIs / packages

… highlights
✓ use GR as a backend for Matplotlib 
➟ mix GR, MPL and GR3 (OpenGL) code
✓ IP[y]: / Jupyter notebook integration
✓ display continuous data streams
✓ create video animations on the ﬂy
✓ native GUI integration (Qt4, wx) 
➟ interactivity
✓ simultaneous output to multiple output
devices

GR in action …
6

Matplotlib using the GR backend
7
Circle
Rectangle
Wedge
Polygon
Ellipse
Arrow
PathPatch
FancyBoxPatch
Line2D
! % # % & ( * $#
+
! %
#
%
&
(
*
$#
$%
$&
$(
-
#"#
%"'
'"#
)"'
$#"#
$%"'
$'"#
$)"'
%#"#
%%"'
+,
'
%!
$!
#!
"!
!
"!
#!
$!
%!
(
%!
$!
#!
"!
!
"!
#!
$!
%!
)
"!!
&!
!
&!
"!!
# '$% # &$* # &$% # %$* %$% %$* &$% &$* '$%
# '$%
# &$*
# &$%
# %$*
%$%
%$*
&$%
&$*
'$%
051..-7- 7/67 !'%% 431276"
# %$(+
# %$')
# %$&'
%$%%
%$&'
%$')
%$(+
%$),
$"# $"( %"# %"( &"# &"( '"# '"( ("#
$"#
$"(
%"#
%"(
&"#
&"(
'"#
'"(
("#
4+30351,6. :/7. 0,9,06
! #"*
! #")
! #"&
#"#
#"&
#")
#"*
$"%
$"( %"# %"( &"# &"( '"# '"(
$"(
%"#
%"(
&"#
&"(
'"#
'"(
+327385- :/7. 0,9,06
! #"*
! #")
! #"&
#"#
#"&
#")
#"*
$"%
0°
45°
90°
135°
180°
225°
270°
315°
2
4
6
8
10
(
&
$
"
$
&
( (
&
$
"
$
&
(
#!"#
"!)+
"!'(
"!%&
"!##
"!##
"!%&
"!'(
"!)+
#!"#
"!*
"!(
"!&
"!$
"!"
"!$
"!&
"!(
"!*
# &$% # %$' %$% %$' &$%
# &$%
# %$'
%$%
%$'
&$%
)/07,5/2- * (,0*82*9 1,6.
!*440/+*7/32 73 ./-.# 5,63087/32 75/+327385/2-"
#!"
"!&
"!"
"!&
#!" #!"
"!&
"!"
"!&
#!"
"!'
"!%
"!$
"!"
"!$
"!%
"!'

click images to view notebooks …

Performance analysis
0
100
200
300
400
500
600
700
MPL MPL+GR GR
fps
ncalls cumtime filename:lineno(function)
398 6.852 {method'draw'of'_macosx.FigureCanvas'objects}
29378/397 6.771 artist.py:57(draw_wrapper)
397 6.769 figure.py:1004(draw)
397 6.574 _base.py:1989(draw)
794 5.894 axis.py:1106(draw)
5161 4.601 axis.py:232(draw)
199 3.616 pyplot.py:175(pause)
10719 3.609 lines.py:661(draw)
199 3.480 pyplot.py:551(draw)
7940 1.044 text.py:581(draw)
199 4.412 pyplot.py:551(draw)
199 4.410 backend_gr.py:227(draw)
14726/199 4.237 artist.py:57(draw_wrapper)
199 4.236 figure.py:1004(draw)
199 4.138 _base.py:1989(draw)
398 3.770 axis.py:1106(draw)
2587 3.073 axis.py:232(draw)
5373 2.642 lines.py:661(draw)
5174 1.202 backend_bases.py:237(draw_markers)
199 3.263 __init__.py:1910(plot)
199 3.184 __init__.py:250(updatews)
MPL
MPL + GR
GR
most time is spent in the
artist layer (pure Python)
???
No room for further
optimizations on the
backend side

Inline graphics
Matplotlib
GR
~ 10 times faster
%matplotlib inline
import matplotlib.pyplot as mpl
fig, ax = mpl.subplots()
for i in arange(1, 200):
clear_output(wait=True)
ax.cla()
ax.plot(x, sin(x + i / 10.0))
display(fig)
mpl.close()
from gr import inline
from gr.pygr import plot
inline()
for i in arange(1, 200):
plot(x, sin(x + i / 10.0))

GR + GR3 + MPL interop
from os import environ
environ['MPLBACKEND'] = 'module://gr.matplotlib.backend_gr'
import matplotlib.pyplot as mpl
import mogli
molecules = mogli.read("data/700K.xyz")
import gr
gr.inline("mov")
gr.setregenflags(gr.MPL_POSTPONE_UPDATE)
import numpy as np
angles = np.load("data/700K.npy")
lens = []
for t in range(100):
mpl.cla()
fig = mpl.subplot(133)
fig.xaxis.set_ticks([-100, 0, 100])
fig.yaxis.set_ticks([])
mpl.ylim([0, 1000])
mpl.hist(angles[t], 20, normed=0, facecolor='g', alpha=0.5)
mpl.show()
gr.setviewport(0.05, 0.7, 0.05, 0.7)
gr.setwindow(0, 1, 0, 1)
mogli.draw(molecules[t])
gr.settextalign(gr.TEXT_HALIGN_CENTER, gr.TEXT_VALIGN_HALF)
gr.text(0.35, 0.7, "700K (%.1f ps) # of bonds: %d" %
(t / 10.0, np.size(angles[t])))
lens.append(np.size(angles[t]))
if t > 0:
gr.setwindow(0, 10, 3500, 5000)
gr.setviewport(0.1, 0.6, 0.05, 0.1)
gr.axes(1, 0, 0, 3500, 2, 0, 0.005)
gr.polyline(np.arange(t+1) / 10.0, lens)
gr.updatews()
import gr3
gr3.export("data/700K.html", 600, 600)
Matplotlib
GR3
GR
Important:
tells MPL backend not to update
GR3

Demos
✓ Animated graphics performance comparison: Matplotlib vs. GR 
(01-anim.ipynb)
✓ Inline graphics performance comparison: Matplotlib vs. GR 
(02-inline.ipynb)
✓ GR / mogli / Matplotlib interoperability example (03-interop.ipynb)
✓ Simple spectral analysis (04-specgram.ipynb)
✓ Visualize detector data from a scattering instrument (05-kws.ipynb)

GR can be transpiled to JS ➟ gr.js 
(Emscripten: LLVM-to-JavaScript compiler)
➟ call GR functions from JS
 
Use cases:
✓ embed JS code in IP[y]: or IJulia (Jypyter)
✓ interpret GR display list in the browser 
(06-anim-js.ipynb)
Current activities
JavaScript
<canvas id="canvas" width="500" height="500"></canvas>
<script type="text/javascript" src="gr.js"></script>
<script type="text/javascript">
GR.ready(function() {
var gr = new GR();
var t = 0;
var x = new Array(629);
var y = new Array(629);
var draw = function() {
gr_clearws();
var i;
for (i = 0; i < 629; i++) {
x[i] = i / 630.0 * 2 * Math.PI;
y[i] = Math.sin(x[i] + t / 10.0);
}
gr_setviewport(0.1, 0.95, 0.1, 0.95);
gr_setwindow(0, 8, -1, 1);
gr_setcharheight(0.020);
gr_grid(0.5, 0.1, 0, -1, 4, 5);
gr_axes(0.5, 0.1, 0, -1, 4, 5, 0.01);
gr_polyline(629, x, y);
gr_updatews();
t = t + 1;
if (t < 200) {
setTimeout(draw, 1);
}
};
draw();
});
</script>

What else can GR be used for?
pyMolDyn
see Poster session: Embedding visualization applications with pygr by Christian Felder

Conclusions
✓ The speedups when using the GR Matplotlib backend were behind the
expectations – but using GR supersedes “backend hacks”
✓ GR adds more plotting capabilities to Matplotlib allowing to mix 2D drawings
and 3D graphics scenes or create movies on the fly
✓ Producing plots / figures is flexible and much faster with the GR framework
(speedup for plots >20 and >100, respectively) 
➟ e.g. export GKS_WSTYPE=pdf … and then run your (batch) script(s)
15

Outlook
✓ simplify the installation ➟ “self–contained distribution” for Anaconda and
Canopy, or build from scratch
✓ provide more convenience functions (pylab ➟ grlab)
✓ migrate the GR3 library to modern OpenGL (using OpenGL shader language) 
➟ visualize millions of vertices / faces
16

Resources
✓ Website: http://gr-framework.org
✓ GR framework: https://github.com/jheinen/gr
✓ PyPI: https://pypi.python.org/pypi/gr
✓ Talk material: Getting more out of Matplotlib with GR
17

Thank you for your attention
Questions?
Contact:
j.heinen@fz-juelich.de 
@josef_heinen
Thanks to:
Fabian Beule, Steffen Drossard, Christian Felder, Marvin Goblet, Ingo Heimbach,
Daniel Kaiser, Philip Klinkhammer, David Knodt, Florian Rhiem, Jörg Winkler et al.
18

Getting more out of Matplotlib with GR

More Related Content

Similar to Getting more out of Matplotlib with GR (20)

Recently uploaded (20)

Getting more out of Matplotlib with GR