Getting more out of Matplotlib with GR

Size: px

Start display at page:

Download "Getting more out of Matplotlib with GR"

Rodney Ferguson
8 years ago
Views:

1 Member of the Helmholtz Association Getting more out of Matplotlib with GR July 20 th 26 th, 2015 Bilbao EuroPython 2015 Josef

2 Visualization needs visualize and analyzing two- and three-dimensional data sets plot 2D data for real-time monitoring purposes (signal processing) visualize large data sets, probably with a dynamic component, preferably in real-time create publication-quality and web-ready graphics create animations or videos on the fly 2

large data sets, probably with a dynamic component, preferably in real-time

3 Python visualization solutions Plotly Bokeh Matplotlib de-facto standard ( workhorse ) Browser solutions: Bokeh, plot.ly Mayavi (mlab) powerful, but overhead from VTK ggplot, chaco statistical, 2D graphics VTK versatile, but difficult to learn ggplot chaco Vispy, Glumpy, OpenGL fast, but low-level API 2D 3D 3

ly Mayavi (mlab) powerful, but overhead from VTK ggplot, chaco statistical,

4 Problems so far Crux of the matter separated 2D and (hardware accelerated) 3D world Interop some graphics backends "only" produce "figures" no presentation of continuous data streams speed up only by means of backend specific code poor performance on large data sets Speed Quality 4

"figures" no presentation of continuous data streams speed up only by

5 Where to go from here? How can we improve the performance? Several Python modules can be compiled into native code, making them much faster (Cython) Compiling hotspots on the fly (Numba, PyPy) can significantly speed up numerical code segments Use hardware acceleration, but these approaches cannot easily be applied to visualization software! Could another backend speedup Matplotlib and improve interop? 5

hotspots on the fly (Numba, PyPy) can significantly speed up numerical code segments Use hardware

6 Use GR to achieve more graphics performance procedural graphics backend (completely written in C) presentation of continuous data streams builtin support for 2D plotting and OpenGL (GR3) coexistent 2D and 3D world interoperability with GUI toolkits and Web frameworks good user interaction 6

builtin support for 2D plotting and OpenGL (GR3) coexistent 2D and 3D

7 Use GR to extend Matplotlib s capabilities combine the power of Matplotlib and GR next Matplotlib release will allow selecting the backend by setting the environment variable MPLBACKEND produce video contents on the fly by adding a single line of code no need to import an animation module or write extra code create plots containing both 2D and 3D graphics elements 7

MPLBACKEND produce video contents on the fly by adding a single line of code no need to

8 How it works: GR layer architecture Qt console interaction Notebooks support for Python or Julia kernels Use GR as a Matplotlib backend; mix GR, MPL and GR3 code glgr / igr App C / ObjC OpenGL Qt / wx event loop ES socket communication IPython GR off-screen rendering PyPy Swift direct rendering GR3 C / C++ JavaScript generation Browser... POV-Ray generation GLUT GLFW wxglcanvas QGLWidget Generate ray-traced graphics scenes Create HTML5 output GKS WebGL OpenGL (WGL / CGL / GLX) POV-Ray Create video animations on the fly GKS logical device drivers Qt wx X11 Quartz Win32 TikZ PDF MOV gksqt GKSTerm LATEX PS 0MQ HTML5 SVG OpenGL... More logical device drivers / plugins: CGM, GKSM, GIF, RF, UIL WMF, Xfig GS (BMP, JPEG, PNG, TIFF) Highlights: simultaneous output to multiple output devices direct generation of MPEG4 image sequences flicker-free display ("double buffering ) IPython / Jupyter notebook integration

.. POV-Ray generation GLUT GLFW wxglcanvas QGLWidget Generate ray-traced graphics scenes Create HTML5 output GKS WebGL OpenGL (WGL / CGL / GLX) POV-Ray Create video animations on the fly GKS logical

9 Matplotlib using the GR backend $( $& Arrow Line2D %#"# $% $)"' $# * $%"' ( $#"# Ellipse FancyBoxPatch & )"' % '"#! $! "!! %! $! #! "! #! "! # % & + ( #"* #") &"( #"& &"# #"# %"( 135 $"( &"# &"( '"# '"( ("# #"* #") &"( #"& &"# #"# %"( %"#! #") $"( &"( #!"# "!)+ "!'( "!%& "!## "!## "!%& "!'( "!)+ #!"# ( ! #"* &"# #! # &$% # %$') # &$* # %$(+ # '$% # '$% 2! #"& %"( %! $! '"# %"# %! $"% '"( $"( $! 8! #"* %"( #! # %$&' # &$* '"# '"( # &$% # %$* %$% %$* &$% &$* '$% )/07,5/2- * (,0*82*9 1,6.!*440/+*7/32 73./-.# 5,63087/32 75/ /2-" 45! #") :/7. 0,9,06 %$%% &$%! #"& %"# %"# "! #"# $# *!! "! ( $"% '"# $"( %$&' ) %$% %"'!%!% '"( $"# $"# %$') ,6. :/7. 0,9,06 ("# &$% &! ' PathPatch %$(+ # %$* # Polygon &$* %$* %! Circle %$), &! $'"# - Rectangle "!! +, Wedge /67!'%% " '$% %%"' & $ " $ $ & " $ & ( "!' %$' "!* "!( "!& "!$ "!" "!$ "!& "!( "!* "!% "!$ "!" %$% "!$ "!% "!' #!" # %$' "!& #!" & ( ( 270 # &$% # &$% # %$' %$% %$' &$% "!& "!" "!" "!& "!& #!" #!" 9

#"* &"# #! # &$% # %$') # &$* # %$(+ # '$% # '$% 2! #"& %"( %! $! 10 180 '"# %"# %! $"% '"( $"( $! 8! #"* %"( #! # %$&' # &$* '"# '"( # &$% # %$* %$% %$* &$% &$* '$% )/07,5/2- * (,0*82*9 1,6.

10 GR in action 10

11 GR / Jupyter 700 Performance (anim.py) 600 fps MPL MPL+GR GR click images to view notebooks

12 Performance analysis ncalls cumtime filename:lineno(function) fps MPL {method'draw'of'_macosx.figurecanvas'objects} 29378/ artist.py:57(draw_wrapper) figure.py:1004(draw) _base.py:1989(draw) axis.py:1106(draw) axis.py:232(draw) pyplot.py:175(pause) lines.py:661(draw) pyplot.py:551(draw) text.py:581(draw) most time spent in backend wrapper 400 ncalls cumtime filename:lineno(function) ??? MPL + GR pyplot.py:551(draw) backend_gr.py:227(draw) 14726/ artist.py:57(draw_wrapper) figure.py:1004(draw) _base.py:1989(draw) axis.py:1106(draw) axis.py:232(draw) lines.py:661(draw) backend_bases.py:237(draw_markers) No room for further optimizations on the backend side 0 MPL MPL+GR GR GR ncalls cumtime filename:lineno(function) init.py:1910(plot) init.py:250(updatews)

py:551(draw) 7940 1.044 text.py:581(draw) most time spent in backend wrapper 400 ncalls cumtime filename:lineno(function) 300 200 100??? MPL + GR 199 4.412 pyplot.py:551(draw) 199 4.410 backend_gr.

13 GR + GR3 + Matplotlib interop Important: tells MPL backend not to update Matplotlib GR3 GR 13

14 Inline graphics Matplotlib GR ~ 10 times faster

15 Demos Animated graphics performance comparison: Matplotlib vs. GR (anim.ipynb) GR / mogli / Matplotlib interoperability example (interop.ipynb) Inline graphics performance comparison: Matplotlib vs. GR (inline.ipynb) Simple spectral (specgram.ipynb)

16 Outlook (GR release v0.15.0) JavaScript GR + GKS can be transpiled to JS (Emscripten: LLVM-to-JavaScript compiler) Use cases: embed JS code in IP[y]: or IJulia (Jypyter) parse GKS JavaScript logical device driver generated display list in browser

17 What else can GR be used for? pymoldyn see Poster session: Embedding visualization applications with pygr by Christian Felder

18 Conclusions Using the GR Matplotlib backend has not turned out satisfactory as the speedups were not as expected GR adds more plotting capabilities to Matplotlib allowing to mix 2D drawings and 3D graphics scenes or create movies on the fly Producing plots / figures is much faster with the GR framework (speedup for plots > 20, > 100 respectively) 18

to mix 2D drawings and 3D graphics scenes or create movies on the fly Producing plots /

19 What happens next? integrate JavaScript GKS logical device driver provide more convenience function migrate the GR3 library to modern OpenGL (using OpenGL shader language) visualize millions of vertices / faces simplify the installation 19

20 Resources Website: GR framework: PyPI: Talk material: Getting more out of Matplotlib with GR 20

com/jheinen/gr PyPI: https://pypi.python.

21 Thank you for your attention Questions? Thanks to: Fabian Beule, Steffen Drossard, Christian Felder, Marvin Goblet, Ingo Heimbach, Daniel Kaiser, Philip Klinkhammer, David Knodt, Florian Rhiem, Jörg Winkler et al. 21

GR.jl Plotting for Julia based on GR

Member of the Helmholtz Association GR.jl Plotting for Julia based on GR June 24 th 28 th, 2015 Massachusetts Institute of Technology, Cambridge, Massachusetts JuliaCon 2015 Josef Heinen @josef_heinen