Experiences with 2-D and 3-D Mathematical Plots on the Java Platform David Clayworth Maplesoft
What you will learn > Techniques for writing software that plots mathematical and scientific data > How to apply Java tools to these tasks: Java2D for 2D graphs JOGL for 3D graphs > How to make best use of these toolkits, especially JOGL 2
History > Maple is a symbolic math application: Performs math on symbols as well as numbers A huge number of mathematical operations > User interface is all Java (~1M lines) Outputs math as mathematicians, scientists or engineers expect it > Previous toolkit lacked speed, quality, controllability and economy. > Two stage rewrite:2d and then 3D 3
Demo > Maple before 4
Requirements > New Features Interactive annotation Mathematical labels part of the plot Speed and memory improvements Try to use few Java components > Notable Existing Features Plots embeddable in a worksheet Export to bitmap and vector Run on Windows, Linux, Mac and Solaris 5
Design Overview Model View Controller > Maple GUI uses Model-View-Controller > Each symbol, plot, component, axis etc. has a model > Each model has a view > Model update view layout view draw > Mapping from plot to pixel coordinates during layout 6
Design Plot Atoms > Atoms are small objects that store elements in a form that is quick to draw > They may store pixel positions, or a Shape object, or a symbol image and location > Created at layout time and stored in the MVC view > To draw itself a component just draws all its atoms 7
Design Plot Atoms interface Atom { void draw(graphics g); } class PolygonAtom implements Atom { GeneralPath poly; PolygonAtom(float x[],float y[]) { // create the path } void draw(graphics g) { ((Graphics2D)g).fill(poly); } } 8
Design Math in Labels > Mathematical expressions are drawn directly by positioning the view within the plot > Annotations use an existing package which draws over the plot with Java2D 9
Optimizations: float v int coordinates > drawpolyline(int[],int[],int) v draw(shape) > In theory ints are faster but floats more accurate > You need float coordinates for printing or vector graphics (e.g. Postscript) > Considered using int-based calls on screen and float-based for print > Ended using floats throughout: the differences in both accuracy and performance were small (except for printing). 10
Optimizations: Sprites > Rendering performance was acceptable, except when drawing many symbols > Sprites are small images which can be drawn to the screen instead of lines and shapes > Faster to draw an image than even a simple shape > Create a Sprite object which holds an image for the symbol > Sprites are indexed by symbol, size and colour. 11
Optimizations: Sprites Backgrounds of Sprites must be transparent You need to make sure that symbols are symmetric 12
2D Plots The results > The 2D plots were released in 2007. > Speeded up by a factor of 7 > Memory used less than 1/10 th > The new features were appreciated by customers > Spurred demand for the same features in 3D 13
Requirements 3D Project > New Features Interactive annotation, drawn in Java2D over 3D Mathematical labels part of the plot, embedded at 3D positions Other improvements as before > Notable Existing Features as before 14
Toolkits Candidate toolkits for 3D > The choices: Java3D: a complete scenegraph tool written in Java JOGL: a thin Java layer over OpenGL LWJGL: a Java layer over OpenGL designed for games A game engine, such as JMonkeyEngine > LWJGL eliminated as it uses a single window. > Many game engines have LWJGL underneath 15
Toolkits JOGL v Java3D > Java3D advantages: A complete OO toolkit Simpler to learn than OpenGL > JOGL advantages: If you know OpenGL you know JOGL Virtually full OpenGL functionality Known portability Wide use Doesn t impose it s own framework 16
Toolkits JOGL > GLCanvas (heavyweight AWT) or GLJPanel (lightweight Swing) accelerated components > GLEventListener is attached to the component render the scene > GL object passed to the listener provides Java equivalents for OpenGL calls. > You can (and must) use OpenGL documentation. > Each view object has a method to draw itself with GL calls. 17
Design Mixing 2D labels into 3D > Labels are positioned in 3D space > Math is drawn into an image with a transparent background > Set a raster position in 3D, then shift it in 2D to get the alignment right > Image drawn into the scene with JOGL > gl.glrasterpos3fv(arrayxyz, 0); gl.glbitmap(0, 0, 0, 0,xOff, yoff, null); gl.gldrawpixels(w,h,, image); 18
Design Mixing 2D labels into 3D What we expect What we get (without alpha-clipping) 19
Design Mixing 2D labels into 3D (cont.) > When OpenGL writes shape it sets the depth at which it is written (depth buffer) > Subsequent shapes at a greater depth are not written i.e. surfaces behind a surface aren t seen. > But the depth buffer is written even for transparent pixels > We have to use two passes: Write solid pixels and set the depth buffer Write transparent pixels without the depth buffer 20
Design Mixing 2D labels into 3D (cont.) gl.gldepthmask(true); gl.glalphafunc(gl.gl_gequal,alpha); drawallcomponents(); gl.glalphafunc(gl.gl_less, alpha); gl.gldepthmask(false); drawallcomponents(); > OpenGL Programming Guide Alpha Test (p477 Sixth Edition) 21
Rendering Mechanism Limitations of GLJPanel > Started with one GLJPanel per plot > A Maple document may have many plots. > But GLJPanels take resources (video RAM) > After creating so many they stop rendering > JOGL Issue 370 22
Rendering Mechanism Reusing a GLJPanel > Reuse GLJPanel, stealing them from plots that are offscreen. > But you can t reposition and draw a GLJPanel in the same event > There are other problems with writing plots for export. > A complex and impractical solution 23
Rendering Mechanism GLPBuffers > GLPBuffers allow for drawing hardware accelerated graphics offscreen > Render 3D to the GLPBuffer, then the image to the screen > Now easy to draw 2D annotations over the PBuffer image > We have fewer components > We still need to reuse GLPBuffers since they use video memory 24
Rendering Mechanism Creating a GLPBuffer GLDrawableFactory factory = GLDrawableFactory.getFactory(); GLPbuffer buf = factory.createglpbuffer(, width,height, ); GLContext glcontext = buf.createcontext(null); 25
Rendering Mechanism Drawing to a GLPBuffer void draw(graphics g) { try { glcontext.makecurrent(); GL gl = glcontext.getgl(); // make GL calls BufferedImage img = Screenshot.readToBufferedImage(w,h); g.drawimage(img, x, y null); } finally { if (GLContext.getCurrent()==glContext) {glcontext.release();} } } 26
Rendering Mechanism GLPBuffers (cont) > Not all displays support GLPBuffers: GLDrawableFactory. cancreateglpbuffer(); > Creation can still fail, because they need video resources > Catch GLException when creating; and RuntimeException because of JOGL bugs > Some performance cost 27
Rendering Mechanism Offscreen drawable > There is also a unaccelerated offscreen drawable > This is outside the published interface: it is used by GLJPanel. > See GLJPanel source GLDrawableImpl offscreendrawable = GLDrawableFactoryImpl.getFactoryImpl(). createoffscreendrawable(glcap, ); 28
Platform issues: > Not all machine configurations will work with JOGL out of the box. > Working OpenGL doesn t mean working JOGL > Need to call glgeterror() (or use DebugGL) > Catch GLException and process it > Check your display capabilities with GLDrawable.getChosenGLCapabilities(); > Don t make GL calls when your GLContext is not current 29
Platform issues: Mac > None 30
Platform issues: Windows > Few > ATI drivers: need at least v8.231 (Catalyst 8) > Remote Desktop Connection and some laptop external monitors: May not be able to use the 2-pass transparency algorithm Check the supported alpha bits > Some drivers draw bad plots 31
Platform issues: Linux > Some > Many default Linux installs don t work with JOGL > Proprietary video drivers needed ATI Catalyst 8.12 nvidia version 177 > Some drivers crash 32
Demo > The new plots 33
Summary: 2D > Favour precision over speed > Store presentation data close to the Java drawing format > Use sprites for symbols > Java2D provides all you need for this task 34
Summary: 3D > JOGL: Provides all your 3D needs Is good if you already have structure May not be best if you need to create structure > Programming JOGL is programming OpenGL > Write math as images directly to the 3D scene Requires transparency handling > Manage video drivers, esp. Linux 35
Summary: 3D (cont) > PBuffers: Simplify 2D/3D mixing Allow many plots in an app Have little performance cost Reduce component count Need to go outside the spec for unaccelerated images 36
Resources: > Jumping into JOGL: http://today.java.net/pub/a/today/2003/09/11/jogl2d.html/ > JOGL documentatation: http://download.java.net/media/jogl/builds/nightly/javadoc_public/ov http://www.opengl.org/sdk/docs/man/ > JOGL forum: http://www.javagaming.org/index.php/board,25.0.html > PBuffers: http://today.java.net/pub/a/today/2008/10/30/integrating-glpbuffer-a > OpenGL Programming Guide 37
David Clayworth Senior GUI Developer Maplesoft www.maplesoft.com 38