Postprocessing with Python

Transcription

1 Postprocessing with Python Boris Dintrans (CNRS & University of Toulouse) Collaborator: Thomas Gastine (PhD)

2 Outline

3 Outline Introduction - what s Python and why using it? - Installation procedure

4 Outline Introduction - what s Python and why using it? - Installation procedure Python and the Pencil Code - the Python repository and initialization - Migrating from IDL to Python - Some examples & tricks - Parallel Python with Pypar - Doing widgets with PyQt

5 Outline Introduction - what s Python and why using it? - Installation procedure Python and the Pencil Code - the Python repository and initialization - Migrating from IDL to Python - Some examples & tricks - Parallel Python with Pypar - Doing widgets with PyQt Conclusion/Outlook

6 What s Python?

7 What s Python? Python was created in 1991 by Guido van Rossum (CWI, Centrum voor Wiskunde en Informatica, Amsterdam) Benevolent Dictator for Life (BDFL)

8 What s Python? Python was created in 1991 by Guido van Rossum (CWI, Centrum voor Wiskunde en Informatica, Amsterdam) Benevolent Dictator for Life (BDFL)

9 What s Python? Python was created in 1991 by Guido van Rossum (CWI, Centrum voor Wiskunde en Informatica, Amsterdam) Benevolent Dictator for Life (BDFL)

10 The first release in 1991 on alt.sources

11 DARPA funding proposal Computer Programming for Everybody (1999):

12 DARPA funding proposal Computer Programming for Everybody (1999): an easy and intuitive language just as powerful as major competitors open source, so anyone can contribute to its development code that is as understandable as plain English suitability for everyday tasks, allowing for short development times

13 DARPA funding proposal Computer Programming for Everybody (1999): an easy and intuitive language just as powerful as major competitors open source, so anyone can contribute to its development code that is as understandable as plain English suitability for everyday tasks, allowing for short development times year version

14 DARPA funding proposal Computer Programming for Everybody (1999): an easy and intuitive language just as powerful as major competitors open source, so anyone can contribute to its development code that is as understandable as plain English suitability for everyday tasks, allowing for short development times year version Why Python? - it s free! ;-) - quite easy to use; object-oriented; highly modular, etc... - much more rapid than IDL and even PARALLEL

15 The main Python website:

16 The SciPy website:

17 How to install Python?

18 How to install Python? Required: python 2.5: the engine numpy: the scientific computing package (arrays, linear algebra, FFT, random numbers, etc...); [replaces old numarray and numeric] scipy: modules for integrating ODEs, optimizing functions, etc... [tends to federate all of Python scientific modules] matplotlib: MATLAB-inspired mostly-2d plotting modules

19 How to install Python? Required: python 2.5: the engine numpy: the scientific computing package (arrays, linear algebra, FFT, random numbers, etc...); [replaces old numarray and numeric] scipy: modules for integrating ODEs, optimizing functions, etc... [tends to federate all of Python scientific modules] matplotlib: MATLAB-inspired mostly-2d plotting modules Optional: ipython: convenient shell to develop and run Python basemap: map projections Pypar: parallel Python (interface with MPI libraries) PyQt: to do Qt-like widgets VERY easily under Python MayaVi: 3D plotting

20

21 From sources or binary packages?

22 From sources or binary packages? For all platforms: everything can be compiled from sources For Linux, Windows & Mac (at least): binaries are provided (Linux: yum, apt-get, dpkg; Mac: Fink, MacPorts, dmg)

23 From sources or binary packages? For all platforms: everything can be compiled from sources For Linux, Windows & Mac (at least): binaries are provided (Linux: yum, apt-get, dpkg; Mac: Fink, MacPorts, dmg) Linux (FedoraCore 7) Mac OS X 10.4 (Tiger) Python Numpy Scipy Matplotlib ipython

24 Typical installation on a Linux box

25 Python on Mac OSX:

26 Scipy Superpack for OS X:

27 Python packages on my Macbook Pro (SciPy Superpack)

28 Python in the Pencil Code repository: f90/pencil-code/numpy commited by Jeff in fall of 2007 revision 1.1 date: :57: ; author: joishi; state: Exp; * added python scripts for reading pencil code data. they require only the numpy package, but matplotlib is useful for plotting. almost all of these routines are simplified clones of their idl counterparts. i'd love to make a more OO pencil-code package, but my current occupational constraints make that unlikely in the near term. NB: the byte ordering in python is C, not fortran, so these routines return an f array with shape f[nvar,nz,ny,nx]--the reverse of pencil. * added nl2python to take advantage of the amazing perl F90Namelist.pm * modified F90Namelist.pm to output python * i hope these are moderately useful to people!

29 The actual Python tree: 3 directories The reading stuff numpy/pencil/files init.py yzaver.py yaver.py xyaver.py npfile.py dim.py param.py grid.py slices.py zprof.py index.py var.py ts.py

30 The actual Python tree: 3 directories The reading stuff The math stuff numpy/pencil/math numpy/pencil/files init.py yzaver.py yaver.py xyaver.py npfile.py dim.py param.py grid.py slices.py zprof.py index.py var.py ts.py init.py vector_multiplication.py derivatives/ numpy/pencil/math/derivatives init.py der.py div_grad_curl.py der_6th_order_w_ghosts.py

31 ...and the initialization of $PYTHONPATH in f90/pencil-code/sourceme.csh # Set PYTHON path if ($?PYTHONPATH) then setenv PYTHONPATH "${PYTHONPATH}:${PENCIL_HOME}/numpy" else setenv PYTHONPATH "${PENCIL_HOME}/numpy" endif

32 ...and the initialization of $PYTHONPATH in f90/pencil-code/sourceme.csh # Set PYTHON path if ($?PYTHONPATH) then setenv PYTHONPATH "${PYTHONPATH}:${PENCIL_HOME}/numpy" else setenv PYTHONPATH "${PENCIL_HOME}/numpy" endif These modules are loaded when importing the whole pencil directory due to the init.py file

33 cat numpy/pencil/ init.py

34 cat numpy/pencil/ init.py In [1]: import pencil as pc In [2]: pc.read_ts()

35 An important point: Python s classes Python is an object-oriented interpreted language: instead of doing pc.read_ts(), it is better to do a=pc.read_ts()

36 An important point: Python s classes Python is an object-oriented interpreted language: instead of doing pc.read_ts(), it is better to do a=pc.read_ts()... and we can plot the other variables read in time_series.dat and embedded in object a

37 Another example when using pc.read_var()

38 Another example when using pc.read_var()...and we plot the entropy at the top of the 32^3 box

39 Another examples of postprocesing with Python MayaVi: 3D plots Basemap: various kind of map projections

40

41 Be careful: Python s arrays are ordered like f[nvar,mz,my,mx] i.e. REVERSED ORDER COMPARED TO PENCIL- CODE OR IDL!!!

42 Migrating from IDL to Python: some useful Web Guides

43

44 For lazy guys: the i2py converter

45 Some tricks when using Python...

46 Some tricks when using Python... plays with ~/.ipython/ipythonrc to load modules by default (import_all pencil) and thus use read_var instead of pc.read_var(), etc...

47 Some tricks when using Python... plays with ~/.ipython/ipythonrc to load modules by default (import_all pencil) and thus use read_var instead of pc.read_var(), etc... import just what you need! (a cleaning is certainly needed in that respect in the PC tree...)

48 Some tricks when using Python... plays with ~/.ipython/ipythonrc to load modules by default (import_all pencil) and thus use read_var instead of pc.read_var(), etc... import just what you need! (a cleaning is certainly needed in that respect in the PC tree...) launch ipython with the -pylab option to call directly plot, contour, imshow, etc...

49 Some tricks when using Python... plays with ~/.ipython/ipythonrc to load modules by default (import_all pencil) and thus use read_var instead of pc.read_var(), etc... import just what you need! (a cleaning is certainly needed in that respect in the PC tree...) launch ipython with the -pylab option to call directly plot, contour, imshow, etc... accelerate the VAR* reading by passing param, grid, index, etc... [tricks.py]

50 Some tricks when using Python... plays with ~/.ipython/ipythonrc to load modules by default (import_all pencil) and thus use read_var instead of pc.read_var(), etc... import just what you need! (a cleaning is certainly needed in that respect in the PC tree...) launch ipython with the -pylab option to call directly plot, contour, imshow, etc... accelerate the VAR* reading by passing param, grid, index, etc... [tricks.py] accelerate the graphics by using an handle [tricks.py]

51 Some tricks when using Python... plays with ~/.ipython/ipythonrc to load modules by default (import_all pencil) and thus use read_var instead of pc.read_var(), etc... import just what you need! (a cleaning is certainly needed in that respect in the PC tree...) launch ipython with the -pylab option to call directly plot, contour, imshow, etc... accelerate the VAR* reading by passing param, grid, index, etc... [tricks.py] accelerate the graphics by using an handle [tricks.py] take advantage of class and objects (a.shape instead of shape(a))

52 Parallel Python using the Pypar module

53 Pypar example 1: compute a vertical profile in parallel

54 Pypar example 2: write PNG files in parallel for a movie

55 Widgets using Qt Designer + PyQt

56 Conclusion/Outlook

57 Conclusion/Outlook Python can do a very good job in the Pencil Code postprocessing Its using is rapidly increasing in astrophysics (NASA, ESA, ESO, labs,...) More in the Pencil Code philosophy (i.e. under GPL) compared to IDL

58 Conclusion/Outlook Python can do a very good job in the Pencil Code postprocessing Its using is rapidly increasing in astrophysics (NASA, ESA, ESO, labs,...) More in the Pencil Code philosophy (i.e. under GPL) compared to IDL the actual Python subroutines must be rewritten in a more oriented-object form (class inheritance) the Python tree shall maybe be re-organized in something like f90/pencil-code/python or??? what s about the calling of Fortran or C subroutines to increase the speed?