Tools and Techniques for Developing Atmospheric Python Software: Insight from the Python ARM Radar Toolkit

Size: px

Start display at page:

Download "Tools and Techniques for Developing Atmospheric Python Software: Insight from the Python ARM Radar Toolkit"

Estella Murphy
10 years ago
Views:

1 Tools and Techniques for Developing Atmospheric Python Software: Insight from the Python ARM Radar Toolkit Jonathan Helmus1, Scott Giangrande2, Kirk North3, and Scott Collis1 1 2 Argonne National Laboratory Brookhaven National Laboratory 3 McGill University

Scott Giangrande2, Kirk North3, and Scott Collis1 1 2 Argonne

2 Introduction to Py-ART

Py-ART: Plotting import matplotlib.pyplot as plt import matplotlib.pyplot as plt import pyart import pyart radar = pyart.io.read('xsw110520105408.raw7hhf') radar = pyart.io.read('xsw110520113537.

3 Py-ART: Plotting import matplotlib.pyplot as plt import matplotlib.pyplot as plt import pyart import pyart radar = pyart.io.read('xsw raw7hhf') radar = pyart.io.read('xsw raw7hhl') display = pyart.graph.radardisplay(radar) display = pyart.graph.radardisplay(radar) display.plot('reflectivity', 0, vmin=-32, vmax=64.) display.plot('reflectivity', 0, vmin=0, vmax=54.) plt.savefig('ppi_plot.png') display.set_limits(ylim=(0, 8)) plt.savefig('rhi_plot.png')

raw7hhl') display = pyart.graph.radardisplay(radar) display = pyart.graph.radardisplay(radar) display.plot('reflectivity', 0, vmin=-32, vmax=64.

4 Py-ART: Correct - Doppler Velocity Dealiasing from pyart.correct import dealias_fourdd dealias_data = dealias_fourdd(radar, height * 1000., speed, direction, target) radar.add_field('corrected_velocity', dealias_data) Dealiasing performed using U. Washington FourDD algorithm (James and Houze, JTech, 2001)

Py-ART: Correct - LP phase processing import pyart radar = pyart.io.read('095636.mdv') phidp, kdp = pyart.correct.phase_proc_lp(radar, 0.0) radar.

5 Py-ART: Correct - LP phase processing import pyart radar = pyart.io.read(' mdv') phidp, kdp = pyart.correct.phase_proc_lp(radar, 0.0) radar.add_field('corrected_differential_phase', phidp) radar.add_field('corrected_specific_diff_phase', kdp) Phase processing performed using a LP based algorithm (Giangrande et al, JTech, 2013)

add_field('corrected_differential_phase', phidp) radar.

6 Py-ART: Mapping to Cartesian Grids import pyart radar_sw = pyart.io.read(xsapr_sw_file) radar_se = pyart.io.read(xsapr_se_file) grid = pyart.map.grid_from_radars((radar_se, radar_sw), grid_shape=(1, 201, 201), grid_limits=((1000, 1000), (-50000, 40000), (-60000, 40000)), grid_origin = ( , ), max_refl=100.) Gridding using KD-Tree nearest neighbor lookup and Cressman interpolation.

grid_from_radars((radar_se, radar_sw), grid_shape=(1, 201, 201), grid_limits=((1000, 1000),

7 Building Py-ART: SciPy Stack and other libraries

8 Building Py-ART: Interfacing with legacy code Cython files (.pyx,.pxd) Cython C file (.c) C Compiler (gcc) Compiled Python module (.so) External libraries (*.so,.a)

9 Building Py-ART: Cython Example - Cython side cdef extern from "rsl.h": ctypedef struct Radar: cimport _rsl_h cdef class RslFile: Radar_header h cdef _rsl_h.radar * _Radar Volume **v cdef _rsl_h.volume * _Volume ctypedef struct Radar_header: def cinit (self, filename): int month, day, year self._radar = _rsl_h.rsl_anyformat_to_radar(filename) int hour, minute if self._radar is NULL: float sec raise IOError('file cannot be read.') def dealloc (self): ctypedef struct Volume: _rsl_h.rsl_free_radar(self._radar) Volume_header h Sweep **sweep def get_volume(self, int volume_number): rslvolume = _RslVolume() rslvolume.load(self._radar.v[volume_number]) return rslvolume Radar * RSL_anyformat_to_radar(char *infile) property month: void RSL_free_volume(Volume *v) void RSL_free_radar(Radar *r) def get (self): return self._radar.h.month def set (self, int month): _rsl_h.pxd self._radar.h.month = month _rsl_interface.pyx

_radar is NULL: float sec raise IOError('file cannot be read.') def dealloc (self): ctypedef struct Volume: _rsl_h.rsl_free_radar(self.

10 Building Py-ART: Cython Example - Python side _rsl_interface.pyx _rsl_h.pxd >>> from pyart.io import _rsl_interface >>> rslfile = _rsl_interface.rslfile('xsw raw') >>> print rslfile <pyart.io._rsl_interface.rslfile object at 0x107112d20> Cython >>> print rslfile.month 5 >>> rslfile.month = 12 >>> print rslfile.month 12 >>> _rsl_interace.c >>> volume = rslfile.get_volume(1) >>> print volume <pyart.io._rsl_interface._rslvolume object at 0x > C Compiler (gcc) _rsl_interface.so rsl.so

month 5 >>> rslfile.month = 12 >>> print rslfile.month 12 >>> _rsl_interace.c >>> volume = rslfile.

Software Engineering: Version Control $ git log commit dca8e348ba7db19f675747e8b98acf987d634915 Author: Jonathan Helmus <jjhelmus@gmail.

11 Software Engineering: Version Control $ git log commit dca8e348ba7db19f675747e8b98acf987d Author: Jonathan Helmus <[email protected]> Date: Fri Jan 24 11:46: ENH: pyproj no longer an optional dependency pyproj is included in basemap, use the version commit 26c7a36dadfbdcc1c4e1dc1d8bfe188a073e08a4 Author: Jonathan Helmus <[email protected]> Date: Mon Jan 20 15:49: TST: do not install cylp or cvxopt in python 2.6 commit 1eeed39faf89d4aa9eda6014c4dbc1a2458bbb9a Author: Jonathan Helmus <[email protected]> Date: Mon Jan 20 15:42: TST: do not all basemap in python 2.6 commit d1ce060537df08aad39b8ad3a75db13a9728aa1a Author: Jonathan Helmus <[email protected]> Date: Mon Jan 20 15:35: BUG: fixed bugs when running in Python 2.6

Jonathan Helmus <jjhelmus@gmail.com> Date: Mon Jan 20 15:49:15 2014-0600 TST: do not install cylp or cvxopt in python 2.

$Software Engineering: Testing def test_radar_creation(): radar = pyart.testing.make_target_radar() assert isinstance(radar, pyart.io.radar) def test_add_field(): radar = pyart.testing.make_target_radar() dic = {'data': np.$

12 Software Engineering: Testing def test_radar_creation(): radar = pyart.testing.make_target_radar() assert isinstance(radar, pyart.io.radar) def test_add_field(): radar = pyart.testing.make_target_radar() dic = {'data': np.zeros((360, 50)), 'standard_name': 'test'} radar.add_field('test', dic) assert 'test' in radar.fields assert 'data' in radar.fields['test'] assert radar.fields['test']['standard_name'] == 'test' ~/python/pyart$ nosetests -v test_attenuation.test_attenuation ok test_dealias.test_find_time_in_interp_sounde ok test_config.test_filemetadata_custom ok test_config.test_init_load ok test_config.test_intergration ok Ran 386 tests in s OK (SKIP=2)

fields['test'] assert radar.fields['test']['standard_name'] == 'test' ~/python/pyart$ nosetests -v test_attenuation.test_attenuation ok test_dealias.

13 Software Engineering: Continuous Integration

Software Engineering: Embedded Documentation def grid_from_radars(radars, grid_shape, grid_limits, **kwargs): """ Map one or more radars to a Cartesian grid returning a Grid object.

14 Software Engineering: Embedded Documentation def grid_from_radars(radars, grid_shape, grid_limits, **kwargs): """ Map one or more radars to a Cartesian grid returning a Grid object. Additional arguments are passed to :py:func:`map_to_grid` Parameters radars : tuple of Radar Radar objects which grid_shape : 3-tuple of Number of points in objects. will be mapped to the Cartesian grid. floats the grid (z, y, x).

Additional arguments are passed to :py:func:`map_to_grid` Parameters ---------radars : tuple of Radar

15 GitHub

16 Py-ART: File I/O Format Native Read Read w/ RSL Sigmet/IRIS X X MDV X Cf/Radial X (+writing) Universal (UF) X Lassen X NEXRAD Level II DORAD X X

17 Radar 101 Top view Side view Gate Sweep Radial Radar Volume Sweep Sweep Sweep Radial Radial Radial Radial Gate Gate Gate Gate Gate

18 Py-ART: Data layout Radar object attributes Layout ngates : int nrays : int nsweeps : int scan_type : str Radar Sweep layout sweep_start_ray_index : dict sweep_end_ray_index : dict sweep_number : dict sweep_mode : dict fixed_angle: dict Dimensions time : dict range: dict azimuth : dict elevation : dict Location longitude : dict latitude: dict altitude : dict Fields : dict Optional metadata : dict radar_calibration : dict Field attribute keys The format of the Radar object closely follows the Cf/Radial format (Dixon et al) reflectivity : dict data : 2D array units : str spectral_width : dict data : 2D array units : str dopper_velocity : dict data : 2D array units : str differential_phase : dict data : 2D array units : str

Fields : dict Optional metadata : dict radar_calibration : dict Field attribute keys The format of the Radar object closely follows the Cf/Radial format (Dixon et al) reflectivity :

Exercise 0. Although Python(x,y) comes already with a great variety of scientic Python packages, we might have to install additional dependencies:

Exercise 0. Although Python(x,y) comes already with a great variety of scientic Python packages, we might have to install additional dependencies: Exercise 0 Deadline: None Computer Setup Windows Download Python(x,y) via http://code.google.com/p/pythonxy/wiki/downloads and install it. Make sure that before installation the installer does not complain