A Four-Dimensional Multiple-Source Weather Information System for Algorithms and Visualization

Transcription

1 A Four-Dimensional Multiple-Source Weather Information System for Algorithms and Visualization V Lakshmanan, Thomas Vaughan National Severe Storms Laboratory & University of Oklahoma January 24, 2003 Abstract We present a consistent means of accessing, in real-time or play-back, data from disparate sources. The data are geographically and temporally referenced, and placed in one of several source-independent in-memory formats depending on the type of data. We describe how to implement this referencing, and how to take advantage of the referencing and common access paradigms in the development of weather information algorithms and visualization techniques. The concepts described here are implemented in the Warning Decision Support System (WDSS-II). Therefore, the WDSS-II permits easy integration of data from multiple sources. Use of the WDSS-II or a system designed similarly would lead to timely development and implementation of algorithms and displays, especially those that rely on multiple sources of information. 1. Introduction Optimal visualization of weather data and algorithm outputs ( products ) has long been an area of study, but most studies have concentrated on ease of use (Sanger et al. 1995), visualization of new types of data (Smith 1995) or novel ways of visualizing data (Brubaker et al. 1991). Our focus here is different. In this paper, we show how to obtain and visualize data from multiple sources in an integrated manner. Common means of access to streaming data, both base data and products, across disparate systems has been an area that has been completely ignored in the meteorological information systems community. Thus, algorithms and visualization techniques developed with considerable effort on one system need to re-implemented on operational systems. This often leads to long lag times between research and operational use. Integration, especially of time-variant information such as weather data, implies several abilities: Corresponding author address: [email protected] Now with Ball Aerospace, Boulder, CO 1

2 1. To visualize data from different sources of the same type, such as from geographically proximate radars simultaneously. A substantial fraction of the U.S. surface area is covered by more than one radar. 2. To view algorithm outputs from different techniques simultaneously for comparision purposes. 3. To deal with time and scanning strategy variations in the data sources. For example, each radar will start a tilt at a different time, and because each radar may be in a different scanning mode, one radar will require a different amount of time to scan a volume of data than another radar. Even if two radars share the same scanning mode, there is no guarantee that each begins a volume scan at the same time. 4. To visualize data from completely different sources. For example: national lightning data, base radar data, and terrain information simultaneously. 5. To provide data from disparate systems in a common manner. For example: to provide radar data whether from RIDDS, from a LDM feed or from an ORPG in such a way that algorithms and displays deal transparently (through the same interface) with all these data sources. In other words, integration combines data from different sources, of different types, and with varying formats. Moreover, the combination happens in such a way that every client has access to and may process the data in a common manner. It should be noted that we are not talking of data integration or merging data into a common representation. The various data are separate, but treated in a common framework. Where integration has been studied, it has been studied to the task at hand, rather than in a more general sense. For example, Hembree et al. (1997) describe a visualization system that integrates clouds with a flight training system. Integration commonly happens outside of the visualization, in special-purpose algorithms James et al. (2000), for example, present only one time-changing source of information to the visualization system. Techniques for time synchronization amongst different data streams have been ad hoc. Techniques vary from (1) having an algorithm that relies on multiple data streams pick up the latest data every five minutes (Zhang et al. 2000) to (2) doing the processing with the latest products when one of the products (the driving input) arrives (e.g: (Lakshmanan and Witt 1997)) to (3) combining different algorithms into one process to get around the problem of integration (Stumpf et al. 1995). For displaying data from various sources, techniques, driven as they are by the non-rigorous manner of dealing with time, have also been ad hoc. In this paper, we formalize the temporal and geographical integration of disparate sources of data. The concepts described in this paper have been implemented in the second generation Warning Decision Support System (WDSS-II) which was tested in Spring 2001 at the National Weather Service office at Norman, OK. The rest of this paper is organized as follows. In Section 2, we describe the representation of different types of data. In Section 3, we describe the common method of 2

3 accessing those data. In Section 4, we describe how to implement an algorithm in this framework. In Section 5, the design of a multiple-source visualization tool is described. 2. Data Representation Every datum in the system is referenced in a four-dimensional (4D) world three geographical coordinates and one temporal one. The data thus represented range from the sparse and diffuse (lightning ground strikes) to the dense and profuse (every range gate of every radar elevation scan) to the surprising (every row in a table of algorithm output). There are two geographical coordinate systems, either of which may be used for weather data: An arbitrarily oriented and located earth-fixed cartesian coordinate system. An earth-fixed spherical coordinate system of latitude, longitude, and height above mean sea-level. Some data are better referenced in one; some, in the other (more on this later). Hence, we provide ways to get from one coordinate system to another quickly and efficiently. Although, conceptually, every datum on the system possesses a 4D reference, it would be overkill to allocate computer memory to store this 4D reference. Instead, many of the references could be computed on an as-needed basis, if we organize the data. For example, radials from a radar scan could be organized such that we store the reference to the first gate of the radial, the elevation angle of the scan and the width of a gate in kilometers. From these, the 4D reference of any gate in the radial can be computed. Similarly, if an entire table of algorithm output corresponds to just a single event (something that took place at a particular location at a particular time), then storing the same location at every row is wasteful the entire table can be associated with the same 4D reference. In addition to the 4D referencing, data share another attribute: the concept of expiry time. Depending on the source of the data, we can reasonably associate a time beyond which the data are too old. Of course, if newer data arrive from a source (or already exist), then the current data are expired. Associating an expiry time with each data set allows us to deal with multi-source data in a fault-tolerant manner, so that, for example, we can tolerate a radar that goes down for maintenance. In practice, we set the expiry time to be about twice the periodicity of the data (10 minutes for data that commonly arrive every five minutes). a. Units Different data sources can provide data of the same type, but each source may provide the data in a different unit system, or scaled in a different way. If an algorithm is to treat data transparently and independently of source, then the units of data ought to be defined. However, there is often a preferred unit for the data from a source, one that users (and legacy algorithms) have come to expect. 3

4 Thus, we found two separate requirements: (1) An algorithm should be able to access the data in a common format. For example, an algorithm should always be able to obtain velocity data in m/s. (2) If, however, the data are stored as mph, then the information should normally be displayed as mph. To satisfy both requirements, data should be stored in such a manner that at any point of time, the default numerical value as well as the units in which that default value are expressed are available. To ensure transparent access for algorithms, we also provide an extensive unit conversion library that treats all units as integral powers of space, time, mass and charge. For example, the unit of velocity is S 1 T 1 M 0 C 0. Different units for the same quantity differ only in their scalar multipliers. If g/m 2 has a scalar multiplier of 1, then kg/m 2 would have a scalar multiplier of English units can also be similarly expressed. Units may be specified in long form ( Feet ), in a short form ( ft ) or as a combination of a scalar multiplier and the four integral powers. Given data, such as vertically integrated liquid (VIL), in kg/m 2, an algorithm can obtain the numerical equivalent in g/m 2, g/km 2, etc., but not as kg/m. The algorithm typically asks for data in specific units, and the appropriate conversions will be made from the default storage of the data source from which the data comes. b. 2D products We have encountered three types of 2D products: 1. The surface of a cone, e.g., a radar elevation scan. The apex of the cone is the radar and a conical surface is traced out by the radar as it sweeps around. This surface may or may not be a complete cone a terminal doppler weather radar (TDWR) sweeps out only part of the cone. Such data are organized as twodimensional data, where the first dimension is an angle, and the second dimension is a length (azimuth and range). Each gate, then, is referenced by specifying the radial on which it lies as well as the distance along the beam to the radar. Each radial, in turn, is referenced by specifying the angle that the radial makes with geographical north and the angle to the earth s surface. 2. A uniform grid in spherical coordinates, e.g., data projected to the surface of the earth. These two-dimensional data are parameterized by two angles (latitude and longitude). Each pixel is thus referenced by specifying the latitudinal and longitudinal offsets from either the center of the grid or from one of the corners. 3. A uniform grid tangential to a sphere concentric with the earth s surface, e.g., a flat earth projection such as VIL and composite reflectivity products. Each of the two dimensions here is a length, a distance from the tangent point. As can be seen, there are differences among these three types of product. Each is twodimensional, but the topology varies from one to the next. Ignoring these differences will cause errors in geographic association. In spite of this, many radar algorithms and visualization systems do not differentiate among these three types. The problem is not significant in systems where the visualization covers a small domain (Sanger et al. 1995; Hondl et al. 1999; Jain et al. 1998) but becomes significant for multiple-source 4

5 data, distributed over a region whose dimensions are significant in comparison with the radius of the earth. c. 3D products Each 3D product type is the extension of a 2D type: 1. A set of conapical cones, representing a radar volume, in which the three dimensions are azimuthal angle, radial length, and elevation angle. Because of the different volume coverage patterns (VCPs), there is no uniformity in the third dimension. 2. A stack of latitude-longitude (lat-lon) grids in which, besides the two angular dimensions, the third dimension is the height above mean sea level. Some products are uniform in the third dimension (height), but a significant fraction of weather products become sparser as one moves further away from the surface of the earth. 3. A 3D cartesian grid, in which each dimension is a length,. and where the base of the grid is often tangential to the earth s surface. Visualization of products of the third type is what is commonly understood, in the visualization community, by 3D visualization. Hence, most visualization techniques deal with only the third type of 3D product described above. With some approximations, especially on products over a small geographic domain, the visualization techniques developed for the third case can be applied to the second (James et al. 2000). In the case of integrated data from multiple sources, these approximations do not hold. The technique of visualizing products of the first two types in a geographically correct manner is described in Section 5.. d. Non-image products In addition to the 2D and 3D products described above, weather information systems provide some non-image products. Besides the trivial free-text message, these include such products as tabular data, tracks, trends, icons and contours. We can treat tabular data, tracks, trends and icons in a common framework. This makes it convenient for algorithms which can provide one output format, which can then be sliced through in different ways by different clients. A table of data is considered to be a collection of columns, arranged such that corresponding items in different columns refer to the same entity, which is given a 4D reference. This permits the extraction of tabular data as either tables or as point-overlays (with a row s 4D reference used to locate its point). In addition, consecutive tables can be associated in a table history. With a table history, it is now possible to extract the data either as a trend (the selection of the same row-column pair for each time) or as a track (the selection of the same row s 4D reference for each time). In some cases, such as in rainfall basin estimates, the row denotes the same location at every time, and the track extraction does not make sense. In other cases, such as in storm centroids, the row s offset in the table moves in time, and the concept of same row is defined to mean the row that possesses the same storm ID, given by the tracking algorithm. 5

6 In tabular data, as in 2D and 3D products, the units are carried through. A unit is associated with each column. Because each different user prefers to see a table s units in a different way (one column in lb/in 2 and another in kg/m 2, depending on the quantity that is being measured), providing an option to toggle between metric and English units does not solve the problem. In the WDSS-II system, every column of every table can be configured individually by the user. This highly specific configuration includes the units of the quantity, the format of the quantity (how many significant digits), and the coloring of the table cell. The coloring of the table cell can be made to depend either on its own value or on the value of another cell in the same row. 3. Common Access Method To provide access to multiple data sources, we generalized the considerations that are required to access data and to be notified of its availability. These generalization is, conceptually, a database record. It should be emphasized that no unified database actually exists. Instead, a database is constructed ephemerally by each process out of the raw information provided in different forms by different data sources. Conceptually, there is a different database table for each of the sources. a. A Record A database record corresponds to a chunk of data commonly considered as an entity on the data source. For example, there are separate database records for radar elevation scans, all the lightning strikes in a 30-second period, or the output of a hydrometeor classification algorithm at a particular point in time. The database record specifies the means of accessing and building the data, of constructing the appropriate representation of the data, complete with units. The means of constructing the data are specified by an array of character strings. For example, the strings netcdf and mach:/tmp/test.ncf are enough to specify that the data in question is of the netcdf format and can be accessed from the file location /tmp/test.ncf on the machine mach. For data stored in standard formats such as NetCDF (Jenter and Signell 1992) and XML (Bray et al. 2000), specifying the means of access is simple, since the file itself encodes the type of data to be built. If the data is stored in formats that are less self-explanatory, that information has to provided in the record itself. Because it possesses the specification of how to access the data, the database record can be asked to build the data whenever required. It will load the appropriate builder into memory and hand off the specification to the builder. The builder will access the appropriate data store, do the necessary formatting and return the object in a consistent 4D referenced data representation. In addition to the means of accessing the data, the database record also specifies the standard way in which the data will be referred to by algorithms. This is important if we want algorithms and visualization software to deal transparently with different data sources. Thus, the VIL product whether produced by the ORPG or by the WDSS VIL algorithm will be refered to as VIL. The specification consists of these standard elements: 6

7 1. Time stamp (e.g: 2001:06:04-13:12:41 UTC) 2. Data type (e.g: Reflectivity) 3. Optional subtype (e.g: 0.5, the elevation angle) Finally, the record also contains a form of 4D reference. It contains the actual time that the product was constructed (the time stamp is merely a string for human consumption) and the database table to which it belongs. Since a database table is associated with a particular source, we can differentiate between reflectivity products from different radars. b. In-memory database As each record is constructed, it is placed into an in-memory database suitable for navigation, search and update. The records themselves (except for the link to the inmemory database) may be stored on disk for quicker reconstruction. The database is comprised of a number of red-black trees into which records are placed based on their selection criteria and time. i. Search It is possible to search the database for all records that meet certain selection criteria all the records at a particular time, all the reflectivity records after a particular time, etc. It is also possible to get the latest record of any particular datatype (and subtype) available. ii. Navigation Given a database record, it is possible using the index that it is associated with to obtain the next data of that type. For example, given a record corresponding to the 0.5 degree reflectivity scan from KFWS at UTC, it is possible to request the next elevation scan. If one exists, the record corresponding to that scan will be provided. Similarly, it is possible to step backward in time. It is also possible to get the next or previous subtype in radar volumes, this corresponds to being able to step up or down in elevations. iii. Event Records In addition to the data records described above, special event records are also created and added to the database. Such records can be searched and navigated. For example, radar sources provide a new elevation event at the start of every tilt. From one such event, the record corresponding to the start of the previous tilt may be navigated to. Event records are not associated with any data, so no data objects can be created from such records. c. Updates For archived data sets, the database once constructed does not change. In real-time or playback, however, the in-memory database automatically updates when new data become available. So, consecutive searches or navigations could yield different results based on whether new data (or events) have become available. The application can attach any number of listeners to the database. These listeners will be notified whenever any new data arrive. Applications normally discard records 7

8 that they are not interested in (the records are added to the in-memory database in any case, so they can be obtained later if desired). If the record corresponds to data that the application is interested in, the record may be asked to build the data. That data can then be used by the application. The application can stop/restart the notification or attach/detach any listeners at any time. d. Architecture In this section, we will describe the implementation of the concept of common access in the WDSS-II system. Because the system needs to provide access to multiple data sources, we do not insist on a particular data format or system architecture. Instead, our architecture and choice of formats are extremely open. In this section, we will describe the way two different sources are incorporated into the WDSS-II system: dual polarization radar algorithms and ORPG base data and algorithms. i. Polarimetric Radar Products A Hydrometeor Classification Algorithm (HCA, Zrnic et al. (2001)) has been developed that currently runs on data from the Cimarron polarimetric radar located in Central Oklahoma. By applying weighting functions to the polarimetric fields (reflectivity, differential reflectivity, specific differential phase, and correlation coefficient), the HCA determines the dominant precipitation type for all regions of a storm. In the fall of 2001, plans call for data from the Cimarron radar to be replaced with data from a WSR-88D radar with a polarimetric upgrade. These products will then be supplied to the Norman NWS Office as part of the operational phase of the Joint Polarization Experiment, which will serve to demonstrate the operational utility of a polarimetric WSR-88D radar. Since the products from the experiment will be displayed within the context of other weather products at the forecast office, the HCA has to be integrated into the WDSS-II system. The HCA runs on the raw data stream from the polarimetric radar, producing outputs of aggregated radials, KDP, hydrometeor classifier outputs, etc. All these products are written out in the NetCDF format. After a product is written out, an XML record containing the selection criteria, the time of product and means of accessing the data is written into a linear buffer. The linear buffer provides for inter-process communication (Jain et al. 1998). The in-memory database resident in interested processes is notified about additions to the linear buffer and thus, the record gets added to the database. This causes any listeners attached by the applications to be notified of the new record. The architecture of the dual polarization system is shown in Figure 1a. ii. ORPG base data and algorithms The ORPG (Jain et al. 1998) stores its data in the Interface Control Document (ICD) format. The ICD format is not an extensible, self-describing format. Hence, the means of building the object has to specify a lot more information. For example, the means of building a base reflectivity product on the ORPG has to be specified by the following strings: the storage mechanism ( LinearBuffer ), the ICD product code ( 19 ), the name of the product ( Reflectivity ) and the message number of the product in the storage device. This information comes from the product database that is part of the ORPG sys- 8

9 (a) Dual Polarization Source (b) WDSS-II client with ORPG Source Figure 1: (a)the architecture of dual-polarization algorithm output notification, access and build from a WDSS-II client. Compare this with (b) the architecture of similar notification, access and build of ORPG products from a WDSS-II client. Notice that the client structure remains the same although the source architecture changes. tem. This product database may be used as the input for the in-memory WDSS-II database. Every row in the product database table is converted into the type of record (with builder parameters, selection criteria and times) of the type required. The product database can be registered with for notification. Hence additions to the product database show up in the in-memory database which in turn show up in the listeners attached by the application. The architecture of the ORPG to WDSS-II interface is shown in Figure 1b. iii. Database Locators Regardless of the type of in-memory database record built and the architecture of the system, the algorithms and displays see a uniform interface: 1. They obtain an index by specifying a uniform locator. An example locator of the dual polarization index is:xmllb:mach:/data/cimmaron.lb while that of an ORPG index is: orpg:mach:database:23453 The formats of the locators are different, but since they are all just character strings, algorithms can be switched from one source to another by providing a different argument at startup. 2. Having obtained the in-memory index, they can do navigation and search. They can also attach listeners to the index. 3. The listeners receive the same events whenever new data is available. 4. The records themselves share a common interface, providing access to selection criteria and product time. 5. Given any record, the interface to build the data is the same, regardless of what needs to be accomplished under the hood. 9

10 4. Algorithms The multiple source system should make it easy for algorithms to access and build data. It should also provide a common means of notification of the data. The goal is for the algorithm to be developed independent of the system architecture by providing a different database locator string, the algorithm would be working within a completely new system. a. Input and Output It is preferable that the input and output of the algorithm be in standard formats. We recommend NetCDF for large array products but XML for tabular data. Both these formats are self-explanatory, extensible and network-independent (i.e., can be read on machines of all byte orders). There are several advantages to using XML instead of NetCDF for tabular data. Most importantly, the textual nature of XML makes it easy for algorithm developers to analyze the outputs of their algorithm and to search through it. XML data are heirarchically arranged, making it easy to build parts of a table at different times. Commercial and free software exists, off the shelf, to read and manipulate XML data. XML data parsing is an active area of study and it is to be expected that future software will become more efficient. There are two drawbacks to using XML: a. the sizes of the files can be much larger than binary data b. the hierarchical structure of XML makes it neccessary for the data to be parsed, thus slowing down the read process. For tabular data with less than about a thousand cells, the disadvantages are outweighed by the advantages. For larger products, including arrayed products, NetCDF is recommended. The algorithm is completely independent of the data source and the format of the input data since it accesses and builds the data through the database record. It might be advantageous for the algorithm to be similarly independent of the format of its output. It is possible to build a similar method of transparency to output format, so that the algorithm outputs get written in the ouput format prefered by the data source. We have not implemented this, however, because the formats for different data sources are not obvious. This is particularly true if the algorithm in question has outputs of a form that is not envisioned in the design of the data source. Weather information systems should be designed around extensible data formats, but that is not the case right now. Hence, algorithms in WDSS-II are independent of the input data, but simply have to choose a data format to write out in. The WDSS-II system provides ways for algorithms to write out their outputs (and intermediate products, if needed) in either NetCDF or XML. b. Common computations In addition to providing ways for algorithms to write out their outputs in NetCDF and XML, WDSS-II also provides many common calculations. We have already talked about the extensive unit conversion library provided. Geographical navigation (computing with azimuths, ranges, latitudes, longitudes, distances between points, etc.) and time calculations (time differences, conversion between local time and UTC, etc.) are probably the most commonly used. We also provide ways for algorithms to com- 10

11 Figure 2: The design of a typical WDSS-II algorithm. The top part of the figure is the algorithm acting as a client of notification, access and build of the input data streams. The lower parts of the figure is the algorithm itself acting as a data source. Computing the various outputs from the inputs is, of course, the scientific portion of the algorithm. pute the time taken by different modules, a linear-congruent random number generator (Sedgewick 1990), different compression algorithms that can be used to compress algorithm outputs and XML parser interfaces for algorithm configuration.. The architecture of a typical WDSS-II algorithm is shown in Figure Visualization The visualization tool of the WDSS-II system is, for all intents and purposes, just another algorithm. This algorithm is interested in multiple sources, handles time synchronization and is interested in multiple products image products, 3D products and tabular products. However, which products are required is driven interactively by the user. Hence, the visualization tool exercises the limits of the WDSS-II system. The default sources that the display handles are specified through an environment variable at startup. New sources can be added or removed dynamically. These sources are specified by the locator of the in-memory database associated with the source. Products are identified on the basis of the in-memory database from which they are drawn and the record s latter two selection criteria, the type of product and sub-type, if any. Thus, KFWS Reflectivity 0.5 would be a complete description of a product. The time of each product (again obtained from the database record) is used to differentiate periodically occuring products. Because every product is tied to the database record from which it is derived, and because every database record is, in turn, tied to the in-memory database, it is possible to navigate (go up and down in subtype and forward and backward in time) from a given product. 11

12 Figure 3: Time synchronization between products from different sources is shown. The velocity product (top) is from the KTLX radar while the reflectivity product (bottom) is from the KFWS radar. The KTLX product is from 20:38:54 (highlighted in yellow) while the KFWS product is from 20:44:50. The simulation time (shown in white text on the lower right hand corner of the product window) is set to that of the KFWS product. The KTLX product is visible as it is the most current available as of a. Time synchronization Different data sources produce products at different times. Synchronization of data from these sources is done through the concept of a simulation time. Whether or not a product is displayed depends on the simulation time and its relation to the product time and the expiry interval of the product. The product is displayed only if the product time is earlier than the simulation time by less than the expiry interval and if there is no other product later than this one whose time is earlier than the simulation time. This ensures that we are displaying the most current product as of the simulation time. The simulation time itself can be set in two ways. If the user manually selects a product (either from the in-memory database or by navigating from an earlier loaded product), the simulation time is set to be equal to that product s time. Because we show all products that are current as of this time, data from other sources will also be shown. Figure 3 shows the result of loading a product corresponding to a radar in 12

13 Fort Worth, TX, when there is already a product correpsonding to a radar in Norman, OK. The simulation time has been set to that of the Fort Worth product, and the most current Norman product at that time is displayed. This ensures that the time integrity is maintained. The use of the expiry interval ensures that Norman products older than 15 minutes will not be displayed. The second way that the simulation time can be set is by an auto-update. When a product that is being viewed is received, and if the user has requested automatic updating, the product is built and loaded up. If the time of this product is later than the simulation time, the simulation time is moved ahead. Products from other sources, as long as they are not expired, will continue to display. If the simulation time is already ahead of this product s time, this product replaces the product of this time that had been displayed so far. In weather information systems, animation has usually been viewed in terms of the number of frames in the loop (citefixme). In a multiple source, multiple product system, the question arises: number of frames of what? After all, six frames of a loop of one radar could correspond to 10 frames of a loop of another radar. A loop s length in the WDSS-II display is specified in terms of minutes. The simulation time refered to in the above discussion is the end of the simulation interval. The beginning of the simulation interval is set based on the loop length. The simulation time is then steadily advanced in this interval, displaying the most current products as of a particular time. If an automatically updating product is received, the simulation interval is advanced if necessary, keeping the length of the loop constant. b. Oct-tree visualization Every product that is ultimately displayed is associated with a drawable object (henceforth, a drawable), and each drawable has a bounding sphere. For example, a conical radar elevation scan is associated with a sphere that is centered at the radar and with a radius equal to the length of a beam. The volume in which everything is visualized corresponds to a cube whose side length is twice the diameter of the earth and whose center is the center of the earth. This cube is subdivided hierarchically as an oct-tree. Each drawable is placed into the oct-tree node that completely encloses its bounding sphere. The check to determine visibility is made on a per-node basis, rather than on a per-drawable basis. If a large node is completely invisible, then none of its subnodes need be checked for visibility. Also, a node that is too small to be seen from the current viewpoint will not be drawn, and its subnodes will not be checked for angular size. In the case of a product (like a track or an icon) that is associated with a single storm event, the radius of the bounding sphere is a function of both the severity and the extent. Thus, as the user s eye position moves away from the surface of the earth, the weaker storms are culled first. c. Image products Two-dimensional data are efficiently represented in the virtual world by way of the texture map. On each of our supported hardware platforms, texture-map rendering calculations are performed in the graphics hardware, and so the visual simulation can 13

14 maintain interactive frame rates even with rich and detailed graphical objects. A texture map, which has red (R), green (G), blue (B), and transparency (A for alpha ) components, is obtained via a user-defined color map, which is used to transform a gridded 2D scalar field of data into a 2D RGBA texture. This texture is then associated with the appropriate geometry. For example, a single radar scan, which may produce a scalar field representing reflectivity, is transformed into an RGBA texture with the aid of the standard reflectivity color map, and then the texture is wrapped around a cone with the appropriate elevation angle. An efficiently drawable textured surface is one that has a regular shape (such as the rectangle, parallelogram, sphere, cone, etc.). Typically, in weather data, it is possible that data for a location on the grid is missing or unavailable. In such situations, the color corresponding to that location is set to be transparent. By making it transparent, it is possible to overlay a second source and thus obtain data for that location. The latitude-longitude grid data are already uniform in spherical coordinates. Thus, the gridded data are mapped into one or more textures (depending on the maximum texture size supported by the hardware). These textures are then wrapped around the surface of the earth. Thus, the latitude-longitude gridded data are mapped into surface textures. The Cartesian grid data are also uniform, but in a Cartesian coordinate system. The data are mapped into one or more textures which are visualized as parallelograms where the axes of the parallelograms are determined in a coordinate system centered at the earth s center. The elevation data are not uniform, since the radial data are collected separately. The radar rotates between radials and due to mechanical errors, the actual azimuthal angle of the radar can vary. We implemented three different visualization techniques, each of which presents a different trade-off between speed and accuracy: 1. Smearing with an interval. The radials within an elevation scan are assumed to be uniformly distributed within the range of azimuths given by the first and last radial of the elevation scan. This assumption holds true, by and large for the elevation scans from operation Weather Service Radar. The data, assumed to be uniform, are mapped onto a texture that is then wrapped around the surface of a cone. 2. Smearing piecewise. In some situations, the assumption that the radials are uniformly distributed fails. This could happen if some radials are skipped (due to a load distribution problem, for example) or if radials are duplicated, which might happen on experimental radar. Hence, the set of radials in an elevation scan are examined for ranges of azimuths that are uniformly distributed. Each of these ranges are visualized as in the first method, as textures wrapped around the surfaces of cones. 3. Accurate rendering. The conical elevation scan is divided into 360x11 azimuthal bins and the radial corresponding to each bin is determined. Sixteen textures of 256 bins each are constructed, and data from the elevation scan copied onto each bin depending on the radial associated with it. If there is no radial associated with a bin, such as if a radial is missing, the texture is filled with transparent 14

15 colors. These sixteen textures are then wrapped around the surface of a cone and visualized. Although this is the most accurate visualization (within 0.09 degrees of the actual location), it is also the most time and memory intensive. Since the second method works for most situations, a fourth drawing option is presented to the user that of automatically switching from the piecewise method to the accurate rendering if the data are found to be badly distributed. This is the the default visualization, but users interested in maximizing either speed or accuracy are allowed to choose the method of visualization. d. Non-image products Non-image products from algorithms include contour data and tabular data. Contour data are stored as a set of earth-locations along the contour. Tabular data are considered a set of events, with each event having a number of attributes. Thus, a supercell algorithm that identifies supercells, will organize its output such that each supercell is a row in the table. For each row, the algorithm provides an extensive set of attributes, such as [Jason, fill in here]. The algorithm writer then writes the data out as XML files and updates the database of products with the meta-information. For each attribute of a supercell, there is a column in the table. The column may be color-coded according to the actual value of any column in the same table. Each row may be assigned a separate location by providing columns named Latitude, Longitude and/or Height. The algorithm writer typically creates a configuration file for the algorithm output specifying the columns in the table, the units of display and color-coding strategies. It is noteworthy that both image and non-image outputs of algorithms can be visualized along side base data with no source-code modifications. e. GIS data Geographic information data such as state and county lines, power plants or rivers that are present in ArcInfo s Shapefile format may be visualized alongside meteorological data. Terrain information from the USGS SDTS format can also be visualized. GIS objects are treated simply as data with an infinite expiry interval. f. 3D products Three-dimensional products, such as radar virtual volumes and sigma-layered reflectivity volumes Zhang et al. (2000) are visualized by creating a number of low-resolution cross-sections along three perpendicular planes and switching between different such sections based on the user s angle of view. When the user stops interacting with the volume, a high-resolution cross-section is created with the actual angle of view. g. Multiple sources While users may want to display multiple products, there are some products that are exclusive. Analysis of user preferences yielded the following rules: 15

16 Figure 4: Visualization of the volume swept by the radar at KTLX on May 3,

17 Figure 5: Visualization of data gridded from reflectivity data collected at KTLX on May 3, Because the data covers a larger domain, the slicing is done at 4km intervals. This explains the coarseness of the visualization. 17

18 Figure 6: Crosssection drawn through the volume of data shown in Figure 4. 18

19 Figure 7: Crosssection drawn through the volume of data shown in Figure 5. 19

20 1. Different image-type products may be visible at the same time. For example, KOUN Velocity and KFWS Reflectivity may be visible simulatenously. 2. Only one image-type product from a source should be visible at a time. Thus, KFWS VIL should not be displayed if KFWS Reflectivity is being displayed. 3. Non-image type products are not subject to these restrictions. The fact that storm cell overlays are being shown should not restrict the visibility of mesocyclone overlays or storm boundary overlays. These rules, however, are simplistic. Users requirements are actually more involved that this. For example, users wanted to be able to associate certain overlays with certain products (storm cell overlays with reflectivity, mesocyclone overlays with velocity). The distinction based on source is not enough. The problem goes back to the 4D referencing of database records. The data source does not capture visibility concerns. For example, when reflectivity data from Norman, OK is shown, it obscures reflectivity data from Lawton, OK (see Figure FIXME) at places where the beam from Norman is higher than the beam from Lawton. If this effect is not what is required, the only solution is to hide the product whose beams are higher. A better solution would be to divide the covered area into smaller regions and to provide a user-friendly way of choosing the data source for that region. It is not clear how to implement this. 6. Summary We have described the concepts and lessons learned from the implementation of the Warning Decision Support System Integrated Information (WDSS-II). The WDSS-II supports multiple data sources and multiple product types simultaneously. To make this possible, we had to treat all data rigorously in a 4D space. We also showed the considerations that make possible the access, building and notification of data transparent, i.e. independent of the actual source of data. As examples, we showed how two completely different data sources, dual polarization radar and the Open Radar Products Generator (ORPG), were integrated into the WDSS-II. Finally, we discussed the concepts that underly the implementation of a rigorous 4D, multisource display system for weather information. Acknowledgement The work detailed in this paper was supported with funds from the National Science Foundation, NEXRAD Product Improvement group at the National Oceanic and Atmospheric Administration, the Federal Aviation Authority and the National Severe Storms Laboratory (NSSL). The authors would like to thank Don Bailor, Karen Cooper, Charles Kerr, John Krause, Jason Lynn, Lulin Song, Tad Thurston and Lingyan Xin of the University of Oklahoma and Kurt Hondl of NSSL for many contributions to the design and implementation of the WDSS-II system. 20

21 References Bray, T., J. Paoli, C. M. Sperberg-McQueen, and E. Maler: 2000, Extensible markup language (XML). Technical report, World Wide Web Consortium, available at Brubaker, T., R. Huonder, and T. Von der Haar: 1991, Octree encoding for visualization of atmospheric conditions and processes. Journal of Atmospheric and Oceanic Technology, 8, Hembree, L., S. Brand, W. Mayse, M. Cianciolo, and B. Soderberg: 1997, Incorporation of a cloud simulation into a flight mission rehearsal system: Prototype demonstration. Bulletin of the American Meteorological Society, 78, Hondl, K., V. Lakshmanan, V. Ganti, J. Brogden, and R. Cole: 1999, Status update of the wsr-88d open system principal user processor (OPUP) development project. 15th International Conference on Interactive Information and Processing Systems (IIPS) for Meteorology, Oceanography, and Hydrology, Amer. Meteor. Soc., Dallas, TX, Jain, M., Z. Jing, H. Burcham, A. Dodson, E. Forren, J. Horn, D. Priegnitz, S. Smith, and J. Thompson: 1998, Software development of the nexrad open systems radar products generator (ORPG). 14th Intnl. Conf. IIPS, Amer. Meteor. Soc., Phoenix, AZ, James, C., S. Brodzik, H. Edmon, R. H. Jr., and S. Yuter: 2000, Radar data processing and visualization over complex terrain. Weather and Forecasting, 15, Jenter, H. L. and R. P. Signell: 1992, NetCDF: A freely-available software-solution to data-access problems for numerical modelers. Proceedings of the American Society of Civil Engineers Conference on Estuarine and Coastal Modeling, Tampa, Florida. Lakshmanan, V. and A. Witt: 1997, A fuzzy logic approach to detecting severe updrafts. AI Appl., 11, Sanger, S. S., R. Steadham, J. Jarboe, R. Schlegal, and A. Sellakannu: 1995, Human factor contributions to the evolution of an interactive doppler radar and weather detection algorithm display system. 11th Conf. Interactive Information and Processing Systems for Meteorol. Ocean. And Hydrol., American Meteorological Society, Boston, 1 6. Sedgewick, R.: 1990, Algorithms in C++. Addison Wesley, 656 pp. Smith, T.: 1995, Visualization of WSR-88D data in 3d using application visualization software. 14th Conf. on Weather Forecasting. Stumpf, G., C. Marzban, and E. Rasmussen: 1995, The new NSSL mesocyclone detection algorithm: a paradigm shift in the understanding of storm-scale circulation detection. 27th Conference on Radar Meteorology. 21

22 Zhang, J., J. Gourley, K. Howard, and B. Maddox: 2000, Three-dimensional multiple radar reflectivity mosaic. The Second Southwest Weather Symposium, Tucson, AZ. Zrnic, D., A. Ryzhkov, J. Straka, Y. Liu, and J. Vivekanandan: 2001, Testing a procedure for automatic classification of hydrometeor types. J. Atmos. Oceanic Tech., 18,