Data Visualization Tool (DaViTo) Unclassified//Presentation has been reviewed and is approved for public release

Transcription

1 Data Visualization Tool (DaViTo)

2 Problem Statement Problem: Contemporary operational environments create vast amounts of disparate and unstructured data; there is a need for analytical tools and methods designed to extract meaningful information and insights from this data. 2

3 Background of DaViTo January 2010 article by MG Flynn, et. al. indicates that exploration and synthesis of white data could actually answer some of the most important questions surrounding operations in Afghanistan and future battlefields. February 2010 ERDC Human Social Culture Behavior GIS workshop identified exploitation of geospatial statistics as a gap in existing operational analysis. Assessments in OEF continue to be a significant multi-echelon requirement. Tools and techniques used by many analysts create unnecessary bounds on creativity and analytical capability. 3

4 Exploratory Data Analysis Exploratory data analysis of 311 phone complaints in New York shows that from 9 am to 12 pm, when the sun is up, the most common complaint is streetlights. The greatest value of a picture is when it forces us to notice what we never expected to see. - John Tukey. Exploratory Data Analysis,

5 Purpose and Agenda Purpose: To provide an overview of the Data Visualization Tool (DaViTo) project and its current status. Agenda Methodology/Approach. Development Process. DaViTo Capabilities. Proposed Enhancements and Research. 5

6 Constraints, Limitations and Assumptions Constraints The project shall utilize only open source software to perform required functions. ArcGIS and some statistical packages are prohibitively expensive for some users. Current development cycle will be completed by mid July Limitations Development requires a user base willing to provide necessary funding for enhancements. Small development team with many current projects. Assumptions Users have Java installed on the machine running DaViTo. Analysts have some basic knowledge of the data set used and can perform basic modifications to format if necessary. 6

7 DaViTo Vision Statement DaViTo provides visual analytic products that enable decision makers to rapidly glean insights from complex data, including geospatial and temporal data, by leveraging open source software maintained and enhanced by an active user/contributor base, ensuring a more informed and effective force. DaViTo must be Flexible. - Fixed data set isn t needed. Powerful. - DaViTo must be better than its parts alone. Simple. - Intuitive computer interface. - Analysis conducted with very few clicks. Available. - Capable of running on government computers with network and program installation restrictions. 7

8 Analysis Process WITS Trained User DaViTo Software Raw Output Analyst Refined Output Input one or more data set into the system. Some data requires analyst manipulation or processing prior to ingestion by the system. Red activity (CIDNE, WITS, INDURE). Blue activity (TiGRNET, BFT). Polling data (ACK-SYS). Economic indicators (UN WFP, World Bank, USAID). Socio-Political indicators (World Bank, NPS DA, CSP). DaViTo software doesn t require specific data sets, instead it ingests available data and provides visualizations to recognize patterns and unexpected insights. CIDNE: Combined Information Data Network Exchange WITS: Worldwide Incident Tracking System ACK-SYS: Afghanistan Consolidated Knowledge System NPS DA: Defense Analysis Department TiGRNET: Tactical Ground Reporting Network BFT: Blue Force Tracker UN WFP: UN World Food Programme CSP: Center for Systemic Peace 8

9 Development Process Define Requirements Solicit Feedback from Users Identify Potential Solutions Test Implement Solutions 9

10 Major Phases of DaViTo Initial implementation Integrate with R Integrate with OpenMap Follow-on work in progress Value system hierarchy capability Trade-space analysis Future work Increase user base Increase contributor base Enhance functionality 10

11 DaViTo Visualizations (1 of 2) A dot indicates a data point of activity, district centroid for polling data, or food price information. Different shapes can be drawn or shape files loaded for predefined areas (e.g. RC s, provinces, or unit AO). 11

12 DaViTo Visualizations (2 of 2) The user can zoom into an area, break the region into a grid, and normalize the values across charts to identify high incident concentrations. Raw data can be pulled from the map and placed into other software (e.g. JMP, Excel, R). 12

13 DaViTo Interface 13

14 DaViTo Interface Controls (zoom,pan etc.) Data loading Shape files Chart controls Chart combination 14

15 Worldwide Incidents Tracking System (WITS) Data DaViTo chart displaying WITS data from 1 January 2006 to 31 December 2010 grouped by year. 15

16 WITS Data DaViTo chart displaying the same WITS data from 1 January 2006 to 31 December 2010 grouped by month. 16

17 Shape File Generated Plots DaViTo automatically generates charts for any number of regions defined in a shape file. This enables repeatable analysis. 17

18 Incident Deviations # of deviations from annual mean. Mean over entire data set. Annual mean. DaViTo generates control charts allowing analysts to rapidly determine when changes in data are significant. 18

19 Incident Deviations # of events is beyond standard variation expected. # of events is increasing, and less predictable. DaViTo generates control charts allowing analysts to rapidly determine when changes in data are significant. 19

20 Plot for Continuous Data DaViTo generates box and whiskers plots for continuous data, showing not just analysis of the median, but variability in the data as well. 20

21 Incident Wheel The incident wheel functionality makes finding patterns in enemy or friendly activity a straightforward process. 21

22 Heat Map by Afghan Province DaViTo users can also analyze activity within an AOR by Afghan Province. 22

23 Value System Hierarchy Methodology Stability Lines of Effort Medical Security Governance Measures Life expectancy Infant mortality Infectious disease reports % Patrols conducted by local security force Trust in government % Population voting Sampling of attributes that determine the overall status based on user defined weighting scheme. The user created tree structure can be saved and used again on updated or new data or to allow for analysis to be transportable by ing the xml file and data for use in DaViTo. 23

24 Value System Hierarchy Tree-Style Output 24

25 Value System Hierarchy Map-Style Output Output for Sub-Saharan Africa. 25

26 Proposed Enhancements and Research Increased R-Project functionality. Additional premade functions based on feedback from users. Increased user interface functionality to facilitate new premade functions. Increased system stability. Shift to a fully open-source development. Test usability and understanding of charts. Determine how to display information to enable the easiest understanding of the data. Inclusion and automation of a point density heatmapping methodology. Web version in JavaScript utilizing jstat and Google Maps. 26

27 Software Distribution Asymmetric Warfare Group. TRAC. Data cell forward Afghanistan (CODDA). Fort Lee, Virginia. White Sands Missile Range, New Mexico. Fort Leavenworth, Kansas ORSA-MAC. National Defense University. US Corps of Engineers Research and Development Center. SAIC UK Ministry of Defense. USSOCOM. MCCDC. MICoE. ORSA Community. MCoE. UK Defense Science and Technology Laboratory. Raytheon. Center for Army Analysis. University of Virginia. U.S. Military Academy at West Point. Naval Postgraduate School. * Development contributors. 27

28 Interested?