APPLICATION NOTE Integrated Analytics Author: Kathryn Kuhr PerkinElmer, Inc. Shelton, CT PerkinElmer is the exclusive global distributor of the TIBCO Spotfire platform for certain scientific and clinical R&D applications. Management and Visualization of Pesticide Residue Data on Agricultural Commodities Using TIBCO Spotfire Software Introduction Pesticides have long been used to control damaging pests and protect the health of crops throughout their journey from farm to table. This practice serves the interest of parties on both ends of the transaction; growers are rewarded with a more bountiful harvest and consumers are provided with a wider variety of goods in the supermarket. But what effects do pesticides have on the quality of these products? To assist in the search for that answer, the United States Department of Agriculture (USDA) Agricultural Marketing Service (AMS) tests numerous food samples each year as a part of the Pesticide Data Program (PDP) to ensure the safety of the nation s food supply. These results are used by the United States Environmental Protection Agency (EPA) when assessing risk through dietary exposure 1. According to the 2012 and 2013 annual reports published by the AMS, in both years over 99% of the samples tested had residue levels below the tolerances set by the EPA 1,2. This analysis used TIBCO Spotfire to interactively explore and compare the residue results gathered over both these years. In this application note, we explore how using advanced data visualization techniques with pesticide residue data can help scientists quickly and efficiently assess and monitor food safety or quality concerns.
Experimental The data for this analysis was retrieved from the USDA AMS. Each year, as a part of the Pesticide Data Program, the USDA tests various food types for pesticide residue and the results are published in an annual report. The tested food types are selected with prominence given to commodities frequently consumed by infants and young children. Published reports exist on the USDA website for the years 1992-2013. This investigation used only data from the years 2012 and 2013. In 2012, 12,546 different samples were tested to provide a total of 2,152,531 results 1. In 2013, 10,104 different samples were tested to provide a total of 2,023,087 results 2. After downloading data for each year, the sample and result text files were uploaded directly into TIBCO Spotfire. Since the text files do not contain a row of data for the column names, these were edited within the software after the import process. The appropriate column names can be found in the PDP DataDictionary XXXX file within the ZIP folder, where XXXX represents the desired year. In both the sample and result tables, the row contents are abbreviated. To better understand the data, calculated columns were added to each table displaying the data definitions. These calculated columns were written in the form of a case statement as shown below. Case [Column Name] when Arg 1 then Arg 2 when Arg X then Arg X+1 else Error end The descriptions of the codes used in each data table were found in the PDP ReferenceTables XXXX file within the ZIP folder. In the PDP12Samples and PDP13Samples tables, case statements were written to display the full commodity name, commodity marketing claim, collection/distribution facility type and commodity type. Figures 1A and 1B display the same five rows of data before and after the introduction of the calculated column creation respectively. For the PDP12Results and PDP13Results tables, similar case statements were written to display the full commodity name, commodity type, lab name and test class of the compound detected. Results The sample and result files contained data pertaining to food products as well as ground, untreated and finished water; this analysis focused solely on food. Two bar charts were first built to analyze which food types contained the greatest average concentration of pesticide residue. In both 2012 and 2013, only 22 different food products were tested. These selections varied slightly year over year. Despite this difference, plums were found to contain the highest concentration of pesticide residue; this value increased from 0.310 in 2012 to 0.429 in 2013. Table 1 summarizes these results for all commodities and lists the number of samples tested. Figures 2 and 3 visualize this residue concentration data for years 2012 and 2013 respectively. While this analysis does look at the concentration of pesticide residue, it does not compare these findings to EPA regulated tolerances. According to the 2012 annual report, only 0.53% of samples were found to contain residue greater than the tolerance limit 1. Again in 2013, only 0.23% of all samples tested had residue values at or above their tolerance limits 2. Uniquely, of all samples investigated during these two years, in 2013 there was no pesticide residue found in either baby food made from peas or dairy-based infant formula. Figure 2. 2012 average residue concentration vs. commodity name. a b Figure 1a. Five rows of data from the PDP13Samples data table. Figure 1b. Data from Figure 1a after the implementation of calculated columns as case statements. Figure 3. 2013 average residue concentration vs. commodity name. 2
Table 1. Average concentration of pesticide residue and number of samples for each commodity tested. Commodity Name Average Concentration 2012 Results 2013 Results Number of Samples Tested Average Concentration Number of Samples Tested Apple Juice 0.032 396 0.037 379 Avocado 0.139 372 - - Baby Food, Applesauce 0.020 396 0.016 379 Baby Food, Carrots 0.008 792 - - Baby Food, Peaches 0.008 777 - - Baby Food, Peas 0.015 395 0.000 378 Bananas 0.026 559 0.027 708 Broccoli - - 0.058 708 Butter 0.004 792 0.004 756 Cantaloupe 0.031 372 - - Carrots - - 0.022 712 Cauliflower 0.007 737 0.009 532 Celery - - 0.032 708 Cherry Tomatoes 0.042 744 - - Fish, Salmon - - 0.014 352 Grape Juice - - 0.018 176 Green Beans - - 0.118 378 Infant Formula, Dairy - - 0.000 177 Infant Formula, Soy-based - - 0.003 179 Mushrooms 0.268 2 744 0.258 3 532 Nectarines - - 0.262 2 543 Onion 0.011 558 - - Orange Juice 0.013 330 - - Peaches - - 0.185 285 Papaya 0.065 366 - - Plums 0.310 1 697 0.429 1 507 Raspberries - - 0.089 652 Raspberries, Frozen - - 0.187 53 Snap Peas 0.056 743 - - Summer Squash 0.040 186 0.040 709 Sweet Bell Peppers 0.059 186 - - Tangerines 0.158 3 709 - - Wheat grain 0.004 300 - - Winter Squash 0.027 742 0.030 187 Data is split between years 2012 and 2013. 1 Indicates the commodity with the highest average pesticide residue concentration. 3 Indicates the commodity with the third highest average pesticide residue concentration. 2 Indicates the commodity with the second highest average pesticide residue concentration Samples tested in 2012 comprised four main commodity groups: fresh, raw grain, liquid ready-to-serve and puree. Each food type tested fell under only one of these categories. In 2013, the samples tested comprised 6 main commodity groups: fresh, frozen, liquid concentrate, liquid ready-to-serve, powdered, and puree. During 2013, only fish, infant formula, and raspberries were tested from multiple commodity groups. The largest difference in detected concentration resulted from fresh vs. frozen raspberries. These results are summarized in Table 2. Figure 4 displays a bar chart of the sample counts as well as a pie chart showing the percentage of samples per commodity type for both 2012 and 2013 data. Table 2. Average concentration of pesticide residue found on commodities tested in 2013 for various commodity types. Commodity Name Fresh Frozen Commodity Type Concentrate Liquid Ready-to-Serve Powdered Fish, Salmon 0.018 0.013 - - - Dairy - - 0.000 0.000 0.000 Infant Formula Soy-based - - 0.000 0.003 0.000 Raspberries 0.089 0.187 - - - 3
Figure 4. 2012 percentage of samples by commodity type (top left), 2012 commodity sample count (bottom left), 2013 percentage of samples by commodity type (top right), 2013 commodity sample count (bottom right). 4 Creating what is called a Details Visualization allows for the specific drill-down of data from an existing chart. Demonstrated in Figure 5A, a Details Visualization bar chart was created to display the average concentration of pesticide residue per compound class. In order for data to populate this new chart, all bars were selected in the parent visualization. In Figure 5B, only the bar representing plums was selected, thus changing the data in the chart below. This analysis was also carried out for plums for 2012. It should be noted that although the color for the maximum value is presented as red, this is not an indication that the value is over the regulated limit for that residue in food. By analyzing the differences between these two figures, it becomes apparent which classes of pesticides are found more frequently on different commodities. The results of this analysis are summarized in the cross table in Figure 6. Cross tables created in TIBCO Spotfire are analogous to pivot tables in Microsoft Excel. While this analysis does not drill down to connect each pesticide with its organic approval or tolerance, one approach is taken to demonstrate how these samples can be grouped automatically by their categorical similarities. This may be used to uncover and study relations between commodities when food safety and quality is a concern. A treemap was developed to hierarchically organize the samples by marketing claim, commodity name, and distribution state. The purpose of this visualization was to determine if any of the samples which tested positive for pesticide residue, organic and non-organic, originated from a common location. The column distribution state was chosen because the columns representing grow state and pack state consisted largely of empty values. Although some cells do remain empty under distribution state, this did not impact the value of the treemap. The color by rule of the treemap was based on the average concentration of residue detected and the sizing of the cells was based on the number of samples tested. As seen in Figure 7, it becomes apparent that many of the samples were distributed from California. According to the California Department of Food and Agriculture s production statistics, in 2013 the state of California exported 14.7% of the total U.S. agricultural market s food products, 1.6% more than in 2012 3. While from this visualization it may appear that pesticide residue is detected more frequently on commodities distributed from California, the overall percentage of food they supply to consumers puts this into perspective. The functionality of TIBCO Spotfire runs deeper than simply viewing what we already know exists; it s the discovery of new insights and the simplicity from which these become available that holds the true power of data visualization. Complex analyses, or those which contain many repetitive comparisons, can be transformed into automatic files. As new results are acquired and added to the data table source file (.xls,.xlsx,.txt,.csv, etc.), TIBCO Spotfire may be refreshed with one click to incorporate this additional information into the appropriate visualizations, saving countless amounts of time. Through the establishment of color by rules unique filters to different tabs of the analysis, data can be broken down into its exclusive components- pinpointing answers to questions once unknown and bringing data to the forefront of the investigation into food safety and quality.
a b Figure 5a. Visualization from Figure 3 (top), 2013 average concentration vs. test (compound) class (bottom). Figure 5b. Visualization from Figure 3 (top) marked to display only the average concentration per test class for plums (bottom). 5
be used to compare pesticide concentrations year-over-year. As the PDP outlines, the analysis of pesticide residues in food can provide government agencies with the ability to assess exposure levels and make informed decisions about the safety of food products and the use of pesticides while managing the global marketing of U.S. grown agricultural products 4. Utilizing advanced data visualization to accomplish these goals gives users an interactive and holistic approach to data analysis unavailable with traditional methods. Figure 6. Average concentration by compound class, plums only, for 2012 and 2013. Empty values where no pesticides of a particular class were detected are shaded gray. Conclusion This investigation illustrates the value data visualization can bring to the analysis of pesticide residues in food. With its ability to handle large amounts of data, TIBCO Spotfire can References 1. Pesticide Data Program, Annual Summary, Calendar Year 2012, United States Department of Agriculture Agricultural Marketing Service, Feb 2014, Web, 31 Aug 2015. 2. Pesticide Data Progra m, Annual Summary, Calendar Year 2013, United States Department of Agriculture Agricultural Marketing Service, Dec 2014, Web, 31 Aug 2015. 3. California Agricultural Production Statistics, California Department of Food and Agriculture, 2015, Web, 28 Aug 2015. 4. Pesticide Data Program, United States Department of Agriculture Agricultural Marketing Service, n.d., Web, 28 Aug 2015. Figure 7. Treemap arranged hierarchically by Claim, Commodity, and Distribution State for 2012 (top) and 2013 (bottom). PerkinElmer, Inc. 940 Winter Street Waltham, MA 02451 USA P: (800) 762-4000 or (+1) 203-925-4602 www.perkinelmer.com For a complete listing of our global offices, visit www.perkinelmer.com/contactus Copyright 2015, PerkinElmer, Inc. All rights reserved. PerkinElmer is a registered trademark of PerkinElmer, Inc. All other trademarks are the property of their respective owners. 012430_01 PKI