Big Data: Challenges in Agriculture. Big Data Summit, November 2014 Moorea Brega: Agronomic Modeling Lead The Climate Corporation

Transcription

1 Big Data: Challenges in Agriculture Big Data Summit, November 2014 Moorea Brega: Agronomic Modeling Lead The Climate Corporation

2 Outline THE AGRICULTURAL CHALLENGE DATA SCIENCE MEETS AGRICULTURE THE ROLE OF DATA SCIENCE

3 The Agricultural Challenge

4 The Agricultural Challenge

5 Forty decisions, Forty outcomes

6 Key Decisions Each season about 40 management decisions are made equipment selection results analysis seed selection crop selection fertility management water management planting logistics planting practices scouting inputs fertility harvest logistics harvest timing grain marketing

7 Sequential Multi-Arm Bandit Each season about 40 management decisions made many decisions one Bayesian network outcome

8 Forty outcomes A typical farmer will manage 40 seasons a typical farmer manages 40 seasons providing 40 outcomes

9 Data Science Meets Agriculture

10 The Next Revolution? INTENSIFY Apply breeding, fertilization to increase yields. BIOTECH Marker assisted selection, traits, chemistries, microbials. OPTIMIZE Apply data science to optimize management. GREEN REVOLUTION BIOTECH REVOLUTION GREEN DATA REVOLUTION

11 Data Available One Crop, One Season, One Country YIELD MONITOR DATA 14B OBSERVATIONS REMOTE SENSING DATA 260B OBSERVATIONS WEATHER DATA 20B OBSERVATIONS

12 Yield Modeling y = f (g, e, p) + ε yield genetics environment practices variability Yield is a function of genetics, the environment & farming practices

13 Yield Optimization OPTIMIZED YIELD Yield optimized for environment by optimization of genetics and management using predictive model. YIELD Yield optimized for environment by optimization of genetics and management traditional practices.

14 Challenges in Applying Data Science

15 Challenges Data Challenges Spatio-Temporal Data Heterogeneous Data Missing Data Learning Challenges Latent Features Curse of Dimensionality Multi-task Learning Noisy Data

16 Data Challenges Spatial Misalignment Multi-sensor Data high spatial resolution gridded data observed data 2005 Reanalysis Data coarse spatial resolution gridded data produced by deterministic weather models 1950 Gauge Data sparse spatially and temporally observed data 1870

17 Data Challenges Spatial and Temporal Misalignment Low resolution, higher temporal frequency High resolution, lower temporal frequency

18 Data Challenges Heterogeneous Data

19 Data Challenges Missing Data Yield data can be missing due to: pest/disease low yield due to other causes (heat, drought, frost, ponding) equipment malfunction data postprocessing by an outside entity

20 Data Challenges Noisy Data Noise can come from many sources: Clouds and atmospheric disturbance Equipment malfunction/equipment calibration issues Measurement error Human error Mislabeled data or data with no labels

21 Inherent Complexity Genetics, Environment, Practices Soil Processes Nutrient Processes Crop Processes YIELD Zea mays (corn)

22 The Role of Data Science

23 The Role of Data Science A Coherent View of the Field weather sensors remote sensors ground sensors Utilizing multiple sources of data before and during the growing season to provide growers with insights and recommendations.

24 The Role of Data Science Identifying Crop Stress Insights When is the crop under stress? Recommendations What actions can I take to correct this?

25 The Role of Data Science Nutrient Applications Insights How much nitrogen is available to my crop? Grower Practices weather data Recommendation How much fertilizer should I apply?

26 The Role of Data Science Optimize each decision for risk adjusted return Bayesian network step-wise optimization of conditional expected utility

27 The Next Revolution? INTENSIFY Apply breeding, fertilization to increase yields. BIOTECH Marker assisted selection, traits, chemistries, microbials. OPTIMIZE Apply data science to optimize management. GREEN REVOLUTION BIOTECH REVOLUTION GREEN DATA REVOLUTION

28 Questions?