Big Data Effects on Weather and Climate

Transcription

1 Big Data Effects on Weather and Climate Informal Discussions on The New Economics Nancy Grady, PhD, Technical Fellow, Data Science, SAIC David Green, PhD, Emerging Services, NWS Troy Anselmo, Senior Solution Architect, SAIC

2 What is Big Data Big Data: when the size and performance requirements for data management become significant design and decision factors Roger Magoulas and Ben Lorica, O Reilly Release 2.0 Issue 11, 2009 an astute observer can note the faint signals of the coming future, extrapolate from them, and begin to divine the business implications... Tim O Reilly, O Reilly Release 2.0 Issue 1, Weather is the original big data problem Bryson Koeler, CIO Weather Channel,

3 It s Not Just the Volume Powers of Ten Data is growing faster than Moore s Law on all dimensions Volume ever increasing challenge Velocity rapid response needs Variety combining data across domains Veracity quality and authoritative voice Value requires the other 4 V s Business benefits are frequently higher when addressing the variety of data than when addressing volume Mark Beyer and Doug Laney, Gartner, 2012 The Importance of Big Data: A Definition 3

4 Data Science is the Fourth Paradigm Experiment - to know the unknown Theory - to explain the known To predict the unknown Simulation - to predict the unknown when you can t do the experiment Data Science use the Variety of data to know the answers to unknown questions In The Fourth Paradigm: Data-Intensive Scientific Discovery, pioneering computer scientist Jim Gray refers to discovery based on data-intensive science 4

5 Science Now Takes a Community DOMAIN EXPERTISE RESEARCH DATA SCIENCE STATISTICS DATA MINING ANALYTIC SYSTEMS ALGORITHMS PROGRAMMING SKILLS 5

6 Big Data Equals Big Return on Investment While there is a lot of buzz about big data in the market, it isn t hype. Plenty of customers are seeing tangible ROI to address their big data challenges: Healthcare: 20% decrease in patient mortality by analyzing streaming patient data Telco: 92% decrease in processing time by analyzing networking and call data Utilities: 99% improved accuracy in placing power generation resources by analyzing 2.8 petabytes of untapped data 6

7 Big Data Technology and Economics Changes Scalable Cloud Computing On-demand, scalable, high availability Cost effective, pay as needed Shared services Scalable Data Systems Volume, rapid access data clusters Velocity, automatic launch of new processing resources Variety, cross-domain logic at run-time 7

8 Hurricane Sandy Predictions from the Hurricane Center If there are any doubters left about the value of big data and analytics, Hurricane Sandy has surely blown them away... Three-days out, forecasters predicted within 10 miles where landfall would occur. Twenty years ago, they may not even have known that the storm would have taken its unconventional turn left.... thanks to the rapidly growing speed and power of computers, and the ability to collect and analyze data faster and more accurately than before, an even bigger disaster was averted. - Nick Wakeman, Washington Technology 8

9 Changes to Mission Support Reduction in data movement and copies across networks Bring the algorithms to the data, not the data to the algorithms Better visualization for the human-in-the-loop We must feed the cognition of decision makers The experts may not be in your organization Data Science teams may need to combine government, universities and commercial experts to leverage all the data Entrepreneurs are leveraging weather and climate data to build previously impossible business For example, The Climate Corporate selling bad weather insurance to farmers 9

10 Beginning the Conversation How do you take advantage of the economics of cloud-mobiledata-social in the weather data lifecycle? What new capabilities are possible by combining data across disciplines Weather, air travel, safety, financial, health, agriculture What are the new commercial business models or enhanced existing models? What are the potential impacts on public-private partnerships, on the role of government, or government policy? Hoping to elaborate on this at the summer meeting We would welcome your interest and suggestions 10

11 Big Data Technologies Bring Big Opportunities The ground rules have changed. Big Data is upon us How can we best Leverage the technology Connect across domains for new capabilities Construct business models based on the new economics 11

12 Contact Information Nancy Grady,