Teaching Analy-cs, Big Data and Sustainability: An IS perspec-ve

Transcription

1 Teaching Analy-cs, Big Data and Sustainability: An IS perspec-ve Raja Sooriamurthi / Randy Weinberg Informa(on Systems Program Carnegie Mellon University {raja,rweinberg}@cmu.edu

2 Presenta-on Outline The UG Informa(on Systems Program at CMU Two exis(ng courses on Analy(cs / Data mining Classical analy(cs Intelligent Decision Support Systems S13: Big Data and Sustainability * S14 (V2.0) : Big Data Analy(cs * Created with the support of an IBM Watson Solu(ons Faculty Grant

3 The IS Curriculum IS millieux System Devl IS app Fall " " " IS concepts App D&D SD project Spring " " " 1 st year 2 nd year 3 rd year 4 th year Elec(ves in the Analy(cs, Cri(cal Thinking, Problem Solving Space " " " " " Puzzles, Games, and Problem Solving Applied Analy(cs Intelligent Decision Support Systems Big Data and Sustainability Big Data Analy(cs

4 Applied Analy-cs (Classical) Classifica(on [Predic(ve] Clustering [Descrip(ve] Associa(on Rule Discovery [Descrip(ve] Sequen(al Pabern Discovery [Descrip(ve] Regression [Predic(ve] Devia(on Detec(on [Predic(ve]

5 Intelligent Decision Support Systems association" classification" clustering" design" optimization" prediction" recommendation" search/ranking" NN, SOM! SA! GA, GP! RBS,CBR! DT! Bayes! NNMF! SVM! puzzle" fielded application" the concept" demo"

6 Big Data and Sustainability Three Themes Ø Big Data Ø Cognitive Computing Ø Sustainability

7 BD&S: Learning Objec-ves Explain the concept of Big Data and appreciate the value it can bring to an organiza(on. Explain differences between tradi(onal data analy(cs, Big Data compu(ng, and Cogni(ve Compu(ng. Demonstrate basic facility with Hadoop and Google BigQuery Explain the architecture of IBM's Watson compu(ng plagorm and explore how it can be used in the context of the theme of sustainability

8 Background Review Analy(cs: associa(on, classifica(on, clustering, design, op(miza(on, predic(on, recommenda(on, search/ranking Big Data: Volume, Variety, Velocity Volume: terra (1012), peta (1015), exa (1018), zeba (1021), yoba (1024) From the dawn of civiliza8on un8l 2003, humankind generated 5 exabytes of data. Now we produce 5 exabytes every two days. Eric Schmidt Why BD? Can generate, collect, store, process The unreasonable effec(veness of data Algorithms + Data => Insight => Decisions => Value Simple models and a lot of data trump more elaborate models based on less data

9 The Unreasonable Effectiveness of Data Simple models and a lot of data trump more elaborate models based on less data..- Peter Norvig ந ன க ஓட டங கள மற ற ம எங கள தந தகள ம ன ப ஏழ ஆண ட கள ப டப ப ல அ னவர ம சமம என ற ச தந த ரம உணரப பட க ன றத, மற ற ம ம ன ம ழ வ க ம அர ப பண க கப பட ட, இந த கண டத த ல ம ஒர ப த ய தசத த ம ற ம ற வந தத. இப ப த ந ம அந த தசத த, அல லத எந த ந ட ட ன ம கவ ம ச ந த த த ம கவ ம அர ப பண க கப பட ட, ந ண ட ப ற த த க க ள ள ம ட ய ம என ப த பர ச த த த, ஒர பர ய உள ந ட ட ப ர ல ஈட பட ட வர க ன றனர. அந த ப ர ஒர பர ம ப ர -களத த ல சந த த த ர. ந ம இங க அந த ந ட வ ழ வண ட ம என ற அவர கள உய ர ந த த அந த ஒர இற த ந ன ற ப க ம இடத த ல ப ல, அந த த றய ல ஒர பக த ய ஒத க க வந த ர க க றன. அ த ந ம சய ய வண ட ம என ற ம ற ற ல ம ப ர த தம ன சர ய ன உள ளத. Four score and seven years ago our fathers brought forth on this continent a new nation, conceived in liberty, and dedicated to the proposition that all men are created equal. Now we are engaged in a great civil war, testing whether that nation, or any nation so conceived and so dedicated, can long endure. We are met on a great battle-field of that war. We have come to dedicate a portion of that field, as a final resting place for those who here gave their lives that that nation might live. It is altogether fitting and proper that we should do this.

10 Assignments Three Themes Ø Big Data Ø Cognitive Computing Ø Sustainability

11 IBM s grand challenges: Chess vs. Jeopardy Computer Science Engineering Cognitive Computing

12 DeepQA: the technology & architecture behind Watson Learned Models help combine and weigh the Evidence Answer Sources Evidence Sources model" model" model" Initial! Question! Primary Search Candidate Answer Genera(on Answer Scoring Evidence Retrieval Deep Evidence Scoring model" model" model" model" model" model" Question & Topic Analysis" Question" Decomposition" Hypothesis" Generation" Hypothesis " & Evidence Scoring" Synthesis" Final Confidence Merging & Ranking" Hypothesis" Generation" Hypothesis and Evidence Scoring" Answer & Confidence! Hypothesis" Generation" Hypothesis and Evidence Scoring"

13 Guest Speakers

14 Creating Smarter Physical Infrastructure Unleashing Watson-like capabilities in the Built Environment

15 Big Data and TwiWer Analy-cs for Disaster Response If less than.001% of the more than 20million tweets from Hurricane Sandy were Accurate, That leaves over 1000 informa(ve real- (me tweets ~ 15,000 words / or 25 p.

16 The evolution of Watson: Past, Present, Future Watson 2.0 work Automatic classification of diseases using ICD codes Smoothening the patientlab-hospital insurance business process

17 Graph Analytics in Big Data

18 Sustainability: Greatest global predicament? Everything we need for our survival and well-being depends, either directly or indirectly on our natural environment. "

19 Course Project: + Develop a strategic vision and tac(cal plan and to put IBM Watson to work on a problem of environmental sustainability. Sample Project Themes medical plants in the rain forest farm management health of bats disaster response flood and river management

20 Acknowledgements We thank IBM for suppor(ng our pedagogical efforts with a Watson Solu(ons Faculty Grant. We thank IBM colleagues Wayne Balta, David Bartleb, Jerry Haan, Carolyn Calzavara and Saman Haqqi; CMU colleagues Judith Gelernter and Eric Nyberg; and PSC colleagues Bryon Gill, Nick Nystrom, and J. Ray Scob.

21 V2.0 S14: Big Data Analy-cs Split course into two telescoped parts Enrollment by Instruc(on Permission only First eight weeks: Technology and Tools focused Hadoop (BigInsights, Streams, Amazon EMR, Wukong) BigQuery Three hour class once a week ( ) First 80 minutes flipped; less presenta(on more discussion Next 80 minutes demos Grade: A or I (?) Second eight weeks (op(onal): Explore and do something interes(ng with a real world data set