Raul F. Chong Senior program manager Big data, DB2, and Cloud IM Cloud Computing Center of Competence - IBM Toronto Lab, Canada

Transcription

1 What is big data? Raul F. Chong Senior program manager Big data, DB2, and Cloud IM Cloud Computing Center of Competence - IBM Toronto Lab, Canada IBM Corporation

2 Agenda The world is changing What is big data? The IBM big data platform Examples of big data solutions Big data solutions and the cloud Setting up a Hadoop cluster on the cloud IBM Corporation

3 Agenda The world is changing What is big data? The IBM big data platform Examples of big data solutions Big data solutions and the cloud Setting up a Hadoop cluster on the cloud IBM Corporation

4 The world is changing and becoming more 2 Billion internet users 4.6 Billion mobile phones IBM Corporation

5 The world is changing and becoming more Is it really? IBM Corporation

6 Agenda The world is changing What is big data? The IBM big data platform Examples of big data solutions Big data solutions and the cloud Setting up a Hadoop cluster on the cloud IBM Corporation

7 What is big data? Big data are datasets that grow so large that they become awkward to work with using on-hand database management tools. Difficulties include capture, storage, search, sharing, analytics, and visualizing. Source: Wikipedia IBM Corporation

8 Big data characteristics 44x as much data and content over coming decade zettabytes ,000 petabytes Information is growing at a phenomenal rate Source: IDC, The Digital Universe Decade Are You Ready?, May IBM Corporation

9 Big data characteristics About 80%of the world s data is unstructured It may be data we ve been collecting before, but could not process IBM Corporation

10 IBM Corporation

11 Two types of big data Data in movement -streams Twitter / Facebook comments Stock market data Sensors: Vital signs of a newly-born Data at rest -oceans Collection of what has streamed Web logs, s, social media Unstructured documents: forms, claims Structured data from disparate systems IBM Corporation

12 Agenda The world is changing What is big data? The IBM big data platform Examples of big data solutions Big data solutions and the cloud Setting up a Hadoop cluster on the cloud IBM Corporation

13 The IBM Big Data Platform Data Warehouse Client and Partner Solutions IBM Big Data Solutions InfoSphere Warehouse Warehouse Appliances Big Data User Environments Developers End Users Netezza Administrators Master Data Mgmt InfoSphere MDM INTEGRATION AGENTS Big Data Enterprise Engines Open Source Foundational Components Hadoop HBase Pig Lucene Jaql Information Server InfoSphere BigInsights InfoSphere Streams Database DB2 Content Analytics ECM Business Analytics Cognos & SPSS Marketing Unica Data Growth Management InfoSphere Optim IBM Corporation

14 The IBM Big Data Platform Data Warehouse Big Data Platform Enterprise Integration Traditional Sources New Sources IBM Corporation

15 Traditional vs. big data business approaches Traditional Approach Structured & Repeatable Analysis Big Data Approach Iterative & Exploratory Analysis Business Users Determine what question to ask IT Delivers a platform to enable creative discovery IT Structures the data to answer that question Monthly sales reports Profitability analysis Customer surveys Business Explores what questions could be asked Brand sentiment Product strategy Maximum asset utilization IBM Corporation

16 Agenda The world is changing What is big data? The IBM big data platform Examples of big data solutions Big data solutions and the cloud Setting up a Hadoop cluster on the cloud IBM Corporation

17 Examples of big data solutions Multi-channel customer sentiment and experience analysis Detect life-threatening conditions at hospitals in time to intervene Make risk decisions based on real-time transactional data Identify criminals and threats from disparate video, audio, and data feeds Predict weather patterns to plan optimal wind turbine usage, and optimize capital expenditure on asset placement IBM Corporation

18 IBM Corporation

19 IBM Corporation

20 Examples IBM Watson IBM Corporation

21 Agenda The world is changing What is big data? The IBM big data platform Examples of big data solutions Big data solutions and the cloud Setting up a Hadoop cluster on the cloud IBM Corporation

22 Big data solutions and the cloud Big Data solutions and the Cloud are a good fit. The Cloud provides On-demand access to resources Elasticity Utility-like billing Big Data solutions on the Cloud are mainly appropriate for test/development Eg: Hadoop was not designed for virtualized environments IBM Corporation

23 Demo Setting up a Hadoop cluster on the IBM Cloud IBM Corporation

24 IBM Corporation

25 IBM Corporation

26 IBM Corporation

27 IBM Corporation

28 IBM Corporation

29 IBM Corporation

30 IBM Corporation

31 IBM Corporation

32 IBM Corporation

33 IBM Corporation

34 IBM Corporation

35 IBM Corporation

36 IBM Corporation

37 IBM Corporation

38 IBM Corporation

39 INSTRUMENTED INTERCONNECTED INTELLIGENT The resulting explosion of information creates a need for a new kind of intelligence to help build a Smarter Planet IBM Corporation

40 For more information IBM Corporation

41 Thank you! Questions? IBM Corporation