?????? Data Analytics

Transcription

1 ?????? Data Analytics Prof. Dr.-Ing. Lars Linsen Prof. Dr. Adalbert FX Wilhelm Fall 2015

2 0. Organizational Stuff

3 0.1 Syllabus and Organization Data Analytics 3

4 Course website (will be accessible through CampusNet) Data Analytics 4

5 Course description This course provides an introduction to data analytics concepts and methods. The objective of the course is to present methods for gaining insight from data and drawing conclusions for analytical reasoning and decision making. The course starts off by giving real-world examples. Abstracting from these examples leads towards a taxonomy for data types, their characteristics, and relations. The course comprises methods for the analytics of text or document data, image data, high-dimensional data, time-series data, and geospatial data. Moreover, concepts for the analysis of hierarchical, uni-, or bilateral relations are being taught. Data visualization methods are used for visual data representations, visual encoding, and interaction mechanisms, leading to an interactive visual analytics process. Automatic analysis components such as data transformation, aggregation, classification, clustering, and outlier detection are an integral part of the analytics process. Data Analytics 5

6 Lectures Times: - Tuesday, 9:45am 11:00am, - Thursday 8:15am 9:30am. Location:??? Data Analytics 6

7 Instructors Lars Linsen (75%) Office: Res I, 128. Phone: l.linsen Office hours: by appointment Adalbert FX Wilhelm (25%) Office: Res IV, 111. Phone: a.wilhelm Data Analytics 7

8 Lectures Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8 Week 9 Week 10 Week 11 Week 12 Week 13 Week 14 Sep 8 Sep 10 Sep 17 Sep 24 Oct 1 Oct 8 Oct 15 Oct 22 Oct 29 Nov 5 Nov 12 Nov 19 Nov 26 Dec 3 Linsen Wilhelm Linsen Linsen Linsen Linsen Linsen Wilhelm Wilhelm Wilhelm Linsen Linsen Linsen Linsen Data Analytics 8

9 Tuesdays Thursday lectures end with an assignment, where the taught material needs to be applied to a real-world problem. Students will work in groups on a solution. It is intended that group compositions change during the duration of the course. Students will present solutions in the Tuesday slots. Data Analytics 9

10 Exams There will be a written final examination. Date of the exam: tbd (around finals week). There will be no quizzes or midterms. Data Analytics 10

11 Grading Assignments: 60% Final exam: 40% Data Analytics 11

12 Literature??? Alexandru Telea: Data Visualization: Principles and Practice, Wellesley, Mass.: AK Peters, 1st edition, Matthew Ward, Georges Grinstein, Daniel Keim: Interactive Data Visualization: Foundations, Techniques, and Applications. AK Peters, 1st edition, Data Analytics 12

13 Goal This course provides an introduction to data analytics concepts and methods. The objective of the course is to present methods for gaining insight from data and drawing conclusions for analytical reasoning and decision making. Data Analytics 13

14 Topics Introductory examples Taxonomy for data types Supervised and unsupervised learning Visual analytics High-dimensional data analytics Aggregation, clustering, and classification Text and document data analytics Image data analytics Relations Time-series data analytics Geospatial data analytics Data Analytics 14

15 1. Introductory Examples and Taxonomy

16 1.1 Examples for the Digital Era

17 Social media [LinkedIn] Data Analytics 17

18 Twitter Data Analytics 18

19 Twitter Data Analytics 19

20 Twitter Data Analytics 20

21 Twitter Data Analytics 21

22 Instagram Data Analytics 22

23 Instagram Data Analytics 23

24 Some challenges bilateral relations huge network text & document data image data time-varying data geospatial data different heterogeneous sources Data Analytics 24

25 Tasks detect hot topics what goes viral? detect trends detect changes over time detect spatio-temporal patterns Data Analytics 25

26 Movies online Data Analytics 26

27 Netflix competition Data Analytics 27

28 Some challenges massive data: 500k users 20k movies 100m ratings many factors affect ratings actors directors genres high-dimensional data data incomplete Data Analytics 28

29 Tasks detect correlations understand correlations make predictions (related to many other application, cf. online selling, e.g., amazon etc.) Data Analytics 29

30 Human genome Data Analytics 30

31 Microarrays Data Analytics 31

32 Sequencing Data Analytics 32

33 Sequencing costs Data Analytics 33

34 Genome data Data Analytics 34

35 Genome data Data Analytics 35

36 Genome visualization Data Analytics 36

37 Genome visualization Data Analytics 37

38 Genome visualization Data Analytics 38

39 Personalized therapy 10 years from now, each cancer patient is going to want to get a genomic analysis of their cancer and will expect customized therapy based on that information. (Director of The Cancer Genome Atlas, Time Magazine, June 13, 2011) Data Analytics 39

40 Connectome Ramon y Cajal, 1905 Data Analytics 40

41 Connectome workflow Data Analytics 41

42 Ultra-thin eletron microscopy sections Data Analytics 42

43 Automatic reconstruction Data Analytics 43

44 Connectome visualization Data Analytics 44

45 Crime prevention Data Analytics 45

46 Predictive policing [sueddeutsche.de] Data Analytics 46

47 Predictive policing using Tableau Data Analytics 47

48 Internet of things Data Analytics 48

49 Taxi data Data Analytics 49

50 1.2 Big data analytics

51 Big Data Data Analytics 51

52 Big Data Data Analytics 52

53 Big Data Between the dawn of civilization and 2003, we only created five exabyte of information; now we re creating that amount every two days. (Eric Schmidt, Google) Data Analytics 53

54 What is Big Data? Massive data? How many exabytes? Everything we cannot inspect manually. It s not just about the amount of data it s also about the complexity of the data. Data Analytics 54

55 The big V s of Big Data Data Analytics 55

56 The big V s of Big Data Data Analytics 56

57 The big V s of Big Data Data Analytics 57

58 The fourth paradigm Data Analytics 58

59 Ubiquitous data The ability to take data to be able to understand it, to process it, to extract value from it, to visualize it, to communicate it s going to be a hugely important skill in the next decades, not only at the professional level but even at the educational level for elementary school kids, for high-school kids, for college kids. Because now we really do have essentially free and ubiquituous data. (Hal Varian, UC Berkeley) Data Analytics 59

60 Job growth Data Analytics 60

61 Processing pipeline Data Analytics 61

62 1.3 Taxonomy

63 Data example Data Analytics 63

64 Data example Data Analytics 64

65 Data example Data Analytics 65

66 Data example Data Analytics 66

67 Data example Data Analytics 67

68 Taxonomy Data samples are items with attributes. Attributes (stored in tables) can be quantitative continuous numbers (real), discrete numbers (integer), ordinal (ordered sets, rating), or nominal / categorical (unordered sets). Data Analytics 68

69 Taxonomy Nominal / categorical support = relationship oranges, apples, Ordinal obey < relationship small < medium < large Quantitative can do arithmetics on them cm, kg, Data Analytics 69

70 Data dimensions Uni-variate Data Analytics 70

71 Data dimensions Bi-variate Data Analytics 71

72 Data dimensions Tri-variate Data Analytics 72

73 Data dimensions Multi-variate Data Analytics 73

74 Special attributes Geospatial location (longitude, latitude) Time-varying attributes change values over time time series Spatio-temporal geospatial & time-varying Data Analytics 74

75 Other aspects Data Analytics 75

76 Complex data attributes Text / document cf. Twitter Image cf. Instagram Data Analytics 76

77 Data samples may have unilateral relations, bilateral relations, or hierarchical relations. Relations Data Analytics 77