Bretigny, 21st May 2014

Size: px

Start display at page:

Download "Bretigny, 21st May 2014"

Melvin Ray
7 years ago
Views:

1 1

2 Bretigny, 21st May

3 Bretigny, 21st May

4 4 wiki.complexworld.eu

5 5 wiki.complexworld.eu

6 6 wiki.complexworld.eu

7 ComplexWorld Position Paper Uncertainty in ATM Emergent behaviour Non-classical complex metrics Resilience in air transport Complex Data Mining 7 wiki.complexworld.eu

8 Complex Data Mining This section deals with techniques for extracting meaningful information from a complex system, and specifically from air traffic management, by analysing sets of existing data. ComplexWorld Position Paper 8

9 This section deals with techniques for extracting meaningful information from a complex system, and specifically from air traffic management, by analysing sets of existing data. + non-complexity related concepts 9

10 Progress in science depends on new techniques, new discoveries, and new ideas, probably in that order Sydney Brenner, biologist, 2002 Nobel Prize Winner 10

11 new techniques new discoveries new ideas 11

12 new techniques new discoveries new ideas Safety Capacity Passenger delay Data Security Costs models Emissions Resilience Airports Decision Support Fuel usage 12

13 Data science is a set of fundamental principles that support and guide the principled extraction of information and knowledge from data Data science involves facilities, processes and techniques for understanding phenomena via the automatic analysis of data with the ultimate goal of improving data-driven decision making 13

14 How we ended up here? 14

15 How we ended up here? 18th century Bayesian and classical statistics 1920 Parametric models 1980 Non-linear version of parametric models 1990 Highly non-linear relationships in large datasets in complex systems of the real world 2000 Multi-layer neural nets (e.g. SVM) 15

16 How we ended up here? last 30 years focus on better algorithms future? focus on designing processes 16

17 statistical analysis vs data mining Explaining vs predicting Querying (which data satisfy this pattern?) Discovery (which pattern satisfy this data?) Certain properties that are kept in our datasets Algorithms that build patterns, model is actually built with the data. 17

18 How we ended up here? Developments on Information Technologies Storage (Cloud) Computing Communications 18

19 Statistical Modelling Data Mining Machine Learning and Knowledge Discovery Computer Science IT facilities Visual Dashboards DSS 19

20 Data science is a set of fundamental principles that support and guide the principled extraction of information and knowledge from data Data science involves facilities, processes and techniques for understanding phenomena via the automatic analysis of data with the ultimate goal of improving data-driven decision making 20

21 knowledge from data TRL1 facilities, processes and techniques automatic analysis of data data-driven decision making new ideas, new discoveries data-driven operational paradigms data-driven tools and applications TRL9 21

22 Science? 22

23 Science? Multi-disciplinary Statistical Analysis Artificial intelligence Machine Learning Computer Science Data facilities 23

24 Science? Art? Multi-disciplinary Statistical Analysis Artificial intelligence Machine Learning Computer Science Data facilities Visual Dashboards Complexity Science Statistical physics Network Theory Scalable Analytics Domain Knowledge 24

25 Challenges!! 25

26 Challenges!! Data acquisition 26

27 Challenges!! Data facilities Data acquisition 27

28 Challenges!! Data provenance Data acquisition Data facilities 28

29 Challenges!! Algorithm performance Data acquisition Data facilities Data provenance 29

30 Challenges!! Analytic Dashboards Data acquisition Data facilities Data provenance Algorithm performance 30

31 Challenges!! Data acquisition Data facilities Data provenance Algorithm performance Analytic Dashboards Non-concurrency Domain analysis Complexity Volume and Big Data Validation Uncertainty Security Predictive Analytics Prescriptive Analytics Deployment techniques Cultural Shifts Confidentiality, privacy PA Automation 31

32 knowledge from data facilities, processes and techniques automatic analysis of data data-driven decision making New data-driven operational paradigms for air transport In God we trust; all others must bring data W. Edwards Deming, statistician 32

34 2014 Bretigny, 21st May

35 35

Augmented Search for Web Applications. New frontier in big log data analysis and application intelligence

Augmented Search for Web Applications New frontier in big log data analysis and application intelligence Business white paper May 2015 Web applications are the most common business applications today.