Analyzing large and complex data sets in fusion: Modern tools from probability theory and pattern recognition

Transcription

1 FACULTY OF ENGINEERING AND ARCHITECTURE Analyzing large and complex data sets in fusion: Modern tools from probability theory and pattern recognition Geert Verdoolaege Department of Applied Physics Ghent University, Ghent, Belgium Laboratory for Plasma Physics, Royal Military Academy, Brussels, Belgium PRL Seminar Canberra, December 18, 2013

2 Overview 1 Data Science 2 Bayesian methods and integrated analysis Motivation Bayesian methodology Integrated data analysis Example applications 3 Pattern recognition in stochastic data sets Probabilistic manifolds Confinement regime visualization and identification Scaling laws Disruption prediction 4 Conclusion and outlook 1

3 Outline 1 Data Science 2 Bayesian methods and integrated analysis Motivation Bayesian methodology Integrated data analysis Example applications 3 Pattern recognition in stochastic data sets Probabilistic manifolds Confinement regime visualization and identification Scaling laws Disruption prediction 4 Conclusion and outlook 2

4 Evolution of science (1) 1000 years ago: experimental science Last few 100 years: theoretical science 3

5 Evolution of science (2) Last few decades: Computational science Today: Data science 4

6 The data deluge Big data : High-throughput scientific experiments: Telescopes Particle accelerators Fusion devices Sensor networks, satellite surveys Simulations Biology: e.g. human genome Sociology Digital archives... 5

7 The fourth paradigm (1) Jim Gray, Microsoft Research (Turing Award, 1998): Fourth paradigm The new model is for the data to be captured by instruments or generated by simulations before being processed by software and for the resulting information or knowledge to be stored in computers. Scientists only get to look at their data fairly late in this pipeline. The techniques and technologies for such data-intensive science are so different that it is worth distinguishing data-intensive science from computational science as a new, fourth paradigm for scientific exploration. 6

8 The fourth paradigm (2) The Fourth Paradigm: Data-Intensive Scientific Discovery, T. Hey, S. Tansley, K. Tolle, eds., Microsoft Research, Redmond, WA,

9 Data science Challenges: Large data sets Heterogeneous sources and formats Complex nonlinear dependencies Activities: Data analysis: extracting useful information Data visualization Data archiving and retrieval 8

10 What is a data scientist? Opinions vary: Hilary Mason, chief data scientist at bitly: Harvard Business Review, 2012: The sexiest job of the 21 st Century 9

11 Data analysis Level: low intermediate high Descriptive statistics for data visualization Analysis of time series, images, videos and fields (e.g. magnetic) Resampling, (motion) correction,... Fourier/wavelet analysis Inverse problems: Abel inversion, tomography,... Estimation and prediction Event detection, object recognition, object tracking, shape analysis Extracting patterns in data spaces: clusters, regression lines/surfaces Probability theory and statistics for modeling, estimation, hypothesis testing, prediction, error analysis,... 10

12 Fusion data characteristics Challenge Remedy/opportunity Massive databases Real-time requirements (plasma control) Clever and fast algorithms Substantial and heterogeneous uncertainties, stochasticity Probability theory Error propagation studies e.g. Bayesian probability theory Redundancy Pattern recognition: (nonlinear) relations, cluster structure High dimensionality Dimensionality reduction Data visualization 11

13 Outline 1 Data Science 2 Bayesian methods and integrated analysis Motivation Bayesian methodology Integrated data analysis Example applications 3 Pattern recognition in stochastic data sets Probabilistic manifolds Confinement regime visualization and identification Scaling laws Disruption prediction 4 Conclusion and outlook 12

14 Outline 1 Data Science 2 Bayesian methods and integrated analysis Motivation Bayesian methodology Integrated data analysis Example applications 3 Pattern recognition in stochastic data sets Probabilistic manifolds Confinement regime visualization and identification Scaling laws Disruption prediction 4 Conclusion and outlook 13

15 Motivation and objectives Methods: Bayesian probability theory (BPT) Integrated data analysis (IDA) Motivation and objectives: Enhance reliability and robustness of physics results Identify and reduce uncertainty sources Study non-gaussian error propagation Model and quantify systematic uncertainty Integrate heterogeneous data sets: exploit redundancy and interdependencies Diagnostic design ITER and fusion reactors: Reduced space: limited data Reduced access: systematic uncertainty In situ calibration 14

16 Outline 1 Data Science 2 Bayesian methods and integrated analysis Motivation Bayesian methodology Integrated data analysis Example applications 3 Pattern recognition in stochastic data sets Probabilistic manifolds Confinement regime visualization and identification Scaling laws Disruption prediction 4 Conclusion and outlook 15

17 Bayesian recipe Forward model: link (physical) model with data x, e.g.: x = f ( θ ) Assume statistical (random) measurement error, often Gaussian, e.g.: x = f ( θ ) + ν, with ν N (0, σν) 2 = p(x 1 θ ) = exp 2πσν [x f ( θ )] 2 2σ 2 ν Likelihood Propose prior information p( θ ), possibly uninformative (e.g. uniform) Apply Bayes theorem: solve the inverse problem (also profile reconstruction!) 16

18 Bayes theorem p( θ x, I ) = p( x θ, I )p( θ I ) p( x I ) x = data vector, θ = parameter vector, I = implicit assumptions Posterior Likelihood: misfit between model and data Prior: expert or diffuse knowledge Evidence: normalization 17

19 Bayes theorem p( θ x, I ) = p( x θ, I )p( θ I ) p( x I ) x = data vector, θ = parameter vector, I = implicit assumptions Posterior Likelihood: misfit between model and data Prior: expert or diffuse knowledge Evidence: normalization 17

20 Bayes theorem p( θ x, I ) = p( x θ, I )p( θ I ) p( x I ) x = data vector, θ = parameter vector, I = implicit assumptions Posterior Likelihood: misfit between model and data Prior: expert or diffuse knowledge Evidence: normalization 17

21 Bayes theorem p( θ x, I ) = p( x θ, I )p( θ I ) p( x I ) x = data vector, θ = parameter vector, I = implicit assumptions Posterior Likelihood: misfit between model and data Prior: expert or diffuse knowledge Evidence: normalization 17

22 Outline 1 Data Science 2 Bayesian methods and integrated analysis Motivation Bayesian methodology Integrated data analysis Example applications 3 Pattern recognition in stochastic data sets Probabilistic manifolds Confinement regime visualization and identification Scaling laws Disruption prediction 4 Conclusion and outlook 18

23 Principle of IDA Integrate heterogeneous (conflicting?) data sets: likelihoods Model statistical and systematic uncertainties: Measured data Calibration Underlying physical model Estimate model parameters using Bayes theorem Sample the posterior: Markov Chain Monte Carlo 19

24 Example applications n e from Thomson scattering, soft X-ray and interferometry n e profiles from Li-beam and interferometry: factor 400 increase in temporal resolution! Magnetic equilibrium reconstruction: current filaments + Grad-Shafranov Impurity concentrations from CXS + BES Antenna design EM wave propagation via stochastic FDTD: uncertainty in boundary conditions, medium properties, etc. 20

25 Outline 1 Data Science 2 Bayesian methods and integrated analysis Motivation Bayesian methodology Integrated data analysis Example applications 3 Pattern recognition in stochastic data sets Probabilistic manifolds Confinement regime visualization and identification Scaling laws Disruption prediction 4 Conclusion and outlook 21

26 Thomson scattering + soft X-ray + interferometer Fischer et al., Plasma Phys. Control. Fusion, 44, pp ,

27 Bremsstrahlung + CX impurity lines p (x ) n (x cm 3 ) e,1 2 p (x ) n (x cm 3 ) e,2 2 p p Z eff, s ε p p Z eff, s δ Verdoolaege et al., IEEE Trans. Plasma Sci.,, pp.,

28 CXS + BES Helium concentration profile at ITER: 10% error Spectrometer at TEXTOR and later AUG BES: emission from beam Error reduction from 40% 10% He concentration profile with 68% credibility bounds (TEXTOR): Planned at AUG 24

29 Outline 1 Data Science 2 Bayesian methods and integrated analysis Motivation Bayesian methodology Integrated data analysis Example applications 3 Pattern recognition in stochastic data sets Probabilistic manifolds Confinement regime visualization and identification Scaling laws Disruption prediction 4 Conclusion and outlook 25

30 Pattern recognition opportunities Clustering/classification: grouping of data points Dimensionality reduction: data visualization, better learning efficiency Regression: (nonlinear) deterministic relation between variables Objectives 1 Contribute to physics studies by extracting patterns, structure and relations from data 2 Contribute to plasma control through real-time data interpretation Note: Both model-based and purely data-driven approaches are possible One approach does not exclude the other 26

31 Outline 1 Data Science 2 Bayesian methods and integrated analysis Motivation Bayesian methodology Integrated data analysis Example applications 3 Pattern recognition in stochastic data sets Probabilistic manifolds Confinement regime visualization and identification Scaling laws Disruption prediction 4 Conclusion and outlook 27

32 A different view on measurement Traditional measurement: value + error bar Uncertainty is quantized by probability Measurement = sample from underlying (non-gaussian?) probability distribution Goal of measurement = probing the underlying distribution Stochastic component of data descriptive model = probability density function (PDF) Examples: y = β 0 + β 1 x } {{ } Deterministic component y = β 0 + β 1 (x + + ɛ }{{} Stochastic component η ) + ɛ }{{} Error in (independent) variable PDF contains all information about measurement! 28

33 The challenge The probabilistic nature of data The fundamental object resulting from a measurement is a probability distribution. Any further processing of the data (statistical inference, pattern recognition) should respect this inherent probabilistic nature. Pattern recognition in PDF spaces Pattern recognition is based on geometry, primarily distance Geometry of probability distributions Obtain PDF from Repeated measurements (Bayesian) probability theory... 29

34 Probability + geometry: a happy marriage Probabilistic manifold: PDF = point on manifold Coordinates = PDF parameters Distance between PDFs? Information geometry 30

35 Information geometry Riemannian differential geometry Fisher information = unique metric tensor: ) ( Parametric probability model: p x θ = ( ) [ θ g µν = E 2 θ µ θ ν ln p θ = N-dimensional parameter vector ( x θ )], µ, ν = 1... N Line element: ds 2 = g µν dθ µ dθ ν Minimum-length curve: geodesic Geodesic distance (GD) Natural and theoretically well motivated distance between PDFs 31

36 Univariate Gaussian distribution PDF: p(x µ, σ) = 1 ] (x µ)2 exp [ 2πσ 2σ 2 Line element: ds 2 = dµ2 σ 2 + 2dσ2 σ 2 Hyperbolic geometry: Poincaré half-plane model 32

37 Poincaré half-plane p 1 : µ 1 = 4, σ 1 = 0.7; p 2 : µ 2 = 3, σ 2 =

38 Poincaré half-plane p 1 : µ 1 = 4, σ 1 = 0.7; p 2 : µ 2 = 3, σ 2 =

39 Outline 1 Data Science 2 Bayesian methods and integrated analysis Motivation Bayesian methodology Integrated data analysis Example applications 3 Pattern recognition in stochastic data sets Probabilistic manifolds Confinement regime visualization and identification Scaling laws Disruption prediction 4 Conclusion and outlook 35

40 ITPA Confinement Database ITPA Global H Mode Confinement Database (DB3) ITER H-Mode Database Working Group D.C. McDonald et al., Nucl. Fusion 47, pp , entries from 19 tokamaks Approximate error estimates: limited information on PDF! Assume standard deviations Gaussian PDFs (maximum entropy) Different machines different error estimates: difficult to handle using classic approach! 36

41 Confinement regime classification Distinguish between L- and H-mode: 3845 L and 6207 H Identify edge localized mode (ELM) behavior 8 global engineering variables: I p, B t, n e, P loss, R, a, M eff, κ Variables statistically independent product of Gaussians Note: this does not exclude the variables to be related through a deterministic relation! 37

42 Gaussian product manifold Plasma and machine variables: x i x, i = 1,..., 8 Distribution parameters: µ i, σ i Gaussian product distribution: ( x p µ 1,..., µ 8, σ 1,..., σ 8) N = N ( x i µ i, σ i) Measurements ( A and B: µa,b = µ 1 A,B,..., µ8 A,B GD in closed form: ( µa ) GD, σ A µ B, σ B = 2 i=1 ) ( ), σ A,B = σa,b 1,..., σ8 A,B [ 8 i=1 [( µ δab i i = A µ i B ( µ i A µ i B ( 1 + δ ln 2 i ) ] 1/2 AB 1 δab i, ) 2 ( + 2 σ i A σb) i 2 ] 1/2 ) 2 ( ) + 2 σ i A + σb i 2 38

43 Dimensionality reduction for visualization Step 1. Calculate all pairs of GDs proximity matrix [D ij ] Step 2. Plot points arbitrarily in 2D Euclidean space Step 3. Calculate Euclidean proximity matrix [E ij ] Step 4. Minimize i,j (D ij E ij ) 2 Multidimensional scaling Step 5. Plot final configuration 39

44 Confinement visualization Tokamaks Confinement regime Euclidean no errors GD with errors Verdoolaege et al., Fusion Sci. Technol., 62, pp ,

45 ELM behavior Verdoolaege et al., Rev. Sci. Instrum., 83, art. no. 10D715,

46 k-nearest neighbor classification Confinement mode identification Training: 5%, testing: 95% k = 1: nearest neighbor Correct classification rates (%) Mode Euclidean GD GD with w/o errors with errors random errors L H

47 Outline 1 Data Science 2 Bayesian methods and integrated analysis Motivation Bayesian methodology Integrated data analysis Example applications 3 Pattern recognition in stochastic data sets Probabilistic manifolds Confinement regime visualization and identification Scaling laws Disruption prediction 4 Conclusion and outlook 43

48 Geodesic regression p y p x Minimize sum of squared GD Use geodesic centroid 44

49 Synthetic data Original Data Simple regression Errors in variables Geodesic regression y x 45

50 Regression results ITPA global confinement: 10 0 Simple regression Geodesic regression τ E (s) τ reg (s) R 2 OLS = 0.44 R2 EIV = 0.52 R2 GR = 0.71 GR yields full probability distributions Precise error estimates are not required, but may improve estimates 46

51 Outline 1 Data Science 2 Bayesian methods and integrated analysis Motivation Bayesian methodology Integrated data analysis Example applications 3 Pattern recognition in stochastic data sets Probabilistic manifolds Confinement regime visualization and identification Scaling laws Disruption prediction 4 Conclusion and outlook 47

52 Wavelet distributions 1D/2D discrete wavelet transform (DWT) Retain time information Spectral energy distribution identifies signal/image Histograms: zero-mean + heavy-tailed Generalized Gaussian distribution (GGD): p(x α, β) = [ ( ) ] β x β 2αΓ(1/β) exp : features α and β α 48

53 Geodesic distance for GGDs Gaussian (β = 2): GD[N (0, σ 1 ) N (0, σ 2 )] = 2 ln ( α2 α 1 ) Laplace (β = 1): GD[L(0, α 1 ) L(0, α 2 )] = ln ( α2 α 1 ) 49

54 Experiment 1: setup (1) JET campaigns C15 C20 Disruptive shots: 334 Known time of disruption (ToD) Time series for 13 plasma variables Sliding window: 30 ms Regular features: until 1 s before ToD Disruptive features: from 210 ms before ToD Similar to APODIS: Rattá et al., Nucl. Fusion 50, ,

55 Experiment 1: setup (2) Fourier vs. wavelet: Fourier Power spectrum Standard deviation Euclidean and GD Wavelet Detail coefficients Daubechies 4, 3 scales Laplace α GD k-nearest neighbor classifier Training: 65%, testing: 35% Reproduce 20 times 51

56 Experiment 1: results TPR FPR MA FA SR AVG True positive rate False positive rate Missed alarms False alarms Success rate (= 1 (MA + FA)) Average detection time before ToD Performance Fourier Fourier Wavelet measure Euclidean GD GD TPR (%) 76.5 ± ± ± 1.3 FPR (%) 17.4 ± ± ± 0.7 MA (%) 0.6 ± ± ± 0.7 FA (%) 48.0 ± ± ± 2.9 SR (%) 51.4 ± ± ± 2.8 AVG (ms) 61.0 ± ± ± 3.2 Verdoolaege et al., Fusion Sci. Technol., 62, pp , 2012 Verdoolaege et al., SOFT

57 Experiment 2: generalization Generalize to campaigns C21 C27 53

58 Disruptive trajectory Landmark MDS: real-time projections Visual tool in control room JET # disrupted at s due to NTM: 54

59 Outline 1 Data Science 2 Bayesian methods and integrated analysis Motivation Bayesian methodology Integrated data analysis Example applications 3 Pattern recognition in stochastic data sets Probabilistic manifolds Confinement regime visualization and identification Scaling laws Disruption prediction 4 Conclusion and outlook 55

60 Conclusion Emerging field of data science Huge potential for probability theory and pattern recognition in fusion: Physics studies Plasma control Quantification and reduction of uncertainties Probability distributions are maximally informative Full probability structure determines patterns Patterns reflect physics 56