Latent variable and deep modeling with Gaussian processes; application to system identification. Andreas Damianou

Transcription

1 Latent variable and deep modeling with Gaussian processes; application to system identification Andreas Damianou Department of Computer Science, University of Sheffield, UK Brown University, 17 Feb. 2016

2 Outline Part 1: Introduction Recap from yesterday Part 2: Incorporating latent variables Unsupervised GP learning Structure in the latent space - representation learning Bayesian Automatic Relevance Determination Deep GP learning Multi-view GP learning Part 3: Dynamical systems Policy learning Autoregressive Dynamics Going deeper: Deep Recurrent Gaussian Process Regressive dynamics with deep GPs

3 Outline Part 1: Introduction Recap from yesterday Part 2: Incorporating latent variables Unsupervised GP learning Structure in the latent space - representation learning Bayesian Automatic Relevance Determination Deep GP learning Multi-view GP learning Part 3: Dynamical systems Policy learning Autoregressive Dynamics Going deeper: Deep Recurrent Gaussian Process Regressive dynamics with deep GPs

4 Part 1: GP Take-home messages Gaussian processes as infinite dimensional Gaussian distributions can be used as priors over functions Non-parametric: training data act as parameters Uncertainty Quantification Learning from scarce data

5 Notation and graphical models Graphical models x y f x f y White nodes: Observed variables Shaded nodes: Unobserved, or latent variables. Convention: Sometimes the latent function will be placed next to the arrow; sometimes I ll explicitly include the collection of instantiations f

6 Outline Part 1: Introduction Recap from yesterday Part 2: Incorporating latent variables Unsupervised GP learning Structure in the latent space - representation learning Bayesian Automatic Relevance Determination Deep GP learning Multi-view GP learning Part 3: Dynamical systems Policy learning Autoregressive Dynamics Going deeper: Deep Recurrent Gaussian Process Regressive dynamics with deep GPs

7 Unsupervised learning: GP-LVM x y f If X is unobserved, treat it as a parameter and optimize over it. GP-LVM is interpreted as non-linear, non-parametric probabilistic PCA (Lawrence, JMLR 2015). Objective (likelihood function) is similar to GPs: p(y x) = p(y f)p(f x)df = N (y 0, K ff + σ 2 I) but now x s are optimized too. Neil Lawrence, JMLR, 2005.

8 Unsupervised learning: GP-LVM x y f If X is unobserved, treat it as a parameter and optimize over it. GP-LVM is interpreted as non-linear, non-parametric probabilistic PCA (Lawrence, JMLR 2015). Objective (likelihood function) is similar to GPs: p(y x) = p(y f)p(f x)df = N (y 0, K ff + σ 2 I) but now x s are optimized too. Neil Lawrence, JMLR, 2005.

9 Unsupervised learning: GP-LVM x y f If X is unobserved, treat it as a parameter and optimize over it. GP-LVM is interpreted as non-linear, non-parametric probabilistic PCA (Lawrence, JMLR 2015). Objective (likelihood function) is similar to GPs: p(y x) = p(y f)p(f x)df = N (y 0, K ff + σ 2 I) but now x s are optimized too. Neil Lawrence, JMLR, 2005.

10 Fitting the GP-LVM

11 Fitting the GP-LVM 5.5 Figure credits: C. H. Ek

12 Fitting the GP-LVM 5.5 Figure credits: C. H. Ek Additional difficulty: x s are also missing! Improvement: Invoke the Bayesian methodology to find x s too.

13 Bayesian GP-LVM Additionally integrate out the latent space. p(y) = p(y f)p(f x)p(x)df dx where p(x) N (0, I) Titsias and Lawrence 2010, AISTATS; Damianou et al. 2015, JMLR

14 Automatic dimensionality detection In general, X = [x 1,..., x N ] with x (n) R Q. Automatic dimenionality detection by employing automatic relevance determination (ARD) priors for the mapping f. f GP(0, k f ) with: k f (x (i), x (j)) = σ 2 exp 1 Q ( w (q) x (i,q) x (j,q)) 2 2 Example: q=1

15 Outline Part 1: Introduction Recap from yesterday Part 2: Incorporating latent variables Unsupervised GP learning Structure in the latent space - representation learning Bayesian Automatic Relevance Determination Deep GP learning Multi-view GP learning Part 3: Dynamical systems Policy learning Autoregressive Dynamics Going deeper: Deep Recurrent Gaussian Process Regressive dynamics with deep GPs

16 Sampling from a deep GP Input f Unobserved f Y Output

17 Model x = given f h = f h (x) = GP(0, k h (x, x)) h = f h (x) + ɛ h f y = f y (h) = GP(0, k f (h, h)) y = f y (h) + ɛ y So: y = f y (f h (x) + ɛ h ) + ɛ y Objective: p(y x) = p(y f y )p(f y h)p(h f h )p(f h x)df y df h dh Damianou and Lawrence, AISTATS 2013; Damianou, PhD Thesis, 2015

18 Deep GP: Step function (credits for idea to J. Hensman, R. Calandra, M. Deisenroth) (standard GP) (deep GP - 1 hidden layer) (deep GP - 3 hidden layers)

19 Deep GP with three hidden plus one warping layer Standard GP

20 Non-linear feature learning Stacked GP (nonlinear) Stacked PCA (linear) Successive warping creates knots which act as features. Features discovered in one layer remain in the next ones (ie knots are not un-tied) f 1 (f 2 (f 3 (x))) With linear warpings (stacked PCA) we can t achieve this effect: W 1 (W 2 (W 3 x))) = W x Damianou, PhD Thesis, 2015

21 Deep GP: MNIST example (Live sampling video)

22 Outline Part 1: Introduction Recap from yesterday Part 2: Incorporating latent variables Unsupervised GP learning Structure in the latent space - representation learning Bayesian Automatic Relevance Determination Deep GP learning Multi-view GP learning Part 3: Dynamical systems Policy learning Autoregressive Dynamics Going deeper: Deep Recurrent Gaussian Process Regressive dynamics with deep GPs

23 Multi-view: Manifold Relevance Determination (MRD) Multi-view data arise from multiple information sources. These sources naturally contain some overlapping, or shared signal (since they describe the same phenomenon ), but also have some private signal. MRD: Model such data using overlapping sets of latent variables Demo: Faces, mocap-silhouette Damianou et al., ICML 2012; Damianou, PhD Thesis, 2015

24 Multi-view: Manifold Relevance Determination (MRD) Multi-view data arise from multiple information sources. These sources naturally contain some overlapping, or shared signal (since they describe the same phenomenon ), but also have some private signal. MRD: Model such data using overlapping sets of latent variables Demo: Faces, mocap-silhouette Damianou et al., ICML 2012; Damianou, PhD Thesis, 2015

25 Multi-view: Manifold Relevance Determination (MRD) Multi-view data arise from multiple information sources. These sources naturally contain some overlapping, or shared signal (since they describe the same phenomenon ), but also have some private signal. MRD: Model such data using overlapping sets of latent variables Demo: Faces, mocap-silhouette Damianou et al., ICML 2012; Damianou, PhD Thesis, 2015

26 Deep GPs: Another multi-view example X X

27 Automatic structure discovery summary: ARD and MRD Tools: ARD: Eliminate uncessary nodes/connections MRD: Conditional independencies Approximating evidence: Number of layers (?)

28 Automatic structure discovery summary: ARD and MRD Tools: ARD: Eliminate uncessary nodes/connections MRD: Conditional independencies Approximating evidence: Number of layers (?)

29 More deepgp representation learning examples icub interaction Faces - autoencoder

30 Outline Part 1: Introduction Recap from yesterday Part 2: Incorporating latent variables Unsupervised GP learning Structure in the latent space - representation learning Bayesian Automatic Relevance Determination Deep GP learning Multi-view GP learning Part 3: Dynamical systems Policy learning Autoregressive Dynamics Going deeper: Deep Recurrent Gaussian Process Regressive dynamics with deep GPs

31 Dynamical systems examples Through a model, we wish to learn to: perform free simulation by learning patterns coming from the latent generation process (a mechanistic system we do not know) perform inter/extrapolation in time-series data which are very high-dimensional (e.g. video) detect outliers in data coming from a dynamical system optimize policies for control based on a model of the data.

32 Policy learning Model-based policy learning (Cart-pole) (Unicycle) Work by: Marc Deisenroth, Andrew McHutchon, Carl Rasmussen

33 NARX model A standard NARX model considers an input vector x i R D comprised of L y past observed outputs y i R and L u past exogenous inputs u i R: x i = [y i 1,, y i Ly, u i 1,, u i Lu ], y i = f(x i ) + ɛ (y) i, ɛ (y) i Latent auto-regressive GP model: N (ɛ (y) i 0, σy), 2 x i = f(x i 1,, x i Lx u i 1,, u i Lu ) + ɛ (x) i, y i = x i + ɛ (y) i, Contribution 1: Simultaneous auto-regressive and representation learning. Contribution 2: Latents avoid the feedback of possibly corrupted observations into the dynamics. Mattos, Damianou, Barreto, Lawrence, 2016

34 NARX model A standard NARX model considers an input vector x i R D comprised of L y past observed outputs y i R and L u past exogenous inputs u i R: x i = [y i 1,, y i Ly, u i 1,, u i Lu ], y i = f(x i ) + ɛ (y) i, ɛ (y) i Latent auto-regressive GP model: N (ɛ (y) i 0, σy), 2 x i = f(x i 1,, x i Lx u i 1,, u i Lu ) + ɛ (x) i, y i = x i + ɛ (y) i, Contribution 1: Simultaneous auto-regressive and representation learning. Contribution 2: Latents avoid the feedback of possibly corrupted observations into the dynamics. Mattos, Damianou, Barreto, Lawrence, 2016

35 Robustness to outliers Latent auto-regressive GP model: x i = f(x i 1,, x i Lx u i 1,, u i Lu ) + ɛ (x) i, y i = x i + ɛ (y) i, ɛ (x) i ɛ (y) i N (ɛ (x) i 0, σx), 2 N (ɛ (y) 0, τ 1 i i ), τ i Γ(τ i α, β), Contribution 3: Switching-off outliers by including the above Student-t likelihood for the noise. Mattos, Damianou, Barreto, Lawrence, DYCOPS 2016

36 Robust GP autoregressive model: demonstration (a) Artificial 1. (b) Artificial 2. Figure: RMSE values for free simulation on test data with different levels of contamination by outliers.

37 (c) Artificial 3. (d) Artificial 4. (e) Artificial 5.

38 Going deeper: Deep Recurrent Gaussian Process u x (1) x (2) x (H ) y Figure 1: RGP graphical model with H hidden layers. x is the lagged latent function values augmented with the lagged exogenous inputs. Mattos, Dai, Damianou, Barreto, Lawrence, ICLR 2016

39 Inference is tricky... log p(y) N L 2 Tr H+1 h=1 ( ( K (H+1) z log 2πσ 2 h 1 ) 1 Ψ (H+1) (σH+1 2 y Ψ (H+1) )2 1 { ( H + 1 N 2σ 2 h=1 h i=l K (h) z 1 2 K(h) + 1 ( µ (h)) (h) Ψ 2(σh 2)2 1 N ) i=l+1 x (h) i ( q 2σ 2 H+1 ) ) ( y y + Ψ (H+1) 0 K z (H+1) ( K z (H+1) + 1 Ψ (H+1) σh ( λ (h) i + µ (h)) µ (h) + Ψ (h) 0 Tr z + 1 Ψ (h) σh 2 2 ( K z (h) + 1 ) 1 ( Ψ (h) σh 2 2 ( ) L log q + x (h) i x (h) i i=1 x (h) i 1 2 K(H+1) z + 1 σh+1 2 ) 1 ( ) y Ψ (h) 1 ( q x (h) i Ψ (H+1) 1 ( ( K (h) z ) µ (h) ) ( log p Ψ (H+1) 2 ) 1 Ψ (h) 2 x (h) i ) }. ) )

40 Results Results in nonlinear systems identification: 1. artificial dataset 2. drive dataset: by a system with two electric motors that drive a pulley using a flexible belt. input: the sum of voltages applied to the motors output: speed of the belt.

41 RGP GPNARX MLP-NARX LSTM

42 RGP GPNARX MLP-NARX LSTM

43 Avatar control Figure: The generated motion with a step function signal, starting with walking (blue), switching to running (red) and switching back to walking (blue) Videos: Switching between learned speeds Interpolating (un)seen speed Constant unseen speed

44 Regressive dynamics with deep GPs t x h f Instead of coupling f s by encoding the Markov property, we couple them by coupling the f s inputs through another GP with time as input. y = f(x) + ɛ x GP(0, k x (t, t)) f GP(0, k f (x, x)) y Damianou et al., NIPS 2011; Damianou, Titsias and Lawrence, JMLR, 2015

45 Dynamics Dynamics are encoded in the covariance matrix K = k(t, t). We can consider special forms for K. Model individual sequences Model periodic data (missa) (dog) (mocap)

46 Summary Data-driven, model-based approach to real and control problems Gaussian processes: Uncertainty quantification / propagation gives an advantage Deep Gaussian processes: Representation learning + dynamics learning Future work: Deep Gaussian processes + mechanistic information; consider real applications. Thanks!

47 Thanks Thanks to: Neil Lawrence, Michalis Titsias, Carl Henrik Ek, James Hensman, Cesar Lincoln Mattos, Zhenwen Dai, Javier Gonzalez, Tony Prescott, Uriel Martinez-Hernandez, Luke Boorman Thanks to George Karniadakis, Paris Perdikaris for hosting me

48 BACKUP SLIDES 1: Deep GP Optimization - variational inference

49 MAP optimisation? x f 1 h 1 f 2 h 2 f 3 y Joint Distr. = p(y h 2 )p(h 2 h 1 )p(h 1 x) MAP optimization is extremely problematic because: Dimensionality of hs has to be decided a priori Prone to overfitting, if h are treated as parameters Deep structures are not supported by the model s objective but have to be forced [Lawrence & Moore 07] We want: To use the marginal likelihood as the objective: marg. lik. = h 2,h 1 p(y h 2 )p(h 2 h 1 )p(h 1 x) Further regularization tools.

50 Marginal likelihood is intractable Let s try to marginalize out the top layer only: p(h 2 ) = p(h 2 h 1 )p(h 1 )dh 1 = p(h 2 f 2 )p(f 2 h 1 )p(h 1 )df 2 h 1 = p(h 2 f 2 )[ ] p(f 2 h 1 )p(h 1 )dh 1 df 2 } {{ } Intractable! Intractability: h 1 appears non-linearly in p(f 2 h 1 ), inside K 1 (and also the determinant term), where K = k(h 1, h 1 ).

51 Solution: Variational Inference Similar issues arise for 1-layer models. Solution was given by Titsias and Lawrence, A small modification to that solution does the trick in deep GPs too. Extend Titsias method for variational learning of inducing variables in Sparse GPs. Analytic variational bound F p(y x) Approximately marginalise out h Hence obtain the approximate posterior q(h)

52 Inducing points: sparseness, tractability and Big Data h (1) f (1) h (2) f (2) h (30) f (30) h (31) f (31) h (N) f (N)

53 Inducing points: sparseness, tractability and Big Data h (1) f (1) h (2) f (2) h (30) f (30) h (i) u (i) h (31) f (31) h (N) f (N)

54 Inducing points: sparseness, tractability and Big Data h (1) f (1) h (2) f (2) h (30) f (30) h (i) u (i) h (31) f (31) h (N) f (N) Inducing points originally introduced for faster (sparse) GPs But this also induces tractability in our models, due to the conditional independencies assumed Viewing them as global variables extension to Big Data [Hensman et al., UAI 2013]

55 Bayesian regularization F = Data Fit KL (q(h 1 ) p(h 1 )) + } {{ } Regularisation L H (q(h l )) } {{ } Regularisation l=2