# On Contrastive Divergence Learning

Save this PDF as:

Size: px
Start display at page:

## Transcription

2 difference of two divergences (Hinton, 22): n = KL (p p ) KL (p n p ). In learning, we start the Markov chain at the data distribution p and run the chain for a small number n of steps (e.g. n = 1). This greatly reduces both the computation per gradient step and the variance of the estimated gradient, and experiments show that it results in good parameter estimates (Hinton, 22). has been applied effectively to various problems (Chen and Murray, 23; Teh et al., 23; He et al., 24), using Gibbs sampling or hybrid Monte Carlo as the transition operator for the Markov chain. However, it is hard to know how good the parameter estimates really are, since no comparison was done with the real estimates (which are impractical to compute). There has been a little theoretical investigation of the properties of contrastive divergence (MacKay, 21; Williams and Agakov, 22; Yuille, 24), but important questions remain unanswered: Does it converge? If so how fast, and how are its convergence points related to the true estimates? In this paper we provide some theoretical and empirical evidence that contrastive divergence can, in fact, be the basis for a very effective approach for learning random fields. We concentrate on Boltzmann machines, though our results should be more generally valid. First, we show that provides biased estimates in general: for almost all data distributions, the fixed points of are not fixed points of, and vice versa (section 2). We then show, by comparing and in empirical tests, that this bias is small (sections 3 4) and that an effective approach is to use to perform most of the learning followed by a short run of to clean up the solution (section 5). To eliminate sampling noise from our investigations, we use fairly small models (with e.g. 48 parameters) for which we can compute the exact model distribution and the exact distribution at each step of the Markov chain at each stage of learning. Throughout, we take learning to mean exact learning (i.e., with n in the Markov chain) and n learning to mean learning using the exact distribution of the Markov chain after n steps. The sampling noise in real MCMC estimates would create an additional large advantage that favours over learning, because has much lower variance in its gradient estimates. 1 and learning for two types of Boltzmann machine We will concentrate on two types of Boltzmann machine, which are a particular case of the model of eq. (1). In Boltzmann machines, there are v visible units x = (x 1,..., x v ) T that encode a data vector, and h hidden units y = (y 1,..., y h ) T ; all units are binary and take values in {, 1}. Fully visible Boltzmann machines have h = and the visible units are connected to each other. The energy is then E(x; W) = 1 2 xt Wx, where W = (w ij ) is a symmetric v v matrix of real-valued weights (for simplicity, we do not consider biases). We denote such a machine by VBM(v). In VBMs, the log-likelihood has a unique optimum because its Hessian is negative definite. Since E/ w ij = x i x j, learning takes the form w (τ+1) ij = w (τ) ij + η ( x i x j x i x j ) while n learning takes the form w (τ+1) ij = w (τ) ij + η ( x i x j x i x j n ). Here, p n (x; W) = p n = T n p is the nth-step distribution of the Markov chain with transition matrix 1 T started at the data distribution p. Restricted Boltzmann machines (Smolensky, 1986; Freund and Haussler, 1992) have connections only between a hidden and a visible unit, i.e., they form a bipartite graph. The energy is then E(x, y; W) = y T Wx, where we have v visible units and h hidden units, and W = (w ij ) is an h v matrix of real-valued weights. We denote such a machine by RBM(v, h). By making h large, an RBM can be given far more representational power than a VBM, but the log-likelihood can have multiple maxima. The learning is simpler than in a general Boltzmann machine because the visible units are conditionally independent given the hidden units, and the hidden units are conditionally independent given the visible units. One step of Gibbs sampling can therefore be carried out in two half-steps: the first updates all the hidden units and the second updates all the visible units. Equivalently, we can write T = T x T y where t x;j,i = p(x = j y = i; W) and t y;i,j = p(y = i x = j; W). Since E/ w ij = y i x j, learning takes the form ) w (τ+1) ij = w (τ) ij + η ( y i x j p(y x;w) y ix j while n learning takes the form w (τ+1) ij = w (τ) ij + η ( y i x j p(y x;w) 2 Analysis of the fixed points y i x j n ). A probability distribution over v units is a vector of 2 v real-valued components (from to 2 v 1 in bi- 1 Here and elsewhere we omit the dependence of T on the parameter values W to simplify the notation.

3 nary notation) that lives in the 2 v dimensional simplex 2 v = {x R 2v : x i, 2 v i=1 x i = 1}. Each coordinate axis of R 2v corresponds to a state (binary vector). We write such a distribution as p( ), when emphasising that it is a function, or as a vector p, when emphasising that it is a point in the simplex. We define a Markov chain through its transition operator, which is a stochastic 2 v 2 v matrix T. We PSfrag use the replacements Gibbs sampler as the transition operator because of its simplicity and its wide applicability to many distributions. For Boltzmann machines with finite weights, the Gibbs sampler converges to a stationary distribution which is the model distribution p(x; W). For an initial distribution p we then have p n = T n p for n = 1, 2,... ; and p(x; W) = p = T p for any p 2 v, i.e., T = p 1 T where 1 is the vector of ones. VBMs or RBMs define a manifold M within the simplex, parameterised by W, with w ij R (we ignore the case of infinite weights, corresponding to distributions in the intersection of M and the simplex boundary). The learning ( or ) starts at a point in M and follows the (approximate) gradient of the loglikelihood, thus tracing a trajectory within M. For with gradient learning, the fixed points are the zero-gradient points (maxima, minima and saddles), which satisfy g = g where g = E/ W. For n-step, the fixed points satisfy g = g n. In this section we address the theoretical question of whether the fixed points of are fixed points of and vice versa. We show that, in general, 1 p = (1) T w = w = 1 w = 1 Figure 1: The simplex in 4 dimensions of the VBM(2). The model has a single parameter W = w (, ). The tetrahedron represents the simplex, i.e., the set {p R 4 : 1 T p = 1, p }. The tetrahedron corners correspond to the pure states, i.e., the distributions that assign all the probability to a single state. The red vertical segment is the manifold of VBMs, i.e., the distributions reachable by a VBM(2) for w (, ). The estimate of a data distribution is its orthogonal projection on the model manifold. The estimate only agrees with the one for data distributions in the 3 shaded planes in the inset. p : g = g g 1 p : g = g 1 g. We give a brief explanation of a framework for analysing the fixed points of and ; full details appear in Carreira-Perpiñán and Hinton (24). The idea is to fix a value of the weights and so a value of the moments (defined below), determine which data distributions p have such moments (i.e., the opposite of the learning problem) and then determine under what conditions and agree over those distributions. Call G the W 2 v matrix of energy derivatives, defined by G ix = E w i (x; W) where we consider W as a column vector with W elements and the state x takes values, 1,..., 2 v 1 in the case of v binary variables. We can then write the moments s = E W = g p p of a distribution p as s = Gp, i.e., s is a linear function of p. Call T the transition matrix for the sampling operator with stationary distribution p (so we have p = Tp ). In general, both G and T are functions of W. Now consider a fixed value of W and let p be its associated model distribution. Thus its moments are s = Gp. We define two sets P and P 1 that depend on s. First, the set of data distributions p that have those same moments s is: P = p R 2v : Gp = s 1 T p = 1 p. Each distribution in P gives a fixed point of. Likewise, define the set of data distributions p whose distribution p 1 = Tp (first step in the Markov chain, i.e., what 1 uses instead of p ) has the same moments as s : P 1 = p R 2v : GTp = s 1 T p = 1 p. Both P and P 1 are nonempty since p is in both. Now we can reformulate the problem in terms of the sets P and P 1. For example, a distribution with p P and p / P 1 satisfies g = g g 1 and thus gives a fixed point of but not of (that is, the learning rule would move away from such a p ).

4 In general (and ignoring technical details regarding the inequality p ), P and P 1 are linear subspaces of the same dimension because G is full rank (the moments are l.i.) and T is generally full rank. Thus we cannot generally expect P = P 1, so points with bias are the rule; points in P P 1 have no bias but are the exception (the intersection being a lowerdimensional subspace). We can make the statement precise for a given model. For example, for VBM(2) with Gibbs sampling we have G = ( 1) (G happens to be independent of W for VBMs), we can compute T and we can work out the set P P 1 for every s value. The resulting set, which contains all the data distributions for which and have the same fixed points (i.e., no bias), is the union (intersected with the simplex) of the 3 planes: p 11 = ; p 11 = 1 4 ; 3p 1 + p 11 = 1, where we write a distribution as a 4-dimensional vector p = (p p 1 p 1 p 11 ) T, corresponding to the probabilities of the 4 states,..., 11 (see fig. 1). This set has measure zero in the simplex, so is biased for almost every data distribution. A reachable distribution p M is a fixed point for both and, as it is invariant under T (Hinton, 22). This is consistent with the above argument, as p = p P P 1. The distributions of practical interest are typically unreachable because real data are nearly always more complicated than our computationally tractable model of it. In summary, we expect that for almost every data distribution p, the fixed points of are not fixed points of and vice versa. This means that, in general, is a biased learning algorithm. Our argument can be applied to models other than Boltzmann machines, transition operators other than Gibbs sampling, and to n > 1 (writing T n instead of T). What determines whether is biased are the hyperplanes defined by the matrices G and GT. However, nontrivial models (i.e., defining a lower-dimensional manifold) may exist for which is not biased; an example is Gaussian Boltzmann machines (Williams and Agakov, 22) and Gaussian distributions, at least in 2D (Carreira-Perpiñán and Hinton, 24). This analysis does not imply that learning converges (to a stable fixed point); at present, we do not have a proof for this. But if does converge, as it appears to in practice and in all our experiments, it can only converge to a fixed point. Naturally, does converge to its stable fixed points (maxima) from almost everywhere, since it follows the exact gradient of an objective function; in the noisy sampling case that is used in practice, it also converges provided the learning rate η follows a Robbins-Monro schedule (Benveniste et al., 199), since the rule performs stochastic gradient learning. 3 Experiments with fully visible BMs n Since is biased with respect to for almost all data distributions, we now investigate empirically the magnitude of the bias. In all experiments in the paper, and are tested under exactly the same conditions (unless otherwise stated). Both and learning use the same initial weight vectors, the same constant learning rate η =.9 and the same maximum of 1 iterations (which is rarely reached for VBMs), stopping when e < 1 7 (where e = x i x j x i x j is the gradient vector for, and e = x i x j x i x j 1 is the approximate gradient for 1 ). All the experiments use n = 1 step of Gibbs sampling with fixed ordering of the variables for learning, because this should produce the greatest bias (since ). Although each of our simulated models is necessarily small, our empirical results hold for a range of model sizes and conditions, which suggests they may be more generally valid. In this section we consider fully visible Boltzmann machines, denoted VBM(v), which have a single optimum. It appears that has a single convergence point too: for v = 2 we can prove this, and for v {3,..., 1} we checked empirically by running from many different initial weight vectors that it always converged to the same point (up to a small numerical error). Thus, we assume that has a unique convergence point for VBM(v). This allows us to characterise the bias for this model class by sampling many data distributions and computing the convergence point of and of. For a given value of v we sampled a number (as large as computationally feasible) of data distributions uniformly distributed in the simplex in 2 v variables (see Carreira-Perpiñán and Hinton, 24 for details of how to generate these samples). Then we ran and starting with W = because small weights give faster convergence on average. The results for experiments for v {2,..., 1} were qualitatively similar. For v = 2 it was feasible to sample 1 4 data distributions and the results are summarised in figures 2 5. The histograms in figs. 2 3 show the bias is very small for most distributions. Fig. 4 shows that the KL error (for both and ) is small for data distributions near the simplex centre. For less vague data distributions there is more variability, with some distributions having a low error and some having a much higher one. Generally speaking, the distributions having the highest KL error for (i.e., the distributions that are modelled worst by the VBM) are also the ones that have the highest bias. Most of these lie near the boundaries of the simplex, particularly near the corners. However, not all corners and boundaries are far

5 frag replacements KL (p p ) KL (p p ) Figure 2: Histograms of KL (p p ) and KL (p p ) for VBM(2) after learning on 1 4 data distributions, where p and p are the convergence points for and, respectively. The performance of is very close to on average Bias KL (p, p ) Figure 3: Histogram of the symmetrised KL divergence for VBM(2) between the model distributions found by and for all 1 4 data distributions. This shows that the bias of is almost always very small (< 5% of the KL error obtained by for the same distribution; data not shown). However, data distributions do exist that have a relatively large bias..9.8 from the model manifold; this depends on the geometry of the model. In fig. 4 (for v = 2) we can discern the geometry of the simplex in fig. 1. The discontinuity in the slope at a Euclidean distance p u just less than.3 corresponds to the radius of the inscribed sphere. The branch in the scatterplotpsfrag whichreplacements has low error corresponds to the direction passing through the centre and the simplex corner corresponding to the delta distribution of the (1, 1) state (i.e., along the VBM manifold). The other branch which has high error and more data points corresponds to the directions passing through the centre and any of the other 3 corners (i.e., away from the VBM manifold). As v increases, most of the volume of the simplex concentrates at a distance intermediate between the corners and the centre, close to the radius of the inscribed hypersphere. Consequently, a finite uniform sample contains essentially no points near the boundaries of the simplex, which produce the highest bias. For large v, has very small bias for nearly all randomly chosen data distributions. Only those rare distributions near the simplex boundaries produce a significant bias, but these are important in practice: real-world distributions are often near the boundaries (though not as far as the corners) because large parts of the data space have negligible probability. Fig. 5 shows some typical learning curves. Both and decrease in a similar way, converging at the same rate (first-order), taking the same number of iterations to converge to a given tolerance. yields a KL (p ) Euclidean distance p simplex centre Figure 4: KL error for KL (p p ) (red ) and for KL (p p ) (black +) vs Euclidean distance p u between the data distribution and the uniform distribution (centre of the simplex). This Euclidean distance gives a linear ordering of the data distributions (lowest Euclidean distance: p is the uniform distribution, p u = ; highest: p is one of the corners of the simplex, p u = 1 2 v ). For clarity, not all 1 4 distributions are plotted. higher KL error. In the lower example, the curve increases slightly at the end, suggesting it came close to the optimum but then moved away. In summary, we find the bias to be very small for most distributions and to be highest (but still small) with real-world distributions (near the simplex boundary). This bias is small in relative terms (compared to the KL error for ) and absolute terms (compared to the simplex dimensions). and converge at about the same rate, but an iteration costs much

6 KL (p ) KL (p ) Number of iterations Figure 5: Learning curves for and for 2 randomly chosen data distributions. Axes are in log scale. more than a one in a MCMC implementation. 4 Experiments with restricted BMs RBMs are practically more interesting than VBMs, since they have a higher representational power. They also introduce a new element that complicates our study: the existence of multiple local optima of and. This prevents the characterisation of the bias over a large number of data distributions. Instead, we can only afford to select one data distribution (or a few) and try to characterise the set of all optima of and. Given a data distribution, we generate a collection of 6 random initial weight vectors W and compute all the optima of and that are reachable from any of the initial weight vectors or from the optima found by the other learning method. This requires iterating over the current set of optima with and, until no new optima are found. The result is a bipartite, self-consistent convergence graph where an arrow A B indicates that optimum A converges to optimum B under, or optimum A converges to optimum B under. Using many different initial weight vectors should give a representative collection of optima and a Good-Turing estimator (Good, 1953) can be used as a coarse indicator of how many optima we missed. The graph depends on how we decide whether two very similar optima are really the same. The threshold and number of parameter updates have to be carefully chosen so that truly different optima are not confused but two discoveries of the same optimum are not considered different. We found that using the symmetrised KL distance with a threshold of.1 worked well with 1 5 parameter updates. We ran experiments for various values of v and h and various data distributions. Fig. 6 summarises the results for one representative case, corresponding to: v = 6, h = 4. The data distribution was generated from a data set of 4 binary vectors by adding an extra count of.1 to each possible binary vector and renormalising (thus it is close to the simplex boundary). We used 1 5 iterations and 6 different initial weight vectors: 2 random N (, σ =.1), 2 random N (, σ = 1) and 2 random N (, σ = 1). We found 27 optima and 28 optima, and missed about 3 and 4, respectively (Good-Turing estimate). Panel A shows a 2D visualisation of optima (red ) and optima (black +), i.e., the visible-unit distributions p and p, and their convergence relations. The blue is the data distribution p (of the visible variables). To avoid cluttering the plot, pairs of arrows A B are drawn as a single line without arrowheads A B. Note that many such lines are too short to be distinguished. The 2D view was obtained with SNE (Hinton and Roweis, 23) which tries to preserve the local distances. Using a perplexity of 3 to determine the local neighborhood size, SNE gives a better visualisation than projecting onto the first 2 principal components. This panel shows an important and robust phenomenon: and optima typically come in pairs that converge to each other. The optimum always has a greater or equal KL error than its associated optimum but the difference is small. These pairs are to be expected for n when n is large because becomes as n. However they occur very often even for n = 1, as shown. Panels B C show that the choice of initial weights has a much larger effect on the KL error than the bias. 5 Using to initialise The previous experiments show that takes us close to an optimum, but that a small bias remains. An obvious way to eliminate this bias is to use increasing values of n as training progresses. In this section we explore a crude version of this strategy: run until it is close to convergence then use a short run of

7 A B C KL (p p) KL (p p ) KL (p p ) KL (p p ) Figure 6: Empirical study of the convergence points of and for RBM(6, 4) with a single data distribution. A: 2D SNE visualization of the points (: red ; : black +), and convergence relations among them with and (a line without an arrowhead stands for two arrows, to avoid clutter). B: KL error of vs from the same initial weight vectors, for 6 random initial weight vectors. C: histograms of the KL error of and KL (p ) (1 4 its.) CPU cost (1 4 its.) (1 3 its.) Figure 7: Learning curves for - and, where each iteration is scaled to cost as much as 2 iterations. The axes have a log scale, so - is an order of magnitude faster for about the same final KL. to reduce the bias. We call this strategy -. We use a data distribution which is more representative of a real problem. It is located on the simplex boundary and derived from the statistics of all 3 3 patches in images of handwritten digits from the USPS dataset. The 256 intensity levels are thresholded at 15 to produce 9-dimensional binary vectors (thus v = 9). p is the normalised counts of each of these binary vectors in the patches. We used 6 different initial weight vectors: 2 random N (, σ = 1 3 ), 2 random N (, σ = 1) and 2 random N (, σ = 3). For each starting condition, two types of learning were used: learning for 1 4 iterations; and learning for 1 4 iterations, followed by a shorter run of learning. We ran experiments for h {1,..., 8} and found that there is a unique optimum and several optima of varying degrees of bias. If learning was followed by 1 iterations of, all the optima converged to the optimum. Fig. 7 shows the learning curves (i.e., the error KL (p ) as a function of estimated CPU time) for the different methods with h = 8: (blue line), the short run (1 iterations) following (green line) and (red line), for a selected starting condition. We assume that each iteration costs 2 times as much as each iteration (a reasonable estimate for the size of this RBM). - reaches the same error as but at a small fraction of the cost. Note how sharply the - curve drops when we switch to, suggesting good performance can be achieved with very few of the expensive iterations.

8 6 Conclusion Our first result is negative: for two types of Boltzmann machine we have shown that, in general, the fixed points of differ from those of, and thus is a biased algorithm. This might suggest that is not a competitive method for estimation of random fields. Our remaining, empirical results show otherwise: the bias is generally very small, at least for Gibbs sampling, since typically converges very near an optimum. And this small bias can be eliminated by running for a few iterations after, i.e., using as an initialisation strategy for, with a total computation time that is much smaller than that of full-fledged (which will also have slight bias because the Markov chain cannot be run forever). The theoretical analysis of is difficult because of the complicated form that the p 1 (or p n ) distribution takes; p 1 is a moving target that changes with W in a complicated way, and depends on the sampling scheme used (e.g. Gibbs sampling). As a result, very few theoretical results about exist. MacKay (21) gave some examples of bias, but these used unusual sampling operators. Our analysis applies to any model and operator (through the G and T matrices), in particular generally applicable operators such as Gibbs sampling. Williams and Agakov (22) showed that, for 2D Gaussian Boltzmann machines, is unbiased and typically decreases the variance of the estimates. Yuille (24) gives a condition for to be unbiased, though this condition is difficult to apply in practice. One open theoretical problem is whether the exact version of converges (we believe that it does). Assuming we can prove convergence for the exact case, the right tools to use to prove it in the noisy case are probably those of stochastic approximation (Benveniste et al., 199; Yuille, 24). Acknowledgements This research was funded by NSERC and CFI. GEH is a fellow of CIAR and holds a CRC chair. References A. Benveniste, M. Métivier, and P. Priouret. Adaptive Algorithms and Stochastic Approximations. Springer-Verlag, 199. M. Á. Carreira-Perpiñán and G. E. Hinton. On contrastive divergence () learning. Technical report, Dept. of Computer Science, University of Toronto, 24. In preparation. H. Chen and A. F. Murray. Continuous restricted Boltzmann machine with an implementable training algorithm. IEE Proceedings: Vision, Image and Signal Processing, 15(3): , June Y. Freund and D. Haussler. Unsupervised learning of distributions on binary vectors using 2-layer networks. In NIPS, pages , W. Gilks, S. Richardson, and D. J. Spiegelhalter, editors. Markov Chain Monte Carlo in Practice. Chapman & Hall, I. J. Good. The population frequencies of species and the estimation of population parameters. Biometrika, 4(3/4): , Dec X. He, R. S. Zemel, and M. Á. Carreira-Perpiñán. Multiscale conditional random fields for image labeling. In CVPR, pages , 24. G. Hinton and S. T. Roweis. Stochastic neighbor embedding. In NIPS, pages , 23. G. E. Hinton. Training products of experts by minimizing contrastive divergence. Neural Computation, 14(8): , Aug. 22. S. Z. Li. Markov Random Field Modeling in Image Analysis. Springer-Verlag, 21. D. J. C. MacKay. Failures of the one-step learning algorithm. Available online at abstracts/gbm.html, 21. R. M. Neal. Probabilistic inference using Markov chain Monte Carlo methods. Technical Report CRG TR 93 1, Dept. of Computer Science, University of Toronto, Sept Available online at ftp://ftp.cs.toronto.edu/pub/radford/ review.ps.z. P. Smolensky. Information processing in dynamical systems: Foundations of harmony theory. In D. E. Rumelhart and J. L. MacClelland, editors, Parallel Distributed Computing: Explorations in the Microstructure of Cognition. Vol. 1: Foundations, chapter 6. MIT Press, Y. W. Teh, M. Welling, S. Osindero, and G. E. Hinton. Energy-based models for sparse overcomplete representations. Journal of Machine Learning Research, 4: , Dec. 23. C. K. I. Williams and F. V. Agakov. An analysis of contrastive divergence learning in Gaussian Boltzmann machines. Technical Report EDI INF RR 12, Division of Informatics, University of Edinburgh, May 22. G. Winkler. Image Analysis, Random Fields and Markov Chain Monte Carlo Methods. Springer- Verlag, second edition, 22. A. Yuille. The convergence of contrastive divergences. To appear in NIPS, 24.

### Markov chains and Markov Random Fields (MRFs)

Markov chains and Markov Random Fields (MRFs) 1 Why Markov Models We discuss Markov models now. This is the simplest statistical model in which we don t assume that all variables are independent; we assume

### Supporting Online Material for

www.sciencemag.org/cgi/content/full/313/5786/504/dc1 Supporting Online Material for Reducing the Dimensionality of Data with Neural Networks G. E. Hinton* and R. R. Salakhutdinov *To whom correspondence

### STA 4273H: Statistical Machine Learning

STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.cs.toronto.edu/~rsalakhu/ Lecture 6 Three Approaches to Classification Construct

### Machine Learning and Pattern Recognition Logistic Regression

Machine Learning and Pattern Recognition Logistic Regression Course Lecturer:Amos J Storkey Institute for Adaptive and Neural Computation School of Informatics University of Edinburgh Crichton Street,

### Bayesian Statistics: Indian Buffet Process

Bayesian Statistics: Indian Buffet Process Ilker Yildirim Department of Brain and Cognitive Sciences University of Rochester Rochester, NY 14627 August 2012 Reference: Most of the material in this note

### Using Fast Weights to Improve Persistent Contrastive Divergence

Tijmen Tieleman tijmen@cs.toronto.edu Geoffrey Hinton hinton@cs.toronto.edu Department of Computer Science, University of Toronto, Toronto, Ontario M5S 3G4, Canada Abstract The most commonly used learning

### Segmentation of 2D Gel Electrophoresis Spots Using a Markov Random Field

Segmentation of 2D Gel Electrophoresis Spots Using a Markov Random Field Christopher S. Hoeflich and Jason J. Corso csh7@cse.buffalo.edu, jcorso@cse.buffalo.edu Computer Science and Engineering University

### Introduction to Markov Chain Monte Carlo

Introduction to Markov Chain Monte Carlo Monte Carlo: sample from a distribution to estimate the distribution to compute max, mean Markov Chain Monte Carlo: sampling using local information Generic problem

### Visualization by Linear Projections as Information Retrieval

Visualization by Linear Projections as Information Retrieval Jaakko Peltonen Helsinki University of Technology, Department of Information and Computer Science, P. O. Box 5400, FI-0015 TKK, Finland jaakko.peltonen@tkk.fi

### Markov Chains, part I

Markov Chains, part I December 8, 2010 1 Introduction A Markov Chain is a sequence of random variables X 0, X 1,, where each X i S, such that P(X i+1 = s i+1 X i = s i, X i 1 = s i 1,, X 0 = s 0 ) = P(X

### MATHEMATICAL BACKGROUND

Chapter 1 MATHEMATICAL BACKGROUND This chapter discusses the mathematics that is necessary for the development of the theory of linear programming. We are particularly interested in the solutions of a

### max cx s.t. Ax c where the matrix A, cost vector c and right hand side b are given and x is a vector of variables. For this example we have x

Linear Programming Linear programming refers to problems stated as maximization or minimization of a linear function subject to constraints that are linear equalities and inequalities. Although the study

### PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION Introduction In the previous chapter, we explored a class of regression models having particularly simple analytical

### Lecture 6: The Bayesian Approach

Lecture 6: The Bayesian Approach What Did We Do Up to Now? We are given a model Log-linear model, Markov network, Bayesian network, etc. This model induces a distribution P(X) Learning: estimate a set

### Chapter 15 Introduction to Linear Programming

Chapter 15 Introduction to Linear Programming An Introduction to Optimization Spring, 2014 Wei-Ta Chu 1 Brief History of Linear Programming The goal of linear programming is to determine the values of

### arxiv:1312.6062v2 [cs.lg] 9 Apr 2014

Stopping Criteria in Contrastive Divergence: Alternatives to the Reconstruction Error arxiv:1312.6062v2 [cs.lg] 9 Apr 2014 David Buchaca Prats Departament de Llenguatges i Sistemes Informàtics, Universitat

### Planar Tree Transformation: Results and Counterexample

Planar Tree Transformation: Results and Counterexample Selim G Akl, Kamrul Islam, and Henk Meijer School of Computing, Queen s University Kingston, Ontario, Canada K7L 3N6 Abstract We consider the problem

### Gaussian Process Latent Variable Models for Visualisation of High Dimensional Data

Gaussian Process Latent Variable Models for Visualisation of High Dimensional Data Neil D. Lawrence Department of Computer Science, University of Sheffield, Regent Court, 211 Portobello Street, Sheffield,

### MATHEMATICS (CLASSES XI XII)

MATHEMATICS (CLASSES XI XII) General Guidelines (i) All concepts/identities must be illustrated by situational examples. (ii) The language of word problems must be clear, simple and unambiguous. (iii)

### Tangent and normal lines to conics

4.B. Tangent and normal lines to conics Apollonius work on conics includes a study of tangent and normal lines to these curves. The purpose of this document is to relate his approaches to the modern viewpoints

### Linear Threshold Units

Linear Threshold Units w x hx (... w n x n w We assume that each feature x j and each weight w j is a real number (we will relax this later) We will study three different algorithms for learning linear

### Markov Chain Monte Carlo

CS731 Spring 2011 Advanced Artificial Intelligence Markov Chain Monte Carlo Lecturer: Xiaojin Zhu jerryzhu@cs.wisc.edu A fundamental problem in machine learning is to generate samples from a distribution:

### Probabilistic Graphical Models Homework 1: Due January 29, 2014 at 4 pm

Probabilistic Graphical Models 10-708 Homework 1: Due January 29, 2014 at 4 pm Directions. This homework assignment covers the material presented in Lectures 1-3. You must complete all four problems to

### Bayesian Machine Learning (ML): Modeling And Inference in Big Data. Zhuhua Cai Google, Rice University caizhua@gmail.com

Bayesian Machine Learning (ML): Modeling And Inference in Big Data Zhuhua Cai Google Rice University caizhua@gmail.com 1 Syllabus Bayesian ML Concepts (Today) Bayesian ML on MapReduce (Next morning) Bayesian

### Probabilistic Models for Big Data. Alex Davies and Roger Frigola University of Cambridge 13th February 2014

Probabilistic Models for Big Data Alex Davies and Roger Frigola University of Cambridge 13th February 2014 The State of Big Data Why probabilistic models for Big Data? 1. If you don t have to worry about

### Introduction to Deep Learning Variational Inference, Mean Field Theory

Introduction to Deep Learning Variational Inference, Mean Field Theory 1 Iasonas Kokkinos Iasonas.kokkinos@ecp.fr Center for Visual Computing Ecole Centrale Paris Galen Group INRIA-Saclay Lecture 3: recap

### ABC methods for model choice in Gibbs random fields

ABC methods for model choice in Gibbs random fields Jean-Michel Marin Institut de Mathématiques et Modélisation Université Montpellier 2 Joint work with Aude Grelaud, Christian Robert, François Rodolphe,

### 1 Solving LPs: The Simplex Algorithm of George Dantzig

Solving LPs: The Simplex Algorithm of George Dantzig. Simplex Pivoting: Dictionary Format We illustrate a general solution procedure, called the simplex algorithm, by implementing it on a very simple example.

### 2.3 Convex Constrained Optimization Problems

42 CHAPTER 2. FUNDAMENTAL CONCEPTS IN CONVEX OPTIMIZATION Theorem 15 Let f : R n R and h : R R. Consider g(x) = h(f(x)) for all x R n. The function g is convex if either of the following two conditions

### We call this set an n-dimensional parallelogram (with one vertex 0). We also refer to the vectors x 1,..., x n as the edges of P.

Volumes of parallelograms 1 Chapter 8 Volumes of parallelograms In the present short chapter we are going to discuss the elementary geometrical objects which we call parallelograms. These are going to

### Reinforcement Learning with Factored States and Actions

Journal of Machine Learning Research 5 (2004) 1063 1088 Submitted 3/02; Revised 1/04; Published 8/04 Reinforcement Learning with Factored States and Actions Brian Sallans Austrian Research Institute for

### DATA ANALYSIS II. Matrix Algorithms

DATA ANALYSIS II Matrix Algorithms Similarity Matrix Given a dataset D = {x i }, i=1,..,n consisting of n points in R d, let A denote the n n symmetric similarity matrix between the points, given as where

### Lecture 3: Convex Sets and Functions

EE 227A: Convex Optimization and Applications January 24, 2012 Lecture 3: Convex Sets and Functions Lecturer: Laurent El Ghaoui Reading assignment: Chapters 2 (except 2.6) and sections 3.1, 3.2, 3.3 of

### Gaussian Processes in Machine Learning

Gaussian Processes in Machine Learning Carl Edward Rasmussen Max Planck Institute for Biological Cybernetics, 72076 Tübingen, Germany carl@tuebingen.mpg.de WWW home page: http://www.tuebingen.mpg.de/ carl

### Logistic Regression. Vibhav Gogate The University of Texas at Dallas. Some Slides from Carlos Guestrin, Luke Zettlemoyer and Dan Weld.

Logistic Regression Vibhav Gogate The University of Texas at Dallas Some Slides from Carlos Guestrin, Luke Zettlemoyer and Dan Weld. Generative vs. Discriminative Classifiers Want to Learn: h:x Y X features

### Forecasting Trade Direction and Size of Future Contracts Using Deep Belief Network

Forecasting Trade Direction and Size of Future Contracts Using Deep Belief Network Anthony Lai (aslai), MK Li (lilemon), Foon Wang Pong (ppong) Abstract Algorithmic trading, high frequency trading (HFT)

### LINEAR ALGEBRA: NUMERICAL METHODS. Version: August 12,

LINEAR ALGEBRA: NUMERICAL METHODS. Version: August 12, 2000 45 4 Iterative methods 4.1 What a two year old child can do Suppose we want to find a number x such that cos x = x (in radians). This is a nonlinear

### Lecture 11: Graphical Models for Inference

Lecture 11: Graphical Models for Inference So far we have seen two graphical models that are used for inference - the Bayesian network and the Join tree. These two both represent the same joint probability

### BINARY PRINCIPAL COMPONENT ANALYSIS IN THE NETFLIX COLLABORATIVE FILTERING TASK

BINARY PRINCIPAL COMPONENT ANALYSIS IN THE NETFLIX COLLABORATIVE FILTERING TASK László Kozma, Alexander Ilin, Tapani Raiko Helsinki University of Technology Adaptive Informatics Research Center P.O. Box

### Feature Extraction and Selection. More Info == Better Performance? Curse of Dimensionality. Feature Space. High-Dimensional Spaces

More Info == Better Performance? Feature Extraction and Selection APR Course, Delft, The Netherlands Marco Loog Feature Space Curse of Dimensionality A p-dimensional space, in which each dimension is a

### Introduction to machine learning and pattern recognition Lecture 1 Coryn Bailer-Jones

Introduction to machine learning and pattern recognition Lecture 1 Coryn Bailer-Jones http://www.mpia.de/homes/calj/mlpr_mpia2008.html 1 1 What is machine learning? Data description and interpretation

### Statistical Machine Learning

Statistical Machine Learning UoC Stats 37700, Winter quarter Lecture 4: classical linear and quadratic discriminants. 1 / 25 Linear separation For two classes in R d : simple idea: separate the classes

### Dirichlet Processes A gentle tutorial

Dirichlet Processes A gentle tutorial SELECT Lab Meeting October 14, 2008 Khalid El-Arini Motivation We are given a data set, and are told that it was generated from a mixture of Gaussian distributions.

### Advanced Topics in Machine Learning (Part II)

Advanced Topics in Machine Learning (Part II) 3. Convexity and Optimisation February 6, 2009 Andreas Argyriou 1 Today s Plan Convex sets and functions Types of convex programs Algorithms Convex learning

### INTRODUCTION TO NEURAL NETWORKS

INTRODUCTION TO NEURAL NETWORKS Pictures are taken from http://www.cs.cmu.edu/~tom/mlbook-chapter-slides.html http://research.microsoft.com/~cmbishop/prml/index.htm By Nobel Khandaker Neural Networks An

### Three New Graphical Models for Statistical Language Modelling

Andriy Mnih Geoffrey Hinton Department of Computer Science, University of Toronto, Canada amnih@cs.toronto.edu hinton@cs.toronto.edu Abstract The supremacy of n-gram models in statistical language modelling

### Modern Optimization Methods for Big Data Problems MATH11146 The University of Edinburgh

Modern Optimization Methods for Big Data Problems MATH11146 The University of Edinburgh Peter Richtárik Week 3 Randomized Coordinate Descent With Arbitrary Sampling January 27, 2016 1 / 30 The Problem

### Fundamentals of Operations Research Prof.G. Srinivasan Department of Management Studies Lecture No. # 03 Indian Institute of Technology, Madras

Fundamentals of Operations Research Prof.G. Srinivasan Department of Management Studies Lecture No. # 03 Indian Institute of Technology, Madras Linear Programming solutions - Graphical and Algebraic Methods

### CCNY. BME I5100: Biomedical Signal Processing. Linear Discrimination. Lucas C. Parra Biomedical Engineering Department City College of New York

BME I5100: Biomedical Signal Processing Linear Discrimination Lucas C. Parra Biomedical Engineering Department CCNY 1 Schedule Week 1: Introduction Linear, stationary, normal - the stuff biology is not

### Computing with Finite and Infinite Networks

Computing with Finite and Infinite Networks Ole Winther Theoretical Physics, Lund University Sölvegatan 14 A, S-223 62 Lund, Sweden winther@nimis.thep.lu.se Abstract Using statistical mechanics results,

### Overview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model

Overview of Violations of the Basic Assumptions in the Classical Normal Linear Regression Model 1 September 004 A. Introduction and assumptions The classical normal linear regression model can be written

### Convexity in R N Main Notes 1

John Nachbar Washington University December 16, 2016 Convexity in R N Main Notes 1 1 Introduction. These notes establish basic versions of the Supporting Hyperplane Theorem (Theorem 5) and the Separating

### Neural Networks for Machine Learning. Lecture 13a The ups and downs of backpropagation

Neural Networks for Machine Learning Lecture 13a The ups and downs of backpropagation Geoffrey Hinton Nitish Srivastava, Kevin Swersky Tijmen Tieleman Abdel-rahman Mohamed A brief history of backpropagation

### 5.1 Bipartite Matching

CS787: Advanced Algorithms Lecture 5: Applications of Network Flow In the last lecture, we looked at the problem of finding the maximum flow in a graph, and how it can be efficiently solved using the Ford-Fulkerson

### Tutorial on Markov Chain Monte Carlo

Tutorial on Markov Chain Monte Carlo Kenneth M. Hanson Los Alamos National Laboratory Presented at the 29 th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Technology,

### Lecture 3: Linear methods for classification

Lecture 3: Linear methods for classification Rafael A. Irizarry and Hector Corrada Bravo February, 2010 Today we describe four specific algorithms useful for classification problems: linear regression,

### Math 018 Review Sheet v.3

Math 018 Review Sheet v.3 Tyrone Crisp Spring 007 1.1 - Slopes and Equations of Lines Slopes: Find slopes of lines using the slope formula m y y 1 x x 1. Positive slope the line slopes up to the right.

### Manifold Learning Examples PCA, LLE and ISOMAP

Manifold Learning Examples PCA, LLE and ISOMAP Dan Ventura October 14, 28 Abstract We try to give a helpful concrete example that demonstrates how to use PCA, LLE and Isomap, attempts to provide some intuition

### Applications to Data Smoothing and Image Processing I

Applications to Data Smoothing and Image Processing I MA 348 Kurt Bryan Signals and Images Let t denote time and consider a signal a(t) on some time interval, say t. We ll assume that the signal a(t) is

### The Not-Formula Book for C1

Not The Not-Formula Book for C1 Everything you need to know for Core 1 that won t be in the formula book Examination Board: AQA Brief This document is intended as an aid for revision. Although it includes

### Moving Average Filters

CHAPTER 15 Moving Average Filters The moving average is the most common filter in DSP, mainly because it is the easiest digital filter to understand and use. In spite of its simplicity, the moving average

### Classifiers & Classification

Classifiers & Classification Forsyth & Ponce Computer Vision A Modern Approach chapter 22 Pattern Classification Duda, Hart and Stork School of Computer Science & Statistics Trinity College Dublin Dublin

### Lecture 1: August 27

CS294 Markov Chain Monte Carlo: Foundations & Applications Fall 2009 Lecture 1: August 27 Lecturer: Prof. Alistair Sinclair Scribes: Alistair Sinclair Disclaimer: These notes have not been subjected to

### Least-Squares Intersection of Lines

Least-Squares Intersection of Lines Johannes Traa - UIUC 2013 This write-up derives the least-squares solution for the intersection of lines. In the general case, a set of lines will not intersect at a

### Visualizing Higher-Layer Features of a Deep Network

Visualizing Higher-Layer Features of a Deep Network Dumitru Erhan, Yoshua Bengio, Aaron Courville, and Pascal Vincent Dept. IRO, Université de Montréal P.O. Box 6128, Downtown Branch, Montreal, H3C 3J7,

### Orthogonal Projections

Orthogonal Projections and Reflections (with exercises) by D. Klain Version.. Corrections and comments are welcome! Orthogonal Projections Let X,..., X k be a family of linearly independent (column) vectors

### General theory of stochastic processes

CHAPTER 1 General theory of stochastic processes 1.1. Definition of stochastic process First let us recall the definition of a random variable. A random variable is a random number appearing as a result

### Introduction to Flocking {Stochastic Matrices}

Supelec EECI Graduate School in Control Introduction to Flocking {Stochastic Matrices} A. S. Morse Yale University Gif sur - Yvette May 21, 2012 CRAIG REYNOLDS - 1987 BOIDS The Lion King CRAIG REYNOLDS

### ELEC-E8104 Stochastics models and estimation, Lecture 3b: Linear Estimation in Static Systems

Stochastics models and estimation, Lecture 3b: Linear Estimation in Static Systems Minimum Mean Square Error (MMSE) MMSE estimation of Gaussian random vectors Linear MMSE estimator for arbitrarily distributed

### Vector and Matrix Norms

Chapter 1 Vector and Matrix Norms 11 Vector Spaces Let F be a field (such as the real numbers, R, or complex numbers, C) with elements called scalars A Vector Space, V, over the field F is a non-empty

### Cheng Soon Ong & Christfried Webers. Canberra February June 2016

c Cheng Soon Ong & Christfried Webers Research Group and College of Engineering and Computer Science Canberra February June (Many figures from C. M. Bishop, "Pattern Recognition and ") 1of 31 c Part I

### The Classical Linear Regression Model

The Classical Linear Regression Model 1 September 2004 A. A brief review of some basic concepts associated with vector random variables Let y denote an n x l vector of random variables, i.e., y = (y 1,

### called Restricted Boltzmann Machines for Collaborative Filtering

Restricted Boltzmann Machines for Collaborative Filtering Ruslan Salakhutdinov rsalakhu@cs.toronto.edu Andriy Mnih amnih@cs.toronto.edu Geoffrey Hinton hinton@cs.toronto.edu University of Toronto, 6 King

### Probability & Statistics Primer Gregory J. Hakim University of Washington 2 January 2009 v2.0

Probability & Statistics Primer Gregory J. Hakim University of Washington 2 January 2009 v2.0 This primer provides an overview of basic concepts and definitions in probability and statistics. We shall

### Combining information from different survey samples - a case study with data collected by world wide web and telephone

Combining information from different survey samples - a case study with data collected by world wide web and telephone Magne Aldrin Norwegian Computing Center P.O. Box 114 Blindern N-0314 Oslo Norway E-mail:

### ARE211, Fall2012. Contents. 2. Linear Algebra Preliminary: Level Sets, upper and lower contour sets and Gradient vectors 1

ARE11, Fall1 LINALGEBRA1: THU, SEP 13, 1 PRINTED: SEPTEMBER 19, 1 (LEC# 7) Contents. Linear Algebra 1.1. Preliminary: Level Sets, upper and lower contour sets and Gradient vectors 1.. Vectors as arrows.

### I. Solving Linear Programs by the Simplex Method

Optimization Methods Draft of August 26, 2005 I. Solving Linear Programs by the Simplex Method Robert Fourer Department of Industrial Engineering and Management Sciences Northwestern University Evanston,

### ESTIMATION OF SMALL FAILURE PROBABILITIES IN HIGH DIMENSIONS BY ADAPTIVE LINKED IMPORTANCE SAMPLING

ESTIMATION OF SMALL FAILURE PROBABILITIES IN HIGH DIMENSIONS BY ADAPTIVE LINKED IMPORTANCE SAMPLING L.S. Katafygiotis 1 and K.M. Zuev 1 1 Hong Kong University of Science and Technology, Department of Civil

### Chapter 6. Cuboids. and. vol(conv(p ))

Chapter 6 Cuboids We have already seen that we can efficiently find the bounding box Q(P ) and an arbitrarily good approximation to the smallest enclosing ball B(P ) of a set P R d. Unfortunately, both

### Markov random fields and Gibbs measures

Chapter Markov random fields and Gibbs measures 1. Conditional independence Suppose X i is a random element of (X i, B i ), for i = 1, 2, 3, with all X i defined on the same probability space (.F, P).

### α = u v. In other words, Orthogonal Projection

Orthogonal Projection Given any nonzero vector v, it is possible to decompose an arbitrary vector u into a component that points in the direction of v and one that points in a direction orthogonal to v

### Linear Systems and Gaussian Elimination

Eivind Eriksen Linear Systems and Gaussian Elimination September 2, 2011 BI Norwegian Business School Contents 1 Linear Systems................................................ 1 1.1 Linear Equations...........................................

### Polarization codes and the rate of polarization

Polarization codes and the rate of polarization Erdal Arıkan, Emre Telatar Bilkent U., EPFL Sept 10, 2008 Channel Polarization Given a binary input DMC W, i.i.d. uniformly distributed inputs (X 1,...,

### Counting the number of Sudoku s by importance sampling simulation

Counting the number of Sudoku s by importance sampling simulation Ad Ridder Vrije University, Amsterdam, Netherlands May 31, 2013 Abstract Stochastic simulation can be applied to estimate the number of

### Probabilistic Latent Semantic Analysis (plsa)

Probabilistic Latent Semantic Analysis (plsa) SS 2008 Bayesian Networks Multimedia Computing, Universität Augsburg Rainer.Lienhart@informatik.uni-augsburg.de www.multimedia-computing.{de,org} References

### PSS 27.2 The Electric Field of a Continuous Distribution of Charge

Chapter 27 Solutions PSS 27.2 The Electric Field of a Continuous Distribution of Charge Description: Knight Problem-Solving Strategy 27.2 The Electric Field of a Continuous Distribution of Charge is illustrated.

### Learning. Artificial Intelligence. Learning. Types of Learning. Inductive Learning Method. Inductive Learning. Learning.

Learning Learning is essential for unknown environments, i.e., when designer lacks omniscience Artificial Intelligence Learning Chapter 8 Learning is useful as a system construction method, i.e., expose

### Shiqian Ma, SEEM 5121, Dept. of SEEM, CUHK 1. Chapter 2. Convex Optimization

Shiqian Ma, SEEM 5121, Dept. of SEEM, CUHK 1 Chapter 2 Convex Optimization Shiqian Ma, SEEM 5121, Dept. of SEEM, CUHK 2 2.1. Convex Optimization General optimization problem: min f 0 (x) s.t., f i (x)

### Example: Credit card default, we may be more interested in predicting the probabilty of a default than classifying individuals as default or not.

Statistical Learning: Chapter 4 Classification 4.1 Introduction Supervised learning with a categorical (Qualitative) response Notation: - Feature vector X, - qualitative response Y, taking values in C

### Mixtures of Robust Probabilistic Principal Component Analyzers

Mixtures of Robust Probabilistic Principal Component Analyzers Cédric Archambeau, Nicolas Delannay 2 and Michel Verleysen 2 - University College London, Dept. of Computer Science Gower Street, London WCE

### CSC321 Introduction to Neural Networks and Machine Learning. Lecture 21 Using Boltzmann machines to initialize backpropagation.

CSC321 Introduction to Neural Networks and Machine Learning Lecture 21 Using Boltzmann machines to initialize backpropagation Geoffrey Hinton Some problems with backpropagation The amount of information

### Lecture 1 Newton s method.

Lecture 1 Newton s method. 1 Square roots 2 Newton s method. The guts of the method. A vector version. Implementation. The existence theorem. Basins of attraction. The Babylonian algorithm for finding

### Blind Deconvolution of Corrupted Barcode Signals

Blind Deconvolution of Corrupted Barcode Signals Everardo Uribe and Yifan Zhang Advisors: Ernie Esser and Yifei Lou Interdisciplinary Computational and Applied Mathematics Program University of California,

### Introduction to Statistical Machine Learning

CHAPTER Introduction to Statistical Machine Learning We start with a gentle introduction to statistical machine learning. Readers familiar with machine learning may wish to skip directly to Section 2,

### Section Notes 3. The Simplex Algorithm. Applied Math 121. Week of February 14, 2011

Section Notes 3 The Simplex Algorithm Applied Math 121 Week of February 14, 2011 Goals for the week understand how to get from an LP to a simplex tableau. be familiar with reduced costs, optimal solutions,

### 3. INNER PRODUCT SPACES

. INNER PRODUCT SPACES.. Definition So far we have studied abstract vector spaces. These are a generalisation of the geometric spaces R and R. But these have more structure than just that of a vector space.

### A Foundation for Geometry

MODULE 4 A Foundation for Geometry There is no royal road to geometry Euclid 1. Points and Lines We are ready (finally!) to talk about geometry. Our first task is to say something about points and lines.

### Recurrent Neural Networks

Recurrent Neural Networks Neural Computation : Lecture 12 John A. Bullinaria, 2015 1. Recurrent Neural Network Architectures 2. State Space Models and Dynamical Systems 3. Backpropagation Through Time