Machine Learning and Applications in

Transcription

1

2

3 Machine Learning and Applications in Performance Based Budgeting Abolfazl Shabani

4 Forecasting Predicting and planning future estimate tools which relies on the past and certain data is called forecasting.

5 Forecasting Types Forecasting Qualitative: Based on Experienced and knowledgeable advisors Quantitative: Based on Statistical and mathematical Knowledge

6 What Affects Forecasting? Trends Seasonality cyclical elements Autocorrelation random variation.

7 Drew Conway (2010): THE DATA SCIENCE VENN DIAGRAM

8 What is Machine Learning? Arthur Samuel(1959): Field of study that gives computers the ability to learn without being explicitly programmed. Tom M. Mitchell(1998): A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.

9 What is Machine Learning? Machine learning is the science of designing efficient and accurate prediction algorithms which receive the past data and learn from experiences in it to do forecasting.

10 Efficient Machine Learning Algorithms and Their Complexity Machine Learning Algorithms Complexity Time Complexity Space Complexity Sample Data Complexity

11 Properties of a Good Learning Algorithm Efficient in time; in space; in learning sample. General to handle a variety of different learning problems. Accurate to deal with large-scale data sets.

12 Large-Scale Data Problems Complex curve simple curve More accurate, but hard to define and hard to generalize. Less accurate, but Easier in defining and generalizing.

13 Why we need machine learningin PBB? Store forecasted and actual data. We have a huge amount of past data. We want to learning from these data. Compare the forecasted quantities with actual ones. Predict future quantities by the Algorithm. Design an algorithm based on what had been learnt.

14 Diagram of a typical learning problem

15 Applications of Learning Algorithms Classification Regression Ranking Clustering Dimensionality reduction

16 Definitions and Terminology Examples: Items or instances of data used for learning or evaluation. Labels: Values or categories assigned to examples. Features: The set of attributes, often represented as a vector, associated to an example. Feathers have an important role in learning. They can make learning easy or difficult, effective or ineffective. Good features are correlated with labels. Training sample: Examples used to train a learning algorithm.

17 Definitions and Terminology Validation sample: Examples used to tune the parameters of a learning algorithm when working with labeled data. Learning algorithms typically have one or more free parameters, and the validation sample is used to select appropriate values for these model parameters. Test sample: Examples used to evaluate the performance of a learning algorithm. The test sample is separate from the training and validation data and is not made available in the learning stage. Loss function: A function that measures the difference, or loss, between a predicted label and a true label. Hypothesis set: A set of functions mapping features (feature vectors) to the set of labels Y.

18 Learning Scenarios Supervised learning Unsupervised learning Semi-supervised learning Transductive inference On-line learning Reinforcement learning Active learning

19 Bayesian hierarchical modeling: Inverse probability (i.e. Bayes Theorem) allows us to infer unknown quantities, adapt our models, make predictions and learn from data.

20 Bayesian hierarchical modeling: Inverse probability (i.e. Bayes Theorem) allows us to infer unknown quantities, adapt our models, make predictions and learn from data. From Bayes theorem one can infer that the relation between probability of certain data and probability of unknown future is ( ) P unknown data = ( ) ( ) P unknown P data unknown P ( data )

21 Comparing models by the Bayesian rule and prediction Given two modelsm 1 and M 2with parametersθ 1, θ 2 and associated parameter priors, P ( x, θ M ) = P ( x θ, M ) P ( θ M ) and P ( x, θ M ) = P ( x θ, M ) P ( θ M ) = and We compare the performance of the models in fitting a set of data D = { x x }. Let P ( M ), ( i = 1,2 ), be the prior beliefs in the 1,..., N i appropriateness of each model. The model posterior probability P is ( D M i ) P ( M i ) 2 P ( M i D ) = where P ( D ) = P ( D M i ) P ( M i ). P ( D ) i = 1 The model likelihood of M i is given by =. P ( D M ) P ( D θ, M ) P ( θ M ) dθ i i i i i i

22 Comparing models by the Bayesian rule and prediction Now we are able to compare two models by Bayes' factor P( D M i ) P ( D M j ) in this way: P ( M i D ) P ( D M i ) P ( M i ) = P ( M D ) P ( D M ) P ( M ) j By the Bayes rule the prediction of a variable x via model M is P ( x D, M ) P ( x θ ) P ( θ D, M ) dθ j =. j

23 Multivariate Gaussian distribution: X = X, 1, X k is said to have the multivariate normal (Gaussian) distribution if every linear combination of its components Y= a1 X1 + + a k X k is normally A random vector ( ) distributed.

24 Gaussian Processes Gaussian process which generalizes multivariate Gaussian distributions over finite dimensional vectors to infinite dimensionality is a stochastic process in which all the finite-dimensional distributions are multivariate Gaussian distributions for any finite choice of variables.

25 Properties of Gaussian Process Gaussian process is distribution over functions. Gaussian processes are non-parametric. Gaussian processes can be used in a Bayesian setting. Gaussian process is fully specified by a mean function and a covariance function..

26 Definition of Gaussian Process Let f = ( f ( x1 ), f( x2 ),..., f ( x N )) be an N-dimensional vector of function values evaluated at N points x i X. P( f ) is a Gaussian N process if for any finite subset { x } 1 X i i=, the marginal distribution over that finite subset has a multivariate Gaussian distribution.

27 Definition of Gaussian Process We say a real process ( ) f x is Gaussian process distributed with a mean function m( x) and a covariance function C( x, x ), written as f GP ( m ( x ), C ( x, x ) ). Where the argument x of the random function f ( x ) plays the role of the index.

28 Examples of covariance functions

29 Applications of Gaussian Processes regression classification

30 Regression by Gaussian processes: Recover a functional dependency y = f ( x ) + εfrom N observed N training data points { ( x, y, where y R i i )} 1 is the noisy observed i = i d output at input location R. The idea of Gaussian process xi regression is to place a prior directly on the space of functions without parametrizing the function. i i

31 Regression by Gaussian processes: Likelihood Function: 2 Assuming independent and normally distributed noise termsε N(0, σ ). i noise The likelihood model on an output vector y R N and an input matrix N d X R will be Y f X N f I 2, ( x, σ noise )

32 Regression by Gaussian processes: Predictive Distribution: The Goal of regression is to make an output prediction for a novel input x *, given a set of input-output training points N {( x i, y i )} i = 1. If the marginal distribution over the training inputs is f X N (0, k ), the marginal distribution over training outputs will be 2 ( ) = ( X ) ( X ) X = (0, +σ noise ) P Y X P Y f P f df N K I

33 Regression by Gaussian processes: Predictive Distribution: The joint distribution over sets of training points Y and the quantity we wish to predict y * is ( N + 1) ( N + 1) given by P ( Y, y * X, x * ) = N (0, C ), where C is the joint covariance matrix given by C K + σ I k 2 noise X, x * = * k k ( x, x ) + σ T X, x * * * X, x 1 * N * 2 noise k = ( k( x, x ),..., k( x, x )) N

34 Regression by Gaussian processes: Predictive Distribution: Using a standard Gaussian property on computing conditional distribution from a joint Gaussian distribution, the predictive distribution is given by P ( y x, X, Y ) = N ( µ, σ ) 2 * * * * T 2 1 such that µ * = k X, x ( K + σ ) * noise I Y is the mean function and σ = k ( x, x ) k ( K + σ I ) k + σ 2 T * * * X, x noise X, x noise * * is the covariance function of the posterior process in for any novel input.

35 Regression by Gaussian processes: Optimal point prediction: Let true y be the true value of y, y predictbe the predicted value for y, and L( y, y ) be the loss function. The optimal point prediction can be true predict computedby minimizing the expected loss as y x = arg min L( y, y ) P ( y x, X, Y ) dy pre dict optim al * y * predict * * *

36 Gaussian Processes in Machine Learning Gaussian processes were first formalized for machine learning tasks by Williams and Rasmussen and Neal(1996). Learning with Gaussian Processes is equivalent to place priors on hyper-parameters that covariance function depends on, to get posterior distribution of hyper-parameters and finally optimize these hyper-parameters directly.

37 Gaussian Process Reinforcement Learning: Gaussian process reinforcement learning is a class of learning problems concerned with Gaussian process reinforcement learning is a class of learning problems concerned with achieving long-term goals in unfamiliar, uncertain, and dynamic environments.

38 Gaussian Process Reinforcement Learning Methods Model-based methods: Gaussian processes are used to learn the transition and reward model of the Markov decision process underlying the Reinforcement Learning problem. Model-free methods: no explicit representation of the Markov decision process is Model-free methods: no explicit representation of the Markov decision process is maintained. Gaussian processes are used to learn either the Markov decision process value function, state-action value function, or some other quantity that may be used to solve the Markov decision process.

39 An example of Machine learning in PBB B(t) = [X 1(t),X 2(t),...,X n(t)]and B (t) = [X 1(t),X 2(t),...,X n (t)] are respectively the actual and forecasted budgets. X (t) Define n to be the proportional change of a forecasted financial variable X n (t) to X n (t) its pervious actual X n (t-1): X n (t) = 1 X (t-1) n

40 An example of Machine learning in PBB Define B ( t) [ X (t), X (t),..., X (t)] = to be the change of the forecast budget. 1 2 n Assign a class label A(t)to the vector B ( t ) by the following way: 1 Y ( t) < Y ( t) A( t) = ; 0 Y ( t) Y ( t) Where Y ( t ) and Y ( t ) are the actual and forecast values of a budget performance indicator. The label will be 0if the actual budget under-performs the forecast budget and it will be 1 otherwise.

41 An example of Machine learning in PBB

42 References: 1. MalteKussand Carl Edward Rasmussen, Assessing Approximations for Gaussian Process Classification, Neural Information Processing Systems, C. Rasmussen and C. Williams, Gaussian Processes for Machine Learning, MIT Press, C. Rasmussen, Gaussian Processes in Machine Learning, Advanced Lectures on Machine Learning: ML Summer Schools, Canberra, Australia, Claude Sammut, Geoffrey I. Webb, Encyclopedia of Machine Learning, Springer 5. Kai Leung Yip, Determining the Accuracy of Budgets, Unitec Insitute of Technology, David Barber, Bayesian Reasoning and Machine Learning, Cambride University Press, 2012