Adaptive Classification by Hybrid EKF with Truncated Filtering: Brain Computer Interfacing
|
|
- Leslie Daniels
- 8 years ago
- Views:
Transcription
1 Adaptive Classification by Hybrid EKF with Truncated Filtering: Brain Computer Interfacing Ji Won Yoon, Stephen J. Roberts, Matthew Dyson 2, and John Q. Gan 2 Pattern Analysis and Machine Learning Group Department of Engineering Science, University of Oxford, UK jwyoon,sjrob}@robots.ox.ac.uk parg 2 Department of Computing and Electronic Systems University of Essex, UK mdyson,jqgan}@essex.ac.uk Abstract. This paper proposes a robust algorithm for adaptive modelling of EEG signal classification using a modified Extended Kalman Filter (EKF). This modified EKF combines Radial Basis functions (RBF) and Autoregressive (AR) modeling and obtains better classification performance by truncating the filtering distribution when new observations are very informative. Keywords: Extended Kalman Filter, Informative Observation, Logistic Classification, Truncated Filtering. Introduction The goal of Brain Computer Interfacing (BCI) is to enable people with severe neurological disabilities to operate computers by manipulation of the brain s electrical activity rather than by physical means. It is known that the generation and control of electrical brain activity (the electroencephalogram or EEG) signals for a BCI system often requires extensive subject training before a reliable communication channel can be formed between a human subject and a computer interface [,2. In order to reduce the overheads of training and, importantly, to cope with new subjects, adaptive approaches to the core data modelling have been developed for BCI systems. Such adaptive approaches differ from the typical methodology, in which an algorithm is trained off-line on retrospective data, in so much that the process of learning is continuously taking place rather than being confined to a section of training data. In the signal processing and machine learning communities this is referred to as sequential classification. Previous research in this area applied to BCI data has used state space modelling of the time series [3,4,5. In particular Extended Kalman Filter (EKF) approaches have been effective [3,5. In these previous models, based upon the EKF, the variances of the observation noise and hidden state noise are re-estimated using C. Fyfe et al. (Eds.): IDEAL 28, LNCS 5326, pp , 28. c Springer-Verlag Berlin Heidelberg 28
2 Adaptive Classification by Hybrid EKF with Truncated Filtering 37 a maximum-likelihood style framework, ultimately due to the work of Jazwinski [7. In previous work [6, we developed a simple, computationally practial, modification of the conventional EKF approach in which we marginalise (integrate out) the unknown variance parameters for variances by applying Bayesian conjugate priors [8. This enabled us to avoid the need for spurious parameter estimation steps which decrease the robustness of the algorithm in highly non-stationary and noisy regions of data. The EKF offers one framework for approximating the non-linear linkage between observations and decisions (via the logistic function). In all this prior work, however, the output function assumed the target labels (the decision classes) to be time independent, i.e. there is no explict Markov dependency from one decision to the next. This is a poor model for the characteristics of the BCI interface, at the sample rate we operate at (over.2s intervals), in which sequences of labels of the same class persist for several seconds before making a transition. The algorithm presented here employs both a set of non-linear basis functions (thus making it a dynamic Radial Basis Function (RBF) classifier) and an autoregressive (AR) model to tackle this Markov dependency of the labeling. In addition, our approach modifies a filtering step by truncating the filtering distribution in terms of new observations. This modified filtering step provides a robust algorithm when given labels (i.e. as training data) are very informative. Moreover, this algorithm does not have to approximate the convolution of a logistic function and Normal distribution using a probit function, which can lead to poor performance for the EKF [5,6. 2 Mathematical Model We consider an observed input stream (i.e. a time series) x t at time t =, 2,,T which we project into a non-linear basis space, x t ϕ(x t ). We also (partially) observe z t, } such that Pr(z t = x t, w t )=g(f(x t, w t )) where g( ) can be logistic model or probit link function which takes the latent variables to the outcome decision variable. In this paper, we use the logistic function, i.e. g(s) =/( + e s ). The state space model for the BCI system can hence be regarded as a hierarchical model in that the noise of observations influences the model indirectly through the logistic regression model g( ). Such indirect influence makes for a more complicated model, but we can circumvent much of this complexity by forming a three stage state space model and by introducing an auxiliary variable y t. The latter variable acts so as to link the indirect relationships between observations and the logistic regression model given by p(z t y t )=g(y t ) zt ( g(y t )) zt y t = abw T t ϕ t(x t )+( a)y t + v t w t = Aw t + Lh t () where v t N(,κ)andh t N(,τI). The variable a denotes a convex mixing ratio between the Radial Basis Function (RBF) and the Autoregressive (AR) model. To simplify notation, we use ϕ t instead of ϕ t (x t ) i.e.
3 372 J.W. Yoon et al. ϕ t = ϕ t (x t )= [ x T t, φ() t (x t )} T,, φ (N b) t (x t )} T, T (2) and φ (i) t (x t ) is the response of the ith Gaussian basis function for i,,n b } at time t. Here, N b is the number of Gaussian basis functions and the basis functions are chosen in random. In adaptive classification, there are two steps in our state space model which perform model inference: Prediction: p(w t,y t z :t, x :t ) Filtering: p(w t,y t z :t, x :t ) after which we obtain ȳ t t = y t y t [ w t p(w t,y t z :t, x :t )dw t dy t. 2. Prediction p(w t,y t z :t, x :t ) = p(w t,y t, w t,y t z :t, x :t )dw t dy t = p(y t w t,y t )p(w t w t )p(w t,y t z :t, x :t )dw t dy t [ [ = p(y t w t,y t,κ)p(κ)dκ p(w t w t,τ)p(τ)dτ κ τ p(w t,y t z :t, x :t )dw t dy t (3) ( ( ) ) ( α N y t ; aϕ T ( ) ) t Bw t +( a)y t, κ α N w t ; Aw t, τ LL T N = N = N ([ yt w t ([ yt ([ yt where w t w t ; β κ ) ; μ t t,σ t t dw t dy t [ [ ) a aϕ T t BA yt,σ N A w t ; Sμ t t,σ+ SΣ t t S T ) [ a aϕ T t BA A [ Σy Σ,Σ = c, Σ c Σ w ([ yt w t ; μ t t,σ t t ) dw t dy t S = ( ) α Σ y = E[(y t E(y t ))(y t E(y t )) T =a 2 ϕ T ( ) t BL τ α L T B T ϕ t + κ, ( α Σ w = E[(w t E(w t ))(w t E(w t )) T =L τ ) L T, ( ) α Σ c = E[(y t E(y t ))(w t E(w t )) T =aϕ T t BL τ L T. (3) β κ (4) 2.2 Filtering In the filtering step, our EKF uses a new observation z t. In general, p(w t,y t z :t, x :t ) is used with the following form:
4 Adaptive Classification by Hybrid EKF with Truncated Filtering 373 p(w t,y t z :t, x :t ) p(z t, w t,y t z :t, x :t ) = p(z t y t )p(w t,y t z :t, x :t ). (4) It is difficult to obtain a full distribution for the EKF filtering step so previous approaches approximated the convolution between the normal distribution and a logistic function using the probit function [5,6. However, if the new observation is very informative, filtering can be improved without the need for the approximation, which is one of the major sources of non-linearity in logistic EKF filtering. We can obtain better filtering distributions by truncation as we do not need the convolution approximation. Hence we obtain p(w t,y t z :t, x :t ) p(wt,y = p(w t,y t z :t, x :t,z t )= t z :t, x :t )I yt (y t ) if z t = p(w t,y t z :t, x :t )I yt<(y t ) if z t =, if yt satisfies C where I C (y t )=, otherwise, p(wt y = t, z :t, x :t )p(y t z :t, x :t )I yt (y t ) if z t = p(w t y t, z :t, x :t )p(y t z :t, x :t )I yt<(y t ) if z t = p(wt y t, z :t, x :t )[p(y t z :t, x :t )I yt (y t ) if z t = p(w t y t, z :t, x :t )[p(y t z :t, x :t )I yt<(y t ) if z t = N (wt ;ˆμ = w,t t, ˆΣ w,t t ) [ N (y t ; μ y,t t,σ y,t t )I yt (y t ) if z t = N (w t ;ˆμ w,t t, ˆΣ w,t t ) [ N (y t ; μ y,t t,σ y,t t )I yt<(y t ) if z t = ˆμw,t t = μ w,t t + Σ c,t t Σ y,t t where (y t μ y,t t ) ˆΣ w,t t = Σ w,t t Σc,t t T Σ y,t t Σ c,t t N(w t ;ˆμ w,t t, ˆΣ w,t t )N (y t ; μ y,zt,σ y,zt )forz t, } where μ y,zt= = E N (yt;μ y,t t,σ y,t t )I yt (y t)(y t ) μ y,zt= = E N (yt;μ y,t t,σ y,t t )I yt <(y t)(y t ) Σ y,zt= = V N (yt;μ y,t t,σ y,t t )I yt (y t)(y t ) Σ y,zt= = V N (yt;μ y,t t,σ y,t t )I yt <(y t)(y t ) (5) and E( ) and V( ) denote mean and covariance of the truncated normal distribution respectively. The calculation of the mean and covariance of the truncated normal distribution is described in the appendix. Therefore, we have p(w t,y t z :t, x :t) =N yt w t yt μy,t t ; μ t t,σ t t = N ;, w t μ w,t t Σy,t t Σ c,t t Σ c,t t Σ w,t t (6)
5 374 J.W. Yoon et al. where μ y,t t = μ yt,zt Σ y,t t = Σ yt,zt Σ c,t t = Σ c,t t Σ y,t t Σyt,zt μ w,t t = μ w,t t + Σ c,t t Σ y,t t (μy,zt μ y,t t ) Σ w,t t = Σ w,t t + Σ c,t t Σ y,t t Σy,zt Σ y,t t Σ c,t t Σ c,t t Σ y,t t ΣT c,t t (7) 3 Results for Experimental Data Set 3. Data Acquisition Data used in this experiment consisted of two channels of EEG, recorded at 256Hz placed over the central portion of the head and one channel of muscle electrical activity (EMG), recorded at 24Hz over the muscles of the right fore-arm. The EMG was then down-sampled to 256Hz and muscle contraction strength for movement and non-movement detection was evaluated via a simple windowed peak and trough detection; this then formed a movement / non-movement label. 3.2 Feature Extraction and Basis Formation The second reflection coefficient of a second-order autoregressive (AR) model [ were calculated over each EEG signal once every 78ms using a sliding one-second- a) Underlying true labels Missing labels b) c) d) Fig.. Comparison of different EKF algorithms: a) trajectories with true labels (blue) and missing labels (red), b) Standard EKF (fixed τ and κ are used), c) Modified EKF (marginalising both τ and κ), d) hybrid EKF using marginalistion and truncation
6 Adaptive Classification by Hybrid EKF with Truncated Filtering 375 Cross Entropy 2 8 Values Standard EKF Marginalised EKF hybrid EKF:(a=.5) Fig. 2. Comparison of Cross Entropy: the method proposed in this paper outperforms other EKF approaches long window, forming a set of feature vectors x t. These vectors were projected into a non-linear latent space using a set of Gaussian basis functions (N b = ), providing the stream of ϕ t. 3.3 Performance Comparison In this paper, hyper-parameters defining prior distributions over τ and κ are given by α τ =2, =, α κ =5,andβ κ =.. Since there is no proper model in dynamics for our model, we used a transition kernel based on random walks so that the matrices of Eq. () are known as A = B = L = I Nw where I Nw denotes an N w N w size identity matrix and N w is the number of elements of w t.for comparison, fixed τ and κ are chosen by τ = E(τ α τ, )= τp(τ α τ, )dτ and κ = E(κ α κ,β κ )= κp(κ α κ,β κ )dκ where τ. Also,p(w )=N ( ; μ,σ ) where μ = and Σ = τ I respectively. Here τ = 2. Fig. shows the comparison of several methods. The first plot of the figure shows the labels: inactive (z t = ), active (z t = ) and unknown labels (z t = ) in the movement respectively. The algorithm receives no label information after t = samples. Fig. 2 demonstrates the comparison of the cross entropy of the several methods. The cross entropy of the tth sample is defined by zt = log p(z t z t )= log z t ( z t ) zt} (8) E cross t where z t and z t represent for underlying labels and the posterior of estimated labels respectively. 4 Conclusion and Future Work In this paper, we propose a dynamic Bayesian model for adaptive classification in time series based on a Bayesian version of the modified Extended Kalman filter
7 376 J.W. Yoon et al. which combines Radial basis function and autoregressive model. The AR model explains the Markov dependency of the output function so that it provides the possibility of improvement in conventional EKF classification. This proposed algorithm also employs truncation of filtering distribution to obtain a better classifier when the observation is very informative. The critical data-specific sensitivity of our model is to the hyper-parameter a. In principle this can be inferred from a section of labelled data or set via knowledge of the typical withinstate times. This requires a more extensive testing against a larger number of data sets than presented in this pilot study and this will be the focus of future work. Acknowledgment This project is supported by UK EPSRC funding to whom we are most grateful. References. Wopaw, J.R., McFarland, D.J., Neat, D.J., Forneris, C.A.: An EEG-based braincomputer interface for cursor control. Electroencephalogr. Clin. Neurophysiol. 78, (99) 2. Pfurtscheller, G., Flotzinger, D., Kalcher, J.: Brain-computer interface: a new communication device for handicapped persons. Journal of Microcomputer Applications archive 6(3), (993) 3. Sykacek, P., Roberts, S.J., Stokes, M.: Adaptive BCI Based on Variational Bayesian Kalman Filtering: An Empirical Evaluation. IEEE Transactions on Biomedical Engineering 5(5), (24) 4. Penny, W.D., Roberts, S.J., Curran, E.A., Stokes, M.: EEG-based communication: a pattern recognition approach. IEEE Transactions on Rehabilitation Engineering 8(2), (2) 5. Lowne, D.R., Roberts, S.J., Garnett, R.: Sequential Non-stationary Dynamic Classification. Machine Learning (submitted, 28) 6. Yoon, J., Roberts, S.J., Dyson, M., Gan, J.Q.: Sequential Bayesian Estimation for Adaptive Classification. Multisensor Fusion and Integration for Intelligent Systems (accepted for publication, 28) 7. Jazwinski, A.H., Bailie, A.E.: Adaptive filtering Interim report. NASA Technical Reports (March, 967) 8. Bernardo, J.M., Smith, A.F.M.: Bayesian Theory. Wiley, Chichester (994) 9. Bishop, C.M.: Pattern Recognition and Machine Learning, st edn. Springer, Heidelberg (988). Pardey, J., Roberts, S.J., Tarassenko, L.: A Review of Parametric Modelling Techniques for EEG Analysis. Med. Eng. Phys. 8(), 2 (996) Appendix 4. Laplace Approximation In Eq. (3), the non-gaussian distribution generated from the marginalisation and logistic model are approximated by Gaussian distributions using the Laplace
8 Adaptive Classification by Hybrid EKF with Truncated Filtering 377 approximation. Let π(w t ) be the non-gaussian distribution of interest. Then, we have π(w t )=(2π) Nw/2 βατ τ Γ (α τ [ ) Γ (α τ ) βτ + 2 (w t w t ) T (w t w t ) α. (9) τ In order to find the mode of π(w t ), we use log of the distribution given by [ =logπ(w t )= α q log + 2 (w t w t ) T (w t w t ) + c () L wt where c = N w /2log(2π) +α τ log log Γ (α τ ) + log(α τ ). Using the first derivate of L wt in terms of w t,wehave dl α τ = dw t + 2 (w t w t ) T (w t w t ) (w t w t ) = () and a mode can be easily obtained by w t = w t. In addition, using the second derivative of the L wt, we can obtain the covariance of the approximated distribution as follows: d 2 L dw 2 t wt=w t = α τ. (2) Now, we have an approximated distribution ( ( ) ) α π(w t ) N w t ; w t, τ. (3) 4.2 Truncated Normal Distribution Suppose we condition on Y t θ =[θ,θ 2, where <θ <θ 2 <. When we have the normal distribution with mean μ = μ y,t t and standard deviation σ = Σ y,t t, the conditional density of Y t is σ p(y t θ) = φ ( y t μ) ) σ ),θ y t θ 2 (4) Φ Φ ( θ2 μ σ ( θ μ σ where φ( ) andφ( ) denote normal distribution function and normal cumulative distribution function respectively. μ y,zt= = E[Y t Y t >==μ + σ φ( μ σ ) Φ( μ σ ) μ y,zt= = E[Y t Y t < = μ σ φ( μ σ ) Φ( μ σ ) Σ y,zt= = V[Y t Y t >==σ 2 + μ σ φ( μ σ ) [ φ( μ Φ( μ σ ) σ ) 2 } Φ( μ σ ) Σ y,zt= = V[Y t Y t < = σ 2 + μ σ φ( μ σ ) Φ( μ σ ) [ φ( μ σ ) Φ( μ σ ) 2 }. (5)
Christfried Webers. Canberra February June 2015
c Statistical Group and College of Engineering and Computer Science Canberra February June (Many figures from C. M. Bishop, "Pattern Recognition and ") 1of 829 c Part VIII Linear Classification 2 Logistic
More informationSTA 4273H: Statistical Machine Learning
STA 4273H: Statistical Machine Learning Russ Salakhutdinov Department of Statistics! rsalakhu@utstat.toronto.edu! http://www.cs.toronto.edu/~rsalakhu/ Lecture 6 Three Approaches to Classification Construct
More information11. Time series and dynamic linear models
11. Time series and dynamic linear models Objective To introduce the Bayesian approach to the modeling and forecasting of time series. Recommended reading West, M. and Harrison, J. (1997). models, (2 nd
More informationLinear Classification. Volker Tresp Summer 2015
Linear Classification Volker Tresp Summer 2015 1 Classification Classification is the central task of pattern recognition Sensors supply information about an object: to which class do the object belong
More informationStatistical Machine Learning
Statistical Machine Learning UoC Stats 37700, Winter quarter Lecture 4: classical linear and quadratic discriminants. 1 / 25 Linear separation For two classes in R d : simple idea: separate the classes
More informationBayes and Naïve Bayes. cs534-machine Learning
Bayes and aïve Bayes cs534-machine Learning Bayes Classifier Generative model learns Prediction is made by and where This is often referred to as the Bayes Classifier, because of the use of the Bayes rule
More informationDeterministic Sampling-based Switching Kalman Filtering for Vehicle Tracking
Proceedings of the IEEE ITSC 2006 2006 IEEE Intelligent Transportation Systems Conference Toronto, Canada, September 17-20, 2006 WA4.1 Deterministic Sampling-based Switching Kalman Filtering for Vehicle
More informationBasics of Statistical Machine Learning
CS761 Spring 2013 Advanced Machine Learning Basics of Statistical Machine Learning Lecturer: Xiaojin Zhu jerryzhu@cs.wisc.edu Modern machine learning is rooted in statistics. You will find many familiar
More informationTime Series Analysis
Time Series Analysis hm@imm.dtu.dk Informatics and Mathematical Modelling Technical University of Denmark DK-2800 Kgs. Lyngby 1 Outline of the lecture Identification of univariate time series models, cont.:
More informationCCNY. BME I5100: Biomedical Signal Processing. Linear Discrimination. Lucas C. Parra Biomedical Engineering Department City College of New York
BME I5100: Biomedical Signal Processing Linear Discrimination Lucas C. Parra Biomedical Engineering Department CCNY 1 Schedule Week 1: Introduction Linear, stationary, normal - the stuff biology is not
More informationGaussian Processes in Machine Learning
Gaussian Processes in Machine Learning Carl Edward Rasmussen Max Planck Institute for Biological Cybernetics, 72076 Tübingen, Germany carl@tuebingen.mpg.de WWW home page: http://www.tuebingen.mpg.de/ carl
More informationMonte Carlo and Empirical Methods for Stochastic Inference (MASM11/FMS091)
Monte Carlo and Empirical Methods for Stochastic Inference (MASM11/FMS091) Magnus Wiktorsson Centre for Mathematical Sciences Lund University, Sweden Lecture 5 Sequential Monte Carlo methods I February
More informationLeast Squares Estimation
Least Squares Estimation SARA A VAN DE GEER Volume 2, pp 1041 1045 in Encyclopedia of Statistics in Behavioral Science ISBN-13: 978-0-470-86080-9 ISBN-10: 0-470-86080-4 Editors Brian S Everitt & David
More informationAutomated Stellar Classification for Large Surveys with EKF and RBF Neural Networks
Chin. J. Astron. Astrophys. Vol. 5 (2005), No. 2, 203 210 (http:/www.chjaa.org) Chinese Journal of Astronomy and Astrophysics Automated Stellar Classification for Large Surveys with EKF and RBF Neural
More informationExample: Credit card default, we may be more interested in predicting the probabilty of a default than classifying individuals as default or not.
Statistical Learning: Chapter 4 Classification 4.1 Introduction Supervised learning with a categorical (Qualitative) response Notation: - Feature vector X, - qualitative response Y, taking values in C
More informationStatistics Graduate Courses
Statistics Graduate Courses STAT 7002--Topics in Statistics-Biological/Physical/Mathematics (cr.arr.).organized study of selected topics. Subjects and earnable credit may vary from semester to semester.
More informationD-optimal plans in observational studies
D-optimal plans in observational studies Constanze Pumplün Stefan Rüping Katharina Morik Claus Weihs October 11, 2005 Abstract This paper investigates the use of Design of Experiments in observational
More informationMonte Carlo-based statistical methods (MASM11/FMS091)
Monte Carlo-based statistical methods (MASM11/FMS091) Jimmy Olsson Centre for Mathematical Sciences Lund University, Sweden Lecture 5 Sequential Monte Carlo methods I February 5, 2013 J. Olsson Monte Carlo-based
More informationNote on the EM Algorithm in Linear Regression Model
International Mathematical Forum 4 2009 no. 38 1883-1889 Note on the M Algorithm in Linear Regression Model Ji-Xia Wang and Yu Miao College of Mathematics and Information Science Henan Normal University
More informationA Study on the Comparison of Electricity Forecasting Models: Korea and China
Communications for Statistical Applications and Methods 2015, Vol. 22, No. 6, 675 683 DOI: http://dx.doi.org/10.5351/csam.2015.22.6.675 Print ISSN 2287-7843 / Online ISSN 2383-4757 A Study on the Comparison
More informationReject Inference in Credit Scoring. Jie-Men Mok
Reject Inference in Credit Scoring Jie-Men Mok BMI paper January 2009 ii Preface In the Master programme of Business Mathematics and Informatics (BMI), it is required to perform research on a business
More informationA Multi-Model Filter for Mobile Terminal Location Tracking
A Multi-Model Filter for Mobile Terminal Location Tracking M. McGuire, K.N. Plataniotis The Edward S. Rogers Sr. Department of Electrical and Computer Engineering, University of Toronto, 1 King s College
More informationKristine L. Bell and Harry L. Van Trees. Center of Excellence in C 3 I George Mason University Fairfax, VA 22030-4444, USA kbell@gmu.edu, hlv@gmu.
POSERIOR CRAMÉR-RAO BOUND FOR RACKING ARGE BEARING Kristine L. Bell and Harry L. Van rees Center of Excellence in C 3 I George Mason University Fairfax, VA 22030-4444, USA bell@gmu.edu, hlv@gmu.edu ABSRAC
More informationHandling attrition and non-response in longitudinal data
Longitudinal and Life Course Studies 2009 Volume 1 Issue 1 Pp 63-72 Handling attrition and non-response in longitudinal data Harvey Goldstein University of Bristol Correspondence. Professor H. Goldstein
More informationBayesian Image Super-Resolution
Bayesian Image Super-Resolution Michael E. Tipping and Christopher M. Bishop Microsoft Research, Cambridge, U.K..................................................................... Published as: Bayesian
More informationSpatial Statistics Chapter 3 Basics of areal data and areal data modeling
Spatial Statistics Chapter 3 Basics of areal data and areal data modeling Recall areal data also known as lattice data are data Y (s), s D where D is a discrete index set. This usually corresponds to data
More informationElectrophysiology of the expectancy process: Processing the CNV potential
Electrophysiology of the expectancy process: Processing the CNV potential *, Išgum V. ** *, Boživska L. Laboratory of Neurophysiology, Institute of Physiology, Medical Faculty, University Sv. Kiril i Metodij,
More information6.2.8 Neural networks for data mining
6.2.8 Neural networks for data mining Walter Kosters 1 In many application areas neural networks are known to be valuable tools. This also holds for data mining. In this chapter we discuss the use of neural
More informationFiltered Gaussian Processes for Learning with Large Data-Sets
Filtered Gaussian Processes for Learning with Large Data-Sets Jian Qing Shi, Roderick Murray-Smith 2,3, D. Mike Titterington 4,and Barak A. Pearlmutter 3 School of Mathematics and Statistics, University
More informationPredict the Popularity of YouTube Videos Using Early View Data
000 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050
More informationMultiplier-accelerator Models on Time Scales
International Journal of Statistics and Economics; [Formerly known as the Bulletin of Statistics & Economics ISSN 0973-7022)]; ISSN 0975-556X,; Spring 2010, Volume 4, Number S10; Copyright 2010 by CESER
More informationAuxiliary Variables in Mixture Modeling: 3-Step Approaches Using Mplus
Auxiliary Variables in Mixture Modeling: 3-Step Approaches Using Mplus Tihomir Asparouhov and Bengt Muthén Mplus Web Notes: No. 15 Version 8, August 5, 2014 1 Abstract This paper discusses alternatives
More informationHT2015: SC4 Statistical Data Mining and Machine Learning
HT2015: SC4 Statistical Data Mining and Machine Learning Dino Sejdinovic Department of Statistics Oxford http://www.stats.ox.ac.uk/~sejdinov/sdmml.html Bayesian Nonparametrics Parametric vs Nonparametric
More informationEnhancing the SNR of the Fiber Optic Rotation Sensor using the LMS Algorithm
1 Enhancing the SNR of the Fiber Optic Rotation Sensor using the LMS Algorithm Hani Mehrpouyan, Student Member, IEEE, Department of Electrical and Computer Engineering Queen s University, Kingston, Ontario,
More informationDiscrete Frobenius-Perron Tracking
Discrete Frobenius-Perron Tracing Barend J. van Wy and Michaël A. van Wy French South-African Technical Institute in Electronics at the Tshwane University of Technology Staatsartillerie Road, Pretoria,
More informationBayesX - Software for Bayesian Inference in Structured Additive Regression
BayesX - Software for Bayesian Inference in Structured Additive Regression Thomas Kneib Faculty of Mathematics and Economics, University of Ulm Department of Statistics, Ludwig-Maximilians-University Munich
More informationCell Phone based Activity Detection using Markov Logic Network
Cell Phone based Activity Detection using Markov Logic Network Somdeb Sarkhel sxs104721@utdallas.edu 1 Introduction Mobile devices are becoming increasingly sophisticated and the latest generation of smart
More informationStock Option Pricing Using Bayes Filters
Stock Option Pricing Using Bayes Filters Lin Liao liaolin@cs.washington.edu Abstract When using Black-Scholes formula to price options, the key is the estimation of the stochastic return variance. In this
More informationA LOGNORMAL MODEL FOR INSURANCE CLAIMS DATA
REVSTAT Statistical Journal Volume 4, Number 2, June 2006, 131 142 A LOGNORMAL MODEL FOR INSURANCE CLAIMS DATA Authors: Daiane Aparecida Zuanetti Departamento de Estatística, Universidade Federal de São
More informationPattern Analysis. Logistic Regression. 12. Mai 2009. Joachim Hornegger. Chair of Pattern Recognition Erlangen University
Pattern Analysis Logistic Regression 12. Mai 2009 Joachim Hornegger Chair of Pattern Recognition Erlangen University Pattern Analysis 2 / 43 1 Logistic Regression Posteriors and the Logistic Function Decision
More informationElectroencephalography Analysis Using Neural Network and Support Vector Machine during Sleep
Engineering, 23, 5, 88-92 doi:.4236/eng.23.55b8 Published Online May 23 (http://www.scirp.org/journal/eng) Electroencephalography Analysis Using Neural Network and Support Vector Machine during Sleep JeeEun
More informationBrain Computer Interface Research at the Wadsworth Center
222 IEEE TRANSACTIONS ON REHABILITATION ENGINEERING, VOL. 8, NO. 2, JUNE 2000 [9] G. Pfurtscheller, C. Neuper, C. Andrew, and G. Edlinger, Foot and hand area mu rhythms, Int. J. Psychophysiol., vol. 26,
More informationPerformance evaluation of multi-camera visual tracking
Performance evaluation of multi-camera visual tracking Lucio Marcenaro, Pietro Morerio, Mauricio Soto, Andrea Zunino, Carlo S. Regazzoni DITEN, University of Genova Via Opera Pia 11A 16145 Genoa - Italy
More informationClassification Problems
Classification Read Chapter 4 in the text by Bishop, except omit Sections 4.1.6, 4.1.7, 4.2.4, 4.3.3, 4.3.5, 4.3.6, 4.4, and 4.5. Also, review sections 1.5.1, 1.5.2, 1.5.3, and 1.5.4. Classification Problems
More informationChapter 2 The Research on Fault Diagnosis of Building Electrical System Based on RBF Neural Network
Chapter 2 The Research on Fault Diagnosis of Building Electrical System Based on RBF Neural Network Qian Wu, Yahui Wang, Long Zhang and Li Shen Abstract Building electrical system fault diagnosis is the
More informationAdvanced Signal Processing and Digital Noise Reduction
Advanced Signal Processing and Digital Noise Reduction Saeed V. Vaseghi Queen's University of Belfast UK WILEY HTEUBNER A Partnership between John Wiley & Sons and B. G. Teubner Publishers Chichester New
More informationCortical Source Localization of Human Scalp EEG. Kaushik Majumdar Indian Statistical Institute Bangalore Center
Cortical Source Localization of Human Scalp EEG Kaushik Majumdar Indian Statistical Institute Bangalore Center Cortical Basis of Scalp EEG Baillet et al, IEEE Sig Proc Mag, Nov 2001, p 16 Mountcastle,
More informationBayesian Hyperspectral Image Segmentation with Discriminative Class Learning
Bayesian Hyperspectral Image Segmentation with Discriminative Class Learning Janete S. Borges 1,José M. Bioucas-Dias 2, and André R.S.Marçal 1 1 Faculdade de Ciências, Universidade do Porto 2 Instituto
More informationMAN-BITES-DOG BUSINESS CYCLES ONLINE APPENDIX
MAN-BITES-DOG BUSINESS CYCLES ONLINE APPENDIX KRISTOFFER P. NIMARK The next section derives the equilibrium expressions for the beauty contest model from Section 3 of the main paper. This is followed by
More informationThese slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by Christopher Bishop
Music and Machine Learning (IFT6080 Winter 08) Prof. Douglas Eck, Université de Montréal These slides follow closely the (English) course textbook Pattern Recognition and Machine Learning by Christopher
More information2. Illustration of the Nikkei 225 option data
1. Introduction 2. Illustration of the Nikkei 225 option data 2.1 A brief outline of the Nikkei 225 options market τ 2.2 Estimation of the theoretical price τ = + ε ε = = + ε + = + + + = + ε + ε + ε =
More informationLogistic Regression. Jia Li. Department of Statistics The Pennsylvania State University. Logistic Regression
Logistic Regression Department of Statistics The Pennsylvania State University Email: jiali@stat.psu.edu Logistic Regression Preserve linear classification boundaries. By the Bayes rule: Ĝ(x) = arg max
More informationApplications of R Software in Bayesian Data Analysis
Article International Journal of Information Science and System, 2012, 1(1): 7-23 International Journal of Information Science and System Journal homepage: www.modernscientificpress.com/journals/ijinfosci.aspx
More informationAdaptive Demand-Forecasting Approach based on Principal Components Time-series an application of data-mining technique to detection of market movement
Adaptive Demand-Forecasting Approach based on Principal Components Time-series an application of data-mining technique to detection of market movement Toshio Sugihara Abstract In this study, an adaptive
More informationPresence research and EEG. Summary
Presence research and EEG Alois Schlögl 1, Mel Slater, Gert Pfurtscheller 1 1 Institute for Biomedical Engineering, University of Technology Graz Inffeldgasse 16a, A-81 Graz, AUSTRIA Department of Computer
More informationChapter 14 Managing Operational Risks with Bayesian Networks
Chapter 14 Managing Operational Risks with Bayesian Networks Carol Alexander This chapter introduces Bayesian belief and decision networks as quantitative management tools for operational risks. Bayesian
More informationTracking in flussi video 3D. Ing. Samuele Salti
Seminari XXIII ciclo Tracking in flussi video 3D Ing. Tutors: Prof. Tullio Salmon Cinotti Prof. Luigi Di Stefano The Tracking problem Detection Object model, Track initiation, Track termination, Tracking
More informationExact Inference for Gaussian Process Regression in case of Big Data with the Cartesian Product Structure
Exact Inference for Gaussian Process Regression in case of Big Data with the Cartesian Product Structure Belyaev Mikhail 1,2,3, Burnaev Evgeny 1,2,3, Kapushev Yermek 1,2 1 Institute for Information Transmission
More informationPATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION
PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 4: LINEAR MODELS FOR CLASSIFICATION Introduction In the previous chapter, we explored a class of regression models having particularly simple analytical
More informationComponent Ordering in Independent Component Analysis Based on Data Power
Component Ordering in Independent Component Analysis Based on Data Power Anne Hendrikse Raymond Veldhuis University of Twente University of Twente Fac. EEMCS, Signals and Systems Group Fac. EEMCS, Signals
More informationAnalysis of Bayesian Dynamic Linear Models
Analysis of Bayesian Dynamic Linear Models Emily M. Casleton December 17, 2010 1 Introduction The main purpose of this project is to explore the Bayesian analysis of Dynamic Linear Models (DLMs). The main
More informationCS 2750 Machine Learning. Lecture 1. Machine Learning. http://www.cs.pitt.edu/~milos/courses/cs2750/ CS 2750 Machine Learning.
Lecture Machine Learning Milos Hauskrecht milos@cs.pitt.edu 539 Sennott Square, x5 http://www.cs.pitt.edu/~milos/courses/cs75/ Administration Instructor: Milos Hauskrecht milos@cs.pitt.edu 539 Sennott
More informationSAS Software to Fit the Generalized Linear Model
SAS Software to Fit the Generalized Linear Model Gordon Johnston, SAS Institute Inc., Cary, NC Abstract In recent years, the class of generalized linear models has gained popularity as a statistical modeling
More informationWeb-based Supplementary Materials for Bayesian Effect Estimation. Accounting for Adjustment Uncertainty by Chi Wang, Giovanni
1 Web-based Supplementary Materials for Bayesian Effect Estimation Accounting for Adjustment Uncertainty by Chi Wang, Giovanni Parmigiani, and Francesca Dominici In Web Appendix A, we provide detailed
More informationStock Trading by Modelling Price Trend with Dynamic Bayesian Networks
Stock Trading by Modelling Price Trend with Dynamic Bayesian Networks Jangmin O 1,JaeWonLee 2, Sung-Bae Park 1, and Byoung-Tak Zhang 1 1 School of Computer Science and Engineering, Seoul National University
More informationHardware Implementation of Probabilistic State Machine for Word Recognition
IJECT Vo l. 4, Is s u e Sp l - 5, Ju l y - Se p t 2013 ISSN : 2230-7109 (Online) ISSN : 2230-9543 (Print) Hardware Implementation of Probabilistic State Machine for Word Recognition 1 Soorya Asokan, 2
More informationMODEL MIXING FOR LONG-TERM EXTRAPOLATION
MODEL MIXING FOR LONG-TERM EXTRAPOLATION Pavel Ettler 1, Miroslav Kárný 2, Petr Nedoma 2 1 COMPUREG Plzeň, s.r.o. 36 34 Plzeň, Czech Republic 2 Institute of Information Theory and Automation (UTIA) 182
More informationAn Application of Inverse Reinforcement Learning to Medical Records of Diabetes Treatment
An Application of Inverse Reinforcement Learning to Medical Records of Diabetes Treatment Hideki Asoh 1, Masanori Shiro 1 Shotaro Akaho 1, Toshihiro Kamishima 1, Koiti Hasida 1, Eiji Aramaki 2, and Takahide
More informationBig Data Visualisations. Professor Ian Nabney i.t.nabney@aston.ac.uk NCRG
Big Data Visualisations Professor Ian Nabney i.t.nabney@aston.ac.uk NCRG Overview Why visualise data? How we can visualise data Big Data Institute What is Visualisation? Goal of visualisation is to present
More informationModeling the Distribution of Environmental Radon Levels in Iowa: Combining Multiple Sources of Spatially Misaligned Data
Modeling the Distribution of Environmental Radon Levels in Iowa: Combining Multiple Sources of Spatially Misaligned Data Brian J. Smith, Ph.D. The University of Iowa Joint Statistical Meetings August 10,
More informationConditional guidance as a response to supply uncertainty
1 Conditional guidance as a response to supply uncertainty Appendix to the speech given by Ben Broadbent, External Member of the Monetary Policy Committee, Bank of England At the London Business School,
More informationAdvanced Ensemble Strategies for Polynomial Models
Advanced Ensemble Strategies for Polynomial Models Pavel Kordík 1, Jan Černý 2 1 Dept. of Computer Science, Faculty of Information Technology, Czech Technical University in Prague, 2 Dept. of Computer
More informationClass #6: Non-linear classification. ML4Bio 2012 February 17 th, 2012 Quaid Morris
Class #6: Non-linear classification ML4Bio 2012 February 17 th, 2012 Quaid Morris 1 Module #: Title of Module 2 Review Overview Linear separability Non-linear classification Linear Support Vector Machines
More informationStandard errors of marginal effects in the heteroskedastic probit model
Standard errors of marginal effects in the heteroskedastic probit model Thomas Cornelißen Discussion Paper No. 320 August 2005 ISSN: 0949 9962 Abstract In non-linear regression models, such as the heteroskedastic
More informationTutorial on Markov Chain Monte Carlo
Tutorial on Markov Chain Monte Carlo Kenneth M. Hanson Los Alamos National Laboratory Presented at the 29 th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Technology,
More informationSystem Identification for Acoustic Comms.:
System Identification for Acoustic Comms.: New Insights and Approaches for Tracking Sparse and Rapidly Fluctuating Channels Weichang Li and James Preisig Woods Hole Oceanographic Institution The demodulation
More informationMoving averages. Rob J Hyndman. November 8, 2009
Moving averages Rob J Hyndman November 8, 009 A moving average is a time series constructed by taking averages of several sequential values of another time series. It is a type of mathematical convolution.
More informationMachine Learning in FX Carry Basket Prediction
Machine Learning in FX Carry Basket Prediction Tristan Fletcher, Fabian Redpath and Joe D Alessandro Abstract Artificial Neural Networks ANN), Support Vector Machines SVM) and Relevance Vector Machines
More informationINDIRECT INFERENCE (prepared for: The New Palgrave Dictionary of Economics, Second Edition)
INDIRECT INFERENCE (prepared for: The New Palgrave Dictionary of Economics, Second Edition) Abstract Indirect inference is a simulation-based method for estimating the parameters of economic models. Its
More informationProbabilistic Models for Big Data. Alex Davies and Roger Frigola University of Cambridge 13th February 2014
Probabilistic Models for Big Data Alex Davies and Roger Frigola University of Cambridge 13th February 2014 The State of Big Data Why probabilistic models for Big Data? 1. If you don t have to worry about
More informationJava Modules for Time Series Analysis
Java Modules for Time Series Analysis Agenda Clustering Non-normal distributions Multifactor modeling Implied ratings Time series prediction 1. Clustering + Cluster 1 Synthetic Clustering + Time series
More informationMachine Learning in Statistical Arbitrage
Machine Learning in Statistical Arbitrage Xing Fu, Avinash Patra December 11, 2009 Abstract We apply machine learning methods to obtain an index arbitrage strategy. In particular, we employ linear regression
More informationCS 688 Pattern Recognition Lecture 4. Linear Models for Classification
CS 688 Pattern Recognition Lecture 4 Linear Models for Classification Probabilistic generative models Probabilistic discriminative models 1 Generative Approach ( x ) p C k p( C k ) Ck p ( ) ( x Ck ) p(
More informationIntroduction to General and Generalized Linear Models
Introduction to General and Generalized Linear Models General Linear Models - part I Henrik Madsen Poul Thyregod Informatics and Mathematical Modelling Technical University of Denmark DK-2800 Kgs. Lyngby
More informationPart 2: One-parameter models
Part 2: One-parameter models Bernoilli/binomial models Return to iid Y 1,...,Y n Bin(1, θ). The sampling model/likelihood is p(y 1,...,y n θ) =θ P y i (1 θ) n P y i When combined with a prior p(θ), Bayes
More informationFitting Subject-specific Curves to Grouped Longitudinal Data
Fitting Subject-specific Curves to Grouped Longitudinal Data Djeundje, Viani Heriot-Watt University, Department of Actuarial Mathematics & Statistics Edinburgh, EH14 4AS, UK E-mail: vad5@hw.ac.uk Currie,
More informationLecture 3: Linear methods for classification
Lecture 3: Linear methods for classification Rafael A. Irizarry and Hector Corrada Bravo February, 2010 Today we describe four specific algorithms useful for classification problems: linear regression,
More informationProbabilistic user behavior models in online stores for recommender systems
Probabilistic user behavior models in online stores for recommender systems Tomoharu Iwata Abstract Recommender systems are widely used in online stores because they are expected to improve both user
More informationFractionally integrated data and the autodistributed lag model: results from a simulation study
Fractionally integrated data and the autodistributed lag model: results from a simulation study Justin Esarey July 1, 215 Abstract Two contributions in this issue, Grant and Lebo (215) and Keele, Linn
More informationEE 570: Location and Navigation
EE 570: Location and Navigation On-Line Bayesian Tracking Aly El-Osery 1 Stephen Bruder 2 1 Electrical Engineering Department, New Mexico Tech Socorro, New Mexico, USA 2 Electrical and Computer Engineering
More informationThe Proportional Odds Model for Assessing Rater Agreement with Multiple Modalities
The Proportional Odds Model for Assessing Rater Agreement with Multiple Modalities Elizabeth Garrett-Mayer, PhD Assistant Professor Sidney Kimmel Comprehensive Cancer Center Johns Hopkins University 1
More informationEstimating an ARMA Process
Statistics 910, #12 1 Overview Estimating an ARMA Process 1. Main ideas 2. Fitting autoregressions 3. Fitting with moving average components 4. Standard errors 5. Examples 6. Appendix: Simple estimators
More informationContinuous Time Bayesian Networks for Inferring Users Presence and Activities with Extensions for Modeling and Evaluation
Continuous Time Bayesian Networks for Inferring Users Presence and Activities with Extensions for Modeling and Evaluation Uri Nodelman 1 Eric Horvitz Microsoft Research One Microsoft Way Redmond, WA 98052
More informationTesting for Granger causality between stock prices and economic growth
MPRA Munich Personal RePEc Archive Testing for Granger causality between stock prices and economic growth Pasquale Foresti 2006 Online at http://mpra.ub.uni-muenchen.de/2962/ MPRA Paper No. 2962, posted
More informationCourse 8. An Introduction to the Kalman Filter
Course 8 An Introduction to the Kalman Filter Speakers Greg Welch Gary Bishop Kalman Filters in 2 hours? Hah! No magic. Pretty simple to apply. Tolerant of abuse. Notes are a standalone reference. These
More informationNew Ensemble Combination Scheme
New Ensemble Combination Scheme Namhyoung Kim, Youngdoo Son, and Jaewook Lee, Member, IEEE Abstract Recently many statistical learning techniques are successfully developed and used in several areas However,
More informationTowards Online Recognition of Handwritten Mathematics
Towards Online Recognition of Handwritten Mathematics Vadim Mazalov, joint work with Oleg Golubitsky and Stephen M. Watt Ontario Research Centre for Computer Algebra Department of Computer Science Western
More informationJPEG compression of monochrome 2D-barcode images using DCT coefficient distributions
Edith Cowan University Research Online ECU Publications Pre. JPEG compression of monochrome D-barcode images using DCT coefficient distributions Keng Teong Tan Hong Kong Baptist University Douglas Chai
More informationPrincipled Hybrids of Generative and Discriminative Models
Principled Hybrids of Generative and Discriminative Models Julia A. Lasserre University of Cambridge Cambridge, UK jal62@cam.ac.uk Christopher M. Bishop Microsoft Research Cambridge, UK cmbishop@microsoft.com
More informationLarge Time-Varying Parameter VARs
Large Time-Varying Parameter VARs Gary Koop University of Strathclyde Dimitris Korobilis University of Glasgow Abstract In this paper we develop methods for estimation and forecasting in large timevarying
More information