Hidden Markov Models: Fundamentals and Applications Part 1: Markov Chains and Mixture Models

Size: px
Start display at page:

Download "Hidden Markov Models: Fundamentals and Applications Part 1: Markov Chains and Mixture Models"

Transcription

1 Also appears n the Onlne Symposum for Electroncs Engneer 000 Hdden Marov Models: Fundamentals and Applcatons Part : Marov Chans and Mxture Models Valery A. Petrushn Center for Strategc Technology Research Accenture 3773 Wllo Rd. Northbroo, Illnos 6006, USA. Abstract The obectve of ths tutoral s to ntroduce basc concepts of a Hdden Marov Model (HMM) as a fuson of more smple models such as a Marov chan and a Gaussan mxture model. The tutoral s ntended for the practcng engneer, bologst, lngust or programmer ho ould le to learn more about the above mentoned fascnatng mathematcal models and nclude them nto one s repertore. Ths lecture presents Marov Chans and Gaussan mxture models, hch consttute the prelmnary noledge for understandng Hdden Marov Models. Introducton Why t s so mportant to learn about these models? Frst, the models have proved to be ndspensable for a de range of applcatons n such areas as sgnal processng, bonformatcs, mage processng, lngustcs, and others, hch deal th sequences or mxtures of components. Second, the ey algorthm used for estmatng the models the socalled Expectaton Maxmzaton Algorthm -- has much broader applcaton potental and deserves to be non to every practcng engneer or scentst. And last, but not least, the beauty of the models and algorthms maes t orthhle to devote some tme and efforts to learnng and enoyng them. Introducton nto Marov chans. Pavng the path to the shrne problem Welcome to the planet Medan! Medan s the captal planet of the Stochastc Empre, hch conquered all orlds of the observable part of the Unverse and spread ts nfluence to the hdden parts. Ruled by the Glorous and Far ruler, Emperor Probabl the Great, the Empre reached the most flourshng state of commerce, scence and ndustry ever. Welcome to Bayes-Cty, the captal of the Empre! Today, the Emperor s heralds had the honor of announcng a ne gant proect. The four maor Provnces of the Empre have to pave the paths from ther resdence ctes on Medan to the St. ely Hood Shrne, located on the Mean Square n Fgure.. Pavng problem Bayes-Cty. Each path s,000 mles long and has to be paved usng large tles of the follong precous stones: ruby (red), emerald (green) and topaz (blue) n accordance to ts

2 Provnce s random process hch states that the color of next tle depends only on the color of the current tle. The square around the St. ely Hood Shrne ll be secretly dvded nto sectors and paved by the teams from all four Provnces over one nght (Fgure.). A plgrm ll be alloed to enter the Shrne to enoy the ve of the Bayesan Belef Netor, the log that served as a char to ely Hood, and the other relcs, and also to tal to the hgh prests of scence durng the hgh confdence tme ntervals f and only f he or she can determne for three sectors of pavement around the Shrne hch sector has been paved by the team of hch Provnce. Imagne you are a plgrm ho ourneyed more than,000 mles on foot to see the Stochastc Empre s relcs. Ho can you solve the problem?. Marov chans We deal th a stochastc or random process hch s characterzed by the rule that only the current state of the process can nfluence the choce of the next state. It means the process has no memory of ts prevous states. Such a process s called a Marov process after the name of a promnent Russan mathematcan Andrey Marov (856-9). If e assume that the process has only a fnte or countable set of states, then t s called a Marov chan. Marov chans can be consdered both n dscrete and contnuous tme, but e shall lmt our tutoral to the dscrete tme fnte Marov chans. (a) (b) π = ( ) π = ( ) Fgure.. Marov chan models for a based con (a), and the pavng model M N Such chans can be descrbed by dagrams (Fgure.). The nodes of the dagram represent the states (n our case, a state corresponds to a choce of a tle of a partcular color) and the edges represent transtons beteen the states. A transton probablty s assgned to each edge. The probabltes of all edges outgong from a node must sum to one. Besde that, there s an ntal state probablty dstrbuton to defne the frst state of the chan. M = π, A π = ( π ), π,..., π N { a }, =, N + = + (.) Thus, a Marov chan s unquely defned by a par (.), here π s the vector of ntal probablty dstrbuton and A s a stochastc transton matrx. The Marov process characterstc property s represented by (.). Fgure. presents Marov chan models for a based con and tle generaton..3 Tranng algorthm ( q q,..., q ) P( q q ) P (.) We assume that each Provnce uses ts on Marov chan to generate sequences of tles. We also assume that all tles have ther ordnal numbers carved on them and e have no problem determnng the order of tles n the sequence (n spte of the fact that the tles cover the todmensonal surface of the path). The resultng paths could be vsually dfferent for pavements of dfferent Provnces. For example, North Provnce, hch s famous for ts deposts of emerald, mght have larger probabltes for transtons to emerald (green) state and generate greensh paths. But the dfferences among models could be rather small and the paths produced by the models could loo very smlar, especally for the paths of short length. That s hy e have to fnd a formal approach for estmatng the model's parameters based on

3 data. Thans to the Emperor, e have enough data to tran our models --,000 mles of data per model! If a tle has the dameter of one yard and the tles are lned up n ros of 0 tles per ro then e have 35,00 tles per mle. To estmate parameters of a model th 3 states, e need to calculate values: a 3-element vector of ntal state probabltes and a 3 by 3 state transton probablty matrx. et Q = qq... q be a tranng sequence of states. As estmatons for the ntal state probabltes, e use frequences of tles c n the tranng sequence (.3). To estmate transton probabltes, e have to count the number of pars of tles for each combnaton of colors c (,=,N) and then normalze values of each ro (.4). Table. presents three estmates of parameters for the ncreasng length of the tranng sequence. Table.. Marov chan tranng results π = True =000 =0000 =3500 ( ) c, π = a c c = N = N, =, = N N N = = c = (.3) (.4) π = ( ) π = ( ) π = ( ) No you no hat to do! You should al along each path, collect data and use (.3) and (.4) to create Marov chans for each Provnce..4 Recognton algorthm et us assume that you have already bult the models for each Provnce: MN, ME, MS, and MW. Ho can you use them to determne hch Provnce s team paved a sector Q? A natural approach to solve ths problem s to calculate the condtonal probablty of the model M for the gven sequence of tles Q: P(M Q). Usng the Bayes Rule (.5), e can see that P(Q) does not depend on the model. Thus, e should pc up the model, hch maxmzes the numerator of the formula (.5). P( M P( Q M ) P( M ) Q) = P( Q) here P(M Q) s the posteror probablty of model M for the gven sequence Q; P(Q M ) s the lelhood of Q to be generated by M ; P(M ) s the pror probablty of M ; P(Q) s the probablty of Q to be generated by all models. Ths crteron s called maxmum aposteror probablty (MAP) estmaton and can be formally expressed as (.6). Hoever, t s often the case that the pror probabltes are not non or are unform over all the models. Then e can reduce (.6) to (.7). Ths crteron s called maxmum lelhood (M) estmaton. M M = arg max{ P( Q M * M = argmax{ P( Q M )} ) P( M * M )} (.5) (.6) (.7)

4 Ho can you calculate the lelhood? et us consder the sequence 33 and trace on the dagram n Fgure.b to see ho ths sequence could be generated. The frst state can be generated th the probablty π 3, the transton probablty from 3 to s a 3, and from to 3 s a 3. The probablty of the sequence s equal to the product of these three factors (.8). P Q M ) = π a a = (.8) ( = Generalzng our dervaton for the sequence of arbtrary length e obtan (.9). Typcally the logarthm of lelhood or log-lelhood s used for calculatons (.0). P( Q M ) = π a a... a = π q qq qq3 q q q log P( Q M ) = logπ q + loga = Table. presents the accuracy of recognton of,000 sequences of dfferent length. The test sequences ere generated by the true models and recognzed by the estmated models. For recognton e used the M crteron,.e. e calculated the log-lelhood of the gven sequence for each of four models and pced up the model th the maxmal value. The set of models as traned on one-mle data for each model. You can see that the accuracy s poor for the short sequences and gradually mproves th grong length of sequences. For relable recognton e need sequences of more than 00 symbols. q q = a q q (.9) (.0) Table.. Recognton accuracy for test sequences of dfferent length ength Accuracy (%) True models: North East South West π = ( ) π = ( ) = ( ) π = π ( ) Estmated one-mle models: North East South West π = ( ) π = ( ) π = ( ) π = ( 3 3 4) Generalzatons To mportant generalzatons of the Marov chan model descrbed above are orth to mentonng. They are hgh-order Marov chans and contnuous-tme Marov chans. In the case of a hgh-order Marov chan of order n, here n >, e assume that the choce of the next state depends on n prevous states, ncludng the current state (.). P ( q ) ( ) + n q, q,..., q,..., q+ n = P q+ n q,..., q + n (.) In the case of a contnuous-tme Marov chan, the process ats for a random tme at the current state before t umps to the next state. The atng tme at each state can have ts on probablty densty functon or the common functon for all states, for example, an exponental dstrbuton (.). f T λt ( t) = e t 0 λ (.)

5 .6 References and Applcatons The theory of Marov chans s ell establshed and has vast lterature. Tang all responsblty for my slant choce, I ould le to menton only to boos. Norrs boo [] s a concse ntroducton nto the theory of Marov chans, and Steart s boo [] pays attenton to numercal aspects of Marov chans and ther applcatons. Both boos have some recommendatons and references for further readng. Marov Chans have a de range of applcatons. One of the most mportant and elaborated areas of applcatons s analyss and desgn of queues and queung netors. It covers a range of problems from productvty analyss of a carash staton to the desgn of optmal computer and telecommuncaton netors. Steart s boo [] gves a lot of detals on ths type of modelng and descrbes a softare pacage that can be used for queung netors desgn and analyss (http://.csc.ncsu.edu/faculty/wsteart/marca/marca.html). Hstorcally, the frst applcaton of Marov Chans as made by Andrey Marov hmself n the area of language modelng. Marov as fond of poetry and he appled hs model to studes of poetc style. Today you can have fun usng the Shannonzer hch s a Web-based program (see that can rerte your message n the style of Mar Tan or es Carroll. Another example of Marov chans applcaton n lngustcs s stochastc language modelng. It as shon that alphabet level Marov chans of order 5 and 6 can be used for recognton texts n dfferent languages. The ord level Marov chans of order and are used for language modelng for speech recognton (http://.s.cs.cmu.edu/is.speech.lm.html). DNA sequences can be consdered as texts n the alphabet of four letters that represent the nucleotdes. The dfference n stochastc propertes of Marov chan models for codng and non-codng regons of DNA can be used for gene fndng. GeneMar s the frst system that mplemented ths approach (see The other group of models non as branchng processes as desgned to cover such applcaton as modelng chan reacton n chemstry and nuclear physcs, populaton genetcs (http://pespmc.vub.ac.be/mathmpg.html), and even game analyss for such games as baseball (http://.retrosheet.org/mdp/marov/theory.htm), curlng, etc. And the last applcaton area, hch I d le to menton, s so called Marov chan Monte Carlo algorthms. They use Marov chans to generate random numbers that belong exactly to the desred dstrbuton or, speang n other ords, they create a perfectly random samplng. See the for addtonal nformaton about these algorthms. Introducton nto Gaussan mxture models. Revealng the old mon s secret problem All respectful famles on Medan have ther tradtonal vast gardens of sculptures. The gardens, hch are covered by sand composed of mcroscopc grans of precous stones, exhbt both the sculptures of concrete thngs and persons and abstract concepts. The color of the sand s as mportant as the artstc and scentfc value of sculptures. The sand recpes are ept n secret and passed from one generaton to another. A ne rch man (let us call hm Mr. Nerch), ho nterpreted the slogan eep off Medan! as permsson to explore and explot poor developed suburbs of the Empre, returned to the captal planet and settled hmself n a prestgous area. But hen he started to desgn hs garden of sculptures, he ran nto a problem of sand recpe. After several

6 unsuccessful attempts to buy a recpe he decded to steal some sand from the orshop of an old mon ho served as a sandeeper for a noble and respectful, but not too rch famly, and reveal the secret of the recpe usng the stolen sand as data. Mr. Nerch s helpers measured the avelength of reflected lght for 0,000 sand grans. Fgure. presents the hstogram for the data. Three concentratons of data near 400 nm, 40 nm and 450 nm allo us to form a hypothess that sand s a mxture of three components. Fgure.. Hstogram of sand data. et us pretend that you are the chef scentst at Allempre Consultng, Inc., and Mr. Nerch pays you a fortune to decode the data. Ho can you determne the characterstcs of each component and proporton of the mxture? It seems that stochastc modelng approach called Gaussan Mxture Models can do the ob.. Gaussan mxture model et us assume that components of the mxture have Gaussan dstrbuton th mean µ and varance σ and probablty densty functon (.). The choce of the Gaussan dstrbuton s natural and very common hen e deal th a natural obect or phenomenon, such as sand. But the mxture model approach can be appled to the other dstrbutons from the exponental famly. σ π ( x µ ) ( ;, ) σ f x µ σ = e (.) We let o,o,,o denote our observatons or data ponts. Then each data pont s assumed to be an ndependent realzaton of the random varable O th the three-component mxture probablty densty functon (.) and log-lelhood functon (.3). Wth the maxmum lelhood approach to the estmaton of our model M = ν, N ( µ, σ ),..., N ( µ, σ ), e need to fnd such values of ν, µ and σ that maxmze the functon (.3). To solve the problem, the Expectaton Maxmzaton (EM) Algorthm s used. f ( o; M ) = ν f = log ( M ) = ( o; µ, σ ) log( = = The EM Algorthm s also non as the Fuzzy -mean algorthm. The ey dea of the algorthm s to assgn to each data pont o a vector that has as many elements as there are components n the mxture (three n our case). Each element of the vector (e shall call them eghts) represents the confdence or probablty that the -th data pont o belongs to the ν f ( o ν = ; µ, σ )) = (.) (.3)

7 -th mxture component. All eghts for a gven data pont must sum to one (.4). For example, a to-component mxture eght vector (, 0) means that the data pont certanly belongs to the frst mxture component, but (, 0.7) means than the data pont belongs more to the second mxture component than to the frst one. = P( N( µ, σ ) o ), = (.4) For each teraton, e use the current parameters of the model to calculate eghts for all data pont (expectaton step or E-step), and then e use these updated eghts to recalculate parameters of the model (maxmzaton step or M-step). et us see ho the EM algorthm ors n some detals. Frst, e have to provde an ntal approxmaton of model M (0) usng one of the follong ays: () Parttonng data ponts arbtrarly among the mxture components and then calculatng the parameters of the model. () Settng up the parameters randomly or based on our noledge about the problem. After ths, e start to terate E- and M-steps untl a stoppng crteron gets true. For the E-step the expected component membershp eghts are calculated for each data pont based on parameters of the current model. Ths s done usng the Bayes Rule that calculates the eght as the posteror probablty of membershp of -th data pont n -th mxture component (.5). Compare the formula (.5) to the Bayes Rule (.5). h= For the M-step, the parameters of the model are recalculated usng formulas (.6), (.7) and (.8), based on our refned noledge about membershp of data ponts. A nce feature of the EM algorthm s that the lelhood functon (.3) can never decrease; hence t eventually converges to a local maxmum. In practce, the algorthm stops hen the dfference beteen to consecutve values of lelhood functon s less than a threshold (.9). ( t+ ) ( t ) log ( M ) log ( M ) < ε (.9) In order to better understand ho the EM algorthm ors, let us trace t for the case of to mxture components th a small amount of data ponts. Suppose e have 0 data ponts that are arbtrarly parttoned beteen to components (see Fgure.3a). The ntal model has large varances and means that le close to each other (see Table.). Usng ths model, the algorthm calculates a ne model th larger dstance beteen means and smaller varances. After teratons the algorthm produces the model that maxmzes the lelhood functon and optmzes the membershp probablty dstrbutons for each data pont (see Fgure.3 and Table.). The resultng soluton s depcted as to green bell-shaped curves on Fgure.3d. You can see that e have perfect mxture proporton, and good approxmaton for the mean of the frst component and varance of the second component. In general, the results are amazngly good for a data set of ten ponts. = = ν f ( o ; µ, σ ) ν f ( o ; µ, σ ) ( t+ ) ν = =, µ σ ( t+ ) ( t+ ) = = = = = o = ( o µ ) h = h h (.5) (.6) (.7) (.8)

8 Fgure.3. EM algorthm convergence. Table. EM algorthm convergence (0 ponts) Applyng the EM algorthm to Mr. Nerch s data, e obtan the soluton presented n Table.. Due to a rather good ntal model t too only 7 teratons to converge. The old mon s secret s revealed! Table.. Mon s secret problem

9 The mxture model approach and the EM algorthm have been generalzed to accommodate multvarate data and other nd of probablty dstrbutons. Another method of generalzaton s to splt multvarate data nto several sets or streams and create a mxture model for each stream, but have the ont lelhood functon. The specal beauty of mxture models s that they or n cases hen the components are heavly ntersected. For example, for to-dmensonal data of 000 ponts and threecomponent mxture th to components havng equal means but dfferent varances, e can get a good approxmaton n 58 teratons (see Fgure.4). The dar sde of the EM algorthm s that t s very slo and s consdered to be mpractcal for large data sets. Fgure.4. Three-component mxture th to components havng equal means. The EM algorthm as nvented n late 970s, but the 990s mared a ne ave of research revved by grong nterest to data mnng. Recent research s gong n to drectons: () Generalzaton of the EM algorthm and extenson to other probablty dstrbutons, and () Improvng the speed of EM algorthm usng herarchcal data representatons..3 References and Applcatons There s a suffcent body of lterature devoted to the mxture models. Most of t s ntended for researches th a strong mathematcal bacground. I refer to to boos [3-4] rtten by Australan researchers. There are also several commercal and free softare pacages devoted to mxture models, most of hch are avalable on the Web. Commercal softare: MPUS MIX

10 Freeare: EMMIX MCUST MIXCUS NORMIX CAMAN Mxture Models have been n use for more than 30 years. It s a very poerful approach to model-based clusterng and classfcaton. It has been appled to numerous problems. Some of the more nterestng applcatons are descrbed belo. Mxture models are dely used n mage processng for mage representaton and segmentaton, obect recognton, and content-based mage retreval : Mxture models also ere used for text classfcaton and clusterng: Gaussan mxture models proved to be very successful for solvng the speaer recognton and speaer verfcaton problems: There s grong nterest n applyng ths technque to exploratory analyss of data or data mnng, and medcal dagnoss: You can learn more about these applcatons by follong the URs presented above. References [] J.R. Norrs (997) Marov Chans. Cambrdge Unversty Press. [] W.J. Steart (994) Introducton to the Numercal Solutons of Marov Chans. Prnceton Unversty Press. [3] G.J. Mcachlan and.e. Basford (988) Mxture Models: Inference and Applcaton to Clusterng. Ne Yor: Marcel Deer. [4] G.J. Mcachlan and T. rshnan (997) The EM Algorthm and Extensons. Ne Yor: Wley.

Descriptive Models. Cluster Analysis. Example. General Applications of Clustering. Examples of Clustering Applications

Descriptive Models. Cluster Analysis. Example. General Applications of Clustering. Examples of Clustering Applications CMSC828G Prncples of Data Mnng Lecture #9 Today s Readng: HMS, chapter 9 Today s Lecture: Descrptve Modelng Clusterng Algorthms Descrptve Models model presents the man features of the data, a global summary

More information

What is Candidate Sampling

What is Candidate Sampling What s Canddate Samplng Say we have a multclass or mult label problem where each tranng example ( x, T ) conssts of a context x a small (mult)set of target classes T out of a large unverse L of possble

More information

CS 2750 Machine Learning. Lecture 17a. Clustering. CS 2750 Machine Learning. Clustering

CS 2750 Machine Learning. Lecture 17a. Clustering. CS 2750 Machine Learning. Clustering Lecture 7a Clusterng Mlos Hauskrecht mlos@cs.ptt.edu 539 Sennott Square Clusterng Groups together smlar nstances n the data sample Basc clusterng problem: dstrbute data nto k dfferent groups such that

More information

The Development of Web Log Mining Based on Improve-K-Means Clustering Analysis

The Development of Web Log Mining Based on Improve-K-Means Clustering Analysis The Development of Web Log Mnng Based on Improve-K-Means Clusterng Analyss TngZhong Wang * College of Informaton Technology, Luoyang Normal Unversty, Luoyang, 471022, Chna wangtngzhong2@sna.cn Abstract.

More information

benefit is 2, paid if the policyholder dies within the year, and probability of death within the year is ).

benefit is 2, paid if the policyholder dies within the year, and probability of death within the year is ). REVIEW OF RISK MANAGEMENT CONCEPTS LOSS DISTRIBUTIONS AND INSURANCE Loss and nsurance: When someone s subject to the rsk of ncurrng a fnancal loss, the loss s generally modeled usng a random varable or

More information

Lecture 2: Single Layer Perceptrons Kevin Swingler

Lecture 2: Single Layer Perceptrons Kevin Swingler Lecture 2: Sngle Layer Perceptrons Kevn Sngler kms@cs.str.ac.uk Recap: McCulloch-Ptts Neuron Ths vastly smplfed model of real neurons s also knon as a Threshold Logc Unt: W 2 A Y 3 n W n. A set of synapses

More information

Forecasting the Demand of Emergency Supplies: Based on the CBR Theory and BP Neural Network

Forecasting the Demand of Emergency Supplies: Based on the CBR Theory and BP Neural Network 700 Proceedngs of the 8th Internatonal Conference on Innovaton & Management Forecastng the Demand of Emergency Supples: Based on the CBR Theory and BP Neural Network Fu Deqang, Lu Yun, L Changbng School

More information

CS 2750 Machine Learning. Lecture 3. Density estimation. CS 2750 Machine Learning. Announcements

CS 2750 Machine Learning. Lecture 3. Density estimation. CS 2750 Machine Learning. Announcements Lecture 3 Densty estmaton Mlos Hauskrecht mlos@cs.ptt.edu 5329 Sennott Square Next lecture: Matlab tutoral Announcements Rules for attendng the class: Regstered for credt Regstered for audt (only f there

More information

Module 2 LOSSLESS IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur

Module 2 LOSSLESS IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur Module LOSSLESS IMAGE COMPRESSION SYSTEMS Lesson 3 Lossless Compresson: Huffman Codng Instructonal Objectves At the end of ths lesson, the students should be able to:. Defne and measure source entropy..

More information

PSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 12

PSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 12 14 The Ch-squared dstrbuton PSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 1 If a normal varable X, havng mean µ and varance σ, s standardsed, the new varable Z has a mean 0 and varance 1. When ths standardsed

More information

Recurrence. 1 Definitions and main statements

Recurrence. 1 Definitions and main statements Recurrence 1 Defntons and man statements Let X n, n = 0, 1, 2,... be a MC wth the state space S = (1, 2,...), transton probabltes p j = P {X n+1 = j X n = }, and the transton matrx P = (p j ),j S def.

More information

A DATA MINING APPLICATION IN A STUDENT DATABASE

A DATA MINING APPLICATION IN A STUDENT DATABASE JOURNAL OF AERONAUTICS AND SPACE TECHNOLOGIES JULY 005 VOLUME NUMBER (53-57) A DATA MINING APPLICATION IN A STUDENT DATABASE Şenol Zafer ERDOĞAN Maltepe Ünversty Faculty of Engneerng Büyükbakkalköy-Istanbul

More information

Probabilities and Probabilistic Models

Probabilities and Probabilistic Models Probabltes and Probablstc Models Probablstc models A model means a system that smulates an obect under consderaton. A probablstc model s a model that produces dfferent outcomes wth dfferent probabltes

More information

DEFINING %COMPLETE IN MICROSOFT PROJECT

DEFINING %COMPLETE IN MICROSOFT PROJECT CelersSystems DEFINING %COMPLETE IN MICROSOFT PROJECT PREPARED BY James E Aksel, PMP, PMI-SP, MVP For Addtonal Informaton about Earned Value Management Systems and reportng, please contact: CelersSystems,

More information

1 Approximation Algorithms

1 Approximation Algorithms CME 305: Dscrete Mathematcs and Algorthms 1 Approxmaton Algorthms In lght of the apparent ntractablty of the problems we beleve not to le n P, t makes sense to pursue deas other than complete solutons

More information

8.5 UNITARY AND HERMITIAN MATRICES. The conjugate transpose of a complex matrix A, denoted by A*, is given by

8.5 UNITARY AND HERMITIAN MATRICES. The conjugate transpose of a complex matrix A, denoted by A*, is given by 6 CHAPTER 8 COMPLEX VECTOR SPACES 5. Fnd the kernel of the lnear transformaton gven n Exercse 5. In Exercses 55 and 56, fnd the mage of v, for the ndcated composton, where and are gven by the followng

More information

1. Fundamentals of probability theory 2. Emergence of communication traffic 3. Stochastic & Markovian Processes (SP & MP)

1. Fundamentals of probability theory 2. Emergence of communication traffic 3. Stochastic & Markovian Processes (SP & MP) 6.3 / -- Communcaton Networks II (Görg) SS20 -- www.comnets.un-bremen.de Communcaton Networks II Contents. Fundamentals of probablty theory 2. Emergence of communcaton traffc 3. Stochastc & Markovan Processes

More information

Clustering Gene Expression Data. (Slides thanks to Dr. Mark Craven)

Clustering Gene Expression Data. (Slides thanks to Dr. Mark Craven) Clusterng Gene Epresson Data Sldes thanks to Dr. Mark Craven Gene Epresson Proles we ll assume we have a D matr o gene epresson measurements rows represent genes columns represent derent eperments tme

More information

BERNSTEIN POLYNOMIALS

BERNSTEIN POLYNOMIALS On-Lne Geometrc Modelng Notes BERNSTEIN POLYNOMIALS Kenneth I. Joy Vsualzaton and Graphcs Research Group Department of Computer Scence Unversty of Calforna, Davs Overvew Polynomals are ncredbly useful

More information

L10: Linear discriminants analysis

L10: Linear discriminants analysis L0: Lnear dscrmnants analyss Lnear dscrmnant analyss, two classes Lnear dscrmnant analyss, C classes LDA vs. PCA Lmtatons of LDA Varants of LDA Other dmensonalty reducton methods CSCE 666 Pattern Analyss

More information

Cluster Analysis of Data Points using Partitioning and Probabilistic Model-based Algorithms

Cluster Analysis of Data Points using Partitioning and Probabilistic Model-based Algorithms Internatonal Journal of Appled Informaton Systems (IJAIS) ISSN : 2249-0868 Foundaton of Computer Scence FCS, New York, USA Volume 7 No.7, August 2014 www.jas.org Cluster Analyss of Data Ponts usng Parttonng

More information

Efficient Project Portfolio as a tool for Enterprise Risk Management

Efficient Project Portfolio as a tool for Enterprise Risk Management Effcent Proect Portfolo as a tool for Enterprse Rsk Management Valentn O. Nkonov Ural State Techncal Unversty Growth Traectory Consultng Company January 5, 27 Effcent Proect Portfolo as a tool for Enterprse

More information

Meta-Analysis of Hazard Ratios

Meta-Analysis of Hazard Ratios NCSS Statstcal Softare Chapter 458 Meta-Analyss of Hazard Ratos Introducton Ths module performs a meta-analyss on a set of to-group, tme to event (survval), studes n hch some data may be censored. These

More information

Lecture 18: Clustering & classification

Lecture 18: Clustering & classification O CPS260/BGT204. Algorthms n Computatonal Bology October 30, 2003 Lecturer: Pana K. Agarwal Lecture 8: Clusterng & classfcaton Scrbe: Daun Hou Open Problem In HomeWor 2, problem 5 has an open problem whch

More information

How Sets of Coherent Probabilities May Serve as Models for Degrees of Incoherence

How Sets of Coherent Probabilities May Serve as Models for Degrees of Incoherence 1 st Internatonal Symposum on Imprecse Probabltes and Ther Applcatons, Ghent, Belgum, 29 June 2 July 1999 How Sets of Coherent Probabltes May Serve as Models for Degrees of Incoherence Mar J. Schervsh

More information

THE DISTRIBUTION OF LOAN PORTFOLIO VALUE * Oldrich Alfons Vasicek

THE DISTRIBUTION OF LOAN PORTFOLIO VALUE * Oldrich Alfons Vasicek HE DISRIBUION OF LOAN PORFOLIO VALUE * Oldrch Alfons Vascek he amount of captal necessary to support a portfolo of debt securtes depends on the probablty dstrbuton of the portfolo loss. Consder a portfolo

More information

Forecasting the Direction and Strength of Stock Market Movement

Forecasting the Direction and Strength of Stock Market Movement Forecastng the Drecton and Strength of Stock Market Movement Jngwe Chen Mng Chen Nan Ye cjngwe@stanford.edu mchen5@stanford.edu nanye@stanford.edu Abstract - Stock market s one of the most complcated systems

More information

Face Verification Problem. Face Recognition Problem. Application: Access Control. Biometric Authentication. Face Verification (1:1 matching)

Face Verification Problem. Face Recognition Problem. Application: Access Control. Biometric Authentication. Face Verification (1:1 matching) Face Recognton Problem Face Verfcaton Problem Face Verfcaton (1:1 matchng) Querymage face query Face Recognton (1:N matchng) database Applcaton: Access Control www.vsage.com www.vsoncs.com Bometrc Authentcaton

More information

Risk-based Fatigue Estimate of Deep Water Risers -- Course Project for EM388F: Fracture Mechanics, Spring 2008

Risk-based Fatigue Estimate of Deep Water Risers -- Course Project for EM388F: Fracture Mechanics, Spring 2008 Rsk-based Fatgue Estmate of Deep Water Rsers -- Course Project for EM388F: Fracture Mechancs, Sprng 2008 Chen Sh Department of Cvl, Archtectural, and Envronmental Engneerng The Unversty of Texas at Austn

More information

Prediction of Disability Frequencies in Life Insurance

Prediction of Disability Frequencies in Life Insurance Predcton of Dsablty Frequences n Lfe Insurance Bernhard Köng Fran Weber Maro V. Wüthrch October 28, 2011 Abstract For the predcton of dsablty frequences, not only the observed, but also the ncurred but

More information

Causal, Explanatory Forecasting. Analysis. Regression Analysis. Simple Linear Regression. Which is Independent? Forecasting

Causal, Explanatory Forecasting. Analysis. Regression Analysis. Simple Linear Regression. Which is Independent? Forecasting Causal, Explanatory Forecastng Assumes cause-and-effect relatonshp between system nputs and ts output Forecastng wth Regresson Analyss Rchard S. Barr Inputs System Cause + Effect Relatonshp The job of

More information

A hybrid global optimization algorithm based on parallel chaos optimization and outlook algorithm

A hybrid global optimization algorithm based on parallel chaos optimization and outlook algorithm Avalable onlne www.ocpr.com Journal of Chemcal and Pharmaceutcal Research, 2014, 6(7):1884-1889 Research Artcle ISSN : 0975-7384 CODEN(USA) : JCPRC5 A hybrd global optmzaton algorthm based on parallel

More information

Lecture 5,6 Linear Methods for Classification. Summary

Lecture 5,6 Linear Methods for Classification. Summary Lecture 5,6 Lnear Methods for Classfcaton Rce ELEC 697 Farnaz Koushanfar Fall 2006 Summary Bayes Classfers Lnear Classfers Lnear regresson of an ndcator matrx Lnear dscrmnant analyss (LDA) Logstc regresson

More information

Project Networks With Mixed-Time Constraints

Project Networks With Mixed-Time Constraints Project Networs Wth Mxed-Tme Constrants L Caccetta and B Wattananon Western Australan Centre of Excellence n Industral Optmsaton (WACEIO) Curtn Unversty of Technology GPO Box U1987 Perth Western Australa

More information

Realistic Image Synthesis

Realistic Image Synthesis Realstc Image Synthess - Combned Samplng and Path Tracng - Phlpp Slusallek Karol Myszkowsk Vncent Pegoraro Overvew: Today Combned Samplng (Multple Importance Samplng) Renderng and Measurng Equaton Random

More information

Performance attribution for multi-layered investment decisions

Performance attribution for multi-layered investment decisions Performance attrbuton for mult-layered nvestment decsons 880 Thrd Avenue 7th Floor Ne Yor, NY 10022 212.866.9200 t 212.866.9201 f qsnvestors.com Inna Oounova Head of Strategc Asset Allocaton Portfolo Management

More information

A DYNAMIC CRASHING METHOD FOR PROJECT MANAGEMENT USING SIMULATION-BASED OPTIMIZATION. Michael E. Kuhl Radhamés A. Tolentino-Peña

A DYNAMIC CRASHING METHOD FOR PROJECT MANAGEMENT USING SIMULATION-BASED OPTIMIZATION. Michael E. Kuhl Radhamés A. Tolentino-Peña Proceedngs of the 2008 Wnter Smulaton Conference S. J. Mason, R. R. Hll, L. Mönch, O. Rose, T. Jefferson, J. W. Fowler eds. A DYNAMIC CRASHING METHOD FOR PROJECT MANAGEMENT USING SIMULATION-BASED OPTIMIZATION

More information

"Research Note" APPLICATION OF CHARGE SIMULATION METHOD TO ELECTRIC FIELD CALCULATION IN THE POWER CABLES *

Research Note APPLICATION OF CHARGE SIMULATION METHOD TO ELECTRIC FIELD CALCULATION IN THE POWER CABLES * Iranan Journal of Scence & Technology, Transacton B, Engneerng, ol. 30, No. B6, 789-794 rnted n The Islamc Republc of Iran, 006 Shraz Unversty "Research Note" ALICATION OF CHARGE SIMULATION METHOD TO ELECTRIC

More information

Logistic Regression. Lecture 4: More classifiers and classes. Logistic regression. Adaboost. Optimization. Multiple class classification

Logistic Regression. Lecture 4: More classifiers and classes. Logistic regression. Adaboost. Optimization. Multiple class classification Lecture 4: More classfers and classes C4B Machne Learnng Hlary 20 A. Zsserman Logstc regresson Loss functons revsted Adaboost Loss functons revsted Optmzaton Multple class classfcaton Logstc Regresson

More information

RESEARCH ON DUAL-SHAKER SINE VIBRATION CONTROL. Yaoqi FENG 1, Hanping QIU 1. China Academy of Space Technology (CAST) yaoqi.feng@yahoo.

RESEARCH ON DUAL-SHAKER SINE VIBRATION CONTROL. Yaoqi FENG 1, Hanping QIU 1. China Academy of Space Technology (CAST) yaoqi.feng@yahoo. ICSV4 Carns Australa 9- July, 007 RESEARCH ON DUAL-SHAKER SINE VIBRATION CONTROL Yaoq FENG, Hanpng QIU Dynamc Test Laboratory, BISEE Chna Academy of Space Technology (CAST) yaoq.feng@yahoo.com Abstract

More information

Abstract. Clustering ensembles have emerged as a powerful method for improving both the

Abstract. Clustering ensembles have emerged as a powerful method for improving both the Clusterng Ensembles: {topchyal, Models jan, of punch}@cse.msu.edu Consensus and Weak Parttons * Alexander Topchy, Anl K. Jan, and Wllam Punch Department of Computer Scence and Engneerng, Mchgan State Unversty

More information

Optimal Bidding Strategies for Generation Companies in a Day-Ahead Electricity Market with Risk Management Taken into Account

Optimal Bidding Strategies for Generation Companies in a Day-Ahead Electricity Market with Risk Management Taken into Account Amercan J. of Engneerng and Appled Scences (): 8-6, 009 ISSN 94-700 009 Scence Publcatons Optmal Bddng Strateges for Generaton Companes n a Day-Ahead Electrcty Market wth Rsk Management Taken nto Account

More information

Time Series Analysis in Studies of AGN Variability. Bradley M. Peterson The Ohio State University

Time Series Analysis in Studies of AGN Variability. Bradley M. Peterson The Ohio State University Tme Seres Analyss n Studes of AGN Varablty Bradley M. Peterson The Oho State Unversty 1 Lnear Correlaton Degree to whch two parameters are lnearly correlated can be expressed n terms of the lnear correlaton

More information

Lecture 3: Force of Interest, Real Interest Rate, Annuity

Lecture 3: Force of Interest, Real Interest Rate, Annuity Lecture 3: Force of Interest, Real Interest Rate, Annuty Goals: Study contnuous compoundng and force of nterest Dscuss real nterest rate Learn annuty-mmedate, and ts present value Study annuty-due, and

More information

Credit Limit Optimization (CLO) for Credit Cards

Credit Limit Optimization (CLO) for Credit Cards Credt Lmt Optmzaton (CLO) for Credt Cards Vay S. Desa CSCC IX, Ednburgh September 8, 2005 Copyrght 2003, SAS Insttute Inc. All rghts reserved. SAS Propretary Agenda Background Tradtonal approaches to credt

More information

Ring structure of splines on triangulations

Ring structure of splines on triangulations www.oeaw.ac.at Rng structure of splnes on trangulatons N. Vllamzar RICAM-Report 2014-48 www.rcam.oeaw.ac.at RING STRUCTURE OF SPLINES ON TRIANGULATIONS NELLY VILLAMIZAR Introducton For a trangulated regon

More information

Prediction of Disability Frequencies in Life Insurance

Prediction of Disability Frequencies in Life Insurance 1 Predcton of Dsablty Frequences n Lfe Insurance Bernhard Köng 1, Fran Weber 1, Maro V. Wüthrch 2 Abstract: For the predcton of dsablty frequences, not only the observed, but also the ncurred but not yet

More information

ECE544NA Final Project: Robust Machine Learning Hardware via Classifier Ensemble

ECE544NA Final Project: Robust Machine Learning Hardware via Classifier Ensemble 1 ECE544NA Fnal Project: Robust Machne Learnng Hardware va Classfer Ensemble Sa Zhang, szhang12@llnos.edu Dept. of Electr. & Comput. Eng., Unv. of Illnos at Urbana-Champagn, Urbana, IL, USA Abstract In

More information

An Enhanced Super-Resolution System with Improved Image Registration, Automatic Image Selection, and Image Enhancement

An Enhanced Super-Resolution System with Improved Image Registration, Automatic Image Selection, and Image Enhancement An Enhanced Super-Resoluton System wth Improved Image Regstraton, Automatc Image Selecton, and Image Enhancement Yu-Chuan Kuo ( ), Chen-Yu Chen ( ), and Chou-Shann Fuh ( ) Department of Computer Scence

More information

Feature selection for intrusion detection. Slobodan Petrović NISlab, Gjøvik University College

Feature selection for intrusion detection. Slobodan Petrović NISlab, Gjøvik University College Feature selecton for ntruson detecton Slobodan Petrovć NISlab, Gjøvk Unversty College Contents The feature selecton problem Intruson detecton Traffc features relevant for IDS The CFS measure The mrmr measure

More information

A Simple Approach to Clustering in Excel

A Simple Approach to Clustering in Excel A Smple Approach to Clusterng n Excel Aravnd H Center for Computatonal Engneerng and Networng Amrta Vshwa Vdyapeetham, Combatore, Inda C Rajgopal Center for Computatonal Engneerng and Networng Amrta Vshwa

More information

Cluster Analysis. Cluster Analysis

Cluster Analysis. Cluster Analysis Cluster Analyss Cluster Analyss What s Cluster Analyss? Types of Data n Cluster Analyss A Categorzaton of Maor Clusterng Methos Parttonng Methos Herarchcal Methos Densty-Base Methos Gr-Base Methos Moel-Base

More information

INVESTIGATION OF VEHICULAR USERS FAIRNESS IN CDMA-HDR NETWORKS

INVESTIGATION OF VEHICULAR USERS FAIRNESS IN CDMA-HDR NETWORKS 21 22 September 2007, BULGARIA 119 Proceedngs of the Internatonal Conference on Informaton Technologes (InfoTech-2007) 21 st 22 nd September 2007, Bulgara vol. 2 INVESTIGATION OF VEHICULAR USERS FAIRNESS

More information

Mining Feature Importance: Applying Evolutionary Algorithms within a Web-based Educational System

Mining Feature Importance: Applying Evolutionary Algorithms within a Web-based Educational System Mnng Feature Importance: Applyng Evolutonary Algorthms wthn a Web-based Educatonal System Behrouz MINAEI-BIDGOLI 1, and Gerd KORTEMEYER 2, and Wllam F. PUNCH 1 1 Genetc Algorthms Research and Applcatons

More information

The OC Curve of Attribute Acceptance Plans

The OC Curve of Attribute Acceptance Plans The OC Curve of Attrbute Acceptance Plans The Operatng Characterstc (OC) curve descrbes the probablty of acceptng a lot as a functon of the lot s qualty. Fgure 1 shows a typcal OC Curve. 10 8 6 4 1 3 4

More information

DI Fund Sufficiency Evaluation Methodological Recommendations and DIA Russia Practice

DI Fund Sufficiency Evaluation Methodological Recommendations and DIA Russia Practice DI Fund Suffcency Evaluaton Methodologcal Recommendatons and DIA Russa Practce Andre G. Melnkov Deputy General Drector DIA Russa THE DEPOSIT INSURANCE CONFERENCE IN THE MENA REGION AMMAN-JORDAN, 18 20

More information

Vision Mouse. Saurabh Sarkar a* University of Cincinnati, Cincinnati, USA ABSTRACT 1. INTRODUCTION

Vision Mouse. Saurabh Sarkar a* University of Cincinnati, Cincinnati, USA ABSTRACT 1. INTRODUCTION Vson Mouse Saurabh Sarkar a* a Unversty of Cncnnat, Cncnnat, USA ABSTRACT The report dscusses a vson based approach towards trackng of eyes and fngers. The report descrbes the process of locatng the possble

More information

Single and multiple stage classifiers implementing logistic discrimination

Single and multiple stage classifiers implementing logistic discrimination Sngle and multple stage classfers mplementng logstc dscrmnaton Hélo Radke Bttencourt 1 Dens Alter de Olvera Moraes 2 Vctor Haertel 2 1 Pontfíca Unversdade Católca do Ro Grande do Sul - PUCRS Av. Ipranga,

More information

Using Mixture Covariance Matrices to Improve Face and Facial Expression Recognitions

Using Mixture Covariance Matrices to Improve Face and Facial Expression Recognitions Usng Mxture Covarance Matrces to Improve Face and Facal Expresson Recogntons Carlos E. homaz, Duncan F. Glles and Raul Q. Fetosa 2 Imperal College of Scence echnology and Medcne, Department of Computng,

More information

GRAVITY DATA VALIDATION AND OUTLIER DETECTION USING L 1 -NORM

GRAVITY DATA VALIDATION AND OUTLIER DETECTION USING L 1 -NORM GRAVITY DATA VALIDATION AND OUTLIER DETECTION USING L 1 -NORM BARRIOT Jean-Perre, SARRAILH Mchel BGI/CNES 18.av.E.Beln 31401 TOULOUSE Cedex 4 (France) Emal: jean-perre.barrot@cnes.fr 1/Introducton The

More information

The Application of Fractional Brownian Motion in Option Pricing

The Application of Fractional Brownian Motion in Option Pricing Vol. 0, No. (05), pp. 73-8 http://dx.do.org/0.457/jmue.05.0..6 The Applcaton of Fractonal Brownan Moton n Opton Prcng Qng-xn Zhou School of Basc Scence,arbn Unversty of Commerce,arbn zhouqngxn98@6.com

More information

Calculation of Sampling Weights

Calculation of Sampling Weights Perre Foy Statstcs Canada 4 Calculaton of Samplng Weghts 4.1 OVERVIEW The basc sample desgn used n TIMSS Populatons 1 and 2 was a two-stage stratfed cluster desgn. 1 The frst stage conssted of a sample

More information

Abstract. 1. Introduction

Abstract. 1. Introduction System and Methodology for Usng Moble Phones n Lve Remote Montorng of Physcal Actvtes Hamed Ketabdar and Matt Lyra Qualty and Usablty Lab, Deutsche Telekom Laboratores, TU Berln hamed.ketabdar@telekom.de,

More information

Binomial Link Functions. Lori Murray, Phil Munz

Binomial Link Functions. Lori Murray, Phil Munz Bnomal Lnk Functons Lor Murray, Phl Munz Bnomal Lnk Functons Logt Lnk functon: ( p) p ln 1 p Probt Lnk functon: ( p) 1 ( p) Complentary Log Log functon: ( p) ln( ln(1 p)) Motvatng Example A researcher

More information

STATISTICAL DATA ANALYSIS IN EXCEL

STATISTICAL DATA ANALYSIS IN EXCEL Mcroarray Center STATISTICAL DATA ANALYSIS IN EXCEL Lecture 6 Some Advanced Topcs Dr. Petr Nazarov 14-01-013 petr.nazarov@crp-sante.lu Statstcal data analyss n Ecel. 6. Some advanced topcs Correcton for

More information

Inter-Ing 2007. INTERDISCIPLINARITY IN ENGINEERING SCIENTIFIC INTERNATIONAL CONFERENCE, TG. MUREŞ ROMÂNIA, 15-16 November 2007.

Inter-Ing 2007. INTERDISCIPLINARITY IN ENGINEERING SCIENTIFIC INTERNATIONAL CONFERENCE, TG. MUREŞ ROMÂNIA, 15-16 November 2007. Inter-Ing 2007 INTERDISCIPLINARITY IN ENGINEERING SCIENTIFIC INTERNATIONAL CONFERENCE, TG. MUREŞ ROMÂNIA, 15-16 November 2007. UNCERTAINTY REGION SIMULATION FOR A SERIAL ROBOT STRUCTURE MARIUS SEBASTIAN

More information

An Alternative Way to Measure Private Equity Performance

An Alternative Way to Measure Private Equity Performance An Alternatve Way to Measure Prvate Equty Performance Peter Todd Parlux Investment Technology LLC Summary Internal Rate of Return (IRR) s probably the most common way to measure the performance of prvate

More information

An Interest-Oriented Network Evolution Mechanism for Online Communities

An Interest-Oriented Network Evolution Mechanism for Online Communities An Interest-Orented Network Evoluton Mechansm for Onlne Communtes Cahong Sun and Xaopng Yang School of Informaton, Renmn Unversty of Chna, Bejng 100872, P.R. Chna {chsun,yang}@ruc.edu.cn Abstract. Onlne

More information

Estimating the Number of Clusters in Genetics of Acute Lymphoblastic Leukemia Data

Estimating the Number of Clusters in Genetics of Acute Lymphoblastic Leukemia Data Journal of Al Azhar Unversty-Gaza (Natural Scences), 2011, 13 : 109-118 Estmatng the Number of Clusters n Genetcs of Acute Lymphoblastc Leukema Data Mahmoud K. Okasha, Khaled I.A. Almghar Department of

More information

Luby s Alg. for Maximal Independent Sets using Pairwise Independence

Luby s Alg. for Maximal Independent Sets using Pairwise Independence Lecture Notes for Randomzed Algorthms Luby s Alg. for Maxmal Independent Sets usng Parwse Independence Last Updated by Erc Vgoda on February, 006 8. Maxmal Independent Sets For a graph G = (V, E), an ndependent

More information

SCHEDULING OF CONSTRUCTION PROJECTS BY MEANS OF EVOLUTIONARY ALGORITHMS

SCHEDULING OF CONSTRUCTION PROJECTS BY MEANS OF EVOLUTIONARY ALGORITHMS SCHEDULING OF CONSTRUCTION PROJECTS BY MEANS OF EVOLUTIONARY ALGORITHMS Magdalena Rogalska 1, Wocech Bożeko 2,Zdzsław Heduck 3, 1 Lubln Unversty of Technology, 2- Lubln, Nadbystrzycka 4., Poland. E-mal:rogalska@akropols.pol.lubln.pl

More information

THE METHOD OF LEAST SQUARES THE METHOD OF LEAST SQUARES

THE METHOD OF LEAST SQUARES THE METHOD OF LEAST SQUARES The goal: to measure (determne) an unknown quantty x (the value of a RV X) Realsaton: n results: y 1, y 2,..., y j,..., y n, (the measured values of Y 1, Y 2,..., Y j,..., Y n ) every result s encumbered

More information

The covariance is the two variable analog to the variance. The formula for the covariance between two variables is

The covariance is the two variable analog to the variance. The formula for the covariance between two variables is Regresson Lectures So far we have talked only about statstcs that descrbe one varable. What we are gong to be dscussng for much of the remander of the course s relatonshps between two or more varables.

More information

MAPP. MERIS level 3 cloud and water vapour products. Issue: 1. Revision: 0. Date: 9.12.1998. Function Name Organisation Signature Date

MAPP. MERIS level 3 cloud and water vapour products. Issue: 1. Revision: 0. Date: 9.12.1998. Function Name Organisation Signature Date Ttel: Project: Doc. No.: MERIS level 3 cloud and water vapour products MAPP MAPP-ATBD-ClWVL3 Issue: 1 Revson: 0 Date: 9.12.1998 Functon Name Organsaton Sgnature Date Author: Bennartz FUB Preusker FUB Schüller

More information

Extending Probabilistic Dynamic Epistemic Logic

Extending Probabilistic Dynamic Epistemic Logic Extendng Probablstc Dynamc Epstemc Logc Joshua Sack May 29, 2008 Probablty Space Defnton A probablty space s a tuple (S, A, µ), where 1 S s a set called the sample space. 2 A P(S) s a σ-algebra: a set

More information

PAS: A Packet Accounting System to Limit the Effects of DoS & DDoS. Debish Fesehaye & Klara Naherstedt University of Illinois-Urbana Champaign

PAS: A Packet Accounting System to Limit the Effects of DoS & DDoS. Debish Fesehaye & Klara Naherstedt University of Illinois-Urbana Champaign PAS: A Packet Accountng System to Lmt the Effects of DoS & DDoS Debsh Fesehaye & Klara Naherstedt Unversty of Illnos-Urbana Champagn DoS and DDoS DDoS attacks are ncreasng threats to our dgtal world. Exstng

More information

PREDICTION OF MISSING DATA IN CARDIOTOCOGRAMS USING THE EXPECTATION MAXIMIZATION ALGORITHM

PREDICTION OF MISSING DATA IN CARDIOTOCOGRAMS USING THE EXPECTATION MAXIMIZATION ALGORITHM 18-19 October 2001, Hotel Kontokal Bay, Corfu PREDICTIO OF MISSIG DATA I CARDIOTOCOGRAMS USIG THE EXPECTATIO MAXIMIZATIO ALGORITHM G. okas Department of Electrcal and Computer Engneerng, Unversty of Patras,

More information

Robust Design of Public Storage Warehouses. Yeming (Yale) Gong EMLYON Business School

Robust Design of Public Storage Warehouses. Yeming (Yale) Gong EMLYON Business School Robust Desgn of Publc Storage Warehouses Yemng (Yale) Gong EMLYON Busness School Rene de Koster Rotterdam school of management, Erasmus Unversty Abstract We apply robust optmzaton and revenue management

More information

1. Math 210 Finite Mathematics

1. Math 210 Finite Mathematics 1. ath 210 Fnte athematcs Chapter 5.2 and 5.3 Annutes ortgages Amortzaton Professor Rchard Blecksmth Dept. of athematcal Scences Northern Illnos Unversty ath 210 Webste: http://math.nu.edu/courses/math210

More information

THE APPLICATION OF DATA MINING TECHNIQUES AND MULTIPLE CLASSIFIERS TO MARKETING DECISION

THE APPLICATION OF DATA MINING TECHNIQUES AND MULTIPLE CLASSIFIERS TO MARKETING DECISION Internatonal Journal of Electronc Busness Management, Vol. 3, No. 4, pp. 30-30 (2005) 30 THE APPLICATION OF DATA MINING TECHNIQUES AND MULTIPLE CLASSIFIERS TO MARKETING DECISION Yu-Mn Chang *, Yu-Cheh

More information

Bayesian Cluster Ensembles

Bayesian Cluster Ensembles Bayesan Cluster Ensembles Hongjun Wang 1, Hanhua Shan 2 and Arndam Banerjee 2 1 Informaton Research Insttute, Southwest Jaotong Unversty, Chengdu, Schuan, 610031, Chna 2 Department of Computer Scence &

More information

NPAR TESTS. One-Sample Chi-Square Test. Cell Specification. Observed Frequencies 1O i 6. Expected Frequencies 1EXP i 6

NPAR TESTS. One-Sample Chi-Square Test. Cell Specification. Observed Frequencies 1O i 6. Expected Frequencies 1EXP i 6 PAR TESTS If a WEIGHT varable s specfed, t s used to replcate a case as many tmes as ndcated by the weght value rounded to the nearest nteger. If the workspace requrements are exceeded and samplng has

More information

Mixtures of Factor Analyzers with Common Factor Loadings for the Clustering and Visualisation of High-Dimensional Data

Mixtures of Factor Analyzers with Common Factor Loadings for the Clustering and Visualisation of High-Dimensional Data Mxtures of Factor Analyzers wth Common Factor Loadngs for the Clusterng and Vsualsaton of Hgh-Dmensonal Data Jangsun Baek 1 and Geoffrey J. McLachlan 2 1 Department of Statstcs, Chonnam Natonal Unversty,

More information

MATHEMATICAL ENGINEERING TECHNICAL REPORTS. Sequential Optimizing Investing Strategy with Neural Networks

MATHEMATICAL ENGINEERING TECHNICAL REPORTS. Sequential Optimizing Investing Strategy with Neural Networks MATHEMATICAL ENGINEERING TECHNICAL REPORTS Sequental Optmzng Investng Strategy wth Neural Networks Ryo ADACHI and Akmch TAKEMURA METR 2010 03 February 2010 DEPARTMENT OF MATHEMATICAL INFORMATICS GRADUATE

More information

Georey E. Hinton. University oftoronto. Email: zoubin@cs.toronto.edu. Technical Report CRG-TR-96-1. May 21, 1996 (revised Feb 27, 1997) Abstract

Georey E. Hinton. University oftoronto. Email: zoubin@cs.toronto.edu. Technical Report CRG-TR-96-1. May 21, 1996 (revised Feb 27, 1997) Abstract The EM Algorthm for Mxtures of Factor Analyzers Zoubn Ghahraman Georey E. Hnton Department of Computer Scence Unversty oftoronto 6 Kng's College Road Toronto, Canada M5S A4 Emal: zoubn@cs.toronto.edu Techncal

More information

Rate-Based Daily Arrival Process Models with Application to Call Centers

Rate-Based Daily Arrival Process Models with Application to Call Centers Submtted to Operatons Research manuscrpt (Please, provde the manuscrpt number!) Authors are encouraged to submt new papers to INFORMS journals by means of a style fle template, whch ncludes the journal

More information

Data Visualization by Pairwise Distortion Minimization

Data Visualization by Pairwise Distortion Minimization Communcatons n Statstcs, Theory and Methods 34 (6), 005 Data Vsualzaton by Parwse Dstorton Mnmzaton By Marc Sobel, and Longn Jan Lateck* Department of Statstcs and Department of Computer and Informaton

More information

Texas Instruments 30X IIS Calculator

Texas Instruments 30X IIS Calculator Texas Instruments 30X IIS Calculator Keystrokes for the TI-30X IIS are shown for a few topcs n whch keystrokes are unque. Start by readng the Quk Start secton. Then, before begnnng a specfc unt of the

More information

A Probabilistic Theory of Coherence

A Probabilistic Theory of Coherence A Probablstc Theory of Coherence BRANDEN FITELSON. The Coherence Measure C Let E be a set of n propostons E,..., E n. We seek a probablstc measure C(E) of the degree of coherence of E. Intutvely, we want

More information

The Greedy Method. Introduction. 0/1 Knapsack Problem

The Greedy Method. Introduction. 0/1 Knapsack Problem The Greedy Method Introducton We have completed data structures. We now are gong to look at algorthm desgn methods. Often we are lookng at optmzaton problems whose performance s exponental. For an optmzaton

More information

Mooring Pattern Optimization using Genetic Algorithms

Mooring Pattern Optimization using Genetic Algorithms 6th World Congresses of Structural and Multdscplnary Optmzaton Ro de Janero, 30 May - 03 June 005, Brazl Moorng Pattern Optmzaton usng Genetc Algorthms Alonso J. Juvnao Carbono, Ivan F. M. Menezes Luz

More information

Portfolio Loss Distribution

Portfolio Loss Distribution Portfolo Loss Dstrbuton Rsky assets n loan ortfolo hghly llqud assets hold-to-maturty n the bank s balance sheet Outstandngs The orton of the bank asset that has already been extended to borrowers. Commtment

More information

CHAPTER 14 MORE ABOUT REGRESSION

CHAPTER 14 MORE ABOUT REGRESSION CHAPTER 14 MORE ABOUT REGRESSION We learned n Chapter 5 that often a straght lne descrbes the pattern of a relatonshp between two quanttatve varables. For nstance, n Example 5.1 we explored the relatonshp

More information

Probabilistic Linear Classifier: Logistic Regression. CS534-Machine Learning

Probabilistic Linear Classifier: Logistic Regression. CS534-Machine Learning robablstc Lnear Classfer: Logstc Regresson CS534-Machne Learnng Three Man Approaches to learnng a Classfer Learn a classfer: a functon f, ŷ f Learn a probablstc dscrmnatve model,.e., the condtonal dstrbuton

More information

FORECASTING TELECOMMUNICATION NEW SERVICE DEMAND BY ANALOGY METHOD AND COMBINED FORECAST

FORECASTING TELECOMMUNICATION NEW SERVICE DEMAND BY ANALOGY METHOD AND COMBINED FORECAST Yugoslav Journal of Operatons Research 5 (005), Number, 97-07 FORECAING ELECOMMUNICAION NEW ERVICE DEMAND BY ANALOGY MEHOD AND COMBINED FORECA Feng-Jenq LIN Department of Appled Economcs Natonal I-Lan

More information

Using Series to Analyze Financial Situations: Present Value

Using Series to Analyze Financial Situations: Present Value 2.8 Usng Seres to Analyze Fnancal Stuatons: Present Value In the prevous secton, you learned how to calculate the amount, or future value, of an ordnary smple annuty. The amount s the sum of the accumulated

More information

Risk Model of Long-Term Production Scheduling in Open Pit Gold Mining

Risk Model of Long-Term Production Scheduling in Open Pit Gold Mining Rsk Model of Long-Term Producton Schedulng n Open Pt Gold Mnng R Halatchev 1 and P Lever 2 ABSTRACT Open pt gold mnng s an mportant sector of the Australan mnng ndustry. It uses large amounts of nvestments,

More information

On the Optimal Control of a Cascade of Hydro-Electric Power Stations

On the Optimal Control of a Cascade of Hydro-Electric Power Stations On the Optmal Control of a Cascade of Hydro-Electrc Power Statons M.C.M. Guedes a, A.F. Rbero a, G.V. Smrnov b and S. Vlela c a Department of Mathematcs, School of Scences, Unversty of Porto, Portugal;

More information

Optimal outpatient appointment scheduling

Optimal outpatient appointment scheduling Health Care Manage Sc (27) 1:217 229 DOI 1.17/s1729-7-915- Optmal outpatent appontment schedulng Gudo C. Kaandorp Ger Koole Receved: 15 March 26 / Accepted: 28 February 27 / Publshed onlne: 23 May 27 Sprnger

More information

A Novel Methodology of Working Capital Management for Large. Public Constructions by Using Fuzzy S-curve Regression

A Novel Methodology of Working Capital Management for Large. Public Constructions by Using Fuzzy S-curve Regression Novel Methodology of Workng Captal Management for Large Publc Constructons by Usng Fuzzy S-curve Regresson Cheng-Wu Chen, Morrs H. L. Wang and Tng-Ya Hseh Department of Cvl Engneerng, Natonal Central Unversty,

More information