# Gaussian Processes for Regression: A Quick Introduction

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Gaussian Processes for Regression A Quick Introduction M Ebden, August 28 Coents to MOTIVATION Figure illustrates a typical eaple of a prediction proble given soe noisy observations of a dependent variable at certain values of the independent variable, what is our best estiate of the dependent variable at a new value,? If we epect the underlying function to be linear, and can ake soe assuptions about the input data, we ight use a least-squares ethod to fit a straight line (linear regression) Moreover, if we suspect ay also be quadratic, cubic, or even nonpolynoial, we can use the principles of odel selection to choose aong the various possibilities Gaussian process regression (GPR) is an even finer approach than this Rather than claiing relates to soe specific odels (eg ), a Gaussian process can represent obliquely, but rigorously, by letting the data speak ore clearly for theselves GPR is still a for of supervised learning, but the training data are harnessed in a subtler way As such, GPR is a less paraetric tool However, it s not copletely free-for, and if we re unwilling to ake even basic assuptions about, then ore general techniques should be considered, including those underpinned by the principle of aiu entropy; Chapter 6 of Sivia and Skilling (26) offers an introduction 5 5? y Figure Given si noisy data points (error bars are indicated with vertical lines), we are interested in estiating a seventh at

2 ( ( ( ( ( Y WW 2 DEFINITION OF A GAUSSIAN PROCESS Gaussian processes (GPs) etend ultivariate Gaussian distributions to infinite diensionality Forally, a Gaussian process generates data located throughout soe doain such that any finite subset of the range follows a ultivariate Gaussian distribution Now, the observations in an arbitrary data set,! "#%\$'&, can always be iagined as a single point sapled fro soe ultivariate ( -variate) Gaussian distribution, after enough thought Hence, working backwards, this data set can be partnered with a GP Thus GPs are as universal as they are siple Very often, it s assued that the ean of this partner GP is zero everywhere What )*'+, relates one observation to another in such cases is just the covariance function, ( A popular choice is the squared eponential, -# + /2 +, ; () %< = where the aiu allowable covariance is defined as / 3 this should be high for functions which cover a broad range on the ais If?>'+, then ( -* +, approaches this aiu, eaning is nearly perfectly correlated with This is good for our function to look sooth, neighbours ust be alike Now if is distant fro '+, we have instead ( -* +A B>C, ie the two points cannot see each other So, for eaple, during interpolation at new values, distant observations will have negligible effect How uch effect this separation has will depend on the length paraeter, <, so there is uch fleibility built into () Not quite enough fleibility though the data are often noisy as well, fro easureent errors and so on Each observation can be thought of as related to an underlying function through a Gaussian noise odel D -FE G#/ \$ H (2) soething which should look failiar to those who ve done regression before Re- gression is the search for Purely for siplicity of eposition in the net page, we take the novel approach of folding the noise into ( -# '+,, by writing )* + / 3 '+, 4I576J8 K< L/ = \$7M -# + I (3) where M -# '+, is the Kronecker delta function (When ost people use Gaussian processes, they keep /G\$ separate fro ( -* +, However, our redefinition of ( )* is equally suitable for working with probles of the sort posed in Figure So, given observations, our objective is to predict, not the actual ; their epected values are identical according to (2), but their variances differ owing to the observational noise process eg in Figure, the epected value of, and of, is the dot at ) To prepare for GPR, we calculate the covariance function, (3), aong all possible cobinations of these points, suarizing our findings in three atrices 2;# ( ;* TS S!S ( 2;# '\$ VXW N POQ # ( * TS S!S ( # '\$ RQQ (4) '\$2# I ( \$ * US S!S ( '\$2#'\$ N [Z ( %*2 ( %* \S!S S ( K* \$G ^] N _ ( * I N (5) Confir for yourself that the diagonal eleents of are / 3 `/ \$, and that its etree off-diagonal eleents tend to zero when spans a large enough doain 2

3 N Z QO Q QQ QRQ N W WW W W Y 3 HOW TO REGRESS USING GAUSSIAN PROCESSES Since the key assuption in GP odelling is that our data can be represented as a saple fro a ultivariate Gaussian distribution, we have that 8 a =cb EedGf- 8 N Nhg N i =kj (6) on given the data, how likely is a certain prediction for? As where l indicates atri transposition We are of course interested in the conditional probability eplained ore slowly in the Appendi, the probability follows a Gaussian distribution n b E N NLp N i N Nqp N g H (7) Our best estiate for is the ean of this distribution N NLp (8) and the uncertainty in our estiate is captured in its variance r;skt! N _ We re now ready to tackle the data in Figure There are vu observations, at wh y% o B B Gz%y N N p N g (9) G { G oy G o ] We know /G\$D G } fro the error bars With judicious choices of / 3 and < (ore on this later), we have enough to calculate a covariance atri using (4) zk {a ~z zk y VXW {a zk y%u }K{ z y%u zk y { G ~azt }%{ y z; yk {o~ Gz% {a yk z; yku G y G az {o~ yku z; N Fro (5) we also have iƒ zk and N Z G }o~tgzk %} }oy {ou yk~ ] 2 Fro (8) and (9), G y and r;skt v 3 Figure shows a data point with a question ark underneath, representing the estiation of the dependent variable at G We can repeat the above procedure for various other points spread over soe portion of the ais, as shown in Figure 2 (In fact, equivalently, we could N avoid the repetition by perforing the above procedure once with suitably larger N and N _ atrices In this case, since there are, test points spread over the ais, _ would be of size,,) Rather than plotting siple error bars, we ve decided to plot %ugˆ r;skt, giving a 95% confidence interval 3

4 ( 5 5 y Figure 2 The solid line indicates an estiation of for, values of Pointwise 95% confidence intervals are shaded 4 GPR IN THE REAL WORLD The reliability of our regression is dependent on how well we select the covariance function Clearly if its paraeters call the Š <Œi/ 3 #/G\$ & are not chosen sensibly, the result is nonsense Our aiu a posteriori estiate of occurs when n w is at its greatest Bayes theore tells us that, assuing we have little prior knowledge about what should be, this corresponds to aiizing Ž % " given by Ž o " n w g NLp Ž % n N n n w, Ž % K () Siply run your favourite ultivariate optiization algorith (eg conjugate gradients, Nelder-Mead siple, etc) on this equation and you ve found a pretty good choice for ; in our eaple, <- and / 3 oz It s only pretty good because, of course, Thoas Bayes is rolling in his grave Why coend just one answer for, when you can integrate everything over the any different possible choices for? Chapter 5 of Rasussen and Willias (26) presents the equations necessary in this case Finally, if you feel you ve grasped the toy proble in Figure 2, the net two eaples handle ore coplicated cases Figure 3(a), in addition to a long-ter downward trend, has soe fluctuations, so we ight use a ore sophisticated covariance function -* + / '+, 3 4I576 8 K< / = 3 '+, 4I576J8 K< L/ = \$ M -# + I () The first ter takes into account the sall vicissitudes of the dependent variable, and the second ter has a longer length paraeter ( < > u%< ) to represent its long-ter 4

5 ( (a) (b) y y Figure 3 Estiation of (solid line) for a function with (a) short-ter and long-ter dynaics, and (b) long-ter dynaics and a periodic eleent Observations are shown as crosses trend Covariance functions can be grown in this way ad infinitu, to suit the copleity of your particular data The function looks as if it ight contain a periodic eleent, but it s difficult to be sure Let s consider another function, which we re told has a periodic eleent The solid line in Figure 3(b) was regressed with the following covariance function )* + v/ 3 4I576J8 '+, K< = 4I576 # š7% œa + ^ ž&ÿl/ \$M )* + I (2) The first ter represents the hill-like trend over the long ter, and the second ter gives periodicity with frequency œ This is the first tie we ve encountered a case -* +,? > for where and '+ can be distant and yet still see each other (that is, ( + ) What if the dependent variable has other dynaics which, a priori, you epect to )*'+, N can be, provided is positive appear? There s no liit to how coplicated ( definite Chapter 4 of Rasussen and Willias (26) offers a good outline of the range of covariance functions you should keep in your toolkit Hang on a inute, you ask, isn t choosing a covariance function fro a toolkit a lot like choosing a odel type, such as linear versus cubic which we discussed at the outset? Well, there are indeed siilarities In fact, there is no way to perfor regression without iposing at least a odicu of structure on the data set; such is the nature of generative odelling However, it s worth repeating that Gaussian processes do allow the data to speak very clearly For eaple, there eists ecellent theoretical justification for the use of () in any settings (Rasussen and Willias (26), Section 43) You will still want to investigate carefully which covariance functions are appropriate for your data set Essentially, choosing aong alternative functions is a way of reflecting various fors of prior knowledge about the physical process under investigation 5

6 g g ª 5 DISCUSSION We ve presented a brief outline of the atheatics of GPR, but practical ipleentation of the above ideas requires the solution of a few algorithic hurdles as opposed to those of data analysis If you aren t a good coputer prograer, then the code for Figures and 2 is at ftp//ftprobotsoacuk/pub/outgoing/ebden/isc/gptutzip, and ore general code can be found at http//wwwgaussianprocessorg/gpl We ve erely scratched the surface of a powerful technique (MacKay, 998) First, although the focus has been on one-diensional inputs, it s siple to accept those of higher diension Whereas would then change fro a scalar to a vector, ( )* + would reain a scalar and so the aths overall would be virtually unchanged Second, the zero vector representing the ean of the ultivariate Gaussian distribution in (6) can be replaced with functions of Third, in addition to their use in regression, GPs are applicable to integration, global optiization, iture-of-eperts odels, unsupervised learning odels, and ore see Chapter 9 of Rasussen and Willias (26) The net tutorial will focus on their use in classification REFERENCES MacKay, D (998) In CM Bishop (Ed), Neural networks and achine learning (NATO ASI Series, Series F, Coputer and Systes Sciences, Vol 68, pp 33-66) Dordrecht Kluwer Acadeic Press Rasussen, C and C Willias (26) Gaussian Processes for Machine Learning MIT Press Sivia, D and J Skilling (26) Data Analysis A Bayesian Tutorial (second ed) Oford Science Publications APPENDIX Iagine a data saple taken fro soe ultivariate Gaussian distribution with zero ean and a covariance given by atri Now decopose arbitrarily into two consecutive subvectors and in other words, writing E f- b would be the sae as writing 8 =Db Eed f" 8 ª =«j (3) where, ª, and are the corresponding bits and pieces that ake up Interestingly, the conditional distribution of given is itself Gaussian-distributed If the covariance atri were diagonal or even block diagonal, then knowing wouldn t tell us anything about specifically, n E f" b On the other hand, if were nonzero, then soe atri algebra leads us to n E p b p g H ª (4) p The ean,, is known as the atri of regression coefficients, and the variance, ª, is the Schur copleent of in p In suary, if we know soe of, we can use that to infor our estiate of what the rest of ight be, thanks to the revealing off-diagonal eleents of 6

7 N Gaussian Processes for Classification A Quick Introduction M Ebden, August 28 Prerequisite reading Gaussian Processes for Regression OVERVIEW As entioned in the previous docuent, GPs can be applied to probles other than, it can regression For eaple, if the output of a GP is squashed onto the range represent the probability of a data point belonging to one of say two types, and voilà, we can ascertain classifications This is the subject of the current docuent The big difference between GPR and GPC is how the output data,, are linked to the underlying function outputs, They are no longer connected siply via a noise process as in (2) in the previous docuent, but are instead now discrete say L precisely for one class and B for the other In principle, we could try fitting a GP that produces an output of approiately for soe values of and approiately B for others, siulating this discretization Instead, we interpose the GP between the data and a squashing function; then, classification of a new data point involves two steps instead of one Evaluate a latent function which odels qualitatively how the likelihood of one class versus the other changes over the ais This is the GP using any sigoidal function, 2 Squash the output of this latent function onto G prob D n Writing these two steps scheatically, data, GP 7 latent function, %n sigoid H* class probability, The net section will walk you through ore slowly how such a classifier operates Section 3 eplains how to train the classifier, so perhaps we re presenting things in reverse order! Section 4 handles classification when there are ore than two classes Before we get started, a quick note on Although other fors will do, here we will prescribe it to be the cuulative Gaussian distribution, This ± -shaped function satisfies our needs, apping high into ²>, and low into «> A second quick note, revisiting (6) and (7) in the first docuent confir for yourself that, if there were no noise ( /G\$D v ), the two equations could be rewritten as 8 = b N g E d f- 8 N N _ =Ÿj () and n b E N NLp N _ N NLp N g H (2)

8 ˆ \$ 2 USING THE CLASSIFIER Suppose we ve trained a classifier fro input data, w, and their corresponding epertlabelled output data, And suppose that in the process we fored soe GP outputs corresponding to these data, which have soe uncertainty but ean values given by We re now ready to input a new data point,, in the left side of our scheatic, in order to deterine at the other end the probability 2 of its class ebership In the first step, finding the probability n is siilar to GPR, ie we adapt (2) n ² E N NLp N i N N + p N g I (3) N ( + will be eplained soon, N but for now consider it to be very siilar to ) In the second step, we squash to find the probability of class ebership, The epected value is µ %n * (4) This is the integral of a cuulative Gaussian ties a Gaussian, which can be solved analytically By Section 39 of Rasussen and Willias (26), the solution is d An eaple is depicted in Figure r;skt! ;j (5) 3 TRAINING THE GP IN THE CLASSIFIER Our objective now is to find N and +, so that we know everything about the GP producing (3), the first step of the classifier The second step of the classifier does not require training as it s a fied sigoidal function Aong the any GPs which could be partnered with our data set, naturally we d like to copare their usefulness quantitatively Considering the outputs of a certain GP, how likely they are to be appropriate for the training data can be decoposed using Bayes theore n w Ÿ n n w- n w- (6) Let s focus on the two factors in the nuerator Assuing the data set is iid, n ² ¹ º ¹ n ¹ I (7) Dropping the subscripts in the product, n is infored by our sigoid function, Specifically, n is by definition, and to coplete the picture, n? B» A terse way of cobining these two cases is to write n The second factor in the nuerator is n w- This is related to the output of the first step of our scheatic drawing, but first we re interested in the value of n w- which aiizes the posterior probability n w This occurs when the derivative 2

9 Predictive probability Latent function f() Figure (a) Toy classification dataset, where circles and crosses indicate class ebership of the input (training) data, at locations w is for one class and B for another, but for illustrative purposes we pretend ¼ë instead of B in this figure The solid line is the (ean) probability ) prob ƒ n!, ie the answer to our proble after successfully perforing GPC (b) The corresponding distribution of the latent function, not constrained to lie between and of (6) with respect to is zero, or equivalently and ore siply, when the derivative of its logarith is zero Doing this, and using the sae logic that produced () in the previous docuent, we find that N9½ Ž o " is the best for our proble Unfortunately, n I (8) where appears on both sides of the equation, so we ake an initial guess (zero is fine) and go through a few iterations The answer to (8) can be used directly in (3), so we ve found one of the two quantities we seek therein The variance of is given by the negative second derivative of the logarith of (6), N p which turns out to be ¾ p, with ¾À ½Á½ Ž % 2 n Making a Laplace approiation, we pretend n w is Gaussian distributed, ie n w bâ n w Ÿ LE N p ¾ p H (9) (This assuption is occasionally inaccurate, so if it yields poor classifications, better ways of characterizing the uncertainty in should be considered, for eaple via epectation propagation) 3

10 Ê g Now for a subtle point The fact that can vary eans that using (2) directly is inappropriate in particular, its ean is correct but its variance no longer tells the whole N story This is why we use the adapted version, (3), with + N instead of Since the varying quantity in (2), N, is being ultiplied by N p, we N N add N p cov N p N g to the variance in (2) Siplification leads to (3), in which +' N ¾ p With the GP now copletely specified, we re ready to use the classifier as described in the previous section GPC in the Real World As with GPR, the reliability of our classification is dependent on how well we select the covariance function in the GP in our first step The paraeters are ë <Œi/ 3 &, one fewer now because / \$ [ However, as usual, is optiized by aiizing n w, or (oitting on the righthand side of the equation), n w µ n This can be siplified, using a Laplace approiation, to yield n w g Nqp Ž o " n n w- * () Ž % n N n S n Nqp ¾ n H () This is the equation to run your favourite optiizer on, as perfored in GPR 4 MULTI-CLASS GPC We ve described binary classification, where the nuber of possible classes,, is just two In the case of ÄÃ classes, one approach is to fit an for each class In the first of the two steps of classification, our GP values are concatenated as!!2 \$! 2 \$! 2 Å! 2 Å \$ g (2) Let be a vector of the sae length as which, for each Æ!2, is for the class which is the label N and for the other Ç N entries Let grow to being block diagonal in the atrices! 2 N Å is a lengthening So the first change we see for ÈÃ of the GP Section 35 of Rasussen and Willias (26) offers hints on keeping the coputations anageable The second change is that the (erely one-diensional) cuulative Gaussian distribution is no longer sufficient to describe the squashing function in our classifier; instead we use the softa function For the Æ th data point, É ¹ n ¹ ² -É ¹ 4I576 ž ¹ É É^Ë ¹ É Ë (3) where ¹ is a nonconsecutive subset of, viz ¹ Ä ¹ ¹!!2 ¹ Å & We can suarize our results with Ì!!!2* \$ *!!"* \$!"* Å!2* \$ Å & Now that we ve presented the two big changes needed to go fro binary- to ulticlass GPC, we continue as before Setting to zero the derivative of the logarith of the coponents in (6), we replace (8) with N Ì H N p (4) The corresponding variance is ¾ p as before, but now ¾Š diag Ì ÎÍ Í, where Í is a v L atri obtained by stacking vertically the diagonal atrices diag Ì É, if Ì É is the subvector of Ì pertaining to class 4

11 \$ Å With these quantities estiated, we have enough to generalize (3) to ž É n ² LEÏ N É N É p É N diag _ N É g N É ¾ É p p N É g Ð (5) where É N, É, and ¾ É represent the class-relevant inforation only Finally, () is replaced with n w ² g N p g Ñ ¹ º Ž % hò Ñ É º 4I576 É ¹ Ó Ž o Ïin N n S n N p ¾ We won t present an eaple of ulti-class GPC, but hopefully you get the idea n Ð (6) 5 DISCUSSION As with GPR, classification can be etended to accept values with ultiple diensions, while keeping ost of the atheatics unchanged Other possible etensions include using the epectation propagation ethod in lieu of the Laplace approiation as entioned previously, putting confidence intervals on the classification probabilities, calculating the derivatives of (6) to aid the optiizer, or using the variational Gaussian process classifiers described in MacKay (998), to nae but four etensions Second, we repeat the Bayesian call fro the previous docuent to integrate over a range of possible covariance function paraeters This should be done regardless of how uch prior knowledge is available see for eaple Chapter 5 of Sivia and Skilling (26) on how to choose priors in the ost opaque situations Third, we ve again spared you a few practical algorithic details; coputer code is available at http//wwwgaussianprocessorg/gpl, with eaples ACKNOWLEDGMENTS Thanks are due to Prof Stephen Roberts and ebers of his Pattern Analysis Research Group, as well as the ALADDIN project (wwwaladdinprojectorg) REFERENCES MacKay, D (998) In CM Bishop (Ed), Neural networks and achine learning (NATO ASI Series, Series F, Coputer and Systes Sciences, Vol 68, pp 33-66) Dordrecht Kluwer Acadeic Press Rasussen, C and C Willias (26) Gaussian Processes for Machine Learning MIT Press Sivia, D and J Skilling (26) Data Analysis A Bayesian Tutorial (second ed) Oford Science Publications 5

### Example: Suppose that we deposit \$1000 in a bank account offering 3% interest, compounded monthly. How will our money grow?

Finance 111 Finance We have to work with oney every day. While balancing your checkbook or calculating your onthly expenditures on espresso requires only arithetic, when we start saving, planning for retireent,

### Machine Learning Applications in Grid Computing

Machine Learning Applications in Grid Coputing George Cybenko, Guofei Jiang and Daniel Bilar Thayer School of Engineering Dartouth College Hanover, NH 03755, USA gvc@dartouth.edu, guofei.jiang@dartouth.edu

### An Integrated Approach for Monitoring Service Level Parameters of Software-Defined Networking

International Journal of Future Generation Counication and Networking Vol. 8, No. 6 (15), pp. 197-4 http://d.doi.org/1.1457/ijfgcn.15.8.6.19 An Integrated Approach for Monitoring Service Level Paraeters

### INTEGRATED ENVIRONMENT FOR STORING AND HANDLING INFORMATION IN TASKS OF INDUCTIVE MODELLING FOR BUSINESS INTELLIGENCE SYSTEMS

Artificial Intelligence Methods and Techniques for Business and Engineering Applications 210 INTEGRATED ENVIRONMENT FOR STORING AND HANDLING INFORMATION IN TASKS OF INDUCTIVE MODELLING FOR BUSINESS INTELLIGENCE

### Graphing Exponential Functions

Graphing Eponential Functions Another coonly used type of non-linear function is the eponential function. We focus in this section on eponential graphs and solving equations. Another section is loaded

### AUC Optimization vs. Error Rate Minimization

AUC Optiization vs. Error Rate Miniization Corinna Cortes and Mehryar Mohri AT&T Labs Research 180 Park Avenue, Florha Park, NJ 0793, USA {corinna, ohri}@research.att.co Abstract The area under an ROC

### Use of extrapolation to forecast the working capital in the mechanical engineering companies

ECONTECHMOD. AN INTERNATIONAL QUARTERLY JOURNAL 2014. Vol. 1. No. 1. 23 28 Use of extrapolation to forecast the working capital in the echanical engineering copanies A. Cherep, Y. Shvets Departent of finance

### Analyzing Spatiotemporal Characteristics of Education Network Traffic with Flexible Multiscale Entropy

Vol. 9, No. 5 (2016), pp.303-312 http://dx.doi.org/10.14257/ijgdc.2016.9.5.26 Analyzing Spatioteporal Characteristics of Education Network Traffic with Flexible Multiscale Entropy Chen Yang, Renjie Zhou

### Performance Evaluation of Machine Learning Techniques using Software Cost Drivers

Perforance Evaluation of Machine Learning Techniques using Software Cost Drivers Manas Gaur Departent of Coputer Engineering, Delhi Technological University Delhi, India ABSTRACT There is a treendous rise

### Analysis of the purchase option of computers

Analysis of the of coputers N. Ahituv and I. Borovits Faculty of Manageent, The Leon Recanati Graduate School of Business Adinistration, Tel-Aviv University, University Capus, Raat-Aviv, Tel-Aviv, Israel

### 2012-13 and 2013-14 Transitional Comprehensive Curriculum. Algebra I Unit 7: Exponents, Exponential Functions, and Nonlinear Graphs

01-13 and 013-14 Transitional Coprehensive Curriculu Algebra I Unit 7: Eponents, Eponential Functions, and Nonlinear Graphs Tie Frae: Approiately five weeks Unit Description This unit is an introduction

### Fuzzy Sets in HR Management

Acta Polytechnica Hungarica Vol. 8, No. 3, 2011 Fuzzy Sets in HR Manageent Blanka Zeková AXIOM SW, s.r.o., 760 01 Zlín, Czech Republic blanka.zekova@sezna.cz Jana Talašová Faculty of Science, Palacký Univerzity,

### Online Bagging and Boosting

Abstract Bagging and boosting are two of the ost well-known enseble learning ethods due to their theoretical perforance guarantees and strong experiental results. However, these algoriths have been used

### An Expert Decision Support System for Crisis Management. Boumediene Belkhouche Reda Bakeer Salah Al-Saleh

An Expert Decision Support Syste for Crisis Manageent Bouediene Belkhouche Reda Bakeer Salah Al-Saleh EECS Departent Tulane University New Orleans, LA 70118 (504) 862-3372 (phone) (504) 862-3293 (fax)

### Shoe Size Data Investigation

Spring 013 Shoe Size Data Investigation Participants note shoe size rounded to the nearest whole nuber. For ales, add 1.5 and round to the nearest whole nuber, this will equalize the ale/feale size differences.

### 6. Time (or Space) Series Analysis

ATM 55 otes: Tie Series Analysis - Section 6a Page 8 6. Tie (or Space) Series Analysis In this chapter we will consider soe coon aspects of tie series analysis including autocorrelation, statistical prediction,

### Calculating the Return on Investment (ROI) for DMSMS Management. The Problem with Cost Avoidance

Calculating the Return on nvestent () for DMSMS Manageent Peter Sandborn CALCE, Departent of Mechanical Engineering (31) 45-3167 sandborn@calce.ud.edu www.ene.ud.edu/escml/obsolescence.ht October 28, 21

### International Journal of Management & Information Systems First Quarter 2012 Volume 16, Number 1

International Journal of Manageent & Inforation Systes First Quarter 2012 Volue 16, Nuber 1 Proposal And Effectiveness Of A Highly Copelling Direct Mail Method - Establishent And Deployent Of PMOS-DM Hisatoshi

### Image restoration for a rectangular poor-pixels detector

Iage restoration for a rectangular poor-pixels detector Pengcheng Wen 1, Xiangjun Wang 1, Hong Wei 2 1 State Key Laboratory of Precision Measuring Technology and Instruents, Tianjin University, China 2

### CRM FACTORS ASSESSMENT USING ANALYTIC HIERARCHY PROCESS

641 CRM FACTORS ASSESSMENT USING ANALYTIC HIERARCHY PROCESS Marketa Zajarosova 1* *Ph.D. VSB - Technical University of Ostrava, THE CZECH REPUBLIC arketa.zajarosova@vsb.cz Abstract Custoer relationship

### Lecture L9 - Linear Impulse and Momentum. Collisions

J. Peraire, S. Widnall 16.07 Dynaics Fall 009 Version.0 Lecture L9 - Linear Ipulse and Moentu. Collisions In this lecture, we will consider the equations that result fro integrating Newton s second law,

### Physics 211: Lab Oscillations. Simple Harmonic Motion.

Physics 11: Lab Oscillations. Siple Haronic Motion. Reading Assignent: Chapter 15 Introduction: As we learned in class, physical systes will undergo an oscillatory otion, when displaced fro a stable equilibriu.

### Quality evaluation of the model-based forecasts of implied volatility index

Quality evaluation of the odel-based forecasts of iplied volatility index Katarzyna Łęczycka 1 Abstract Influence of volatility on financial arket forecasts is very high. It appears as a specific factor

### Plane Trusses. Section 7: Flexibility Method - Trusses. A plane truss is defined as a twodimensional

lane Trusses A plane truss is defined as a twodiensional fraework of straight prisatic ebers connected at their ends by frictionless hinged joints, and subjected to loads and reactions that act only at

### Comment on On Discriminative vs. Generative Classifiers: A Comparison of Logistic Regression and Naive Bayes

Coent on On Discriinative vs. Generative Classifiers: A Coparison of Logistic Regression and Naive Bayes Jing-Hao Xue (jinghao@stats.gla.ac.uk) and D. Michael Titterington (ike@stats.gla.ac.uk) Departent

### Lecture L26-3D Rigid Body Dynamics: The Inertia Tensor

J. Peraire, S. Widnall 16.07 Dynaics Fall 008 Lecture L6-3D Rigid Body Dynaics: The Inertia Tensor Version.1 In this lecture, we will derive an expression for the angular oentu of a 3D rigid body. We shall

### Reliability Constrained Packet-sizing for Linear Multi-hop Wireless Networks

Reliability Constrained acket-sizing for inear Multi-hop Wireless Networks Ning Wen, and Randall A. Berry Departent of Electrical Engineering and Coputer Science Northwestern University, Evanston, Illinois

### Applying Multiple Neural Networks on Large Scale Data

0 International Conference on Inforation and Electronics Engineering IPCSIT vol6 (0) (0) IACSIT Press, Singapore Applying Multiple Neural Networks on Large Scale Data Kritsanatt Boonkiatpong and Sukree

### arxiv:0805.1434v1 [math.pr] 9 May 2008

Degree-distribution stability of scale-free networs Zhenting Hou, Xiangxing Kong, Dinghua Shi,2, and Guanrong Chen 3 School of Matheatics, Central South University, Changsha 40083, China 2 Departent of

### PERFORMANCE METRICS FOR THE IT SERVICES PORTFOLIO

Bulletin of the Transilvania University of Braşov Series I: Engineering Sciences Vol. 4 (53) No. - 0 PERFORMANCE METRICS FOR THE IT SERVICES PORTFOLIO V. CAZACU I. SZÉKELY F. SANDU 3 T. BĂLAN Abstract:

### Vectors & Newton's Laws I

Physics 6 Vectors & Newton's Laws I Introduction In this laboratory you will eplore a few aspects of Newton s Laws ug a force table in Part I and in Part II, force sensors and DataStudio. By establishing

### Exercise 4 INVESTIGATION OF THE ONE-DEGREE-OF-FREEDOM SYSTEM

Eercise 4 IVESTIGATIO OF THE OE-DEGREE-OF-FREEDOM SYSTEM 1. Ai of the eercise Identification of paraeters of the euation describing a one-degree-of- freedo (1 DOF) atheatical odel of the real vibrating

### Calculation Method for evaluating Solar Assisted Heat Pump Systems in SAP 2009. 15 July 2013

Calculation Method for evaluating Solar Assisted Heat Pup Systes in SAP 2009 15 July 2013 Page 1 of 17 1 Introduction This docuent describes how Solar Assisted Heat Pup Systes are recognised in the National

### Draft Evaluation Approach for Shade Impact Mitigation Devices

Draft Evaluation Approach for Shade Ipact Mitigation Devices Prepared For: California Energy Coission Prepared By: BEW Engineering, Inc. 2303 Caino Raon, Suite 220 San Raon, CA 94583 (925)867-3330 www.bewengineering.co

### Factored Models for Probabilistic Modal Logic

Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence (2008 Factored Models for Probabilistic Modal Logic Afsaneh Shirazi and Eyal Air Coputer Science Departent, University of Illinois

### Dynamic Placement for Clustered Web Applications

Dynaic laceent for Clustered Web Applications A. Karve, T. Kibrel, G. acifici, M. Spreitzer, M. Steinder, M. Sviridenko, and A. Tantawi IBM T.J. Watson Research Center {karve,kibrel,giovanni,spreitz,steinder,sviri,tantawi}@us.ib.co

### This paper studies a rental firm that offers reusable products to price- and quality-of-service sensitive

MANUFACTURING & SERVICE OPERATIONS MANAGEMENT Vol., No. 3, Suer 28, pp. 429 447 issn 523-464 eissn 526-5498 8 3 429 infors doi.287/so.7.8 28 INFORMS INFORMS holds copyright to this article and distributed

### Construction Economics & Finance. Module 3 Lecture-1

Depreciation:- Construction Econoics & Finance Module 3 Lecture- It represents the reduction in arket value of an asset due to age, wear and tear and obsolescence. The physical deterioration of the asset

### ON SELF-ROUTING IN CLOS CONNECTION NETWORKS. BARRY G. DOUGLASS Electrical Engineering Department Texas A&M University College Station, TX 77843-3128

ON SELF-ROUTING IN CLOS CONNECTION NETWORKS BARRY G. DOUGLASS Electrical Engineering Departent Texas A&M University College Station, TX 778-8 A. YAVUZ ORUÇ Electrical Engineering Departent and Institute

### AUTOMATIC SATELLITE IMAGE REGISTRATION BY COMBINATION OF STEREO MATCHING AND RANDOM SAMPLE CONSENSUS

AUTOATIC SATELLITE IAGE REGISTRATION BY COBINATION OF STEREO ATCHING AND RANDO SAPLE CONSENSUS Taejung Ki* Yong-Jo I** *Satellite Technology Research Center Korea Advanced Institute of Science and Technology

### Design of Model Reference Self Tuning Mechanism for PID like Fuzzy Controller

Research Article International Journal of Current Engineering and Technology EISSN 77 46, PISSN 347 56 4 INPRESSCO, All Rights Reserved Available at http://inpressco.co/category/ijcet Design of Model Reference

### ESTIMATING LIQUIDITY PREMIA IN THE SPANISH GOVERNMENT SECURITIES MARKET

ESTIMATING LIQUIDITY PREMIA IN THE SPANISH GOVERNMENT SECURITIES MARKET Francisco Alonso, Roberto Blanco, Ana del Río and Alicia Sanchis Banco de España Banco de España Servicio de Estudios Docuento de

### RECURSIVE DYNAMIC PROGRAMMING: HEURISTIC RULES, BOUNDING AND STATE SPACE REDUCTION. Henrik Kure

RECURSIVE DYNAMIC PROGRAMMING: HEURISTIC RULES, BOUNDING AND STATE SPACE REDUCTION Henrik Kure Dina, Danish Inforatics Network In the Agricultural Sciences Royal Veterinary and Agricultural University

### Investing in corporate bonds?

Investing in corporate bonds? This independent guide fro the Australian Securities and Investents Coission (ASIC) can help you look past the return and assess the risks of corporate bonds. If you re thinking

### The Virtual Spring Mass System

The Virtual Spring Mass Syste J. S. Freudenberg EECS 6 Ebedded Control Systes Huan Coputer Interaction A force feedbac syste, such as the haptic heel used in the EECS 6 lab, is capable of exhibiting a

### AN ALGORITHM FOR REDUCING THE DIMENSION AND SIZE OF A SAMPLE FOR DATA EXPLORATION PROCEDURES

Int. J. Appl. Math. Coput. Sci., 2014, Vol. 24, No. 1, 133 149 DOI: 10.2478/acs-2014-0011 AN ALGORITHM FOR REDUCING THE DIMENSION AND SIZE OF A SAMPLE FOR DATA EXPLORATION PROCEDURES PIOTR KULCZYCKI,,

### Leak detection in open water channels

Proceedings of the 17th World Congress The International Federation of Autoatic Control Seoul, Korea, July 6-11, 28 Leak detection in open water channels Erik Weyer Georges Bastin Departent of Electrical

### Standards and Protocols for the Collection and Dissemination of Graduating Student Initial Career Outcomes Information For Undergraduates

National Association of Colleges and Eployers Standards and Protocols for the Collection and Disseination of Graduating Student Initial Career Outcoes Inforation For Undergraduates Developed by the NACE

### A quantum secret ballot. Abstract

A quantu secret ballot Shahar Dolev and Itaar Pitowsky The Edelstein Center, Levi Building, The Hebrerw University, Givat Ra, Jerusale, Israel Boaz Tair arxiv:quant-ph/060087v 8 Mar 006 Departent of Philosophy

### Managing Complex Network Operation with Predictive Analytics

Managing Coplex Network Operation with Predictive Analytics Zhenyu Huang, Pak Chung Wong, Patrick Mackey, Yousu Chen, Jian Ma, Kevin Schneider, and Frank L. Greitzer Pacific Northwest National Laboratory

### Searching strategy for multi-target discovery in wireless networks

Searching strategy for ulti-target discovery in wireless networks Zhao Cheng, Wendi B. Heinzelan Departent of Electrical and Coputer Engineering University of Rochester Rochester, NY 467 (585) 75-{878,

### Preference-based Search and Multi-criteria Optimization

Fro: AAAI-02 Proceedings. Copyright 2002, AAAI (www.aaai.org). All rights reserved. Preference-based Search and Multi-criteria Optiization Ulrich Junker ILOG 1681, route des Dolines F-06560 Valbonne ujunker@ilog.fr

### Cooperative Caching for Adaptive Bit Rate Streaming in Content Delivery Networks

Cooperative Caching for Adaptive Bit Rate Streaing in Content Delivery Networs Phuong Luu Vo Departent of Coputer Science and Engineering, International University - VNUHCM, Vietna vtlphuong@hciu.edu.vn

### A framework for performance monitoring, load balancing, adaptive timeouts and quality of service in digital libraries

Int J Digit Libr (2000) 3: 9 35 INTERNATIONAL JOURNAL ON Digital Libraries Springer-Verlag 2000 A fraework for perforance onitoring, load balancing, adaptive tieouts and quality of service in digital libraries

### Modeling Cooperative Gene Regulation Using Fast Orthogonal Search

8 The Open Bioinforatics Journal, 28, 2, 8-89 Open Access odeling Cooperative Gene Regulation Using Fast Orthogonal Search Ian inz* and ichael J. Korenberg* Departent of Electrical and Coputer Engineering,

### A Hybrid Grey-Game-MCDM Method for ERP Selecting Based on BSC. M. H. Kamfiroozi, 2 A. BonyadiNaeini

Int. J. Manag. Bus. Res., 3 (1), 13-20, Winter 2013 IAU A Hybrid Grey-Gae-MCDM Method for ERP Selecting Based on BSC 1 M. H. Kafiroozi, 2 A. BonyadiNaeini 1,2 Departent of Industrial Engineering, Iran

### Adaptive Modulation and Coding for Unmanned Aerial Vehicle (UAV) Radio Channel

Recent Advances in Counications Adaptive odulation and Coding for Unanned Aerial Vehicle (UAV) Radio Channel Airhossein Fereidountabar,Gian Carlo Cardarilli, Rocco Fazzolari,Luca Di Nunzio Abstract In

### Investing in corporate bonds?

Investing in corporate bonds? This independent guide fro the Australian Securities and Investents Coission (ASIC) can help you look past the return and assess the risks of corporate bonds. If you re thinking

### Factor Model. Arbitrage Pricing Theory. Systematic Versus Non-Systematic Risk. Intuitive Argument

Ross [1],[]) presents the aritrage pricing theory. The idea is that the structure of asset returns leads naturally to a odel of risk preia, for otherwise there would exist an opportunity for aritrage profit.

### Guidelines for calculating sample size in 2x2 crossover trials : a simulation study

J.Natn.Sci.Foundation Sri Lanka 011 39 (1): 77-89 RESEARCH ARTICLE Guidelines for calculating saple size in x crossover trials : a siulation study N.M. Siyasinghe and M.R. Sooriyarachchi Departent of Statistics,

### 4.3 The Graph of a Rational Function

4.3 The Graph of a Rational Function Section 4.3 Notes Page EXAMPLE: Find the intercepts, asyptotes, and graph of + y =. 9 First we will find the -intercept by setting the top equal to zero: + = 0 so =

### Modeling operational risk data reported above a time-varying threshold

Modeling operational risk data reported above a tie-varying threshold Pavel V. Shevchenko CSIRO Matheatical and Inforation Sciences, Sydney, Locked bag 7, North Ryde, NSW, 670, Australia. e-ail: Pavel.Shevchenko@csiro.au

### Considerations on Distributed Load Balancing for Fully Heterogeneous Machines: Two Particular Cases

Considerations on Distributed Load Balancing for Fully Heterogeneous Machines: Two Particular Cases Nathanaël Cheriere Departent of Coputer Science ENS Rennes Rennes, France nathanael.cheriere@ens-rennes.fr

### The individual neurons are complicated. They have a myriad of parts, subsystems and control mechanisms. They convey information via a host of

CHAPTER 4 ARTIFICIAL NEURAL NETWORKS 4. INTRODUCTION Artificial Neural Networks (ANNs) are relatively crude electronic odels based on the neural structure of the brain. The brain learns fro experience.

### Semi-invariants IMOTC 2013 Simple semi-invariants

Sei-invariants IMOTC 2013 Siple sei-invariants These are soe notes (written by Tejaswi Navilarekallu) used at the International Matheatical Olypiad Training Cap (IMOTC) 2013 held in Mubai during April-May,

### Generating Certification Authority Authenticated Public Keys in Ad Hoc Networks

SECURITY AND COMMUNICATION NETWORKS Published online in Wiley InterScience (www.interscience.wiley.co). Generating Certification Authority Authenticated Public Keys in Ad Hoc Networks G. Kounga 1, C. J.

### A Comparative Study of Parametric and Nonparametric Regressions

Iranian Econoic Review, Vol.16, No.3, Fall 11 A Coparative Study of Paraetric and Nonparaetric Regressions Shahra Fattahi Received: 1/8/8 Accepted: 11/4/4 Abstract his paper evaluates inflation forecasts

### An Approach to Combating Free-riding in Peer-to-Peer Networks

An Approach to Cobating Free-riding in Peer-to-Peer Networks Victor Ponce, Jie Wu, and Xiuqi Li Departent of Coputer Science and Engineering Florida Atlantic University Boca Raton, FL 33431 April 7, 2008

### ( C) CLASS 10. TEMPERATURE AND ATOMS

CLASS 10. EMPERAURE AND AOMS 10.1. INRODUCION Boyle s understanding of the pressure-volue relationship for gases occurred in the late 1600 s. he relationships between volue and teperature, and between

### HW 2. Q v. kt Step 1: Calculate N using one of two equivalent methods. Problem 4.2. a. To Find:

HW 2 Proble 4.2 a. To Find: Nuber of vacancies per cubic eter at a given teperature. b. Given: T 850 degrees C 1123 K Q v 1.08 ev/ato Density of Fe ( ρ ) 7.65 g/cc Fe toic weight of iron ( c. ssuptions:

### IV Approximation of Rational Functions 1. IV.C Bounding (Rational) Functions on Intervals... 4

Contents IV Approxiation of Rational Functions 1 IV.A Constant Approxiation..................................... 1 IV.B Linear Approxiation....................................... 3 IV.C Bounding (Rational)

### Software Quality Characteristics Tested For Mobile Application Development

Thesis no: MGSE-2015-02 Software Quality Characteristics Tested For Mobile Application Developent Literature Review and Epirical Survey WALEED ANWAR Faculty of Coputing Blekinge Institute of Technology

### Storing and Accessing Live Mashup Content in the Cloud

Storing and Accessing Live Mashup Content in the Cloud Krzysztof Ostrowski Cornell University Ithaca, NY 14853, USA krzys@cs.cornell.edu Ken Biran Cornell University Ithaca, NY 14853, USA ken@cs.cornell.edu

### Binary Embedding: Fundamental Limits and Fast Algorithm

Binary Ebedding: Fundaental Liits and Fast Algorith Xinyang Yi The University of Texas at Austin yixy@utexas.edu Eric Price The University of Texas at Austin ecprice@cs.utexas.edu Constantine Caraanis

### Data Set Generation for Rectangular Placement Problems

Data Set Generation for Rectangular Placeent Probles Christine L. Valenzuela (Muford) Pearl Y. Wang School of Coputer Science & Inforatics Departent of Coputer Science MS 4A5 Cardiff University George

### HIGH FREQUENCY FILTER DESIGN

HIGH FREQUENCY FILTER DESIGN for High-Frequency Circuit Design Elective by Michael Tse Septeber 2003 CONTENTS. Introduction. Types of filters.2 Monolithic filters.3 Integrators.4 Siple first-order g-c

### Evaluating Inventory Management Performance: a Preliminary Desk-Simulation Study Based on IOC Model

Evaluating Inventory Manageent Perforance: a Preliinary Desk-Siulation Study Based on IOC Model Flora Bernardel, Roberto Panizzolo, and Davide Martinazzo Abstract The focus of this study is on preliinary

### Modeling Parallel Applications Performance on Heterogeneous Systems

Modeling Parallel Applications Perforance on Heterogeneous Systes Jaeela Al-Jaroodi, Nader Mohaed, Hong Jiang and David Swanson Departent of Coputer Science and Engineering University of Nebraska Lincoln

### Exact Matrix Completion via Convex Optimization

Exact Matrix Copletion via Convex Optiization Eanuel J. Candès and Benjain Recht Applied and Coputational Matheatics, Caltech, Pasadena, CA 91125 Center for the Matheatics of Inforation, Caltech, Pasadena,

### ASIC Design Project Management Supported by Multi Agent Simulation

ASIC Design Project Manageent Supported by Multi Agent Siulation Jana Blaschke, Christian Sebeke, Wolfgang Rosenstiel Abstract The coplexity of Application Specific Integrated Circuits (ASICs) is continuously

### Metric of Carbon Equity: Carbon Gini Index Based on Historical Cumulative Emission per Capita

ADVANCES IN CLIMATE CHANGE RESEARCH 2(3): 134 140, 2011 www.cliatechange.cn DOI: 10.3724/SP.J.1248.2011.00134 POLICY FORUM Metric of Carbon Equity: Carbon Gini Index Based on Historical Cuulative Eission

An Innovate Dynaic Load Balancing Algorith Based on Task Classification Hong-bin Wang,,a, Zhi-yi Fang, b, Guan-nan Qu,*,c, Xiao-dan Ren,d College of Coputer Science and Technology, Jilin University, Changchun

### Non-Price Equilibria in Markets of Discrete Goods

Non-Price Equilibria in Markets of Discrete Goods (working paper) Avinatan Hassidi Hai Kaplan Yishay Mansour Noa Nisan ABSTRACT We study arkets of indivisible ites in which price-based (Walrasian) equilibria

### SUPPORTING YOUR HIPAA COMPLIANCE EFFORTS

WHITE PAPER SUPPORTING YOUR HIPAA COMPLIANCE EFFORTS Quanti Solutions. Advancing HIM through Innovation HEALTHCARE SUPPORTING YOUR HIPAA COMPLIANCE EFFORTS Quanti Solutions. Advancing HIM through Innovation

### Modified Latin Hypercube Sampling Monte Carlo (MLHSMC) Estimation for Average Quality Index

Analog Integrated Circuits and Signal Processing, vol. 9, no., April 999. Abstract Modified Latin Hypercube Sapling Monte Carlo (MLHSMC) Estiation for Average Quality Index Mansour Keraat and Richard Kielbasa

### The Research of Measuring Approach and Energy Efficiency for Hadoop Periodic Jobs

Send Orders for Reprints to reprints@benthascience.ae 206 The Open Fuels & Energy Science Journal, 2015, 8, 206-210 Open Access The Research of Measuring Approach and Energy Efficiency for Hadoop Periodic

### The Velocities of Gas Molecules

he Velocities of Gas Molecules by Flick Colean Departent of Cheistry Wellesley College Wellesley MA 8 Copyright Flick Colean 996 All rights reserved You are welcoe to use this docuent in your own classes

### Real Time Target Tracking with Binary Sensor Networks and Parallel Computing

Real Tie Target Tracking with Binary Sensor Networks and Parallel Coputing Hong Lin, John Rushing, Sara J. Graves, Steve Tanner, and Evans Criswell Abstract A parallel real tie data fusion and target tracking

### parallelmcmccombine: An R Package for Bayesian Methods for Big Data and Analytics

parallelcccobine: An R Package for Bayesian ethods for Big Data and Analytics Alexey iroshnikov 1, Erin. Conlon 1* 1 Departent of atheatics and Statistics, University of assachusetts, Aherst, assachusetts,

ADJUSTING FOR QUALITY CHANGE 7 Introduction 7.1 The easureent of changes in the level of consuer prices is coplicated by the appearance and disappearance of new and old goods and services, as well as changes

### Extended-Horizon Analysis of Pressure Sensitivities for Leak Detection in Water Distribution Networks: Application to the Barcelona Network

2013 European Control Conference (ECC) July 17-19, 2013, Zürich, Switzerland. Extended-Horizon Analysis of Pressure Sensitivities for Leak Detection in Water Distribution Networks: Application to the Barcelona

### Simple Harmonic Motion

SHM-1 Siple Haronic Motion A pendulu, a ass on a spring, and any other kinds of oscillators ehibit a special kind of oscillatory otion called Siple Haronic Motion (SHM). SHM occurs whenever : i. there

### Data Streaming Algorithms for Estimating Entropy of Network Traffic

Data Streaing Algoriths for Estiating Entropy of Network Traffic Ashwin Lall University of Rochester Vyas Sekar Carnegie Mellon University Mitsunori Ogihara University of Rochester Jun (Ji) Xu Georgia

### Markovian inventory policy with application to the paper industry

Coputers and Cheical Engineering 26 (2002) 1399 1413 www.elsevier.co/locate/copcheeng Markovian inventory policy with application to the paper industry K. Karen Yin a, *, Hu Liu a,1, Neil E. Johnson b,2

### Implementation of Active Queue Management in a Combined Input and Output Queued Switch

pleentation of Active Queue Manageent in a obined nput and Output Queued Switch Bartek Wydrowski and Moshe Zukeran AR Special Research entre for Ultra-Broadband nforation Networks, EEE Departent, The University

### ABSTRACT KEYWORDS. Comonotonicity, dependence, correlation, concordance, copula, multivariate. 1. INTRODUCTION

MEASURING COMONOTONICITY IN M-DIMENSIONAL VECTORS BY INGE KOCH AND ANN DE SCHEPPER ABSTRACT In this contribution, a new easure of coonotonicity for -diensional vectors is introduced, with values between

### REQUIREMENTS FOR A COMPUTER SCIENCE CURRICULUM EMPHASIZING INFORMATION TECHNOLOGY SUBJECT AREA: CURRICULUM ISSUES

REQUIREMENTS FOR A COMPUTER SCIENCE CURRICULUM EMPHASIZING INFORMATION TECHNOLOGY SUBJECT AREA: CURRICULUM ISSUES Charles Reynolds Christopher Fox reynolds @cs.ju.edu fox@cs.ju.edu Departent of Coputer

### Pure Bending Determination of Stress-Strain Curves for an Aluminum Alloy

Proceedings of the World Congress on Engineering 0 Vol III WCE 0, July 6-8, 0, London, U.K. Pure Bending Deterination of Stress-Strain Curves for an Aluinu Alloy D. Torres-Franco, G. Urriolagoitia-Sosa,