Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations

Size: px
Start display at page:

Download "Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations"

Transcription

1 Real-Time Computig Without Stable States: A New Framework for Neural Computatio Based o Perturbatios Wolfgag aass+, Thomas Natschläger+ & Hery arkram* + Istitute for Theoretical Computer Sciece, Techische Uiversität Graz; A-8010 Graz, Austria * Brai id Istitute, Ecole Polytechique Federale de Lausae, CH-1015 Lausae, Switzerlad Wolfgag aass & Thomas Natschlaeger Istitute for Theoretical Computer Sciece Techische Uiversitaet Graz Iffeldgasse 16b, A-8010 Graz, Austria Tel: Fax: Hery arkram Brai id Istitute Ecole Polytechique Federale de Lausae CE-Ecubles, CH-1015 Lausae, Switzerlad Tel: Fax: Address for Correspodece: Wolfgag aass 1

2 A key challege for eural modelig is to explai how a cotiuous stream of multi-modal iput from a rapidly chagig eviromet ca be processed by stereotypical recurret circuits of itegrate-ad-fire euros i real-time. We propose a ew computatioal model for real-time computig o time-varyig iput that provides a alterative to paradigms based o Turig machies or attractor eural etworks. It does ot require a taskdepedet costructio of eural circuits. Istead it is based o priciples of high dimesioal dyamical systems i combiatio with statistical learig theory, ad ca be implemeted o geeric evolved or foud recurret circuitry. It is show that the iheret trasiet dyamics of the high-dimesioal dyamical system formed by a sufficietly large ad heterogeeous eural circuit may serve as uiversal aalog fadig memory. Readout euros ca lear to extract i real-time from the curret state of such recurret eural circuit iformatio about curret ad past iputs that may be eeded for diverse tasks. Stable iteral states are ot required for givig a stable output, sice trasiet iteral states ca be trasformed by readout euros ito stable target outputs due to the high dimesioality of the dyamical system. Our approach is based o a rigorous computatioal model, the liquid state machie, that ulike Turig machies, does ot require sequetial trasitios betwee well-defied discrete iteral states. It is supported, like the Turig machie, by rigorous mathematical results that predict uiversal computatioal power uder idealized coditios, but for the biologically more realistic sceario of real-time processig of time-varyig iputs. Our approach provides ew perspectives for the iterpretatio of eural codig, for the desig of experimets ad dataaalysis i europhysiology, ad for the solutio of problems i robotics ad eurotechology. 2

3 Itroductio Itricate topographically orgaized feed-forward pathways project rapidly chagig spatio-temporal iformatio about the eviromet ito the eocortex. This iformatio is processed by extremely complex but surprisigly stereotypic microcircuits that ca perform a wide spectrum of tasks (Shepherd, 1988, Douglas et al., 1998, vo elcher et al., 2000). The microcircuit features that eable this seemigly uiversal computatioal power, is a mystery. Oe particular feature, the multiple recurret loops that form a immesely complicated etwork usig as may as 80% of all the syapses withi a fuctioal eocortical colum, has preseted a itractable problem both for computatioal models ispired by curret artificial computig machiery (Savage, 1998), ad for attractor eural etwork models. The difficulty to uderstad computatios withi recurret etworks of itegrate-ad-fire euros comes from the fact that their dyamics takes o a life of its ow whe challeged with rapidly chagig iputs. This is particularly true for the very high dimesioal dyamical system formed by a eural microcircuit, whose compoets are highly heterogeeous ad where each euro ad each syapse adds degrees of freedom to the dyamics of the system. The most commo approach for modelig computig i recurret eural circuits has bee to try to take cotrol of their high dimesioal dyamics. ethods for cotrollig the dyamics of recurret eural etworks through adaptive mechaisms are reviewed i (Pearlmutter, 1995). So far oe has ot bee able to apply these to the case of etworks of spikig euros. Other approaches towards modelig computatio i biological eural systems are based o costructios of artificial eural etworks that simulate Turig machies or other models for digital computatio, see for example (Pollack, 1991), (Giles et al., 1992), (Siegelma et al., 1994), (Hyoetyiemi, 1996), (oore, 1998). Amog these there are models, such as dyamical recogizers, which are capable of real-time computig o olie iput (i discrete time). Noe of these approaches has bee demostrated to work for etworks of spikig euros, or ay more realistic models for eural microcircuits. It was show i (aass, 1996) that oe also ca costruct recurret circuits of spikig euros that ca simulate arbitrary Turig machies. But all of these approaches require sychroizatio of all euros by a cetral clock, a feature that appears to missig i eural microcircuits. I additio they require the costructio of particular recurret circuits, ad caot be implemeted by evolvig or adaptig a give circuit. Furthermore the results of (aass et al., 1999) o the impact of oise o the computatioal power of recurret eural etworks suggest that all these approaches break dow as soo as oe assumes that the uderlyig aalog computatioal uits are subject to Gaussia or other realistic oise distributios. Attractor eural etworks o the other had allow oise robust computatio, but their attractor ladscape is i geeral hard to cotrol, ad they eed to have a very large set of attractors i order to store saliet iformatio o past iputs (for example 1024 attractors i order to store 10 bits). I additio they are less suitable for real-time computig o rapidly varyig iput streams because of the time required for covergece to a attractor. Fially, oe of these approaches allows that several real-time computatios are carried out i parallel withi the same circuitry, which appears to be a geeric feature of eural microcircuits. I this article we aalyze the dyamics of eural microcircuits from the poit of view of a readout euro, whose task is to extract iformatio ad report results from a eural microcircuit to other circuits. A huma observer of the dyamics i a eural microcircuit would be lookig for clearly distict ad temporally stable features, such as covergece to attractors. We show that a readout euro, that receives iputs from hudreds or thousads of euros i a eural 3

4 microcircuit, ca lear to extract saliet iformatio from the high dimesioal trasiet states of the circuit, ad ca trasform trasiet circuit states ito stable readouts. I particular each readout ca lear to defie its ow otio of equivalece of dyamical states withi the eural microcircuit, ad ca the perform its task o ovel iputs. This uexpected fidig of readoutassiged equivalet states of a dyamical system explais how ivariat readout is possible despite the fact that the eural microcircuit may ever re-visit the same state. Furthermore we show that multiple readout modules ca be traied to perform differet tasks o the same state trajectories of a recurret eural circuit, thereby eablig parallel real-time computig. We preset the mathematical framework for a computatioal model that does ot require covergece to stable iteral states or attractors (eve if they do occur), sice iformatio about past iputs is automatically captured i the perturbatios of a dyamical system, i.e. i the cotiuous trajectory of trasiet iteral states. Special cases of this mechaism were already reported i (Buoomao et al., 1995) ad (Domiey et al., 1995). Similar ideas have bee discovered idepedetly by Herbert Jaeger (Jaeger, 2001) i the cotext of artificial eural etworks. Computig without Attractors As a illustratio for our geeral approach towards real-time computig cosider a series of trasiet perturbatios caused i a excitable medium (see (Holde et al., 1991)), for example a liquid, by a sequece of exteral disturbaces ("iputs") such as wid, soud, or sequeces of pebbles dropped ito the liquid. Viewed as a attractor eural etwork, the liquid has oly oe attractor state the restig state ad may therefore seem useless for computatioal purposes. However, the perturbed state of the liquid, at ay momet i time, represets preset as well as past iputs, potetially providig the iformatio eeded for a aalysis of various dyamic aspects of the eviromet. I order for such a liquid to serve as a source of saliet iformatio about preset ad past stimuli without relyig o stable states, the perturbatios must be sesitive to salietly differet iputs but o-chaotic. The maer i which perturbatios are formed ad maitaied would vary for differet types of liquids ad would determie how useful the perturbatios are for such retrograde aalysis. Limitatios o the computatioal capabilities of liquids are imposed by their time-costat for relaxatio, ad the strictly local iteractios ad homogeeity of the elemets of a liquid. Neural microcircuits, however, appear to be ideal liquids for computig o perturbatios because of the large diversity of their elemets, euros ad syapses (see (Gupta et al., 2000)), ad the large variety of mechaisms ad time costats characterizig their iteractios, ivolvig recurret coectios o multiple spatial scales ("loops withi loops"). The foudatio for our aalysis of computatios without stable states is a rigorous computatioal model: the liquid state machie. Two macroscopic properties emerge from our theoretical aalysis ad computer simulatios as ecessary ad sufficiet coditios for powerful real-time computig o perturbatios: a separatio property, SP, ad a approximatio property, AP. SP addresses the amout of separatio betwee the trajectories of iteral states of the system that are caused by two differet iput streams (i the case of a physical liquid, SP could reflect the differece betwee the wave patters resultig from differet sequeces of disturbaces). 4

5 AP addresses the resolutio ad recodig capabilities of the readout mechaisms - more precisely its capability to distiguish ad trasform differet iteral states of the liquid ito give target outputs (whereas SP depeds mostly o the complexity of the liquid, AP depeds mostly o the adaptability of the readout mechaism to the required task). Liquid State achies Like the Turig machie (Savage, 1998), the model of a liquid state machie (LS) is based o a rigorous mathematical framework that guaratees, uder idealized coditios, uiversal computatioal power. Turig machies, however, have uiversal computatioal power for off-lie computatio o (static) discrete iputs, while LSs have i a very specific sese uiversal computatioal power for real-time computig with fadig memory o aalog fuctios i cotiuous time. The iput fuctio u ( ) ca be a cotiuous sequece of disturbaces, ad the target output ca be some chose fuctio y ( ) of time that provides a real-time aalysis of this sequece. I order for a machie to map iput fuctios of time u ( ) to output fuctios y( ) of time, we assume that it geerates, at every time t, a iteral liquid state x (t), which costitutes its curret respose to precedig perturbatios, i.e., to precedig iputs u (s) for s t (Figure 1). I cotrast to the fiite state of a fiite state machie (or fiite automato) this liquid state cosists of aalog values that may chage cotiuously over time. Whereas the state set ad the state trasitio fuctio of a fiite state machie is i geeral costructed for a Figure 1: A: Architecture of a LS. A fuctio of time (time series) u( ) is ijected as iput ito the L liquid filter, creatig at time t the liquid state x (t), which is trasformed by a memory-less readout map f to geerate a output y(t). specific task, the liquid states ad the trasitios betwee them eed ot be customized for a specific task. I a physical implemetatio this liquid state cosists of all iformatio about the curret iteral state of a dyamical system that is accessible to the readout modules. I 5

6 mathematical terms, this liquid state is simply the curret output of some operator or filter 1 that maps iput fuctios u ( ) oto fuctios x (t) : x ( t) = ( L u)( t). L I the followig we will refer to this filter L ofte as liquid filter, or liquid circuit if implemeted by a circuit. If it is implemeted by a eural circuit, we refer to the euros i that circuit as liquid euros. The secod compoet of a LS is a memory-less readout map f that trasforms, at every time t, the curret liquid state x (t) ito the output y( t) = f ( x ( t)). I cotrast to the liquid filter L, this readout map f is i geeral chose i a task-specific maer (ad there may be may differet readout maps, that extract differet task-specific iformatio i parallel from the curret output of L ). Note that i a fiite state machie there exists o aalogo to such task-specific readout maps, sice there the iteral fiite states are already costructed i a task-specific maer. Accordig to the precedig defiitio readout maps are i geeral memory-less 2. Hece all iformatio about iputs u (s) from precedig time poits s t that is eeded to produce a target output y(t) at time t has to be cotaied i the curret liquid state x (t). odels for computatio that have origiated i computer sciece store such iformatio about the past i stable states (for example i memory buffers or tapped delay lies). We argue, however, that this is ot ecessary sice large computatioal power o fuctios of time ca also be realized eve if all memory traces are cotiuously decayig. Istead of worryig about the code ad locatio where iformatio about past iputs is stored, ad how this iformatio decays, it is eough to address the separatio questio: For which later time poits t will ay two sigificatly differet iput fuctios of time u( ) ad v( ) cause sigificatly differet liquid states x u (t) ad (t). Good separatio capability, i combiatio x v 1 Fuctios F that map iput fuctios of time u ( ) o output fuctios y ( ) of time are usually called operators i mathematics, but are commoly referred to as filters i egieerig ad eurosciece. We use the term filter i the followig, ad we write ( Fu )( t) for the output of the filter F at time t whe F is applied to the iput fuctio u ( ). Formally, such filter F is a map from U ito ( R R ) k, where R R is the set of all real-valued fuctios of time, vectors cosistig of k such fuctios of time, U is some subset of vectors cosistig of fuctios of time i U. 2 The term "memory-less" refers to the fact that the readout map R R, ad ( R k R ) is the set of U is the set of f is ot required to retai ay memory of previous states x ( s), s < t, of the liquid. However, i a biological cotext, the readout map will i geeral be subject to plasticity, ad may also cotribute to the memory capability of the system. We do ot explore this issues i this article because the differetiatio ito a memory-less readout map ad a liquid that serves as a memory device is made for coceptual clarificatio, ad is ot essetial to the model. 6

7 with a adequate readout map f, allows us to discard the requiremet of storig bits "util further otice" i stable states of the computatioal system. Uiversal Computatioal Power of LSs for Time Varyig Iputs We say that a class of machies has uiversal power for computatios with fadig memory o fuctios of time if ay filter F, i.e., ay map from fuctios of time u ( ) to fuctios of time y ( ), that is time ivariat 3 ad has fadig memory 4, ca be approximated by machies from this class, to ay degree of precisio. Arguably, these filters F are approximated accordig to this defiitio iclude all maps from iput fuctios of time to output fuctios of time that a behavig orgaism might eed to compute. A mathematical theorem (see Appedix A) guaratees that LSs have this uiversal computatioal power regardless of specific structure or implemetatio, provided that two abstract properties are met: the class of basis filters from which the liquid filters L are composed satisfies the poit-wise separatio property ad the class of fuctios from which the readout maps f are draw, satisfies the approximatio property. These two properties provide the mathematical basis for the separatio property SP ad the approximatio property AP that were previously discussed. Theorem 1 i Appedix A implies that there are o serious a priori limits for the computatioal power of LSs o cotiuous fuctios of time, ad thereby provides a theoretical foudatio for our approach towards modelig eural computatio. I particular, sice this theorem makes o specific requiremet regardig the exact ature or behaviour of the basis filters, as log as they satisfy the separatio property (for the iputs i questio), it provides theoretical support for employig istead of circuits that were costructed 3 A filter F is called time ivariat if ay temporal shift of the iput fuctio u ( ) by some amout t 0 causes a temporal shift of the output fuctio y = Fu by the same amout t 0, i.e., ( Fu t 0 )( t) = ( Fu)( t + t 0) for all t,t 0 R, where u t 0 ( t) : = u( t + t 0). Note that if U is closed uder R k temporal shifts, the a time ivariat filter F : U ( R ) ca be idetified uiquely by the values y ( 0) = ( Fu)(0) of its output fuctios y ( ) at time 0. 4 Fadig memory (Boyd at al., 1985) is a cotiuity property of filters F which demads that for ay iput fuctio u ) U ay other iput fuctios ( the output )(0) v ) U [ T,0]. Formally oe defies that (Fu ca be approximated by the outputs (Fv)(0) for ( that approximate ( ) F : U ) u o a sufficietly log time iterval R k ( R has fadig memory if for every u U ad every ε > 0 there exist δ > 0 ad T > 0 so that ( Fv )(0) ( Fu)(0) < ε for all v U with u ( t) v( t) <δ for all t [ T,0]. Iformally a filter F has fadig memory if the most sigificat bits of its curret output value (Fu)(0) deped just o the most sigificat bits of the values of its iput fuctio ( ) T,0 ito the past. Thus, i order to compute the most sigificat bits of (Fu)(0) it is ot ecessary to kow the precise value of the iput fuctio u (s) for ay time s, ad it is also ot ecessary to kow aythig about the values of u ( ) for more tha a fiite time iterval back ito the past. Note that a filter that has fadig memory is automatically causal. u from some fiite time widow [ ] 7

8 for a specific task, partially evolved or eve rather arbitrary foud computatioal modules for purposeful computatios. This feature highlights a importat differece to computatioal theories based o Turig machies or fiite state machies, which are ofte used as coceptual basis for modelig eural computatio. The mathematical theory of LSs ca also be exteded to cover computatio o spike u of the iput u( ) trais (discrete evets i cotiuous time) as iputs. Here the i th compoet ( ) is a fuctio that assumes oly the values 0 ad 1, with ( t) = 1 at time t. Thus ( ) u i if the i th precedig euro fires u is ot a cotiuous fuctio but a sequece of poit evets. Theorem 2 i i Appedix A provides a theoretical foudatio for approximatig ay biologically relevat computatio o spike trais by LSs. i Neural icrocircuits as Implemetatios of LSs I order to test the applicability of this coceptual framework to modelig computatio i eural microcircuits, we carried out computer simulatios where a geeric recurret circuit of itegrate-ad-fire euros (see Appedix B for details) was employed as liquid filter. I other words: computer models for eural microcircuits were viewed as implemetatio of the liquid filter L of a LS. I order to test the theoretically predicted uiversal real-time computig capabilities of these eural implemetatios of LSs, we evaluated their performace o a wide variety of challegig bechmark tasks. The iput to the eural circuit was via oe or several iput spike trais, which diverged to iject curret ito 30% radomly chose liquid euros. The amplitudes of the iput syapses were chose from a Gaussia distributio, so that each euro i the liquid circuit received a slightly differet iput (a form of topographic ijectio). The liquid state of the eural microcircuit at time t was defied as all iformatio that a readout euro could extract at time t from the circuit, i.e. the output at time t of all the liquid euros represeted the curret liquid state of this istatiatio of a LS. ore precisely, sice the readout euros were modeled as I&F euros with a biologically realistic membrae time costat of 30 ms, the liquid state x (t) at time t cosisted of the vector of cotributios of all the liquid euros to the membrae potetial at time t of a geeric readout euro (with uit syaptic weights). athematically this liquid state x (t) ca be defied as the vector of output values at time t of liear filters with expoetial decay (time costat 30 ms) applied to the spike trais emitted by the liquid euros. Each readout map f was implemeted by a separate populatio P of itegrate-ad-fire euros (referred to as "readout euros") that received iput from all the "liquid euros", but had o lateral or recurret coectios 5. The curret firig activity p(t) of the populatio P, that is the fractio of euros i P firig durig a time bi of 20ms, was iterpreted as the aalog output of f at time t (oe ofte refers to such represetatio of aalog values by the curret firig activity i a pool of euros as space rate codig). Theoretically the class of readout maps that ca be implemeted i this fashio satisfies the approximatio property AP (aass, 2000, Auer et al., 2001), ad is accordig to Theorem 1 i priciple sufficiet for approximatig arbitrary give fadig memory filters F. I cases where a readout with discrete values 1 ad 0 5 For coceptual purposes we separate the liquid ad readout elemets i this paper, although dual liquid-readout fuctios ca also be implemeted. 8

9 suffices, oe ca implemet a readout map eve by a sigle I&F euro that represets these discrete output values by firig/o-firig at time t. I cases where the target output cosists of slowly varyig aalog values, a sigle readout euro ca be traied to represet these values through its time-varyig firig rate. I ay case the readout euros ca be traied to perform a specific task by adjustig the stregths of syapses projected oto them from the liquid euros usig a perceptro-like local learig rule (Auer et al., 2001). The fial leared state of the readout euros eables them to take particular weighted sums of the curret outputs x (t) of the liquid euros ad geerate a respose f ( x (t) ) that approximates the target value y(t). As a first test of these eural implemetatios of LSs we evaluated the separatio property SP of computer models for eural microcircuits o spike trai iputs. A large set of pairs of Poisso spike trais u( ) ad v( ) were radomly geerated ad ijected (i separate trials) as iput to the recurret eural circuit. The resultig trajectories x ( ) ad Figure 2: Average distace of liquid states for two differet iput spike trais u ad v (give as iput to the eural circuit i separate trials, each time with a idepedetly chose radom iitial state of the eural circuit, see Appedix B) plotted as a fuctio of time t. The state distace icreases with the distace d(u,v) betwee the two iput spike trais u ad v. Plotted o the y-axis is the average value of x ( t) x ( t) u v, where. deotes the Euclidea orm, ad x u (t), (t) x v deote the liquid states at time t for iput spike trais u ad v. The plotted results for the values 0.1, 0.2, 0.4 of the iput differece d' represet the average over 200 radomly geerated pairs u ad v of spike trais such that d ' d( u, v) < Parameters of the liquid: 1 colum, degree of coectivity λ = 2 (see Appedix B for details). x v ( ) of liquid states of the recurret circuit were recorded for each of these time-varyig iputs u( ) ad v( ). The average distace x ( t) x ( t) betwee these liquid states was plotted i Figure 2 as a fuctio of the time t after the oset of the iput, for various fixed values of the distace d(u,v) 6 betwee the two spike trai iputs u ad v. These curves show that the distace betwee these liquids states is well above the oise level, i.e. above the average liquid state differeces caused by the same spike trai applied with two differet radomly chose iitial coditios of the circuit (idicated by the solid curve). Furthermore these curves show that the differece i liquid states is after the first 30 ms roughly proportioal to the distace betwee the correspodig iput 6 I order to defie the distace d ( u, v) betwee two spike trais u ad v we replaced each 2 spike by a Gaussia exp( ( t / τ) ) for τ = 5ms (to be precise, u ad v are covolved with the 2 Gaussia kerel exp( ( t / τ) ) ) ad defied d ( u, v) as the distace of the resultig two cotiuous fuctios i the L2-orm (divided by the maximal legths 0.5 s of the spike trais u ad v). u u v 9

10 spike trais. Note i particular the absece of chaotic effects for these geeric eural microcircuit models with biologically realistic itermediate coectio legths. Explorig the Computatioal Power of odels for Neural icrocircuit As a first test of its computatioal power this simple geeric circuit was applied to a previously cosidered classificatio task (Hopfield & Brody, 2001), where spoke words were represeted by oise-corrupted spatio-temporal spike patters over a rather log time iterval (40-chael spike patters over 0.5s). This classificatio task had bee solved i (Hopfield & Brody, 2001) by a etwork of euros desiged for this task (relyig o ukow mechaisms that could provide smooth decays of firig activity over loger time periods, ad apparetly requirig substatially larger etworks of I&F euros if fully implemeted with I&F euros). The architecture of that etwork, which had bee customized for this task, limited its classificatio power to spike trais cosistig of a sigle spike per chael. We foud that the same, but also a more geeral versio of this spatio-temporal patter recogitio task that allowed several spikes per iput chael, ca be solved by a geeric recurret circuit as described i the previous sectio. Furthermore the output of this etwork was available at ay time, ad was usually correct as soo as the liquid state of the eural circuit had absorbed eough iformatio about the iput (the iitial value of the correctess just reflects the iitial guess of the readout). Formally we defied the correctess of the eural readout at time s by the term 1 target output y(s) readout activity p(s), where the target output y(s) cosisted i this case of the costat values 1 or 0, depedig o the iput patter. Plotted i Fig. 3 is for ay time t durig the presetatio of the iput patters i additio to the correctess as a fuctio of t also the certaity of the output at time t, which is defied as the average correctess up to that time t. Whereas the etwork costructed by Hopfield ad Brody was costructed to be ivariat with regard to liear time warpig of iputs (provided that oly oe spike arrives i each chael), the readouts of the geeric recurret circuit that we cosidered could be traied to be ivariat with regard to a large class of differet types of oises. The results show i Fig. 3 are for a oise where each iput spike is moved idepedetly by a amout draw from a Gaussia distributio with mea 0 ad SD 32 ms. 10

11 Figure 3: Applicatio of a geeric recurret etwork of I&F euros modeled as LS to a more difficult versio of a well-studied classificatio task (Hopfield & Brody, 2001). Five radomly draw patters (called zero, "oe ", "two",..), each cosistig of 40 parallel Poisso spike trais over 0.5s, were chose. Five readout modules, each cosistig of 50 itegrate-ad-fire euros, were traied with 20 oisy versios of each iput patter to respod selectively to oisy versios of just oe of these patters (oise was ijected by radomly movig each spike by a amout draw idepedetly from a Gaussia distributio with mea 0 ad variace 32ms; i additio the iitial state of the liquid euros was chose radomly at the begiig of each trial). The resposes of the readout which had bee traied * to detect the patter "zero" is show for a ew, previously ot show, oisy versios of two of the iput patters. The correctess ad certaity (= average correctess so far) are show as fuctios of time from the oset of the stimulus at the bottom. The correctess is calculated as 1 p(t)- y(t) where p(t) is the ormalized firig activity i the readout pool (ormalized to the rage [0 1]; 1 correspodig to a activity of 180Hz; biwidth 20ms) ad y(t) is the target output. (Correctess starts at a level of 0 for patter zero where this readout pool is supposed to become active, ad at a value of 1 for patter oe, because the readout pool starts i a iactive state). I cotrast to most circuits of spikig euros that have bee costructed for specific computatioal task, the spike trais of liquid ad readout euros show i this figure look rather realistic. * The familiar delta-rule was applied or ot applied to each readout euro, depedig o whether the curret firig activity i the readout pool was too high, too low, or about right, thus requirig at most two bits of global commuicatio. The precise versio of the learig rule was the p-delta rule that is discussed i Auer et al., (2001). 11

12 Givig a costat output for a time-varyig liquid state (caused by a time-varyig iput) is a serious challege for a LS, sice it caot rely o attractor states, ad the memory-less readout has to trasform the trasiet ad cotiuously chagig states of the liquid ito a stable output (see the discussio below ad Fig. 9 for details). I order to explore the limits of this simple eural implemetatio of a LS for computig o time-varyig iput, we chose aother classificatio task where all iformatio of the iput is cotaied i its temporal evolutio, more precisely i the iterspike itervals of a sigle iput spike trai. I this test, 8 radomly geerated Poisso spike trais over 250 ms, or equivaletly 2 Poisso spike trais over 1000 ms partitioed Figure 4: Evaluatig the fadig memory of a geeric eural microcircuit: the task. I this more challegig classificatio task all spike trais are of legth 1000 ms ad cosist of 4 segmets of legth 250 ms each. For each segmet 2 templates were geerated radomly (Poisso spike trai with a frequecy of 20 Hz); see upper traces. The actual iput spike trais of legth 1000 ms used for traiig ad testig were geerated by choosig for each segmet oe of the two associated templates, ad the geeratig a oisy versio by movig each spike by a amout draw from a Gaussia distributio with mea 0 ad a SD that we refer to as jitter (see lower trace for a visualizatio of the jitter with a SD of 4 ms). The task is to output with 4 differet readouts at time t = 1000 ms for each of the precedig 4 iput segmets the umber of the template from which the correspodig segmet of the iput was geerated. Results are summarized i Figures 5 ad 6. ito 4 segmets each (see top of Figure 4), were chose as template patters. Other spike trais over 1000 ms were geerated by choosig for each 250 ms segmet oe of the two templates for this segmet, ad by jitterig each spike i the templates (more precisely: each spike was moved by a amout draw from a Gaussia distributio with mea 0 ad a SD that we refer to as jitter, see bottom of Figure 4). A typical spike trai geerated i this way is show i the middle of Figure 4. Because of the oisy dislocatio of spikes it was impossible to recogize a specific template from a sigle iterspike iterval (ad there were o spatial cues cotaied i this sigle chael iput). Istead, a patter formed by several iterspike itervals had to be recogized ad classified retrospectively. Furthermore readouts were ot oly traied to classify at time t = 1000 ms (i.e., at after the iput spike trai had etered the circuit) the template from which the last 250 ms segmet of this iput spike trai had bee geerated, but other readouts were traied to classify simultaeously also the templates from which precedig segmets of the iput (which had etered the circuit several hudred ms earlier) had bee geerated. Obviously 12

13 the latter classificatio task is substatially more demadig, sice the correspodig earlier segmets of the iput spike trai may have left a clear trace i the curret firig activity of the recurret circuit just after they had etered the circuit, but this trace was subsequetly overwritte by the ext segmets of the iput spike trai (which had o correlatio with the choice of the earlier segmets). Altogether there were i this experimet 4 readouts f 1 to f 4, where f i had bee traied to classify at time t = 1000 ms the i-th idepedetly chose 250 ms segmet of the precedig iput spike trai. The performace of the LS, with a geeric recurret etwork of 135 I&F euros as liquid filter (see Appedix B), was evaluated after traiig of the readout pools o iputs from the same distributio (for jitter = 4 ms), but with a example that the LS had ot see before. The accuracy of the 4 readouts is plotted i pael A of Figure 5. It demostrates the fadig memory of a geeric recurret circuit of I&F euros, where iformatio about iputs that occurred several hudred ms ago ca be recovered eve after that iput segmet was subsequetly overwritte. Sice readout euros (ad euros withi the liquid circuit) were modeled with a realistic time costat of just 30 ms, the questio arises where this iformatio about earlier iputs had bee stored for Figure 5: Evaluatig the fadig memory of a geeric eural microcircuit: results. 4 readout modules f 1 to f 4, each cosistig of a sigle perceptro, were traied for their task by liear regressio. The readout module f i was traied to output 1 at time t=1000 ms if the i-th segmet of the previously preseted iput spike trai had bee costructed from the correspodig template 1, ad to output 0 at time t=1000 ms otherwise. Correctess (percetage of correct classificatio o a idepedet set of 500 iputs ot used for traiig) is calculated as average over 50 trials. I each trial ew Poisso spike trais were draw as templates, a ew radomly coected circuit was costructed (1 colum, λ=2; see Appedix B), ad the readout modules f 1 to f 4 were traied with1000 traiig examples geerated by the distributio described i Figure 4. A: Average correctess of the 4 readouts for ovel test iputs draw from the same distributio. B: Firig activity i the liquid circuit (time iterval [0.5 s, 0.8 s]) for a typical iput spike trai. C: Results of a cotrol experimet where all dyamic syapses i the liquid circuit had bee replaced by static syapses (the mea values of the syaptic stregths were uiformly re-scaled so that the average liquid activity is approximately the same as for dyamic syapses). The liquid state of this circuit cotaied substatially less iformatio about earlier iput segmets. D: Firig activity i the liquid circuit with static syapses used for the classificatio results reported i pael C. The circuit respose to each of the 4 iput spikes that etered the circuit durig the observed time iterval [0.5 s, 0.8 s] is quite stereotypical without dyamic syapses (except for the secod iput spike that arrives just 20 ms after the first oe). I cotrast the firig respose of the liquid circuit with dyamic syapses (pael B) is differet for each of the 4 iput spikes, showig that dyamic syapses edow these circuits with the capability to process ew iput differetly depedig o the cotext set by precedig iput, eve if that precedig iput occurred several hudred ms before. 13

14 several hudred ms. As a cotrol we repeated the same experimet with a liquid circuit where the dyamic syapses had bee replaced by static syapses (with syaptic weights that achieved about the same level of firig activity as the circuit with dyamic syapses). Pael C of Fig. 5 shows that this results i a sigificat loss i performace for the classificatio of all except for the last iput segmet. A possible explaatio is provided by the raster plots of firig activity i the liquid circuit with (pael B) ad without dyamic syapses (pael D), show here with high temporal resolutio. I the circuit with dyamic syapses the recurret activity differs for each of the 4 spikes that etered the circuit durig the time period show, demostratig that each ew spike is processed by the circuit i a idividual maer that depeds o the cotext defied by precedig iput spikes. I cotrast, the firig respose is very stereotypical for the same 4 iput spikes i the circuit without dyamic syapses, except for the respose to the secod spike that arrives withi 20 ms of the first oe (see the period betwee 500 ad 600 ms i pael D). This idicates Figure 6: Average correctess depeds o the parameter λ that cotrols the distributio of radom coectios withi the liquid circuit. Plotted is the average correctess (at time t=1000 ms, calculated as average over 50 trials as i Figure 5; same umber of traiig ad test examples) of the readout module f 3 (which is traied to classify retroactively the secod to last segmet of the precedig spike trai) as a fuctio of λ. The bad performace for λ=0 (o recurret coectios withi the circuit) shows that recurret coectios are essetial for achievig a satisfactory separatio property i eural microcircuits. Too large values of λ also decrease the performace because they support a chaotic respose. that the short term dyamics of syapses may play a essetial role i the itegratio of iformatio for real-time processig i eural microcircuits. Figure 6 examies aother aspect of eural microcircuits that appears to be importat for their separatio property: the statistical distributio of coectio legths withi the recurret circuit. Six types of liquid circuits, each cosistig of 135 I&F euros but with differet values of the parameter λ which regulated the average umber of coectios ad the average spatial legth of coectios (see Appedix B), were traied ad evaluated accordig to the same protocol ad for the same task as i Fig. 5. Show i Fig. 6 is for each of these 6 types of liquid circuits the average correctess of the readout f 3 o ovel iputs, after it had bee traied to classify the secod to last segmet of the iput spike trai. The performace was fairly low for circuits without recurret coectios (λ = 0). It also was fairly low for recurret circuits with large values of λ, whose largely legthidepedet distributio of coectios homogeized the microcircuit ad facilitated chaotic behavior. Hece for this classificatio task the ideal liquid circuit is a microcircuit that has i additio to local coectios to eighborig euros also a few log-rage coectios, thereby iterpolatig betwee the customarily cosidered extremes of strictly total coectivity (like i a cellular automato) o oe had, ad the locality-igorig global coectivity of a Hopfield et o the other had. 14

15 The performace results of eural implemetatios of LSs that were reported i this sectio should ot be viewed as absolute data o the computatioal power of recurret eural circuits. Rather the geeral theory suggests that their computatioal power icreases with ay improvemet i their separatio or approximatio property. Sice the approximatio property AP was already close to optimal for these etworks (icreasig the umber of euros i the readout module did ot icrease the performace sigificatly; ot show), the primary limitatio i performace lay i the separatio property SP. Ituitively it is clear that the liquid circuit eeds to be sufficietly complex to hold the details required for the particular task, but should reduce iformatio that is ot relevat to the task (for example spike time jitter). SP ca be egieered i may ways such as icorporatig euro diversity, implemetig specific syaptic architectures, alterig microcircuit coectivity, or simply recruitig more colums. The last optio is of particular iterest because it is ot available i most computatioal models. It will be explored i the ext sectio. Addig Computatioal Power A iterestig structural differece betwee eural systems ad our curret geeratio of artificial computig machiery is that the computatioal power of eural systems ca apparetly be elarged by recruitig more circuitry (without the eed to rewire old or ew circuits). We explored the cosequeces of recruitig additioal colums for eural implemetatios of LSs (see pael B of Fig. 7), ad compared it with the optio of just addig further coectios to the Figure 7: Separatio property ad performace of liquid circuits with larger umbers of coectios or euros. A ad B: Schematic drawigs of LSs cosistig of oe colum (A) ad four colums (B). Each colum cosists of = 135 I&F euros. C: Separatio property depeds o the structure of the liquid. Average state distace (at time t = 100 ms) calculated as described i Figure 2. A colum with high iteral coectivity (high λ) achieves higher separatio as a sigle colum with lower coectivity, but teds to chaotic behavior where it becomes equally sesitive to small ad large iput differeces d(u,v). O the other had the characteristic curve for a liquid cosistig of 4 colums with small λ is lower for values of d(u,v) lyig i the rage of jittered versios u ad v of the same spike trai patter (d(u,v) 0.1 for jitter 8 ms) ad higher for values of d(u,v) i the rage typical for spike trais u ad v from differet classes (mea: 0.22). D: Evaluatio of the same three types of liquid circuits for oise robust classificatio. Plotted is the average performace for the same task as i Fig. 6, but for various values of the jitter i iput spike times. Several colums (ot itercoected) with low iteral coectivity yield a better performig implemetatio of a LS for this computatioal task, as predicted by the aalysis of their separatio property. 15

16 primary oe-colum-liquid that we used so far (135 I&F euros with λ = 2, see pael A of Fig. 7). Pael C of Fig. 7 demostrates that the recruitmet of additioal colums icreases the separatio property of the liquid circuit i a desirable maer, where the distace betwee subsequet liquid states (always recorded at time t = 1000 ms i this experimet) is proportioal to the distace betwee the spike trai iputs that had previously etered the liquid circuit (spike trai distace measured i the same way as for Fig. 2). I cotrast the additio of more coectios to a sigle colum (λ = 8, see Appedix B) also icreases the separatio betwee subsequet liquid states, but i a quasi-chaotic maer where small iput distaces cause about the same distaces betwee subsequet liquid states as small iput differeces. I particular the subsequet liquid state distace is about equally large for two jittered versios of the iput spike trai state (yieldig typically a value of d(u,v) aroud 0.1) as for sigificatly differet iput spike trais that require differet outputs of the readouts. Thus improvig SP by alterig the itrisic microcircuitry of a sigle colum icreases sesitivity for the task, but also icreases sesitivity to oise. The performace of these differet types of liquid circuits for the same classificatio task as i Fig. 6 is cosistet with this aalysis of their characteristic separatio property. Show i pael D of Fig. 7 is their performace for various values of the spike time jitter i the iput spike trais. The optimizatio of SP for a specific distributio of iputs ad a specific group of readout modules is likely to arrive at a specific balace betwee the itrisic complexity of the microcircuitry ad the umber of repeatig colums. Parallel Computig i Real-Time o Novel Iputs Sice the liquid of the LS does ot have to be traied for a particular task, it supports parallel computig i real-time. This was demostrated by a test i which multiple spike trais were ijected ito the liquid ad multiple readout euros were traied to perform differet tasks i parallel. We added 6 readout modules to a liquid cosistig of 2 colums with differet values of λ 7. Each of the 6 readout modules was traied idepedetly for a completely differet olie task that required a output value at ay time t. We focused here o tasks that require diverse ad rapidly chagig aalog output resposes y(t). Figure 8 shows that after traiig each of these 6 tasks ca be performed i real-time with high accuracy. The performace show is for a ovel iput that was ot draw from the same distributio as the traiig examples, ad differs i several aspects from the traiig examples (thereby demostratig the possibility of extrageeralizatio i eural microcircuits, due to their iheret bias, that goes beyod the usual defiitio of geeralizatio i statistical learig theory). Readout-Assiged Equivalet States of a Dyamical System Real-time computatio o ovel iputs implies that the readout must be able to geerate a ivariat or appropriately scaled respose for ay iput eve though the liquid state may ever repeat. Ideed, Figure 3 showed already that the dyamics of readout pools ca become quite idepedet from the dyamics of the liquid eve though the liquid euros are the oly source 7 I order to combie high sesitivity with good geeralizatio performace we chose here a liquid cosistig of two colums as before, oe with λ=2, the other with λ=8 ad the iterval [ ] for the uiform distributio of the ospecific backgroud curret I b. 16

17 of iput. To examie the uderlyig mechaism for this relatively idepedet readout respose, we re-examied the readout pool from Figure 3. Whereas the firig activity withi the liquid circuit was highly dyamic, the firig activity i the readout pool was almost costat after traiig. The stability of the readout respose does ot simply come about because the readout oly samples a few uusual liquid euros as show by the distributio of syaptic weights oto a sample readout euro (Figure 9F). Sice the syaptic weights do ot chage after learig, this idicates that the readout euros have leared to defie a otio of equivalece for dyamic states of the liquid. Ideed, equivalece classes are a ievitable cosequece of collapsig the high dimesioal space of liquid states ito a sigle dimesio, but what is surprisig is that the equivalece classes are meaigful i terms of the task, allowig ivariat ad appropriately scaled readout resposes ad therefore real-time computatio o ovel iputs. Furthermore, while the iput rate may cotai saliet iformatio that is costat for a particular readout elemet, it may ot be for aother (see for example Fig. 8), idicatig that equivalece classes ad dyamic stability exist purely from the perspective of the readout elemets. 17

18 Figure 8: ulti-taskig i real-time. 4 iput spike trais of legth 2 s (show at the top) are ijected ito a liquid module cosistig of 2 colums (radomly costructed with the same parameters; see Appedix B), which is coected to multiple readout modules. Each readout module is traied to extract iformatio for a differet realtime computig task. The target fuctios are plotted as dashed lie, ad populatio respose of the correspodig readout module as solid lie. The tasks assiged to the 6 readout modules were the followig: Represet the sum of rates: at time t, output the sum of firig rates of all 4 iput spike trais withi the last 30ms. Represet the itegral of the sum of rates: at time t, output the total activity i all 4 iputs itegrated over the last 200ms. Patter detectio: output a high value if a specific spatio temporal spike patter appears. Represet a switch i spatial distributio of rates: output a high value if a specific iput patter occurs where the rate of iput spike trais 1 ad 2 goes up ad simultaeously the rate of iput spike trais 3 ad 4 goes dow, otherwise remai low. Represet the firig correlatio: at time t, output the umber of spike coicideces (ormalized ito the rage [0 1]) durig the last 75 ms for iputs 1 ad 3 ad separately for iputs 1 ad 2. Target readout values are plotted as dashed lies, actual outputs of the readout modules as solid lies, all i the same time scale as the 4 spike trais show at the top that eter the liquid circuit durig this 2 s time iterval. Results show are for a ovel iput that was ot draw from the same distributio as the traiig examples. 150 traiig examples were draw radomly from the followig distributio. Each iput spike trai was a idepedetly draw Possio spike trai with a time varyig rate of r(t) = A+B si (2 π f t + α). The parameters A, B, ad f where draw radomly from the followig itervals (the phase was fixed at α=0 deg): A [0Hz, 30Hz] ad [70Hz, 100Hz], B [0Hz, 30Hz] ad [70Hz, 100Hz], f [0.5Hz, 1Hz] ad [3Hz, 5Hz]. O this backgroud activity 4 differet patters had bee superimposed (always i the same order durig traiig): rate switch to iputs 1 ad 3, a burst patter, rate switch to iputs 1 ad 2, ad fially a spatio temporal spike patter. The results show are for a test iput that could ot be geerated by the same distributio as the traiig examples, because its base level (A=50Hz), as well as the amplitude (B=50Hz), frequecy (f=2hz) ad phase (α=180 deg) of the uderlyig time varyig firig rate of the Poisso iput spike trais were chose to lie i the middle of the gaps betwee the two itervals that were used for these parameters durig traiig. Furthermore the spatio-temporal patters (a burst patter, rate switch to iputs 1 ad 3, ad rate switch to iputs 1 ad 2), that were superimposed to achieve more iput variatio withi the observed 2 s, ever occured i this order ad at these time poits for ay traiig iput. Hece the accurate performace for this ovel iput demostrates substatial geeralizatio capabilities of the readouts after traiig. 18

19 Discussio We itroduce the liquid state machie, a ew paradigm for real-time computig o timevaryig iput streams. I cotrast to most computatioal models it does ot require the costructio of a circuit or program for a specific computatioal task. Rather, it relies o priciples of high-dimesioal dyamical systems ad learig theory that allow it to adapt uspecific evolved or foud recurret circuitry for a give computatioal task. Sice oly the readouts, ot the recurret circuit itself, have to be adapted for specific computatioal tasks, the same recurret circuit ca support completely differet real-time computatios i parallel. The uderlyig abstract computatioal model of a liquid state machie (LS) emphasizes the importace of perturbatios i dyamical systems for real-time computig, sice eve without stable states or attractors the separatio property ad the approximatio property may edow a dyamical system with virtually ulimited computatioal power o time-varyig iputs. I particular we have demostrated the computatioal uiversality of geeric recurret circuits of itegrate-ad-fire euros (eve with quite arbitrary coectio structure), if viewed as special cases of LSs. Apparetly this is the first stable ad geerally applicable method for usig geeric recurret etworks of itegrate-ad-fire euros to carry out a wide family of complex real-time computatios o spike trais as iputs. Hece this approach provides a platform for explorig the computatioal role of specific aspects of biological eural microcircuits. The computer simulatios reported i this article provide possible explaatios ot oly for the computatioal role of the highly recurret coectivity structure of eural circuits, but also for their characteristic distributio of coectio legths, which places their coectivity structure betwee the extremes of strictly local coectivity (cellular automata or coupled map lattices) ad uiform global coectivity (Hopfield ets) that are usually addressed i theoretical studies. Furthermore our computer simulatios suggest a importat computatioal role of dyamic syapses for real-time computig o time-varyig iputs. Fially, we reveal a most uexpected ad remarkable priciple that readout elemets ca establish their ow equivalece relatioships o high-dimesioal trasiet states of a dyamical system, makig it possible to geerate stable ad appropriately scaled output resposes eve if the iteral state ever coverges to a attractor state. I cotrast to virtually all computatioal models from computer sciece or artificial eural etworks, this computatioal model is ehaced rather tha hampered by the presece of diverse computatioal uits. Hece it may also provide isight ito the computatioal role of the complexity ad diversity of euros ad syapses (see for example (Gupta et al., 2000)). While there are may plausible models for spatial aspects of eural computatio, a biologically realistic framework for modelig temporal aspects of eural computatio has bee missig. I cotrast to models ispired by computer sciece, the liquid state machie does ot try to reduce these temporal aspects to trasitios betwee stable states or limit cycles, ad it does ot require delay lies or buffers. Istead it proposes that the trajectory of iteral states of a recurret eural circuit provides a raw, ubiased, ad uiversal source of temporally itegrated iformatio, from which specific readout elemets ca extract specific iformatio about past iputs for their idividual task. Hece the otorious trial-to-trial stimulus respose variatios i sigle ad populatios of euros observed experimetally, may reflect a accumulatio of iformatio from previous iputs i the trajectory of iteral states, rather tha oise (see also (Arieli et al., 1996)). This would imply that averagig over trials or biig, peels out most of the iformatio processed by recurret microcircuits ad leaves mostly topographic iformatio. 19

20 This approach also offers ew ideas for models of the computatioal orgaisatio of cogitio. It suggests that it may ot be ecessary to scatter all iformatio about sesory iput by recodig it through feedforward processig as output vector of a esemble of feature Figure 9: Readout assiged equivalet states of a dyamical system. A LS (liquid circuit as i Figure 3) was traied for the classificatio task as described i Figure 3. Results show are for a ovel test iput (draw from the same distributio as the traiig examples). A: The test iput cosists of 40 Poisso spike trais, each with a costat rate of 5 Hz. B: Raster plot of the 135 liquid euros i respose to this iput. Note the large variety of liquid states that arise durig this time period. C: Populatio rate of the liquid (bi-size 20 ms). Note that this populatio rate chages quit a bit over time. D: Readout respose (solid lie) ad target respose (dashed lie). The target respose had a costat value of 1 for this iput. The output of the traied readout module is also almost costat for this test example (except for the begiig), although its iput, the liquid states of the recurret circuit, varied quit a bit durig this time period. F: Weight distributio of a sigle readout euro. detectors with fixed receptive fields (thereby creatig the "bidig problem"). It proposes that at the same time more global iformatio about precedig iputs ca be preserved i the trajectories of very high dimesioal dyamical systems, from which multiple readout modules extract ad combie the iformatio eeded for their specific tasks. This approach is evertheless compatible with experimetal data that cofirm the existece of special maps of feature detectors. These 20

Modified Line Search Method for Global Optimization

Modified Line Search Method for Global Optimization Modified Lie Search Method for Global Optimizatio Cria Grosa ad Ajith Abraham Ceter of Excellece for Quatifiable Quality of Service Norwegia Uiversity of Sciece ad Techology Trodheim, Norway {cria, ajith}@q2s.tu.o

More information

I. Chi-squared Distributions

I. Chi-squared Distributions 1 M 358K Supplemet to Chapter 23: CHI-SQUARED DISTRIBUTIONS, T-DISTRIBUTIONS, AND DEGREES OF FREEDOM To uderstad t-distributios, we first eed to look at aother family of distributios, the chi-squared distributios.

More information

In nite Sequences. Dr. Philippe B. Laval Kennesaw State University. October 9, 2008

In nite Sequences. Dr. Philippe B. Laval Kennesaw State University. October 9, 2008 I ite Sequeces Dr. Philippe B. Laval Keesaw State Uiversity October 9, 2008 Abstract This had out is a itroductio to i ite sequeces. mai de itios ad presets some elemetary results. It gives the I ite Sequeces

More information

Case Study. Normal and t Distributions. Density Plot. Normal Distributions

Case Study. Normal and t Distributions. Density Plot. Normal Distributions Case Study Normal ad t Distributios Bret Halo ad Bret Larget Departmet of Statistics Uiversity of Wiscosi Madiso October 11 13, 2011 Case Study Body temperature varies withi idividuals over time (it ca

More information

Non-life insurance mathematics. Nils F. Haavardsson, University of Oslo and DNB Skadeforsikring

Non-life insurance mathematics. Nils F. Haavardsson, University of Oslo and DNB Skadeforsikring No-life isurace mathematics Nils F. Haavardsso, Uiversity of Oslo ad DNB Skadeforsikrig Mai issues so far Why does isurace work? How is risk premium defied ad why is it importat? How ca claim frequecy

More information

Properties of MLE: consistency, asymptotic normality. Fisher information.

Properties of MLE: consistency, asymptotic normality. Fisher information. Lecture 3 Properties of MLE: cosistecy, asymptotic ormality. Fisher iformatio. I this sectio we will try to uderstad why MLEs are good. Let us recall two facts from probability that we be used ofte throughout

More information

3. Covariance and Correlation

3. Covariance and Correlation Virtual Laboratories > 3. Expected Value > 1 2 3 4 5 6 3. Covariace ad Correlatio Recall that by takig the expected value of various trasformatios of a radom variable, we ca measure may iterestig characteristics

More information

LECTURE 13: Cross-validation

LECTURE 13: Cross-validation LECTURE 3: Cross-validatio Resampli methods Cross Validatio Bootstrap Bias ad variace estimatio with the Bootstrap Three-way data partitioi Itroductio to Patter Aalysis Ricardo Gutierrez-Osua Texas A&M

More information

Confidence Intervals for One Mean

Confidence Intervals for One Mean Chapter 420 Cofidece Itervals for Oe Mea Itroductio This routie calculates the sample size ecessary to achieve a specified distace from the mea to the cofidece limit(s) at a stated cofidece level for a

More information

Vladimir N. Burkov, Dmitri A. Novikov MODELS AND METHODS OF MULTIPROJECTS MANAGEMENT

Vladimir N. Burkov, Dmitri A. Novikov MODELS AND METHODS OF MULTIPROJECTS MANAGEMENT Keywords: project maagemet, resource allocatio, etwork plaig Vladimir N Burkov, Dmitri A Novikov MODELS AND METHODS OF MULTIPROJECTS MANAGEMENT The paper deals with the problems of resource allocatio betwee

More information

7. Sample Covariance and Correlation

7. Sample Covariance and Correlation 1 of 8 7/16/2009 6:06 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 7. Sample Covariace ad Correlatio The Bivariate Model Suppose agai that we have a basic radom experimet, ad that X ad Y

More information

Analyzing Longitudinal Data from Complex Surveys Using SUDAAN

Analyzing Longitudinal Data from Complex Surveys Using SUDAAN Aalyzig Logitudial Data from Complex Surveys Usig SUDAAN Darryl Creel Statistics ad Epidemiology, RTI Iteratioal, 312 Trotter Farm Drive, Rockville, MD, 20850 Abstract SUDAAN: Software for the Statistical

More information

Chapter 6: Variance, the law of large numbers and the Monte-Carlo method

Chapter 6: Variance, the law of large numbers and the Monte-Carlo method Chapter 6: Variace, the law of large umbers ad the Mote-Carlo method Expected value, variace, ad Chebyshev iequality. If X is a radom variable recall that the expected value of X, E[X] is the average value

More information

Department of Computer Science, University of Otago

Department of Computer Science, University of Otago Departmet of Computer Sciece, Uiversity of Otago Techical Report OUCS-2006-09 Permutatios Cotaiig May Patters Authors: M.H. Albert Departmet of Computer Sciece, Uiversity of Otago Micah Colema, Rya Fly

More information

CHAPTER 3 DIGITAL CODING OF SIGNALS

CHAPTER 3 DIGITAL CODING OF SIGNALS CHAPTER 3 DIGITAL CODING OF SIGNALS Computers are ofte used to automate the recordig of measuremets. The trasducers ad sigal coditioig circuits produce a voltage sigal that is proportioal to a quatity

More information

Hypothesis testing. Null and alternative hypotheses

Hypothesis testing. Null and alternative hypotheses Hypothesis testig Aother importat use of samplig distributios is to test hypotheses about populatio parameters, e.g. mea, proportio, regressio coefficiets, etc. For example, it is possible to stipulate

More information

1 Computing the Standard Deviation of Sample Means

1 Computing the Standard Deviation of Sample Means Computig the Stadard Deviatio of Sample Meas Quality cotrol charts are based o sample meas ot o idividual values withi a sample. A sample is a group of items, which are cosidered all together for our aalysis.

More information

Center, Spread, and Shape in Inference: Claims, Caveats, and Insights

Center, Spread, and Shape in Inference: Claims, Caveats, and Insights Ceter, Spread, ad Shape i Iferece: Claims, Caveats, ad Isights Dr. Nacy Pfeig (Uiversity of Pittsburgh) AMATYC November 2008 Prelimiary Activities 1. I would like to produce a iterval estimate for the

More information

Overview of some probability distributions.

Overview of some probability distributions. Lecture Overview of some probability distributios. I this lecture we will review several commo distributios that will be used ofte throughtout the class. Each distributio is usually described by its probability

More information

Trading the randomness - Designing an optimal trading strategy under a drifted random walk price model

Trading the randomness - Designing an optimal trading strategy under a drifted random walk price model Tradig the radomess - Desigig a optimal tradig strategy uder a drifted radom walk price model Yuao Wu Math 20 Project Paper Professor Zachary Hamaker Abstract: I this paper the author iteds to explore

More information

Chair for Network Architectures and Services Institute of Informatics TU München Prof. Carle. Network Security. Chapter 2 Basics

Chair for Network Architectures and Services Institute of Informatics TU München Prof. Carle. Network Security. Chapter 2 Basics Chair for Network Architectures ad Services Istitute of Iformatics TU Müche Prof. Carle Network Security Chapter 2 Basics 2.4 Radom Number Geeratio for Cryptographic Protocols Motivatio It is crucial to

More information

Normal Distribution.

Normal Distribution. Normal Distributio www.icrf.l Normal distributio I probability theory, the ormal or Gaussia distributio, is a cotiuous probability distributio that is ofte used as a first approimatio to describe realvalued

More information

5: Introduction to Estimation

5: Introduction to Estimation 5: Itroductio to Estimatio Cotets Acroyms ad symbols... 1 Statistical iferece... Estimatig µ with cofidece... 3 Samplig distributio of the mea... 3 Cofidece Iterval for μ whe σ is kow before had... 4 Sample

More information

Chapter 7 Methods of Finding Estimators

Chapter 7 Methods of Finding Estimators Chapter 7 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 011 Chapter 7 Methods of Fidig Estimators Sectio 7.1 Itroductio Defiitio 7.1.1 A poit estimator is ay fuctio W( X) W( X1, X,, X ) of

More information

Subject CT5 Contingencies Core Technical Syllabus

Subject CT5 Contingencies Core Technical Syllabus Subject CT5 Cotigecies Core Techical Syllabus for the 2015 exams 1 Jue 2014 Aim The aim of the Cotigecies subject is to provide a groudig i the mathematical techiques which ca be used to model ad value

More information

5 Boolean Decision Trees (February 11)

5 Boolean Decision Trees (February 11) 5 Boolea Decisio Trees (February 11) 5.1 Graph Coectivity Suppose we are give a udirected graph G, represeted as a boolea adjacecy matrix = (a ij ), where a ij = 1 if ad oly if vertices i ad j are coected

More information

1 Correlation and Regression Analysis

1 Correlation and Regression Analysis 1 Correlatio ad Regressio Aalysis I this sectio we will be ivestigatig the relatioship betwee two cotiuous variable, such as height ad weight, the cocetratio of a ijected drug ad heart rate, or the cosumptio

More information

Lesson 17 Pearson s Correlation Coefficient

Lesson 17 Pearson s Correlation Coefficient Outlie Measures of Relatioships Pearso s Correlatio Coefficiet (r) -types of data -scatter plots -measure of directio -measure of stregth Computatio -covariatio of X ad Y -uique variatio i X ad Y -measurig

More information

Determining the sample size

Determining the sample size Determiig the sample size Oe of the most commo questios ay statisticia gets asked is How large a sample size do I eed? Researchers are ofte surprised to fid out that the aswer depeds o a umber of factors

More information

INVESTMENT PERFORMANCE COUNCIL (IPC) Guidance Statement on Calculation Methodology

INVESTMENT PERFORMANCE COUNCIL (IPC) Guidance Statement on Calculation Methodology Adoptio Date: 4 March 2004 Effective Date: 1 Jue 2004 Retroactive Applicatio: No Public Commet Period: Aug Nov 2002 INVESTMENT PERFORMANCE COUNCIL (IPC) Preface Guidace Statemet o Calculatio Methodology

More information

Escola Federal de Engenharia de Itajubá

Escola Federal de Engenharia de Itajubá Escola Federal de Egeharia de Itajubá Departameto de Egeharia Mecâica Pós-Graduação em Egeharia Mecâica MPF04 ANÁLISE DE SINAIS E AQUISÇÃO DE DADOS SINAIS E SISTEMAS Trabalho 02 (MATLAB) Prof. Dr. José

More information

Domain 1 - Describe Cisco VoIP Implementations

Domain 1 - Describe Cisco VoIP Implementations Maual ONT (642-8) 1-800-418-6789 Domai 1 - Describe Cisco VoIP Implemetatios Advatages of VoIP Over Traditioal Switches Voice over IP etworks have may advatages over traditioal circuit switched voice etworks.

More information

Recursion and Recurrences

Recursion and Recurrences Chapter 5 Recursio ad Recurreces 5.1 Growth Rates of Solutios to Recurreces Divide ad Coquer Algorithms Oe of the most basic ad powerful algorithmic techiques is divide ad coquer. Cosider, for example,

More information

Week 3 Conditional probabilities, Bayes formula, WEEK 3 page 1 Expected value of a random variable

Week 3 Conditional probabilities, Bayes formula, WEEK 3 page 1 Expected value of a random variable Week 3 Coditioal probabilities, Bayes formula, WEEK 3 page 1 Expected value of a radom variable We recall our discussio of 5 card poker hads. Example 13 : a) What is the probability of evet A that a 5

More information

*The most important feature of MRP as compared with ordinary inventory control analysis is its time phasing feature.

*The most important feature of MRP as compared with ordinary inventory control analysis is its time phasing feature. Itegrated Productio ad Ivetory Cotrol System MRP ad MRP II Framework of Maufacturig System Ivetory cotrol, productio schedulig, capacity plaig ad fiacial ad busiess decisios i a productio system are iterrelated.

More information

Taking DCOP to the Real World: Efficient Complete Solutions for Distributed Multi-Event Scheduling

Taking DCOP to the Real World: Efficient Complete Solutions for Distributed Multi-Event Scheduling Taig DCOP to the Real World: Efficiet Complete Solutios for Distributed Multi-Evet Schedulig Rajiv T. Maheswara, Milid Tambe, Emma Bowrig, Joatha P. Pearce, ad Pradeep araatham Uiversity of Souther Califoria

More information

Project Deliverables. CS 361, Lecture 28. Outline. Project Deliverables. Administrative. Project Comments

Project Deliverables. CS 361, Lecture 28. Outline. Project Deliverables. Administrative. Project Comments Project Deliverables CS 361, Lecture 28 Jared Saia Uiversity of New Mexico Each Group should tur i oe group project cosistig of: About 6-12 pages of text (ca be loger with appedix) 6-12 figures (please

More information

Lecture 13. Lecturer: Jonathan Kelner Scribe: Jonathan Pines (2009)

Lecture 13. Lecturer: Jonathan Kelner Scribe: Jonathan Pines (2009) 18.409 A Algorithmist s Toolkit October 27, 2009 Lecture 13 Lecturer: Joatha Keler Scribe: Joatha Pies (2009) 1 Outlie Last time, we proved the Bru-Mikowski iequality for boxes. Today we ll go over the

More information

INVESTMENT PERFORMANCE COUNCIL (IPC)

INVESTMENT PERFORMANCE COUNCIL (IPC) INVESTMENT PEFOMANCE COUNCIL (IPC) INVITATION TO COMMENT: Global Ivestmet Performace Stadards (GIPS ) Guidace Statemet o Calculatio Methodology The Associatio for Ivestmet Maagemet ad esearch (AIM) seeks

More information

Evaluation of Different Fitness Functions for the Evolutionary Testing of an Autonomous Parking System

Evaluation of Different Fitness Functions for the Evolutionary Testing of an Autonomous Parking System Evaluatio of Differet Fitess Fuctios for the Evolutioary Testig of a Autoomous Parkig System Joachim Wegeer 1, Oliver Bühler 2 1 DaimlerChrysler AG, Research ad Techology, Alt-Moabit 96 a, D-1559 Berli,

More information

Engineering Data Management

Engineering Data Management BaaERP 5.0c Maufacturig Egieerig Data Maagemet Module Procedure UP128A US Documetiformatio Documet Documet code : UP128A US Documet group : User Documetatio Documet title : Egieerig Data Maagemet Applicatio/Package

More information

Discrete Mathematics and Probability Theory Spring 2014 Anant Sahai Note 13

Discrete Mathematics and Probability Theory Spring 2014 Anant Sahai Note 13 EECS 70 Discrete Mathematics ad Probability Theory Sprig 2014 Aat Sahai Note 13 Itroductio At this poit, we have see eough examples that it is worth just takig stock of our model of probability ad may

More information

0.7 0.6 0.2 0 0 96 96.5 97 97.5 98 98.5 99 99.5 100 100.5 96.5 97 97.5 98 98.5 99 99.5 100 100.5

0.7 0.6 0.2 0 0 96 96.5 97 97.5 98 98.5 99 99.5 100 100.5 96.5 97 97.5 98 98.5 99 99.5 100 100.5 Sectio 13 Kolmogorov-Smirov test. Suppose that we have a i.i.d. sample X 1,..., X with some ukow distributio P ad we would like to test the hypothesis that P is equal to a particular distributio P 0, i.e.

More information

Universal coding for classes of sources

Universal coding for classes of sources Coexios module: m46228 Uiversal codig for classes of sources Dever Greee This work is produced by The Coexios Project ad licesed uder the Creative Commos Attributio Licese We have discussed several parametric

More information

The second difference is the sequence of differences of the first difference sequence, 2

The second difference is the sequence of differences of the first difference sequence, 2 Differece Equatios I differetial equatios, you look for a fuctio that satisfies ad equatio ivolvig derivatives. I differece equatios, istead of a fuctio of a cotiuous variable (such as time), we look for

More information

(VCP-310) 1-800-418-6789

(VCP-310) 1-800-418-6789 Maual VMware Lesso 1: Uderstadig the VMware Product Lie I this lesso, you will first lear what virtualizatio is. Next, you ll explore the products offered by VMware that provide virtualizatio services.

More information

4.1 Sigma Notation and Riemann Sums

4.1 Sigma Notation and Riemann Sums 0 the itegral. Sigma Notatio ad Riema Sums Oe strategy for calculatig the area of a regio is to cut the regio ito simple shapes, calculate the area of each simple shape, ad the add these smaller areas

More information

SECTION 1.5 : SUMMATION NOTATION + WORK WITH SEQUENCES

SECTION 1.5 : SUMMATION NOTATION + WORK WITH SEQUENCES SECTION 1.5 : SUMMATION NOTATION + WORK WITH SEQUENCES Read Sectio 1.5 (pages 5 9) Overview I Sectio 1.5 we lear to work with summatio otatio ad formulas. We will also itroduce a brief overview of sequeces,

More information

Basic Measurement Issues. Sampling Theory and Analog-to-Digital Conversion

Basic Measurement Issues. Sampling Theory and Analog-to-Digital Conversion Theory ad Aalog-to-Digital Coversio Itroductio/Defiitios Aalog-to-digital coversio Rate Frequecy Aalysis Basic Measuremet Issues Reliability the extet to which a measuremet procedure yields the same results

More information

The Forgotten Middle. research readiness results. Executive Summary

The Forgotten Middle. research readiness results. Executive Summary The Forgotte Middle Esurig that All Studets Are o Target for College ad Career Readiess before High School Executive Summary Today, college readiess also meas career readiess. While ot every high school

More information

Approximating Area under a curve with rectangles. To find the area under a curve we approximate the area using rectangles and then use limits to find

Approximating Area under a curve with rectangles. To find the area under a curve we approximate the area using rectangles and then use limits to find 1.8 Approximatig Area uder a curve with rectagles 1.6 To fid the area uder a curve we approximate the area usig rectagles ad the use limits to fid 1.4 the area. Example 1 Suppose we wat to estimate 1.

More information

Research Article Sign Data Derivative Recovery

Research Article Sign Data Derivative Recovery Iteratioal Scholarly Research Network ISRN Applied Mathematics Volume 0, Article ID 63070, 7 pages doi:0.540/0/63070 Research Article Sig Data Derivative Recovery L. M. Housto, G. A. Glass, ad A. D. Dymikov

More information

A probabilistic proof of a binomial identity

A probabilistic proof of a binomial identity A probabilistic proof of a biomial idetity Joatho Peterso Abstract We give a elemetary probabilistic proof of a biomial idetity. The proof is obtaied by computig the probability of a certai evet i two

More information

Key Ideas Section 8-1: Overview hypothesis testing Hypothesis Hypothesis Test Section 8-2: Basics of Hypothesis Testing Null Hypothesis

Key Ideas Section 8-1: Overview hypothesis testing Hypothesis Hypothesis Test Section 8-2: Basics of Hypothesis Testing Null Hypothesis Chapter 8 Key Ideas Hypothesis (Null ad Alterative), Hypothesis Test, Test Statistic, P-value Type I Error, Type II Error, Sigificace Level, Power Sectio 8-1: Overview Cofidece Itervals (Chapter 7) are

More information

Class Meeting # 16: The Fourier Transform on R n

Class Meeting # 16: The Fourier Transform on R n MATH 18.152 COUSE NOTES - CLASS MEETING # 16 18.152 Itroductio to PDEs, Fall 2011 Professor: Jared Speck Class Meetig # 16: The Fourier Trasform o 1. Itroductio to the Fourier Trasform Earlier i the course,

More information

Domain 1: Designing a SQL Server Instance and a Database Solution

Domain 1: Designing a SQL Server Instance and a Database Solution Maual SQL Server 2008 Desig, Optimize ad Maitai (70-450) 1-800-418-6789 Domai 1: Desigig a SQL Server Istace ad a Database Solutio Desigig for CPU, Memory ad Storage Capacity Requiremets Whe desigig a

More information

Designing Incentives for Online Question and Answer Forums

Designing Incentives for Online Question and Answer Forums Desigig Icetives for Olie Questio ad Aswer Forums Shaili Jai School of Egieerig ad Applied Scieces Harvard Uiversity Cambridge, MA 0238 USA shailij@eecs.harvard.edu Yilig Che School of Egieerig ad Applied

More information

Research Method (I) --Knowledge on Sampling (Simple Random Sampling)

Research Method (I) --Knowledge on Sampling (Simple Random Sampling) Research Method (I) --Kowledge o Samplig (Simple Radom Samplig) 1. Itroductio to samplig 1.1 Defiitio of samplig Samplig ca be defied as selectig part of the elemets i a populatio. It results i the fact

More information

.04. This means $1000 is multiplied by 1.02 five times, once for each of the remaining sixmonth

.04. This means $1000 is multiplied by 1.02 five times, once for each of the remaining sixmonth Questio 1: What is a ordiary auity? Let s look at a ordiary auity that is certai ad simple. By this, we mea a auity over a fixed term whose paymet period matches the iterest coversio period. Additioally,

More information

Chapter 7 - Sampling Distributions. 1 Introduction. What is statistics? It consist of three major areas:

Chapter 7 - Sampling Distributions. 1 Introduction. What is statistics? It consist of three major areas: Chapter 7 - Samplig Distributios 1 Itroductio What is statistics? It cosist of three major areas: Data Collectio: samplig plas ad experimetal desigs Descriptive Statistics: umerical ad graphical summaries

More information

CHAPTER 3 THE TIME VALUE OF MONEY

CHAPTER 3 THE TIME VALUE OF MONEY CHAPTER 3 THE TIME VALUE OF MONEY OVERVIEW A dollar i the had today is worth more tha a dollar to be received i the future because, if you had it ow, you could ivest that dollar ad ear iterest. Of all

More information

GENERATING A FRACTAL SQUARE

GENERATING A FRACTAL SQUARE GENERATING A FRACTAL SQUARE I 194 the Swedish mathematicia Helge vo Koch(187-194 itroduced oe o the earliest ow ractals, amely, the Koch Sowlae. It is a closed cotiuous curve with discotiuities i its derivative

More information

Biology 171L Environment and Ecology Lab Lab 2: Descriptive Statistics, Presenting Data and Graphing Relationships

Biology 171L Environment and Ecology Lab Lab 2: Descriptive Statistics, Presenting Data and Graphing Relationships Biology 171L Eviromet ad Ecology Lab Lab : Descriptive Statistics, Presetig Data ad Graphig Relatioships Itroductio Log lists of data are ofte ot very useful for idetifyig geeral treds i the data or the

More information

PROCEEDINGS OF THE YEREVAN STATE UNIVERSITY AN ALTERNATIVE MODEL FOR BONUS-MALUS SYSTEM

PROCEEDINGS OF THE YEREVAN STATE UNIVERSITY AN ALTERNATIVE MODEL FOR BONUS-MALUS SYSTEM PROCEEDINGS OF THE YEREVAN STATE UNIVERSITY Physical ad Mathematical Scieces 2015, 1, p. 15 19 M a t h e m a t i c s AN ALTERNATIVE MODEL FOR BONUS-MALUS SYSTEM A. G. GULYAN Chair of Actuarial Mathematics

More information

Running Time ( 3.1) Analysis of Algorithms. Experimental Studies ( 3.1.1) Limitations of Experiments. Pseudocode ( 3.1.2) Theoretical Analysis

Running Time ( 3.1) Analysis of Algorithms. Experimental Studies ( 3.1.1) Limitations of Experiments. Pseudocode ( 3.1.2) Theoretical Analysis Ruig Time ( 3.) Aalysis of Algorithms Iput Algorithm Output A algorithm is a step-by-step procedure for solvig a problem i a fiite amout of time. Most algorithms trasform iput objects ito output objects.

More information

Here are a couple of warnings to my students who may be here to get a copy of what happened on a day that you missed.

Here are a couple of warnings to my students who may be here to get a copy of what happened on a day that you missed. This documet was writte ad copyrighted by Paul Dawkis. Use of this documet ad its olie versio is govered by the Terms ad Coditios of Use located at http://tutorial.math.lamar.edu/terms.asp. The olie versio

More information

hp calculators HP 12C Statistics - average and standard deviation Average and standard deviation concepts HP12C average and standard deviation

hp calculators HP 12C Statistics - average and standard deviation Average and standard deviation concepts HP12C average and standard deviation HP 1C Statistics - average ad stadard deviatio Average ad stadard deviatio cocepts HP1C average ad stadard deviatio Practice calculatig averages ad stadard deviatios with oe or two variables HP 1C Statistics

More information

Incremental calculation of weighted mean and variance

Incremental calculation of weighted mean and variance Icremetal calculatio of weighted mea ad variace Toy Fich faf@cam.ac.uk dot@dotat.at Uiversity of Cambridge Computig Service February 009 Abstract I these otes I eplai how to derive formulae for umerically

More information

Your organization has a Class B IP address of 166.144.0.0 Before you implement subnetting, the Network ID and Host ID are divided as follows:

Your organization has a Class B IP address of 166.144.0.0 Before you implement subnetting, the Network ID and Host ID are divided as follows: Subettig Subettig is used to subdivide a sigle class of etwork i to multiple smaller etworks. Example: Your orgaizatio has a Class B IP address of 166.144.0.0 Before you implemet subettig, the Network

More information

Chapter 5 Unit 1. IET 350 Engineering Economics. Learning Objectives Chapter 5. Learning Objectives Unit 1. Annual Amount and Gradient Functions

Chapter 5 Unit 1. IET 350 Engineering Economics. Learning Objectives Chapter 5. Learning Objectives Unit 1. Annual Amount and Gradient Functions Chapter 5 Uit Aual Amout ad Gradiet Fuctios IET 350 Egieerig Ecoomics Learig Objectives Chapter 5 Upo completio of this chapter you should uderstad: Calculatig future values from aual amouts. Calculatig

More information

Notes on exponential generating functions and structures.

Notes on exponential generating functions and structures. Notes o expoetial geeratig fuctios ad structures. 1. The cocept of a structure. Cosider the followig coutig problems: (1) to fid for each the umber of partitios of a -elemet set, (2) to fid for each the

More information

iprox sensors iprox inductive sensors iprox programming tools ProxView programming software iprox the world s most versatile proximity sensor

iprox sensors iprox inductive sensors iprox programming tools ProxView programming software iprox the world s most versatile proximity sensor iprox sesors iprox iductive sesors iprox programmig tools ProxView programmig software iprox the world s most versatile proximity sesor The world s most versatile proximity sesor Eato s iproxe is syoymous

More information

Plug-in martingales for testing exchangeability on-line

Plug-in martingales for testing exchangeability on-line Plug-i martigales for testig exchageability o-lie Valetia Fedorova, Alex Gammerma, Ilia Nouretdiov, ad Vladimir Vovk Computer Learig Research Cetre Royal Holloway, Uiversity of Lodo, UK {valetia,ilia,alex,vovk}@cs.rhul.ac.uk

More information

Enhancing Oracle Business Intelligence with cubus EV How users of Oracle BI on Essbase cubes can benefit from cubus outperform EV Analytics (cubus EV)

Enhancing Oracle Business Intelligence with cubus EV How users of Oracle BI on Essbase cubes can benefit from cubus outperform EV Analytics (cubus EV) Ehacig Oracle Busiess Itelligece with cubus EV How users of Oracle BI o Essbase cubes ca beefit from cubus outperform EV Aalytics (cubus EV) CONTENT 01 cubus EV as a ehacemet to Oracle BI o Essbase 02

More information

Trackless online algorithms for the server problem

Trackless online algorithms for the server problem Iformatio Processig Letters 74 (2000) 73 79 Trackless olie algorithms for the server problem Wolfgag W. Bei,LawreceL.Larmore 1 Departmet of Computer Sciece, Uiversity of Nevada, Las Vegas, NV 89154, USA

More information

9.8: THE POWER OF A TEST

9.8: THE POWER OF A TEST 9.8: The Power of a Test CD9-1 9.8: THE POWER OF A TEST I the iitial discussio of statistical hypothesis testig, the two types of risks that are take whe decisios are made about populatio parameters based

More information

Maximum Likelihood Estimators.

Maximum Likelihood Estimators. Lecture 2 Maximum Likelihood Estimators. Matlab example. As a motivatio, let us look at oe Matlab example. Let us geerate a radom sample of size 00 from beta distributio Beta(5, 2). We will lear the defiitio

More information

Quadrat Sampling in Population Ecology

Quadrat Sampling in Population Ecology Quadrat Samplig i Populatio Ecology Backgroud Estimatig the abudace of orgaisms. Ecology is ofte referred to as the "study of distributio ad abudace". This beig true, we would ofte like to kow how may

More information

Output Analysis (2, Chapters 10 &11 Law)

Output Analysis (2, Chapters 10 &11 Law) B. Maddah ENMG 6 Simulatio 05/0/07 Output Aalysis (, Chapters 10 &11 Law) Comparig alterative system cofiguratio Sice the output of a simulatio is radom, the comparig differet systems via simulatio should

More information

Measures of Spread and Boxplots Discrete Math, Section 9.4

Measures of Spread and Boxplots Discrete Math, Section 9.4 Measures of Spread ad Boxplots Discrete Math, Sectio 9.4 We start with a example: Example 1: Comparig Mea ad Media Compute the mea ad media of each data set: S 1 = {4, 6, 8, 10, 1, 14, 16} S = {4, 7, 9,

More information

Exploratory Data Analysis

Exploratory Data Analysis 1 Exploratory Data Aalysis Exploratory data aalysis is ofte the rst step i a statistical aalysis, for it helps uderstadig the mai features of the particular sample that a aalyst is usig. Itelliget descriptios

More information

NEW HIGH PERFORMANCE COMPUTATIONAL METHODS FOR MORTGAGES AND ANNUITIES. Yuri Shestopaloff,

NEW HIGH PERFORMANCE COMPUTATIONAL METHODS FOR MORTGAGES AND ANNUITIES. Yuri Shestopaloff, NEW HIGH PERFORMNCE COMPUTTIONL METHODS FOR MORTGGES ND NNUITIES Yuri Shestopaloff, Geerally, mortgage ad auity equatios do ot have aalytical solutios for ukow iterest rate, which has to be foud usig umerical

More information

STUDENTS PARTICIPATION IN ONLINE LEARNING IN BUSINESS COURSES AT UNIVERSITAS TERBUKA, INDONESIA. Maya Maria, Universitas Terbuka, Indonesia

STUDENTS PARTICIPATION IN ONLINE LEARNING IN BUSINESS COURSES AT UNIVERSITAS TERBUKA, INDONESIA. Maya Maria, Universitas Terbuka, Indonesia STUDENTS PARTICIPATION IN ONLINE LEARNING IN BUSINESS COURSES AT UNIVERSITAS TERBUKA, INDONESIA Maya Maria, Uiversitas Terbuka, Idoesia Co-author: Amiuddi Zuhairi, Uiversitas Terbuka, Idoesia Kuria Edah

More information

Asymptotic Growth of Functions

Asymptotic Growth of Functions CMPS Itroductio to Aalysis of Algorithms Fall 3 Asymptotic Growth of Fuctios We itroduce several types of asymptotic otatio which are used to compare the performace ad efficiecy of algorithms As we ll

More information

Cantilever Beam Experiment

Cantilever Beam Experiment Mechaical Egieerig Departmet Uiversity of Massachusetts Lowell Catilever Beam Experimet Backgroud A disk drive maufacturer is redesigig several disk drive armature mechaisms. This is the result of evaluatio

More information

Page 1. Real Options for Engineering Systems. What are we up to? Today s agenda. J1: Real Options for Engineering Systems. Richard de Neufville

Page 1. Real Options for Engineering Systems. What are we up to? Today s agenda. J1: Real Options for Engineering Systems. Richard de Neufville Real Optios for Egieerig Systems J: Real Optios for Egieerig Systems By (MIT) Stefa Scholtes (CU) Course website: http://msl.mit.edu/cmi/ardet_2002 Stefa Scholtes Judge Istitute of Maagemet, CU Slide What

More information

Multiplexers and Demultiplexers

Multiplexers and Demultiplexers I this lesso, you will lear about: Multiplexers ad Demultiplexers 1. Multiplexers 2. Combiatioal circuit implemetatio with multiplexers 3. Demultiplexers 4. Some examples Multiplexer A Multiplexer (see

More information

MARTINGALES AND A BASIC APPLICATION

MARTINGALES AND A BASIC APPLICATION MARTINGALES AND A BASIC APPLICATION TURNER SMITH Abstract. This paper will develop the measure-theoretic approach to probability i order to preset the defiitio of martigales. From there we will apply this

More information

Electrostatic solutions for better efficiency

Electrostatic solutions for better efficiency Electrostatic solutios for better efficiecy idustry for egieers, professioals ad techicias i developmet, productio ad istallatio. www.kerste.de/e Electrostatic solutios kerste has bee the leadig supplier

More information

A Balanced Scorecard

A Balanced Scorecard A Balaced Scorecard with VISION A Visio Iteratioal White Paper Visio Iteratioal A/S Aarhusgade 88, DK-2100 Copehage, Demark Phoe +45 35430086 Fax +45 35434646 www.balaced-scorecard.com 1 1. Itroductio

More information

Definition. A variable X that takes on values X 1, X 2, X 3,...X k with respective frequencies f 1, f 2, f 3,...f k has mean

Definition. A variable X that takes on values X 1, X 2, X 3,...X k with respective frequencies f 1, f 2, f 3,...f k has mean 1 Social Studies 201 October 13, 2004 Note: The examples i these otes may be differet tha used i class. However, the examples are similar ad the methods used are idetical to what was preseted i class.

More information

Entropy of bi-capacities

Entropy of bi-capacities Etropy of bi-capacities Iva Kojadiovic LINA CNRS FRE 2729 Site école polytechique de l uiv. de Nates Rue Christia Pauc 44306 Nates, Frace iva.kojadiovic@uiv-ates.fr Jea-Luc Marichal Applied Mathematics

More information

Mathematical goals. Starting points. Materials required. Time needed

Mathematical goals. Starting points. Materials required. Time needed Level A1 of challege: C A1 Mathematical goals Startig poits Materials required Time eeded Iterpretig algebraic expressios To help learers to: traslate betwee words, symbols, tables, ad area represetatios

More information

Systems Design Project: Indoor Location of Wireless Devices

Systems Design Project: Indoor Location of Wireless Devices Systems Desig Project: Idoor Locatio of Wireless Devices Prepared By: Bria Murphy Seior Systems Sciece ad Egieerig Washigto Uiversity i St. Louis Phoe: (805) 698-5295 Email: bcm1@cec.wustl.edu Supervised

More information

GCSE STATISTICS. 4) How to calculate the range: The difference between the biggest number and the smallest number.

GCSE STATISTICS. 4) How to calculate the range: The difference between the biggest number and the smallest number. GCSE STATISTICS You should kow: 1) How to draw a frequecy diagram: e.g. NUMBER TALLY FREQUENCY 1 3 5 ) How to draw a bar chart, a pictogram, ad a pie chart. 3) How to use averages: a) Mea - add up all

More information

Section 11.3: The Integral Test

Section 11.3: The Integral Test Sectio.3: The Itegral Test Most of the series we have looked at have either diverged or have coverged ad we have bee able to fid what they coverge to. I geeral however, the problem is much more difficult

More information

Chapter 7: Confidence Interval and Sample Size

Chapter 7: Confidence Interval and Sample Size Chapter 7: Cofidece Iterval ad Sample Size Learig Objectives Upo successful completio of Chapter 7, you will be able to: Fid the cofidece iterval for the mea, proportio, ad variace. Determie the miimum

More information

Convexity, Inequalities, and Norms

Convexity, Inequalities, and Norms Covexity, Iequalities, ad Norms Covex Fuctios You are probably familiar with the otio of cocavity of fuctios. Give a twicedifferetiable fuctio ϕ: R R, We say that ϕ is covex (or cocave up) if ϕ (x) 0 for

More information

On the Capacity of Hybrid Wireless Networks

On the Capacity of Hybrid Wireless Networks O the Capacity of Hybrid ireless Networks Beyua Liu,ZheLiu +,DoTowsley Departmet of Computer Sciece Uiversity of Massachusetts Amherst, MA 0002 + IBM T.J. atso Research Ceter P.O. Box 704 Yorktow Heights,

More information

where: T = number of years of cash flow in investment's life n = the year in which the cash flow X n i = IRR = the internal rate of return

where: T = number of years of cash flow in investment's life n = the year in which the cash flow X n i = IRR = the internal rate of return EVALUATING ALTERNATIVE CAPITAL INVESTMENT PROGRAMS By Ke D. Duft, Extesio Ecoomist I the March 98 issue of this publicatio we reviewed the procedure by which a capital ivestmet project was assessed. The

More information