RealTime Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations


 Rodger Small
 3 years ago
 Views:
Transcription
1 RealTime Computig Without Stable States: A New Framework for Neural Computatio Based o Perturbatios Wolfgag aass+, Thomas Natschläger+ & Hery arkram* + Istitute for Theoretical Computer Sciece, Techische Uiversität Graz; A8010 Graz, Austria * Brai id Istitute, Ecole Polytechique Federale de Lausae, CH1015 Lausae, Switzerlad Wolfgag aass & Thomas Natschlaeger Istitute for Theoretical Computer Sciece Techische Uiversitaet Graz Iffeldgasse 16b, A8010 Graz, Austria Tel: Fax: Hery arkram Brai id Istitute Ecole Polytechique Federale de Lausae CEEcubles, CH1015 Lausae, Switzerlad Tel: Fax: Address for Correspodece: Wolfgag aass 1
2 A key challege for eural modelig is to explai how a cotiuous stream of multimodal iput from a rapidly chagig eviromet ca be processed by stereotypical recurret circuits of itegrateadfire euros i realtime. We propose a ew computatioal model for realtime computig o timevaryig iput that provides a alterative to paradigms based o Turig machies or attractor eural etworks. It does ot require a taskdepedet costructio of eural circuits. Istead it is based o priciples of high dimesioal dyamical systems i combiatio with statistical learig theory, ad ca be implemeted o geeric evolved or foud recurret circuitry. It is show that the iheret trasiet dyamics of the highdimesioal dyamical system formed by a sufficietly large ad heterogeeous eural circuit may serve as uiversal aalog fadig memory. Readout euros ca lear to extract i realtime from the curret state of such recurret eural circuit iformatio about curret ad past iputs that may be eeded for diverse tasks. Stable iteral states are ot required for givig a stable output, sice trasiet iteral states ca be trasformed by readout euros ito stable target outputs due to the high dimesioality of the dyamical system. Our approach is based o a rigorous computatioal model, the liquid state machie, that ulike Turig machies, does ot require sequetial trasitios betwee welldefied discrete iteral states. It is supported, like the Turig machie, by rigorous mathematical results that predict uiversal computatioal power uder idealized coditios, but for the biologically more realistic sceario of realtime processig of timevaryig iputs. Our approach provides ew perspectives for the iterpretatio of eural codig, for the desig of experimets ad dataaalysis i europhysiology, ad for the solutio of problems i robotics ad eurotechology. 2
3 Itroductio Itricate topographically orgaized feedforward pathways project rapidly chagig spatiotemporal iformatio about the eviromet ito the eocortex. This iformatio is processed by extremely complex but surprisigly stereotypic microcircuits that ca perform a wide spectrum of tasks (Shepherd, 1988, Douglas et al., 1998, vo elcher et al., 2000). The microcircuit features that eable this seemigly uiversal computatioal power, is a mystery. Oe particular feature, the multiple recurret loops that form a immesely complicated etwork usig as may as 80% of all the syapses withi a fuctioal eocortical colum, has preseted a itractable problem both for computatioal models ispired by curret artificial computig machiery (Savage, 1998), ad for attractor eural etwork models. The difficulty to uderstad computatios withi recurret etworks of itegrateadfire euros comes from the fact that their dyamics takes o a life of its ow whe challeged with rapidly chagig iputs. This is particularly true for the very high dimesioal dyamical system formed by a eural microcircuit, whose compoets are highly heterogeeous ad where each euro ad each syapse adds degrees of freedom to the dyamics of the system. The most commo approach for modelig computig i recurret eural circuits has bee to try to take cotrol of their high dimesioal dyamics. ethods for cotrollig the dyamics of recurret eural etworks through adaptive mechaisms are reviewed i (Pearlmutter, 1995). So far oe has ot bee able to apply these to the case of etworks of spikig euros. Other approaches towards modelig computatio i biological eural systems are based o costructios of artificial eural etworks that simulate Turig machies or other models for digital computatio, see for example (Pollack, 1991), (Giles et al., 1992), (Siegelma et al., 1994), (Hyoetyiemi, 1996), (oore, 1998). Amog these there are models, such as dyamical recogizers, which are capable of realtime computig o olie iput (i discrete time). Noe of these approaches has bee demostrated to work for etworks of spikig euros, or ay more realistic models for eural microcircuits. It was show i (aass, 1996) that oe also ca costruct recurret circuits of spikig euros that ca simulate arbitrary Turig machies. But all of these approaches require sychroizatio of all euros by a cetral clock, a feature that appears to missig i eural microcircuits. I additio they require the costructio of particular recurret circuits, ad caot be implemeted by evolvig or adaptig a give circuit. Furthermore the results of (aass et al., 1999) o the impact of oise o the computatioal power of recurret eural etworks suggest that all these approaches break dow as soo as oe assumes that the uderlyig aalog computatioal uits are subject to Gaussia or other realistic oise distributios. Attractor eural etworks o the other had allow oise robust computatio, but their attractor ladscape is i geeral hard to cotrol, ad they eed to have a very large set of attractors i order to store saliet iformatio o past iputs (for example 1024 attractors i order to store 10 bits). I additio they are less suitable for realtime computig o rapidly varyig iput streams because of the time required for covergece to a attractor. Fially, oe of these approaches allows that several realtime computatios are carried out i parallel withi the same circuitry, which appears to be a geeric feature of eural microcircuits. I this article we aalyze the dyamics of eural microcircuits from the poit of view of a readout euro, whose task is to extract iformatio ad report results from a eural microcircuit to other circuits. A huma observer of the dyamics i a eural microcircuit would be lookig for clearly distict ad temporally stable features, such as covergece to attractors. We show that a readout euro, that receives iputs from hudreds or thousads of euros i a eural 3
4 microcircuit, ca lear to extract saliet iformatio from the high dimesioal trasiet states of the circuit, ad ca trasform trasiet circuit states ito stable readouts. I particular each readout ca lear to defie its ow otio of equivalece of dyamical states withi the eural microcircuit, ad ca the perform its task o ovel iputs. This uexpected fidig of readoutassiged equivalet states of a dyamical system explais how ivariat readout is possible despite the fact that the eural microcircuit may ever revisit the same state. Furthermore we show that multiple readout modules ca be traied to perform differet tasks o the same state trajectories of a recurret eural circuit, thereby eablig parallel realtime computig. We preset the mathematical framework for a computatioal model that does ot require covergece to stable iteral states or attractors (eve if they do occur), sice iformatio about past iputs is automatically captured i the perturbatios of a dyamical system, i.e. i the cotiuous trajectory of trasiet iteral states. Special cases of this mechaism were already reported i (Buoomao et al., 1995) ad (Domiey et al., 1995). Similar ideas have bee discovered idepedetly by Herbert Jaeger (Jaeger, 2001) i the cotext of artificial eural etworks. Computig without Attractors As a illustratio for our geeral approach towards realtime computig cosider a series of trasiet perturbatios caused i a excitable medium (see (Holde et al., 1991)), for example a liquid, by a sequece of exteral disturbaces ("iputs") such as wid, soud, or sequeces of pebbles dropped ito the liquid. Viewed as a attractor eural etwork, the liquid has oly oe attractor state the restig state ad may therefore seem useless for computatioal purposes. However, the perturbed state of the liquid, at ay momet i time, represets preset as well as past iputs, potetially providig the iformatio eeded for a aalysis of various dyamic aspects of the eviromet. I order for such a liquid to serve as a source of saliet iformatio about preset ad past stimuli without relyig o stable states, the perturbatios must be sesitive to salietly differet iputs but ochaotic. The maer i which perturbatios are formed ad maitaied would vary for differet types of liquids ad would determie how useful the perturbatios are for such retrograde aalysis. Limitatios o the computatioal capabilities of liquids are imposed by their timecostat for relaxatio, ad the strictly local iteractios ad homogeeity of the elemets of a liquid. Neural microcircuits, however, appear to be ideal liquids for computig o perturbatios because of the large diversity of their elemets, euros ad syapses (see (Gupta et al., 2000)), ad the large variety of mechaisms ad time costats characterizig their iteractios, ivolvig recurret coectios o multiple spatial scales ("loops withi loops"). The foudatio for our aalysis of computatios without stable states is a rigorous computatioal model: the liquid state machie. Two macroscopic properties emerge from our theoretical aalysis ad computer simulatios as ecessary ad sufficiet coditios for powerful realtime computig o perturbatios: a separatio property, SP, ad a approximatio property, AP. SP addresses the amout of separatio betwee the trajectories of iteral states of the system that are caused by two differet iput streams (i the case of a physical liquid, SP could reflect the differece betwee the wave patters resultig from differet sequeces of disturbaces). 4
5 AP addresses the resolutio ad recodig capabilities of the readout mechaisms  more precisely its capability to distiguish ad trasform differet iteral states of the liquid ito give target outputs (whereas SP depeds mostly o the complexity of the liquid, AP depeds mostly o the adaptability of the readout mechaism to the required task). Liquid State achies Like the Turig machie (Savage, 1998), the model of a liquid state machie (LS) is based o a rigorous mathematical framework that guaratees, uder idealized coditios, uiversal computatioal power. Turig machies, however, have uiversal computatioal power for offlie computatio o (static) discrete iputs, while LSs have i a very specific sese uiversal computatioal power for realtime computig with fadig memory o aalog fuctios i cotiuous time. The iput fuctio u ( ) ca be a cotiuous sequece of disturbaces, ad the target output ca be some chose fuctio y ( ) of time that provides a realtime aalysis of this sequece. I order for a machie to map iput fuctios of time u ( ) to output fuctios y( ) of time, we assume that it geerates, at every time t, a iteral liquid state x (t), which costitutes its curret respose to precedig perturbatios, i.e., to precedig iputs u (s) for s t (Figure 1). I cotrast to the fiite state of a fiite state machie (or fiite automato) this liquid state cosists of aalog values that may chage cotiuously over time. Whereas the state set ad the state trasitio fuctio of a fiite state machie is i geeral costructed for a Figure 1: A: Architecture of a LS. A fuctio of time (time series) u( ) is ijected as iput ito the L liquid filter, creatig at time t the liquid state x (t), which is trasformed by a memoryless readout map f to geerate a output y(t). specific task, the liquid states ad the trasitios betwee them eed ot be customized for a specific task. I a physical implemetatio this liquid state cosists of all iformatio about the curret iteral state of a dyamical system that is accessible to the readout modules. I 5
6 mathematical terms, this liquid state is simply the curret output of some operator or filter 1 that maps iput fuctios u ( ) oto fuctios x (t) : x ( t) = ( L u)( t). L I the followig we will refer to this filter L ofte as liquid filter, or liquid circuit if implemeted by a circuit. If it is implemeted by a eural circuit, we refer to the euros i that circuit as liquid euros. The secod compoet of a LS is a memoryless readout map f that trasforms, at every time t, the curret liquid state x (t) ito the output y( t) = f ( x ( t)). I cotrast to the liquid filter L, this readout map f is i geeral chose i a taskspecific maer (ad there may be may differet readout maps, that extract differet taskspecific iformatio i parallel from the curret output of L ). Note that i a fiite state machie there exists o aalogo to such taskspecific readout maps, sice there the iteral fiite states are already costructed i a taskspecific maer. Accordig to the precedig defiitio readout maps are i geeral memoryless 2. Hece all iformatio about iputs u (s) from precedig time poits s t that is eeded to produce a target output y(t) at time t has to be cotaied i the curret liquid state x (t). odels for computatio that have origiated i computer sciece store such iformatio about the past i stable states (for example i memory buffers or tapped delay lies). We argue, however, that this is ot ecessary sice large computatioal power o fuctios of time ca also be realized eve if all memory traces are cotiuously decayig. Istead of worryig about the code ad locatio where iformatio about past iputs is stored, ad how this iformatio decays, it is eough to address the separatio questio: For which later time poits t will ay two sigificatly differet iput fuctios of time u( ) ad v( ) cause sigificatly differet liquid states x u (t) ad (t). Good separatio capability, i combiatio x v 1 Fuctios F that map iput fuctios of time u ( ) o output fuctios y ( ) of time are usually called operators i mathematics, but are commoly referred to as filters i egieerig ad eurosciece. We use the term filter i the followig, ad we write ( Fu )( t) for the output of the filter F at time t whe F is applied to the iput fuctio u ( ). Formally, such filter F is a map from U ito ( R R ) k, where R R is the set of all realvalued fuctios of time, vectors cosistig of k such fuctios of time, U is some subset of vectors cosistig of fuctios of time i U. 2 The term "memoryless" refers to the fact that the readout map R R, ad ( R k R ) is the set of U is the set of f is ot required to retai ay memory of previous states x ( s), s < t, of the liquid. However, i a biological cotext, the readout map will i geeral be subject to plasticity, ad may also cotribute to the memory capability of the system. We do ot explore this issues i this article because the differetiatio ito a memoryless readout map ad a liquid that serves as a memory device is made for coceptual clarificatio, ad is ot essetial to the model. 6
7 with a adequate readout map f, allows us to discard the requiremet of storig bits "util further otice" i stable states of the computatioal system. Uiversal Computatioal Power of LSs for Time Varyig Iputs We say that a class of machies has uiversal power for computatios with fadig memory o fuctios of time if ay filter F, i.e., ay map from fuctios of time u ( ) to fuctios of time y ( ), that is time ivariat 3 ad has fadig memory 4, ca be approximated by machies from this class, to ay degree of precisio. Arguably, these filters F are approximated accordig to this defiitio iclude all maps from iput fuctios of time to output fuctios of time that a behavig orgaism might eed to compute. A mathematical theorem (see Appedix A) guaratees that LSs have this uiversal computatioal power regardless of specific structure or implemetatio, provided that two abstract properties are met: the class of basis filters from which the liquid filters L are composed satisfies the poitwise separatio property ad the class of fuctios from which the readout maps f are draw, satisfies the approximatio property. These two properties provide the mathematical basis for the separatio property SP ad the approximatio property AP that were previously discussed. Theorem 1 i Appedix A implies that there are o serious a priori limits for the computatioal power of LSs o cotiuous fuctios of time, ad thereby provides a theoretical foudatio for our approach towards modelig eural computatio. I particular, sice this theorem makes o specific requiremet regardig the exact ature or behaviour of the basis filters, as log as they satisfy the separatio property (for the iputs i questio), it provides theoretical support for employig istead of circuits that were costructed 3 A filter F is called time ivariat if ay temporal shift of the iput fuctio u ( ) by some amout t 0 causes a temporal shift of the output fuctio y = Fu by the same amout t 0, i.e., ( Fu t 0 )( t) = ( Fu)( t + t 0) for all t,t 0 R, where u t 0 ( t) : = u( t + t 0). Note that if U is closed uder R k temporal shifts, the a time ivariat filter F : U ( R ) ca be idetified uiquely by the values y ( 0) = ( Fu)(0) of its output fuctios y ( ) at time 0. 4 Fadig memory (Boyd at al., 1985) is a cotiuity property of filters F which demads that for ay iput fuctio u ) U ay other iput fuctios ( the output )(0) v ) U [ T,0]. Formally oe defies that (Fu ca be approximated by the outputs (Fv)(0) for ( that approximate ( ) F : U ) u o a sufficietly log time iterval R k ( R has fadig memory if for every u U ad every ε > 0 there exist δ > 0 ad T > 0 so that ( Fv )(0) ( Fu)(0) < ε for all v U with u ( t) v( t) <δ for all t [ T,0]. Iformally a filter F has fadig memory if the most sigificat bits of its curret output value (Fu)(0) deped just o the most sigificat bits of the values of its iput fuctio ( ) T,0 ito the past. Thus, i order to compute the most sigificat bits of (Fu)(0) it is ot ecessary to kow the precise value of the iput fuctio u (s) for ay time s, ad it is also ot ecessary to kow aythig about the values of u ( ) for more tha a fiite time iterval back ito the past. Note that a filter that has fadig memory is automatically causal. u from some fiite time widow [ ] 7
8 for a specific task, partially evolved or eve rather arbitrary foud computatioal modules for purposeful computatios. This feature highlights a importat differece to computatioal theories based o Turig machies or fiite state machies, which are ofte used as coceptual basis for modelig eural computatio. The mathematical theory of LSs ca also be exteded to cover computatio o spike u of the iput u( ) trais (discrete evets i cotiuous time) as iputs. Here the i th compoet ( ) is a fuctio that assumes oly the values 0 ad 1, with ( t) = 1 at time t. Thus ( ) u i if the i th precedig euro fires u is ot a cotiuous fuctio but a sequece of poit evets. Theorem 2 i i Appedix A provides a theoretical foudatio for approximatig ay biologically relevat computatio o spike trais by LSs. i Neural icrocircuits as Implemetatios of LSs I order to test the applicability of this coceptual framework to modelig computatio i eural microcircuits, we carried out computer simulatios where a geeric recurret circuit of itegrateadfire euros (see Appedix B for details) was employed as liquid filter. I other words: computer models for eural microcircuits were viewed as implemetatio of the liquid filter L of a LS. I order to test the theoretically predicted uiversal realtime computig capabilities of these eural implemetatios of LSs, we evaluated their performace o a wide variety of challegig bechmark tasks. The iput to the eural circuit was via oe or several iput spike trais, which diverged to iject curret ito 30% radomly chose liquid euros. The amplitudes of the iput syapses were chose from a Gaussia distributio, so that each euro i the liquid circuit received a slightly differet iput (a form of topographic ijectio). The liquid state of the eural microcircuit at time t was defied as all iformatio that a readout euro could extract at time t from the circuit, i.e. the output at time t of all the liquid euros represeted the curret liquid state of this istatiatio of a LS. ore precisely, sice the readout euros were modeled as I&F euros with a biologically realistic membrae time costat of 30 ms, the liquid state x (t) at time t cosisted of the vector of cotributios of all the liquid euros to the membrae potetial at time t of a geeric readout euro (with uit syaptic weights). athematically this liquid state x (t) ca be defied as the vector of output values at time t of liear filters with expoetial decay (time costat 30 ms) applied to the spike trais emitted by the liquid euros. Each readout map f was implemeted by a separate populatio P of itegrateadfire euros (referred to as "readout euros") that received iput from all the "liquid euros", but had o lateral or recurret coectios 5. The curret firig activity p(t) of the populatio P, that is the fractio of euros i P firig durig a time bi of 20ms, was iterpreted as the aalog output of f at time t (oe ofte refers to such represetatio of aalog values by the curret firig activity i a pool of euros as space rate codig). Theoretically the class of readout maps that ca be implemeted i this fashio satisfies the approximatio property AP (aass, 2000, Auer et al., 2001), ad is accordig to Theorem 1 i priciple sufficiet for approximatig arbitrary give fadig memory filters F. I cases where a readout with discrete values 1 ad 0 5 For coceptual purposes we separate the liquid ad readout elemets i this paper, although dual liquidreadout fuctios ca also be implemeted. 8
9 suffices, oe ca implemet a readout map eve by a sigle I&F euro that represets these discrete output values by firig/ofirig at time t. I cases where the target output cosists of slowly varyig aalog values, a sigle readout euro ca be traied to represet these values through its timevaryig firig rate. I ay case the readout euros ca be traied to perform a specific task by adjustig the stregths of syapses projected oto them from the liquid euros usig a perceptrolike local learig rule (Auer et al., 2001). The fial leared state of the readout euros eables them to take particular weighted sums of the curret outputs x (t) of the liquid euros ad geerate a respose f ( x (t) ) that approximates the target value y(t). As a first test of these eural implemetatios of LSs we evaluated the separatio property SP of computer models for eural microcircuits o spike trai iputs. A large set of pairs of Poisso spike trais u( ) ad v( ) were radomly geerated ad ijected (i separate trials) as iput to the recurret eural circuit. The resultig trajectories x ( ) ad Figure 2: Average distace of liquid states for two differet iput spike trais u ad v (give as iput to the eural circuit i separate trials, each time with a idepedetly chose radom iitial state of the eural circuit, see Appedix B) plotted as a fuctio of time t. The state distace icreases with the distace d(u,v) betwee the two iput spike trais u ad v. Plotted o the yaxis is the average value of x ( t) x ( t) u v, where. deotes the Euclidea orm, ad x u (t), (t) x v deote the liquid states at time t for iput spike trais u ad v. The plotted results for the values 0.1, 0.2, 0.4 of the iput differece d' represet the average over 200 radomly geerated pairs u ad v of spike trais such that d ' d( u, v) < Parameters of the liquid: 1 colum, degree of coectivity λ = 2 (see Appedix B for details). x v ( ) of liquid states of the recurret circuit were recorded for each of these timevaryig iputs u( ) ad v( ). The average distace x ( t) x ( t) betwee these liquid states was plotted i Figure 2 as a fuctio of the time t after the oset of the iput, for various fixed values of the distace d(u,v) 6 betwee the two spike trai iputs u ad v. These curves show that the distace betwee these liquids states is well above the oise level, i.e. above the average liquid state differeces caused by the same spike trai applied with two differet radomly chose iitial coditios of the circuit (idicated by the solid curve). Furthermore these curves show that the differece i liquid states is after the first 30 ms roughly proportioal to the distace betwee the correspodig iput 6 I order to defie the distace d ( u, v) betwee two spike trais u ad v we replaced each 2 spike by a Gaussia exp( ( t / τ) ) for τ = 5ms (to be precise, u ad v are covolved with the 2 Gaussia kerel exp( ( t / τ) ) ) ad defied d ( u, v) as the distace of the resultig two cotiuous fuctios i the L2orm (divided by the maximal legths 0.5 s of the spike trais u ad v). u u v 9
10 spike trais. Note i particular the absece of chaotic effects for these geeric eural microcircuit models with biologically realistic itermediate coectio legths. Explorig the Computatioal Power of odels for Neural icrocircuit As a first test of its computatioal power this simple geeric circuit was applied to a previously cosidered classificatio task (Hopfield & Brody, 2001), where spoke words were represeted by oisecorrupted spatiotemporal spike patters over a rather log time iterval (40chael spike patters over 0.5s). This classificatio task had bee solved i (Hopfield & Brody, 2001) by a etwork of euros desiged for this task (relyig o ukow mechaisms that could provide smooth decays of firig activity over loger time periods, ad apparetly requirig substatially larger etworks of I&F euros if fully implemeted with I&F euros). The architecture of that etwork, which had bee customized for this task, limited its classificatio power to spike trais cosistig of a sigle spike per chael. We foud that the same, but also a more geeral versio of this spatiotemporal patter recogitio task that allowed several spikes per iput chael, ca be solved by a geeric recurret circuit as described i the previous sectio. Furthermore the output of this etwork was available at ay time, ad was usually correct as soo as the liquid state of the eural circuit had absorbed eough iformatio about the iput (the iitial value of the correctess just reflects the iitial guess of the readout). Formally we defied the correctess of the eural readout at time s by the term 1 target output y(s) readout activity p(s), where the target output y(s) cosisted i this case of the costat values 1 or 0, depedig o the iput patter. Plotted i Fig. 3 is for ay time t durig the presetatio of the iput patters i additio to the correctess as a fuctio of t also the certaity of the output at time t, which is defied as the average correctess up to that time t. Whereas the etwork costructed by Hopfield ad Brody was costructed to be ivariat with regard to liear time warpig of iputs (provided that oly oe spike arrives i each chael), the readouts of the geeric recurret circuit that we cosidered could be traied to be ivariat with regard to a large class of differet types of oises. The results show i Fig. 3 are for a oise where each iput spike is moved idepedetly by a amout draw from a Gaussia distributio with mea 0 ad SD 32 ms. 10
11 Figure 3: Applicatio of a geeric recurret etwork of I&F euros modeled as LS to a more difficult versio of a wellstudied classificatio task (Hopfield & Brody, 2001). Five radomly draw patters (called zero, "oe ", "two",..), each cosistig of 40 parallel Poisso spike trais over 0.5s, were chose. Five readout modules, each cosistig of 50 itegrateadfire euros, were traied with 20 oisy versios of each iput patter to respod selectively to oisy versios of just oe of these patters (oise was ijected by radomly movig each spike by a amout draw idepedetly from a Gaussia distributio with mea 0 ad variace 32ms; i additio the iitial state of the liquid euros was chose radomly at the begiig of each trial). The resposes of the readout which had bee traied * to detect the patter "zero" is show for a ew, previously ot show, oisy versios of two of the iput patters. The correctess ad certaity (= average correctess so far) are show as fuctios of time from the oset of the stimulus at the bottom. The correctess is calculated as 1 p(t) y(t) where p(t) is the ormalized firig activity i the readout pool (ormalized to the rage [0 1]; 1 correspodig to a activity of 180Hz; biwidth 20ms) ad y(t) is the target output. (Correctess starts at a level of 0 for patter zero where this readout pool is supposed to become active, ad at a value of 1 for patter oe, because the readout pool starts i a iactive state). I cotrast to most circuits of spikig euros that have bee costructed for specific computatioal task, the spike trais of liquid ad readout euros show i this figure look rather realistic. * The familiar deltarule was applied or ot applied to each readout euro, depedig o whether the curret firig activity i the readout pool was too high, too low, or about right, thus requirig at most two bits of global commuicatio. The precise versio of the learig rule was the pdelta rule that is discussed i Auer et al., (2001). 11
12 Givig a costat output for a timevaryig liquid state (caused by a timevaryig iput) is a serious challege for a LS, sice it caot rely o attractor states, ad the memoryless readout has to trasform the trasiet ad cotiuously chagig states of the liquid ito a stable output (see the discussio below ad Fig. 9 for details). I order to explore the limits of this simple eural implemetatio of a LS for computig o timevaryig iput, we chose aother classificatio task where all iformatio of the iput is cotaied i its temporal evolutio, more precisely i the iterspike itervals of a sigle iput spike trai. I this test, 8 radomly geerated Poisso spike trais over 250 ms, or equivaletly 2 Poisso spike trais over 1000 ms partitioed Figure 4: Evaluatig the fadig memory of a geeric eural microcircuit: the task. I this more challegig classificatio task all spike trais are of legth 1000 ms ad cosist of 4 segmets of legth 250 ms each. For each segmet 2 templates were geerated radomly (Poisso spike trai with a frequecy of 20 Hz); see upper traces. The actual iput spike trais of legth 1000 ms used for traiig ad testig were geerated by choosig for each segmet oe of the two associated templates, ad the geeratig a oisy versio by movig each spike by a amout draw from a Gaussia distributio with mea 0 ad a SD that we refer to as jitter (see lower trace for a visualizatio of the jitter with a SD of 4 ms). The task is to output with 4 differet readouts at time t = 1000 ms for each of the precedig 4 iput segmets the umber of the template from which the correspodig segmet of the iput was geerated. Results are summarized i Figures 5 ad 6. ito 4 segmets each (see top of Figure 4), were chose as template patters. Other spike trais over 1000 ms were geerated by choosig for each 250 ms segmet oe of the two templates for this segmet, ad by jitterig each spike i the templates (more precisely: each spike was moved by a amout draw from a Gaussia distributio with mea 0 ad a SD that we refer to as jitter, see bottom of Figure 4). A typical spike trai geerated i this way is show i the middle of Figure 4. Because of the oisy dislocatio of spikes it was impossible to recogize a specific template from a sigle iterspike iterval (ad there were o spatial cues cotaied i this sigle chael iput). Istead, a patter formed by several iterspike itervals had to be recogized ad classified retrospectively. Furthermore readouts were ot oly traied to classify at time t = 1000 ms (i.e., at after the iput spike trai had etered the circuit) the template from which the last 250 ms segmet of this iput spike trai had bee geerated, but other readouts were traied to classify simultaeously also the templates from which precedig segmets of the iput (which had etered the circuit several hudred ms earlier) had bee geerated. Obviously 12
13 the latter classificatio task is substatially more demadig, sice the correspodig earlier segmets of the iput spike trai may have left a clear trace i the curret firig activity of the recurret circuit just after they had etered the circuit, but this trace was subsequetly overwritte by the ext segmets of the iput spike trai (which had o correlatio with the choice of the earlier segmets). Altogether there were i this experimet 4 readouts f 1 to f 4, where f i had bee traied to classify at time t = 1000 ms the ith idepedetly chose 250 ms segmet of the precedig iput spike trai. The performace of the LS, with a geeric recurret etwork of 135 I&F euros as liquid filter (see Appedix B), was evaluated after traiig of the readout pools o iputs from the same distributio (for jitter = 4 ms), but with a example that the LS had ot see before. The accuracy of the 4 readouts is plotted i pael A of Figure 5. It demostrates the fadig memory of a geeric recurret circuit of I&F euros, where iformatio about iputs that occurred several hudred ms ago ca be recovered eve after that iput segmet was subsequetly overwritte. Sice readout euros (ad euros withi the liquid circuit) were modeled with a realistic time costat of just 30 ms, the questio arises where this iformatio about earlier iputs had bee stored for Figure 5: Evaluatig the fadig memory of a geeric eural microcircuit: results. 4 readout modules f 1 to f 4, each cosistig of a sigle perceptro, were traied for their task by liear regressio. The readout module f i was traied to output 1 at time t=1000 ms if the ith segmet of the previously preseted iput spike trai had bee costructed from the correspodig template 1, ad to output 0 at time t=1000 ms otherwise. Correctess (percetage of correct classificatio o a idepedet set of 500 iputs ot used for traiig) is calculated as average over 50 trials. I each trial ew Poisso spike trais were draw as templates, a ew radomly coected circuit was costructed (1 colum, λ=2; see Appedix B), ad the readout modules f 1 to f 4 were traied with1000 traiig examples geerated by the distributio described i Figure 4. A: Average correctess of the 4 readouts for ovel test iputs draw from the same distributio. B: Firig activity i the liquid circuit (time iterval [0.5 s, 0.8 s]) for a typical iput spike trai. C: Results of a cotrol experimet where all dyamic syapses i the liquid circuit had bee replaced by static syapses (the mea values of the syaptic stregths were uiformly rescaled so that the average liquid activity is approximately the same as for dyamic syapses). The liquid state of this circuit cotaied substatially less iformatio about earlier iput segmets. D: Firig activity i the liquid circuit with static syapses used for the classificatio results reported i pael C. The circuit respose to each of the 4 iput spikes that etered the circuit durig the observed time iterval [0.5 s, 0.8 s] is quite stereotypical without dyamic syapses (except for the secod iput spike that arrives just 20 ms after the first oe). I cotrast the firig respose of the liquid circuit with dyamic syapses (pael B) is differet for each of the 4 iput spikes, showig that dyamic syapses edow these circuits with the capability to process ew iput differetly depedig o the cotext set by precedig iput, eve if that precedig iput occurred several hudred ms before. 13
14 several hudred ms. As a cotrol we repeated the same experimet with a liquid circuit where the dyamic syapses had bee replaced by static syapses (with syaptic weights that achieved about the same level of firig activity as the circuit with dyamic syapses). Pael C of Fig. 5 shows that this results i a sigificat loss i performace for the classificatio of all except for the last iput segmet. A possible explaatio is provided by the raster plots of firig activity i the liquid circuit with (pael B) ad without dyamic syapses (pael D), show here with high temporal resolutio. I the circuit with dyamic syapses the recurret activity differs for each of the 4 spikes that etered the circuit durig the time period show, demostratig that each ew spike is processed by the circuit i a idividual maer that depeds o the cotext defied by precedig iput spikes. I cotrast, the firig respose is very stereotypical for the same 4 iput spikes i the circuit without dyamic syapses, except for the respose to the secod spike that arrives withi 20 ms of the first oe (see the period betwee 500 ad 600 ms i pael D). This idicates Figure 6: Average correctess depeds o the parameter λ that cotrols the distributio of radom coectios withi the liquid circuit. Plotted is the average correctess (at time t=1000 ms, calculated as average over 50 trials as i Figure 5; same umber of traiig ad test examples) of the readout module f 3 (which is traied to classify retroactively the secod to last segmet of the precedig spike trai) as a fuctio of λ. The bad performace for λ=0 (o recurret coectios withi the circuit) shows that recurret coectios are essetial for achievig a satisfactory separatio property i eural microcircuits. Too large values of λ also decrease the performace because they support a chaotic respose. that the short term dyamics of syapses may play a essetial role i the itegratio of iformatio for realtime processig i eural microcircuits. Figure 6 examies aother aspect of eural microcircuits that appears to be importat for their separatio property: the statistical distributio of coectio legths withi the recurret circuit. Six types of liquid circuits, each cosistig of 135 I&F euros but with differet values of the parameter λ which regulated the average umber of coectios ad the average spatial legth of coectios (see Appedix B), were traied ad evaluated accordig to the same protocol ad for the same task as i Fig. 5. Show i Fig. 6 is for each of these 6 types of liquid circuits the average correctess of the readout f 3 o ovel iputs, after it had bee traied to classify the secod to last segmet of the iput spike trai. The performace was fairly low for circuits without recurret coectios (λ = 0). It also was fairly low for recurret circuits with large values of λ, whose largely legthidepedet distributio of coectios homogeized the microcircuit ad facilitated chaotic behavior. Hece for this classificatio task the ideal liquid circuit is a microcircuit that has i additio to local coectios to eighborig euros also a few lograge coectios, thereby iterpolatig betwee the customarily cosidered extremes of strictly total coectivity (like i a cellular automato) o oe had, ad the localityigorig global coectivity of a Hopfield et o the other had. 14
15 The performace results of eural implemetatios of LSs that were reported i this sectio should ot be viewed as absolute data o the computatioal power of recurret eural circuits. Rather the geeral theory suggests that their computatioal power icreases with ay improvemet i their separatio or approximatio property. Sice the approximatio property AP was already close to optimal for these etworks (icreasig the umber of euros i the readout module did ot icrease the performace sigificatly; ot show), the primary limitatio i performace lay i the separatio property SP. Ituitively it is clear that the liquid circuit eeds to be sufficietly complex to hold the details required for the particular task, but should reduce iformatio that is ot relevat to the task (for example spike time jitter). SP ca be egieered i may ways such as icorporatig euro diversity, implemetig specific syaptic architectures, alterig microcircuit coectivity, or simply recruitig more colums. The last optio is of particular iterest because it is ot available i most computatioal models. It will be explored i the ext sectio. Addig Computatioal Power A iterestig structural differece betwee eural systems ad our curret geeratio of artificial computig machiery is that the computatioal power of eural systems ca apparetly be elarged by recruitig more circuitry (without the eed to rewire old or ew circuits). We explored the cosequeces of recruitig additioal colums for eural implemetatios of LSs (see pael B of Fig. 7), ad compared it with the optio of just addig further coectios to the Figure 7: Separatio property ad performace of liquid circuits with larger umbers of coectios or euros. A ad B: Schematic drawigs of LSs cosistig of oe colum (A) ad four colums (B). Each colum cosists of = 135 I&F euros. C: Separatio property depeds o the structure of the liquid. Average state distace (at time t = 100 ms) calculated as described i Figure 2. A colum with high iteral coectivity (high λ) achieves higher separatio as a sigle colum with lower coectivity, but teds to chaotic behavior where it becomes equally sesitive to small ad large iput differeces d(u,v). O the other had the characteristic curve for a liquid cosistig of 4 colums with small λ is lower for values of d(u,v) lyig i the rage of jittered versios u ad v of the same spike trai patter (d(u,v) 0.1 for jitter 8 ms) ad higher for values of d(u,v) i the rage typical for spike trais u ad v from differet classes (mea: 0.22). D: Evaluatio of the same three types of liquid circuits for oise robust classificatio. Plotted is the average performace for the same task as i Fig. 6, but for various values of the jitter i iput spike times. Several colums (ot itercoected) with low iteral coectivity yield a better performig implemetatio of a LS for this computatioal task, as predicted by the aalysis of their separatio property. 15
16 primary oecolumliquid that we used so far (135 I&F euros with λ = 2, see pael A of Fig. 7). Pael C of Fig. 7 demostrates that the recruitmet of additioal colums icreases the separatio property of the liquid circuit i a desirable maer, where the distace betwee subsequet liquid states (always recorded at time t = 1000 ms i this experimet) is proportioal to the distace betwee the spike trai iputs that had previously etered the liquid circuit (spike trai distace measured i the same way as for Fig. 2). I cotrast the additio of more coectios to a sigle colum (λ = 8, see Appedix B) also icreases the separatio betwee subsequet liquid states, but i a quasichaotic maer where small iput distaces cause about the same distaces betwee subsequet liquid states as small iput differeces. I particular the subsequet liquid state distace is about equally large for two jittered versios of the iput spike trai state (yieldig typically a value of d(u,v) aroud 0.1) as for sigificatly differet iput spike trais that require differet outputs of the readouts. Thus improvig SP by alterig the itrisic microcircuitry of a sigle colum icreases sesitivity for the task, but also icreases sesitivity to oise. The performace of these differet types of liquid circuits for the same classificatio task as i Fig. 6 is cosistet with this aalysis of their characteristic separatio property. Show i pael D of Fig. 7 is their performace for various values of the spike time jitter i the iput spike trais. The optimizatio of SP for a specific distributio of iputs ad a specific group of readout modules is likely to arrive at a specific balace betwee the itrisic complexity of the microcircuitry ad the umber of repeatig colums. Parallel Computig i RealTime o Novel Iputs Sice the liquid of the LS does ot have to be traied for a particular task, it supports parallel computig i realtime. This was demostrated by a test i which multiple spike trais were ijected ito the liquid ad multiple readout euros were traied to perform differet tasks i parallel. We added 6 readout modules to a liquid cosistig of 2 colums with differet values of λ 7. Each of the 6 readout modules was traied idepedetly for a completely differet olie task that required a output value at ay time t. We focused here o tasks that require diverse ad rapidly chagig aalog output resposes y(t). Figure 8 shows that after traiig each of these 6 tasks ca be performed i realtime with high accuracy. The performace show is for a ovel iput that was ot draw from the same distributio as the traiig examples, ad differs i several aspects from the traiig examples (thereby demostratig the possibility of extrageeralizatio i eural microcircuits, due to their iheret bias, that goes beyod the usual defiitio of geeralizatio i statistical learig theory). ReadoutAssiged Equivalet States of a Dyamical System Realtime computatio o ovel iputs implies that the readout must be able to geerate a ivariat or appropriately scaled respose for ay iput eve though the liquid state may ever repeat. Ideed, Figure 3 showed already that the dyamics of readout pools ca become quite idepedet from the dyamics of the liquid eve though the liquid euros are the oly source 7 I order to combie high sesitivity with good geeralizatio performace we chose here a liquid cosistig of two colums as before, oe with λ=2, the other with λ=8 ad the iterval [ ] for the uiform distributio of the ospecific backgroud curret I b. 16
17 of iput. To examie the uderlyig mechaism for this relatively idepedet readout respose, we reexamied the readout pool from Figure 3. Whereas the firig activity withi the liquid circuit was highly dyamic, the firig activity i the readout pool was almost costat after traiig. The stability of the readout respose does ot simply come about because the readout oly samples a few uusual liquid euros as show by the distributio of syaptic weights oto a sample readout euro (Figure 9F). Sice the syaptic weights do ot chage after learig, this idicates that the readout euros have leared to defie a otio of equivalece for dyamic states of the liquid. Ideed, equivalece classes are a ievitable cosequece of collapsig the high dimesioal space of liquid states ito a sigle dimesio, but what is surprisig is that the equivalece classes are meaigful i terms of the task, allowig ivariat ad appropriately scaled readout resposes ad therefore realtime computatio o ovel iputs. Furthermore, while the iput rate may cotai saliet iformatio that is costat for a particular readout elemet, it may ot be for aother (see for example Fig. 8), idicatig that equivalece classes ad dyamic stability exist purely from the perspective of the readout elemets. 17
18 Figure 8: ultitaskig i realtime. 4 iput spike trais of legth 2 s (show at the top) are ijected ito a liquid module cosistig of 2 colums (radomly costructed with the same parameters; see Appedix B), which is coected to multiple readout modules. Each readout module is traied to extract iformatio for a differet realtime computig task. The target fuctios are plotted as dashed lie, ad populatio respose of the correspodig readout module as solid lie. The tasks assiged to the 6 readout modules were the followig: Represet the sum of rates: at time t, output the sum of firig rates of all 4 iput spike trais withi the last 30ms. Represet the itegral of the sum of rates: at time t, output the total activity i all 4 iputs itegrated over the last 200ms. Patter detectio: output a high value if a specific spatio temporal spike patter appears. Represet a switch i spatial distributio of rates: output a high value if a specific iput patter occurs where the rate of iput spike trais 1 ad 2 goes up ad simultaeously the rate of iput spike trais 3 ad 4 goes dow, otherwise remai low. Represet the firig correlatio: at time t, output the umber of spike coicideces (ormalized ito the rage [0 1]) durig the last 75 ms for iputs 1 ad 3 ad separately for iputs 1 ad 2. Target readout values are plotted as dashed lies, actual outputs of the readout modules as solid lies, all i the same time scale as the 4 spike trais show at the top that eter the liquid circuit durig this 2 s time iterval. Results show are for a ovel iput that was ot draw from the same distributio as the traiig examples. 150 traiig examples were draw radomly from the followig distributio. Each iput spike trai was a idepedetly draw Possio spike trai with a time varyig rate of r(t) = A+B si (2 π f t + α). The parameters A, B, ad f where draw radomly from the followig itervals (the phase was fixed at α=0 deg): A [0Hz, 30Hz] ad [70Hz, 100Hz], B [0Hz, 30Hz] ad [70Hz, 100Hz], f [0.5Hz, 1Hz] ad [3Hz, 5Hz]. O this backgroud activity 4 differet patters had bee superimposed (always i the same order durig traiig): rate switch to iputs 1 ad 3, a burst patter, rate switch to iputs 1 ad 2, ad fially a spatio temporal spike patter. The results show are for a test iput that could ot be geerated by the same distributio as the traiig examples, because its base level (A=50Hz), as well as the amplitude (B=50Hz), frequecy (f=2hz) ad phase (α=180 deg) of the uderlyig time varyig firig rate of the Poisso iput spike trais were chose to lie i the middle of the gaps betwee the two itervals that were used for these parameters durig traiig. Furthermore the spatiotemporal patters (a burst patter, rate switch to iputs 1 ad 3, ad rate switch to iputs 1 ad 2), that were superimposed to achieve more iput variatio withi the observed 2 s, ever occured i this order ad at these time poits for ay traiig iput. Hece the accurate performace for this ovel iput demostrates substatial geeralizatio capabilities of the readouts after traiig. 18
19 Discussio We itroduce the liquid state machie, a ew paradigm for realtime computig o timevaryig iput streams. I cotrast to most computatioal models it does ot require the costructio of a circuit or program for a specific computatioal task. Rather, it relies o priciples of highdimesioal dyamical systems ad learig theory that allow it to adapt uspecific evolved or foud recurret circuitry for a give computatioal task. Sice oly the readouts, ot the recurret circuit itself, have to be adapted for specific computatioal tasks, the same recurret circuit ca support completely differet realtime computatios i parallel. The uderlyig abstract computatioal model of a liquid state machie (LS) emphasizes the importace of perturbatios i dyamical systems for realtime computig, sice eve without stable states or attractors the separatio property ad the approximatio property may edow a dyamical system with virtually ulimited computatioal power o timevaryig iputs. I particular we have demostrated the computatioal uiversality of geeric recurret circuits of itegrateadfire euros (eve with quite arbitrary coectio structure), if viewed as special cases of LSs. Apparetly this is the first stable ad geerally applicable method for usig geeric recurret etworks of itegrateadfire euros to carry out a wide family of complex realtime computatios o spike trais as iputs. Hece this approach provides a platform for explorig the computatioal role of specific aspects of biological eural microcircuits. The computer simulatios reported i this article provide possible explaatios ot oly for the computatioal role of the highly recurret coectivity structure of eural circuits, but also for their characteristic distributio of coectio legths, which places their coectivity structure betwee the extremes of strictly local coectivity (cellular automata or coupled map lattices) ad uiform global coectivity (Hopfield ets) that are usually addressed i theoretical studies. Furthermore our computer simulatios suggest a importat computatioal role of dyamic syapses for realtime computig o timevaryig iputs. Fially, we reveal a most uexpected ad remarkable priciple that readout elemets ca establish their ow equivalece relatioships o highdimesioal trasiet states of a dyamical system, makig it possible to geerate stable ad appropriately scaled output resposes eve if the iteral state ever coverges to a attractor state. I cotrast to virtually all computatioal models from computer sciece or artificial eural etworks, this computatioal model is ehaced rather tha hampered by the presece of diverse computatioal uits. Hece it may also provide isight ito the computatioal role of the complexity ad diversity of euros ad syapses (see for example (Gupta et al., 2000)). While there are may plausible models for spatial aspects of eural computatio, a biologically realistic framework for modelig temporal aspects of eural computatio has bee missig. I cotrast to models ispired by computer sciece, the liquid state machie does ot try to reduce these temporal aspects to trasitios betwee stable states or limit cycles, ad it does ot require delay lies or buffers. Istead it proposes that the trajectory of iteral states of a recurret eural circuit provides a raw, ubiased, ad uiversal source of temporally itegrated iformatio, from which specific readout elemets ca extract specific iformatio about past iputs for their idividual task. Hece the otorious trialtotrial stimulus respose variatios i sigle ad populatios of euros observed experimetally, may reflect a accumulatio of iformatio from previous iputs i the trajectory of iteral states, rather tha oise (see also (Arieli et al., 1996)). This would imply that averagig over trials or biig, peels out most of the iformatio processed by recurret microcircuits ad leaves mostly topographic iformatio. 19
20 This approach also offers ew ideas for models of the computatioal orgaisatio of cogitio. It suggests that it may ot be ecessary to scatter all iformatio about sesory iput by recodig it through feedforward processig as output vector of a esemble of feature Figure 9: Readout assiged equivalet states of a dyamical system. A LS (liquid circuit as i Figure 3) was traied for the classificatio task as described i Figure 3. Results show are for a ovel test iput (draw from the same distributio as the traiig examples). A: The test iput cosists of 40 Poisso spike trais, each with a costat rate of 5 Hz. B: Raster plot of the 135 liquid euros i respose to this iput. Note the large variety of liquid states that arise durig this time period. C: Populatio rate of the liquid (bisize 20 ms). Note that this populatio rate chages quit a bit over time. D: Readout respose (solid lie) ad target respose (dashed lie). The target respose had a costat value of 1 for this iput. The output of the traied readout module is also almost costat for this test example (except for the begiig), although its iput, the liquid states of the recurret circuit, varied quit a bit durig this time period. F: Weight distributio of a sigle readout euro. detectors with fixed receptive fields (thereby creatig the "bidig problem"). It proposes that at the same time more global iformatio about precedig iputs ca be preserved i the trajectories of very high dimesioal dyamical systems, from which multiple readout modules extract ad combie the iformatio eeded for their specific tasks. This approach is evertheless compatible with experimetal data that cofirm the existece of special maps of feature detectors. These 20
Modified Line Search Method for Global Optimization
Modified Lie Search Method for Global Optimizatio Cria Grosa ad Ajith Abraham Ceter of Excellece for Quatifiable Quality of Service Norwegia Uiversity of Sciece ad Techology Trodheim, Norway {cria, ajith}@q2s.tu.o
More informationI. Chisquared Distributions
1 M 358K Supplemet to Chapter 23: CHISQUARED DISTRIBUTIONS, TDISTRIBUTIONS, AND DEGREES OF FREEDOM To uderstad tdistributios, we first eed to look at aother family of distributios, the chisquared distributios.
More informationIn nite Sequences. Dr. Philippe B. Laval Kennesaw State University. October 9, 2008
I ite Sequeces Dr. Philippe B. Laval Keesaw State Uiversity October 9, 2008 Abstract This had out is a itroductio to i ite sequeces. mai de itios ad presets some elemetary results. It gives the I ite Sequeces
More informationCase Study. Normal and t Distributions. Density Plot. Normal Distributions
Case Study Normal ad t Distributios Bret Halo ad Bret Larget Departmet of Statistics Uiversity of Wiscosi Madiso October 11 13, 2011 Case Study Body temperature varies withi idividuals over time (it ca
More informationNonlife insurance mathematics. Nils F. Haavardsson, University of Oslo and DNB Skadeforsikring
Nolife isurace mathematics Nils F. Haavardsso, Uiversity of Oslo ad DNB Skadeforsikrig Mai issues so far Why does isurace work? How is risk premium defied ad why is it importat? How ca claim frequecy
More informationProperties of MLE: consistency, asymptotic normality. Fisher information.
Lecture 3 Properties of MLE: cosistecy, asymptotic ormality. Fisher iformatio. I this sectio we will try to uderstad why MLEs are good. Let us recall two facts from probability that we be used ofte throughout
More informationConfidence Intervals for One Mean with Tolerance Probability
Chapter 421 Cofidece Itervals for Oe Mea with Tolerace Probability Itroductio This procedure calculates the sample size ecessary to achieve a specified distace from the mea to the cofidece limit(s) with
More information3. Covariance and Correlation
Virtual Laboratories > 3. Expected Value > 1 2 3 4 5 6 3. Covariace ad Correlatio Recall that by takig the expected value of various trasformatios of a radom variable, we ca measure may iterestig characteristics
More informationLECTURE 13: Crossvalidation
LECTURE 3: Crossvalidatio Resampli methods Cross Validatio Bootstrap Bias ad variace estimatio with the Bootstrap Threeway data partitioi Itroductio to Patter Aalysis Ricardo GutierrezOsua Texas A&M
More information7. Sample Covariance and Correlation
1 of 8 7/16/2009 6:06 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 7. Sample Covariace ad Correlatio The Bivariate Model Suppose agai that we have a basic radom experimet, ad that X ad Y
More information3. Continuous Random Variables
Statistics ad probability: 31 3. Cotiuous Radom Variables A cotiuous radom variable is a radom variable which ca take values measured o a cotiuous scale e.g. weights, stregths, times or legths. For ay
More informationConfidence Intervals for One Mean
Chapter 420 Cofidece Itervals for Oe Mea Itroductio This routie calculates the sample size ecessary to achieve a specified distace from the mea to the cofidece limit(s) at a stated cofidece level for a
More informationDefinition. Definition. 72 Estimating a Population Proportion. Definition. Definition
7 stimatig a Populatio Proportio I this sectio we preset methods for usig a sample proportio to estimate the value of a populatio proportio. The sample proportio is the best poit estimate of the populatio
More informationVladimir N. Burkov, Dmitri A. Novikov MODELS AND METHODS OF MULTIPROJECTS MANAGEMENT
Keywords: project maagemet, resource allocatio, etwork plaig Vladimir N Burkov, Dmitri A Novikov MODELS AND METHODS OF MULTIPROJECTS MANAGEMENT The paper deals with the problems of resource allocatio betwee
More information{{1}, {2, 4}, {3}} {{1, 3, 4}, {2}} {{1}, {2}, {3, 4}} 5.4 Stirling Numbers
. Stirlig Numbers Whe coutig various types of fuctios from., we quicly discovered that eumeratig the umber of oto fuctios was a difficult problem. For a domai of five elemets ad a rage of four elemets,
More informationNPTEL STRUCTURAL RELIABILITY
NPTEL Course O STRUCTURAL RELIABILITY Module # 0 Lecture 1 Course Format: Web Istructor: Dr. Aruasis Chakraborty Departmet of Civil Egieerig Idia Istitute of Techology Guwahati 1. Lecture 01: Basic Statistics
More informationAnalyzing Longitudinal Data from Complex Surveys Using SUDAAN
Aalyzig Logitudial Data from Complex Surveys Usig SUDAAN Darryl Creel Statistics ad Epidemiology, RTI Iteratioal, 312 Trotter Farm Drive, Rockville, MD, 20850 Abstract SUDAAN: Software for the Statistical
More informationStandard Errors and Confidence Intervals
Stadard Errors ad Cofidece Itervals Itroductio I the documet Data Descriptio, Populatios ad the Normal Distributio a sample had bee obtaied from the populatio of heights of 5yearold boys. If we assume
More informationCHAPTER 3 DIGITAL CODING OF SIGNALS
CHAPTER 3 DIGITAL CODING OF SIGNALS Computers are ofte used to automate the recordig of measuremets. The trasducers ad sigal coditioig circuits produce a voltage sigal that is proportioal to a quatity
More informationChapter 6: Variance, the law of large numbers and the MonteCarlo method
Chapter 6: Variace, the law of large umbers ad the MoteCarlo method Expected value, variace, ad Chebyshev iequality. If X is a radom variable recall that the expected value of X, E[X] is the average value
More informationHypothesis testing. Null and alternative hypotheses
Hypothesis testig Aother importat use of samplig distributios is to test hypotheses about populatio parameters, e.g. mea, proportio, regressio coefficiets, etc. For example, it is possible to stipulate
More information1 Computing the Standard Deviation of Sample Means
Computig the Stadard Deviatio of Sample Meas Quality cotrol charts are based o sample meas ot o idividual values withi a sample. A sample is a group of items, which are cosidered all together for our aalysis.
More informationHypothesis Tests Applied to Means
The Samplig Distributio of the Mea Hypothesis Tests Applied to Meas Recall that the samplig distributio of the mea is the distributio of sample meas that would be obtaied from a particular populatio (with
More informationDepartment of Computer Science, University of Otago
Departmet of Computer Sciece, Uiversity of Otago Techical Report OUCS200609 Permutatios Cotaiig May Patters Authors: M.H. Albert Departmet of Computer Sciece, Uiversity of Otago Micah Colema, Rya Fly
More informationOverview of some probability distributions.
Lecture Overview of some probability distributios. I this lecture we will review several commo distributios that will be used ofte throughtout the class. Each distributio is usually described by its probability
More informationG r a d e. 5 M a t h e M a t i c s. Patterns and relations
G r a d e 5 M a t h e M a t i c s Patters ad relatios Grade 5: Patters ad Relatios (Patters) (5.PR.1) Edurig Uderstadigs: Number patters ad relatioships ca be represeted usig variables. Geeral Outcome:
More informationNormal Distribution.
Normal Distributio www.icrf.l Normal distributio I probability theory, the ormal or Gaussia distributio, is a cotiuous probability distributio that is ofte used as a first approimatio to describe realvalued
More informationCenter, Spread, and Shape in Inference: Claims, Caveats, and Insights
Ceter, Spread, ad Shape i Iferece: Claims, Caveats, ad Isights Dr. Nacy Pfeig (Uiversity of Pittsburgh) AMATYC November 2008 Prelimiary Activities 1. I would like to produce a iterval estimate for the
More informationB1. Fourier Analysis of Discrete Time Signals
B. Fourier Aalysis of Discrete Time Sigals Objectives Itroduce discrete time periodic sigals Defie the Discrete Fourier Series (DFS) expasio of periodic sigals Defie the Discrete Fourier Trasform (DFT)
More informationChair for Network Architectures and Services Institute of Informatics TU München Prof. Carle. Network Security. Chapter 2 Basics
Chair for Network Architectures ad Services Istitute of Iformatics TU Müche Prof. Carle Network Security Chapter 2 Basics 2.4 Radom Number Geeratio for Cryptographic Protocols Motivatio It is crucial to
More informationTrading the randomness  Designing an optimal trading strategy under a drifted random walk price model
Tradig the radomess  Desigig a optimal tradig strategy uder a drifted radom walk price model Yuao Wu Math 20 Project Paper Professor Zachary Hamaker Abstract: I this paper the author iteds to explore
More informationGrade 7. Strand: Number Specific Learning Outcomes It is expected that students will:
Strad: Number Specific Learig Outcomes It is expected that studets will: 7.N.1. Determie ad explai why a umber is divisible by 2, 3, 4, 5, 6, 8, 9, or 10, ad why a umber caot be divided by 0. [C, R] [C]
More information5: Introduction to Estimation
5: Itroductio to Estimatio Cotets Acroyms ad symbols... 1 Statistical iferece... Estimatig µ with cofidece... 3 Samplig distributio of the mea... 3 Cofidece Iterval for μ whe σ is kow before had... 4 Sample
More informationChapter 7 Methods of Finding Estimators
Chapter 7 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 011 Chapter 7 Methods of Fidig Estimators Sectio 7.1 Itroductio Defiitio 7.1.1 A poit estimator is ay fuctio W( X) W( X1, X,, X ) of
More information1 Correlation and Regression Analysis
1 Correlatio ad Regressio Aalysis I this sectio we will be ivestigatig the relatioship betwee two cotiuous variable, such as height ad weight, the cocetratio of a ijected drug ad heart rate, or the cosumptio
More informationLesson 17 Pearson s Correlation Coefficient
Outlie Measures of Relatioships Pearso s Correlatio Coefficiet (r) types of data scatter plots measure of directio measure of stregth Computatio covariatio of X ad Y uique variatio i X ad Y measurig
More information5 Boolean Decision Trees (February 11)
5 Boolea Decisio Trees (February 11) 5.1 Graph Coectivity Suppose we are give a udirected graph G, represeted as a boolea adjacecy matrix = (a ij ), where a ij = 1 if ad oly if vertices i ad j are coected
More informationSubject CT5 Contingencies Core Technical Syllabus
Subject CT5 Cotigecies Core Techical Syllabus for the 2015 exams 1 Jue 2014 Aim The aim of the Cotigecies subject is to provide a groudig i the mathematical techiques which ca be used to model ad value
More informationWeek 3 Conditional probabilities, Bayes formula, WEEK 3 page 1 Expected value of a random variable
Week 3 Coditioal probabilities, Bayes formula, WEEK 3 page 1 Expected value of a radom variable We recall our discussio of 5 card poker hads. Example 13 : a) What is the probability of evet A that a 5
More informationORDERS OF GROWTH KEITH CONRAD
ORDERS OF GROWTH KEITH CONRAD Itroductio Gaiig a ituitive feel for the relative growth of fuctios is importat if you really wat to uderstad their behavior It also helps you better grasp topics i calculus
More informationDetermining the sample size
Determiig the sample size Oe of the most commo questios ay statisticia gets asked is How large a sample size do I eed? Researchers are ofte surprised to fid out that the aswer depeds o a umber of factors
More informationRecursion and Recurrences
Chapter 5 Recursio ad Recurreces 5.1 Growth Rates of Solutios to Recurreces Divide ad Coquer Algorithms Oe of the most basic ad powerful algorithmic techiques is divide ad coquer. Cosider, for example,
More informationINVESTMENT PERFORMANCE COUNCIL (IPC) Guidance Statement on Calculation Methodology
Adoptio Date: 4 March 2004 Effective Date: 1 Jue 2004 Retroactive Applicatio: No Public Commet Period: Aug Nov 2002 INVESTMENT PERFORMANCE COUNCIL (IPC) Preface Guidace Statemet o Calculatio Methodology
More informationLecture 7: Borel Sets and Lebesgue Measure
EE50: Probability Foudatios for Electrical Egieers JulyNovember 205 Lecture 7: Borel Sets ad Lebesgue Measure Lecturer: Dr. Krisha Jagaatha Scribes: Ravi Kolla, Aseem Sharma, Vishakh Hegde I this lecture,
More information*The most important feature of MRP as compared with ordinary inventory control analysis is its time phasing feature.
Itegrated Productio ad Ivetory Cotrol System MRP ad MRP II Framework of Maufacturig System Ivetory cotrol, productio schedulig, capacity plaig ad fiacial ad busiess decisios i a productio system are iterrelated.
More informationLecture 13. Lecturer: Jonathan Kelner Scribe: Jonathan Pines (2009)
18.409 A Algorithmist s Toolkit October 27, 2009 Lecture 13 Lecturer: Joatha Keler Scribe: Joatha Pies (2009) 1 Outlie Last time, we proved the BruMikowski iequality for boxes. Today we ll go over the
More informationThe second difference is the sequence of differences of the first difference sequence, 2
Differece Equatios I differetial equatios, you look for a fuctio that satisfies ad equatio ivolvig derivatives. I differece equatios, istead of a fuctio of a cotiuous variable (such as time), we look for
More informationProject Deliverables. CS 361, Lecture 28. Outline. Project Deliverables. Administrative. Project Comments
Project Deliverables CS 361, Lecture 28 Jared Saia Uiversity of New Mexico Each Group should tur i oe group project cosistig of: About 612 pages of text (ca be loger with appedix) 612 figures (please
More informationTaking DCOP to the Real World: Efficient Complete Solutions for Distributed MultiEvent Scheduling
Taig DCOP to the Real World: Efficiet Complete Solutios for Distributed MultiEvet Schedulig Rajiv T. Maheswara, Milid Tambe, Emma Bowrig, Joatha P. Pearce, ad Pradeep araatham Uiversity of Souther Califoria
More informationEscola Federal de Engenharia de Itajubá
Escola Federal de Egeharia de Itajubá Departameto de Egeharia Mecâica PósGraduação em Egeharia Mecâica MPF04 ANÁLISE DE SINAIS E AQUISÇÃO DE DADOS SINAIS E SISTEMAS Trabalho 02 (MATLAB) Prof. Dr. José
More informationMESSAGE TO TEACHERS: NOTE TO EDUCATORS:
MESSAGE TO TEACHERS: NOTE TO EDUCATORS: Attached herewith, please fid suggested lesso plas for term 1 of MATHEMATICS Grade 12. Please ote that these lesso plas are to be used oly as a guide ad teachers
More informationDomain 1  Describe Cisco VoIP Implementations
Maual ONT (6428) 18004186789 Domai 1  Describe Cisco VoIP Implemetatios Advatages of VoIP Over Traditioal Switches Voice over IP etworks have may advatages over traditioal circuit switched voice etworks.
More informationKey Ideas Section 81: Overview hypothesis testing Hypothesis Hypothesis Test Section 82: Basics of Hypothesis Testing Null Hypothesis
Chapter 8 Key Ideas Hypothesis (Null ad Alterative), Hypothesis Test, Test Statistic, Pvalue Type I Error, Type II Error, Sigificace Level, Power Sectio 81: Overview Cofidece Itervals (Chapter 7) are
More informationEngineering Data Management
BaaERP 5.0c Maufacturig Egieerig Data Maagemet Module Procedure UP128A US Documetiformatio Documet Documet code : UP128A US Documet group : User Documetatio Documet title : Egieerig Data Maagemet Applicatio/Package
More informationINVESTMENT PERFORMANCE COUNCIL (IPC)
INVESTMENT PEFOMANCE COUNCIL (IPC) INVITATION TO COMMENT: Global Ivestmet Performace Stadards (GIPS ) Guidace Statemet o Calculatio Methodology The Associatio for Ivestmet Maagemet ad esearch (AIM) seeks
More informationME 101 Measurement Demonstration (MD 1) DEFINITIONS Precision  A measure of agreement between repeated measurements (repeatability).
INTRODUCTION This laboratory ivestigatio ivolves makig both legth ad mass measuremets of a populatio, ad the assessig statistical parameters to describe that populatio. For example, oe may wat to determie
More information5KHz low pass filter. unity at sample 1. Aligned with coeff. 3
Copyright 997, Lavry Egieerig Uderstadig FIR (Fiite Impulse Respose) Filters  A Ituitive Approach by Da Lavry, Lavry Egieerig People less familiar with digital sigal processig, ofte view the theory as
More informationAQA STATISTICS 1 REVISION NOTES
AQA STATISTICS 1 REVISION NOTES AVERAGES AND MEASURES OF SPREAD www.mathsbox.org.uk Mode : the most commo or most popular data value the oly average that ca be used for qualitative data ot suitable if
More information0.7 0.6 0.2 0 0 96 96.5 97 97.5 98 98.5 99 99.5 100 100.5 96.5 97 97.5 98 98.5 99 99.5 100 100.5
Sectio 13 KolmogorovSmirov test. Suppose that we have a i.i.d. sample X 1,..., X with some ukow distributio P ad we would like to test the hypothesis that P is equal to a particular distributio P 0, i.e.
More information(VCP310) 18004186789
Maual VMware Lesso 1: Uderstadig the VMware Product Lie I this lesso, you will first lear what virtualizatio is. Next, you ll explore the products offered by VMware that provide virtualizatio services.
More information4.1 Sigma Notation and Riemann Sums
0 the itegral. Sigma Notatio ad Riema Sums Oe strategy for calculatig the area of a regio is to cut the regio ito simple shapes, calculate the area of each simple shape, ad the add these smaller areas
More informationSECTION 1.5 : SUMMATION NOTATION + WORK WITH SEQUENCES
SECTION 1.5 : SUMMATION NOTATION + WORK WITH SEQUENCES Read Sectio 1.5 (pages 5 9) Overview I Sectio 1.5 we lear to work with summatio otatio ad formulas. We will also itroduce a brief overview of sequeces,
More informationDiscrete Mathematics and Probability Theory Spring 2014 Anant Sahai Note 13
EECS 70 Discrete Mathematics ad Probability Theory Sprig 2014 Aat Sahai Note 13 Itroductio At this poit, we have see eough examples that it is worth just takig stock of our model of probability ad may
More informationSequences, Series and Convergence with the TI 92. Roger G. Brown Monash University
Sequeces, Series ad Covergece with the TI 92. Roger G. Brow Moash Uiversity email: rgbrow@deaki.edu.au Itroductio. Studets erollig i calculus at Moash Uiversity, like may other calculus courses, are itroduced
More informationEvaluation of Different Fitness Functions for the Evolutionary Testing of an Autonomous Parking System
Evaluatio of Differet Fitess Fuctios for the Evolutioary Testig of a Autoomous Parkig System Joachim Wegeer 1, Oliver Bühler 2 1 DaimlerChrysler AG, Research ad Techology, AltMoabit 96 a, D1559 Berli,
More informationUniversal coding for classes of sources
Coexios module: m46228 Uiversal codig for classes of sources Dever Greee This work is produced by The Coexios Project ad licesed uder the Creative Commos Attributio Licese We have discussed several parametric
More informationBasic Measurement Issues. Sampling Theory and AnalogtoDigital Conversion
Theory ad AalogtoDigital Coversio Itroductio/Defiitios Aalogtodigital coversio Rate Frequecy Aalysis Basic Measuremet Issues Reliability the extet to which a measuremet procedure yields the same results
More informationEstimating the Mean and Variance of a Normal Distribution
Estimatig the Mea ad Variace of a Normal Distributio Learig Objectives After completig this module, the studet will be able to eplai the value of repeatig eperimets eplai the role of the law of large umbers
More informationRunning Time ( 3.1) Analysis of Algorithms. Experimental Studies ( 3.1.1) Limitations of Experiments. Pseudocode ( 3.1.2) Theoretical Analysis
Ruig Time ( 3.) Aalysis of Algorithms Iput Algorithm Output A algorithm is a stepbystep procedure for solvig a problem i a fiite amout of time. Most algorithms trasform iput objects ito output objects.
More information1 Introduction to reducing variance in Monte Carlo simulations
Copyright c 007 by Karl Sigma 1 Itroductio to reducig variace i Mote Carlo simulatios 11 Review of cofidece itervals for estimatig a mea I statistics, we estimate a uow mea µ = E(X) of a distributio by
More informationChapter Gaussian Elimination
Chapter 04.06 Gaussia Elimiatio After readig this chapter, you should be able to:. solve a set of simultaeous liear equatios usig Naïve Gauss elimiatio,. lear the pitfalls of the Naïve Gauss elimiatio
More informationApproximating Area under a curve with rectangles. To find the area under a curve we approximate the area using rectangles and then use limits to find
1.8 Approximatig Area uder a curve with rectagles 1.6 To fid the area uder a curve we approximate the area usig rectagles ad the use limits to fid 1.4 the area. Example 1 Suppose we wat to estimate 1.
More informationA probabilistic proof of a binomial identity
A probabilistic proof of a biomial idetity Joatho Peterso Abstract We give a elemetary probabilistic proof of a biomial idetity. The proof is obtaied by computig the probability of a certai evet i two
More informationWe have seen that the physically observable properties of a quantum system are represented
Chapter 14 Probability, Expectatio Value ad Ucertaity We have see that the physically observable properties of a quatum system are represeted by Hermitea operators (also referred to as observables ) such
More informationResearch Article Sign Data Derivative Recovery
Iteratioal Scholarly Research Network ISRN Applied Mathematics Volume 0, Article ID 63070, 7 pages doi:0.540/0/63070 Research Article Sig Data Derivative Recovery L. M. Housto, G. A. Glass, ad A. D. Dymikov
More informationThe Forgotten Middle. research readiness results. Executive Summary
The Forgotte Middle Esurig that All Studets Are o Target for College ad Career Readiess before High School Executive Summary Today, college readiess also meas career readiess. While ot every high school
More informationLecture Notes CMSC 251
We have this messy summatio to solve though First observe that the value remais costat throughout the sum, ad so we ca pull it out frot Also ote that we ca write 3 i / i ad (3/) i T () = log 3 (log ) 1
More informationChapter 7  Sampling Distributions. 1 Introduction. What is statistics? It consist of three major areas:
Chapter 7  Samplig Distributios 1 Itroductio What is statistics? It cosist of three major areas: Data Collectio: samplig plas ad experimetal desigs Descriptive Statistics: umerical ad graphical summaries
More informationResearch Method (I) Knowledge on Sampling (Simple Random Sampling)
Research Method (I) Kowledge o Samplig (Simple Radom Samplig) 1. Itroductio to samplig 1.1 Defiitio of samplig Samplig ca be defied as selectig part of the elemets i a populatio. It results i the fact
More informationThroughput of Ideally Routed Wireless Ad Hoc Networks
Throughput of Ideally Routed Wireless Ad Hoc Networks Gábor Németh, Zoltá Richárd Turáyi, 2 ad Adrás Valkó 2 Commuicatio Networks Laboratory 2 Traffic Lab, Ericsso Research, Hugary I. INTRODUCTION At the
More informationA Study on the False Alarm Rates of X, EWMA and CUSUM Control Charts when Parameters are Estimated
11 Iteratioal Coferece o Circuits, System ad Simulatio IPCSIT vol.7 (11) (11) IACSIT Press, Sigapore A Study o the False Alarm Rates of X, EWMA ad CUSUM Cotrol Charts whe Parameters are Estimated Michael
More informationClass Meeting # 16: The Fourier Transform on R n
MATH 18.152 COUSE NOTES  CLASS MEETING # 16 18.152 Itroductio to PDEs, Fall 2011 Professor: Jared Speck Class Meetig # 16: The Fourier Trasform o 1. Itroductio to the Fourier Trasform Earlier i the course,
More informationDomain 1: Designing a SQL Server Instance and a Database Solution
Maual SQL Server 2008 Desig, Optimize ad Maitai (70450) 18004186789 Domai 1: Desigig a SQL Server Istace ad a Database Solutio Desigig for CPU, Memory ad Storage Capacity Requiremets Whe desigig a
More informationARITHMETIC AND GEOMETRIC PROGRESSIONS
Arithmetic Ad Geometric Progressios Sequeces Ad ARITHMETIC AND GEOMETRIC PROGRESSIONS Successio of umbers of which oe umber is desigated as the first, other as the secod, aother as the third ad so o gives
More informationFourier Series and the Wave Equation Part 2
Fourier Series ad the Wave Equatio Part There are two big ideas i our work this week. The first is the use of liearity to break complicated problems ito simple pieces. The secod is the use of the symmetries
More information.04. This means $1000 is multiplied by 1.02 five times, once for each of the remaining sixmonth
Questio 1: What is a ordiary auity? Let s look at a ordiary auity that is certai ad simple. By this, we mea a auity over a fixed term whose paymet period matches the iterest coversio period. Additioally,
More informationBiology 171L Environment and Ecology Lab Lab 2: Descriptive Statistics, Presenting Data and Graphing Relationships
Biology 171L Eviromet ad Ecology Lab Lab : Descriptive Statistics, Presetig Data ad Graphig Relatioships Itroductio Log lists of data are ofte ot very useful for idetifyig geeral treds i the data or the
More informationif A S, then X \ A S, and if (A n ) n is a sequence of sets in S, then n A n S,
Lecture 5: Borel Sets Topologically, the Borel sets i a topological space are the σalgebra geerated by the ope sets. Oe ca build up the Borel sets from the ope sets by iteratig the operatios of complemetatio
More informationSection 73 Estimating a Population. Requirements
Sectio 73 Estimatig a Populatio Mea: σ Kow Key Cocept This sectio presets methods for usig sample data to fid a poit estimate ad cofidece iterval estimate of a populatio mea. A key requiremet i this sectio
More informationHere are a couple of warnings to my students who may be here to get a copy of what happened on a day that you missed.
This documet was writte ad copyrighted by Paul Dawkis. Use of this documet ad its olie versio is govered by the Terms ad Coditios of Use located at http://tutorial.math.lamar.edu/terms.asp. The olie versio
More informationYour organization has a Class B IP address of 166.144.0.0 Before you implement subnetting, the Network ID and Host ID are divided as follows:
Subettig Subettig is used to subdivide a sigle class of etwork i to multiple smaller etworks. Example: Your orgaizatio has a Class B IP address of 166.144.0.0 Before you implemet subettig, the Network
More informationhp calculators HP 12C Statistics  average and standard deviation Average and standard deviation concepts HP12C average and standard deviation
HP 1C Statistics  average ad stadard deviatio Average ad stadard deviatio cocepts HP1C average ad stadard deviatio Practice calculatig averages ad stadard deviatios with oe or two variables HP 1C Statistics
More informationIncremental calculation of weighted mean and variance
Icremetal calculatio of weighted mea ad variace Toy Fich faf@cam.ac.uk dot@dotat.at Uiversity of Cambridge Computig Service February 009 Abstract I these otes I eplai how to derive formulae for umerically
More informationCHAPTER 3 THE TIME VALUE OF MONEY
CHAPTER 3 THE TIME VALUE OF MONEY OVERVIEW A dollar i the had today is worth more tha a dollar to be received i the future because, if you had it ow, you could ivest that dollar ad ear iterest. Of all
More informationGENERATING A FRACTAL SQUARE
GENERATING A FRACTAL SQUARE I 194 the Swedish mathematicia Helge vo Koch(187194 itroduced oe o the earliest ow ractals, amely, the Koch Sowlae. It is a closed cotiuous curve with discotiuities i its derivative
More informationwhen n = 1, 2, 3, 4, 5, 6, This list represents the amount of dollars you have after n days. Note: The use of is read as and so on.
Geometric eries Before we defie what is meat by a series, we eed to itroduce a related topic, that of sequeces. Formally, a sequece is a fuctio that computes a ordered list. uppose that o day 1, you have
More informationConfidence Intervals for the Mean of Nonnormal Data Class 23, 18.05, Spring 2014 Jeremy Orloff and Jonathan Bloom
Cofidece Itervals for the Mea of Noormal Data Class 23, 8.05, Sprig 204 Jeremy Orloff ad Joatha Bloom Learig Goals. Be able to derive the formula for coservative ormal cofidece itervals for the proportio
More informationMaximum Likelihood Estimators.
Lecture 2 Maximum Likelihood Estimators. Matlab example. As a motivatio, let us look at oe Matlab example. Let us geerate a radom sample of size 00 from beta distributio Beta(5, 2). We will lear the defiitio
More informationDesigning Incentives for Online Question and Answer Forums
Desigig Icetives for Olie Questio ad Aswer Forums Shaili Jai School of Egieerig ad Applied Scieces Harvard Uiversity Cambridge, MA 0238 USA shailij@eecs.harvard.edu Yilig Che School of Egieerig ad Applied
More informationBASIC STATISTICS. Discrete. Mass Probability Function: P(X=x i ) Only one finite set of values is considered {x 1, x 2,...} Prob. t = 1.
BASIC STATISTICS 1.) Basic Cocepts: Statistics: is a sciece that aalyzes iformatio variables (for istace, populatio age, height of a basketball team, the temperatures of summer moths, etc.) ad attempts
More information