Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations

Transcription

1 Real-Time Computig Without Stable States: A New Framework for Neural Computatio Based o Perturbatios Wolfgag aass+, Thomas Natschläger+ & Hery arkram* + Istitute for Theoretical Computer Sciece, Techische Uiversität Graz; A-8010 Graz, Austria * Brai id Istitute, Ecole Polytechique Federale de Lausae, CH-1015 Lausae, Switzerlad Wolfgag aass & Thomas Natschlaeger Istitute for Theoretical Computer Sciece Techische Uiversitaet Graz Iffeldgasse 16b, A-8010 Graz, Austria Tel: Fax: [email protected], [email protected] Hery arkram Brai id Istitute Ecole Polytechique Federale de Lausae CE-Ecubles, CH-1015 Lausae, Switzerlad Tel: Fax: [email protected] Address for Correspodece: Wolfgag aass 1

2 A key challege for eural modelig is to explai how a cotiuous stream of multi-modal iput from a rapidly chagig eviromet ca be processed by stereotypical recurret circuits of itegrate-ad-fire euros i real-time. We propose a ew computatioal model for real-time computig o time-varyig iput that provides a alterative to paradigms based o Turig machies or attractor eural etworks. It does ot require a taskdepedet costructio of eural circuits. Istead it is based o priciples of high dimesioal dyamical systems i combiatio with statistical learig theory, ad ca be implemeted o geeric evolved or foud recurret circuitry. It is show that the iheret trasiet dyamics of the high-dimesioal dyamical system formed by a sufficietly large ad heterogeeous eural circuit may serve as uiversal aalog fadig memory. Readout euros ca lear to extract i real-time from the curret state of such recurret eural circuit iformatio about curret ad past iputs that may be eeded for diverse tasks. Stable iteral states are ot required for givig a stable output, sice trasiet iteral states ca be trasformed by readout euros ito stable target outputs due to the high dimesioality of the dyamical system. Our approach is based o a rigorous computatioal model, the liquid state machie, that ulike Turig machies, does ot require sequetial trasitios betwee well-defied discrete iteral states. It is supported, like the Turig machie, by rigorous mathematical results that predict uiversal computatioal power uder idealized coditios, but for the biologically more realistic sceario of real-time processig of time-varyig iputs. Our approach provides ew perspectives for the iterpretatio of eural codig, for the desig of experimets ad dataaalysis i europhysiology, ad for the solutio of problems i robotics ad eurotechology. 2

3 Itroductio Itricate topographically orgaized feed-forward pathways project rapidly chagig spatio-temporal iformatio about the eviromet ito the eocortex. This iformatio is processed by extremely complex but surprisigly stereotypic microcircuits that ca perform a wide spectrum of tasks (Shepherd, 1988, Douglas et al., 1998, vo elcher et al., 2000). The microcircuit features that eable this seemigly uiversal computatioal power, is a mystery. Oe particular feature, the multiple recurret loops that form a immesely complicated etwork usig as may as 80% of all the syapses withi a fuctioal eocortical colum, has preseted a itractable problem both for computatioal models ispired by curret artificial computig machiery (Savage, 1998), ad for attractor eural etwork models. The difficulty to uderstad computatios withi recurret etworks of itegrate-ad-fire euros comes from the fact that their dyamics takes o a life of its ow whe challeged with rapidly chagig iputs. This is particularly true for the very high dimesioal dyamical system formed by a eural microcircuit, whose compoets are highly heterogeeous ad where each euro ad each syapse adds degrees of freedom to the dyamics of the system. The most commo approach for modelig computig i recurret eural circuits has bee to try to take cotrol of their high dimesioal dyamics. ethods for cotrollig the dyamics of recurret eural etworks through adaptive mechaisms are reviewed i (Pearlmutter, 1995). So far oe has ot bee able to apply these to the case of etworks of spikig euros. Other approaches towards modelig computatio i biological eural systems are based o costructios of artificial eural etworks that simulate Turig machies or other models for digital computatio, see for example (Pollack, 1991), (Giles et al., 1992), (Siegelma et al., 1994), (Hyoetyiemi, 1996), (oore, 1998). Amog these there are models, such as dyamical recogizers, which are capable of real-time computig o olie iput (i discrete time). Noe of these approaches has bee demostrated to work for etworks of spikig euros, or ay more realistic models for eural microcircuits. It was show i (aass, 1996) that oe also ca costruct recurret circuits of spikig euros that ca simulate arbitrary Turig machies. But all of these approaches require sychroizatio of all euros by a cetral clock, a feature that appears to missig i eural microcircuits. I additio they require the costructio of particular recurret circuits, ad caot be implemeted by evolvig or adaptig a give circuit. Furthermore the results of (aass et al., 1999) o the impact of oise o the computatioal power of recurret eural etworks suggest that all these approaches break dow as soo as oe assumes that the uderlyig aalog computatioal uits are subject to Gaussia or other realistic oise distributios. Attractor eural etworks o the other had allow oise robust computatio, but their attractor ladscape is i geeral hard to cotrol, ad they eed to have a very large set of attractors i order to store saliet iformatio o past iputs (for example 1024 attractors i order to store 10 bits). I additio they are less suitable for real-time computig o rapidly varyig iput streams because of the time required for covergece to a attractor. Fially, oe of these approaches allows that several real-time computatios are carried out i parallel withi the same circuitry, which appears to be a geeric feature of eural microcircuits. I this article we aalyze the dyamics of eural microcircuits from the poit of view of a readout euro, whose task is to extract iformatio ad report results from a eural microcircuit to other circuits. A huma observer of the dyamics i a eural microcircuit would be lookig for clearly distict ad temporally stable features, such as covergece to attractors. We show that a readout euro, that receives iputs from hudreds or thousads of euros i a eural 3

4 microcircuit, ca lear to extract saliet iformatio from the high dimesioal trasiet states of the circuit, ad ca trasform trasiet circuit states ito stable readouts. I particular each readout ca lear to defie its ow otio of equivalece of dyamical states withi the eural microcircuit, ad ca the perform its task o ovel iputs. This uexpected fidig of readoutassiged equivalet states of a dyamical system explais how ivariat readout is possible despite the fact that the eural microcircuit may ever re-visit the same state. Furthermore we show that multiple readout modules ca be traied to perform differet tasks o the same state trajectories of a recurret eural circuit, thereby eablig parallel real-time computig. We preset the mathematical framework for a computatioal model that does ot require covergece to stable iteral states or attractors (eve if they do occur), sice iformatio about past iputs is automatically captured i the perturbatios of a dyamical system, i.e. i the cotiuous trajectory of trasiet iteral states. Special cases of this mechaism were already reported i (Buoomao et al., 1995) ad (Domiey et al., 1995). Similar ideas have bee discovered idepedetly by Herbert Jaeger (Jaeger, 2001) i the cotext of artificial eural etworks. Computig without Attractors As a illustratio for our geeral approach towards real-time computig cosider a series of trasiet perturbatios caused i a excitable medium (see (Holde et al., 1991)), for example a liquid, by a sequece of exteral disturbaces ("iputs") such as wid, soud, or sequeces of pebbles dropped ito the liquid. Viewed as a attractor eural etwork, the liquid has oly oe attractor state the restig state ad may therefore seem useless for computatioal purposes. However, the perturbed state of the liquid, at ay momet i time, represets preset as well as past iputs, potetially providig the iformatio eeded for a aalysis of various dyamic aspects of the eviromet. I order for such a liquid to serve as a source of saliet iformatio about preset ad past stimuli without relyig o stable states, the perturbatios must be sesitive to salietly differet iputs but o-chaotic. The maer i which perturbatios are formed ad maitaied would vary for differet types of liquids ad would determie how useful the perturbatios are for such retrograde aalysis. Limitatios o the computatioal capabilities of liquids are imposed by their time-costat for relaxatio, ad the strictly local iteractios ad homogeeity of the elemets of a liquid. Neural microcircuits, however, appear to be ideal liquids for computig o perturbatios because of the large diversity of their elemets, euros ad syapses (see (Gupta et al., 2000)), ad the large variety of mechaisms ad time costats characterizig their iteractios, ivolvig recurret coectios o multiple spatial scales ("loops withi loops"). The foudatio for our aalysis of computatios without stable states is a rigorous computatioal model: the liquid state machie. Two macroscopic properties emerge from our theoretical aalysis ad computer simulatios as ecessary ad sufficiet coditios for powerful real-time computig o perturbatios: a separatio property, SP, ad a approximatio property, AP. SP addresses the amout of separatio betwee the trajectories of iteral states of the system that are caused by two differet iput streams (i the case of a physical liquid, SP could reflect the differece betwee the wave patters resultig from differet sequeces of disturbaces). 4

5 AP addresses the resolutio ad recodig capabilities of the readout mechaisms - more precisely its capability to distiguish ad trasform differet iteral states of the liquid ito give target outputs (whereas SP depeds mostly o the complexity of the liquid, AP depeds mostly o the adaptability of the readout mechaism to the required task). Liquid State achies Like the Turig machie (Savage, 1998), the model of a liquid state machie (LS) is based o a rigorous mathematical framework that guaratees, uder idealized coditios, uiversal computatioal power. Turig machies, however, have uiversal computatioal power for off-lie computatio o (static) discrete iputs, while LSs have i a very specific sese uiversal computatioal power for real-time computig with fadig memory o aalog fuctios i cotiuous time. The iput fuctio u ( ) ca be a cotiuous sequece of disturbaces, ad the target output ca be some chose fuctio y ( ) of time that provides a real-time aalysis of this sequece. I order for a machie to map iput fuctios of time u ( ) to output fuctios y( ) of time, we assume that it geerates, at every time t, a iteral liquid state x (t), which costitutes its curret respose to precedig perturbatios, i.e., to precedig iputs u (s) for s t (Figure 1). I cotrast to the fiite state of a fiite state machie (or fiite automato) this liquid state cosists of aalog values that may chage cotiuously over time. Whereas the state set ad the state trasitio fuctio of a fiite state machie is i geeral costructed for a Figure 1: A: Architecture of a LS. A fuctio of time (time series) u( ) is ijected as iput ito the L liquid filter, creatig at time t the liquid state x (t), which is trasformed by a memory-less readout map f to geerate a output y(t). specific task, the liquid states ad the trasitios betwee them eed ot be customized for a specific task. I a physical implemetatio this liquid state cosists of all iformatio about the curret iteral state of a dyamical system that is accessible to the readout modules. I 5

6 mathematical terms, this liquid state is simply the curret output of some operator or filter 1 that maps iput fuctios u ( ) oto fuctios x (t) : x ( t) = ( L u)( t). L I the followig we will refer to this filter L ofte as liquid filter, or liquid circuit if implemeted by a circuit. If it is implemeted by a eural circuit, we refer to the euros i that circuit as liquid euros. The secod compoet of a LS is a memory-less readout map f that trasforms, at every time t, the curret liquid state x (t) ito the output y( t) = f ( x ( t)). I cotrast to the liquid filter L, this readout map f is i geeral chose i a task-specific maer (ad there may be may differet readout maps, that extract differet task-specific iformatio i parallel from the curret output of L ). Note that i a fiite state machie there exists o aalogo to such task-specific readout maps, sice there the iteral fiite states are already costructed i a task-specific maer. Accordig to the precedig defiitio readout maps are i geeral memory-less 2. Hece all iformatio about iputs u (s) from precedig time poits s t that is eeded to produce a target output y(t) at time t has to be cotaied i the curret liquid state x (t). odels for computatio that have origiated i computer sciece store such iformatio about the past i stable states (for example i memory buffers or tapped delay lies). We argue, however, that this is ot ecessary sice large computatioal power o fuctios of time ca also be realized eve if all memory traces are cotiuously decayig. Istead of worryig about the code ad locatio where iformatio about past iputs is stored, ad how this iformatio decays, it is eough to address the separatio questio: For which later time poits t will ay two sigificatly differet iput fuctios of time u( ) ad v( ) cause sigificatly differet liquid states x u (t) ad (t). Good separatio capability, i combiatio x v 1 Fuctios F that map iput fuctios of time u ( ) o output fuctios y ( ) of time are usually called operators i mathematics, but are commoly referred to as filters i egieerig ad eurosciece. We use the term filter i the followig, ad we write ( Fu )( t) for the output of the filter F at time t whe F is applied to the iput fuctio u ( ). Formally, such filter F is a map from U ito ( R R ) k, where R R is the set of all real-valued fuctios of time, vectors cosistig of k such fuctios of time, U is some subset of vectors cosistig of fuctios of time i U. 2 The term "memory-less" refers to the fact that the readout map R R, ad ( R k R ) is the set of U is the set of f is ot required to retai ay memory of previous states x ( s), s < t, of the liquid. However, i a biological cotext, the readout map will i geeral be subject to plasticity, ad may also cotribute to the memory capability of the system. We do ot explore this issues i this article because the differetiatio ito a memory-less readout map ad a liquid that serves as a memory device is made for coceptual clarificatio, ad is ot essetial to the model. 6

7 with a adequate readout map f, allows us to discard the requiremet of storig bits "util further otice" i stable states of the computatioal system. Uiversal Computatioal Power of LSs for Time Varyig Iputs We say that a class of machies has uiversal power for computatios with fadig memory o fuctios of time if ay filter F, i.e., ay map from fuctios of time u ( ) to fuctios of time y ( ), that is time ivariat 3 ad has fadig memory 4, ca be approximated by machies from this class, to ay degree of precisio. Arguably, these filters F are approximated accordig to this defiitio iclude all maps from iput fuctios of time to output fuctios of time that a behavig orgaism might eed to compute. A mathematical theorem (see Appedix A) guaratees that LSs have this uiversal computatioal power regardless of specific structure or implemetatio, provided that two abstract properties are met: the class of basis filters from which the liquid filters L are composed satisfies the poit-wise separatio property ad the class of fuctios from which the readout maps f are draw, satisfies the approximatio property. These two properties provide the mathematical basis for the separatio property SP ad the approximatio property AP that were previously discussed. Theorem 1 i Appedix A implies that there are o serious a priori limits for the computatioal power of LSs o cotiuous fuctios of time, ad thereby provides a theoretical foudatio for our approach towards modelig eural computatio. I particular, sice this theorem makes o specific requiremet regardig the exact ature or behaviour of the basis filters, as log as they satisfy the separatio property (for the iputs i questio), it provides theoretical support for employig istead of circuits that were costructed 3 A filter F is called time ivariat if ay temporal shift of the iput fuctio u ( ) by some amout t 0 causes a temporal shift of the output fuctio y = Fu by the same amout t 0, i.e., ( Fu t 0 )( t) = ( Fu)( t + t 0) for all t,t 0 R, where u t 0 ( t) : = u( t + t 0). Note that if U is closed uder R k temporal shifts, the a time ivariat filter F : U ( R ) ca be idetified uiquely by the values y ( 0) = ( Fu)(0) of its output fuctios y ( ) at time 0. 4 Fadig memory (Boyd at al., 1985) is a cotiuity property of filters F which demads that for ay iput fuctio u ) U ay other iput fuctios ( the output )(0) v ) U [ T,0]. Formally oe defies that (Fu ca be approximated by the outputs (Fv)(0) for ( that approximate ( ) F : U ) u o a sufficietly log time iterval R k ( R has fadig memory if for every u U ad every ε > 0 there exist δ > 0 ad T > 0 so that ( Fv )(0) ( Fu)(0) < ε for all v U with u ( t) v( t) <δ for all t [ T,0]. Iformally a filter F has fadig memory if the most sigificat bits of its curret output value (Fu)(0) deped just o the most sigificat bits of the values of its iput fuctio ( ) T,0 ito the past. Thus, i order to compute the most sigificat bits of (Fu)(0) it is ot ecessary to kow the precise value of the iput fuctio u (s) for ay time s, ad it is also ot ecessary to kow aythig about the values of u ( ) for more tha a fiite time iterval back ito the past. Note that a filter that has fadig memory is automatically causal. u from some fiite time widow [ ] 7

8 for a specific task, partially evolved or eve rather arbitrary foud computatioal modules for purposeful computatios. This feature highlights a importat differece to computatioal theories based o Turig machies or fiite state machies, which are ofte used as coceptual basis for modelig eural computatio. The mathematical theory of LSs ca also be exteded to cover computatio o spike u of the iput u( ) trais (discrete evets i cotiuous time) as iputs. Here the i th compoet ( ) is a fuctio that assumes oly the values 0 ad 1, with ( t) = 1 at time t. Thus ( ) u i if the i th precedig euro fires u is ot a cotiuous fuctio but a sequece of poit evets. Theorem 2 i i Appedix A provides a theoretical foudatio for approximatig ay biologically relevat computatio o spike trais by LSs. i Neural icrocircuits as Implemetatios of LSs I order to test the applicability of this coceptual framework to modelig computatio i eural microcircuits, we carried out computer simulatios where a geeric recurret circuit of itegrate-ad-fire euros (see Appedix B for details) was employed as liquid filter. I other words: computer models for eural microcircuits were viewed as implemetatio of the liquid filter L of a LS. I order to test the theoretically predicted uiversal real-time computig capabilities of these eural implemetatios of LSs, we evaluated their performace o a wide variety of challegig bechmark tasks. The iput to the eural circuit was via oe or several iput spike trais, which diverged to iject curret ito 30% radomly chose liquid euros. The amplitudes of the iput syapses were chose from a Gaussia distributio, so that each euro i the liquid circuit received a slightly differet iput (a form of topographic ijectio). The liquid state of the eural microcircuit at time t was defied as all iformatio that a readout euro could extract at time t from the circuit, i.e. the output at time t of all the liquid euros represeted the curret liquid state of this istatiatio of a LS. ore precisely, sice the readout euros were modeled as I&F euros with a biologically realistic membrae time costat of 30 ms, the liquid state x (t) at time t cosisted of the vector of cotributios of all the liquid euros to the membrae potetial at time t of a geeric readout euro (with uit syaptic weights). athematically this liquid state x (t) ca be defied as the vector of output values at time t of liear filters with expoetial decay (time costat 30 ms) applied to the spike trais emitted by the liquid euros. Each readout map f was implemeted by a separate populatio P of itegrate-ad-fire euros (referred to as "readout euros") that received iput from all the "liquid euros", but had o lateral or recurret coectios 5. The curret firig activity p(t) of the populatio P, that is the fractio of euros i P firig durig a time bi of 20ms, was iterpreted as the aalog output of f at time t (oe ofte refers to such represetatio of aalog values by the curret firig activity i a pool of euros as space rate codig). Theoretically the class of readout maps that ca be implemeted i this fashio satisfies the approximatio property AP (aass, 2000, Auer et al., 2001), ad is accordig to Theorem 1 i priciple sufficiet for approximatig arbitrary give fadig memory filters F. I cases where a readout with discrete values 1 ad 0 5 For coceptual purposes we separate the liquid ad readout elemets i this paper, although dual liquid-readout fuctios ca also be implemeted. 8

9 suffices, oe ca implemet a readout map eve by a sigle I&F euro that represets these discrete output values by firig/o-firig at time t. I cases where the target output cosists of slowly varyig aalog values, a sigle readout euro ca be traied to represet these values through its time-varyig firig rate. I ay case the readout euros ca be traied to perform a specific task by adjustig the stregths of syapses projected oto them from the liquid euros usig a perceptro-like local learig rule (Auer et al., 2001). The fial leared state of the readout euros eables them to take particular weighted sums of the curret outputs x (t) of the liquid euros ad geerate a respose f ( x (t) ) that approximates the target value y(t). As a first test of these eural implemetatios of LSs we evaluated the separatio property SP of computer models for eural microcircuits o spike trai iputs. A large set of pairs of Poisso spike trais u( ) ad v( ) were radomly geerated ad ijected (i separate trials) as iput to the recurret eural circuit. The resultig trajectories x ( ) ad Figure 2: Average distace of liquid states for two differet iput spike trais u ad v (give as iput to the eural circuit i separate trials, each time with a idepedetly chose radom iitial state of the eural circuit, see Appedix B) plotted as a fuctio of time t. The state distace icreases with the distace d(u,v) betwee the two iput spike trais u ad v. Plotted o the y-axis is the average value of x ( t) x ( t) u v, where. deotes the Euclidea orm, ad x u (t), (t) x v deote the liquid states at time t for iput spike trais u ad v. The plotted results for the values 0.1, 0.2, 0.4 of the iput differece d' represet the average over 200 radomly geerated pairs u ad v of spike trais such that d ' d( u, v) < Parameters of the liquid: 1 colum, degree of coectivity λ = 2 (see Appedix B for details). x v ( ) of liquid states of the recurret circuit were recorded for each of these time-varyig iputs u( ) ad v( ). The average distace x ( t) x ( t) betwee these liquid states was plotted i Figure 2 as a fuctio of the time t after the oset of the iput, for various fixed values of the distace d(u,v) 6 betwee the two spike trai iputs u ad v. These curves show that the distace betwee these liquids states is well above the oise level, i.e. above the average liquid state differeces caused by the same spike trai applied with two differet radomly chose iitial coditios of the circuit (idicated by the solid curve). Furthermore these curves show that the differece i liquid states is after the first 30 ms roughly proportioal to the distace betwee the correspodig iput 6 I order to defie the distace d ( u, v) betwee two spike trais u ad v we replaced each 2 spike by a Gaussia exp( ( t / τ) ) for τ = 5ms (to be precise, u ad v are covolved with the 2 Gaussia kerel exp( ( t / τ) ) ) ad defied d ( u, v) as the distace of the resultig two cotiuous fuctios i the L2-orm (divided by the maximal legths 0.5 s of the spike trais u ad v). u u v 9

10 spike trais. Note i particular the absece of chaotic effects for these geeric eural microcircuit models with biologically realistic itermediate coectio legths. Explorig the Computatioal Power of odels for Neural icrocircuit As a first test of its computatioal power this simple geeric circuit was applied to a previously cosidered classificatio task (Hopfield & Brody, 2001), where spoke words were represeted by oise-corrupted spatio-temporal spike patters over a rather log time iterval (40-chael spike patters over 0.5s). This classificatio task had bee solved i (Hopfield & Brody, 2001) by a etwork of euros desiged for this task (relyig o ukow mechaisms that could provide smooth decays of firig activity over loger time periods, ad apparetly requirig substatially larger etworks of I&F euros if fully implemeted with I&F euros). The architecture of that etwork, which had bee customized for this task, limited its classificatio power to spike trais cosistig of a sigle spike per chael. We foud that the same, but also a more geeral versio of this spatio-temporal patter recogitio task that allowed several spikes per iput chael, ca be solved by a geeric recurret circuit as described i the previous sectio. Furthermore the output of this etwork was available at ay time, ad was usually correct as soo as the liquid state of the eural circuit had absorbed eough iformatio about the iput (the iitial value of the correctess just reflects the iitial guess of the readout). Formally we defied the correctess of the eural readout at time s by the term 1 target output y(s) readout activity p(s), where the target output y(s) cosisted i this case of the costat values 1 or 0, depedig o the iput patter. Plotted i Fig. 3 is for ay time t durig the presetatio of the iput patters i additio to the correctess as a fuctio of t also the certaity of the output at time t, which is defied as the average correctess up to that time t. Whereas the etwork costructed by Hopfield ad Brody was costructed to be ivariat with regard to liear time warpig of iputs (provided that oly oe spike arrives i each chael), the readouts of the geeric recurret circuit that we cosidered could be traied to be ivariat with regard to a large class of differet types of oises. The results show i Fig. 3 are for a oise where each iput spike is moved idepedetly by a amout draw from a Gaussia distributio with mea 0 ad SD 32 ms. 10

11 Figure 3: Applicatio of a geeric recurret etwork of I&F euros modeled as LS to a more difficult versio of a well-studied classificatio task (Hopfield & Brody, 2001). Five radomly draw patters (called zero, "oe ", "two",..), each cosistig of 40 parallel Poisso spike trais over 0.5s, were chose. Five readout modules, each cosistig of 50 itegrate-ad-fire euros, were traied with 20 oisy versios of each iput patter to respod selectively to oisy versios of just oe of these patters (oise was ijected by radomly movig each spike by a amout draw idepedetly from a Gaussia distributio with mea 0 ad variace 32ms; i additio the iitial state of the liquid euros was chose radomly at the begiig of each trial). The resposes of the readout which had bee traied * to detect the patter "zero" is show for a ew, previously ot show, oisy versios of two of the iput patters. The correctess ad certaity (= average correctess so far) are show as fuctios of time from the oset of the stimulus at the bottom. The correctess is calculated as 1 p(t)- y(t) where p(t) is the ormalized firig activity i the readout pool (ormalized to the rage [0 1]; 1 correspodig to a activity of 180Hz; biwidth 20ms) ad y(t) is the target output. (Correctess starts at a level of 0 for patter zero where this readout pool is supposed to become active, ad at a value of 1 for patter oe, because the readout pool starts i a iactive state). I cotrast to most circuits of spikig euros that have bee costructed for specific computatioal task, the spike trais of liquid ad readout euros show i this figure look rather realistic. * The familiar delta-rule was applied or ot applied to each readout euro, depedig o whether the curret firig activity i the readout pool was too high, too low, or about right, thus requirig at most two bits of global commuicatio. The precise versio of the learig rule was the p-delta rule that is discussed i Auer et al., (2001). 11

12 Givig a costat output for a time-varyig liquid state (caused by a time-varyig iput) is a serious challege for a LS, sice it caot rely o attractor states, ad the memory-less readout has to trasform the trasiet ad cotiuously chagig states of the liquid ito a stable output (see the discussio below ad Fig. 9 for details). I order to explore the limits of this simple eural implemetatio of a LS for computig o time-varyig iput, we chose aother classificatio task where all iformatio of the iput is cotaied i its temporal evolutio, more precisely i the iterspike itervals of a sigle iput spike trai. I this test, 8 radomly geerated Poisso spike trais over 250 ms, or equivaletly 2 Poisso spike trais over 1000 ms partitioed Figure 4: Evaluatig the fadig memory of a geeric eural microcircuit: the task. I this more challegig classificatio task all spike trais are of legth 1000 ms ad cosist of 4 segmets of legth 250 ms each. For each segmet 2 templates were geerated radomly (Poisso spike trai with a frequecy of 20 Hz); see upper traces. The actual iput spike trais of legth 1000 ms used for traiig ad testig were geerated by choosig for each segmet oe of the two associated templates, ad the geeratig a oisy versio by movig each spike by a amout draw from a Gaussia distributio with mea 0 ad a SD that we refer to as jitter (see lower trace for a visualizatio of the jitter with a SD of 4 ms). The task is to output with 4 differet readouts at time t = 1000 ms for each of the precedig 4 iput segmets the umber of the template from which the correspodig segmet of the iput was geerated. Results are summarized i Figures 5 ad 6. ito 4 segmets each (see top of Figure 4), were chose as template patters. Other spike trais over 1000 ms were geerated by choosig for each 250 ms segmet oe of the two templates for this segmet, ad by jitterig each spike i the templates (more precisely: each spike was moved by a amout draw from a Gaussia distributio with mea 0 ad a SD that we refer to as jitter, see bottom of Figure 4). A typical spike trai geerated i this way is show i the middle of Figure 4. Because of the oisy dislocatio of spikes it was impossible to recogize a specific template from a sigle iterspike iterval (ad there were o spatial cues cotaied i this sigle chael iput). Istead, a patter formed by several iterspike itervals had to be recogized ad classified retrospectively. Furthermore readouts were ot oly traied to classify at time t = 1000 ms (i.e., at after the iput spike trai had etered the circuit) the template from which the last 250 ms segmet of this iput spike trai had bee geerated, but other readouts were traied to classify simultaeously also the templates from which precedig segmets of the iput (which had etered the circuit several hudred ms earlier) had bee geerated. Obviously 12

13 the latter classificatio task is substatially more demadig, sice the correspodig earlier segmets of the iput spike trai may have left a clear trace i the curret firig activity of the recurret circuit just after they had etered the circuit, but this trace was subsequetly overwritte by the ext segmets of the iput spike trai (which had o correlatio with the choice of the earlier segmets). Altogether there were i this experimet 4 readouts f 1 to f 4, where f i had bee traied to classify at time t = 1000 ms the i-th idepedetly chose 250 ms segmet of the precedig iput spike trai. The performace of the LS, with a geeric recurret etwork of 135 I&F euros as liquid filter (see Appedix B), was evaluated after traiig of the readout pools o iputs from the same distributio (for jitter = 4 ms), but with a example that the LS had ot see before. The accuracy of the 4 readouts is plotted i pael A of Figure 5. It demostrates the fadig memory of a geeric recurret circuit of I&F euros, where iformatio about iputs that occurred several hudred ms ago ca be recovered eve after that iput segmet was subsequetly overwritte. Sice readout euros (ad euros withi the liquid circuit) were modeled with a realistic time costat of just 30 ms, the questio arises where this iformatio about earlier iputs had bee stored for Figure 5: Evaluatig the fadig memory of a geeric eural microcircuit: results. 4 readout modules f 1 to f 4, each cosistig of a sigle perceptro, were traied for their task by liear regressio. The readout module f i was traied to output 1 at time t=1000 ms if the i-th segmet of the previously preseted iput spike trai had bee costructed from the correspodig template 1, ad to output 0 at time t=1000 ms otherwise. Correctess (percetage of correct classificatio o a idepedet set of 500 iputs ot used for traiig) is calculated as average over 50 trials. I each trial ew Poisso spike trais were draw as templates, a ew radomly coected circuit was costructed (1 colum, λ=2; see Appedix B), ad the readout modules f 1 to f 4 were traied with1000 traiig examples geerated by the distributio described i Figure 4. A: Average correctess of the 4 readouts for ovel test iputs draw from the same distributio. B: Firig activity i the liquid circuit (time iterval [0.5 s, 0.8 s]) for a typical iput spike trai. C: Results of a cotrol experimet where all dyamic syapses i the liquid circuit had bee replaced by static syapses (the mea values of the syaptic stregths were uiformly re-scaled so that the average liquid activity is approximately the same as for dyamic syapses). The liquid state of this circuit cotaied substatially less iformatio about earlier iput segmets. D: Firig activity i the liquid circuit with static syapses used for the classificatio results reported i pael C. The circuit respose to each of the 4 iput spikes that etered the circuit durig the observed time iterval [0.5 s, 0.8 s] is quite stereotypical without dyamic syapses (except for the secod iput spike that arrives just 20 ms after the first oe). I cotrast the firig respose of the liquid circuit with dyamic syapses (pael B) is differet for each of the 4 iput spikes, showig that dyamic syapses edow these circuits with the capability to process ew iput differetly depedig o the cotext set by precedig iput, eve if that precedig iput occurred several hudred ms before. 13

14 several hudred ms. As a cotrol we repeated the same experimet with a liquid circuit where the dyamic syapses had bee replaced by static syapses (with syaptic weights that achieved about the same level of firig activity as the circuit with dyamic syapses). Pael C of Fig. 5 shows that this results i a sigificat loss i performace for the classificatio of all except for the last iput segmet. A possible explaatio is provided by the raster plots of firig activity i the liquid circuit with (pael B) ad without dyamic syapses (pael D), show here with high temporal resolutio. I the circuit with dyamic syapses the recurret activity differs for each of the 4 spikes that etered the circuit durig the time period show, demostratig that each ew spike is processed by the circuit i a idividual maer that depeds o the cotext defied by precedig iput spikes. I cotrast, the firig respose is very stereotypical for the same 4 iput spikes i the circuit without dyamic syapses, except for the respose to the secod spike that arrives withi 20 ms of the first oe (see the period betwee 500 ad 600 ms i pael D). This idicates Figure 6: Average correctess depeds o the parameter λ that cotrols the distributio of radom coectios withi the liquid circuit. Plotted is the average correctess (at time t=1000 ms, calculated as average over 50 trials as i Figure 5; same umber of traiig ad test examples) of the readout module f 3 (which is traied to classify retroactively the secod to last segmet of the precedig spike trai) as a fuctio of λ. The bad performace for λ=0 (o recurret coectios withi the circuit) shows that recurret coectios are essetial for achievig a satisfactory separatio property i eural microcircuits. Too large values of λ also decrease the performace because they support a chaotic respose. that the short term dyamics of syapses may play a essetial role i the itegratio of iformatio for real-time processig i eural microcircuits. Figure 6 examies aother aspect of eural microcircuits that appears to be importat for their separatio property: the statistical distributio of coectio legths withi the recurret circuit. Six types of liquid circuits, each cosistig of 135 I&F euros but with differet values of the parameter λ which regulated the average umber of coectios ad the average spatial legth of coectios (see Appedix B), were traied ad evaluated accordig to the same protocol ad for the same task as i Fig. 5. Show i Fig. 6 is for each of these 6 types of liquid circuits the average correctess of the readout f 3 o ovel iputs, after it had bee traied to classify the secod to last segmet of the iput spike trai. The performace was fairly low for circuits without recurret coectios (λ = 0). It also was fairly low for recurret circuits with large values of λ, whose largely legthidepedet distributio of coectios homogeized the microcircuit ad facilitated chaotic behavior. Hece for this classificatio task the ideal liquid circuit is a microcircuit that has i additio to local coectios to eighborig euros also a few log-rage coectios, thereby iterpolatig betwee the customarily cosidered extremes of strictly total coectivity (like i a cellular automato) o oe had, ad the locality-igorig global coectivity of a Hopfield et o the other had. 14

15 The performace results of eural implemetatios of LSs that were reported i this sectio should ot be viewed as absolute data o the computatioal power of recurret eural circuits. Rather the geeral theory suggests that their computatioal power icreases with ay improvemet i their separatio or approximatio property. Sice the approximatio property AP was already close to optimal for these etworks (icreasig the umber of euros i the readout module did ot icrease the performace sigificatly; ot show), the primary limitatio i performace lay i the separatio property SP. Ituitively it is clear that the liquid circuit eeds to be sufficietly complex to hold the details required for the particular task, but should reduce iformatio that is ot relevat to the task (for example spike time jitter). SP ca be egieered i may ways such as icorporatig euro diversity, implemetig specific syaptic architectures, alterig microcircuit coectivity, or simply recruitig more colums. The last optio is of particular iterest because it is ot available i most computatioal models. It will be explored i the ext sectio. Addig Computatioal Power A iterestig structural differece betwee eural systems ad our curret geeratio of artificial computig machiery is that the computatioal power of eural systems ca apparetly be elarged by recruitig more circuitry (without the eed to rewire old or ew circuits). We explored the cosequeces of recruitig additioal colums for eural implemetatios of LSs (see pael B of Fig. 7), ad compared it with the optio of just addig further coectios to the Figure 7: Separatio property ad performace of liquid circuits with larger umbers of coectios or euros. A ad B: Schematic drawigs of LSs cosistig of oe colum (A) ad four colums (B). Each colum cosists of = 135 I&F euros. C: Separatio property depeds o the structure of the liquid. Average state distace (at time t = 100 ms) calculated as described i Figure 2. A colum with high iteral coectivity (high λ) achieves higher separatio as a sigle colum with lower coectivity, but teds to chaotic behavior where it becomes equally sesitive to small ad large iput differeces d(u,v). O the other had the characteristic curve for a liquid cosistig of 4 colums with small λ is lower for values of d(u,v) lyig i the rage of jittered versios u ad v of the same spike trai patter (d(u,v) 0.1 for jitter 8 ms) ad higher for values of d(u,v) i the rage typical for spike trais u ad v from differet classes (mea: 0.22). D: Evaluatio of the same three types of liquid circuits for oise robust classificatio. Plotted is the average performace for the same task as i Fig. 6, but for various values of the jitter i iput spike times. Several colums (ot itercoected) with low iteral coectivity yield a better performig implemetatio of a LS for this computatioal task, as predicted by the aalysis of their separatio property. 15

16 primary oe-colum-liquid that we used so far (135 I&F euros with λ = 2, see pael A of Fig. 7). Pael C of Fig. 7 demostrates that the recruitmet of additioal colums icreases the separatio property of the liquid circuit i a desirable maer, where the distace betwee subsequet liquid states (always recorded at time t = 1000 ms i this experimet) is proportioal to the distace betwee the spike trai iputs that had previously etered the liquid circuit (spike trai distace measured i the same way as for Fig. 2). I cotrast the additio of more coectios to a sigle colum (λ = 8, see Appedix B) also icreases the separatio betwee subsequet liquid states, but i a quasi-chaotic maer where small iput distaces cause about the same distaces betwee subsequet liquid states as small iput differeces. I particular the subsequet liquid state distace is about equally large for two jittered versios of the iput spike trai state (yieldig typically a value of d(u,v) aroud 0.1) as for sigificatly differet iput spike trais that require differet outputs of the readouts. Thus improvig SP by alterig the itrisic microcircuitry of a sigle colum icreases sesitivity for the task, but also icreases sesitivity to oise. The performace of these differet types of liquid circuits for the same classificatio task as i Fig. 6 is cosistet with this aalysis of their characteristic separatio property. Show i pael D of Fig. 7 is their performace for various values of the spike time jitter i the iput spike trais. The optimizatio of SP for a specific distributio of iputs ad a specific group of readout modules is likely to arrive at a specific balace betwee the itrisic complexity of the microcircuitry ad the umber of repeatig colums. Parallel Computig i Real-Time o Novel Iputs Sice the liquid of the LS does ot have to be traied for a particular task, it supports parallel computig i real-time. This was demostrated by a test i which multiple spike trais were ijected ito the liquid ad multiple readout euros were traied to perform differet tasks i parallel. We added 6 readout modules to a liquid cosistig of 2 colums with differet values of λ 7. Each of the 6 readout modules was traied idepedetly for a completely differet olie task that required a output value at ay time t. We focused here o tasks that require diverse ad rapidly chagig aalog output resposes y(t). Figure 8 shows that after traiig each of these 6 tasks ca be performed i real-time with high accuracy. The performace show is for a ovel iput that was ot draw from the same distributio as the traiig examples, ad differs i several aspects from the traiig examples (thereby demostratig the possibility of extrageeralizatio i eural microcircuits, due to their iheret bias, that goes beyod the usual defiitio of geeralizatio i statistical learig theory). Readout-Assiged Equivalet States of a Dyamical System Real-time computatio o ovel iputs implies that the readout must be able to geerate a ivariat or appropriately scaled respose for ay iput eve though the liquid state may ever repeat. Ideed, Figure 3 showed already that the dyamics of readout pools ca become quite idepedet from the dyamics of the liquid eve though the liquid euros are the oly source 7 I order to combie high sesitivity with good geeralizatio performace we chose here a liquid cosistig of two colums as before, oe with λ=2, the other with λ=8 ad the iterval [ ] for the uiform distributio of the ospecific backgroud curret I b. 16

17 of iput. To examie the uderlyig mechaism for this relatively idepedet readout respose, we re-examied the readout pool from Figure 3. Whereas the firig activity withi the liquid circuit was highly dyamic, the firig activity i the readout pool was almost costat after traiig. The stability of the readout respose does ot simply come about because the readout oly samples a few uusual liquid euros as show by the distributio of syaptic weights oto a sample readout euro (Figure 9F). Sice the syaptic weights do ot chage after learig, this idicates that the readout euros have leared to defie a otio of equivalece for dyamic states of the liquid. Ideed, equivalece classes are a ievitable cosequece of collapsig the high dimesioal space of liquid states ito a sigle dimesio, but what is surprisig is that the equivalece classes are meaigful i terms of the task, allowig ivariat ad appropriately scaled readout resposes ad therefore real-time computatio o ovel iputs. Furthermore, while the iput rate may cotai saliet iformatio that is costat for a particular readout elemet, it may ot be for aother (see for example Fig. 8), idicatig that equivalece classes ad dyamic stability exist purely from the perspective of the readout elemets. 17

18 Figure 8: ulti-taskig i real-time. 4 iput spike trais of legth 2 s (show at the top) are ijected ito a liquid module cosistig of 2 colums (radomly costructed with the same parameters; see Appedix B), which is coected to multiple readout modules. Each readout module is traied to extract iformatio for a differet realtime computig task. The target fuctios are plotted as dashed lie, ad populatio respose of the correspodig readout module as solid lie. The tasks assiged to the 6 readout modules were the followig: Represet the sum of rates: at time t, output the sum of firig rates of all 4 iput spike trais withi the last 30ms. Represet the itegral of the sum of rates: at time t, output the total activity i all 4 iputs itegrated over the last 200ms. Patter detectio: output a high value if a specific spatio temporal spike patter appears. Represet a switch i spatial distributio of rates: output a high value if a specific iput patter occurs where the rate of iput spike trais 1 ad 2 goes up ad simultaeously the rate of iput spike trais 3 ad 4 goes dow, otherwise remai low. Represet the firig correlatio: at time t, output the umber of spike coicideces (ormalized ito the rage [0 1]) durig the last 75 ms for iputs 1 ad 3 ad separately for iputs 1 ad 2. Target readout values are plotted as dashed lies, actual outputs of the readout modules as solid lies, all i the same time scale as the 4 spike trais show at the top that eter the liquid circuit durig this 2 s time iterval. Results show are for a ovel iput that was ot draw from the same distributio as the traiig examples. 150 traiig examples were draw radomly from the followig distributio. Each iput spike trai was a idepedetly draw Possio spike trai with a time varyig rate of r(t) = A+B si (2 π f t + α). The parameters A, B, ad f where draw radomly from the followig itervals (the phase was fixed at α=0 deg): A [0Hz, 30Hz] ad [70Hz, 100Hz], B [0Hz, 30Hz] ad [70Hz, 100Hz], f [0.5Hz, 1Hz] ad [3Hz, 5Hz]. O this backgroud activity 4 differet patters had bee superimposed (always i the same order durig traiig): rate switch to iputs 1 ad 3, a burst patter, rate switch to iputs 1 ad 2, ad fially a spatio temporal spike patter. The results show are for a test iput that could ot be geerated by the same distributio as the traiig examples, because its base level (A=50Hz), as well as the amplitude (B=50Hz), frequecy (f=2hz) ad phase (α=180 deg) of the uderlyig time varyig firig rate of the Poisso iput spike trais were chose to lie i the middle of the gaps betwee the two itervals that were used for these parameters durig traiig. Furthermore the spatio-temporal patters (a burst patter, rate switch to iputs 1 ad 3, ad rate switch to iputs 1 ad 2), that were superimposed to achieve more iput variatio withi the observed 2 s, ever occured i this order ad at these time poits for ay traiig iput. Hece the accurate performace for this ovel iput demostrates substatial geeralizatio capabilities of the readouts after traiig. 18

19 Discussio We itroduce the liquid state machie, a ew paradigm for real-time computig o timevaryig iput streams. I cotrast to most computatioal models it does ot require the costructio of a circuit or program for a specific computatioal task. Rather, it relies o priciples of high-dimesioal dyamical systems ad learig theory that allow it to adapt uspecific evolved or foud recurret circuitry for a give computatioal task. Sice oly the readouts, ot the recurret circuit itself, have to be adapted for specific computatioal tasks, the same recurret circuit ca support completely differet real-time computatios i parallel. The uderlyig abstract computatioal model of a liquid state machie (LS) emphasizes the importace of perturbatios i dyamical systems for real-time computig, sice eve without stable states or attractors the separatio property ad the approximatio property may edow a dyamical system with virtually ulimited computatioal power o time-varyig iputs. I particular we have demostrated the computatioal uiversality of geeric recurret circuits of itegrate-ad-fire euros (eve with quite arbitrary coectio structure), if viewed as special cases of LSs. Apparetly this is the first stable ad geerally applicable method for usig geeric recurret etworks of itegrate-ad-fire euros to carry out a wide family of complex real-time computatios o spike trais as iputs. Hece this approach provides a platform for explorig the computatioal role of specific aspects of biological eural microcircuits. The computer simulatios reported i this article provide possible explaatios ot oly for the computatioal role of the highly recurret coectivity structure of eural circuits, but also for their characteristic distributio of coectio legths, which places their coectivity structure betwee the extremes of strictly local coectivity (cellular automata or coupled map lattices) ad uiform global coectivity (Hopfield ets) that are usually addressed i theoretical studies. Furthermore our computer simulatios suggest a importat computatioal role of dyamic syapses for real-time computig o time-varyig iputs. Fially, we reveal a most uexpected ad remarkable priciple that readout elemets ca establish their ow equivalece relatioships o high-dimesioal trasiet states of a dyamical system, makig it possible to geerate stable ad appropriately scaled output resposes eve if the iteral state ever coverges to a attractor state. I cotrast to virtually all computatioal models from computer sciece or artificial eural etworks, this computatioal model is ehaced rather tha hampered by the presece of diverse computatioal uits. Hece it may also provide isight ito the computatioal role of the complexity ad diversity of euros ad syapses (see for example (Gupta et al., 2000)). While there are may plausible models for spatial aspects of eural computatio, a biologically realistic framework for modelig temporal aspects of eural computatio has bee missig. I cotrast to models ispired by computer sciece, the liquid state machie does ot try to reduce these temporal aspects to trasitios betwee stable states or limit cycles, ad it does ot require delay lies or buffers. Istead it proposes that the trajectory of iteral states of a recurret eural circuit provides a raw, ubiased, ad uiversal source of temporally itegrated iformatio, from which specific readout elemets ca extract specific iformatio about past iputs for their idividual task. Hece the otorious trial-to-trial stimulus respose variatios i sigle ad populatios of euros observed experimetally, may reflect a accumulatio of iformatio from previous iputs i the trajectory of iteral states, rather tha oise (see also (Arieli et al., 1996)). This would imply that averagig over trials or biig, peels out most of the iformatio processed by recurret microcircuits ad leaves mostly topographic iformatio. 19

20 This approach also offers ew ideas for models of the computatioal orgaisatio of cogitio. It suggests that it may ot be ecessary to scatter all iformatio about sesory iput by recodig it through feedforward processig as output vector of a esemble of feature Figure 9: Readout assiged equivalet states of a dyamical system. A LS (liquid circuit as i Figure 3) was traied for the classificatio task as described i Figure 3. Results show are for a ovel test iput (draw from the same distributio as the traiig examples). A: The test iput cosists of 40 Poisso spike trais, each with a costat rate of 5 Hz. B: Raster plot of the 135 liquid euros i respose to this iput. Note the large variety of liquid states that arise durig this time period. C: Populatio rate of the liquid (bi-size 20 ms). Note that this populatio rate chages quit a bit over time. D: Readout respose (solid lie) ad target respose (dashed lie). The target respose had a costat value of 1 for this iput. The output of the traied readout module is also almost costat for this test example (except for the begiig), although its iput, the liquid states of the recurret circuit, varied quit a bit durig this time period. F: Weight distributio of a sigle readout euro. detectors with fixed receptive fields (thereby creatig the "bidig problem"). It proposes that at the same time more global iformatio about precedig iputs ca be preserved i the trajectories of very high dimesioal dyamical systems, from which multiple readout modules extract ad combie the iformatio eeded for their specific tasks. This approach is evertheless compatible with experimetal data that cofirm the existece of special maps of feature detectors. These 20

21 could reflect specific readouts, but also specialized compoets of a liquid circuit, that have bee optimized geetically ad through developmet to ehace the separatio property of a eural microcircuit for a particular iput distributio. The ew coceptual framework preseted i this article suggests to complemet the experimetal ivestigatio of eural codig by a systematic ivestigatio of the trajectories of iteral states of eural microcircuits or systems, which are compared o oe had with iputs to the circuit, ad o the other had with resposes of differet readout projectios. The liquid computig framework suggests that recurret eural microcircuits, rather tha idividual euros, might be viewed as basic computatioal uits of cortical computatio, ad therefore may give rise to a ew geeratio of cortical models that lik LS colums to form cortical areas where eighborig colums read out differet aspects of aother colum ad where each of the stereotypic colums serve both liquid ad readout fuctios. I fact, the classificatio of euros ito liquid- ad readout euros is primarily made for coceptual reasos. Aother coceptual simplificatio was made by restrictig plasticity to syapses oto readout euros. However syapses i the liquid circuit are likely to be also plastic, for example to support the extractio of idepedet compoets of iformatio about precedig time varyig iputs for a particular distributio of atural stimuli ad thereby ehace the separatio property of eural microcircuits. This plasticity withi the liquid would be iput-drive ad less task specific, ad might be most promiet durig developmet of a orgaism. I additio, the iformatio processig capabilities of hierarchies or other structured etworks of LSs remai to be explored, which may provide a basis for modelig larger cortical areas. Apart from biological modelig, the computatioal model discussed i this article may also be iterest for some areas of computer sciece. I computer applicatios where real-time processig of complex iput streams is required, such as for example i robotics, there is o eed to work with complicated heterogeeous recurret etworks of itegrate-ad-fire euros as i biological modelig. Istead, oe ca use simple devices such as tapped delay lies for storig iformatio about past iputs. Furthermore oe ca use ay oe of large selectio of powerful tools for static patter recogitio (such as feedforward eural etworks, support vector machies, or decisio trees) to extract from the curret cotet of such tapped delay lie iformatio about a precedig iput time series, i order to predict that time series, to classify that time series, or to propose actios based o that time series. This works fie, except that oe has deal with the problems caused by local miima i the error fuctios of such highly oliear patter recogitio devices, which may result i slow learig ad suboptimal geeralizatio. I geeral the escape from such local miima requires further traiig examples, or timecosumig offlie computatios such as repetitio of backprop for may differet iitial weights, or the solutio of a quadratic optimizatio problem i the case of support vector machies. Hece these approaches ted to be icompatible with real-time requiremets, where a classificatio or predictio of the past iput time series is istatly eeded. Furthermore these stadard approaches provide o support for multi-taskig, sice oe has to ru for each idividual classificatio or predictio task a separate copy of the time-cosumig patter recogitio algorithm. I cotrast, the alterative computatioal paradigm discussed i this article suggests to replace the tapped delay lie by a oliear olie projectio of the iput time series ito a highdimesioal space, i combiatio with liear readouts from that high-dimesioal itermediate space. The oliear olie preprocessig could eve be implemeted by iexpesive (eve partially faulty) aalog circuitry, sice the details of this olie preprocessig do ot matter, as log as the separatio property is satisfied for all relevat iputs. If this task-idepedet olie 21

22 preprocessig maps iput streams ito a sufficietly high-dimesioal space, all subsequet liear patter recogitio devices, such as perceptros, receive essetially the same classificatio ad regressio capability for the time varyig iputs to the system as oliear classifiers without preprocessig. The traiig of such liear readouts has a importat advatage compared with traiig oliear readouts. While the error miimizatio for a oliear readout is likely to get stuck i local miima, the sum of squared errors for a liear readout has just a sigle local miimum, which is automatically the global miimum of this error fuctio. Furthermore the weights of a liear readouts ca be adapted i a olie maer by very simple local learig rules so that the weight vector moves towards this global miimum. Related mathematical facts are exploited by support vector machies i machie learig (Vapik, 1998), although the boostig of the expressive power of liear readouts is implemeted there i a differet fashio that is ot suitable for real-time computig. Fially, the ew approach towards real-time eural computatio preseted i this article may provide ew ideas for euromorphic egieerig ad aalog VLSI. Besides implemetig recurret circuits of spikig euros i silico oe could examie a wide variety of other materials ad circuits that may potetially eable iexpesive implemetatio of liquid modules with suitable separatio properties, to which a variety of simple adaptive readout devices may be attached to execute multiple tasks. Ackowledgemet We would like to thak Rodey Douglas, Herbert Jaeger, Wulfram Gerster, Ala urray, isha Tsodyks, Thomas Poggio, Lee Segal, Tali Tishby, Ida Segev, Phil Goodma & ark Pisky for their commets o a draft of this article. The work was supported by project # P15386 of the Austria Sciece Fud, the NeuroCOLT project of the EU, the Office of Naval Research, HFSP, Dolfi & Eber Ceter ad the Edith Blum Foudatio. H is the icumbet of the Diller Family Chair i Neurosciece. Refereces Arieli, A., Sterki, A., Grivald, A., & Aertse, A. (1996). Dyamics of ogoig activity: explaatio of the large variability i evoked cortical resposes. Sciece, 273, Auer, P., Burgsteier, H., & aass, W. (2001) The p-delta rule for parallel perceptros. Submitted for publicatio, olie available at Boyd, S., & Chua, L.O. (1985). Fadig memory ad the problem of approximatig oliear operators with Volterra series. IEEE Tras. o Circuits ad Systems, 32, Buoomao, D.V., & erzeich,.. (1995). Temporal iformatio trasformed ito spatial code by a eural etwork with realistic properties. Sciece, 267, Domiey P., Arbib,., & Joseph, J.P. (1995). A model of corticostriatal plasticity for learig oculomotor associatio ad sequeces. J. Cog. Neurosci. 7(3),

23 Douglas, R., & arti, K. (1998). Neocortex. I: The Syaptic Orgaizatio of the Brai. G.. Shepherd, Ed. (Oxford Uiversity Press), Giles, C.L., iller, C.B., Che, D., H.H., Su, G.Z., & Lee, Y.C. (1992). Learig ad extractig fiite state automata with secod-order recurret eural etworks. Neural Computatio, 4, Gupta, A., Wag, Y., & arkram, H. (2000). Orgaizig priciples for a diversity of GABAergic itereuros ad syapses i the eocortex. Sciece 287, 2000, Hertz, J., Krogh, A., & Palmer, R.G. (1991). Itroductio to the Theory of Neural Computatio. (Addiso-Wesley, Redwood City, Ca). Holde, A.V., Tucker, J.V., & Thompso, B.C. (1991). Ca excitable media be cosidered as computatioal systems? Physica D, 49, Hopfield, J.J., & Brody, C.D. (2001). What is a momet? Trasiet sychroy as a collective mechaism for spatio-temporal itegratio. Proc. Natl. Acad. Sci., USA, 89(3), Hyoetyiemi, H. (1996). Turig machies are recurret eural etworks. Proc. of SteP 96 Gees, Nets ad Symbols. Alader, J., Hokela, T. & Jacobsso,., editors, Fiish Artificial Itelligece Society, Jaeger, H. (2001). The echo state approach to aalyzig ad traiig recurret eural etworks, submitted for publicatio. aass, W. (1996). Lower bouds for the computatioal power of etworks of spikig euros. Neural Computatio, 8(1), 1-40 aass, W. (2000). O the computatioal power of wier-take-all. Neural Computatio, 12(11): aass, W., & Sotag, E.D. (1999). Aalog eural ets with Gaussia or other commo oise distributios caot recogize arbitrary regular laguages. Neural Computatio, 11: aass, W., & Sotag, E.D. (2000). Neural systems as oliear filters. Neural Computatio, 12(8): arkram, H., Wag, Y., & Tsodyks,. (1998). Differetial sigalig via the same axo of eocortical pyramidal euros. Proc. Natl. Acad. Sci., 95, oore, C. (1998). Dyamical recogizers: real-time laguage recogitio by aalog computers. Theoretical Computer Sciece, 201, Pearlmutter, B.A. (1995). Gradiet calculatio for dyamic recurret eural etworks: a survey. IEEE Tras. O Neural Networks, 6(5):

24 Pollack, J.B. (1991). The iductio of dyamical recogizers. achie Learig, 7, Savage, J.E. (1998). odels of Computatio: Explorig the Power of Computig. (Addiso- Wesley, Readig, A). Shepherd, G.. (1988). A basic circuit for cortical orgaizatio. I: Perspectives i emory Research,. Gazzaiga, Ed. (IT Press), Siegelma, H., & Sotag, E.D. (1994). Aalog computatio via eural etworks. Theoretical Computer Sciece, 131: Tsodyks,., Uziel, A., & arkram, H. (2000). Sychroy geeratio i recurret etworks with frequecy-depedet syapses. J. Neurosciece, Vol. 20 RC50. Vapik, V.N. (1998). Statistical Learig Theory. Joh Wiley, New York. vo elcher, L., Pallas, S.L., & Sur,. (2000). Visual behaviour mediated by retial projectio directed to the auditory pathway. Nature, 2000 Apr 20; 404:

25 Appedix A: athematical Theory We say that a class CB of filters has the poit-wise separatio property with regard to iput fuctios from U if for ay two fuctios u( ), v( ) U with u( s) v( s) for some s 0 there exists some filter B CB that separates u ( ) ad v ( ), i.e., ( Bu )(0) ( Bv)(0). Note that it is ot required that there exists a filter B CB with ( Bu )(0) ( Bv)(0) for ay two fuctios u ( ), v( ) U with ( s) v( s) u for some s 0. Simple examples for classes CB of filters that have t0 this property are the class of all delay filters u( ) a u ( ) (for t R ), the class of all liear at filters with impulse resposes of the form h( t) = e with a > 0, ad the class of filters defied by stadard models for dyamic syapses, see (aass ad Sotag, 2000). A liquid filter L of a LS is said to be composed of filters from CB if there are fiitely may filters B, K 1, B i m CB to which we refer as basis filters i this cotext so that ( L u)( t) = ( B1u )( t), K,( B u)( t) for all t R ad all iput fuctios u ( ) i U. I other words: the output of L for a particular iput u is simply the vector of outputs give by these fiitely may basis filters for this iput u. A class CF of fuctios has the approximatio property if for ay m N, ay compact m (i.e., closed ad bouded) set X R, ay cotiuous fuctio h : X R ad ay give ρ > 0 there exists some f i CF so that that h ( x) f ( x) ρ for all x X. The defiitio for the case of fuctios with multi-dimesioal output is aalogous. Theorem 1: Cosider a space U of iput fuctios where U = u : R B, B : u( t) u( s) B' t s for all t, s R for some B, B' > 0 (thus U is a { [ ] } class of uiformly bouded ad Lipschitz-cotiuous fuctios). Assume that CB is some arbitrary class of time ivariat filters with fadig memory that has the poit-wise separatio property. Furthermore, assume that CF is some arbitrary class of fuctios that satisfies the approximatio property. The ay give time ivariat filter F that has fadig memory ca be approximated by LSs with liquid filters L composed from basis filters i CB ad readout maps f chose from CF. ore precisely: For every ε > 0 there exist m N, B,..., B 1 m CB ad f CF so that the output y ( ) of the liquid state machie with liquid filter composed of B 1,..., Bm, i.e., ( L u)( t) = ( B1u )( t), K,( Bmu)( t), ad readout map all u ) U ( ad all R t ( Fu )( t) y( t) ε. 0 m L f satisfies for The proof of this theorem follows from the Stoe-Weierstrass Approximatio Theorem, similarly as the proof of Theorem 1 i Boyd & Chua (1985).Oe ca easily show that the iverse of Theorem 1 also holds: If the fuctios i CF are cotiuous, the ay filter F that ca be approximated by the liquid state machies cosidered i Theorem 1 is time ivariat ad has fadig memory. I combiatio with Theorem 1, this provides a complete characterizatio of the computatioal power of LSs. 25

26 I order to exted Theorem 1 to the case where the iputs are fiite or ifiite spike trais, rather tha cotiuous fuctios of time, oe eeds to cosider a appropriate otio of fadig memory for filters o spike trais. The traditioal defiitio, give i footote 4, is ot suitable for the followig reaso. If u ( ) ad v ( ) are fuctios with values i {0, 1} that represet spike trais ad if δ 1, the the coditio u ( t) v( t) < δ is too strog: it would require that u ( t) = v( t). Hece we defie for the case where the domai U cosists of spike trais (i. e. 0 1 R k valued fuctios) that a filter F : U ( R ) has fadig memory o spike trais if for every u = u1,..., u U ad every ε > 0 there exist δ > 0 ad m Í so that ( Fv)(0) ( Fu)(0) < ε for all v = v1,..., v U with the property that for i = 1,..., the last m spikes i v i (before time 0 ) each have a distace of at most δ from the correspodig oes amog the last m spikes i u i. Ituitively this says that a filter has fadig memory o spike trais if the most sigificat bits of the filter output ca already be determied from the approximate times of the last few spikes i the iput spike trai. For this otio of fadig memory o spike trais oe ca prove: Theorem 2: Cosider the space U of iput fuctios where U is the class of spike trais with some miimal distace betwee successive spikes (e. g., = 1 ms). Assume that CB is some arbitrary class of time ivariat filters with fadig memory o spike trais that has the poit-wise separatio property. Furthermore, assume that CF is some arbitrary class of fuctios that satisfies the approximatio property. The ay give time ivariat filter F that has fadig memory o spike trais ca be approximated by liquid state machies with liquid filters L composed from basis filters i CB ad readout maps f chose from CF. The proof for Theorem 2 is obtaied by showig that for all filters F fadig memory o spike trais is equivalet to cotiuity with regard to a suitable metric o spike trais that turs the domai of spike trais ito a compact metric space. Hece, oe ca apply the Stoe- Weierstrass Approximatio Theorem also to this case of computatios o spike trais. Appedix B: Details of the Computer Simulatio We used a radomly coected circuit cosistig of 135 itegrate-ad-fire euros, 20% of which were radomly chose to be ihibitory, as a sigle "colum" of eural circuitry (Tsodyks et al., 2000). Neuro parameters: membrae time costat 30ms, absolute refractory period 3ms (excitatory euros), 2ms (ihibitory euros), threshold 15mV (for a restig membrae potetial assumed to be 0), reset voltage 13.5mV, costat ospecific backgroud curret I b =13.5A, iput resistace 1 Ω. Coectivity structure: The probability of a syaptic coectio from euro a to euro b (as well as that of a syaptic coectio from euro b to euro a ) was defied as 2 ( D( a, b) / λ) C e, where λ is a parameter which cotrols both the average umber of coectios ad the average distace betwee euros that are syaptically coected. We assumed that the 135 euros were located o the iteger poits of a colum i space, where D ( a, b) is 26

27 the Euclidea distace betwee euros a ad b. Depedig o whether a ad b were excitatory ( E ) or ihibitory ( I ), the value of C was 0.3 (EE), 0.2 (EI), 0.4 (IE), 0.1 (II). I the case of a syaptic coectio from a to b we modeled the syaptic dyamics accordig to the model proposed i (arkram et al., 1998), with the syaptic parameters U (use), D (time costat for depressio), F (time costat for facilitatio) radomly chose from Gaussia distributios that were based o empirically foud data for such coectios. Depedig o whether a, b were excitatory ( E ) or ihibitory ( I ), the mea values of these three parameters (with D, F expressed i secod, s) were chose to be.5, 1.1,.05 (EE),.05,.125, 1.2 (EI),.25,.7,.02 (IE),.32,.144,.06 (II). The SD of each parameter was chose to be 50% of its mea (with egative values replaced by values chose from a appropriate uiform distributio). The mea of the scalig parameter A (i A) was chose to be 30 (EE), 60 (EI), -19 (IE), -19 (II). I the case of iput syapses the parameter A had a value of 18 A if projectig oto a excitatory euro ad 9.0 A if projectig oto a ihibitory euro. ). The SD of the A parameter was chose to be 100% of its mea ad was draw from a gamma distributio. The postsyaptic curret was modeled as a expoetial decay exp(-t/τ s ) with τ s =3ms (τ s =6ms) for excitatory (ihibitory) syapses. The trasmissio delays betwee liquid euros were chose uiformly to be 1.5 ms (EE), ad 0.8 for the other coectios. For each simulatio, the iitial coditios of each leaky-itegrate ad fire euro, i.e. the membrae voltage at time t=0, were draw radomly (uiform distributio) from the iterval [13.5mV, 15.0mV]. Together with the spike time jitter i the iput these radomly draw iitial coditios served as implemetatio of oise i our simulatios (i order to test the oise robustess of our approach). Readout elemets used i the simulatios of Figures 3, 8, ad 9 were made of 51 itegrate-ad-fire euros (ucoected). A variatio of the perceptro learig rule ( the delta rule, see (Hertz et al., 1991)) was applied to scale the syapses of these readout euros: the p- delta rule discussed i (Auer et al., 2001). The p-delta rule is a geeralizatio of the delta rule that trais a populatio of perceptros to adopt a give populatio respose (i terms of the umber of perceptros that are above threshold), requirig very little overhead commuicatio. This rule, which formally requires to adjust the weights ad the threshold of perceptros, was applied i such a maer that the backgroud curret of a itegrate-ad-fire euro is adjusted istead of the threshold of a perceptro (while the firig threshold was kept costat at 15mV). I Fig. 8 ad 9 the readout euros are ot fully modeled as itegrate ad fire euros, but just as perceptros (with a low pass filter i frot that trasforms syaptic currets ito PSPs, time costat 30 ms); i order to save computatio time. I this case the "membrae potetial" of each perceptro is checked every 20 ms, ad it is said to "fire" at this time poit if this "membrae potetial" is curretly above the 15 mv threshold. No refractory effects are modeled, ad o reset after firig. The percetage of readout euros that fire durig a 20 ms time bi is iterpreted as the curret output of this readout module (assumig values i [0, 1] ). I the simulatios for Figures 5, 6, ad 7 we used just sigle perceptros as readout elemets. The weights of such a sigle perceptro have bee traied usig stadard liear regressio: the target value for the liear regressio problem was +1 (-1) if the perceptro should output 1 (0) for the give iput. The output of the perceptro after learig was 1 (0) if the weighted sum of iputs was 0 (< 0). 27