A computational model of supervised learning in the hippocampal area CA1

A computational model of supervised learning in the hippocampal area CA1 Technical Report BCCN Freiburg, 12.02.2009 Filip Ponulak 1,2 1 Institute of Control and Information Engineering, Poznań University of Technology, Poland, Poznań 60-965, ul.piotrowo 3a, phone: +48 (61) 665-2365, fax: +48 (61) 665-2563 2 Bernstein Center for Computational Neuroscience, Albert-Ludwigs University Freiburg, Germany, Freiburg 79-104, Hansastr. 9A, phone: +49 (761) 203-9324, fax: +49 (761) 203-9559 e-mail: ponulak@bccn.uni-freiburg.de Abstract In this report we investigate a new model of learning in hippocampal CA1 pyramidal cells. The model is derived based on a hypothesis that direct cortical input to the CA1 pyramidal neurons provides instructive signals associating cell s activity with selected spatio-temporal patterns of input signals from area CA3. We explore hetero- and homosynaptic plasticity mechanisms recently observed in area CA1, as means for the implementation of supervised learning. We demonstrate that modelled pyramidal cells trained according to the proposed learning rules are able to discriminate different input spike patterns. Classification results can be communicated by the neurons using a variety of information coding schemes, such as: firing/not-firing, rate-based or phase coding. 1 Introduction Hippocampal CA1 pyramidal cells (PC) receive two streams of excitatory input originating from the entorhinal cortex: a direct pathway, the perforant path (PP), synapses at distal apical dendritic sites in the stratum lacunosum-moleculare; and an indirect trisynaptic pathway, formed by the sequential activation of dentate granule cells and CA3 pyramidal neurons, which innervates CA1 pyramids at proximal apical dendritic sites in the stratum radiatum as Schaffer collateral (SC) synapses. Recently Dudman and colleagues [4] have demonstrated that low-frequency pairing of PP and SC EPSPs leads to the input-specific, potentiation or depression of the SC EPSP, depending on the timing between the PP and SC EPSPs (cf.fig.1.b). This phenomenon has been termed Input-Timing- Dependent-Plasticity (ITDP). The goal of the recent study is to investigate in a computational model whether ITDP can provide a means for supervised learning in the hippocampal CA1 pyramidal cells. It has been demonstrated that presynaptically triggered heterosynaptic plasticity rules, similar to ITDP, can indeed play an important role in supervised learning in spiking neural network models [13, 14]. ITDP exhibits several interesting properties that make it a good candidate for such a learning mechanism: the learning window of ITDP is highly selective - it strengthens only these SC inputs which are active within a short time range (10-40ms) after the PP spikes. Under the assumptions that both, PP and SC inputs to the CA1 pyramidal cell, display thetarhythm firing, ITDP potentiates the SC inputs that satisfy the following conditions: 1. fire at the same rate as an instructive PP signal, 2. fire with a certain theta-phase shift with respect to the PP signal. The fact that ITDP is triggered presynaptically (without the need for postsynaptic action potentials) satisfies the condition necessary for establishing new associations between synaptic input and neuronal output. This condition cannot be satisfied by the traditional Hebbian models, which require a temporal correlation of the pre- and postsynaptic neural activity, in order to produce the synaptic changes. 1

However, ITDP alone can potentially lead to the maximal strengthening of the SC synaptic efficacies. This effect can increase the reliability of the established association on one hand; on the other hand it can result also in the crosstalk phenomenon, i.e. undesired cell activation in response to some other input patterns (cf.fig.2.b). In order to avoid such an effect, an additional plasticity mechanism is necessary, to: (a) suppress the undesired PC activity, while (b) preserving (maintaining) the intended association. Several forms of synaptic plasticity observed in the hippocampal CA1 neurons can be considered for such a counter mechanism. Homosynaptic LTD seems to be a natural candidate to meet the condition (a), while LTP to support (b). Both mechanisms are known to be induced in CA1 under different stimulation protocols [11, 12, 2]. However, the reported conditions for the transition from LTD to LTP are often inconsistent or even contradictory to each other. The picture becomes even more complex if one considers the fact the the similar stimulation protocols induce also bidirectional STDP [3, 7]. This discrepancy is somewhat confusing and naturally leads to a question on how the particular plasticity forms are related to each other and how they should be taken into account in a model. Recent analysis of the synaptic plasticity in the area CA1 performed by Wittenberg and Wang [16] shed new light on this picture. The authors found that under certain conditions rules for potentiation and depression underlying spike-timing-dependent-plasticity can be separated from one another and their interaction can lead to the induction of pure-ltd, pure-ltp, or bidirectional STDP, depending on different proportions of potentiation and depression (Fig.1.C). This proportion is thought to depend on the factor related to the firing rate of the postsynaptic neuron, such as e.g. magnitude of the rise in intracellular calcium [8]. The results of Wittenberg and Wang provide means for investigating the particular forms of synaptic plasticity within a unifying concept, termed malleability of spike-timing-dependent-plasticity (we denote it by mstdp for short). In order to investigate whether mstdp complements ITDP in our model of supervised learning for the CA1 neurons, let us summarize conditions for the induction of the particular forms of plasticity within the mstdp framework, as reported by Wittenberg and Wang in [16]: 1. Depression-only learning is observed after pairing of single pre- and postsynaptic action potentials (both, in the causal and reverse order) at low stimulation frequencies (0.1-5 Hz). 2. Bidirectional learning (sombrero-shaped learning curve) with timing windows for both, LTP and LTD, is observed when single EPSPs are paired with postsynaptic action potential doublets at 5Hz. 3. Potentiation requires pairing of presynaptic EPSPs with postsynaptic burst firing at 5Hz or higher, a firing pattern that occurs during the theta rhythm. Induction of LTP in this protocol requires as few as 20 pairings. These results demonstrate that: (a) hippocampal synaptic plasticity can reinforce brief, rapid activity sequences, while (b) weakening nonspecific activity sequences spread over longer timescales. This is consistent with our requirements for stable supervised learning under the reasonable assumption that the trained neuron responds with theta-bursts to the associated input patterns (which corresponds to (a)) and with low-frequency, irregular spiking to non-associated input patterns (which corresponds to (b)). In the next section we present a set of computational experiments demonstrating that the combination of the considered learning rules with ITDP leads to supervised learning, resulting in the ability of PC to discriminate selected SC spatio-temporal patterns. We will show that this can be achieved using various learning paradigms and various information coding schemes. First, however, we briefly summarize our model. 2 Methods Circuit model: A diagram of the modelled neural microcircuit is presented in Fig.1.A. Our model consists of a single pyramidal cell (PC) receiving a single PP input and N = [500,1000] SC inputs (the number of SC inputs varies in the particular experiments). A Gaussian current (with mean zero and a given variance) is injected into PC in order to model a background noise. 2

A. Circuit model B. ITDP learning curve C. mstdp learning surface PP inputs A i pos SC inputs - PC BC T i pos T i Apos A i neg t pp -t sc 1 m 0 T i neg Figure 1: (A) Model of a single pyramidal cell (PC) with SC inputs, PP (supervisory) inputs and recurrent inhibition through a basket cell (BC) model, (B) Learning curve of the input-timing-dependent plasticity model used in our simulations, (C) Malleability of spike-timing-dependent-plasticity (adapted from [16]). The amount of depression or potentiation depends on the factor m. In our model m is proportional to the level of postsynaptic neural activity. The PC is recurrently inhibited by a single basket cell model (BC). The strength and latency of the inhibitory loop is set to promote PC s spiking at theta frequency [1]. Both, PC and BC neurons, are modelled using simple point-process leaky-integrate-and-fire units [6]. In our recent model we assume that the CA3-CA1 connections are the only plastic synapses and their efficacies are modified according to the homosynaptic mstdp and heterosynaptic ITDP processes. ITDP model: For each individual SC input we assume an additive ITDP model, where the changes of the synaptic efficacies are determined by the the learning window (Fig.1.B) defined over a time difference t i = (t i pp t i sc) between the firing times of the PP and SC inputs, respectively. We neglect the dependence of ITDP on the synaptic and pre-/post-neuronal states, due to the lack of relevant experimental data. The learning window for ITDP model is assumed symmetric with a positive central part (with amplitude A i pos and width T i pos) surrounded by two negative parts (with amplitudes A i neg and width T i neg = 3 T i pos). The peak of the positive part in the ITDP learning window is centred around time t i = T i A pos. The ITDP learning window is approximated with a set of straight lines. mstdp model: We consider an mstdp model with a symmetric learning window defined over a time difference t s = (t s pc t s sc) between the firing times of PC and the SC EPSPs, respectively. The learning window is described by the parameters: A s pos, T s pos, A s neg, T s neg, t s A pos, which correspond to the parameters used for the ITDP model. In contrast to ITDP, however, the parameters of mstdp are not fixed but depend on the variable m (m [0, 1]), which scales linearly the particular parameters in the range between their assumed minimum and maximum values (as depicted in Fig.1.C). The variable m is assumed to reflect the (normalized) level of activity of the PC neuron. 3 Results 3.1 Pattern discrimination with firing/not-firing code Experiment description: PC is trained to respond with theta-burst firing to the selected SC input patterns, and to become silent (or fire with low-frequency) to other patterns. Schematic illustration of the experiment is presented in Fig.2.A. It is assumed that in the initial state (I.) the neuron is not selective to any of four input patterns (A,B,C,D). During the coding phase (II.) PP input is supposed to associate PC s activity with pattern A. The established association is 3

5mV 2nV A SC SC pattern discrimination - experiment diagram I. Init state II. III.a. III.b. A B C D A B C D A B C D C D A B PP PC B SC input Pattern A Simulation results Pattern B Init state (I) PC activity in respone to Pattern A PC activity in respone to Pattern B PP input (II) (III.a) (III.b) Synaptic weights 0 50 100 150 200 250 300 350 400 450 500 0 50 100 150 200 250 300 350 400 450 500 synaptic inputs synaptic inputs Figure 2: Spatio-temporal pattern discrimination. (A) PC is trained to respond with theta-burst activity to SC input pattern A and to respond with low activity to other patterns. (B) Simulation results for pattern A (left column) and pattern B (right column). From top to bottom: (SC input) Raster plot of the SC input spikes (each row represents another input) constituting the particular spatio-temporal patterns A or B; (PP input) Pattern of activity at the PP input - supposed to play a role of an instructive signal binding PC s activity with pattern A; (Init State, ) Membrane voltage trace and the corresponding PC spikes in response to the SC inputs before and during the memory storage phase; () PC membrane potentials and spikes during information retrieval - directly after coding (III.a) or after several other retrieval rounds (III.b); (Synaptic weights) CA3-CA1 synaptic efficacies after the experiment. maintained by mstdp, so during the retrieval phase (III.a and III.b) the neuron fires with theta-bursts whenever pattern A is presented at the SC inputs. Methods: 500 SC inputs; the particular inputs are supposed to transmit sequences of spikes (15±3 spikes) with theta frequency (here 8Hz); for every SC input we randomly set a firing onset time t on in the range of 0 to 300 ms, individually for every pattern A-D (thus patterns A-D differ only in the temporal order of the particular SC inputs, as illustrated for A and B in Fig.2.B). single PP input is supposed to transmit a seq.of spikes (15 ± 3 spikes, freq. 8Hz, random onset t on [0,300]ms) whenever pattern A is presented at the SC inputs. 4

Results: Results of the experiment are presented in Fig.2.B. For clarity we show the case for pattern A (left column) and for one of the not-associated patterns only (pattern B, right column). Initially, PC is not selective to any pattern (Init State). The association of the PC s activity with pattern A can be established already in one-shot (, left graph). The learning leads to the crosstalk effect - PC starts to fire also in response to pattern B (, right graph). During the retrieval phase ( III.a and III.b) the results are purified through mstdp - PC s activity in response to pattern A is maintained, while neuronal responses to B are gradually suppressed. Analysis of the synaptic weights observed after the experiment (bottom graphs) reveals high selectivity of ITDP - only several SC inputs are potentiated, while other inputs remain relatively weak. 3.2 Pattern discrimination via rate coding Experiment description: PC is trained to respond selectively to patterns A and C. Rate coding scheme is used to further distinguish between both patterns, i.e. PC is supposed to fire with firing rates individually assigned to the particular associated input patterns (Fig.3.A). Methods: 1000 SC inputs; the particular SC inputs are supposed to fire sequences of spikes (15±3 spikes) with frequency 7.5±1Hz; for every SC input we randomly set a firing onset t on in the range of 0 to 300 ms, individually for every pattern A-D. PP input: 15 ± 3 spikes, frequency 7 or 8Hz, random onset t on [0,300]ms. It is assumed that a PP spike sequence with frequency 8Hz (7Hz) is temporally correlated with SC pattern A (C). Results: Results of the experiment (Fig.3) confirm that PC can be trained to respond to the selected pattern with the target firing rates defined by the PP input (Fig.3.B). Deviations from the target rate observed in PC (Fig.3.C) can be minimized by increasing the number of decorrelated SC inputs (results not shown). 3.3 Pattern discrimination via phase coding Experiment description: Similarly as in the previous experiment PC is trained to respond selectively to patterns A and C. Here, however, a spike-timing code is used to reinforce PC to fire precisely at the specified theta phase (Fig.4.A). The task is to maintain the relative phase of the PC firing times in response to A and C. Methods: 1000 SC inputs; the particular SC inputs are supposed to fire sequences of spikes (15±3 spikes) with frequency 8Hz; for every SC input we randomly set a firing onset t on within a single theta cycle, individually for every pattern A-D. PP input: 15 ± 3 spikes, firing rate: 8Hz. The firing onset is set such that PP fires at phase φ A =0 o w.r.t. the theta-cycle whenever pattern A is presented, and with φ C = 120 o for pattern C, thus the target relative phase φ trg CA = (φ C φ A ) = 120 o. 5

Information coding via firing rate A SC PP PC A B C D A B C D B Pattern A PP freq.= 8 Hz C 8.00Hz 8.03Hz PP freq.= 7 Hz firing rate [Hz] 8.0 7.5 7.0 6.5 6.0 7.00Hz 7.24Hz PP PC PP PC Pattern A Pattern C Pattern C Figure 3: Spatio-temporal pattern discrimination with rate-coding. (A) Discrimination of the selected input patterns A and C is communicated by firing sequences of spikes with the predefined firing rates, assigned to the particular input patterns. (B) Here PC is trained to respond with rate 8Hz to pattern A, and with rate 7Hz to pattern C. (C) Results observed in PC (over all retrieval steps) are close to the target firing rates defined by PP. Results: We observe (Fig.4.B) that φ trg CA is maintained by PC, both, during the coding stage as well as during the retrieval. This is confirmed also in Fig.4.C., where we present mean values and standard deviation of φ CA calculated for every pair of the corresponding spikes recorded in PC in response to A and C patterns (mean and var. is calculated over all retrieval steps presented in Fig.4.B). It is observed that on average φ CA = 117 ± 6 o, which closely matches the target value. 3.4 Learning with single instructive spikes Experiment description: In this experiment we assume that PP transmits low-frequency, isolated action potentials [5]. We investigate whether the particular single EPSPs evoked by the PP input are able to selectively associate PC s activity with the ongoing SC activity at times specified by the PP EPSPs (Fig.5). Methods: 1000 SC inputs; the particular SC inputs are supposed to transmit one or more sequences of spikes (15±3 spikes; frequency 8Hz); the onset times of the particular sequences are generated randomly according to the Poissonian distribution. PP input: isolated spikes, Poissonian stimuli with frequency 0.1Hz. Results: Results of this experiment (Fig.5) confirm our prediction that the isolated PP EPSPs are able to associate the PC activity with the SC patterns temporally correlated with the particular PP EPSPs. We observe that the PC responses to the these patterns are enhanced in the consecutive coding and retrieval steps. These results correspond well with the observed expansion of place fields after several passes across the given fields [10]. Plot of the CA3-CA1 synaptic efficacies (Fig.5, bottom graph) shows which of the SC inputs are associated with the PC activity (the strongest connections indicated by the highest bars). 6

Information coding via theta-phase A SC PP PC I. II. A B C D A D C B B Pattern A o PP phase: A = 0 1 2 3 4 5 6 7 8 9 10 C relative phase [deg] o 180 PC ( C- A) = 117 6 120 60 0 1 2 3 4 5 6 7 8 9 10 Pattern C o PP phase: C = 120 1 2 3 4 5 6 7 8 9 10 Figure 4: Spatio-temporal pattern discrimination with rate-coding. (A) Presentation of the selected input patterns is communicated by PC through sequences of spikes fired at the fixed phase with respect to the theta cycle. (B) PC is trained to fire at phase φ A = 0 o in response to pattern A, and with φ C = 120 o in response to pattern C. (C) The target relative phase (φ C φ A) = 120 o is preserved along all spikes in all retrievals. 4 Discussion Results of the presented experiments reveal interesting learning properties of the considered plasticity model. We demonstrated the ability of our model to discriminate different input patterns using several neural information coding schemes, such as: firing/not firing code, rate coding or phase coding. We investigated also two learning paradigms where PP signals were assumed to have a form of sequences of spikes or isolated spikes. In both cases learning successfully led to the establishing of the intended associations. As demonstrated in Section 3.1 our model can also account for a single-shot learning phenomenon suggested for the hippocampus [15]. Due to the specific properties of the mstdp model, our learning rules reinforce bursting PC patterns, while suppressing isolated low-frequency spikes. This properties can potentially be utilized in the model of learning to erase memory of the CA1 neurons. Synaptic connections potentiated during the learning phase can be depressed again e.g. if the PC neuron is driven with low-frequency SC patterns. One prominent source of such prolonged low-frequency activity in the hippocampus could be slow-wave sleep and quiet wakefulness [9]. Limitations: Despite interesting learning properties our model has also some limitations: there is no mechanism to dissociate undesired associations (induced e.g. due to the crosstalk effect) - ones a neuron starts to fire sequences of bursts in response to some given spatiotemporal input pattern, there is no mechanism to suppress this activity, the model is sensitive to the parameter controlling the maximal efficacy of the particular SC synaptic inputs - in the recent implementation such a parameter had to be tuned individually for each experiment in order to obtain stable learning. Further work: In this report we presented just a first, modest approach to modelling supervised learning in the CA1 region. Further work is needed to implement the following, important extensions in our model: detailed models of CA1 pyramidal cells, their interneurons and the local microcircuitry, plasticity models for all synaptic connections (not only for SC inputs), biologically more realistic activity patterns for SC and PP inputs. 7

Assoc. A Assoc. B Assoc. C SC input Init state 2 s PP input PC activity Synaptic weights 20 nv 0 100 200 300 400 500 600 700 800 900 1000 synaptic inputs Figure 5: Supervised learning with single instructive spikes. PC neuron receiving a continuous stream of SC inputs is supposed to selectively respond to the ongoing input activity at the times specified by the PP spikes. Initially PC fires isolated spikes spread out uniformly throughout the whole simulation time. After the memory storage phase () PC exhibits high activity around the times specified by the PP input. These PC responses are further enhanced after every retrieval step. Bottom graph: CA3-CA1 synaptic efficacies observed after the experiment. Acknowledgement This project is carried out in collaboration with Dr Joshua T. Dudman. from the Howard Hughes Medical Institutes at the Janelia Farm Research Campus. The work was partially supported by the German Federal Ministry of Education and Research (grant 01GQ0420 to BCCN Freiburg). References [1] E. Buhl, K. Halasy, and P. Somogyi. Diverse sources of hippocampal unitary inhibitory postsynaptic potentials and the number of synaptic release sites. Nature, 368:823 828, 1994. [2] D. Debanne, B. H. Gähwiler, and S. M. Thompson. Cooperative Interactions in the Induction of Long-Term Potentiation and Depression of Synaptic Excitation Between Hippocampal CA3-CA1 Cell Pairs In Vitro. Proceedings of the National Academy of Science USA, 93:11225 11230, 1996. [3] D. Debanne, B. H. Gähwiler, and S. M. Thompson. Bidirectional Associative Plasticity of Unitary CA3-CA1 EPSPs in the Rat Hippocampus In Vitro. Journal of Neurophysiology, 77:2851 2855, 1997. [4] J. T. Dudman, D. Tsay, and S. A. Siegelbaum. A role for distal synaptic inputs: instructive signals for hippocampal synaptic plasticity. Neuron, 56(5):866 879, 2007. [5] L. Frank, E. Brown, and M. Wilson. A comparison of the firing properties of putative excitatory and inhibitory neurons from CA1 and the entorhinal cortex. Journal of Neurophysiology, 86:2029 2040, 2001. [6] W. Gerstner and W. Kistler. Spiking Neuron Models. Single Neurons, Populations, Plasticity. Cambridge University Press, Cambridge, 2002. [7] A. Heynen, W. Abraham, and M. Bear. Bidirectional modification of CA1 synapses in the adult hippocampus in vivo. Nature, 381:163 166, 1996. 8

[8] D. Johnston, B. R. Christie, A. Frick, R. Gray, D. A. Hoffman, L. Schexnayder, S. Watanabe, and L.-L. Yuan. Active dendrites, potassium channels and synaptic plasticity. Phil. Trans. R. Soc. Lond. B, 358:667 674, 2003. [9] A. Lee and M. Wilson. Memory of sequential experience in the hippocampus during slow wave sleep. Neuron, 36:1183 1194, 2002. [10] M. Mehta, M. Quirk, and M. Wilson. Experience-dependent asymmetric shape of hippocampal receptive fields. Neuron, 25:707 715, 2000. [11] T. Mizuno, I. Kanazawa, and M. Sakurai. Differential induction of LTP and LTD is not determined solely by instantaneous calcium concentration: an essential involvement of a temporal factor. European Journal of Neuroscience, 14:701 708, 2001. [12] F. Pike, R. Meredith, A. Olding, and O. Paulsen. Rapid report: postsynaptic bursting is essential for Hebbian induction of associative long-term potentiation at excitatory synapses in rat hippocampus. Journal of Physiology (London), 518:571 576, 1999. [13] F. Ponulak. Supervised Learning in Spiking Neural Networks with ReSuMe Method. PhD thesis, Institute of Control and Information Engineering, Poznań University of Technology, Poland, 2006. Available from: http://d1.cie.put.poznan.pl/~fp/. [14] F. Ponulak and A. Kasiński. Supervised Learning in Spiking Neural Networks with ReSuMe: Sequence Learning, Classification and Spike-Shifting. Neural Computation (under review), 2008. [15] S. Remy and N. Spruston. Dendritic spikes induce single-burst long-term potentiation. Proceedings of the National Academy of Sciences of USA, 104(43):17192 17197, 2007. [16] G. M. Wittenberg and S. S.-H. Wang. Malleability of Spike-Timing-Dependent Plasticity at the CA3 CA1 Synapse. The Journal of Neuroscience, 26(24):6610 6617, 2006. 9