Estimating Simple and Complex Cell Receptive Fields from Natural Image Stimuli and 2-Photon Imaging Recordings of the V1 in Ferrets

Transcription

1 Estimating Simple and Complex Cell Receptive Fields from Natural Image Stimuli and 2-Photon Imaging Recordings of the V1 in Ferrets Philipp John Frederic Rüdiger E H U N I V E R S I T Y T O H F G R E D I N B U Master of Science by Research Institute for Adaptive and Neural Computation School of Informatics University of Edinburgh 2011

2 Abstract The visual cortex in many species displays a columnar organisation into topographic domains, which was for a long time fundamentally associated with cortical computation. More recently experiments have shown that most rodent species do not display this topographic organisation of neurons into maps, calling unifying theories of cortical computation into question. The characteristics of individual neurons are defined by so called receptive fields, which can be visualised using a variety of estimation techniques. The receptive field properties of V1 neurons in have generally been assumed to vary little across species although no detailed comparison of entire neuronal populations have so far been performed. By extending two-photon calcium imaging protocols previously applied in mice at Tom Mrsic-Flogel s lab entire populations of neurons in the primary visual cortex can be imaged at single cell resolution and their responses to visual stimuli be recorded. This project develops the necessary procedures and protocols to perform receptive field estimation in ferrets, including the stimuli, pre-processing procedures and receptive field estimation techniques. Two novel stimuli are implemented in the laboratory software toolbox used by various labs including Stephen van Hooser s at Brandeis University. In addition, the development of pre-processing tools to convert the raw luminance signal from two-photon calcium imaging into the appropriate formats for further analysis is demonstrated. To complete the toolchain a variety of receptive field estimation techniques are tested on simulated and real datasets, including electrophysiology recordings from ferret and macaque and a single twophoton calcium imaging dataset from mouse recordings at Tom Mrsic-Flogel s lab. The ferret dataset was recorded as part of the visit to Stephen van Hooser s lab using the stimulus protocol that is part of this project. Moving forward this work provides the foundation to collect detailed datasets of V1 volumes in ferrets, which can be compared to already existing datasets of mice. The ultimate aim will be to provide clear data about the differences in functional organisation, development and information processing in species with and without topographic maps. i

3 Acknowledgements Many thanks to Steve van Hooser the PI of the lab at Brandeis University who graciously accommodated me and answered my constant questions. Thanks also to his postdoctoral fellow Arani Roy, who patiently explained to me the operation of the two-photon calcium imaging rig and let me observe the entire experimental procedure. I also wish to express my gratitude to Neil Ritter, the postgraduate student in the Brandeis lab, who prepared the electrophysiology experiment and gave me further insights into the experimental methods. Jan Antolik also deserves honorable mention having been involved in previous experiments at Tom Mrsic- Flogel s lab at UCL and providing essential help with the analysis code as well as invaluable advice about the viability of different approaches to analysis. Thanks to Tom Mrsic-Flogel for allowing me access to his two-photon calcium data recorded in mice and to the Collaborative Research in Computational Neuroscience group, who have provided open access to an electrophysiology dataset from the Dario Ringach lab at UCLA. Finally, special thanks to Jim Bednar for his kind and thoughtful supervision, which provided crucial guidance in the vast field of vision research. ii

4 Declaration I declare that this thesis was composed by myself, that the work contained herein is my own except where explicitly stated otherwise in the text, and that this work has not been submitted for any other degree or professional qualification except as specified. (Philipp John Frederic Rüdiger) iii

5 Table of Contents 1 Background Introduction The Visual System Primary Visual Cortex: Topographic Maps, Simple and Complex Cells Cortical Architecture: Development and Species Variability Recording Techniques Electrophysiology Classical Optical Imaging Techniques Two-Photon Excitation Calcium Fluorescence Imaging List of Aims Developing Experimental Procedures Stimuli Grating Stimuli Stochastic Stimuli Natural Image Stimuli Hartley Stimuli Summary Experimental Protocol Original Protocol Changes to the Protocol Pre-Processing Calcium signal pre-processing Electrophysiology Analysis Methods Spike Triggered Averaging (STA) Spike Triggered Covariance Wavelet Models Testing the Wavelet Model iv

6 2.4.5 Structure based models Experimental Work, Results and Analysis Experimental Work Results and Analysis Ferret Data Mouse Data Macaque Data Discussion and Conclusion Discussion Stimuli Experimental Protocol Pre-Processing Analysis Results Future Work Conclusion A Testing Computer Specification 57 Bibliography 58 v

7 List of Figures 1.1 The early visual pathway in primates. Reprinted from Solomon and Lennie (2007) The centre-surround receptive field structure of some retinal ganglion cells and LGN neurons. Adapted from Bear et al. (2007) Orientation preference map and magnifications thereof. Adapted from Bosking et al. (1997) Reconstruction of ocular dominance columns in the right occipital lobe of a macaque monkey. Adapted from Hubel and Freeman (1977) Gabor patches at different orientations Orientation preferences of single cells in layer 2/3 of the rat V1. Adapted from (Ohki et al., 2005) Development of orientation map in ferret visual cortex. Adapted from (Chapman et al., 1996) Table of species differences. Reprinted from (Van Hooser, 2007) Timing of development of major visual circuit properties in ferret and mouse. Reprinted from (Huberman et al., 2008) Horizontal connectivity in mammals with and without orientation maps. Reprinted from (Van Hooser, 2007) Diagram of two-photon calcium imaging (Provided by Tom Mrsic-Flogel) Stochastic Grid Stimulus GUI and trinary stimulus Natural Image Stimulus GUI Sample Natural Image Stimuli Spatial Contrast Sensitivity in different species (Reprinted from Uhlrich et al. (1981)) and Power Spectrum of Natural Images Natural Image filtered at two spatial frequencies Hartley stimulus basis set Hartley Stimulus GUI vi

8 2.8 Control Diagram of VHLab environment taken from internal documentation on the VHTools website VHLab Calcium signal extraction environment. Allows calcium imaging stacks to be loaded and cells to be identified, correcting for any drift and outputting the calculated luminance signal Filtered and unfiltered Calcium luminance signal Preprocessed luminance signal for a single neuron Spike rates extracted from sample time course using the method described in Vogelstein et al. (2010) Spike2 Software including Waveform Analysis and K-Means Clustering Visualisation of STA. Reprinted from Schwartz et al. (2006) Effects of Noise on STA Prediction Ridge Regression STA Noise Performance Visual representation of spike triggered covariance (STC). Reprinted from Schwartz et al. (2006) Spatio-temporal receptive field modelled using temporally and spatially shifted Gabor filters STA and STC and estimates of the complex cell STRF Gradient and coordinate descent optimisation of Gabor wavelet model Wavelets constructed from Gabor pyramid based on the natural image statistics Tuning curves of two ferret V1 neurons recorded by means of single cell electrophysiology at Stephen van Hooser s lab at Brandeis University Regularised Pseudoinverse STA receptive field estimates of ferret neurons Gabor wavelet model estimation of ferret receptive field Receptive field estimates of 102 cells in the mouse V Selected mouse receptive fields Comparison of selected mouse receptive fields for different methods Monkey receptive fields estimated using rpsta Comparison of Macaque receptive field estimates using rpsta and Wavelet Analysis vii

9 List of Tables 2.1 Average time per frame for the stochastic grids stimulus Mouse receptive field validation set correlations viii

10 Chapter 1 Background 1.1 Introduction When Vernon Mountcastle first described a columnar functional organisation of the cerebral cortex, it was assumed this was a universal feature of the mammalian cortex and seemed to provide the basis for a universal theory of cortical computation. The lack of sensory topographic maps in rats (Girman et al., 1999), mice (Metin et al., 1988) but also in the grey squirrel (Van Hooser et al., 2005), which is known for its high visual acuity, calls their function as a fundamental organisational principle into question. The precise differences in functional cell types and connectivity in species with and without topographic maps have so far not been very well described. New techniques in imaging and analysis now allow for in depth studies of the differences between these two groups to be carried out. Orientation maps are the most common sensory feature maps and have been intensely studied. Individual columns within the orientation maps exhibit very similar receptive field properties and respond similarly to visual stimuli with certain orientations. Receptive fields of visual neurons describe the spatial and temporal features in the visual space the neuron responds to and are used to classify visually responsive neurons as either simple or complex. While simple cells display clear ON and OFF regions within its receptive fields, either driving or supressing activity, complex cells exhibit phase invariance responding even when the stimulus is shifted within the neurons receptive field. Although the general organisational principle differs between species, the individual cells appear to have very similar characteristics. It is therefore unclear whether the organisation of neurons into fundamentally different arrangements are evidence of truly different visual processing mechanisms or whether similar functional connectivity is simply achieved by different means perhaps due to constraints placed on individual species in their evolutionary history. In order to truly answer this questions detailed datasets comparing functional cell properties, connectivity and developmental processes in species with and without topographic maps are 1

11 Chapter 1. Background 2 required. In the past such studies would have been carried out using electrophysiology, optical imaging or fmri all of which have properties that make it prohibitive to study entire neuron populations. At best electrophysiology can sample the cortex with a very coarse spatial resolution. Optical and fmri imaging both have very low spatial and temporal resolution, which make it impossible to even observe individual neurons. The advent of two-photon calcium imaging therefore truly provides a great leap forward as it allows activity from large populations neurons in a superficial cortical volume to be imaged at single cell resolution. Although the calcium transients to not provide the same temporal resolution as electrophysiology, techniques reconstruct spike trains are quickly being developed. Detailed recordings from mouse V1 volumes are already underway at Tom Mrsic-Flogels lab at the University College London. What is missing is a collaborator to collect data in another species, which does exhibit maps. This project aims to provide the necessary tools for the Stephen van Hooser lab and other labs, which share the same software toolkit, to collect equivalent datasets in ferrets. This Masters thesis details the development of the protocols necessary to record datasets from ferrets, the tools to pre-process the raw data and finally different methods to analyse the datasets in a meaningful way. The report also details the visit made to the van Hooser lab at Brandeis University to integrate the tools in their system and collect data in the V1 of ferrets using their electrophysiology equipment and two-photon microscope. Further, it will test different analysis techniques on simulated and real datasets presenting the preliminary results. Finally, the report will focus on further experiments that may be done using this protocol to better understand species variability of cortical maps, their development and their function in cortical processing. In the following sections a detailed introduction and background into the current literature on the visual system, cortical information processing, the functional and anatomical differences between species and different recording methods will be provided. 1.2 The Visual System The first stage of the early mammalian visual system (pictured in figure 1.1) can be found in the retina in the back of the eye, where two types of photoreceptors, the rods and the cones, convert the optical signal into electrical and chemical changes in cell membrane. The photoreceptors are activated as photons strike various opsin proteins causing conformational changes thereby altering the flow of ions in and out of the membrane and increasing or decreasing the membrane potential of the cell. This signal is then sent down synaptic connections via retinal bipolar and ganglion cells, down the optic nerve to the lateral geniculate nucleus (LGN). Connections from

12 Chapter 1. Background 3 the two eyes cross over at the optic chiasm to form projections of the right and left visual field contralaterally, so that information about the left half of the visual field is projected to the right hemisphere and vice versa. Even as information from each half of the visual field is sent to the contralateral hemisphere, the streams of from each eye remain separated in the LGN and are only combined at later stages of the visual system. The connections from the retina map retinotopically onto the LGN, which ensures that nearby areas of LGN respond to nearby portions of the visual field. After the initial processing in the retina and LGN the visual stream is projected onto the primary visual cortex (V1). Retinal processing begins, as previously mentioned, with the photoreceptors, which release glutamate into the synaptic terminal connecting them to bipolar or horizontal cells. The bipolar cells respond either as ON or OFF cells as the glutamate either de- or hyperpolarizes them basically turning on or off in response to light stimulation. The bipolar cells can make connections to a small or large number of photoreceptors or even be indirectly connected to them via horizontal cells, which leads to a variety of receptive field sizes. The term receptive field, in this context, refers to the area of the visual field the cell responds to when stimulated with light and is extended to include its spatial and temporal characteristics. This can also give rise to so called centre-surround receptive fields in which a central field is combined with an antagonistic peripheral field, where either the centre or surround field responds to light and the other is suppressed by light. These more complex ON-centre and OFF-centre receptive fields can arise in bipolar cells but are more commonly associated with retinal ganglion cells (RGCs). This receptive field type responds most strongly to spots of light/dark moving through the visual field (as shown in figure 1.2). The ganglion cells provide the only output from the retina to the rest of the brain and connect to their thalamic targets in the LGN through very long axons, which form the optic nerve. Much like the retina and many other areas of the brain, the LGN forms a laminar architecture, receiving input from RGCs and projecting axons to the primary visual cortex via the optic radiation. The separation of LGN cells into layers also corresponds to functional separations, as layers 1, 4 and 6 usually receive contralateral input, while layers 2, 3 and 5 receive ipsilateral inputs from the retina. Furthermore, different layers consist of different cell types, with ventral layers 1 and 2 containing larger so called magnocellular (M) neurons and dorsal layers 3, 4, 5 and 6 containing smaller parvocellular (P) neurons with intralaminar neurons being referred to as koniocellular (K). Since these three cell types are also present in the retina and make connections mainly with their own cell type the idea that each carries its own parallel information stream has been posited. Functionally, the M-cell has a larger receptive field than the P-cell and responds with transient rather than persistent activity. In addition to the feedforward retinal input, the LGN also receives feedback connections from V1, the influence of which has not yet been fully clarified.

13 Chapter 1. Background 4 Optic radiation V1 Layer V A 4B 4C 4C 5 6 White matter Koniocellular Magnocellullar Parvocellular LGN input LGN Ipsilateral input Magnocellullar Koniocellular Parvocellular Optic tract Optic nerve Chiasm LGN Retinal nerve cells Pigmented cell Rod Photoreceptors Cone Horizontal cell Bipolar cell Amacrine cell Ganglion cell Figure 1.1: The early visual pathway in primates(at least superficially the same for most other mammals) from the retina to the primary visual cortex (V1) via the lateral geniculate nucleus (LGN) of the thalamus. The left panel shows the pathway, while the right panels highlight noteworthy sections including the structure of the retina, the LGN and V1 broken down into their different layers and showing different cell types. Adapted from Solomon and Lennie (2007).

14 Chapter 1. Background 5 Figure 1.2: The centre-surround receptive field structure of some retinal ganglion cells and LGN neurons, illustrating how a contrast edge activates different portions of the field and thereby results in different activation patterns. From left to right one can see that as the light-dark edge moves into the ON surround field spontaneous activity is suppressed and as it moves further over the OFF centre field is deactivated causing activity to sharply spike. Adapted from Bear et al. (2007). 1.3 Primary Visual Cortex: Topographic Maps, Simple and Complex Cells The primary visual cortex (V1) or striate cortex provides the first cortical stage of processing of visual information. The cortex was classically divided into six layers by Korbinian Brodmann but since many subdivisions have been added after functional sub-groups were discovered. The exact architecture has been studied thoroughly and is described in (Binzegger 2004; Chalupa 2008; Lund 1988) Van Hooser (2007); Douglas and Martin (2004). Although the gross inter-, intra- and extracortical connectivity is quite uniform across species, a variety of differences have been uncovered particularly in the abundance, distribution and fine connectivity of different cell types. Generally input from the LGN is received in layer 4, still divided laminarly into M, P and K cells. The cells in layer 4 then send projections up to layer 2/3, which has a diverse lateral intralaminar network of connection but also sends intracortical projections to higher visual areas such as V2, V3, V4 and MT. The neurons in layers 5 and 6 provide feedback to the LGN, for reasons that are not yet understood. Neurons in the primary visual cortex (V1) are tuned to respond to a variety of different features or complex combinations of such features, including orientation, spatial and temporal frequency, motion direction, colour or ocular origin. In many mammal species especially primates and carnivores, these response preferences map smoothly and topographically onto the cortical surface. This mapping extends vertically through the layers of the cortex, such that nearby neurons respond to similar input properties, giving rise to the notion of distinct cortical columns. Retinotopy arises due to the mapping of visual information straight from the retina to the LGN and then to the cortex. Other response preferences such as orientation and direction selectivity rarely arise in the LGN and are usually thought of as an emergent phenomenon in

15 Chapter 1. Background 6 Figure 1.3: A) Orientation preference map in a ferret generated by overlaying the activity maps for different orientations and artificially colouring each area according to the orientation preference laid out in the legend below. The image also highlights three recurrent features of orientation maps in white. The square highlights a saddle point, where a patch of cortex selective for a particular direction is almost bisected by a patch selective to another direction. The circle highlights a pinwheel arrangement, where different orientations preference patches are arranged in a circular shape. Finally the rectangular shape highlights a linear zone in which orientation preference change continuously. B) Magnifications of a linear zone and two pinwheel arrangements. Adapted from Bosking et al. (1997). the cortex. Since their discovery, they have become the target of intense study since it was long thought that they revealed something deeper about the computational processes carried out by the cortex. However, more recent discoveries has shown that such topographic mapping is not present in all species including rats (Girman et al., 1999), mice (Metin et al., 1988) but also in the highly visually acute grey squirrel (Van Hooser et al., 2005), which has thrown the idea of cortical columns as a functional organisational principle into question and has led some to suggest that it is in fact a structure without a function (Horton and Adams, 2005). Research into cortical columns and topographic maps has been extensive especially since it seemed to offer a unifying principle for understanding cortical function. Since Mountcastle first discovered them (Mountcastle, 1957), the main technique for mapping cortical columns has been using optical imaging techniques in conjunction with activity sensitive dyes and grating or sinusoidal stimuli. Techniques since have been refined and will be discussed in more detail in later sections. Today we have highly detailed maps of orientation preference (as shown in figure 1.3) and other topographic mapping phenomena such as ocular dominance columns (figure 1.4) and direction selectivity. The receptive fields of V1 neurons are different in that they are no longer simple ON- or

16 Chapter 1. Background 7 Figure 1.4: Reconstruction of ocular dominance columns in the right occipital lobe of a macaque monkey. Prepared by staining using the silver method of Liesegang and taking serial sections tangential to the cortical surface of the occipatal lobe. Adapted from Hubel and Freeman (1977). OFF-centre surround fields, forming more complex spatio-temporal patterns. They are commonly modelled using Gabor filters as shown in figure 1.5 which have elongated ON and OFF regions, achieved by localising a full-field sine grating with a Gaussian envelope. Orientation selectivity and spatial frequency preference are determined by the ON- and OFF- region angles and spacing respectively. It is also possible for V1 cells to filter temporal patterns by employing spatio-temporal shifts in their ON and OFF regions, giving rise to direction selectivity. Orientation selective neurons can generally be classed as simple or complex cells, depending on whether they display some form of spatial invariance. In reality this classification is less clear with cells being somewhere on a gradient from pure simple cells to a complex cell with the degree of spatial invariance being the determining factor. Apart from spatial invariance the neurons may also exhibit contrast invariance, such that even at very low contrast they will respond more strongly to their preferred orientation than to the orthogonal orientation. While functional cell types are conserved across species their distribution and organisation are not. 1.4 Cortical Architecture: Development and Species Variability The general structure of the primary visual cortex may be shared across mammalian species; nonetheless there are some significant differences in the exact organisation of neuron distribution and connectivity. The most prominent difference between species is the existence of topo-

17 Chapter 1. Background 8 Figure 1.5: Gabor Patches at 0 degree and 90/180 degree orientations with clearly visible ON (white) and OFF (black) regions. graphic maps and cortical columns in predatory mammals (Hubel and Wiesel, 1963; Mountcastle, 1957; Ohki et al., 2005) and primates (Hubel and Wiesel, 1968) and their apparent lack of existence in rodents (Metin et al., 1988; Ohki et al., 2005) and lagomorphs. Determining what processes drive the development of these different architectures may go a long way to explaining how alternate functional connectivity arises in different species. Developmental studies involve imaging the same area of the cortex over a number of days and investigating what drives development of orientation maps and other topographic arrangements. The ferret is a particularly common model in developmental work since they are born in a relatively early developmental stage. Early developmental studies found, using relatively limited optical imaging techniques on ferrets shortly after eye opening, that the iso-orientation domains in the V1 develop very early in development and subsequently show very little change (Chapman et al., 1996) (pictured in figure 1.7). These and other experiments (White and Fitzpatrick, 2007) showed that orientation preference develops even in absence of visual input although the maps do not fully mature, while this does not happen when the eye lids are sutured. This seems to suggest that orientation maps and other topographic organisations develop initially even in absence of visual input through genetic determination but then require external stimulation to fully mature. In the initial stages of development various preprogrammed guidance cues set up the basic connectivity between the LGN and the cortical processing areas. Some successful models of development have focused on the afferent connections between the LGN ON and OFF cells and their targets in the visual cortex as the driving force behind the development of orientation columns (Jin et al., 2011). These wiring processes are driven by various axon guidance cues and other molecules. By setting up the molecular gradients axons in the LGN make connections with their cortical targets in the V1. There has even been speculation that retinal ganglion cell mosaics guide the wiring between thalamus and cortex, setting up basic feature maps (Ringach, 2007). Later, but still starting in prenatal development, retinal waves, consisting of periodic ac-

18 Chapter 1. Background 9 Figure 1.6: Orientation preferences of single cells in layer 2/3 of the rat V1 imaged using in vivo two-photon calcium imaging. Visually responsive cells coloured according to their preferred orientation. Scale bar = 100 µm. Adapted from (Ohki et al., 2005). Figure 1.7: Development of orientation map in ferret visual cortex from postnatal day 31 to 42 revealed using chronic optical imaging of intrinsic signals. Adapted from (Chapman et al., 1996).

19 Chapter 1. Background 10 tivity in retinal ganglion cells (RGCs), spread across the retina driving neighbouring RGCs to fire in a correlated fashion. Evidence suggests that these may not only drive the development of retinal circuits but are also responsible for effecting the ocular separation of inputs to the LGN and V1 (Firth et al., 2005). In the context of correlation based theories of development this makes sense since uncorrelated activity in the two retinas would effect exactly this kind of separation in the LGN and V1. When modelling development based feedforward effects of retinal waves it is very difficult to predict actual patterns of spontaneous activity, which is to be expected since the LGN only receives a relatively small proportion of its inputs from the retina, while intrinsic interneurons, inhibitory inputs from the thalamic reticular nucleus and feedback connection from layer 6 of the V1 make up the majority of its afferents. The balance between the main drivers of functional map development including orientation columnation is still unclear. The most prominent proposals in theory and in models have been focused the termination pattern of both the geniculocortical afferents in layer 4 (Katz et al., 2000; Ringach, 2007) and adjustments in lateral connectivity through activity dependent, competitive processes (Bednar and Miikkulainen, 2003). While the emergence of orientation selectivity isn t at all settled the processes involved in maturation of the maps are generally accepted to be driven by activity dependent weight modification in form of Hebbian learning or some variant thereof. Since traditional Hebbian learning seems to result in unstable development due to positive feedback, more recently, homeostatic plasticity has successfully applied in developmental models to replicate some of the effects observed in experiments (Law, 2009). These models can account for the development of orientation preference maps through both intrinsic activity including retinal, spindle and PGO waves of REM sleep (Bednar and Miikkulainen, 2003) and visually induced retinal activity. Experiments show that at the very least these processes are required to achieve the nely tuned precision, which can now be observed at single cell resolution (Ohki et al., 2006; White and Fitzpatrick, 2007). Further experiments will have to resolve the degree to which each of these processes is responsible for orientation maps, which may well resolve why they appear in some species but not others. The functional architecture behind orientation maps should theoretically allow one to explain the differences between species and determine whether they share common processing and development algorithms. Comparisons between single neuron properties show very little variability as the table compiled by Van Hooser shows (figure 1.8). The orientation tuning and selectivity are highly similar across species suggesting that any differences must be in their distribution and connectivity. Although there s some variability in the distribution of cell types across species (figure 1.9) these differences largely reflect the different visual requirements of species such as the large abundance of direction selective cells in predatory animals. There s no clear distinction in the distributions of cells between animals with and without topographic

20 Chapter 1. Background 11 Figure 1.8: Table of species differences. Reprinted from (Van Hooser, 2007). maps. That leaves the connections as the one factor, which may truly distinguish species from another. Apart from the vertical intercortical connection, lateral connections have been found to extend intra-laminarly (Gilbert and Wiesel, 1983) and may underlie the development of orientation columns to some degree. Experiments in layer 2/3 of the tree shrew involving orientation preference mapping and subsequent axonal staining have shown that although short range connections show no preference in their terminations, long range connections longer than 500 µm, preferentially link neurons with co-oriented and co-axially aligned receptive fields (Bosking et al., 1997). In iso-orientation regions cells therefore make short-range connections largely with cells that prefer the same direction as them, while the connectivity at pinwheels shortrange connections are made with cells with a wide range of orientation preferences (Ohki et al., 2006). This is highly similar to the connectivity found in animals without maps, where spike timing correlation studies have shown a strong relationship between correlation strength and distance between cells but none between correlation strength and orientation preference (Van Hooser et al., 2006). The stylised drawing of connectivity in figure 1.10 shows a variety of possibilities with regard to the horizontal connectivity in mammals with and without maps. It is a possibility that long range horizontal connections, which selectively link neurons with similar orientation preferences make up for the lack of short range connections of this kind in rodents. Until recently there was no indication this was the case but a very recent study found that in mice uni- and bi-directional connections were more likely to be present between V1 layer 2/3

21 Chapter 1. Background 12 Figure 1.9: Timing of development of major visual circuit properties in ferret and mouse. Retinal waves are classified as mediated by: Blue - cholinergic, gap-junction activity; Turquoise - Nicotinic Acetylcholine receptor-mediated activity: Orange - Glutamate receptor activity. Black icons in yellow bar represent time in which visually evoked activity can be elicited through the closed eye-lids. Reprinted from (Huberman et al., 2008). pyramidal cells that had strongly correlated visual responses (Ko et al., 2011). Further work will be necessary to determine whether the difference in connectivity between species means they have fundamentally different ways of processing the visual information. The problem with addressing the above question is that the exact function of horizontal connections is far from settled since it is still unclear whether they contribute directly to the tuning properties of cells or if they serve to modulate the responses of cells. Some developmental evidence suggests that they do not directly affect orientation maps since afferent connections from the LGN arrive in V1 before the horizontal connections begin to form clusters (White and Fitzpatrick, 2007) and are therefore merely involved in maturing the already existing map. In summary there are a variety of processes that drive the development of the early visual pathway ranging from predetermined connectivity patterns to Hebbian processes activated through intrinsically and extrinsically stimulated activity with differences in both among species. This is reflected by the wide range of modelling work that is currently being done in this field. Further evidence informing the models in regard to the function of lateral connections is therefore desperately needed and may well be solved by applying some of the new imaging methods in combination with single cell tracing or spike-response correlation techniques to identify the underlying architecture and provide an in depth description of individual neurons.

22 Chapter 1. Background 13 Figure 1.10: Horizontal connectivity in mammals with and without orientation maps. A) In animals with orientation maps short-range connections seem not to be selective for orientation selectivity with the exception of cells in iso-orientation regions. Longer range connections have however been shown to make patchy connections to regions with similar orientation selectivity. B) In mammals without maps local short-range connections seem to be unspecific much like at pinwheels and orientation selectivity fault lines in mammals with maps. So far no conclusive evidence has been found for long range connections, which selectively target cells with similar orientation preferences. Reprinted from (Van Hooser, 2007).

23 Chapter 1. Background Recording Techniques Although the different techniques to functionally characterize individual cells and functionally map their response properties across the surface of the cortex have already been mentioned in passing it s important to describe how they differ and what their particular advantages and disadvantages are Electrophysiology Electrophysiology is the oldest and most common technique for functionally characterizing neurons in vivo and in vitro. It was famously pioneered by Hodgkin and Huxley (Hodgkin and Huxley, 1952) and later used by Hubel and Wiesel to characterize an orientation selective cells in the V1 of the cat and the macaque(hubel and Wiesel, 1968). It is used extensively to this day in various forms. Electrophysiology can be separated into intra-cellular and extra-cellular recordings, where electrodes are placed either inside the cell membrane or in the extra-cellular medium respectively. Intra-cellular recordings are very useful for characterising membrane currents or changes in potential but are very hard to perform in vivo and are therefore not of much interest with regard to this project. Extra-cellular recordings pick up the rapid changes in membrane potential during a spike from one or multiple cells using one or multiple microelectrodes, which are placed near the cell. It can therefore be further divided into single-unit and multi-unit recordings. The high temporal resolution of electrode recordings allow spikes to be picked up with unparalleled precision, which is essential for the use of spike time correlation to establish the functional connectivity between cells. Multi-electrode recording in particular has made huge advances as large scale multi-electrode arrays and columns are now available, which allow recording from entire patches of tissue or numbers of neurons in a cortical column. Therefore it is still a widely used technique today and has been successfully applied to receptive field estimation in a number of species (David et al., 2004; Felsen et al., 2005; Nauhaus and Ringach, 2007; Touryan et al., 2005) Classical Optical Imaging Techniques Classical optical imaging of the cortical surface relies on the fluctuation of intrinsic signals of the brain surface to determine activity. Since it s not a direct measure of activity the temporal resolution of this method is particularly low compared to other methods. The spatial resolution is a large step up from electrophysiology, which can at best sample the cortical volume with a very crude spatial resolution. For a long time it was one of the best methods to determine orientation maps in animals. It also provides the benefit that it isn t as invasive as other methods and therefore lends itself to developmental studies. The intrinsic signals being recorded are

24 Chapter 1. Background 15 changes in surface properties such as refraction, absorption and fluorescence, which vary due to blood oxygenation and other activity dependent phenomena. It was used to obtain the orientation preference maps in a number of species (Bosking et al., 1997; Chapman et al., 1996) but has now largely been superseded by measurement of extrinsic signals such as Ca 2+ sensitive dye using two-photon calcium imaging, which provides better temporal and spatial resolution in addition to an extra imaging plane allowing entire volumes of cortex to be imaged Two-Photon Excitation Calcium Fluorescence Imaging Two-photon excitation microscopy, pioneered by Denk et al. (1990), relies on the excitation of a fluorophore using a laser, which allows entire volumes of cortical surface to be imaged at single cell resolution. The concept works by exciting a fluorophore using two low energy photons, which strike at almost the same point in time and cause the fluorophore to release a higher energy photon. By focusing a laser beam which provides the necessary photons very high rejection ratios of out of focus signals can be achieved resulting in single cell resolution. Furthermore, the technique can image up to a depth of anywhere between 400 µm and 1 mm, providing three-dimensional image stacks of the cortical surface and several layers beneath it several hundred micrometres wide in all dimensions. The technique works with a variety of fluorescent dyes but the most relevant to the study of receptive fields is calcium sensitive dye first used by Stosiek et al. (2003). A schematic diagram of a two-photon imaging rig is shown in Figure The protocol of two-photon calcium imaging begins with the preparation of the animal, by putting it in a slightly anaesthetised but stable state. A craniotomy is then performed and a membrane-permeable calcium-sensitive fluorescent dye is injected into the extracellular space of the cortex using a micropipette, from where it diffuses into cells (Stosiek et al., 2003). The microscope is then focused on the region of interest scanning the cortical volume providing a high-resolution stack of images from which the calcium signal can be extracted and either be analysed in its raw form or be pre-processed to extract spikes using a non-negative deconvolution technique (Vogelstein et al., 2010). This technique provides a huge leap forward for imaging studies of orientationally selective cells in mammals with and without orientation maps since it allows imaging at the map level with single cell resolution. It has already led to several new insights since its relatively recent introduction. It has provided classifications of individual neurons in specific features of orientation maps such as pinwheels (Ohki et al., 2006) and provided insights into the previously poorly studied local connections of mice V1 neurons (Ohki et al., 2005; Smith and Häusser, 2010). In addition, developmental studies have been successfully demonstrated (Rochefort et al., 2011) providing great insights into the development of direction selectivity in mice, which appear to be largely developed pre-natally. By combining the technique in vivo and in vitro with genetic labelling of cell types, functional and anatomical features can be studied at an unprecedented

25 Chapter 1. Background 16 Figure 1.11: A) Diagram of two-photon calcium imaging showing dye loaded cortical surface and the objective. B) Imaged cortical volume. C) Two-dimensional patch taken from the imaged stack of cortical volume, with three labelled neurons. D) Luminance signal extracted from the three labelled cells. (Provided by Tom Mrsic-Flogel at UCL) level (Ko et al., 2011). The potential of two-photon calcium has only just started to be tapped and will provide the data necessary to bring about a better understanding of the development and operation of the cortical micro-architecture in different mammalian species. 1.6 List of Aims The main focus of this project is to provide a complete toolchain from stimulus presentation and recording to the data analysis to be deployed in a number of labs with the future purpose of collecting systematic two-photon measurements of visual responses in networks of V1 neurons at various stages of development. The ultimate goal is to determine the exact differences in receptive field properties and connectivity patterns between species, although no specific answers can be expected within the short duration of this project. The goal of this project can be broken down into several aims: 1. Program, optimize and integrate the most important stimuli for receptive field estimation into the lab software toolbox of Steve van Hoosers lab at Brandeis University, which will potentially also be distributed to a number of other labs including those of David Fitzpatrick and Sacha Nelson. 2. Observing the experimental procedures involved in electrophysiology and two-photon calcium imaging experiments and recording data in response to the programmed stimuli, if possible. The exposure to experimental neuroscience being a major requirement of the Doctoral Training Program in Computational Neuroscience and of the Masters thesis thereof.

26 Chapter 1. Background Obtaining, programming and applying the necessary tools to convert the raw data into formats accepted by the analysis techniques and thereby demonstrating the ability of software development in a scientific context. 4. Evaluating different receptive field estimation techniques such as standard and regularized STA methods, higher order STC analysis and other linear and non-linear estimation models using datasets from mice obtained in Tom Mrsic-Flogel s lab, data recorded as part of the visit to Brandeis and other openly available V1 datasets. 5. Providing a roadmap of questions which may be addressed and ideas that could be tested on the basis of forthcoming data. This project therefore encompasses a wide variety of tasks ranging from Experimental to Computational Neuroscience and Neuroinformatics, demonstrating the ability to perform scientific research in the converging fields of neuroscience and informatics.

27 Chapter 2 Developing Experimental Procedures The background and literature review in the previous section has provided a thorough account of the current understanding of early visual processing and its development in different species. This section will expound the experimental and analysis techniques involved in experiments such as these and detail the development of an entire toolchain from stimulus presentation to analysis. This work will lay the foundation to perform detailed studies on and rigorously evaluate the differences in the receptive fields of mammals with and without orientation maps. The VHLab toolbox developed in the Fitzpatrick lab at Duke University and the Nelson lab at Brandeis University, now also in use at the van Hooser lab at Brandeis University provides the framework to integrate stimulus presentation, response recording and some superficial analysis. Some basic stimuli, including stationary and moving gratings and a customizable grid of stochastic pixels were already prepgrogrammed. The initial challenge was to extend the toolbox to include natural image and Hartley stimuli, which have both shown to be useful in characterising receptive field properties. After detailing the development and integration of stimuli within the existing environment, the appropriate tools to convert both response and stimuli to the necessary file formats are presented. These include calcium signal extraction, averaging and filtering as well as basic conversion of various data types. Finally some analysis methods are evaluated using simulated data for simple and complex cells with and without noise, to then later be applied to real data collected from various sources. 2.1 Stimuli The stimulus provided to the visual system can greatly influence the response of the visual system so it is important to characterize and understand the advantages and disadvantages of using a particular stimulus types. Classically receptive fields have been mapped using simple two-dimensional gratings or even oriented bars, more recent computational techniques have used white-noise or other orthogonal stimuli sets to accurately reconstruct receptive fields. 18

28 Chapter 2. Developing Experimental Procedures 19 Only in the last few years have natural image stimuli become more commonly used spurred on by new receptive field estimation techniques. This section will detail the implementation of a variety of stimuli in the VHLab environment but also focus on the particular benefits of each stimulus type, how they may be useful and how they can be adjusted to appropriately drive visual response in different species. The VHLab toolbox relies on Psychtoolbox for stimulus presentations. Since it was developed when Psychtoolbox was still relatively new, it has a variety of backwards compatibility features, which had to be taken into account when programming new stimuli or updating the old ones. A few of the old stimuli needed to be updated to make use of new features, while ensuring maximum timing accuracy and frame rates. Older versions had used colour lookup table animation, which is no longer universally supported and runs into problems particularly when running in Windows. Such compatibility issues were one major concern when designing the stimuli. The newer texture animation required loading textures into memory, drawing them offscreen and then flipping the offscreen buffers on screen at the appropriate time. The system also includes triggers, which are sent to the recording equipment to pick up exact stimulus and frame times Grating Stimuli Grating stimuli come in a variety of forms ranging from simple bars as they were used by Hubel and Wiesel to stationary or moving sinusoidal gratings. Traditionally they were used simply to identify orientation selectivity of cells but have now also been applied to simple and complex cell receptive field estimation. Moving sinusoidal gratings have also become a common tool to study the development of direction selectivity (Li et al., 2008). More importantly they are used in the run up to experiments to identify the spatial position of a neuron in the visual space, which simplifies the later analysis of the data. The most common grating stimulus is the counter-phase sinusoidal grating also implemented in VHLab as periodicstim. It is described by the function: s(x,y,t) = Acos(KxcosΘ + KysinΘ Φ)cos(ωt) (2.1) where the spatial frequency is specified by K, the frequency by ω, the orientation by Ω, the spatial phase by Φ and the amplitude A. The gratings can either be stationary or moving and are usually randomly interleaved. The grating stimulus was already integrated within the VHLab environment and served as the template for the new texture drawing routines introduced in the more recent versions of Psychtoolbox.

29 Chapter 2. Developing Experimental Procedures 20 Figure 2.1: Stochastic Grid Stimulus GUI and trinary stimulus Stochastic Stimuli Stochastic stimuli or more specifically white noise stimuli are widely used in reconstructing receptive fields from neuronal responses. Since they have a flat power spectrum they allow different parameters of the neuronal receptive field to be driven with equal strength. This means that the stimulus makes no a priori assumptions about a neuron s behaviour and can be used in conjunction with reverse correlation and spike triggered averaging (STA) analysis to reveal receptive fields in a relatively small number of stimulus presentations. The disadvantages are that nearby pixels aren t correlated and therefore do not sufficiently drive the neurons response to correlated features as natural scenes might (Willmore and Smyth, 2003). The only requirement to the stimulus is that it is uncorrelated in both space and time such that: 1 T dts(x,y,t)s(x,y,t + τ) = τ s T 2δ(x x )δ(y y ) (2.2) 0 Since the actual pattern will be discretized in both space and time as pixels and frames respectively one can only ever produce an approximation, especially at higher frequencies. As long the the power spectrum remains flat in the bandwidth picked up by the neuron however this poses no problem (Dayan and Abbott, 2001). The VHLab environment already included a stochastic stimulus in form of stochasticgrid it was however programmed to run using the old colour lookup table animation routines, which are no longer fully supported in all operating systems. It was therefore necessary to update this stimulus to make use of the newer texture drawing routines while making sure it would still run with high timing accuracy and frame rates. The GUI (figure 2.1) was already existent in it s current form and specified all the necessary parameters. The parameters specified the size and position of the grid, the randomly generated colours, their relative probabilities, the angle of the grid on the screen, the size of each pixel, the number of frames and the frame rate. The idea was that it would generate a grid of pixels of adjustable size, each of which would ran-

30 Chapter 2. Developing Experimental Procedures 21 Resolution Pixel Size Average time/frame in s 800x x x x x Table 2.1: Average time per frame for the stochastic grids stimulus at the specified resolution and pixel size measured by averaging the load times over 100 repetitions using Matlab s tic toc functions. Demonstrates that depending on the parameters frame rates of between 4 Hz to 65 Hz are possible at the usual full screen resolution for stimulus presenation. For testing computer specification see appendix A. domly change to be one of three colours. This allows it to be programmed as either a binary or trinary white noise stimulus. The old drawing routine was preserved but for all newer versions of Psychtoolbox the stimulus was generated by loading textures into memory, just before it is displayed on screen. Although loading the next frame into memory while the previous is still being displayed limits the frame rate, it provides the only solution for long stimulus protocols with high resolutions without excessive memory use. In order to reduce memory and computational load whenever possible lower resolution textures were loaded into memory and a larger destination rectangle was sent to the drawing routine, which would automatically scale and interpolate, making up for the missing resolution. This results in huge performance gains as the pixel size increases as table 2.1 shows. This work has updated the old stochasticgridstim to use newer drawing routines and been optimized to generate white noise stimuli at high frame rates in real time Natural Image Stimuli Natural Images have only recently become useful in estimating the spatial and temporal receptive field structure, mainly due advances in analysis techniques, which have allowed more complex receptive field maps to be extracted from the data. Willmore and Smyth (2003) for example test several methods to reconstruct simple cell receptive fields in their 2003 paper and although they are generally more complex than classic reverse correlation techniques they provide better fits to the data and identify a variety of complex interactions, which cannot be studied using stochastic stimuli. The receptive fields also appear to activate in a different manner than when driven by synthetic stimuli, characterised mainly by stronger late inhibtion and shifts in spatial tuning (David et al., 2004). Since then a variety of effects have been observed

31 Chapter 2. Developing Experimental Procedures 22 Figure 2.2: Natural Image Stimulus GUI populated with display parameters including display size, duration and location. and analysis techniques have been extended to work even with complex cells, so that natural stimuli have now become the de facto standard of such studies. The natural image stimulus was the first that was programmed from scratch, although on the basis of the code templates of the other stimuli and the protocol used in the experiments by Tom Mrsic-Flogel and Jan Antolik. The first consideration was how the loading procedure was to be implemented, i.e. whether to implement a large buffer where the images would be preloaded or loading each image as the previous image is still being displayed. Given that huge sets (up to 10,000 images) may be displayed in a single run, memory constraints make buffering entire image sets prohibitive. Therefore each image has to be loaded to the video buffer just in time. To speed this up the whole set of images indexed and randomised when the stimulus protocol is loaded and the loop merely has to access an array with the destination filenames and load the correct image into memory. Testing the average load time per image using the stimulus computer described in appendix A and a set of monochrome images (384x208 pixels) gave an averaged seconds to load each image into memory. This is low enough to ensure there will be no timing issues in any forseeable stimulus protocol. The GUI was populated with all necessary parameters, including the now redundant buffer size, position, image number, background colour, time per frame, time for blank screens to be shown between frames, the random seed, the image directory and filename prefixes. The stimulus was therewith ready to display any stimulus set contained within a single file or folder.

32 Chapter 2. Developing Experimental Procedures 23 Figure 2.3: Sample Natural Image Stimuli captured by Tom Mrsic-Flogel from a variety of David Attenborough documentaries. The natural image set used as part of the experiments performed as part of the experimental portion of this project at the van Hooser lab consisted of around 13,800 monochrome images extracted from David Attenboroughs documentary and previously used in Tom Mrsic-Flogels lab and provided by Jan Antolik. They had already been converted to greyscale in 256 linearly spaced luminance steps. Each of the images existed in one of four forms, flipped horizontally, vertically with and without reversed phase (six example stimulus frames are shown in figure 2.3). Since different animals have acute vision in only a specific range of spatial frequencies it makes sense to analyse the power spectrum of the images over all frequencies, which allows one to determine whether a particular image set will be an efficient driver of responses in visually receptive neurons of a particular species. The average power spectrum was extracted from the image set by iteratively applying a two-dimensional Fast Fourier transform to each image, shifting the zero frequency component to the center and averaging the resulting power spectra. The resulting spectrum is shown in Figure 2.4, which also shows the visual acuity limits for a number of species. Comparing the two graphs that the power spectrum overlaps quite well with the most contrast sensitive spatial frequencies of the different species, with the majority of the power spectrum in low spatial frequencies. Even though ferrets pups and mature ferrets respond mostly within the narrow range of 0.08 and 0.25 cycles/degree (Baker et al., 1998) they too pick up a good portion of the natural image power spectrum so the stimuli are appropriate for use both in mouse and ferret studies. Sample filters provide a general idea about what image features are extracted at the 0.08 and 0.2 cycles/degree and are shown for a sample image in figure Hartley Stimuli The Hartley stimulus based on the fast Hartley transform (Bracewell, 1984) provides a convenient way to generate subspaces with particular orientations, spatial frequencies and phases,

33 Chapter 2. Developing Experimental Procedures Mean Natural Image Power Spectrum Amplitude Spatial Frequency (Cycles/Degree of Visual Angle) Figure 2.4: A) Spatial Contrast Sensitivity in nine different species. Reprinted from Uhlrich et al. (1981). B) Averaged Power Spectrum of 100 Natural Images with the assumption that the image will be displayed on a screen with a cm diagonal from a viewing distance of 40 cm and displayed on full screen. Figure 2.5: A sample natural image was filtered at the A) lower bound (0.08 cyc/deg) and B) upper bound (0.2 cyc/deg) of ferret pup (P30-P40) vision as given by Stephen van Hooser. The top left in the arrangement is the natural image, top right is the two-dimensional power spectrum of the image, bottom left is the visual filter in the spatial frequency domain and bottom right is the filtered image. Plots were generated using Izumi Ohzawa s toolkit named Twodimensional spatial frequency filtering by FFT using Matlab, freely available online at http: //visiome.neuroinf.jp/modules/xoonips/detail.php?item_id=6106/.

34 Chapter 2. Developing Experimental Procedures 25 Figure 2.6: Hartley Stimulus Basis Set with a grid of stimuli with varying orientation, spatial frequency and in the real case phase. Reprinted from scanned copy of Ringach et al. (1997), not fully reproducing the gradual shift from black to white and visa versa. which exceed white noise stimuli in their ability to fully characterize simple cell receptive fields since they are not overcomplete. They are also orthogonal such that the transpose is equal to its inverse, which greatly simplifies analysis. Ringach et al. (1997) first came up with a spatio-temporal linear filter, which would model a neurons response from a set of responses to a Hartley stimulus. A full basis set of Hartley stimuli can be seen in figure 2.6 and is defined by ( ) 2π(kx l + k y m) H(k x,k y ) = cas 0 l,m (M 1) (2.3) M where cas = cos(θ) + sin(θ), M is the width and height of the grid, k x and k y the coordinates within it and l and m are the individual pixels of each square. Just as before the GUI (figure 2.7 was populated with the necessary parameters, which would specify the number of image presentations, the random seed but also the values to restrict

35 Chapter 2. Developing Experimental Procedures 26 Figure 2.7: Hartley Stimulus GUI the Hartley basis set to the spatial frequency space that the animal most responds to. This includes the screen diagonal, pixel density, resolution and upper and lower limits of the animals visual acuity. This square subspace is defined by its maximum wave number Ω such that S Ω span{h(k x,k y max( k x, k y ) Ω}. (2.4) This allows for control over both pixel size and spectral content of the images. Ultimately this allows for specific aspects of a cells receptive field to be measured independent as band- or low-pass filtered subsets of a basis are shown independently, for example as to characterize orientation tuning only at a specific spatial frequency (Ringach et al., 1997). The stimulus is loaded as one large batch, although only an eighth of the subspace actually has to be in memory since the remaining set can be obtained by randomly rotating each texture during the display call. This reduces memory requirements by an eighth. Future implementations should focus on implementing the stimulus using only one-dimensional textures, which are appropriately placed, rotated and masked to further reduce memory, the basics of which were already developed but had to be abandoned due to time constraints Summary Each of these stimulus types plays an important role in characterising different aspects of cell responses. Basic gratings can be used to determine the spatial location of receptive fields, stochastic stimuli to determine basic receptive fields and natural stimuli, which activate complex, higher order effects in the cells. The exact protocol of both stimulus presentation but also data collection is detailed in the next section.

36 Chapter 2. Developing Experimental Procedures Experimental Protocol The experimental protocol developed by Tom Mrsic-Flogel and used to perform initial mouse experiments in mice described below were adapted for the laboratory setup at Stephen van Hoosers laboratory at Brandeis University. This section describes both the original protocol and any changes that were made due to differences in species and equipment Original Protocol The original experiments at the Mrsic-Flogel lab at University College London were carried out in mice in postnatal days 30 to 40, the experimental protocol is taken from the paper by Antolik et al. (2011), which is still in preperation. The mice were anesthetised for initial preperation using Fentanyl (0.05 mg/kg), Midazolam (5.0 mg/kg), and Medetomidin (0.5 mg/kg). During the imaging they were kept under light anesthesia with isofluorane ( %) and a 60 : 40% mixture of O 2 : N 2 O. A craniotomy, 1-2 mm in size was performed over the visual cortex and sealed again after dye injection using 1.6% Agarose in Hepes-buffered ACSF and a cover slip. The dye loading was done using the calcium-sensitive dye Oregon Green Bapta-1 AM, which was dissolved in 4 l DMSO containing 20% Pluronic, and further diluted (1/11) in dye buffer (150 mm NaCl, 2.5 mm KCl, and 10 mm HEPES [ph 7.4]) yielding a final concentration of 0.9 mm. The dye was then injected into the right visual cortex under visual control by twophoton imaging at a depth of µm with a micro-pipette (35M, 3-10 psi, 24 min). Once the dye had diffused into cells the activity of cortical neurons was monitored by imaging fluorescence changes with a custom-built microscope and mode-locked Ti:Sapphire laser at 830 nm and an average laser power of <50 mw through a 40x water immersion objective. The scanning and image acquisition was controlled using National Instruments LabView software. The imaging frames of size 256x256 pixels were acquired at 7.6 Hz and realigned with the focal plane and imaging position of the initial frame. Cell outlines were initially identified a semi-automated algorithm based on morphological measurements of of cell intensity, size and shape and then confirmed by visual inspection. Further processing of the imaging frames fall into the upcoming pre-processing section. The stimuli were presented on 60 Hz LCD monitors of size 43.5x27.5 cm at a resolution of 1024x768 pixels. A retinotopic mapping protocol was used to ensure the screen covered the animals receptive field of the recorded neurons, consisting of moving gratings being presented for 1.4 seconds at 12 locations on the screen with 1.5 second gaps between presentations. The monitor was then centred on the animals receptive field. The natural images set described previously were then presented for 500 ms each and interleaved with 1860 ms of blank gray screens. The images were split into a single trial of 1,800 images and 10 trials of 50 different images, such that a total of 2300 presentations of 1850 different images were carried out. The

37 Chapter 2. Developing Experimental Procedures 28 Figure 2.8: Control Diagram of VHLab environment taken from internal documentation on the VHTools website. onset of image presentation was aligned with the frame rate of scanning Changes to the Protocol The original protocol could of course not be reproduced exactly at the van Hooser lab so the adjustments in experimental procedure, which were investigated as part of this project will be detailed here. The initial difference between procedures at the van Hooser lab was of course largely down to the fact the recordings here were made from ferrets. However, the differences in equipment required a lot of trial and error testing of both the imaging and stimulus protocol. The VHLab environment is shown in figure 2.8 showing the complete control and data flow from the stimulus through the recording equipment and data processing Two-Photon Calcium Imaging The dye loading procedures for two-photon calcium imaging at the lab at Brandeis University were largely identical to those described in the original experiment. Not being directly involved, although keenly observing the experimental procedures, the focus was on getting comparable imaging frames from a different set-up. The first issue was with the timing of the imaging rig at the van Hooser lab, which was at first unable to reproduce the high imaging frame rates of the original experiment. At an

38 Chapter 2. Developing Experimental Procedures 29 imaging resolution of 256x256 pixels and under the standard settings for pixel dwell time, zoom and rotation the period per frame was 697 ms far below the 7.6 Hz specified in the original experiments and too slow for spike train prediction. Therefore adjustments needed to be made, increasing the zoom and reducing the dwell time and region of interest (ROI), effectively cropping the imaging frames, reduced the period per frame to values consistent with the original experiment, although considerable fine adjustment had to be made. Reducing the dwell time per pixel from its standard 1.2 µs to 0.8 µs provides a trade off in the signal to noise ratio, which was essential in achieving frame rates of >7 Hz. Another trade off could be made in selecting a smaller ROI as it could force the exclusion of unresponsive neurons. In addition differences in the ROI from one experiment to the next would result in different frame rates for each experiment. Although this is viable and could easily be implemented using some simple calculations it alters the experimental procedures each time, which is not desirable. Therefore two alternatives were proposed and tested. First, the ROI could be consistently reduced to a particular size, which would cut off some visually responsive neurons in some experiments but ensure equal imaging frame lengths throughout. Secondly, fine adjustments of the zoom and dwell time could be used in conjunction with the ROI size to ensure the frame rate would be the same each time. Using both these approaches imaging frames were captured at a constant 112 ms, a frame rate of Hz. This in turn required the length of the blank frame between frames to be adjusted, to line up with the start of each imaging frame. Since the calcium signal requires up to 2 s to fully decay, the length of the frame could not be altered substantially. With the natural image frames being displayed for a constant 500 ms the combined duration of an image frame and the blank stimulus in between would have to be a multiple of 112 ms. The total duration of these two frames was therefore decided to be s, leaving s for the blank stimulus. The proper operation of this protocol was tested on an animal in which dye loading had failed, which confirmed correct frame triggering and timestamping Electrophysiology As a back up in case two-photon calcium imaging studies couldn t be performed the stimulus and recording protocol was adjusted so that it could also be used for electrophysiology studies. Since electrophysiology provide much higher temporal resolution, the presentation of stimuli could be sped up considerably. This required very little adjustment, although much larger stimulus sets were prepared from the full image repository provided by Tom Mrsic-Flogel s lab. It was decided that the natural images would be displayed for 100 ms with another 100 ms blank screen gap, a vast increase in stimulus frame rate over the two-photon imaging stimulus sets. Compared to other electrophysiology experiments this is still relatively slow but was done to maximize the chances of getting usable data in a single experiment and could always

39 Chapter 2. Developing Experimental Procedures 30 be adjusted in subsequent experiments. The training set was increased to a set of 10,000 and the validation set to a size of 1,000, which was to be shown 10 times. The protocols in preparing the ferret were the same as in two-photon Calcium imaging. Once the animal was prepared a single channel tungsten electrode from World Precision Instruments was inserted into the primary visual cortex and kept at a particular depth until the neuron became unresponsive at which point it was readjusted and recording was continued from another visually responsive neuron. 2.3 Pre-Processing Pre-processing of both two-photon calcium imaging frames and electrophysiology is necessary to feed the results into the analysis methods. This section details the extraction of the signals from the raw data streams, data format conversion and other related issues. It specifies clearly, which processes are carried out by the dedicated software tools and which were programmed as part of this project Calcium signal pre-processing The pre-processing of the two-photon calcium imaging frames involves the automatic detection of cells, averaging their activation, high-pass filtering and excluding inactive neurons but also spike extraction. The VHLab environment already performs automatic cell outline detection and allows for manual visual inspection and then performs averaging of the luminance for each region of interest. The raw luminance signal can therefore easily be exported using the interface in Figure 2.9. The environment also allows for drift correction and completely manual identification of cells. Once the raw luminance signal has been exported, further pre-processing is required before the data can be fed into analysis scripts. For this purpose a preprocessing script was developed on the basis of the pre-processing described in the Antolik et al. (2011) paper that is still in preperation. This includes a low-pass filter, extraction of the maximum activation and a Kolmogorov-Smirnov test to exclude any visually unresponsive neurons. The script accepts arrays with the number of cells and the temporal time course as its dimension, such as the the raw luminance data exported by the VHLab tools environment. It then applies a forward and backward pass of a Chebyshev high-pass filter of order 16 with a cut-off frequency of 0.02 Hz to each time course, this reduces slow fluctuations in the image signal and the high order and double-pass ensure flat phase and frequency responses in the filter (figure 2.10 shows a raw and filtered luminance signal). The script then then breaks the data down into the luminance values for each image presentation and determines for which imaging frame activation is highest, averaged over the entire data set. Next it then extracts and averages the activations within a

40 Chapter 2. Developing Experimental Procedures 31 Figure 2.9: VHLab Calcium signal extraction environment. Allows calcium imaging stacks to be loaded and cells to be identified, correcting for any drift and outputting the calculated luminance signal. Figure 2.10: A) Unfiltered calcium luminance signal time course B) Same luminance signal time course filtered using a forward and backward pass of a order 16 Chebyshev high-pass filter with 0.02 Hz cut-off frequency.

41 Chapter 2. Developing Experimental Procedures 32 Figure 2.11: Preprocessed calcium luminance signal for a single neuron for individual imaging frames. Figure 2.12: Spike rates extracted from a sample time course using the fast non-negative deconvolution method described in Vogelstein et al. (2010). window ±1 of the maximum activation frame, which reduces the data to single values for each stimulus presentation. Finally it performs a Kolmogorov-Smirnov test to establish whether the neuron is in fact visually responsive and saves those that are to a.dat file. The sample output for the neuron shown in its raw and filtered form is shown in figure Instead of working with the raw luminance values, the freely available fast non-negative deconvolution filter for spike train inference from population calcium imaging (Vogelstein et al., 2010) may be used to eliminate sub-threshold activity and thereby potentially increase the predictive power of analysis methods. This filter performs optimal linear deconvolution on the data, runs in linear time and only requires the fluorescence data as its input. The same sample time course as above was run through this filter and returned the spike rates in figure 2.13 as its maximum likelihood estimate, which can be thresholded to produce a spike train Electrophysiology The electrophysiology data is much easier to pre-process since the raw signal only has to be thresholded to arrive at a spike train although some analysis may need to be performed to ex-

42 Chapter 2. Developing Experimental Procedures 33 Figure 2.13: A) Recording Channels including frame triggers, spikes and the raw voltage data. B) Waveform analysis extracts templates of different spiking waveforms for clustering analysis C) K-means clustering analysis using principal component analysis (PCA) to find group similar waveforms and thereby seperates signals from different sources. clude noise and interference from other nearby neurons. The van Hooser lab uses the Spike2 software by Cambridge Electronic Design bundled with their equipment to record the membrane voltage, detect spikes and do k-means clustering analysis, which is used to detect noise and whether one or more cells contributed to the signal on the basis of the spiking waveforms. To extract spike times only from one particular source, the recorded voltage timecourse was thresholded and spike waveform templates were created using the tool shown in figure 2.13B. Similar waveform templates were combined and principal component analysis was performed on spike waveform data on the basis of which k-means clustering was carried out, giving rise to distinct clusters of neuronal activity as seen in figure 2.13C. Clusters corresponding to clear spiking waveforms were saved to external.dat files. These steps were applied to all data recorded as part of this project. Since the software only outputs spike times a small script loaded the frame times and grouped each spike into a bin so

43 Chapter 2. Developing Experimental Procedures 34 that the resulting data could be saved as rates per stimulus presentation although more detailed analysis could potentially be obtained in future by analysing precise spike timings. 2.4 Analysis Methods The analysis is the one of the most important element of this project as it allows the receptive field estimates to be extracted from the data. The most common methods are so called spike triggered averaging and spike triggered covariance, which are still commonly used but have also been extended to work with natural stimuli. Additionally there are now a variety of wavelet models, which try to fit combination of wavelets with varying preferences to the data, in order to achieve a non-linear model of the receptive field and structural models which take thalamocortical and lateral activity into account. All these methods will be explained and a few will be tested on simulated data to confirm they are working under various conditions Spike Triggered Averaging (STA) Theory The spike triggered average stimulus is the average value of the stimulus at a time interval τ before the stimulus is fired. It can be defined as: A = 1 N N n=1 s(t n ) (2.5) where t n is the time of the nth spike, s(t n ) is a vector representing the stimuli in the time interval τ before the spike was triggered and N is the total number of spikes. Since time is discrete several spikes can occur within on time t n, so the stimulus vector is multiplied by the number of spikes in this case. A good visualisation of the spike triggered averaging can be seen in figure By performing this analysis for several time windows spatio-temporal receptive fields may be estimated. Obtaining the STA can be expressed as an inverse problem by estimating the kernel which best predicts the neuron s response to a stimulus (Willmore and Smyth, 2003). While a complete description in theory requires an infinite number of kernels an estimate can be attained using only the first-order kernel, which provides the equivalent of an STA analysis. Mathematically a receptive field of arbitrary dimensionality containing n p pixels can be expressed as the vector n p x1 vector f, where the pixel values are flattened into one dimension. The n s x1 response vector r of a neuron to a stimulus, expressed as the n s xn p matrix S, where n s is the number of stimuli, is given by: Sf = r (2.6)

44 Chapter 2. Developing Experimental Procedures 35 Figure 2.14: Visualisation of spike triggered averaging (STA) shows stimuli at spike time intervals t τ being summed and averaged to give rise to the spike triggered average stimulus. Reprinted from Schwartz et al. (2006). which can be inverted such that recovering the receptive field from stimulus and the neuron s response becomes a matter of solving f = S 1 r (2.7) for f is non-trivial however since it enquires inversion of the stimulus matrix S, which only exists for choices of S with specific properties. Therefore a variety of methods have been devised to address this problem, generally classed into three categories, bias removal, leastsquares solutions and regularized solutions. Reverse correlation (Marmarelis and Naka, 1972) as described above consists simply of calculating the response weighted average of the stimulus. The main problem with this method is that it only produces unbiased estimates of the receptive field when the stimulus matrix, S is both square and orthogonal, in which case the inverse of the stimulus matrix equals simply its transpose, which can be quickly found. A variety of stimuli have been designed to fullfill this requirements including dense white noise patterns (Reid et al., 1997) and the Hartley stimuli devised by Ringach et al. (1997). As Theunissen et al. (2001) found, using non-orthogonal stimuli introduces a systematic bias into the receptive field estimate. A variety of methods were devised to correct this bias starting with simple spectrum corrections in the Fourier domain. Alternatively attempts can be made to solve equation 2.6 directly by finding its least squares solution. A variety of methods to do this have been devised including singular value decomposition (SVD) and iterative methods. The most succesful solutions to this problem also involve regularisation, which ensures a priori information is taken into account when estimating the receptive field. In a detailed comparison Willmore and Smyth (2003) showed that the regularised pseudoinverse provides the most efficient estimator of the first-order kernel, at least for simulated