What is High-level vision? General Issues Scenes Objects Faces Selective Attention. High-Level Vision 2/10/2005 ZOL 867 1

Transcription

1 What is High-level vision? General Issues Scenes Objects Faces Selective Attention High-Level Vision 2/10/2005 ZOL 867 1

2 Defined by Cognitive Scientists From MIT Encyclopedia of Cognitive Science Aspects of vision that reflect influences from memory, context, or intention are considered "high-level vision," a term originating in a hierarchical approach to vision. In currently popular interactive hierarchical models, however, it is almost impossible to distinguish where one level of processing ends and another begins. This is because partial outputs from lowerlevel processes initiate higher-level processes, and the outputs of higher-level processes feed back to influence processing at the lower levels (McClelland and Rumelhart 1986). Thus, the distinctions between processes residing at high, intermediate, and low levels are difficult to draw. Indeed, substantial empirical evidence indicates that some high-level processes influence behaviors that are traditionally considered low-level or MID-LEVEL VISION. With this caveat in mind, the following topics will be considered under the heading "high-level vision": object and face recognition, scene perception and context effects, effects of intention and object knowledge on perception, and the mental structures used to integrate across successive glances at an object or a scene. (Mary Peterson) 2/10/2005 ZOL 867 2

3 Defined by Visual Neuroscientists high-level perception such as recognition and categorization i.e., visual processes that rely on neural activities in inferior temporal cortex and beyond. Li, VanRullen, Koch, & Perona, PNAS, 2000, 99 (14), /10/2005 ZOL 867 3

4 Defined by Visual Neuroscientists (cont d) Low Level Vision (Eye to Cortex) Retina Optic nerve Subcortical structures Cortex lateral geniculate nucleus (LGNd) V1 and V2 superior colliculus (SC) Division of labor in the LGNd: Magno: Colorblind, fast, high contrast, low spatial resolution Parvo: Color selective, slow, low contrast, high spatial resolution Division of labor in V1 and V2: Layers 2 and 3, blob, thin stripe: Color Layers 2 and 3, interblob, interstripe: color-form Layer 4b, thick stripe: motion and dynamic-form 2/10/2005 ZOL 867 4

5 Defined by Visual Neuroscientists (cont d) Intermediate Level Vision (Color, Form, Movement) V1 and V2 parcel out the visual signal into separate regions in prestriate cortex, whose cells respond to different features: V5: Motion V4 (via blob regions of V1): Color V4 (via interblob regions of V1): Color-form V3 and V3a: Dynamic-form 2/10/2005 ZOL 867 5

6 Defined by Visual Neuroscientists (cont d) High Level Vision (Recognition and Action) Ventral System. V4 inferior temporal cortex Forms representations that are abstract, i.e. independent of a specific vantage point, lighting, location and, perhaps, orientation; implicated in object recognition Dorsal System. V5 posterior parietal cortex (PP): coordination of visually guided actions (e.g., eye movements and orienting grasp) localization (perhaps) modulation of selective attention (perhaps) These pathways are sometimes called the what and where pathways 2/10/2005 ZOL 867 6

7 Topics in High-Level Visual Processing Object recognition Face recognition Scene recognition (new) Effects of semantic knowledge (beliefs,interpretation) on perception Transsaccadic Integration Visual Attention and Active Vision (new) 2/10/2005 ZOL 867 7

8 Meta-Issue in Visual Perception Perception vs Visual Processing Is consciousness critical to study of perception? A central concern in classic study of perception is why things look the way they do i.e., phenomenology of perception (closely related to consciousness Marr s view: vision... is a process that produces from images of the external world a description that is useful to the viewer and not cluttered with irrelevant information. (Marr, 1982) 2/10/2005 ZOL 867 8

9 Meta-Issue in Visual Perception Perception vs Visual Processing The world is experienced as a unified whole, but sensory systems do not deliver it to the brain in this way. Signals from different sensory modalities are initially registered in separate brain areas. How does this information become bound together in experience? [emphasis added] Lynn Robertson (2003), Trends in Cognitive Sciences, 4, /10/2005 ZOL 867 9

10 Classic Example: Perceptual Learning This is thought to require higher cortical processing But bees can do it too Hard Easy Visual system improves in ability to recognize object in ambiguous or noisy pattern 2/10/2005 Easy ZOL

11 Meta-Issue in Visual Perception Computational Approach What are the internal visual representations that are available for additional visual and cognitive computations (and implemented in the brain), without concern for whether they give rise to perceptual experience or are open to awareness. Representations that directly control behavior Representations that support other cognitive processes Computational approach is compatible with computer vision, animal (at least invertebrate) vision 2/10/2005 ZOL

12 Meta-Issue in Visual Perception Computational Approach We don t see or experience the operation of gabor filters or edge detection algorithms or structural descriptions (that s why we have to devise clever experiments). Behavioral indicators (whether they give rise to awareness or not) are critical for evaluating of underlying visual processing. These include reaction time, errors, implicit measures like eye movements. 2/10/2005 ZOL

13 Scenes: Transaccadic Stability Why and how do we experience a complete, detailed, full color visual world despite the fact that (a) the retinas cannot deliver this high-fidelity input within a given fixation, and (b) the visual system cannot fuse together discrete retinotopic images to generate a composite internal picture Data are experiential: did you see the world shift? 2/10/2005 ZOL

14 Scenes: Transaccadic Integration What is the nature of the internal representation that is retained across a saccade and integrated with the visual input acquired during the next fixation regardless of whether this representation is functional in generating experience. Data are behavioral: was some behavioral measure affected by the manipulation, e.g., fixation time 2/10/2005 ZOL

15 Objects ( If it looks and walks and quacks like a duck Some Terminology: Type A general class or category of object, such as coffee cup, car, postage stamp Token A particular object within a given class 2/10/2005 ZOL

16 Objects (cont d) I. Eidetic, or pictoral images: system recognizes whole shape, but is confused by rotational changes Bees confuse these patterns 2/10/2005 ZOL

17 Objects (cont d) II. Featural models: system recognizes diagnostic features (ethologists would call these sign stimuli, which could be learned or hardwired) Herring gull chicks innately recognize adult gull head (red spot is sign stimulus) Pigeons confuse these patterns rick/default.htm#rbc 2/10/2005 ZOL

18 Objects (cont d) III. Structural models: system recognizes assemblage of features, according to the form and arrangement of individual components (akin to visual words and syntax _ Example: Recognition by components (Biedermann) (Geon model) Recognition is achieved when relative relationships among components matches Rules are defined according to relative spatial relationships, so this system is robust to variance in size, rotation, or translation Same features, different structure 2/10/2005 ZOL

19 Objects (cont d) III. Statistical models: system recognizes object based on degree of resemblance (in whatever visual dimensions) to previously experienced exemplars This leads to recognition that is robust to variance in rotation, etc., provided experience includes exposure to a wide variety of viewpoints, contexts, etc. 2/10/2005 ZOL

20 Faces 2/10/2005 ZOL

21 Faces Functional Neuroimaging Probably single-most important methodology in the modern era of cognitive neuroscience 2/10/2005 ZOL