Next Generation Artificial Vision Systems

Transcription

1 Next Generation Artificial Vision Systems Reverse Engineering the Human Visual System Anil Bharath Maria Petrou Imperial College London ARTECH H O U S E BOSTON LONDON artechhouse.com

2 Contents Preface xiii CHAPTER 1 The Human Visual System: An Engineering Challenge Introduction Overview of the Human Visual System The Human Eye Issues to Be Investigated Lateral Geniculate Nucleus (LGN) The VI Region of the Visual Cortex Issues to Be Investigated Motion Analysis and V Issues to Be Investigated Conclusions 15 References 17 The Physiology and Psychology of Vision 19 CHAPTER 2 Retinal Physiology and Neuronal Modeling Introduction Retinal Anatomy Retinal Physiology Mathematical Modeling Single Cells of the Retina Mathematical Modeling The Retina and Its Functions A Flexible, Dynamical Model of Retinal Function Foveal Structure Differential Equations Color Mechanisms Foveal Image Representation Modeling Retinal Motion Numerical Simulation Examples Parameters and Visual Stimuli Temporal Characteristics Spatial Characteristics Color Characteristics Conclusions 45 References 46 v

3 VI Contents CHAPTER 3 Ä Review of VI 3.1 Introduction 3.2 Two Aspects of Organization and Functions in VI Single-Neuron Responses Organization of Individual Cells in VI Orientation Selectivity Color Selectivity Scale Selectivity Phase Selectivity 3.3 Computational Understanding of the Feed Forward VI VI Cell Interactions and Global Computation Theory and Model of Intracortical Interactions in VI 3.4 Conclusions References CHAPTER 4 Testing the Hypothesis That VI Creates a Bottom-Up Saliency Map Introduction Materials and Methods Results Interference by Task-Irrelevant Features The Color-Orientation Asymmetry in Interference Advantage for Color-Orientation Double Feature but Not Orientation-Orientation Double Feature Emergent Grouping of Orientation Features by Spatial Configurations 4.4 Discussion 4.5 Conclusions References The Mathematics of Vision 103 CHAPTER 5 VI Wavelet Models and Visual Inference 5.1 Introduction Wavelets Wavelets in Image Analysis and Vision Wavelet Choices Linear vs Nonlinear Mappings 5.2 A Polar Separable Complex Wavelet Design

4 Contents VII Design Overview Filter Designs: Radial Frequency Angular Frequency Response Filter Kernels Steering and Orientation Estimation The Use of Vl-Like Wavelet Models in Computer Vision Overview Generating Orientation Maps Corner Likelihood Response Phase Estimation Inference from Vl-Like Representations Vector Image Fields Formulation of Detection Samplingof (B,X) The Notion of "Expected" Vector Fields An Analytic Example: Uniform Intensity Circle Vector Model Plausibility and Extension Vector Fields: A Variable Contrast Model Plausibility by Demonstration Plausibility from Real Image Data Divisive Normalization Evaluating Shape Detection Algorithms Circle-and-Square Discrimination Test Grouping Phase-Invariant Feature Maps Keypoint Detection Using DTCWT Summary and Conclusions 140 References 141 CHAPTER 6 Beyond the Representation of Images by Rectangular Grids Introduction Linear Image Processing Interpolation of Irregularly Sampled Data Kriging Iterative Error Correction Normalized Convolution DFT from Irregularly Sampled Data Nonlinear Image Processing Vl-Inspired Edge Detection Beyond the Conventional Data Representations and Object Descriptors The Trace Transform Features from the Trace Transform 165

5 VIII Contents 6.4 Reverse Engineering Some Aspect of the Human Visual System Conclusions 168 References 169 CHAPTER 7 Reverse Engineering of Human Vision: Hyperacuity and Super-Resolution Introduction Hyperacuity and Super-Resolution Super-Resolution Image Reconstruction Methods Constrained Least Squares Approach Projection onto Convex Sets Maximum A Posteriori Formulation Markov Random Field Prior Comparison of the Super-Resolution Methods Image Registration Applications of Super-Resolution Application in Minimally Invasive Surgery Other Applications Conclusions and Further Challenges 188 References 188 CHAPTER 8 Eye Tracking and Depth from Vergence Introduction Eye-Tracking Techniques Applications of Eye Tracking Psychology/Psychiatry and Cognitive Sciences Behavior Analysis Medicine Human-Computer Interaction Gaze-Contingent Control for Robotic Surgery Ocular Vergence for Depth Recovery Binocular Eye-Tracking Calibration Depth Recovery and Motion Stabilization Discussion and Conclusions 209 References 210 CHAPTER 9 Motion Detection and Tracking by Mimicking Neurological Dorsal/Ventral Pathways Introduction Motion Processing in the Human Visual System Motion Detection 219

6 Contents IX Temporal Edge Detection Wavelet Decomposition The Spatiotemporal Haar Wavelet Computational Cost Dual-Channel Tracking Paradigm Appearance Model Early Approaches to Prediction Tracking by Blob Sorting Behavior Recognition and Understanding A Theory of Tracking Concluding Remarks 241 References 242 UESUÜUJJ Hardware Technologies for Vision 249 CHAPTER 10 Organic and Inorganic Semiconductor Photoreceptors Mimicking the Human Rods and Cones Introduction Phototransduction in the Human Eye The Physiology of the Eye Phototransduction Cascade Light Activation of the Cascade Deactivation of the Cascade Light Adaptation of Photoreceptors: Weber-Fechner's Law Some Engineering Aspects of Photoreceptor Cells Phototransduction in Silicon CCD Photodetector Arrays CMOS Photodetector Arrays Color Filtering Scaling Considerations Phototransduction with Organic Semiconductor Devices Principles of Organic Semiconductors Organic Photodetection Organic Photodiode Structure Organic Photodiode Electronic Characteristics Photocurrent and Efficiency The Equivalent Circuit and Shunt Resistance Spectral Response Characteristics Fabrication Contact Printing Printing on CMOS Conclusions 285 References 286

7 X Contents CHAPTER 11 Analog Retinomorphic Circuitry to Perform Retinal and Retinal-Inspired Processing Introduction Principles of Analog Processing The Metal Oxide Semiconductor Field Effect Transistor Transistor Operation nmos and pmos Devices Transconductance Characteristics Inversion Characteristics MOSFET Weak Inversion and Biological Gap Junctions Analog vs Digital Methodologies Photo Electric Transduction Logarithmic Sensors Feedback Buffers Integration-Based Photodetection Circuits Photocurrent Current-Mode Readout Retinimorphic Circuit Processing Voltage Mode Resistive Networks Limitations with This Approach Current Mode Approaches to Receptive Field Convolution Improved Horizontal Cell Circuitry Novel Bipolar Circuitry Bidirectional Current Mode Processing Dealing with Multiple High Impedance Processing Channels The Current Comparator Reconfigurable Fields Intelligent Ganglion Cells ON-OFF Ganglion Cells Pulse Width Encoding Address Event Representation The Arbitration Tree Collisions Sparse Coding Collision Reduction Adaptive Foveation System Algorithm Circuit Implementation The Future Conclusions 330 References 330

8 Contents XI CHAPTER 12 Analog VI Platforms Analog Processing: Obsolete? The Cellular Neural Network The Linear CNN CNNs and Mixed Domain Spatiotemporal Transfer Functions Networks with Temporal Derivative Diffusion Stability A Signal Flow Graph-Based Implementation Continuous Time Signal Flow Graphs On SFG Relations with the MLCNN Examples A Spatiotemporal Cone Filter Visual Cortical Receptive Field Modelling Modeling of Complex Cell Receptive Fields Summary and Conclusions 363 References 364 CHAPTER 13 From Algorithms to Hardware Implementation Introduction Field Programmable Gate Arrays Circuit Design Design Process Mapping Two-Dimensional Filters onto FPGAs Implementation of Complex Wavelet Pyramid on FPGA FPGA Design Host Control Implementation Analysis Performance Analysis Corner Detection Conclusions Hardware Implementation of the Trace Transform Introduction to the Trace Transform Computational Complexity Füll Trace Transform System Acceleration Methods Target Board System Overview Top-Level Control Rotation Block Functional Blocks Initialization Flexible Functionals for Exploration Type A Functional Block 388

9 XII Contents Type B Functional Block Type C Functional Block Functional Coverage Performance and Area Results Conclusions Summary 391 References 392 CHAPTER 14 Real-Time Spatiotemporal Saliency Introduction The Framework Overview Realization of the Framework Two-Dimensional Feature Detection Feature Tracker Prediction Distribution Distance Suppression Performance Evaluation Adaptive Saliency Responses Complex Scene Saliency Analysis Conclusions 413 References 413 Acronyms and Abbreviations 415 About the Editors 419 List of Contributors 420 Index 423