Sensors: Introduction

Transcription

1 Sensors: Introduction Sensor is device that allows robot to interact with world Sensing must be considered as a module consisting of 1. Physical sensor 2. Software to extract relevant info from signal Module incorporated as perceptual schema Process depicted in following diagram: Note that percept is defined as a data structure Task of perceptual schema is to extract appropriate info from sensory input and to store it in percept Following terminology used to distinguish among sensor types: 1. Sensor: Device to measure some attribute of the world 2. Transducer: part of sensor that converts what is being measured into another form of energy 3. Passive sensor: Relies on environment to produce energy being sensed 4. Active sensor: Generates energy being sensed 5. Modality: Type of energy being sensed 1

2 Sensors: Introduction (2) 6. Logical sensor: Abstract sensor independent of any physical implementation Entails all available alternative methods of obtaining percept Methods may vary in terms of Conditions under which they operate Time to produce signal Modality of operation,... All are logically equivalent Included is algorithm for selecting appropriate method Definition has been generalized to standard meaning of logical Black box that performs a given task Perceptual schema is logical sensor wrt this definition Implication is that physical sensor can be used as many logical sensors 2

3 Sensors: Sensor Fusion Sensor fusion: Process of combining input from multiple sensors into a single percept Basic ways of combining sensors: 1. Redundancy/competition Use multiple devices to sense same percept Useful In noisy environments With sensors that are imprecise Types of redundancy: Physical redundancy: Sensors have same modality Logical redundancy: Sensors use different modalities to sense same percept Types of sensor misreadings: False positive: Sensing of signal that isn t there False negative: Failure to detect signal 2. Complementation Use multiple sensors, each id ing different aspect of percept 3. Coordination Use sensors in sequence Often used to focus attention 3

4 Sensors: Sensor Fusion (2) Fusion can be incorporated into behaviors in a number of ways: 1. Sensor fission Coined by Brooks He argued that fusion does not occur at behavioral level Sensor s outputs not combined Rather overall behavior result of primitive competing behaviors Uses one sensor per behavior Behaviors can independently share sensors Resultant behavior is result of competition among triggered behaviors 2. Action-oriented sensor fusion Cognitive psychology shows that behavioral fusion does indeed occur Stimuli travel over separate paths to multiple behavior areas of brain Fusion of sensory inputs occurs at a particular behavior Result is common representation Fusion occurs to support a particular action associated with a particular behavior 4

5 3. Sensor fashion Sensors: Sensor Fusion (3) Refers to coordination of sensors (addressed by neither fission or fusion) Percept chosen depends on circumstances 5

6 Sensors: Sensor Suites Sensor suite: Set of sensors Operating environment determines set of sensors needed for robot Sensors can be categorized as: 1. Proprioceptive: Measure movements of robot wrt internal frame of reference 2. Exteroceptive: Measure environment wrt internal frame of reference 3. Exproprioceptive: Measure position of robot wrt environment Reactive robots always have exteroceptive sensors Hardware/sensor characteristics to be considered: 1. Field of view 2. Range 3. Accuracy Inaccuracy usable if always off by consistent amount 4. Repeatability 5. Resolution 6. Responsiveness (in domain) Must have adequate signal-to-noise ratio 7. Power consumption Hotel load: Amount of power needed to support sensor suite Locomotion load: Amount of power needed to move robot 8. Hardware reliability 9. Size Factor of operating environment Directly affects power requirements 6

7 Sensors: Sensor Suites (2) Software characteristics to be considered: 1. Computational complexity 2. Interpretation reliability How reliable is algorithm? Desirable characteristics: 1. Simplicity Implies ease of maintenance and interpretation 2. Modularity Implies ease of reconfiguration 3. Redundancy Implies reliability Major problem is for robot to identify when a sensor has failed Fault tolerance: Ability to survive a failure 7

8 Sensors: Types of Sensors 1. Location sensors (a) Shaft encoders Proprioceptive Measure distance Associated with motor Accuracy dependent on locomotion method and terrain May affect traction (b) Inertial navigation system (c) GPS Exteroceptive Based on accelerometers Large and costly Accuracy dependent on smoothness of motion Uses triangulation wrt 4 satellites Not reliable in cities Military and civilian versions Neither proprioceptive nor exteroceptive 8

9 Sensors: Types of Sensors (2) 2. Proximity sensors (a) Sonar (echo location) Field of view 30 o (5 foot range) Produces main lobe (primary sound wave) and side lobes (secondary waves) Generally assume echo produced by main lobe Generally consider main lobe with FOV = 8-15 degrees Cannot detect objects within 11 inches due to time to damp vibrations 4 regions of interest: i. Inside range ii. Outside range iii. Within range iv. Behind object 9

10 Sensors: Types of Sensors (3) Problems: i. Specular reflection: reflection at an acute angle Signal not returned ii. Multiple reflection (result of specular) False distance (farther than actual) iii. Foreshortening: False distance based on assumption that echo is from dead ahead iv. Crosstalk: Interference from multiple sensors Eliminate by firing in sequence v. Size (of obstacles) Small objects do not generate sufficient echo As distance increases, obstacles produce reduced signal vi. Power consumption vii. Generally high viii. Need a strong signal 10

11 (b) IR Problems: i. Bright environments ii. Dark objects iii. Short range (3-5 inches) (c) Touch sensors (tactile sensors) Types of Sensors (4) Problems: i. Placement important - must insure parts of robot do not extend beyond sensor, or that objects may be missed 11

12 Based on electromagnetic energy Sensors: Computer Vision - Introduction Maps multiple readings from 3D space onto 2D grid Each grid cell called a pixel Modality of camera determines what image measures: Light (visual) image Heat map... Computer vision separate field Variety of algorithms developed for Edge detection Noise filtering Image enhancement... These generally not amenable for mobile robots due to complexity and memory requirements 12

13 Sensors: Computer Vision - Devices Video camera Consists of array of charged couple devices (CCDs) Each detector is metal-oxide semiconductor (MOS) capacitor Responsive to visible light Each represents a rectangular pixel Signal is analog Image can be captured One row at a time (line transfer) Entire array at a time (frame transfer) Signal must be digitized Use A/D converter Process too slow for real-time control To compensate: Use multiple buffers to create a pipeline of frames Use low frame rate Frame grabber Digitizes analog image 13

14 Sensors: Computer Vision - Color Grayscale Monochrome 8 bit representation Color RGB Additive primaries Used for CRTs Color cube Plotted on Cartesian coordinate system Black at origin White at FUR corner Main diagonal represents grays Usually represented using 8 bits per primary 2 ways to represent internally: 1. Interleaved Each pixel represented as 3 contiguous values: #define RED 0 #define GREEN 1 #define BLUE 2 int image[row][col][col_plane]; red = image[row][col][red]; green = image[row][col][green]; blue = image[row][col][blue]; displaycolor(red, green, blue); 14

15 Sensors: Computer Vision - Color (2) 2. Separately Pixel represented as entry across 3 color arrays: #define RED 0 #define GREEN 1 #define BLUE 2 int imagered[row][col]; int imagegreen[row][col]; int imageblue[row][col]; red = imagered[row][col]; green = imagegreen[row][col]; blue = imageblue[row][col]; displaycolor(red, green, blue); Problems for robotics: 1. Perceived color dependent on (a) Color of illumination source (b) Surface characteristics of object (c) Sensitivity of camera 2. Visual erosion: Distance of robot from object and angular relation to illumination source affect perceived intensity 3. CCD devices less sensitive to red than green and blue 15

16 HSV Sensors: Computer Vision - Color (3) Based on Hue: Absolute color Saturation: Purity of color Value: Intensity Color cone Cylindrical coordinate system Vertical axis V (value) - represents lightness H (hue) measured as angle around V axis R at 0 o S (saturation) measured as distance from V axis to edge Edge represents S = 1 Apex of cone at origin Black at V = 0, S = 0 White at V = 1, S = 0 Pure colors at V = 1, S = 1 Advantages over RGB model: Hue measures absolute wavelength of color Easy to represent internally Problems: 1. Need special cameras and frame grabbers - expensive (Standard models use RGB) 2. Conversion to HSV from RGB Computationally expensive Singularities exists where R == G == B Since CCD devices have flat response to R, increases liklihood of singularities 16

17 SCT Sensors: Computer Vision - Color (4) Spherical Coordinate Transform Based on RGB color cube Represents colors in terms of polar coordinate system: Vector r represents color Angular coordinates represent hue Length represents intensity 17

18 Sensors: Computer Vision - Color (5) 18

19 Sensors: Computer Vision - Region Segmentation Major task of behavior-based robots is to id objects based on color General process called region segmentation Need to separate image into 1. Foreground: Pixels that represent object of interest 2. Background: Remaining pixels Two general steps involved: 1. Thresholding Id pixels in region with same color 2. Region growing Generate clusters of those pixels Thresholding In simplest case, assume binary image Pixels have color-of-interest or not Code: for (i = 0; i < rows; i++) for (j = 0; j < cols; j++) if ((image[i][j][red] == redvalue) && (image[i][j][green] == greenvalue) && (image[i][j][blue] == bluevalue)) imageout[i][j] = 255; else imageout[i][j] = 0; Use 0 and 255 because color eye cannot distinguish between output levels of 0 and 1 on devices using 8+ bits per pixel 19

20 Sensors: Computer Vision - Region Segmentation (2) Use multivalued image Since object s perceived color sensitive to environment, will not be seen as blob of constant color Use threshold to id pixels of interest: for (i = 0; i < rows; i++) for (j = 0; j < cols; j++) if (((image[i][j][red] >= redlowvalue) && (image[i][j][red] <= redhighvalue)) && ((image[i][j][green] >= greenlowvalue) && (image[i][j][green] <= greenhighvalue)) && ((image[i][j][blue] >= bluelowvalue) && (image[i][j][blue] <= bluehighvalue))) imageout[i][j] = 255; else imageout[i][j] = 0; Region growing Want to id center of region of interest Performed by perceptual schema Approaches: 1. Use weighted centroid of all pixels of interest 2. Find largest area of adjacent pixels of interest and find its centroid 20

21 Sensors: Computer Vision - Histogramming Thresholding based on multicolored objects Histogram consist of buckets Each bucket contains number of pixels that fall within a specified range Implementation for grayscale and HSV simple Implementation for RGB more difficult: Need 1 histogram per color plane Recognition achieved via histogram intersection Simply subtract 1 histogram from another on bucket-by-bucket basis Difference is per cent of pixels that do not match intersection = b j=1 abs(i j E j ) bj=1 E j where b is number of buckets, I and E are histograms Can be used to implement releasers Store histogram of releasing image in perceptual schema Intersect current image with it If difference small enough, set releaser Does not violate principles of behavior-based robots Intersection also useful to represent stimulus strength 21

22 Sensors: Computer Vision - Range Finding: Stereopsis Cameras can be used in several ways to determine distance of robot from an object: 1. Stereopsis 2. Light striping 3. Laser ranging Stereopsis (stereo disparity, binocular vision) uses 2 cameras in tandem Called stereo pair Cameras mounted on a fixed base Distance between them called baseline Result will be a depth map 2D grayscale image Intensity represents distance 2 general approaches: 22

23 1. Vergeance Sensors: Computer Vision - Range Finding: Stereopsis (2) Each camera can rotate about a fixed axis Cameras aimed at same point Angles between line of sight and main axis determined Distance can be determined by triangulation Vergeance refers to process of focusing cameras at same point Problems: (a) Expensive (b) Correspondence problem: Insuring both cameras aimed at same point in space Interest operator: Algorithm for id ing pixels of interest - those with unique values Difficult due to problems associated with perceived color Interest operator usually id s set of pixels and matching algorithm tries to find best correlation between left and right projections Once aligned, assign depth values 23

24 2. Fixed stereo pair Sensors: Computer Vision - Range Finding: Stereopsis (3) Cameras aimed straight ahead with optic axes parallel Projection of pixel of interest intersects planes at different points Disparity: Difference between points Rectified images: Images after they have been correlated Pixel of interest should appear in same row in each image Epipolar lines: Paired lines of images Id of corresponding points computationally easy as only compare points in lines, not in whole 2D image Problems: (a) Must have perfect alignment Calibration process frequently used to insure Stereopsis expensive Stereo matching algorithms O(n 2 m 2 ) for an n m grid Color segmentation O(nm) 24

25 Sensors: Computer Vision - Range Finding: Light Striping Principle is to cast a light pattern onto a surface Flat surface results in continuous image Observed discontinuities provide depth, size and shape info Granularity of pattern determines amount of depth info extracted Relatively cheap 1. Do not need to be as precise as stereopsis devices 2. Can use regular light 3. Processing cheaper - do not consider every pixel Problems: 1. Well suited for recognition, but not so much for reactive robots 2. Difficulty when reflected light and object s color similar 3. Environmental light may interfere with reflection 25

26 Sensors: Computer Vision - Range Finding: Laser Ranging Also called laser radar, ladar, lidar Based on same principle as sonar range finder FOV much narrower than for sonar Scans environment like CRT Can generate frame at rates of pixels per second Two grayscale images generated: 1. Depth 2. Intensity Problems: 1. Differences in reflectivity of different surfaces 2. Expensive 3. Generally need 2 26