Images 1
Image processing, image analysis Image Processing : In: image, Out: image Improve image quality, make things clearer Image Analysis is: In: image, Out: information/insight Interpret/understand images. Image analysis is typically a pipeline of techniques, that starts with image processing (e.g. denoising, feature extraction, etc). 2
Example No border What s this? 3
Example Source: http://www.hamangia.freeserve.co.uk/ 4
Example Source: http://www.hamangia.freeserve.co.uk/ 5
Example Source: Australian Bureau of Meteorology 6
Why process/analyze images? Many different fields: Image data transmission (coding, compression, steganography); Image filtering (enhancement, de-blurring, special effects) Image understanding (segmentation, pattern recognition, machine intelligence). Some application areas: Science and technology (medicine, materials science, biology, astronomy, remote sensing) Law enforcement (face recognition) Arts (special effects, film restoration, communication) 7
When process/analyze images? Is is economically sensible do use image processing: Automation of visual tasks (surveillance, quality control, industrial vision) Cheap and/or plentiful sensors (cameras are everywhere) When there is no other choice Hubble space telescope in its early years Robots in remote or dangerous locations When we want to make full use of the data High-depth, high resolution sensors, Multi-spectral data, enhanced senses (IR, UV, X-rays) Rendering of 3D data And far more. 8
So, what is an image? Typically a 2D array of single values (gray-level images) or a 2D array of 3-component vectors (RGB, color images). A single element of these arrays is called a pixel (picture element). Input and output spaces can be higher-dimensional (3D images in medicine, movies, hyperspectral images). Image collections can be related spatially (stereo pairs). 9 9
2-D arrays of 8-bit data Simplest image representation: collection of 1 (graylevel) or 3 (color) 2-D arrays of 8-bit data. Enough to represent most visible grey levels (256), and most visible colors (255^3 = 16 millions). Most problems that occur with more dimensions, more channels, more bit depths, etc. can be illustrated within that framework. 10
Spatial resolution Full 1/2 1/4 1/8 11
Number of grey levels (bit depth) 12
Sensors: photography First photography: ca 1816, now lost. Among oldest photos on record is the following, original also lost in late 19th century. 13
Images often start with optics Pinhole model Advantages: Simple geometry, always in focus, useful model. Drawbacks: Too simplistic: no diffraction, no aberrations. 14
Thin lens camera Actually a somewhat realistic model for some instruments, e.g. telescopes. Issues: Depth of field, F number, Field of view, Diffraction, etc. 15
Real cameras have: Thick, multiple lens Geometric aberrations: spherical lens, barrel-pincushion, vignetting, etc. Chromatic aberrations: due to refraction index dependent of wavelength. 16
Real cameras: spherical aberration 17
Visible light is a subset of the electro-magnetic spectrum, from 380 to 720nm wavelength. A light source is characterized by (i) its strength and (ii) its spectrum. A monochromatic source is characterized by a wavelength and its luminance L. The eye: What is color? Cannot distinguish individual frequencies (unlike the hear). Instead the addition of N colors is equivalent to the addition of a white flux Lw and the flux of a single resulting monochromatic color Lr of wavelength r (3 variables) 18
(Human) color perception There are 3 types of color sensors in the eye (cones). This suggests a 3-stimulus color representation scheme : 19
CIE XYZ standard In 1931, the Commission Internationale de l Éclairage (CIE) did the color matching experiment using 3 monochromatic sources: red (645.2 nm), green (526.3.1 nm) and blue (444.4 nm). This gave rise to the CIE RGB system. With this system, it is possible to represent most natural colors Some wavelengths require negative weights. 20
Other color systems RGB (Red, Green, Blue) HSL (Hue, Saturation, Lightness) CMYK (Cyan, Magenta, Yellow, Black) and also: CIE Lab 21
It s complicated (the eye/brain system is not a CCD chip) Spatial and temporal effects are ignored Chromatic adaptation: ability of the visual system to adapt to a dominant color. Assimilation: influence of surrounding colors towards these colors Contrast: influence of surrounding color to move away from these colors. Everyone is different. About 10% of males have some sort of color blindness, i.e. lack of one or more kind of receptors, usually the red cones. Some rare females can have 4 kinds of receptors. Illumination is a big factor. We are all color blind at night (rod receptors are more sensitive than cones). 22
Example of contrast perception Which is the darkest inner rectangle? 23
color perception test 24
Lines perception aberration 25
Questions? 26