Virtual Reality for Human Computer Interaction

Transcription

1 Virtual Reality for Human Computer Interaction Appearance Appearance

2 Appearance Objects have been described so far by their spatial attributes position, location and shape (using vertices, surfaces and transformations). The next task is to determine their appearance: 1. Render type: vertices, lines, surfaces, 2. Lighting: Description or model of light-object-eye interaction. 3. Shading: Algorithmical lighting application during rendering across a primitive. The applied methods can be loosely divided as follows: 1. Local models: Do not take object-object reflections into account. Example: Gouraud and Phong shader using Phong lighting model. 2. Global models: Take object-object reflections into account. Example: Ray-tracer, Radiosity Most Realtime 3D systems currently use local models and texturing... but local models are often extended to capture global attributes, e.g., using Virtual Reality for Human Computer Interaction Appearance: Visual perception Light and Color see: (van Dam et al., 1996, pp )

3 Achromatic/Colored Light Achromatic/Colored Light Achromatic light

4 Achromatic/Colored Light Achromatic light Chromatic color Achromatic/Colored Light Achromatic light Chromatic color Color models for raster graphics

5 Achromatic/Colored Light Achromatic light Chromatic color Color models for raster graphics Reproducing color Achromatic/Colored Light Achromatic light Chromatic color Color models for raster graphics Reproducing color Using color in computer graphics

6 Color in Computer Graphics Color in Computer Graphics

7 Color in Computer Graphics Physics and measurement for realism what does coding an RGB triple mean? Color in Computer Graphics Physics and measurement for realism what does coding an RGB triple mean? Perception and aesthetics for selecting appropriate user interface colors why a bright red and orange striped bedroom is a bad idea how to put on matching pants and shirt in the morning role of culture and even age e.g., WIRED magazine

8 Color in Computer Graphics Physics and measurement for realism what does coding an RGB triple mean? Perception and aesthetics for selecting appropriate user interface colors why a bright red and orange striped bedroom is a bad idea how to put on matching pants and shirt in the morning role of culture and even age e.g., WIRED magazine Color models for providing users with easy color selection systems for naming and describing colors Color in Computer Graphics Physics and measurement for realism what does coding an RGB triple mean? Perception and aesthetics for selecting appropriate user interface colors why a bright red and orange striped bedroom is a bad idea how to put on matching pants and shirt in the morning role of culture and even age e.g., WIRED magazine Color models for providing users with easy color selection systems for naming and describing colors Color models, measurement and color gamuts for color media conversion why colors on your screen may not be printable, and vice-versa managing color in systems with computers, monitors, scanners, and printers color awareness a highly interdisciplinary field that is often unpredictable and downright bizarre

9 Color in Computer Graphics Physics and measurement for realism what does coding an RGB triple mean? Perception and aesthetics for selecting appropriate user interface colors why a bright red and orange striped bedroom is a bad idea how to put on matching pants and shirt in the morning role of culture and even age e.g., WIRED magazine Color models for providing users with easy color selection systems for naming and describing colors Color models, measurement and color gamuts for color media conversion why colors on your screen may not be printable, and vice-versa managing color in systems with computers, monitors, scanners, and printers color awareness a highly interdisciplinary field that is often unpredictable and downright bizarre Useful background for rendering; provides a good introduction to signal processing also used for image processing and anti-aliasing What Creates Colors? Interaction between Light, Objects, Eyes What is Light? Electromagnetic Radiation of a Specific Spectrum Range Light is a distribution C(I) of intensities I at each wavelength

10 Color difficulties Color difficulties Color is an immensely complex subject, drawing on physics, physiology, psychology, art, and graphic design Many theories, measurement techniques, and standards for colors, yet no one theory of human color perception is universally accepted Color of object depends not only on object itself but also on light source illuminating it, on color of surrounding area, and on human visual system (the eye/brain mechanism) Some objects reflect light (wall, desk, paper), while others also transmit light (cellophane, glass) surface that reflects only pure blue light illuminated with pure red light appears black pure green light viewed through glass that transmits only pure red also appears black

11 Achromatic/Chromatic Light Achromatic/Chromatic Light Achromatic light: intensity (quantity of light) only called intensity or luminance if measure of light s energy or brightness the psychophysical sense of perceived intensity gray levels (e.g., from 0.0 to 1.0) seen on black and white TV or display monitors

12 Achromatic/Chromatic Light Achromatic light: intensity (quantity of light) only called intensity or luminance if measure of light s energy or brightness the psychophysical sense of perceived intensity gray levels (e.g., from 0.0 to 1.0) seen on black and white TV or display monitors Chromatic light visual color sensations brightness/intensity chromaticity/color hue/position in spectrum (red, green, yellow...) saturation/vividness generally need 64 to 256 gray levels for continuous-tone images without contouring Gamma

13 Gamma Gamma (!) is a measure of the nonlinearities of a display Nonlinearity: the response (output) is not directly proportional to the input (term often used incorrectly to refer to nonlinearity of image data) Gamma Gamma (!) is a measure of the nonlinearities of a display Nonlinearity: the response (output) is not directly proportional to the input (term often used incorrectly to refer to nonlinearity of image data) Example: PC monitors have a gamma of roughly 2.5, while Mac monitors have a gamma of 1.8, so Mac images appear dark on PC s:

14 Gamma Gamma (!) is a measure of the nonlinearities of a display Nonlinearity: the response (output) is not directly proportional to the input (term often used incorrectly to refer to nonlinearity of image data) Example: PC monitors have a gamma of roughly 2.5, while Mac monitors have a gamma of 1.8, so Mac images appear dark on PC s: Mac user generates image PC user changes image to make it bright PC user gives image back; it s now too bright Problems in graphics need to maintain color consistency across different platforms and hardware devices (monitor, printer, etc.) even the same type/brand of monitors change gamma value over time proper design, use of color software like ColorBlind Gamma

15 Gamma Nonlinearities are pervasive hardware human visual systems How to distribute 256 different intensities? don t want, for example, first 128 in [0, 0.1] and second 128 in [0.9, 1.0] would create a visible gap from 0.1 to 0.9 but equal distribution of 256 in [0,1.0] ignores important characteristic of the human eye Eye sensitive to ratio: perceives intensities 0.10 and 0.11 as differing just as much as the intensities 0.50 and 0.55 Yet want predictability First, we deal with nonlinearity of the human visual system, then with nonlinearity of CRT (LCD is different) Gamma correction

16 Gamma correction To achieve equal steps in brightness, space logarithmically rather than linearly, so that: I I I I j + 1 = j = j j! 1 r Gamma correction To achieve equal steps in brightness, space logarithmically rather than linearly, so that: I I I I j + 1 = j = j j! 1 r Use the following relations: 2 3 I = I0, I = 1 ri0, I = 2 ri = 1 r I0, I = 3 ri = 2 r I0, 255 I = r I = 1 0 K 255 0,

17 Gamma correction To achieve equal steps in brightness, space logarithmically rather than linearly, so that: I I I I j + 1 = j = j j! 1 r Use the following relations: Therefore: 2 3 I = I0, I = 1 ri0, I = 2 ri = 1 r I0, I = 3 ri = 2 r I0, 255 I = r I = 1 0 K 255 1/255 r (1/ I0), I for0" j " = = = /255 (255 )/255 0 (1/ 0) =! j j j j r I I I0 I0, (13.2) Gamma correction To achieve equal steps in brightness, space logarithmically rather than linearly, so that: I I I I j + 1 = j = j j! 1 r Use the following relations: 2 3 I = I0, I = 1 ri0, I = 2 ri = 1 r I0, I = 3 ri = 2 r I0, 255 I = r I = 1 0 K 255 0, Therefore: In general for n+1 intensities: = = = /255 (255 )/255 0 (1/ 0) =! j j j j r I I I0 I0 1/255 r (1/ I0), I for0" j " 255 1/ ( )/ r (1/ I ) n n! = 0, I = j n j I0 for0" j " n (13.2) (13.3)

18 Gamma correction To achieve equal steps in brightness, space logarithmically rather than linearly, so that: I I I I j + 1 = j = j j! 1 r Use the following relations: 2 3 I = I0, I = 1 ri0, I = 2 ri = 1 r I0, I = 3 ri = 2 r I0, 255 I = r I = 1 0 K 255 0, Therefore: In general for n+1 intensities: = = = /255 (255 )/255 0 (1/ 0) =! j j j j r I I I0 I0 1/255 r (1/ I0), I for0" j " 255 1/ ( )/ r (1/ I ) n n! = 0, I = j n j I0 for0" j " n (13.2) (13.3) Thus for: n = (4 intensities) and I = 1/ 8, r = 2, 3 0 intensity values of 1/8, 1/4, 1/2 and1 Display of Intensities

19 Display of Intensities Dynamic range: ratio of maximum to minimum intensities, i.e., 1/I 0 Display of Intensities Dynamic range: ratio of maximum to minimum intensities, i.e., 1/I 0 Typical on CRT anywhere from 40:1 to 200:1 => I 0 between.005 and.025: for I 0 = 0.02, EQ (13.2) yields r =

20 Display of Intensities Dynamic range: ratio of maximum to minimum intensities, i.e., 1/I 0 Typical on CRT anywhere from 40:1 to 200:1 => I 0 between.005 and.025: for I 0 = 0.02, EQ (13.2) yields r = First few, last two of 256 intensities from EQ (13.1): , , , , , ,, , Display of Intensities Dynamic range: ratio of maximum to minimum intensities, i.e., 1/I 0 Typical on CRT anywhere from 40:1 to 200:1 => I 0 between.005 and.025: for I 0 = 0.02, EQ (13.2) yields r = First few, last two of 256 intensities from EQ (13.1): , , , , , ,, , Pixel values are NOT intensities: need gamma correction to compensate for nonlinearities

21 Display of Intensities Dynamic range: ratio of maximum to minimum intensities, i.e., 1/I 0 Typical on CRT anywhere from 40:1 to 200:1 => I 0 between.005 and.025: for I 0 = 0.02, EQ (13.2) yields r = First few, last two of 256 intensities from EQ (13.1): , , , , , ,, , Pixel values are NOT intensities: need gamma correction to compensate for nonlinearities # I = kn (13.4) Non-linearities in CRT N = number of electronsin beam,proportional to grid voltage,which is proportional to pixel valuev k and# are constants # is typicallyin the range of 2.2 to 2.5 Display of Intensities Dynamic range: ratio of maximum to minimum intensities, i.e., 1/I 0 Typical on CRT anywhere from 40:1 to 200:1 => I 0 between.005 and.025: for I 0 = 0.02, EQ (13.2) yields r = First few, last two of 256 intensities from EQ (13.1): , , , , , ,, , Pixel values are NOT intensities: need gamma correction to compensate for nonlinearities # I = kn (13.4) Non-linearities in CRT N = number of electronsin beam,proportional to grid voltage,which is proportional to pixel valuev k and# are constants # is typicallyin the range of 2.2 to 2.5 Therefore, for some other constant k: I = KV #, or V = ( I / K ) 1 / # (13.5)

22 Display of Intensities Display of Intensities To display intensity I, find nearest I j from a table or: j = ROUND(log r (I/I 0 ))

23 Display of Intensities To display intensity I, find nearest I j from a table or: j = ROUND(log r (I/I 0 )) Then I j = r j I 0 Display of Intensities To display intensity I, find nearest I j from a table or: j = ROUND(log r (I/I 0 )) Then I j = r j I 0 1 / And V = ROUND (( I / K ) # ) j if no look-up table, load V j in pixel if look-up table, load j in pixel, V j in entry j j

24 Display of Intensities To display intensity I, find nearest I j from a table or: j = ROUND(log r (I/I 0 )) Then I j = r j I 0 And V = ROUND j (( I if no look-up table, load V j in pixel if look-up table, load j in pixel, V j in entry j Number of intensities needed for appearance of continuous intensity depends on ratio: need r = 1.01 for I j and I j +1 to be indistinguishable: = 1 / n 1 / n r 1 / I 0 ) or 1.01 (1 / I 0 ) solve for n: n = log 1.01 (1 / I 0 ); 1 / I 0 is dynamic range j / K ) 1 / # ( = ) (13.10) Display of Intensities

25 Display of Intensities Display Media CRT Photographic prints Photographic slides Coated paper printed in B/W Coated paper printed in color Newsprint printed in B/W Typical Dynamic Range No. of Intensities, n Display of Intensities Display Media CRT Photographic prints Photographic slides Coated paper printed in B/W Coated paper printed in color Newsprint printed in B/W Typical Dynamic Range No. of Intensities, n ink bleeding and random noise considerably decreases n in practice

26 Display of Intensities Display Media CRT Photographic prints Photographic slides Coated paper printed in B/W Coated paper printed in color Newsprint printed in B/W Typical Dynamic Range No. of Intensities, n ink bleeding and random noise considerably decreases n in practice Note: a medium s dynamic range (number of intensities) not same as gamut (number of visible colors it can display) Vision: The Eye

27 Vision: The Eye The eye can be viewed as a dynamic, biological camera: it has a lens, a focal length, and an equivalent of film. Vision: The Eye The eye can be viewed as a dynamic, biological camera: it has a lens, a focal length, and an equivalent of film. The lens must focus directly on the retina for perfect vision.

28 Vision: The Eye The eye can be viewed as a dynamic, biological camera: it has a lens, a focal length, and an equivalent of film. The lens must focus directly on the retina for perfect vision. But age, malnutrition and disease can unfocus the eye, leading to near- and farsightedness Vision: The Eye The eye can be viewed as a dynamic, biological camera: it has a lens, a focal length, and an equivalent of film. The lens must focus directly on the retina for perfect vision. But age, malnutrition and disease can unfocus the eye, leading to near- and farsightedness The retina functions as the eye's "film". It is covered with cells sensitive to light. These cells turn the light into electrochemical impulses that are sent to the brain. There are two types of cells, rods and cones

29 Vision: Rods and cones Rods: Sensitive to most visible frequencies (brightness). About 125 million in eye. Located outside of fovea, or center of retina. Used in low light (theaters, night) environments, result in achromatic (b&w) vision. Cones: L cones are sensitive to long wavelengths ($)(red), M to middle $ s (green), and S to short $ s (blue). About >6 million in eye. Highly concentrated in fovea, with S cones more evenly distributed than the others (but only about 12% are S cones). Used for high detail color vision. Vision: Rods and cones Rods: Sensitive to most visible frequencies (brightness). About 125 million in eye. Located outside of fovea, or center of retina. Used in low light (theaters, night) environments, result in achromatic (b&w) vision. rod/cone normalized absorption spectrum: Cones: L cones are sensitive to long wavelengths ($)(red), M to middle $ s (green), and S to short $ s (blue). About >6 million in eye. Highly concentrated in fovea, with S cones more evenly distributed than the others (but only about 12% are S cones). Used for high detail color vision.

30 Vision: Rods and cones Rods: Sensitive to most visible frequencies (brightness). About 125 million in eye. Located outside of fovea, or center of retina. Used in low light (theaters, night) environments, result in achromatic (b&w) vision. rod/cone normalized absorption spectrum: Cones: L cones are sensitive to long wavelengths ($)(red), M to middle $ s (green), and S to short $ s (blue). About >6 million in eye. Highly concentrated in fovea, with S cones more evenly distributed than the others (but only about 12% are S cones). Used for high detail color vision. rod/cone distribution: cones rods # rods/cones position on retina blind spot Vision: Sensitivity vs. Acuity

31 Vision: Sensitivity vs. Acuity Sensitivity is a measure of the dimmest light the eye can detect. Vision: Sensitivity vs. Acuity Sensitivity is a measure of the dimmest light the eye can detect. Acuity is a measure of the smallest object the eye can see.

32 Vision: Sensitivity vs. Acuity Sensitivity is a measure of the dimmest light the eye can detect. Acuity is a measure of the smallest object the eye can see. These two capabilities are in competition: In the fovea cones are closely packed. Acuity at its highest, sensitivity at its lowest. Outside the fovea acuity decreases rapidly. Sensitivity increases correspondingly. Blind spot examples

33 Stimuli response Stimuli response We draw a frequency response curve like this:

34 Stimuli response We draw a frequency response curve like this: to indicate how much a receptor responds to light of uniform intensity for each wavelength Stimuli response We draw a frequency response curve like this: to indicate how much a receptor responds to light of uniform intensity for each wavelength To compute response to incoming band (frequency distribution) of light, like this:

35 Stimuli response We draw a frequency response curve like this: to indicate how much a receptor responds to light of uniform intensity for each wavelength To compute response to incoming band (frequency distribution) of light, like this: we multiply the curves, wavelength by wavelength, to compute receptor response to each amount of stimulus across spectrum Stimuli response f ($) I($) R($)

36 Stimuli response Response Curve f ($) Incoming Light Distribution I($) Product of functions R($) $ $ Stimuli response Response Curve f ($) Gray area under product curve represents how much receptor sees, i.e., total response to incoming light Incoming Light Distribution I($) Product of functions R($) $ $

37 Stimuli response Response Curve f ($) Gray area under product curve represents how much receptor sees, i.e., total response to incoming light Let s call this receptor red, then red perception =!R(")d(") =!I(")f(")d" Incoming Light Distribution I($) Product of functions R($) $ $ Stimuli response Response Curve f ($) Incoming Light Distribution I($) Product of functions R($) $ $ Gray area under product curve represents how much receptor sees, i.e., total response to incoming light Let s call this receptor red, then red perception =!R(")d(") =!I(")f(")d" Response curve also called filter because it determines amplitude of response (i.e., perceived intensity) of each wavelength

38 Stimuli response Response Curve f ($) Incoming Light Distribution I($) Product of functions R($) $ $ Gray area under product curve represents how much receptor sees, i.e., total response to incoming light Let s call this receptor red, then red perception =!R(")d(") =!I(")f(")d" Response curve also called filter because it determines amplitude of response (i.e., perceived intensity) of each wavelength Where filter s amplitude is large, lets through most of incoming signal " strong response Stimuli response Response Curve f ($) Incoming Light Distribution I($) Product of functions R($) $ $ Gray area under product curve represents how much receptor sees, i.e., total response to incoming light Let s call this receptor red, then red perception =!R(")d(") =!I(")f(")d" Response curve also called filter because it determines amplitude of response (i.e., perceived intensity) of each wavelength Where filter s amplitude is large, lets through most of incoming signal " strong response Where filter s amplitude is low, filters out much/most/all of signal " weak response

39 Stimuli response Response Curve f ($) Incoming Light Distribution I($) Product of functions R($) $ $ Gray area under product curve represents how much receptor sees, i.e., total response to incoming light Let s call this receptor red, then red perception =!R(")d(") =!I(")f(")d" Response curve also called filter because it determines amplitude of response (i.e., perceived intensity) of each wavelength Where filter s amplitude is large, lets through most of incoming signal " strong response Where filter s amplitude is low, filters out much/most/all of signal " weak response This is much like impulse response and filtering you ll see in Image Processing Tristimulus Theory

40 Tristimulus Theory Spectral-response functions of f! each of the three types of cones on the human retina (not normalized) Luminous Efficiency Function # $f % (peak sensitivity at yellow-green (550nm)) Tristimulus Theory Spectral-response functions of f! each of the three types of cones on the human retina (not normalized) Luminous Efficiency Function # $f % (peak sensitivity at yellow-green (550nm)) Tristimulus theory does not explain color perception, e.g., not many colors look like mixtures of RGB (violet looks like red and blue, but what about yellow?)

41 Tristimulus Theory Spectral-response functions of f! each of the three types of cones on the human retina (not normalized) Luminous Efficiency Function # $f % (peak sensitivity at yellow-green (550nm)) Tristimulus theory does not explain color perception, e.g., not many colors look like mixtures of RGB (violet looks like red and blue, but what about yellow?) Triple Cell Response Applet: Tristimulus Theory Spectral-response functions of f! each of the three types of cones on the human retina (not normalized) Luminous Efficiency Function # $f % (peak sensitivity at yellow-green (550nm)) Tristimulus theory does not explain color perception, e.g., not many colors look like mixtures of RGB (violet looks like red and blue, but what about yellow?) Triple Cell Response Applet:

42 Vision Vision Chromatic adaption

43 Vision Chromatic adaption If color is just light of a certain wavelength, why does a yellow object always look yellow under different lighting (e.g. interior/exterior)? Vision Chromatic adaption If color is just light of a certain wavelength, why does a yellow object always look yellow under different lighting (e.g. interior/exterior)? This is the phenomenon of color constancy.

44 Vision Chromatic adaption If color is just light of a certain wavelength, why does a yellow object always look yellow under different lighting (e.g. interior/exterior)? This is the phenomenon of color constancy. Colors are constant under different lighting because the brain responds to ratios between the R, G and B cones, and not magnitudes. Vision Chromatic adaption If color is just light of a certain wavelength, why does a yellow object always look yellow under different lighting (e.g. interior/exterior)? This is the phenomenon of color constancy. Colors are constant under different lighting because the brain responds to ratios between the R, G and B cones, and not magnitudes. Metamers

45 Vision Chromatic adaption If color is just light of a certain wavelength, why does a yellow object always look yellow under different lighting (e.g. interior/exterior)? This is the phenomenon of color constancy. Colors are constant under different lighting because the brain responds to ratios between the R, G and B cones, and not magnitudes. Metamers Colors are represented to the brain as ratios of three signals: Vision Chromatic adaption If color is just light of a certain wavelength, why does a yellow object always look yellow under different lighting (e.g. interior/exterior)? This is the phenomenon of color constancy. Colors are constant under different lighting because the brain responds to ratios between the R, G and B cones, and not magnitudes. Metamers Colors are represented to the brain as ratios of three signals:! possible for different frequency combinations to appear as the same color.

46 Vision Chromatic adaption If color is just light of a certain wavelength, why does a yellow object always look yellow under different lighting (e.g. interior/exterior)? This is the phenomenon of color constancy. Colors are constant under different lighting because the brain responds to ratios between the R, G and B cones, and not magnitudes. Metamers Colors are represented to the brain as ratios of three signals:! possible for different frequency combinations to appear as the same color. These combinations are called metamers. This is why RGB color works! B G R metamers for yellow monochromatic Vision Chromatic adaption If color is just light of a certain wavelength, why does a yellow object always look yellow under different lighting (e.g. interior/exterior)? This is the phenomenon of color constancy. Colors are constant under different lighting because the brain responds to ratios between the R, G and B cones, and not magnitudes. Metamers Colors are represented to the brain as ratios of three signals:! possible for different frequency combinations to appear as the same color. These combinations are called metamers. This is why RGB color works! B G R metamers for yellow monochromatic

47 Metamers Metamers Imagine a creature with one receptor type ( red ) with response curve like this:

48 Metamers Imagine a creature with one receptor type ( red ) with response curve like this: How would it respond to each of these two light sources? Metamers Imagine a creature with one receptor type ( red ) with response curve like this: How would it respond to each of these two light sources? Both signals will generate same amount of red perception. They are metamers

49 Metamers Imagine a creature with one receptor type ( red ) with response curve like this: How would it respond to each of these two light sources? Both signals will generate same amount of red perception. They are metamers One receptor type cannot give more than one color sensation (albeit with varying brightness) Metamers Imagine a creature with one receptor type ( red ) with response curve like this: Consider creature with two receptors: I1 I2 I1 How would it respond to each of these two light sources? Both signals will generate same amount of red perception. They are metamers One receptor type cannot give more than one color sensation (albeit with varying brightness)

50 Metamers Imagine a creature with one receptor type ( red ) with response curve like this: Consider creature with two receptors: I1 I2 In principle, an infinite number of frequency distributions can simulate the effect of I2, e.g., I1 I1 How would it respond to each of these two light sources? Both signals will generate same amount of red perception. They are metamers One receptor type cannot give more than one color sensation (albeit with varying brightness) Metamers Imagine a creature with one receptor type ( red ) with response curve like this: How would it respond to each of these two light sources? Consider creature with two receptors: I1 I2 In principle, an infinite number of frequency distributions can simulate the effect of I2, e.g., I1 In practice, for In near base of response curves, amount of light required becomes impractically large. I1 Both signals will generate same amount of red perception. They are metamers One receptor type cannot give more than one color sensation (albeit with varying brightness)

51 Metamers Imagine a creature with one receptor type ( red ) with response curve like this: How would it respond to each of these two light sources? Consider creature with two receptors: I1 I2 In principle, an infinite number of frequency distributions can simulate the effect of I2, e.g., I1 In practice, for In near base of response curves, amount of light required becomes impractically large. For three types of receptors, potentially infinite color distributions (metamers) that will generate identical sensations I1 Both signals will generate same amount of red perception. They are metamers One receptor type cannot give more than one color sensation (albeit with varying brightness) Metamers Imagine a creature with one receptor type ( red ) with response curve like this: How would it respond to each of these two light sources? Both signals will generate same amount of red perception. They are metamers One receptor type cannot give more than one color sensation (albeit with varying brightness) Consider creature with two receptors: I1 I2 In principle, an infinite number of frequency distributions can simulate the effect of I2, e.g., I1 In practice, for In near base of response curves, amount of light required becomes impractically large. For three types of receptors, potentially infinite color distributions (metamers) that will generate identical sensations Conversely, no two monochromatic lights can generate identical receptor responses and therefore all look unique I1

52 Metamers Imagine a creature with one receptor type ( red ) with response curve like this: How would it respond to each of these two light sources? Both signals will generate same amount of red perception. They are metamers One receptor type cannot give more than one color sensation (albeit with varying brightness) Consider creature with two receptors: I1 I2 In principle, an infinite number of frequency distributions can simulate the effect of I2, e.g., I1 In practice, for In near base of response curves, amount of light required becomes impractically large. For three types of receptors, potentially infinite color distributions (metamers) that will generate identical sensations Conversely, no two monochromatic lights can generate identical receptor responses and therefore all look unique Thomas Young in 1801 postulated that we need 3 receptor types to distinguish gamut of colors represented by triples H, S, V (hue, saturation, value) I1 Lateral inhibition

53 Lateral inhibition Receptor cells, A and B, stimulated by neighboring regions of stimulus. A receives moderate light. A s excitation stimulates next neuron on visual chain, cell D, which transmits message toward brain. Transmission impeded by cell B, whose intense excitation inhibits cell D. Cell D fires at reduced rate. Lateral inhibition Receptor cells, A and B, stimulated by neighboring regions of stimulus. A receives moderate light. A s excitation stimulates next neuron on visual chain, cell D, which transmits message toward brain. Transmission impeded by cell B, whose intense excitation inhibits cell D. Cell D fires at reduced rate. Intensity of cell c j =I(c j ) is function of c j s excitation e(c j ) inhibited by its neighbors with attenuation coefficients a k that decrease with distance. Thus, % I ( c ) = e( c ) - ' e( c ) j j k & j k k

54 Lateral inhibition Receptor cells, A and B, stimulated by neighboring regions of stimulus. A receives moderate light. A s excitation stimulates next neuron on visual chain, cell D, which transmits message toward brain. Transmission impeded by cell B, whose intense excitation inhibits cell D. Cell D fires at reduced rate. Intensity of cell c j =I(c j ) is function of c j s excitation e(c j ) inhibited by its neighbors with attenuation coefficients a k that decrease with distance. Thus,! At boundary more excited cells inhibit their less excited neighbors even more and vice versa. Thus, at boundary dark areas even darker than interior dark ones, light areas are lighter than interior light ones. % I ( c ) = e( c ) - ' e( c ) j j k & j k k Lateral inhibition Receptor cells, A and B, stimulated by neighboring regions of stimulus. A receives moderate light. A s excitation stimulates next neuron on visual chain, cell D, which transmits message toward brain. Transmission impeded by cell B, whose intense excitation inhibits cell D. Cell D fires at reduced rate. Intensity of cell c j =I(c j ) is function of c j s excitation e(c j ) inhibited by its neighbors with attenuation coefficients a k that decrease with distance. Thus,! At boundary more excited cells inhibit their less excited neighbors even more and vice versa. Thus, at boundary dark areas even darker than interior dark ones, light areas are lighter than interior light ones.! Nature s edge detection % I ( c ) = e( c ) - ' e( c ) j j k & j k k

55 Lateral inhibition The light striking rods and cones in the retina is not summed uniformly: The nerves that combine the signals from the rods or cones sum with a center/surround opponency. This results in Mach-bending Lateral inhibition The light striking rods and cones in the retina is not summed uniformly: The nerves that combine the signals from the rods or cones sum with a center/surround opponency. This results in Mach-bending Mach-bends: Perceptual artifacts caused by the eye s lateral inhibition which appear at any discontinuity or drastic change in the rate of shading.

56 Lateral inhibition The light striking rods and cones in the retina is not summed uniformly: The nerves that combine the signals from the rods or cones sum with a center/surround opponency. This results in Mach-bending Mach-bends: Perceptual artifacts caused by the eye s lateral inhibition which appear at any discontinuity or drastic change in the rate of shading. When one receptor responds to a high intensity, it inhibits its neighboring receptors responses. Lateral inhibition The light striking rods and cones in the retina is not summed uniformly: The nerves that combine the signals from the rods or cones sum with a center/surround opponency. This results in Mach-bending Mach-bends: Perceptual artifacts caused by the eye s lateral inhibition which appear at any discontinuity or drastic change in the rate of shading. When one receptor responds to a high intensity, it inhibits its neighboring receptors responses. Receptors on the bright side of a discontinuity receive less inhibition from the dark side.

57 Lateral inhibition The light striking rods and cones in the retina is not summed uniformly: The nerves that combine the signals from the rods or cones sum with a center/surround opponency. This results in Mach-bending Mach-bends: Perceptual artifacts caused by the eye s lateral inhibition which appear at any discontinuity or drastic change in the rate of shading. When one receptor responds to a high intensity, it inhibits its neighboring receptors responses. Receptors on the bright side of a discontinuity receive less inhibition from the dark side. Receptors on the dark side of a discontinuity receive more inhibition from the light side. Lateral inhibition The light striking rods and cones in the retina is not summed uniformly: The nerves that combine the signals from the rods or cones sum with a center/surround opponency. This results in Mach-bending Mach-bends: Perceptual artifacts caused by the eye s lateral inhibition which appear at any discontinuity or drastic change in the rate of shading. When one receptor responds to a high intensity, it inhibits its neighboring receptors responses. Receptors on the bright side of a discontinuity receive less inhibition from the dark side. Receptors on the dark side of a discontinuity receive more inhibition from the light side.! Imaginary dark and light lines appear at facet boundaries. Flat shading of more facets does not necessarily look smoother.

58 Color afterimage example + Stare at the plus sign for about 30 seconds (as you do this you probably will see some colors around the blue and green circles). Color afterimage example +

59 Color afterimage example + You probably saw a yellow and desaturated reddish circle. Hering s chromatic opponent channels

60 Hering s chromatic opponent channels Additional neural processing three receptor elements have excitatory and inhibitory connections with neurons higher up that correspond to opponent processes one pole activated by excitation, other by inhibition All colors can be described in Light of 450 nm terms of 4 psychological S I L color primaries R, G, B, and Y However, a color is never reddish-greenish or bluish-yellowish: idea of two antagonistic opponent color channels, red-green and yellow-blue Y-B R-G BK-W Each channel is a weighted sum The blue/yellow and red/green of receptor outputs linear pairs are called complementary Hue: Blue + Red = Violet mapping colors. Mixing the proper shades of them in the proper amounts produces white light. Vision: Beyond the Eye

61 Vision: Beyond the Eye Beyond the eye, visual signals move through different processing stages in the brain. Vision: Beyond the Eye Beyond the eye, visual signals move through different processing stages in the brain. There seem to be two main pathways Magnocellular: low-resolution, motion sensitive, and primarily achromatic pathway Parvocellular: high-resolution, static, and primarily chromatic pathway

62 Vision: Beyond the Eye Beyond the eye, visual signals move through different processing stages in the brain. There seem to be two main pathways Magnocellular: low-resolution, motion sensitive, and primarily achromatic pathway Parvocellular: high-resolution, static, and primarily chromatic pathway Color vision is processed in three dimensions. Perceptual terms: hue, saturation, and luminance Hue: In colorimetry: the dominant wavelength of the light entering the eye Saturation: In colorimetry: exitation purity, inversely related to the amount of white light in the light entering the eye (e.g. red, fully saturated; pink, not fully saturated) Luminance: the intensity of the light entering the eye (e.g. light with a dial) Lightness: luminance from a reflecting object. In colorimetry: luminance Brightness: luminance from a light source. In colorimetry: luminance Vision: Beyond the Eye Beyond the eye, visual signals move through different processing stages in the brain. There seem to be two main pathways Magnocellular: low-resolution, motion sensitive, and primarily achromatic pathway Parvocellular: high-resolution, static, and primarily chromatic pathway Color vision is processed in three dimensions. Perceptual terms: hue, saturation, and luminance Hue: In colorimetry: the dominant wavelength of the light entering the eye Saturation: In colorimetry: exitation purity, inversely related to the amount of white light in the light entering the eye (e.g. red, fully saturated; pink, not fully saturated) Luminance: the intensity of the light entering the eye (e.g. light with a dial) Lightness: luminance from a reflecting object. In colorimetry: luminance Brightness: luminance from a light source. In colorimetry: luminance Chromaticity: the hue and saturation of light (not luminance)

63 Color Models Color Models Used to describe color as accurately as possible.

64 Color Models Used to describe color as accurately as possible. Uses the fact that colors can be described by combinations of three basic colors, called primary colors. Color Models Used to describe color as accurately as possible. Uses the fact that colors can be described by combinations of three basic colors, called primary colors. CIE (Commission International de l'eclairage - International Color Commision) organisation produced two models for defining color: 1931: Measured on 10 subjects (!) on samples subtending 2 (!) degrees of the field of view 1964: Measured on larger number of subjects subtending 10 degrees of field of view The CIE 1931 model is the most commonly used It defines three primary colors X, Y and Z that can be used to describe all visible colors, as well as a standard white, called C. The range of colors that can be described by combinations of other colors is called a color gamut.! Since it is impossible to find three colors with a gamut containing all visible colors, the CIE s three primary colors are imaginary. They cannot be seen, but they can be used to define other visible colors.

65 Doing the Experiment Doing the Experiment People sit in a dark room matching colors: But any three R, G and B can t match all colors... (for reasons we ll be exploring soon): Sometimes need to add some R to the sample you are trying to match: (Expressed mathematically as -R )

66 CIE Space for Color Matching CIE Space for Color Matching Defined X, Y, and Z primaries to replace red, green and blue primaries

67 CIE Space for Color Matching Defined X, Y, and Z primaries to replace red, green and blue primaries x %, y %, and z %, color matching functions for these primaries CIE Space for Color Matching Defined X, Y, and Z primaries to replace red, green and blue primaries x %, y %, and z %, color matching functions for these primaries Y chosen so that y % matches luminous efficiency function

68 CIE Space for Color Matching Defined X, Y, and Z primaries to replace red, green and blue primaries x %, y %, and z %, color matching functions for these primaries Y chosen so that y % matches luminous efficiency function x %, y %, and z % are linear combinations of r %, g %, and b % CIE Space for Color Matching Defined X, Y, and Z primaries to replace red, green and blue primaries x %, y %, and z %, color matching functions for these primaries Y chosen so that y % matches luminous efficiency function x %, y %, and z % are linear combinations of r %, g %, and b % => RGB i! XYZ i via a matrix

69 CIE Space for Color Matching Defined X, Y, and Z primaries to replace red, green and blue primaries x %, y %, and z %, color matching functions for these primaries Y chosen so that y % matches luminous efficiency function x %, y %, and z % are linear combinations of r %, g %, and b % => RGB i! XYZ i via a matrix CIE Space for Color Matching Defined X, Y, and Z primaries to replace red, green and blue primaries x %, y %, and z %, color matching functions for these primaries Y chosen so that y % matches luminous efficiency function x %, y %, and z % are linear combinations of r %, g %, and b % => RGB i! XYZ i via a matrix Mapping the mathematical color matching functions to x! y!, and z! for the 1931 CIE X, Y, and Z primaries. They are defined tabularly at 1 nm intervals for color samples that subtend 2 field of view on retina

70 CIE Space for Color Matching Defined X, Y, and Z primaries to replace red, green and blue primaries x %, y %, and z %, color matching functions for these primaries Y chosen so that y % matches luminous efficiency function x %, y %, and z % are linear combinations of r %, g %, and b % => RGB i! XYZ i via a matrix Wavelength! (nm) Mapping the mathematical color matching functions to x! y!, and z! for the 1931 CIE X, Y, and Z primaries. They are defined tabularly at 1 nm intervals for color samples that subtend 2 field of view on retina CIE 1931 Model

71 CIE 1931 Model CIE 1931 Model To define a color in CIE model, provide weights for the X, Y and Z primaries, just as you would for an RGB display (e.g. color = xx + yy + zz). X, Y and Z form a three dimensional color volume. We can ignore the dimension of luminance by normalizing with total light intensity, x+y+z = 1. This gives chromaticity values: x = x/(x+y+z) y = y/(x+y+z) z = 1 - x - y

72 CIE 1931 Model To define a color in CIE model, provide weights for the X, Y and Z primaries, just as you would for an RGB display (e.g. color = xx + yy + zz). X, Y and Z form a three dimensional color volume. We can ignore the dimension of luminance by normalizing with total light intensity, x+y+z = 1. This gives chromaticity values: x = x/(x+y+z) y = y/(x+y+z) z = 1 - x - y Plotting x and y gives the CIE chromaticity diagram. Color gamuts are found by taking the convex hull of the primary colors. Complements are found by inscribing a line from the color through C to the edge of the diagram. CIE 1931 Model

73 CIE 1931 Model Hue of a color: found by inscribing a line from C (white) through the color to the edge of the diagram. The hue is the wavelength of the color at the intersection of the edge and the line. CIE 1931 Model Hue of a color: found by inscribing a line from C (white) through the color to the edge of the diagram. The hue is the wavelength of the color at the intersection of the edge and the line. Saturation of a color: found by taking the ratio of the distance of the color from C on the above line and the length of the whole line. Complementary colors can be mixed to produce white light (a non-spectral color!) White can be produced by (approx) constant spectral distribution as well as by only two complementary colors, e.g., greenish-blue, D, and reddish-orange, E. Some nonspectral colors (colors not on spectral locus, like G) cannot be defined by dominant wavelength; defined by complementary dominant wavelength.

74 Color gamuts Color gamuts Colors add linearly in CIE: All mixture of I and J lie on the line connecting them.

75 Color gamuts Colors add linearly in CIE: All mixture of I and J lie on the line connecting them. Thus, all possible mixtures of I, J and any third color, K, (or additional colors) lie within their convex hull. Called the color gamut. Color gamuts Colors add linearly in CIE: All mixture of I and J lie on the line connecting them. Thus, all possible mixtures of I, J and any third color, K, (or additional colors) lie within their convex hull. Called the color gamut. No finite number of primaries can include all visible colors!

76 Color gamuts Colors add linearly in CIE: All mixture of I and J lie on the line connecting them. Thus, all possible mixtures of I, J and any third color, K, (or additional colors) lie within their convex hull. Called the color gamut. No finite number of primaries can include all visible colors! Color gamuts Colors add linearly in CIE: All mixture of I and J lie on the line connecting them. Thus, all possible mixtures of I, J and any third color, K, (or additional colors) lie within their convex hull. Called the color gamut. No finite number of primaries can include all visible colors!

77 Color gamuts Colors add linearly in CIE: All mixture of I and J lie on the line connecting them. Thus, all possible mixtures of I, J and any third color, K, (or additional colors) lie within their convex hull. Called the color gamut. No finite number of primaries can include all visible colors! gamut of printer gamut of monitor Color gamuts Colors add linearly in CIE: All mixture of I and J lie on the line connecting them. Thus, all possible mixtures of I, J and any third color, K, (or additional colors) lie within their convex hull. Called the color gamut. No finite number of primaries can include all visible colors! gamut of printer! Device dependent gamut: Mapping colors must be done carefully! gamut of monitor

78 Why the Chromaticity Diagram is not triangular Why the Chromaticity Diagram is not triangular No gamut described by a linear combination of n physical (real) primaries (yielding a convex hull) can simulate the eye s responses to all visible colors

79 Why the Chromaticity Diagram is not triangular No gamut described by a linear combination of n physical (real) primaries (yielding a convex hull) can simulate the eye s responses to all visible colors Review of CIE: shape of CIE space determined by Matching Experiment: subject shown color and asked to create metameric match from colored monochromatic primaries, R, G and B most colors can be matched, some can t because of way response curves overlap would need negative amounts of some primary to match all visible color samples; not physically possible, but can be simulated by adding that color to sample to be matched. to simplify, CIE primaries X, Y, and Z used to get all positive color matching functions Chromatic Opponent Channels on Chromaticity Diagram

80 Chromatic Opponent Channels on Chromaticity Diagram Let s see in another way why 3 (indeed n) physical primaries aren t sufficient to match an arbitrary color by looking at response function. Taken from Falk s Seeing the Light, (Harper and Row, 1986) Chromatic Opponent Channels on Chromaticity Diagram Let s see in another way why 3 (indeed n) physical primaries aren t sufficient to match an arbitrary color by looking at response function. S I L Y-B R-G BK-W Taken from Falk s Seeing the Light, (Harper and Row, 1986) Why would red-green and blue-yellow be useful axes to specify color with?

81 Color Models for Raster Graphics Color Models for Raster Graphics Purpose: specify colors in some gamut Gamut is a subset of all visible chromaticities so model does not contain all visible colors 3D color coordinate system subset containing all colors within a gamut Means to convert to other model(s) Example color model: RGB 3D Cartesian coordinate system unit cube subset Use CIE XYZ space to convert to and from all other models

82 Color Models for Raster Graphics Color Models for Raster Graphics Hardware-oriented models: not intuitive do not relate to concepts of hue, saturation, brightness Green Yellow RGB, used with color CRT monitors YIQ, broadcast TV color system Cyan Black Red CMY (cyan, magenta, yellow) color printing CMYK (cyan, magenta, yellow, black) color printing Blue Magenta IRODORI, six-primary-color projection system

83 Color Models for Raster Graphics Hardware-oriented models: not intuitive do not relate to concepts of hue, saturation, brightness Green Yellow RGB, used with color CRT monitors YIQ, broadcast TV color system Cyan Black Red CMY (cyan, magenta, yellow) color printing CMYK (cyan, magenta, yellow, black) color printing Blue Magenta IRODORI, six-primary-color projection system User-oriented models HSV (hue, saturation, value) also called HSB (B for brightness) HLS (hue, lightness, saturation) The Munsell system CIE Lab The RGB Color Model The RGB cube (Grays on dotted main diagonal)