Image Representation and Description

Size: px

Start display at page:

Download "Image Representation and Description"

Paula Hardy
7 years ago
Views:

1 Lecture Image Representation and Description Shahram Ebadollahi 4/5/8 DIP ELEN E483

2 Image Description Recognition High-level Image Representation Image Understanding 4/5/8

3 Lecture Outline Image Description Shape Descriptors Texture & Texture Descriptors SIFT Motion Descriptors Color Descriptors 4/5/8 3

4 Shape Description Shape Represented by its Boundary Shape Numbers Fourier Descriptors Statistical Moments Shape Represented by its Interior Topological Descriptors Moment Invariants 4/5/8 4

5 Boundary Representation: Freeman Chain Code Boundary representation Original boundary Sub-sampled boundary Chain code of boundary Chain code for 4-neighborhood Chain code for 8-neighborhood 4/5/8 5

Chain Code: example 8-directional chain code 6666666664444444 Starting point normalized chain

6 Chain Code: example 8-directional chain code Starting point normalized chain code Rotation normalized chain code First difference of chain code 4/5/8 6

7 Shape Number A boundary descriptor 4/5/8 7

8 4/5/8 8 + K jy x s / K u e s u a K K u j π / K e u a K s K u K u j π Fourier Descriptor Boundary descriptor Fourier

9 Boundary Reconstruction using Fourier Descriptors % 5% % 5% 868 descriptors Only 8 descriptors.5%.5%.63%.8% 4/5/8 9

10 Boundary Representation: Signatures Represent -D boundary shape using -D signature signal 4/5/8

11 Boundary Representation: Signatures 4/5/8

12 Boundary Description using Statistical Moments A n µ n v vi m p vi n-th moment of v i A m i v p i v i 4/5/8

13 Region Descriptors - Simple Area Perimeter Compactness Circularity Ratio Mean/Median intensity Max/Min intensity Normalized area R c P A / 4π Area of circle with same perimeter as the shape C : R c : r perimeter /Area r b ab 4π 5π π π r 4/5/8 3

14 4/5/8 4

distortions H: # holes in the image C: # connected

15 Topological Region Descriptors Topological properties: Properties of image preserved under rubber-sheet distortions H: # holes in the image C: # connected components H C E- E C-H: Euler Number H C3 E3 V Q + F C H E H C E H C E- 4/5/8 5

16 Geometric Moment Invariants m m p q x y f x y dxdy pq pq M N x y x p p+q-th D geometric moment y q f x y Projection of fxy onto monomial x p y q Why use moments? Geometric moments of different orders represent spatial characteristics of the image intensity distribution m Total intensity of image. For binary image area x y m m / m / m Intensity centroid binary image geometrical center 4/5/8 6

17 Central Moments µ pq M N x y p q x x y y f x y [Translation invariance] µ m µ µ µ µ µ Variance about the centroid covariance Scaled Central Moment λ µ ' / pq pq µ ' p+ q+ / Scale and translation invariant µ pq ' pq p+ q+ α µ Normalized Un-Scaled Central Moment η p+ q+ / pq µ pq µ 4/5/8 7 /

18 Moment Invariants translation scale mirroring rotation φ + η η φ + η φ η η 4 3 η3 η + η η3 4 η3 + η + η η3 φ + φ 5 φ 6 φ 7 ϕ η + η 4/5/8 8

19 Affine Transform & Affine Moment Invariants x' T x y' T y x y x y x' y' m m r r m m r r a b r r x x r r y y In practice: bilinear transform 4 pairs of corresponding points needed to find coefficients In practice: affine transform 3 pairs of corresponding points needed to find coefficients x' a + ax + a y + a3xy ' b + b x + b y + b xy y 3 x' a + ax + a y ' b + b x + b y y Rotation: Scale: Sew: x ' x cosφ + y sin φ y ' xsinφ + y cosφ x ' ax y ' by x ' x + y tanφ y ' y 4/5/8 9

20 Elliptical Shape Descriptors I I Principal moment of inertia: µ + µ + [ µ µ µ + µ [ µ µ + 4µ ] / + 4µ ] / Image ellipse characterizes fundamental shape features and also D position and orientation I + I m / I I / I + I spreadness elongation θ.5 tan µ µ µ θ a I µ b I µ / / / / 4/5/8

21 Texture - Definition 4/5/8

Texture Quantification Methods Statistical: compute local features at each point in image and derive a set of statistics from the distribution of local features st nd and higher-order statistics

22 Texture Quantification Methods Statistical: compute local features at each point in image and derive a set of statistics from the distribution of local features st nd and higher-order statistics based on how many points are used to define local features Structural: texture is considered to be composed of texture elements. Properties of the texture elements and their spatial placement rules characterizes the texture Original texture can be reconstructed from its structural description 4/5/8

23 Statistical Texture Analysis st order statistics image f h f histogram Obtain statistics of the histogram: L Mean: ih i i Variance: Sewness: Entropy: L i L i L i i µ i µ 3 h i h i h ilog h i : average intensity : measure of intensity contrast Measure of variability of intensity 4/5/8 3

24 4/5/8 4

25 Statistical Texture Analysis st order statistics histogram statistics image Sewness.8 Entropy.88 Sewness.44 Entropy.77 Sewness -.9 Entropy.97 4/5/8 5

26 Statistical Texture Analysis nd order statistics: Co-occurrence f m n d θ j P d θ i j Pr[ f m n i f m n j] f m n i L j Joint gray-level histogram of pairs of pixels D histogram i P d θ i j L 4/5/8 6

27 4/5/8 7 Statistical Texture Analysis nd order statistics: Co-occurrence j } : { j n m f i l f d n l m N M N M n m l j i P d θ } : { 45 j n m f i l f d n l d m d n l d m N M N M n m l j i P d θ 9 j i P d θ 35 j i P d θ { } is set cardinality

28 Statistical Texture Analysis nd order statistics: Co-occurrence example image Co-occurrence matrix 4/5/8 8

29 Statistical Texture Analysis nd order statistics: Co-occurrence statistics Angular nd moment energy: measure of image homogeneity Maximum Probability: Entropy: L L i j P d θ i j max P d i j L i j L i j θ P d θ i jlogp d θ i j Contrast: measure of local variations Correlation: measure of image linearity L L L L i j L L i j i j λ d θ i j 4/5/8 9 κ d θ σ x P [ ijp i j] µ µ µ x i P d θ i j σ x i µ x i j σ y L L P d θ i j x y i j

30 4/5/8 3

31 4/5/8 3 Statistical Texture Analysis nd order statistics: Difference Statistics Angular nd moment energy: Mean: Contrast: P L d θ L d P θ log L d d P P θ θ Entropy: L P d θ j i L j i d d j i P P } { θ θ is a subset of co-occurrence matrix

32 Statistical Texture Analysis nd order statistics: Autocorrelation C ff p q M pn q l M M MN p N q f l N l f + p l f l + q Large texture elements autoccorrelation decreases slowly with increasing distance Small texture elements autoccorrelation decreases rapidly with increasing distance Periodic texture elements periodic increase & decrease in autocorrelation value 4/5/8 3

33 4/5/8 33 Statistical Texture Analysis nd order statistics: Fourier Power Spectrum v u F y x f v u F v u P π θ θ r P r P / L r r P P θ θ Power Spectrum u v u v Indicator for size of dominant texture element or texture coarseness Indicator for the directionality of the texture Note: } { v u F F C ff

34 Law s Texture Energy Measures L [] E [ ] 3 3 S3 [ ] L E S R W 5 5 [464] [ ] [ ] [ 46 4] [ ] L T 5 S Convolute different Law s mass with image Compute energy statistics 4/5/8 34

35 4/5/8 35

36 4/5/8 36

37 4/5/8 37

38 4/5/8 38

39 4/5/8 39

40 Motion object Difference image: d j i j if f i j f i ε otherwise No motion direction information! f f object motion d cum i j n a f i j f i j Cumulative difference image Tells us how often the image gray level was different from gray-level of reference image absolute positive negative 4/5/8 4

Motion Field A velocity vector is assigned to each pixel in the image Velocities due to relative motion between camera and the 3D scene Image change

41 Motion Field A velocity vector is assigned to each pixel in the image Velocities due to relative motion between camera and the 3D scene Image change due to motion during a time interval dt Velocity field that represents 3-dimensional motion of object points across -dimensional image Motion field 4/5/8 4

42 Optical flow Motion of brightness patterns in image sequence Assumptions for computing optical flow: Observed brightness of any object point is constant over time Nearby points in the image plane move in a similar manner f f f f f x + dx y + dy t + dt f x y t + dx + dy + dt + O x y t f x y t + f dx + f dy + f dt + O dx x + dx y + dy t + dt f x y t ft f x + dt x y 4/5/8 4 t f y dy dt dx dy c u v dt dt Gray-level difference at same location over time is equivalent to product of spatial gray-level difference and velocity ft f xu + f yv f. c nown unnown

43 Optical Flow Constraints f f u t x + f y v no spatial change in brightness induce no temporal change in brightness no discernible motion motion perpendicular to local gradient induce no temporal change in brightness no discernible motion motion in direction of local gradient induce temporal change in brightness discernible motion only motion in direction of local gradient induces temporal change in brightness and discernible motion 4/5/8 43

44 Optical flow! Motion Field MF OF MF OF 4/5/8 44

45 Which descriptors? Image Feature Evaluation. Prototype Performance Classify Segment image using different features Evaluate which feature is optimal minimum classification error. Figure of Merit Establish functional distance measurements between set of image features large distance low classification error Bhattacharyya distance 4/5/8 45

Geometric Camera Parameters

Geometric Camera Parameters What assumptions have we made so far? -All equations we have derived for far are written in the camera reference frames. -These equations are valid only when: () all distances