Image and Multidimensional Signal Processing

Size: px

Start display at page:

Download "Image and Multidimensional Signal Processing"

Hester Sims
7 years ago
Views:

1 Image and Multidimensional Signal Processing Professor William Hoff Dept of Electrical Engineering &Computer Science

2 Image Compression - Lossy

3 Lossy Compression Reconstructed image is different from original Hopefully differences are unnoticeable, or minor We will look at: Block transform coding methods, using the discrete cosine transform (such as the JPEG standard) Predictive coding 3

4 Block Transform Coding Divides the image into subimages, or blocks Apply a transform (e.g., Fourier) to each block Quantize and encode the coefficients Compression 4

5 Transform Coding General forward transform of image g, size nxn n 1 n 1 T( u, v) g( x, y) r( x, y, u, v) x 0 y 0 Inverse transform n 1 n 1 g( x, y) T( u, v) s( x, y, u, v) u 0 v 0 Example: Fourier transform r,s are the forward and inverse transformation kernels (also called basis functions) 1 r e, s e n j2 ( ux vy)/ n j2 ( ux vy)/ n 2 5

6 Example: Walsh-Hadamard Transform (WHT) Kernels: r( x, y, u, v) s( x, y, u, v) 1 n 1 m 1 i 0 b i ( x) pi ( u) bi ( y) pi ( v) where nxn is the size of the kernel, and n = 2^m b k is the k th bit Summation is done in modulo 2 arithmetic The p s are p 0 (u) = b m-1 (u) p 1 (u) = b m-1 (u) + b m-2 (u) p 2 (u) = b m-2 (u) + b m-3 (u) : p m-1 (u) = b 1 (u) + b 0 (u) 6

7 Example WHT Basis Functions

8 Discrete Cosine Transform (DCT) Kernels: g( x, y, u, v) h( x, y, u, v) (2x 1) u (2y 1) v ( u) ( v)cos cos 2N 2N where ( u) 1 N 2 N for u for u 0 1,2,..., N 1 DCT used in JPEG (wavelets are used in JPEG2000) % Show DCT kernels N = 32; x=0:n-1; y=0:n-1; u = 1; v = 4; au = sqrt(2/n); av = sqrt(2/n); if u==1 au = sqrt(1/n); end if v==1 av = sqrt(1/n); end gx = au*cos((2*x+1)*u*pi/(2*n)); gy = av*cos((2*y+1)*v*pi/(2*n)); figure, plot(x,gx); figure, plot(y,gy); g = gx'*gy; figure, surf(g); 8

9 DCT 4x4 Basis Functions 9

10 Approximation Errors Apply a transform to each 8x8 subimage block Keep highest 50% of coefficients in each block Reconstructed Error Image Then reconstruct image (by taking the inverse transform) using the remaining coefficients RMS error: 2.32 RMS error: 1.78 RMS error:

11 Effect of Subimage Size Image: lena Truncate smallest 75% coefficients in each subimage Figure

12 12

13 Methods: Quantizing Transform Coefficients Threshold coding: Within each 8x8 subimage, keep the top N% of the coefficients; or those with magnitude greater than a threshold Matlab exercise with blkproc Zonal coding: Keep coefficients with maximum variance across all subimages Matlab s dctdemo Then quantize to a fixed or variable number of bits 13

14 Threshold coding Matlab example dct2 The function dct2 performs 2D discrete cosine transform on a matrix B = dct2(a) The function idct2 performs the reverse transformation A2 = idct2(b) blkproc Use blkproc to apply dct2 to each 8x8 block J = blkproc(i,[8 8],@dct2); You could also apply your own function (eg., threshold) to each block J = blkproc(i,[8 8],@mythresh); 14

15 DCT is performed on each 8x8 subimage (block) >> I = imread('cameraman.tif'); >> J = blkproc(i,[8 8],@dct2); DCT coefficients 15

16 A note on functions in Matlab You can write a function and call it from your program Syntax: function B = myfunc(a) % This function computes something from A and returns B : B = Store this in a file called myfunc.m Put in current working directory 16

17 Truncating coefficients Write a function called mytrunc that truncates the smallest 75% coefficients in an image The function should Take the absolute value of each pixel in the image Sort the values in ascending order Find the value that is 75% down the list Threshold the image using that value function B = mytrunc(a) % Truncate the lowest 75% of the magnitudes within A Aabs = abs(a); vals = sort(aabs(:)); % Sort the values from low to high : B = A.* (Aabs > thresh); 17

18 clear all close all I = double(imread('lena.tif')); wsize = 8; J = blkproc(i,[wsize wsize],@dct2); imshow(j,[]), title('j'); % Truncate 75% of the values within each block Jtrunc = blkproc(j,[wsize wsize],@mytrunc); figure, imshow(jtrunc,[]), title('jtrunc'); Try on different block sizes Do you get this result? pct = sum(sum(jtrunc == 0))/(size(I,1)*size(I,2)); fprintf('percentage of zero coeffs: %f\n',100*pct); K=blkproc(Jtrunc,[wsize wsize],@idct2); figure, imshow(k,[]), title('k'); R = I - K; disp('rms error:'); sqrt(mean2(r.^ 2)) 18

19 Methods: Quantizing Transform Coefficients Threshold coding: Within each 8x8 subimage, keep the top N% of the coefficients; or those with magnitude greater than a threshold Matlab exercise with blkproc Zonal coding: Keep coefficients with maximum variance across all subimages Matlab s dctdemo Then quantize to a fixed or variable number of bits 19

20 Zonal coding Matlab dctdemo Apply DCT to each 8x8 block Discard coefficients with the smallest variance Original Saturn Image DCT coefficients Reconstructed Image Error Image 20

21 Ordering sequence Convert a nxn matrix to a one-dimensional vector Typical threshold mask (we keep the coefficients in the shaded positions) Different mask for each subimage Ordering sequence, to convert 8x8 array to a 64x1 vector Resulting vector will have long runs of 0s 21

22 Quantizing Coefficient Magnitudes Once we decide which coefficients to keep, we now quantize the remaining nonzero coefficients We divide each coefficient by a number (depending on its location) and round to integer Smaller magnitudes can be represented by fewer bits Division by a large number will tend to give a zero result Z(u,v) 22

23 Variable quantization You can achieve more or less compression by scaling the normalization matrix Z (i.e., dividing by larger values) Resulting compression: (a) 12:1 (b) 19:1 (c) 30:1 (d) 49:1 (e) 85:1 (f) 182:1 23

24 JPEG Algorithm Divide into 8x8 subimages Discrete cosine transform on each Quantize the coefficients Uses threshold coding Order coefficients in zig-zag pattern Encode the 1D sequence using run-length encoding and Huffman encoding Threshold quantization array Ordering of coefficients 24

25 Input 8x8 subimage Subtract 128 from each value 25

26 Do forward DCT Quantize and truncate values Example: round(-415/16) =

27 JPEG Algorithm (continued) Re-order in zig-zag pattern Use variable length code words to encode non-zero values Use run length encoding to encode # zeros Results bit count = 92 Compression: 512/92 = 5.6:1 27

28 Details of coding the coefficients We use a pre-computed Huffman code (Appendices A.4-A.5) It assumes that values are clustered around zero The code word consists of a base code (which represents the most significant bits), followed by a coding of the least significant bits The base code is determined by the magnitude of the coefficient First find what range the coefficient value lies in, and the corresponding category K Then look up the base code for that category 28

29 Predictive Coding Takes advantage of interpixel redundancy Predict next pixel from previous pixel, encode only the difference from the actual and the predicted 29

30 A simple predictor: f pred (x,y) = f(x,y-1) 30

31 Lossy Predictive Coding Error values are quantized Predictions by encoder and decoder must be same to prevent error buildup 31

32 Optimal quantization of Error Values Lloyd-Max interval quantizer: staircase function with L values 32

33 Assume a Laplacian pdf Choosing Intervals p e ( e) 1 e 2 2 e The optimal 2-bit quantizer is Quantized output Error (e)

34 Lloyd-Max Quantization Quantized output bit (4 level): Error

35 Compression of Image Sequences To compress a video we take advantage of the redundancy between successive frames See NASA Shuttle Movie 1829 color frames (~1 minute) Compression using Quicktime (H.264) Reduction from 5 GB to 45 MB (100:1) Predict the value of each pixel, transmit the residual error Simplest prediction method: Prediction is the value of the pixel in the previous image (forward prediction) Periodically insert I-frames ( independent frames) These are compressed as single images (like DCT) Needed for initialization Or to handle cases where there are too many changes between successive images Can also base the prediction on the next frame (backward prediction) 35

36 Motion Compensation Predict motion of small blocks (e.g, 16x16) Encoder has to estimate motion of each macroblock usually finds dx,dy to minimize mean absolute distortion, which is the average of the absolute values of the differences 36

37 Example std dev = 12.7 std dev =

38 Example Subpixel Motion Estimation std dev = 12.7 std dev = 4.4 Need to interpolate values std dev = 4 std dev =

39 Video Compression Standards P: prediction (forward) B: backward prediction 39

Introduction to image coding

Introduction to image coding Image coding aims at reducing amount of data required for image representation, storage or transmission. This is achieved by removing redundant data from an image, i.e. by