Lab 5: Image Compression

Transcription

1 Lab 5: Image Compression Due Date: There are five group exercises, which must be shown to the TA in Lab or office hours during the weeks of Feb Objective This lab studies lossy and lossless image compression. We experiment with several basic ways of compressing grayscale images: lossless compression lossy compression via: quantization of gray levels eliminating selected frequency components quantization of transform coefficients You will also experiment with aspects of the JPEG compression algorithm. Prelab In this lab, you will implement various compression schemes on 3 different images, so that you can see how the different image content plays a role in what compression schemes work best. Since there are a lot of combinations, you should divide up the work among different members of your team. Read the lab assignment and decide your strategy ahead of time, e.g. by having different people responsible for different images or for different compression methods. All team members will be responsible for answering questions about the lab, so make sure that all team members look at the results for all cases and discuss the differences. Also, make sure everyone is familiar with the MATLAB commands for displaying images, since you ll need to look at a lot of images and make comparisons. Remember that you can look at images side-by-side either by concatenating them into a single variable for display in one figure, or by using the figure command before the subsequent display command. Since you may want to use the zoom-in feature for looking at detailed differences, it may be better to have multiple figures. If you want to print out images side-by-side, then concatenation is easier. In displaying images, consider using >> imshow(image, []); The [ ] in the imshow command indicates that the min value in the image should be mapped to black and the max to white. In this way, you do not need to worry about normalizing to the [0,1] range. This will be useful, since we ll be doing a lot of manipulations of the images in double (vs. uint8) format. 1 Introduction In this lab we focus on compression of digital images. Image compression schemes aim to decrease the number of bits used to represent the image. Using fewer bits allows the image to take up less storage space on a computer, and it can also be transmitted faster (over a network, via satellite, etc.). Compression is a 1

2 general concept that applies to all types of data, not just images. For example, compression algorithms are also used for English text, digital music, and digital movie data. Some of the concepts that we experiment with are general compression concepts that apply to other types of data, while others apply specifically to images. Images that are displayed on webpages (in the.jpg,.gif and.png formats) typically use some type of compression. In this lab, since we want to look at the effect of compression that we do ourselves, it is important to use images that have not been compressed first. Download the following.pgm images from the class webpage: flowers.pgm peppers gray.pgm brainscan 8bit.pgm These images are all 8-bit/256 gray level images of various sizes. Read them into matrix variables in MATLAB using the imread command. In the exercises to follow, you should explore various compression schemes on all three images, so that you can see how the different image content plays a role in what compression schemes work best. (To allow time for exploring multiple methods, you will only implement a few things in MATLAB and will use existing implentations for the more complex algorithms.) By the end of the lab, you should have filled in entries of a table like that below, for each of images above. This means that at the end of the lab, your group should have three versions of the following table filled out, one for each of the three images. For subjective quality you should use the categories: a) same as the original, b) minor differences from the original but not objectionable, c) some distracting artifacts, and d) very poor quality. The objective comparison will be based on root mean squared error (RMSE), described further below. subjective compression Method quality RMSE factor Lossless 2x fewer gray levels 16x fewer gray levels Omit 50% of spatial frequencies (full DCT) Omit 50% of spatial frequencies (block DCT) JPEG Q=90 JPEG Q=50 For each compression scheme and each image, you will assess the differences subjectively (your qualitative impressions from looking at the image) and objectively (using the RMSE function). Do the objective and subjective measures always agree? For each scheme, determine which image looks the worst after compression? Which looks the best? Does your answer vary with the different schemes? If so, comment on what type of images are best suited to the different schemes. For the brainscan, what additional factors beyond human perception should be considered? 2 Lossless compression Lossless compression simply represents the information in an image using fewer bits by taking advantage of redundancy in the image and using variable-length coding to exploit nonuniform distribution of gray levels. After reconstruction, the resulting image is exactly the same as the original. Exercise 1: Compress the three files using the windows lossless compression mechanism (right click on file, then click on send to, then compress). Determine the file size before and after compress to compute the compression ratio for each file. Note that, since compression is lossless, there is no impact on image quality and RMSE=0 (see below about RMSE). 2

3 3 Lossy compression Recall that lossy compression throws out some of the data, so it is not possible to completely reconstruct the original (uncompressed) data from the compressed version. Instead, the goal is to throw out data in a way so that the user/listener/viewer won t notice the difference (or won t be bothered by it). Compression of images can take advantage of the fact that human vision is less sensitive to certain aspects of the image. For example, fine gray-level distinctions near a very strong edge may not be noticeable. Using fewer gray levels is a simple way to compress a grayscale image, which you explore in Exercise 2. We also are less sensitive to high spatial frequency variations in the image, which we will take advantage of in Exercises 3-5. When lossy compression is used, people frequently evaluate the result in terms of the root mean squared error (RMSE), which is the average (over all pixels) squared difference between the old and new values of the pixels: [ RMSE = 1 (original value of pixel i value of pixel i after compression) ], N 2 i where N is the total number of pixels in the image (number of rows times number of columns) and the sum is over all pixels. You will compute RMSE for the examples you experiment with in this lab because of its simplicity. However, it is important to realize that RMSE is not a perfect indicator of quality, because it does not take into account the relative importance of different errors for human perception. Other image distortion measures are covered in [1]. A function for computing RMSE is provided for you on the class web page: rmse.m called as error=rmse(image1,image2). 3.1 Quantization of Gray Levels One very simple method of lossy compression is to reduce the number of gray levels represented, i.e. map some of the gray levels to others. Exercise 2: Implement a simple compression scheme that reduces the number of gray levels by half, by mapping every other gray level to one of its neighbors, e.g. using: >> cap=255*ones(size(myimage)); >> compressed=min(cap,2*round(myimage/2)); where myimage is the matrix representing the original image and compressed now represents the compressed image. Note that you may need to convert uint8 images to double first, using the double() function. The round command rounds a number to its nearest integer. Therefore the above command maps even numbers to themselves, and odd numbers to the even numbers right above them, except 255 is mapped to itself. Since there are fewer possible gray levels we don t need as many bits to specify each pixel. 1 What is the compression factor of this approach? (Hint: It is not 2.) View the uncompressed and compressed images side by side and assess the differences, if any. Compute the RMSE of the compressed image. Repeat the above compression scheme with the other three images provided for this lab. Do you notice a difference with any of these? If so, describe it. Now repeat the above by reducing the number of gray levels by a factor of 16 rather than 2. What is the compression factor in this case? 3.2 Omitting Transform Coefficients We talked briefly in class about the discrete cosine transform (DCT), which is related to the Fourier transform. This transform converts a N N matrix of gray levels (representing pixel values) to an N N matrix of coefficients, where each spot in the matrix corresponds to a spatial frequency and the value of the coefficient in that spot tells you how much of that spatial frequency is present in the image. More simply, 1 The data still range from 0 to 255, since that is easier in working with MATLAB. However, the odd values are not present, so the levels could be mapped to an index with fewer bits, which is how you should think about this problem. 3

4 the DCT allows you to represent a signal in terms of the different cosines that it is composed of. Many similar transforms exist, but we ll work with the DCT because it is simple and very effective for images. If you are using a computer that has the MATLAB image processing toolbox, then a 2-dimensional DCT function and its inverse are included: dct2 and idct2. The dct2 function takes in an image, computes the 2-dimensional DCT, and returns a matrix of coefficients the same size as the image that describes the frequency content of the image. The idct2 function, referred to as an inverse DCT, takes in DCT coefficients and returns an image. To see the magnitude of the different cosines in an image, you need to use the abs() function, and since the coefficients die off very quickly, the pictures are more interesting if you look at log magnitude, as in: >> image=imread( filename.pgm ); >> imcoefs=dct2(image); >> imshow(log(abs(imcoefs)),[]); >> colormap(jet); Note that the colormap command is optional and can be used with different colormaps, but this is what was used to generate the pictures you saw in lecture. As we saw in class, in natural images the high frequencies tend to have very small DCT coefficients, which suggests that we may be able to simply omit these (zero out the coefficients) to reduce what we have to code and thereby save storage. Typically, we would keep coefficients in the upper left corner, which would be all elements (i, j) where i + j m. If the image is 256x256, then the DCT will be the same size, and using m = 256 we keep roughly have the coefficients. If m = 128, we keep roughly 25% of the coefficients. In the next exercise, you will see how the images are affected by throwing out coefficients. Exercise 3: Download the function truncate dct.m from the class webpage. This function that takes in a matrix of dct coefficients and sets everything beyond the upper left triangle (size specified by m, the second argument) to zero. a) For each of the three images, try throwing away half the frequencies and assess the quality of the reconstructed images, e.g.: >> icoefs=dct2(image); >> m=min(size(image)); >> icoefst=truncate dct(icoefs,m); >> newimage=idct2(icoefst); >> imshow(image,[]); >> figure; >> imshow(newimage,[]); The figure command will allow you to look at the images sided by side, so you can assess the difference qualitatively. Compute the RMSE between the original and new image. Compute the compression factor assuming you use 8 bits for each coefficient that is kept. (In fact, you would use more bits for lower coefficients and fewer for higher coefficients, as we ll learn later.) Repeat, throwing away 75% of the coefficients. Describe for your TA any interesting effects that you notice associated with truncating coefficients. b) Instead of doing a DCT on the whole image, we could do mini-dcts on subblocks of the image, throw away the high frequencies there, and then put them back together after reconstruction. (This would require somewhat less computation than the full DCT approach.) Download the mfile subblock truncate.m from the class web page, which is a function that will do block-level truncation for you, for the case where you throw away half the frequencies. Run this on the three images and compare the RMSE values and subjective differences relative to the full DCT approach. It may be helpful to look at the original, the full DCT and the block DCT results all side-by-side for your subjective comparison. 4

5 Table 1: Recommended JPEG quantization matrix, from [1]. 3.3 Non-Uniform Quantization of Transform Coefficients Rather than throwing out the coefficients of high frequencies, we could just use fewer bits to quantize them, which is basically the approach used in JPEG. A simplified outline of the basic JPEG compression process is (see [1]): 1. Divide the image into blocks of 8 8 pixels 2. Perform the discrete cosine transform individually on each block 3. Quantize the coefficients in each block according to a quantization matrix 4. Represent the resulting values efficiently using run-length and Huffman coding The decoder program knows how to interpret images stored in this format. Note that the decoded image has been changed from the original image only in step 3 (i.e., this is the lossy step.) The quantization matrix applies a different factor of quantization to each of the coefficients. It relies on the fact that coefficients at some positions in the matrix (generally, positions corresponding to higher frequency) are not as important as others to our perception of the image. In this part of the lab you will get to play around with the quantization matrix and see how it affects the result. Download compressjpeg.m from the course website. This is a MATLAB function that takes as its two inputs an image and a quantization matrix, and performs the above process (steps 1-3) on the image, returning the resulting image matrix (after decoding). To run the function, you will also need a quantization matrix. Table 1 (originally from [1]) shows a recommended JPEG quantization matrix. Bigger numbers correspond to fewer bits per spatial frequency. This matrix is available on the class webpage as a saved MATLAB variable quantmatrix.mat. Exercise 4: Download the quantization matrix and load it into MATLAB using the command load quantmatrix. Use this matrix with compressjpeg(image,quantmatrix) to compress one of the three grayscale images provided. Compare the following different coding schemes: a) the original matrix b) the same matrix scaled by a factor of 2 (half as many bits) c) the results from Exercise 3b, where you zeroed out high DCT frequencies in subblocks. Note that you should use subblock truncate() rather than compressjpeg() for this. You only need to do this for one image. Compare the results of (a), (b), and (c) above to your original image, both visually and in terms of RMSE. Comment on any differences you observe. (Note: these new results will not go into your table, but you do need to show your findings to your TA.) Exercise 5: Because we did not include step 4 (lossless coding) in the function in exercise 3, the results don t allow you to measure the compression rate of JPEG (although they do allow you to measure RMSE). 5

6 In order to finish filling out the table, we ll use MATLAB s JPEG compression capability, as follows: >> imageo=imread( image.pgm ); >> imwrite(image0, imageq50.jpg, Quality,50); >> imagej=imread( image.jpg ); >> err=rmse(image0,imagej); Repeat using a quality factor of 90. You can compute the compression ratios by looking at the file sizes on the computer. You will of course need to do this for each of the three images you selected, in order to fill in your tables. References [1] R. C. Gonzalez and R. E. Woods, Digital Image Processing, 2nd ed. Upper Saddle River, NJ: Prentice- Hall,