IMAGE AND VIDEO COMPRESSION STANDARDS Algorithms and Architectures. Second Edition

Transcription

1 IMAGE AND VIDEO COMPRESSION STANDARDS Algorithms and Architectures Second Edition

2 THE KLUWER INTERNATIONAL SERIES IN ENGINEERING AND COMPUTER SCIENCE

3 lmage AND VIDEO COMPRESSION STANDARDS Algorithms and Architectures Second Edition by Vasudev Bhaskaran Konstantinos Konstantinides Hewlett-Packard Laboratories SPRINGER SCIENCE+BUSINESS MEDIA, LLC

4 Library of Congress Cataloging-in-Publication Data Bhaskaran, Vasudev, Image and video compression standards : algorithms and architectures / by Vasudev Bhaskaran, Konstantinos Konstantinides nd ed. p. cm. -- (The Kluwer international series in engineering and computer science; SECS 408) Includes bibliographical references and index. ISBN ISBN (ebook) DOI / Image compression. 2. Video compression. 1. Konstantinides, Konstantinos, II. Title. III. Series. TA1632.B '7 --dc CIP Copyright 1997 by Springer Science+Business Media New York. Fourth printing Originally published by Kluwer Academic Publishers in, New York 1997 Softcover reprint of the hardcover 2nd edition 1997 AII rights reserved. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, mechanical, photo-copying, recording, or otherwise, without the prior written permission of the publisher, Springer Science+Business Media, LLC, Printed on acid-free pa per.

5 to Achan. Chechi, Balu, Kishore, Shanti, Paula, Anastasia, Anneta, and Niki

6 CONTENTS PREFACE XIll I COMPRESSION FUNDAMENTALS 1.1 Introduction J.2 Background Compression Taxonomy Issues in Compression Method Selection To Probe Further 14 2 METHODS AND STANDARDS FOR LOSSLESS COMPRESSION Introduction Preliminaries Huffman Encoding Huffman Decoding Huffman Codes with Constrained Length Constrained-Length Huffman Codes: Ad Hoc Design Constrained-Length Huffman Codes: The Voorhis Method Golomb and Rice Coding Arithmetic Coding Implementation Issues Standards for Lossless Compression New Work in IPEG: IPEG-LS To Probe Further 59

7 viii IMAGE AND VIDEO COMPRESSION STANDARDS 3 FUNDAMENTALS OF LOSSY IMAGE COMPRESSION Introduction Preliminaries Basic Coding Schemes for Lossy Compression OCT -based Coding Fast Algorithms for the OCT Rate-Distortion Perfonnance of the OCT To Probe Further 96 4 FUNDAMENTALS OF LOSSY VIDEO COMPRESSION Introduction Video Coding Basics Rate-Distortion Functions in Video Coding Motion-Compensated Prediction Algorithms for Motion Estimation Motion Estimation Using Low Complexity Matching Criteria Sub-pixel-Accurate Motion Estimation Multipicture Motion Estimation To Probe Further THE JPEG STANDARD Introduction DCT-based Coding Processing of Color Images Design of Quantization Tables Entropy Coding JPEG Modes of Operation Implementation Issues JPEG Extensions and Applications To Probe Further THE MPEG VIDEO STANDARDS Introduction 179

8 Contents ix 6.2 The MPEG-I Video Standard MPEG l Implementation Issues The MPEG-2 Video Standard The MPEG-4 Coding Standard Future Directions: MPEG To Probe Further VIDEO TELECONFERENCING STANDARDS Introduction H.320 Video Teleconferencing The H.261 Video Coding Standard Implementation Issues H.324 Video Teleconferencing The H.263 Video Coding Standard Future Directions To Probe Further PROCESSING REQUIREMENTS Introduction Measuring Complexity Distributing the Load To Probe Further RISC AND DSP CORES Introduction The RISC Core The DSP Core RISC versus nsp To Probe Further ARCmTECTURES FOR THE DCT Introduction Vector Processing Distributed Arithmetic for the ocr 274

9 x IMAGE AND VIDEO COMPRESSION STANDARDS 10.4 Commercially Available OCT Processors To Probe Funher HARDWARE FOR MOTION ESTIMATION Introduction Block Matching Using Linear Arrays Hardware for One-bit Motion Estimation Sub-pel Motion Estimation Implementation Examples To Probe Further HARDWARE FOR ENTROPY CODING Introduction Encoder Implementation Decoder Implementation Variable-Length Coding in JPEG To Probe Further IMPLEMENTATION OF JPEG PROCESSORS Introduction Data VO and Memory Interface Color Conversion Commercially Available JPEG Processors To Probe Further INTEGRATED CIRCUITS FOR VIDEO CODERS Introduction Video Signal Processors Multimedia Coprocessors Dedicated Coders To Probe Further MULTIMEDIA ENHANCEMENTS FOR GENERAL PURPOSE PROCESSORS Introduction 347

10 Contents XI 15.2 Generic Operations in Video Processing Multimedia Enhancements in the HP-PA Multimedia Enhancements in the UltraSparc Multimedia Enhancements in the Intel Architecture To Probe Further STANDARDS FOR AUDIO COMPRESSION Introduction The G.12X Standards Fundamentals of Perceptual Coders MPEG Audio Encoding MPEG Audio Decoding The AC-3 Coding Standard Future Directions in MPEG Hardware Implementation of Audio Codecs To Probe Further NONSTANDARD COMPRESSION TECHNIQUES Introduction Vector Quantization Subband Coding Video Coding Schemes for the Desktop To Probe Further 405 A COLOR TRANSFORMATIONS 409 B ABBREVIATIONS AND SYMBOLS 413 C INTERNET DIRECTORY 419 REFERENCES 423 INDEX 447

11 PREFACE This book presents an introduction to the algorithms and architectures that fonn the underpinnings of the image and video compression standards. including JPEG (compression of still-images), H.261 and H.263 (video teleconferencing), and MPEG-I and MPEG-2 (video storage and broadcasting). The next generation of audiovisual coding standards, such as MPEG-4 and MPEG-7. are also briefly described. In addition, the book covers the MPEG and Dolby AC-3 audio coding standards and emerging techniques for image and video compression, such as those based on wavelets and vector quantization. The emphasis of the book is on the foundations of these standards, namely, techniques such as predictive coding, transform-based coding. motion estimation, motion compensation, and entropy coding, as well as how they are applied in the standards. We avoid the implementation details of each standard. However, we do provide all the material necessary to understand the workings of each of the compression standards, including infonnation that can be used by the reader to evaluate the efficiency of various software and hardware implementations confonning to these standards. We place particular emphasis on those algorithms and architectures that have been found to be useful in practical software or hardware implementations. No prior knowledge of image and video compression theory and architectures is assumed; however, some background in high-level hardware design is expected for a better understanding of the chapters covering the hardware implementation of the standards. Chapters 1 to 4 review the fundamentals of image and video compression. In Chapter I we cover the key principles of image compression and explain the basic tenninology. We present a compression taxonomy and discuss design issues and tradeoffs in selecting a compression algorithm. In Chapter 2 we review methods and standards for loss less compression, including differential, Huffman, and arithmetic coding. We present several practical algorithms

12 XIV IMAGE AND VIDEO COMPRESSION STANDARDS for the design of Huffman codes, and we describe the current lossless JPEG algorithm, work in progress on a new lossless and near-iossless JPEG standard (JPEG-LS), and the standards for lossless facsimile transmission (Group 3, Group 4, and IBIG). In Chapter 3 we examine techniques for lossy compression. We review the source-coding theorem and compute the rate-distortion function for several image models relevant to image compression. We place particular emphasis on DCT-based coding. and describe several schemes for the efficient implementation of the OCT. In Chapter 4 we present the fundamentals of video coding, including motion estimation and motion compensation. In Chapter 5 we describe.lossy JPEG and its application to the compression of still-images. We cover recent extensions to JPEG and the lpeg-based standard for color facsimile. In Chapter 6 we describe the coding algorithms and syntax for the MPEG-l and MPEG-2 video compression standards, the basic video coding aspects of MPEG-4. and objectives and target application areas for MPEG-7. In Chapter 7 we describe the H.320 and H.324 video teleconferencing standards, with emphasis on their video coding parts, namely, H.261 and H.263. Simi.larities and differences among the video compression standards are also presented and reviewed. Next, we move to the hardware implementation of the standards. In Chapter 8 we define a measure of computational complexity. and we use it to measure the complexity of various algorithms used in the standards. Bearing in mind that most hardware implementations of the standards are based on either RISC or DSP cores, in Chapter 9 we review the principles of RISC and DSP designs and describe their main differences and similarities. Even though we present the latest designs in image and video processors, we place more emphasis on the designs of the core algorithms; that is, the discrete cosine transfonn (Chapter 10), motion estimation (Chapter 11), and entropy coding (Chapter 12). In Chapter 13 we describe architectures for the JPEG standard, and in Chapter 14 we discuss programmable and dedicated video processors for MPEG and videoconferencing standards. Key features of recent VLIW media processors are also described. In Chapter 15 we review recent developments in the design of multimediaenhanced general-purpose RISC processors, such as the PA-8000 from Hewlett-Packard, the UltraSPARC from Sun Microsystems. and Intel's MMX-

13 Preface xv based Pentium processors. In Chapter 16 we present the key standards on audio coding, and we describe in more detail the MPEG and Dolby AC-3 coding schemes. In Chapter 17 we provide an overview of techniques based on vector quantization and wavelets. Such techniques are used in many proprietary image and video compression schemes and may play a significant role in future standards. Given the wide use of color imagery as input to image and video coders. Appendix A provides a quick reference to basic color transformations. Appendix B provides a summary of common abbreviations used in the text and can be used as a quick reference for the various standard image fonnats (SIF, CIF, and CCIR 601). Information about sites with public software implementations of the standards is included in Appendix C. The material in this book is partitioned into the following categories: fundamenta1s, video standards. architectures. audio standards. and emerging techniques. However. most of the chapters are self contained and allow the reader to easily switch from one category to another. This book is aimed at the professional who wants to have a basic understanding of the latest developments and applications of image compression standards. It is based on tutorials given by the authors at several forums. including the IEEE International Conference on Acoustics Speech and Signal Processing. the ACM Multimedia Conference, and Hewlett-Packard. It provides a reference for any engineer planning to work in this field, either in basic implementation or in research and development. It is intended for self-study; however, it can also be used as a companion textbook in any course on data compression, video coding. computer architecture. signal or image processing. or the design of signal and video processors. New topics in the second edition: Since the publication of the first edition. we have seen a number of new developments in standards-related activities. The original JPEG and MPEG standards have been refined, and several new compression standards have emerged. In 10ssless compression, we introduce IPEG-LS. a new standard for lossless and near-iossless compression of still-images. In video coding, we describe two new MPEG-2 profiles: 4:2:2 and multiview. Activities in two new MPEG standards. MPEG-4 and MPEG-7, are also discussed here. In videoconferencing. we describe the new H.263 standard and compare it with H.261. New developments in fast motion-

14 XVI IMAGE AND VIDEO COMPRESSION STANDARDS estimation algorithms. a major component of all MPEG and videoconferencing standards, are also presented. In audio coding, new standards. such as MPEG-2 AAe and G.723.1, are also discussed. Since the flrst edition, there have been significant advances in the hardware support for multimedia and compression standards. We describe enhancements to several general purpose processors, including the Intel MMX and the newly emerging class of media processors. The information on commercially available audio and video coders has also been updated. Acknowledgments: This book would Dot be possible without the support from the management at Hewlett-Packard Laboratories, especially Fred L. Kitson, Daniel T. Lee, and Ho John Lee. We also thank our col1eagues at Hewlett Packard for many helpful discussions. Balas K. Natarajan has been particularly helpful from the beginning of this project and we thank him for many valuable discussions. comments. and for reviewing parts of this manuscript. The review of our sections on Golomb-Rice coding and JPEG-LS by Guillermo Sapiro and Gadiel Seroussi is gratefully acknowledged. We thank Bob Rau for his comments on our tutorials and Ruby Lee for providing us with a different perspective on the hardware implementation of the video standards. We thank Nariman Farvardin and David Daut for their comments on the computation of the rate-distortion function. We thank Akio Yamamoto for his help in the example on wavelet coding, Jose Fridman for many valuable comments and suggestions, Masoud Khansari for his comments on the video teleconferencing standards. and Elias Manolakos. Feedback from the readers of the first edition of the book has been extremely valuable in helping us to improve the clarity of the presentation and to select new material. We are grateful to all of them, and especially to Thanos Skordas and Aggelos Katsaggelos, for their suggestions and conections. We also thank Robert Holland. Je. and his staff from Kluwer Academic Press for their support, and Suzanne M. Rumsey for helping with many typesetting questions. Finally. we dedicate this book to our teachers and to our families, and especially to Achan. Chechi, Balu, Kishore. Shanti, Paula, Anastasia, Anneta, and Niki.