INTERNATIONAL TELECOMMUNICATION UNION TERMINAL EQUIPMENT AND PROTOCOLS FOR TELEMATIC SERVICES

Transcription

1 INTERNATIONAL TELECOMMUNICATION UNION CCITT T.81 THE INTERNATIONAL (09/92) TELEGRAPH AND TELEPHONE CONSULTATIVE COMMITTEE TERMINAL EQUIPMENT AND PROTOCOLS FOR TELEMATIC SERVICES INFORMATION TECHNOLOGY DIGITAL COMPRESSION AND CODING OF CONTINUOUS-TONE STILL IMAGES REQUIREMENTS AND GUIDELINES Recommendation T.81

2 Foreword ITU (International Telecommunication Union) is the United Nations Specialized Agency in the field of telecommunications. The CCITT (the International Telegraph and Telephone Consultative Committee) is a permanent organ of the ITU. Some 166 member countries, 68 telecom operating entities, 163 scientific and industrial organizations and 39 international organizations participate in CCITT which is the body which sets world telecommunications standards (Recommendations). The approval of Recommendations by the members of CCITT is covered by the procedure laid down in CCITT Resolution. 2 (Melbourne, 1988). In addition, the Plenary Assembly of CCITT, which meets every four years, approves Recommendations submitted to it and establishes the study programme for the following period. In some areas of information technology, which fall within CCITT s purview, the necessary standards are prepared on a collaborative basis with ISO and IEC. The text of CCITT Recommendation T.81 was approved on 18th September The identical text is also published as ISO/IEC International Standard CCITT NOTE In this Recommendation, the expression Administration is used for conciseness to indicate both a telecommunication administration and a recognized private operating agency. ITU 1993 All rights reserved. part of this publication may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and microfilm, without permission in writing from the ITU.

3 Contents Page Introduction... iii 1 Scope rmative references Definitions, abbreviations and symbols General Interchange format requirements Encoder requirements Decoder requirements Annex A Mathematical definitions Annex B Compressed data formats Annex C Huffman table specification Annex D Arithmetic coding Annex E Encoder and decoder control procedures Annex F Sequential DCT-based mode of operation Annex G Progressive DCT-based mode of operation Annex H Lossless mode of operation Annex J Hierarchical mode of operation Annex K Examples and guidelines Annex L Patents Annex M Bibliography CCITT Rec. T.81 (1992 E) i

4 Introduction This CCITT Recommendation ISO/IEC International Standard was prepared by CCITT Study Group VIII and the Joint Photographic Experts Group (JPEG) of ISO/IEC JTC 1/SC 29/WG 10. This Experts Group was formed in 1986 to establish a standard for the sequential progressive encoding of continuous tone grayscale and colour images. Digital Compression and Coding of Continuous-tone Still images, is published in two parts: Requirements and guidelines; Compliance testing. This part, Part 1, sets out requirements and implementation guidelines for continuous-tone still image encoding and decoding processes, and for the coded representation of compressed image data for interchange between applications. These processes and representations are intended to be generic, that is, to be applicable to a broad range of applications for colour and grayscale still images within communications and computer systems. Part 2, sets out tests for determining whether implementations comply with the requirments for the various encoding and decoding processes specified in Part 1. The user s attention is called to the possibility that for some of the coding processes specified herein compliance with this Recommendation International Standard may require use of an invention covered by patent rights. See Annex L for further information. The requirements which these processes must satisfy to be useful for specific image communications applications such as facsimile, Videotex and audiographic conferencing are defined in CCITT Recommendation T.80. The intent is that the generic processes of Recommendation T.80 will be incorporated into the various CCITT Recommendations for terminal equipment for these applications. In addition to the applications addressed by the CCITT and ISO/IEC, the JPEG committee has developped a compression standard to meet the needs of other applications as well, including desktop publishing, graphic arts, medical imaging and scientific imaging. Annexes A, B, C, D, E, F, G, H and J are normative, and thus form an integral part of this Specification. Annexes K, L and M are informative and thus do not form an integral part of this Specification. This Specification aims to follow the guidelines of CCITT and ISO/IEC JTC 1 on Rules for presentation of CCITT ISO/IEC common text.

5 INTERNATIONAL STANDARD ISO/IEC : 1993(E) CCITT Rec. T.81 (1992 E) CCITT RECOMMENDATION INFORMATION TECHNOLOGY DIGITAL COMPRESSION AND CODING OF CONTINUOUS-TONE STILL IMAGES REQUIREMENTS AND GUIDELINES 1 Scope This CCITT Recommendation International Standard is applicable to continuous-tone grayscale or colour digital still image data. It is applicable to a wide range of applications which require use of compressed images. It is not applicable to bi-level image data. This Specification specifies processes for converting source image data to compressed image data; specifies processes for converting compressed image data to reconstructed image data; gives guidance on how to implement these processes in practice; specifies coded representations for compressed image data. NOTE This Specification does not specify a complete coded image representation. Such representations may include certain parameters, such as aspect ratio, component sample registration, and colour space designation, which are applicationdependent. 2 rmative references The following CCITT Recommendations and International Standards contain provisions which, through reference in this text, constitute provisions of this CCITT Recommendation International Standard. At the time of publication, the editions indicated were valid. All Recommendations and Standards are subject to revision, and parties to agreements based on this CCITT Recommendation International Standard are encouraged to investigate the possibility of applying the most recent edition of the Recommendations and Standards listed below. Members of IEC and ISO maintain registers of currently valid International Standards. The CCITT Secretariat maintains a list of currently valid CCITT Recommendations. CCITT Recommendation T.80 (1992), Common components for image compression and communication Basic principles. 3 Definitions, abbreviations and symbols 3.1 Definitions and abbreviations For the purposes of this Specification, the following definitions apply abbreviated format: A representation of compressed image data which is missing some or all of the table specifications required for decoding, or a representation of table-specification data without frame headers, scan headers, and entropy-coded segments AC coefficient: Any DCT coefficient for which the frequency is not zero in at least one dimension (adaptive) (binary) arithmetic decoding: An entropy decoding procedure which recovers the sequence of symbols from the sequence of bits produced by the arithmetic encoder (adaptive) (binary) arithmetic encoding: An entropy encoding procedure which codes by means of a recursive subdivision of the probability of the sequence of symbols coded up to that point application environment: The standards for data representation, communication, or storage which have been established for a particular application. CCITT Rec. T.81 (1992 E) 1

6 3.1.6 arithmetic decoder: An embodiment of arithmetic decoding procedure arithmetic encoder: An embodiment of arithmetic encoding procedure baseline (sequential): A particular sequential DCT-based encoding and decoding process specified in this Specification, and which is required for all DCT-based decoding processes binary decision: Choice between two alternatives bit stream: Partially encoded or decoded sequence of bits comprising an entropy-coded segment block: An 8 8 array of samples or an 8 8 array of DCT coefficient values of one component block-row: A sequence of eight contiguous component lines which are partitioned into 8 8 blocks byte: A group of 8 bits byte stuffing: A procedure in which either the Huffman coder or the arithmetic coder inserts a zero byte into the entropy-coded segment following the generation of an encoded hexadecimal X FF byte carry bit: A bit in the arithmetic encoder code register which is set if a carry-over in the code register overflows the eight bits reserved for the output byte ceiling function: The mathematical procedure in which the greatest integer value of a real number is obtained by selecting the smallest integer value which is greater than or equal to the real number class (of coding process): Lossy or lossless coding processes code register: The arithmetic encoder register containing the least significant bits of the partially completed entropy-coded segment. Alternatively, the arithmetic decoder register containing the most significant bits of a partially decoded entropy-coded segment coder: An embodiment of a coding process coding: Encoding or decoding coding model: A procedure used to convert input data into symbols to be coded (coding) process: A general term for referring to an encoding process, a decoding process, or both colour image: A continuous-tone image that has more than one component columns: Samples per line in a component component: One of the two-dimensional arrays which comprise an image compressed data: Either compressed image data or table specification data or both compressed image data: A coded representation of an image, as specified in this Specification compression: Reduction in the number of bits used to represent source image data conditional exchange: The interchange of MPS and LPS probability intervals whenever the size of the LPS interval is greater than the size of the MPS interval (in arithmetic coding) (conditional) probability estimate: The probability value assigned to the LPS by the probability estimation state machine (in arithmetic coding) conditioning table: The set of parameters which select one of the defined relationships between prior coding decisions and the conditional probability estimates used in arithmetic coding context: The set of previously coded binary decisions which is used to create the index to the probability estimation state machine (in arithmetic coding) continuous-tone image: An image whose components have more than one bit per sample data unit: An 8 8 block of samples of one component in DCT-based processes; a sample in lossless processes. 2 CCITT Rec. T.81 (1992 E)

7 DC coefficient: The DCT coefficient for which the frequency is zero in both dimensions. ISO/IEC : 1993(E) DC prediction: The procedure used by DCT-based encoders whereby the quantized DC coefficient from the previously encoded 8 8 block of the same component is subtracted from the current quantized DC coefficient (DCT) coefficient: The amplitude of a specific cosine basis function may refer to an original DCT coefficient, to a quantized DCT coefficient, or to a dequantized DCT coefficient decoder: An embodiment of a decoding process decoding process: A process which takes as its input compressed image data and outputs a continuous-tone image default conditioning: The values defined for the arithmetic coding conditioning tables at the beginning of coding of an image dequantization: The inverse procedure to quantization by which the decoder recovers a representation of the DCT coefficients differential component: The difference between an input component derived from the source image and the corresponding reference component derived from the preceding frame for that component (in hierarchical mode coding) differential frame: A frame in a hierarchical process in which differential components are either encoded or decoded (digital) reconstructed image (data): A continuous-tone image which is the output of any decoder defined in this Specification (digital) source image (data): A continuous-tone image used as input to any encoder defined in this Specification (digital) (still) image: A set of two-dimensional arrays of integer data discrete cosine transform; DCT: Either the forward discrete cosine transform or the inverse discrete cosine transform downsampling (filter): A procedure by which the spatial resolution of an image is reduced (in hierarchical mode coding) encoder: An embodiment of an encoding process encoding process: A process which takes as its input a continuous-tone image and outputs compressed image data entropy-coded (data) segment: An independently decodable sequence of entropy encoded bytes of compressed image data (entropy-coded segment) pointer: The variable which points to the most recently placed (or fetched) byte in the entropy encoded segment entropy decoder: An embodiment of an entropy decoding procedure entropy decoding: A lossless procedure which recovers the sequence of symbols from the sequence of bits produced by the entropy encoder entropy encoder: An embodiment of an entropy encoding procedure entropy encoding: A lossless procedure which converts a sequence of input symbols into a sequence of bits such that the average number of bits per symbol approaches the entropy of the input symbols extended (DCT-based) process: A descriptive term for DCT-based encoding and decoding processes in which additional capabilities are added to the baseline sequential process forward discrete cosine transform; FDCT: A mathematical transformation using cosine basis functions which converts a block of samples into a corresponding block of original DCT coefficients. CCITT Rec. T.81 (1992 E) 3

8 frame: A group of one or more scans (all using the same DCT-based or lossless process) through the data of one or more of the components in an image frame header: A marker segment that contains a start-of-frame marker and associated frame parameters that are coded at the beginning of a frame frequency: A two-dimensional index into the two-dimensional array of DCT coefficients (frequency) band: A contiguous group of coefficients from the zig-zag sequence (in progressive mode coding) full progression: A process which uses both spectral selection and successive approximation (in progressive mode coding) grayscale image: A continuous-tone image that has only one component hierarchical: A mode of operation for coding an image in which the first frame for a given component is followed by frames which code the differences between the source data and the reconstructed data from the previous frame for that component. Resolution changes are allowed between frames hierarchical decoder: A sequence of decoder processes in which the first frame for each component is followed by frames which decode an array of differences for each component and adds it to the reconstructed data from the preceding frame for that component hierarchical encoder: The mode of operation in which the first frame for each component is followed by frames which encode the array of differences between the source data and the reconstructed data from the preceding frame for that component horizontal sampling factor: The relative number of horizontal data units of a particular component with respect to the number of horizontal data units in the other components Huffman decoder: An embodiment of a Huffman decoding procedure Huffman decoding: An entropy decoding procedure which recovers the symbol from each variable length code produced by the Huffman encoder Huffman encoder: An embodiment of a Huffman encoding procedure Huffman encoding: An entropy encoding procedure which assigns a variable length code to each input symbol Huffman table: The set of variable length codes required in a Huffman encoder and Huffman decoder image data: Either source image data or reconstructed image data interchange format: The representation of compressed image data for exchange between application environments interleaved: The descriptive term applied to the repetitive multiplexing of small groups of data units from each component in a scan in a specific order inverse discrete cosine transform; IDCT: A mathematical transformation using cosine basis functions which converts a block of dequantized DCT coefficients into a corresponding block of samples Joint Photographic Experts Group; JPEG: The informal name of the committee which created this Specification. The joint comes from the CCITT and ISO/IEC collaboration latent output: Output of the arithmetic encoder which is held, pending resolution of carry-over (in arithmetic coding) less probable symbol; LPS: For a binary decision, the decision value which has the smaller probability level shift: A procedure used by DCT-based encoders and decoders whereby each input sample is either converted from an unsigned representation to a two s complement representation or from a two s complement representation to an unsigned representation. 4 CCITT Rec. T.81 (1992 E)

9 lossless: A descriptive term for encoding and decoding processes and procedures in which the output of the decoding procedure(s) is identical to the input to the encoding procedure(s) lossless coding: The mode of operation which refers to any one of the coding processes defined in this Specification in which all of the procedures are lossless (see Annex H) lossy: A descriptive term for encoding and decoding processes which are not lossless marker: A two-byte code in which the first byte is hexadecimal FF (X FF ) and the second byte is a value between 1 and hexadecimal FE (X FE ) marker segment: A marker and associated set of parameters MCU-row: The smallest sequence of MCU which contains at least one line of samples or one block-row from every component in the scan minimum coded unit; MCU: The smallest group of data units that is coded modes (of operation): The four main categories of image coding processes defined in this Specification more probable symbol; MPS: For a binary decision, the decision value which has the larger probability non-differential frame: The first frame for any components in a hierarchical encoder or decoder. The components are encoded or decoded without subtraction from reference components. The term refers also to any frame in modes other than the hierarchical mode non-interleaved: The descriptive term applied to the data unit processing sequence when the scan has only one component parameters: Fixed length integers 4, 8 or 16 bits in length, used in the compressed data formats point transform: Scaling of a sample or DCT coefficient precision: Number of bits allocated to a particular sample or DCT coefficient predictor: A linear combination of previously reconstructed values (in lossless mode coding) probability estimation state machine: An interlinked table of probability values and indices which is used to estimate the probability of the LPS (in arithmetic coding) probability interval: The probability of a particular sequence of binary decisions within the ordered set of all possible sequences (in arithmetic coding) (probability) sub-interval: A portion of a probability interval allocated to either of the two possible binary decision values (in arithmetic coding) procedure: A set of steps which accomplishes one of the tasks which comprise an encoding or decoding process process: See coding process progressive (coding): One of the DCT-based processes defined in this Specification in which each scan typically improves the quality of the reconstructed image progressive DCT-based: The mode of operation which refers to any one of the processes defined in Annex G quantization table: The set of 64 quantization values used to quantize the DCT coefficients quantization value: An integer value used in the quantization procedure quantize: The act of performing the quantization procedure for a DCT coefficient reference (reconstructed) component: Reconstructed component data which is used in a subsequent frame of a hierarchical encoder or decoder process (in hierarchical mode coding). CCITT Rec. T.81 (1992 E) 5

10 renormalization: The doubling of the probability interval and the code register value until the probability interval exceeds a fixed minimum value (in arithmetic coding) restart interval: The integer number of MCUs processed as an independent sequence within a scan restart marker: The marker that separates two restart intervals in a scan run (length): Number of consecutive symbols of the same value sample: One element in the two-dimensional array which comprises a component sample-interleaved: The descriptive term applied to the repetitive multiplexing of small groups of samples from each component in a scan in a specific order scan: A single pass through the data for one or more of the components in an image scan header: A marker segment that contains a start-of-scan marker and associated scan parameters that are coded at the beginning of a scan sequential (coding): One of the lossless or DCT-based coding processes defined in this Specification in which each component of the image is encoded within a single scan sequential DCT-based: The mode of operation which refers to any one of the processes defined in Annex F spectral selection: A progressive coding process in which the zig-zag sequence is divided into bands of one or more contiguous coefficients, and each band is coded in one scan stack counter: The count of X FF bytes which are held, pending resolution of carry-over in the arithmetic encoder statistical conditioning: The selection, based on prior coding decisions, of one estimate out of a set of conditional probability estimates (in arithmetic coding) statistical model: The assignment of a particular conditional probability estimate to each of the binary arithmetic coding decisions statistics area: The array of statistics bins required for a coding process which uses arithmetic coding statistics bin: The storage location where an index is stored which identifies the value of the conditional probability estimate used for a particular arithmetic coding binary decision successive approximation: A progressive coding process in which the coefficients are coded with reduced precision in the first scan, and precision is increased by one bit with each succeeding scan table specification data: The coded representation from which the tables used in the encoder and decoder are generated and their destinations specified transcoder: A procedure for converting compressed image data of one encoder process to compressed image data of another encoder process (uniform) quantization: The procedure by which DCT coefficients are linearly scaled in order to achieve compression upsampling (filter): A procedure by which the spatial resolution of an image is increased (in hierarchical mode coding) vertical sampling factor: The relative number of vertical data units of a particular component with respect to the number of vertical data units in the other components in the frame zero byte: The X 00 byte zig-zag sequence: A specific sequential ordering of the DCT coefficients from (approximately) lowest spatial frequency to highest sample predictor: A linear combination of the three nearest neighbor reconstructed samples to the left and above (in lossless mode coding). 6 CCITT Rec. T.81 (1992 E)

11 3.2 Symbols The symbols used in this Specification are listed below. A AC AC ji Ah Al Ap i APP n B B2 BE BITS BP BPST BR Bx C C i C u C v CE C-low Cm i CNT CODE CODESIZE(V) COM Cs Cs i CT Cx CX d ji d ji k D probability interval AC DCT coefficient AC coefficient predicted from DC values successive approximation bit position, high successive approximation bit position, low ith 8-bit parameter in APP n segment marker reserved for application segments current byte in compressed data next byte in compressed data when B = X FF counter for buffered correction bits for Huffman coding in the successive approximation process 16-byte list containing number of Huffman codes of each length pointer to compressed data pointer to byte before start of entropy-coded segment counter for buffered correction bits for Huffman coding in the successive approximation process byte modified by a carry-over value of bit stream in code register component identifier for frame horizontal frequency dependent scaling factor in DCT vertical frequency dependent scaling factor in DCT conditional exchange low order 16 bits of the arithmetic decoder code register ith 8-bit parameter in COM segment bit counter in NEXTBYTE procedure Huffman code value code size for symbol V comment marker conditioning table value component identifier for scan renormalization shift counter high order 16 bits of arithmetic decoder code register conditional exchange data unit from horizontal position i, vertical position j d ji for component k decision decoded CCITT Rec. T.81 (1992 E) 7

12 Da DAC Db DC DC i DC k DHP DHT DIFF DNL DQT DRI E EC ECS ECS i Eh EHUFCO EHUFSI EOB EOBn EOBx EOB0, EOB1,..., EOB14 EOI Ev EXP FREQ(V) H i H max HUFFCODE HUFFSIZE HUFFVAL i I Index(S) j J in DC coding, the DC difference coded for the previous block from the same component; in lossless coding, the difference coded for the sample immediately to the left define-arithmetic-coding-conditioning marker the difference coded for the sample immediately above DC DCT coefficient DC coefficient for ith block in component kth DC value used in prediction of AC coefficients define hierarchical progression marker define-huffman-tables marker difference between quantized DC and prediction define-number-of-lines marker define-quantization-tables marker define restart interval marker exponent in magnitude category upper bound event counter entropy-coded segment ith entropy-coded segment horizontal expansion parameter in EXP segment Huffman code table for encoder encoder table of Huffman code sizes end-of-block for sequential; end-of-band for progressive run length category for EOB runs position of EOB in previous successive approximation scan run length categories for EOB runs end-of-image marker vertical expansion parameter in EXP segment expand reference components marker frequency of occurrence of symbol V horizontal sampling factor for ith component largest horizontal sampling factor list of Huffman codes corresponding to lengths in HUFFSIZE list of code lengths list of values assigned to each Huffman code subscript index integer variable index to probability estimation state machine table for context index S subscript index integer variable 8 CCITT Rec. T.81 (1992 E)

13 JPG JPG n k K Kmin Kx L L i L i (t) La LASTK Lc Ld Le Lf Lh Lp LPS Lq Lr Ls LSB m m t M Mn MAXCODE MCU marker reserved for JPEG extensions marker reserved for JPEG extensions subscript index integer variable index of 1st AC coefficient in band (1 for sequential DCT) conditioning parameter for AC arithmetic coding model DC and lossless coding conditioning lower bound parameter element in BITS list in DHT segment element in BITS list in the DHT segment for Huffman table t length of parameters in APP n segment largest value of K length of parameters in COM segment length of parameters in DNL segment length of parameters in EXP segment length of frame header parameters length of parameters in DHT segment length of parameters in DAC segment less probable symbol (in arithmetic coding) length of parameters in DQT segment length of parameters in DRI segment length of scan header parameters least significant bit modulo 8 counter for RST m marker number of V i,j parameters for Huffman table t bit mask used in coding magnitude of V nth statistics bin for coding magnitude bit pattern category table with maximum value of Huffman code for each code length minimum coded unit MCU i MCUR MINCODE MPS MPS(S) MSB ith MCU number of MCU required to make up one MCU-row table with minimum value of Huffman code for each code length more probable symbol (in arithmetic coding) more probable symbol for context-index S most significant bit M2, M3, M4,..., M15 designation of context-indices for coding of magnitude bits in the arithmetic coding models n N N/A integer variable data unit counter for MCU coding not applicable CCITT Rec. T.81 (1992 E) 9

14 Nb Next_Index_LPS Next_Index_MPS Nf NL Ns OTHERS(V) P Pq Pq(t) PRED Pt Px Q ji Q vu Q 00 QAC ji QDC k Qe Qe(S) Qk r vu R R vu Ra Rb Rc Rd RES Ri RRRR RS RST m s yx S S vu number of data units in MCU new value of Index(S) after a LPS renormalization new value of Index(S) after a MPS renormalization number of components in frame number of lines defined in DNL segment number of components in scan index to next symbol in chain sample precision quantizer precision parameter in DQT segment quantizer precision parameter in DQT segment for quantization table t quantized DC coefficient from the most recently coded block of the component point transform parameter calculated value of sample quantizer value for coefficient AC ji quantization value for DCT coefficient S vu quantizer value for DC coefficient quantized AC coefficient predicted from DC values kth quantized DC value used in prediction of AC coefficients LPS probability estimate LPS probability estimate for context index S kth element of 64 quantization elements in DQT segment reconstructed image sample length of run of zero amplitude AC coefficients dequantized DCT coefficient reconstructed sample value reconstructed sample value reconstructed sample value rounding in prediction calculation reserved markers restart interval in DRI segment 4-bit value of run length of zero AC coefficients composite value used in Huffman coding of AC coefficients restart marker number m reconstructed value from IDCT context index DCT coefficient at horizontal frequency u, vertical frequency v 10 CCITT Rec. T.81 (1992 E)

15 SC Se SE SI SIGN SIZE SLL SLL α β SN SOF 0 SOF 1 SOF 2 SOF 3 SOF 5 SOF 6 SOF 7 SOF 9 SOF 10 SOF 11 SOF 13 SOF 14 SOF 15 SOI SOS SP Sq vu SRL SRL α β Ss SS SSSS ST Switch_MPS Sz S0 t T context-index for coding of correction bit in successive approximation coding end of spectral selection band in zig-zag sequence context-index for coding of end-of-block or end-of-band Huffman code size 1 if decoded sense of sign is negative and 0 if decoded sense of sign is positive length of a Huffman code shift left logical operation logical shift left of α by β bits context-index for coding of first magnitude category when V is negative baseline DCT process frame marker extended sequential DCT frame marker, Huffman coding progressive DCT frame marker, Huffman coding lossless process frame marker, Huffman coding differential sequential DCT frame marker, Huffman coding differential progressive DCT frame marker, Huffman coding differential lossless process frame marker, Huffman coding sequential DCT frame marker, arithmetic coding progressive DCT frame marker, arithmetic coding lossless process frame marker, arithmetic coding differential sequential DCT frame marker, arithmetic coding differential progressive DCT frame marker, arithmetic coding differential lossless process frame marker, arithmetic coding start-of-image marker start-of-scan marker context-index for coding of first magnitude category when V is positive quantized DCT coefficient shift right logical operation logical shift right of α by β bits start of spectral selection band in zig-zag sequence context-index for coding of sign decision 4-bit size category of DC difference or AC coefficient amplitude stack counter parameter controlling inversion of sense of MPS parameter used in coding magnitude of V context-index for coding of V = 0 decision summation index for parameter limits computation temporary variable CCITT Rec. T.81 (1992 E) 11

16 Ta j Tb Tc Td j TEM Th Tq Tq i U V V i V i,j V max V t VALPTR V1 V2 x i X X i AC entropy table destination selector for jth component in scan arithmetic conditioning table destination identifier Huffman coding or arithmetic coding table class DC entropy table destination selector for jth component in scan temporary marker Huffman table destination identifier in DHT segment quantization table destination identifier in DQT segment quantization table destination selector for ith component in frame DC and lossless coding conditioning upper bound parameter symbol or value being either encoded or decoded vertical sampling factor for ith component jth value for length i in HUFFVAL largest vertical sampling factor temporary variable list of indices for first value in HUFFVAL for each code length symbol value symbol value number of columns in ith component number of samples per line in component with largest horizontal dimension ith statistics bin for coding magnitude category decision X1, X2, X3,..., X15 designation of context-indices for coding of magnitude categories in the arithmetic coding models XHUFCO XHUFSI X values y i Y ZRL ZZ(K) ZZ(0) extended Huffman code table table of sizes of extended Huffman codes values within the quotes are hexadecimal number of lines in ith component number of lines in component with largest vertical dimension value in HUFFVAL assigned to run of 16 zero coefficients Kth element in zig-zag sequence of quantized DCT coefficients quantized DC coefficient in zig-zag sequence order 4 General The purpose of this clause is to give an informative overview of the elements specified in this Specification. Another purpose is to introduce many of the terms which are defined in clause 3. These terms are printed in italics upon first usage in this clause. 12 CCITT Rec. T.81 (1992 E)

17 4.1 Elements specified in this Specification There are three elements specified in this Specification: a) An encoder is an embodiment of an encoding process. As shown in Figure 1, an encoder takes as input digital source image data and table specifications, and by means of a specified set of procedures generates as output compressed image data. b) A decoder is an embodiment of a decoding process. As shown in Figure 2, a decoder takes as input compressed image data and table specifications, and by means of a specified set of procedures generates as output digital reconstructed image data. c) The interchange format, shown in Figure 3, is a compressed image data representation which includes all table specifications used in the encoding process. The interchange format is for exchange between application environments. Encoder TISO /d001 Source image data Table specifications Compressed image data Figure 1 Encoder FIGURE 1 [D01] 5 cm = 195% Decoder TISO /d002 Compressed image data Table specifications Reconstructed image data Figure 2 Decoder FIGURE 2 [D02] 6 cm 234% Figures 1 and 2 illustrate the general case for which the continuous-tone source and reconstructed image data consist of multiple components. (A colour image consists of multiple components; a grayscale image consists only of a single component.) A significant portion of this Specification is concerned with how to handle multiple-component images in a flexible, application-independent way. CCITT Rec. T.81 (1992 E) 13

18 Application environment A Compressed image data, including table specifications Application environment B TISO /d003 Figure 3 Interchange format for compressed image data FIGURE 3 [D03] 9,5cm = 371 % These figures are also meant to show that the same tables specified for an encoder to use to compress a particular image must be provided to a decoder to reconstruct that image. However, this Specification does not specify how applications should associate tables with compressed image data, nor how they should represent source image data generally within their specific environments. Consequently, this Specification also specifies the interchange format shown in Figure 3, in which table specifications are included within compressed image data. An image compressed with a specified encoding process within one application environment, A, is passed to a different environment, B, by means of the interchange format. The interchange format does not specify a complete coded image representation. Application-dependent information, e.g. colour space, is outside the scope of this Specification. 4.2 Lossy and lossless compression This Specification specifies two classes of encoding and decoding processes, lossy and lossless processes. Those based on the discrete cosine transform (DCT) are lossy, thereby allowing substantial compression to be achieved while producing a reconstructed image with high visual fidelity to the encoder s source image. The simplest DCT-based coding process is referred to as the baseline sequential process. It provides a capability which is sufficient for many applications. There are additional DCT-based processes which extend the baseline sequential process to a broader range of applications. In any decoder using extended DCT-based decoding processes, the baseline decoding process is required to be present in order to provide a default decoding capability. The second class of coding processes is not based upon the DCT and is provided to meet the needs of applications requiring lossless compression. These lossless encoding and decoding processes are used independently of any of the DCT-based processes. A table summarizing the relationship among these lossy and lossless coding processes is included in The amount of compression provided by any of the various processes is dependent on the characteristics of the particular image being compressed, as well as on the picture quality desired by the application and the desired speed of compression and decompression. 14 CCITT Rec. T.81 (1992 E)

19 4.3 DCT-based coding Figure 4 shows the main procedures for all encoding processes based on the DCT. It illustrates the special case of a singlecomponent image; this is an appropriate simplification for overview purposes, because all processes specified in this Specification operate on each image component independently. 8 8 blocks DCT-based encoder FDCT Quantizer Entropy encoder Source image data Table specifications Table specifications Compressed image data TISO /d004 Figure 4 DCT-based encoder simplified diagram FIGURE 4 [D04] 7 cm = 273 % In the encoding process the input component s samples are grouped into 8 8 blocks, and each block is transformed by the forward DCT (FDCT) into a set of 64 values referred to as DCT coefficients. One of these values is referred to as the DC coefficient and the other 63 as the AC coefficients. Each of the 64 coefficients is then quantized using one of 64 corresponding values from a quantization table (determined by one of the table specifications shown in Figure 4). default values for quantization tables are specified in this Specification; applications may specify values which customize picture quality for their particular image characteristics, display devices, and viewing conditions. After quantization, the DC coefficient and the 63 AC coefficients are prepared for entropy encoding, as shown in Figure 5. The previous quantized DC coefficient is used to predict the current quantized DC coefficient, and the difference is encoded. The 63 quantized AC coefficients undergo no such differential encoding, but are converted into a onedimensional zig-zag sequence, as shown in Figure 5. The quantized coefficients are then passed to an entropy encoding procedure which compresses the data further. One of two entropy coding procedures can be used, as described in 4.6. If Huffman encoding is used, Huffman table specifications must be provided to the encoder. If arithmetic encoding is used, arithmetic coding conditioning table specifications may be provided, otherwise the default conditioning table specifications shall be used. Figure 6 shows the main procedures for all DCT-based decoding processes. Each step shown performs essentially the inverse of its corresponding main procedure within the encoder. The entropy decoder decodes the zig-zag sequence of quantized DCT coefficients. After dequantization the DCT coefficients are transformed to an 8 8 block of samples by the inverse DCT (IDCT). 4.4 Lossless coding Figure 7 shows the main procedures for the lossless encoding processes. A predictor combines the reconstructed values of up to three neighbourhood samples at positions a, b, and c to form a prediction of the sample at position x as shown in Figure 8. This prediction is then subtracted from the actual value of the sample at position x, and the difference is losslessly entropy-coded by either Huffman or arithmetic coding. CCITT Rec. T.81 (1992 E) 15

20 DC AC AC DC i - 1 DC i Block i - 1 Block i DIFF = DC - DC i i - 1 TISO /d005 Differential DC encoding AC AC Zig-zag order Figure 5 Preparation of quantized coefficients for entropy encoding FIGURE 5 [D05] 8 cm = 313 % DCT-based decoder Entropy decoder Dequantizer IDCT TISO /d006 Compressed image data Table specifications Table specifications Reconstructed image data Figure 6 DCT-based decoder simplified diagram FIGURE 6 [D06] 6,5 cm = 254 % Lossless encoder Predictor Entropy encoder TISO /d007 Source image data Table specifications Compressed image data Figure 7 Lossless encoder simplified diagram FIGURE 7 [D07] 6,5 cm = 254 % 16 CCITT Rec. T.81 (1992 E)

21 c a b x TISO /d008 Figure 8 3-sample prediction neighbourhood FIGURE 8 [D08] 5 cm = 195 % This encoding process may also be used in a slightly modified way, whereby the precision of the input samples is reduced by one or more bits prior to the lossless coding. This achieves higher compression than the lossless process (but lower compression than the DCT-based processes for equivalent visual fidelity), and limits the reconstructed image s worst-case sample error to the amount of input precision reduction. 4.5 Modes of operation There are four distinct modes of operation under which the various coding processes are defined: sequential DCT-based, progressive DCT-based, lossless, and hierarchical. (Implementations are not required to provide all of these.) The lossless mode of operation was described in 4.4. The other modes of operation are compared as follows. For the sequential DCT-based mode, 8 8 sample blocks are typically input block by block from left to right, and blockrow by block-row from top to bottom. After a block has been transformed by the forward DCT, quantized and prepared for entropy encoding, all 64 of its quantized DCT coefficients can be immediately entropy encoded and output as part of the compressed image data (as was described in 4.3), thereby minimizing coefficient storage requirements. For the progressive DCT-based mode, 8 8 blocks are also typically encoded in the same order, but in multiple scans through the image. This is accomplished by adding an image-sized coefficient memory buffer (not shown in Figure 4) between the quantizer and the entropy encoder. As each block is transformed by the forward DCT and quantized, its coefficients are stored in the buffer. The DCT coefficients in the buffer are then partially encoded in each of multiple scans. The typical sequence of image presentation at the output of the decoder for sequential versus progressive modes of operation is shown in Figure 9. There are two procedures by which the quantized coefficients in the buffer may be partially encoded within a scan. First, only a specified band of coefficients from the zig-zag sequence need be encoded. This procedure is called spectral selection, because each band typically contains coefficients which occupy a lower or higher part of the frequency spectrum for that 8 8 block. Secondly, the coefficients within the current band need not be encoded to their full (quantized) accuracy within each scan. Upon a coefficient s first encoding, a specified number of most significant bits is encoded first. In subsequent scans, the less significant bits are then encoded. This procedure is called successive approximation. Either procedure may be used separately, or they may be mixed in flexible combinations. In hierarchical mode, an image is encoded as a sequence of frames. These frames provide reference reconstructed components which are usually needed for prediction in subsequent frames. Except for the first frame for a given component, differential frames encode the difference between source components and reference reconstructed components. The coding of the differences may be done using only DCT-based processes, only lossless processes, or DCT-based processes with a final lossless process for each component. Downsampling and upsampling filters may be used to provide a pyramid of spatial resolutions as shown in Figure 10. Alternatively, the hierarchical mode can be used to improve the quality of the reconstructed components at a given spatial resolution. Hierarchical mode offers a progressive presentation similar to the progressive DCT-based mode but is useful in environments which have multi-resolution requirements. Hierarchical mode also offers the capability of progressive coding to a final lossless stage. CCITT Rec. T.81 (1992 E) 17

22 A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A TISO /d009 Progressive Sequential Figure 9 Progressive versus sequential presentation FIGURE 9 [D09] 9,5 cm = 371 % TISO /d010 Figure 10 Hierarchical multi-resolution encoding FIGURE 10 [D10] 9.5 cm = 374 % 4.6 Entropy coding alternatives Two alternative entropy coding procedures are specified: Huffman coding and arithmetic coding. Huffman coding procedures use Huffman tables, determined by one of the table specifications shown in Figures 1 and 2. Arithmetic coding procedures use arithmetic coding conditioning tables, which may also be determined by a table specification. default values for Huffman tables are specified, so that applications may choose tables appropriate for their own environments. Default tables are defined for the arithmetic coding conditioning. 18 CCITT Rec. T.81 (1992 E)

23 The baseline sequential process uses Huffman coding, while the extended DCT-based and lossless processes may use either Huffman or arithmetic coding. 4.7 Sample precision For DCT-based processes, two alternative sample precisions are specified: either 8 bits or 12 bits per sample. Applications which use samples with other precisions can use either 8-bit or 12-bit precision by shifting their source image samples appropriately. The baseline process uses only 8-bit precision. DCT-based implementations which handle 12-bit source image samples are likely to need greater computational resources than those which handle only 8-bit source images. Consequently in this Specification separate normative requirements are defined for 8-bit and 12-bit DCT-based processes. For lossless processes the sample precision is specified to be from 2 to 16 bits. 4.8 Multiple-component control Subclauses 4.3 and 4.4 give an overview of one major part of the encoding and decoding processes those which operate on the sample values in order to achieve compression. There is another major part as well the procedures which control the order in which the image data from multiple components are processed to create the compressed data, and which ensure that the proper set of table data is applied to the proper data units in the image. (A data unit is a sample for lossless processes and an 8 8 block of samples for DCT-based processes.) Interleaving multiple components Figure 11 shows an example of how an encoding process selects between multiple source image components as well as multiple sets of table data, when performing its encoding procedures. The source image in this example consists of the three components A, B and C, and there are two sets of table specifications. (This simplified view does not distinguish between the quantization tables and entropy coding tables.) A B Encoding process C Source image data Table specification 1 Table specification 2 Compressed image data TISO /d011 Figure 11 Component-interleave and table-switching control FIGURE 11 [D11] 7 cm = 273 % In sequential mode, encoding is non-interleaved if the encoder compresses all image data units in component A before beginning component B, and then in turn all of B before C. Encoding is interleaved if the encoder compresses a data unit from A, a data unit from B, a data unit from C, then back to A, etc. These alternatives are illustrated in Figure 12, which shows a case in which all three image components have identical dimensions: X columns by Y lines, for a total of n data units each. CCITT Rec. T.81 (1992 E) 19

24 X X X A 1 A 2 B1 B2 C1 C2 Y Y Y A n B n C n TISO /d012 A, A,...A, 1 2 n B, B,...B, C, C,...C 1 2 n 1 2 n Scan 1 Scan 2 Scan 3 Data unit encoding order, non-interleaved A 1, B 1, C 1, A 2, B 2, C 2,...A n, B n, Cn Scan 1 Data unit encoding order, interleaved Figure 12 Interleaved versus non-interleaved encoding order FIGURE 12 [D12] 9,5 cm = 371 % These control procedures are also able to handle cases in which the source image components have different dimensions. Figure 13 shows a case in which two of the components, B and C, have half the number of horizontal samples relative to component A. In this case, two data units from A are interleaved with one each from B and C. Cases in which components of an image have more complex relationships, such as different horizontal and vertical dimensions, can be handled as well. (See Annex A.) X X/2 X/2 A 1 A 2 B1 B2 C C 1 2 Y Y Y A n B n/2 C n/2 TISO /d013 A, A, B, C, A, A, B, C,...A, A, B, C n-1 n n/2 n/2 Scan 1 Data unit encoding order, interleaved Figure 13 Interleaved order for components with different dimensions FIGURE 13 [D13] 8 cm = 313 % 20 CCITT Rec. T.81 (1992 E)