Overview: image and video coding standards. JPEG: background

Transcription

1 JPEG Overview: image and video coding standards MPEG-1 and MPEG-2 H.261 and H.263 Progress in video compression Bernd Girod: EE368b Image and Video Compression Coding Standards no. 1 JPEG: background JPEG = Joint Photographic Experts Group: Joint standards committee of ITU-T and ISO Flexible standard for monochrome and color image compression: Digital Compression and Coding of Continuous-tone Still Images Intraframe coding scheme, optimized for still images Flexible picture size Coding of color components separately, arbitrary color spaces possible, best compression for Y/R-Y/B-Y Variable compression ratio Compression 24:1 for ITU-R 601 images without loss of quality Bernd Girod: EE368b Image and Video Compression Coding Standards no. 2 1

2 JPEG derived industry standards JFIF (JPEG File Interchange Format, <xxxx.jpg>) JTIP (JPEG Tiled, Pyramid Format) TIFF (Tagged Image File Format) SPIFF (Still Picture Interchange File Format, JPEG Part 3) FlashPix Developed by Kodak, Hewlett- Packard, Microsoft (1996) Widely used in digital still cameras Bernd Girod: EE368b Image and Video Compression Coding Standards no. 3 JPEG: baseline algorithm Image in Blockwise 8x8 DCT weighting / uniform quantization Huffman Coding Image out Blockwise inverse 8x8 DCT Inverse weighting Huffman Decoding Bernd Girod: EE368b Image and Video Compression Coding Standards no. 4 2

3 JPEG: quantizer stepsize Different weighting matrices are standardized, adapted to human visual contrast sensitivity Example: Inverse weighting for ITU-R 601 images Luminance Chrominance Bernd Girod: EE368b Image and Video Compression Coding Standards no. 5 JPEG: coding of DCT coefficients Differential coding of DC coefficient (0,0) Zig-zag-scan and run-level-coding of AC coefficients Two options for Huffman coding: with predefined code tables (one-pass system) code tables optimized for individual image (two-pass system) Bernd Girod: EE368b Image and Video Compression Coding Standards no. 6 3

4 JPEG-2000 New generation JPEG standard, recently finalized. Discrete wavelet transform Embedded scalar quantization with deadzone Entropy-coding: bit-plane arithmetic coding with context modeling Typically 30% lower bit-rate than old JPEG at the same image quality Bernd Girod: EE368b Image and Video Compression Coding Standards no. 7 Video coding standards ITU-R Mbit/s MPEG-2 digital TV H.261 ISDN picture phone 2-6 Mbps 64 kbps H.263 PSTN picture phone < 28.8 kbps Bernd Girod: EE368b Image and Video Compression Coding Standards no. 8 4

5 Generic encoder block diagram for H.261, H.263, and MPEG Control Video signal S - e Intraframe DCT coder Intraframe Decoder Channel Motion compensated predictor Decoder Bernd Girod: EE368b Image and Video Compression Coding Standards no. 9 Image formats 176 x 144 sub QCIF QCIF ITU-R x 288 CIF Bernd Girod: EE368b Image and Video Compression Coding Standards no. 10 5

6 ISO MPEG MPEG-1 Standard (1991) Target bit-rate about 1.5 Mbit/s Typical image format CIF, no interlace Frame rate fps Main application: video storage for multimedia (e.g., on CD-ROM) MPEG-2 Standard (1994) Extension for interlace, optimized for TV resolution (NTSC: 704 x 480 Pixel) Image quality similar to NTSC, PAL, SECAM at 4-8 Mbit/s HDTV at 20 Mbit/s Bernd Girod: EE368b Image and Video Compression Coding Standards no. 11 MPEG: hierarchical syntax I "Video Sequence" "Group of Pictures" = GOP, GOP structure is very flexible I-Picture P-Picture P-Picture B-Pictures time Bernd Girod: EE368b Image and Video Compression Coding Standards no. 12 6

7 MPEG: hierarchical syntax II "Picture": N x M luminance samples, N/2 x M/2 Cb- and Crsamples "Slice": Sequence of Macroblocks "Macroblock 16x16 luminance samples 8x8 Cbsamples 8x8 Crsamples "Block": 8x8 samples Bernd Girod: EE368b Image and Video Compression Coding Standards no. 13 MPEG: coding of I-pictures I-pictures: intraframe coded 8x8 DCT Arbitrary weighting matrix for coefficients Differential coding of DC-coefficients Uniform quantization Zig-zag-scan, run-level-coding Entropy coding Unfortunately, not quite JPEG Bernd Girod: EE368b Image and Video Compression Coding Standards no. 14 7

8 MPEG: coding of P-pictures Motion-compensated prediction from an encoded I-picture or P-picture (DPCM) Half-pel accuracy of motion compensation, bilinear interpolation One displacement vector per macroblock Differential coding of displacement vectors Coding of prediction error with 8x8-DCT, uniform threshold quantization, zig-zag-scan as in I-pictures Bernd Girod: EE368b Image and Video Compression Coding Standards no. 15 MPEG: coding of B-pictures Motion-compensated prediction from two consecutive P- or I-pictures either or or only forward prediction (1 vector/macroblock) only backward prediction (1 vector/macroblock) Average of forward and backward prediction = interpolation (2 vectors/macroblock) Half-pel accuracy of motion compensation, bilinear interpolation Coding of prediction error with 8x8-DCT, uniform quantization, zig-zag-scan as in I-pictures Bernd Girod: EE368b Image and Video Compression Coding Standards no. 16 8

9 ITU-T Rec. H.261 International standard for ISDN picture phones and for video conferencing systems (1990) Image format: CIF (352 x 288 Y samples) or QCIF (176 * 144 Y samples), frame rate fps Bit-rate: multiple of 64 kbps (= ISDN-channel), typically 128 kbps including audio. Picture quality: for 128 kbps acceptable with limited motion in the scene Stand-alone videoconferencing system or desk-top videoconferencing system, integrated with PC Bernd Girod: EE368b Image and Video Compression Coding Standards no. 17 ITU-T Rec. H.263 International standard for picture phones over analog subscriber lines (1995) Image format usually CIF, QCIF or Sub-QCIF, frame rate usually below 10 fps Bit-rate: arbitrary, typically 20 kbps for PSTN Picture quality: with new options as good as H.261 (at half rate) Software-only PC video phone or TV set-top box Widely used as compression engine for Internet video streaming H.263 is also the compression core of the MPEG-4 standard Example: 8x8 ViaTV phone VC 105 Bernd Girod: EE368b Image and Video Compression Coding Standards no. 18 9

10 Motion compensation H.261 vs H.263 H.261 (1990): integer-pel accuracy, loop filter, 1 motion vector per MB H.263 (1995): half-pel accuracy, no loop filter, 1 motion vector per MB H.263 option Advanced Prediction Mode overlapped block motion compensation (OBMC), switch between 1 or 4 motion vectors per MB H.263 option PB frames Bernd Girod: EE368b Image and Video Compression Coding Standards no. 19 Performance of H.263 and H ) 2) 3) 4) 5) PSNR [db] ) H.263 2) H.263 w/o options 3) H.261 4) H.263 w/o options, integer-pel ME 5) H.261 w/o loop filter rate [kbps] Bernd Girod: EE368b Image and Video Compression Coding Standards no

11 Performance of H.263 SAC mode rate [kbps] 1) SAC-mode 2) w/o options 1) 2) Bernd Girod: EE368b Image and Video Compression Coding Standards no. 21 H.263: overlapped block motion compensation (OBMC) remote luminance block Current luminance block (8x8) remote luminance block remote luminance block remote luminance block M A C R O B L O C K Bernd Girod: EE368b Image and Video Compression Coding Standards no

12 H.263: OBMC weights for MV of current luminance block for remote MV of top/bottom luminance block for remote MV of left/right luminance block Bernd Girod: EE368b Image and Video Compression Coding Standards no. 23 Performance of H.263 AP mode rate [kbps] 1) AP-mode 2) w/o options 1) 2) Bernd Girod: EE368b Image and Video Compression Coding Standards no

13 H.263: PB-frames forward prediction P B P bidirectional prediction bidirectional prediction PB-Frame Bernd Girod: EE368b Image and Video Compression Coding Standards no. 25 Performance of H.263 PB-mode PSNR [db] rate [kbps] 1) 3a) 3b) 2) 1) w/o options (6.25 fps) 2) w/o options (12.5 fps) 3) PB-mode (12.5 fps) a) P-frames b) B-frames Bernd Girod: EE368b Image and Video Compression Coding Standards no

14 Video compression progress Intraframe coding: only spatial correlation exploited DCT [Ahmed, Natarajan, Rao 1974], JPEG [1992] Complexity increases Conditional replenishment, DPCM, scalar quantization H.120 [1984] Frame difference coding H.120 Version 2 [1988] Motion compensation: integer-pel accurate displacements H.261 [1991] Half-pel accurate motion compensation MPEG-1 [1993], MPEG-2/H.262 [1994] Variable block-size motion compensation H.263 [1996], MPEG-4 [1999] Bernd Girod: EE368b Image and Video Compression Coding Standards no. 27 Video compression progress PSNR [db] Variable block size motion compensation (H ) ~40 % Half-pel motion compensation (MPEG ) ~20 % Integer-pel motion compensation (H ) Intraframe DCT coding (JPEG) Conditional Replenishment (H.120) Frame difference coding (H ) Foreman 10 Hz, QCIF 100 frames encoded ~30 % Bit Rate [kbps] 600 Bernd Girod: EE368b Image and Video Compression Coding Standards no

15 Video compression progress PSNR [db] Variable block size motion compensation (H ) Conditional Replenishment (H.120) Mother & Daughter 10 Hz, QCIF 100 frames encoded 34 ~60 % ~70 % Intraframe DCT coding (JPEG) Bit Rate [kbps] Bernd Girod: EE368b Image and Video Compression Coding Standards no. 29 Video compression progress PSNR [db] Variable block size motion compensation (H ) ~ 35 % ~ 40 % Integer-pel motion compensation (H ) Mobile &Calendar 10 Hz, QCIF 100 frames encoded Intraframe DCT coding (JPEG) Bit Rate [kbps] Bernd Girod: EE368b Image and Video Compression Coding Standards no