Sourcing Digital Video. G6DPMM - Lecture 12. Digital Video Playback. Digital Video Compression. Compression Codecs. Common Codecs.

Transcription

1 Sourcing Digital Video G6DPMM - Lecture 12 Digital Video Film as Digital Becoming more common as costs fall May be compressed in the camera or uncompressed Stored on magnetic tape D1, D2, D3, D5, D9 (Digital-S) Digital Betacam, BetacamSX, BetacamIMX DVCAM DVCPRO DV / MiniDV (consumer) Digital-8 (uses Hi-8 tapes) Compressed vs Uncompressed Film as Analogue Digitise with capture board (frame grabber) Wildly variable in price and quality Digital Video Playback Playback Software / Multimedia Framework Windows Media Player (Microsoft) Bundled with Windows (3.1 - XP) Free versions for Mac OSX and Solaris Media framework includes video infrastructure File formats (e.g. AVI / WMV) Quicktime Bundled with Macintosh (OS 7 X) Free version for Windows Media framework includes video infrastructure MOV file format RealNetworks RealPlayer (Windows / Macintosh / Linux / Solaris Originally a streaming technology, now a full blown media framework Digital Video Compression Video consists of a time-series of frames Frames are still images NB still image compression techniques Sequences of frames may be compressed by: Motion estimation Only updating the parts of the frame that have moved E.g. consider a talking head! Motion compensation Finding static parts of the frame and moving them around E.g. consider a pan or zoom Compression Codecs Playback software (media framework) does not usually contain compression/decompression software Compression software implementing a specific algorithm is called a codec Codecs are required by the framework many different codecs can be installed into a given framework NB compatibility and distribution (legality) issues Common Codecs Indeo Cinepak P*64 Sorenson ClearVideo VDOwave RealVideo MPEG NB this is not a single codec MPEG also has many spin-offs such as Microsoft WMV or DivX

2 Video Compression Uncompressed video is expensive Hard to store very large volumes of data Expensive equipment required to manipulate Compression is always lossy Multi-way trade-off File size / transfer rate (bandwidth) Frame rate (typically 15fps for multimedia) Image quality Frame size Different codecs have differing strengths and weaknesses (e.g. Indeo vs Cinepak) Indeo Algorithms developed for DVI Intel i750 chipset Indeo implemented in software Proprietary intel algorithms Two levels of compression PLV (Production Level Video) broadcast quality RTV (Real Time Video) Typical compression ratio of 160:1 Beware versions (5.1 safe on modern machines, 3.2 fairly ubiquitous) MPEG Moving Picture Expert Group Established by ISO in 1988 Remit define standards for movie with soundtracks Digital television Video phones Consumer electronics Digital cinema Video games Family of standards Always lossy Discrete Cosine Transform (DCT) algorithm MPEG-1 (1993) The original MPEG standard Objective: deliver video and audio from CD-ROM with comparable quality to VHS tape and CD-Audio Encodes an audiovisual signal in about 1.5 Mbps Images are progressively scanned This fundamentally limits quality Motion compensation I-Frames (reference) P-Frames (predictive) B-Frames (bidirectional predictive) MPEG Frame Types I-Frames Reference frame Complete image no missing information Subsequent P-frames are based upon an I-frame P-Frames Only contain the difference between themselves and the previous P-frame or I-frame Anything that does not change is not stored B-Frames Contain the difference between themselves and both frames on either side B-frames are not always implemented because of memory requirements The MPEG GOP GOP Structure Group of Pictures Specifies I / P frame structure (e.g. 1 I frame per 11 P frames. IPPPPPPPPPPPIPPPPPPPPPPPIPPPPPPPPPPPI The fewer I frames the better the compression but frame to frame calculations are less accurate leading to artefacts PEL (picture elements) Block of pixels, used as a basis for compression Motion of PEL is predicted based on I, P or B frames

3 The MPEG-1 standard Five parts (separate ISO standards) 1. Systems 2. Video 3. Audio (NB MPEG 1, level 3 MP3) 4. Conformance testing 5. C implementation (encoding and decoding) MPEG Audio MP3 is MPEG 1, audio layer 3 Layer 1: 384, 4:1 compression Layer 2: kbit/s, 6:1..8:1 compression Layer 3: kbit/s, 10:1..12:1 compression Compression Modified discrete cosine transfom (MDCT) 32 band polyphase quadrature filter (PQF) Alias reduction postprocessing Severe design limitations of MP3 Bitrate, time resolution, lack of scaleability especially for high frequencies Applications of MPEG-1 MPEG-1 compression is cheap and rapid Consumer Devices Multimedia / CD-ROM Internet Video Video CD (production is extremely cheap) Internet audio (MP2) Audio only devices (MP3 players) MPEG-2 (1994) A completely different standard from MPEG 1. Objective: Deliver video and audio at comparable quality to broadcast video at roughly 4 times the typical MPEG-1 bit rate Requires between 3 and 15 Mbps Improvements over MPEG-1 Higher resolution Higher image quality Supports interlaced video formats Multiresolution scalability Multiple sound track features Scaleable The MPEG-2 Standard ISO parts 640 patents, held by 24 corporations and one university Video Encoding (grossly simplified) I, P and B frames similar to MPEG-1 Video is separated into: 1 luminance and 2 chrominance channels, 4 luminance macroblocks variable number of chrominance macroblocks Macroblocks of P and B frames are compared, and a motion vector calculated Each block is treated with DCT, the result is run-length encoded and a huffman compression is applied Scaleable MPEG MPEG-2 is scaleable (layered coding) Single coded bitstream is scaleable to various bitrates Layers can be incrementally decoded Basic layer low quality video Subsequent layers higher quality video If the bandwidth is low, only the bitstream from the base layer is sent If the bandwidth changes, more or fewer layers may be used The same principle can be used on noisy connections

4 Applications of MPEG-2 DVD Video HDTV Ditgital satelite / cable TV Digital video recorders Video conferencing Some games Some multimedia (usually where very high quality is essential) NB encoding is processor intensive thus more expensive than MPEG-1 Editing MPEG 1 & 2 High performance compression formats Not designed for editing Only I-frames are stored in their entirety This makes editing, rewinding or even bookmarking extremely difficult MPEG editors tend to be crude Serious editing involves decmopressing and recompressing Quality loss Moral don t edit MPEG if at all possible Therefore don t store digital video as MPEG! MPEG-4 (1998) A different approach to video Designed primarily for streaming video (e.g. on WWW), 3G phones, and other multimedia. MPEG-4 absorbs most of MPEG-1 and MPEG-2 Adds: VRML (3D) support Object-based coding User interactivity Hypermedia capability Streams from different sources can be combined Supports uploading and downloading Rights management protocol Implementation of MPEG-4 Designed to be modular Standard in 20 parts (levels) Each level contains profiles Developers implement components of MPEG-4 Probably no full implementation Various partial implementations, such as: DivX Microsoft ASF Quicktime 6 Object-based coding Media objects Audio and visual objects Natural objects (captured by camera) Synthetic objects (created by computer) Visual objects can be any shape (not just rectangles) Video manipulation (insertion, deletion, rotation etc.) Video indexing and retrieval Texture maps Sprites 3-d models and animation Probably the future of multimedia! MPEG-7 Multimedia content description interface Aimed at addressing issues of content retrieval Intended primarily for digital libraries Object based (as MPEG-4) Supported data: Still images 3D models Sampled audio Speech Video Composition information (combinations of the above) Data representation Binary objects, XML or both

5 MPEG-21 Designed to manage digital media resources Digital item declaration Digital item identification and description Content management and usage Intellectual property management and protection Terminals and networks Content representation Event reporting