Digital Video. Denis Helic

Transcription

1 Digital Video Denis Helic

2 A Video is worth 1000 Pictures :-)) R 1 L R 3 L R 1 R L 2 1 L 2 3 R 5 (1) The left Hand (LH) back-tosses ball 1 (B1) in a low arc toward the centre of the juggle space (and stays right to make its next toss). (2) RH reverse cascades B2, aiming for the left middle of the juggle space, catches B1 at its peak, and carries it over the left arm to the far left. (3) LH throws B3 straight up from under the right arm and catches B2 at the left middle. (4-6) are simply the mirror image of Step 1,2 and 3. [Gillson, Beyond The Cascade] 1 R 3 L 2 1 R L 1 R 3 R L 3 R 3 L 1 L (2/43)

3 Video - General Analog video technology: Recorded analog Digital video technology: Recorded analog, A/D conversion Digital video (DV): Recorded and stored digitally Problem: data size pixel 2.4MB per frame 25 to 30 frames/sec 60MB per second 120 minutes of digital video 432GB Compression!! (3/43)

4 Human Perception of Movement(1/3) Human eye is very slow Fast projecting a sequence of static images creates an illusion of movement images in second is enough to perceive the movement illusion Flickering problem! Flicker-free: >75 images per second (4/43)

5 Cinema 24 images per second (still flickering) Human Perception of Movement(2/3) Breaking images into 2 or 3 images 48 to 72 images per second remove flickering Television PAL: 25 images per second (still flickering) Break an image into two images (odd lines, even lines) Display two half images subsequently 50 half-images per second Interlacing! (5/43)

6 Perception of movement Perception with eye Acceleration (with ear) Physics of the eye: Human Perception of Movement(3/3) Accommodation: different lens-form for different distances Vergency: difference direction of eyes for different distances Decoupling of perception, acceleration, accommodation, vergency may lead to sickness (6/43)

7 Analog Video(1/6) Problem: repeated reading/writing decreases the quality Analog video standards: BAS, FBAS (composite video): Contains Bild, Austastung, Synchronisation If in color: Farbe, Bild, Austastung, Synchronisation (7/43)

8 Analog Video(2/6) Component Video: primary signals according to the applied color model (RGB, YUV, YIQ) PAL-Video: German-speaking Europe Resolution: 768x625 (576 lines visible) 25 frames per second Ratio 4:3 Square pixel Interlaced (50 half frames per second) (8/43)

9 Analog Video(3/6) NTSC Video American television standard Resolution: 640x525 (480 lines visible) frames per second Ratio 4:3 Square pixel YIC color model (9/43)

10 Analog Video(4/6) High Definition Television (HDTV) Different formats/resolutions It might become obselete because of digital video Resolution 640x480 up to 1920x1080 Ratio 4:3 or 16:9 PAL- and NTSC-frequencies supported (10/43)

11 Bandwith for different channels Analog Video(5/6) System Total BW Y channel U/I channel V/Q channel Frequency NTSC 6.0 MHz 4.2 MHz 1.0 MHz 0.6 MHz Interlaced PAL 8.0 MHz 5.5 MHz 1.8 MHz 1.8 MHz 50 Interlaced (11/43)

12 Analog Video(6/6) Y/C (luminance/color) Video (Separated Video): Combination of FBAS and Component Video Color signals (e.g. U,V) form a signal (chrominance) Luminance is separately transmitted Applied in S-VHS and HI-8. (12/43)

13 Digital Video Direct access to single images (frames) Lossless editing Huge amounts of data Codecs (Coder/Decoder): Compression/decompression of video data Standardized interfaces (Windows: Video for Windows, DirectShow, Qucktime; Mac: Quicktime, Linux:???) Cinepak, Indeo, Video-1, MS-RLE, MJPEG, H.261, H.263, MPEG (13/43)

14 Digital Video - Codecs(1/3) Cinepak Key frames Delta frames Compression rate about 7:1 Indeo Developed by Intel for i750 Works on 4x4 makroblocks Compression rate about 7:1 Microsoft Video-1 8, 16, or 24bit color depth Compression rate about 2:1 (14/43)

15 Digital Video - Codecs(2/3) Microsoft RLE only 4 or 8 bit color depth Compression rate even worse than Video-1 (15/43)

16 Motion JPEG (MJPEG) Sequence of JPEG-images No standard for audio-data Digital Video - Codecs(3/3) Stored as AVI (Windows) or Quicktime (Mac) files digital camcorders (DV, MiniDV) - datarate 3.4MB/sec (12GB/h), incl. audio (16/43)

17 Video Compression H.261, H.263(1/4) International video standard H.261 Developed Video-conferencing/telephony by the use of ISDN Target: communication compression/decompression max. 150ms fps (30000/1001) Resolution 352x288 or 176x144 Intraframes (I-Frames): complete image Predicted Frames (P-Frames): contains motion prediction (e.g. difference to previous frame) (17/43)

18 Video Compression H.261, H.263(2/4) I P P P I P P P Zeit (18/43)

19 H.263 Improvements from H.261 Less accuracy in motion prediction Some parts of data stream are optional Video Compression H.261, H.263(3/4) Configurable parameters to increase performance (e.g. change to bidirectional prediction (MPEG)) More resolutions Image indexing using 8bit (H.261: 5bit) Results in less than half datarate (compared to H.261) Improved version (H.245) used in H.323 (IP telephony) (19/43)

20 H.261 Compression: Video Compression H.261, H.263(4/4) I-Frames similar to JPEG (fixed value for quantization) P-Frames: best-match algorithm for 16x16 blocks, transformation vector is transmitted, difference between blocks is compressed with JPEG Compression rates: format datarate uncompressed datarate H.261 CIF 352x288x Mbit/sec 24xISDN = 1.5Mbit/s QCIF 176x144x Mbit/sec 6xISDN = 384kbit/s (20/43)

21 Video Compression MPEG(1/14) Moving Pictures Expert Group 1992 Different standards: MPEG-1 (1992): Average bandwidth 1.892Mbit/sec (1.25Mbit/sec for video) Two audio channels (MPEG1 Layer III = MP3) Resolution 360x fps (CIF Europe) or 352x240x30 (CIF USA) No interlacing VHS recording possible Optimized for multimedia on CD-ROM Video-CD (VCD) (21/43)

22 Video Compression MPEG(2/14) MPEG-2 (1993): the most popular MPEG standard Bandwidths from 2 to 80Mbit/sec 24Hz - 25Hz - 30Hz Interlacing possible Best-match searches half frames as well Resolution up to 16383x16383 pixel Non-linear makroblock-quantization-table Scalable (based on resolutions or bandwidth)) SVCD, DVD (22/43)

23 Video Compression MPEG(3/14) MPEG-4 (1999) Not a fixed standard, but framework Integration of media objects (sprites, audio, 3D, VR-models,...) Synchronization between objects Interaction between users and digital world Extendable Internal scripting language: JavaScript MPEG-J java interface High license costs! (23/43)

24 MPEG-7 Video Compression MPEG(4/14) Work in progress (6 of 7 parts finished end of 2001) Embed meta-data (XML) in video data Allows searching in multimedia data Descriptors (D), Description scheme (DS), Description definition language (DDL) (24/43)

25 Video Compression MPEG(5/14) example DDL: <Creation> <Title>Fawlty Towers</Title> <Abstract> <FreeTextAnnotation> Communication Problems </FreeTextAnnotation> </Abstract> <Creator>BBC</Creator> </Creation> (25/43)

26 Video Compression MPEG(6/14) example DDL: <TextAnnotation> <FreeTextAnnotation> Basil attempts to fix the car without success. </FreeTextAnnotation> <StructuredAnnotation> <Who>Basil</Who> <WhatObject>Car</WhatObject> <WhatAction>Fix</WhatAction> <Where>Carpark</Where> </StructuredAnnotation> </TextAnnotation> (26/43)

27 Video Compression MPEG(7/14) MPEG-21 Started mid-2000 Describes a multimedia framework First results end 2001 resp (sic!) e-commerce data + meta-data + structure (27/43)

28 Video Compression MPEG(8/14) Windows Media Technology Player available on different plattforms, encoder Windows only MPEG-4 (28/43)

29 Video Compression MPEG(9/14) DivX (Digital Video Express) Hacked version of Microsoft MPEG4 V3 Codec Windows Media 7 DVD quality (origin) Based on MPEG4 MP3-audio added DivX 4: legal (DivX Networks, Inc.) DivX 5: bidirectional prediction XviD: open source (29/43)

30 Video Compression MPEG(10/14) MPEG compression: Intraframes (I-Frames) Predicted Frames (P-Frames) Bidirectionally Predicted Frames (B-Frames) (30/43)

31 Video Compression MPEG(11/14) I B B B P B B B P B B B P B B B I Zeit (31/43)

32 Compression rates of frames: Video Compression MPEG(12/14) frame-type size compression I 92 kbyte 7:1 P 32 kbyte 20:1 B 13 kbyte 50:1 average 26 kbyte 25:1 (32/43)

33 Video Compression MPEG(13/14) Order and frequency of frame-types depend on codecs Order of transmission typically not the same as order to display order to display Image Type I B B P B B P B B P B B I order of transmission Image Type I P B B P B B P B B I B B GOB GOB GOB GOB (33/43)

34 I-Frames: coded like H.261 (JPEG) P-Frames: nearly coded like H.261 Video Compression MPEG(14/14) B-Frames: best match using previous and next frame D-Frames: only used for fast-forward, bad image quality (comp. DC-Images of JPEG) (34/43)

35 Video file formats(1/3) Different file formats may use different codecs! Audio-Video-Interleaved Format (AVI) Microsoft Video for Windows based on RIFF specification Mostly use Cinepak, Indeo, MPEG-4 or DivX Supports full PAL resolution (768x576 pixel) Typically 320x240 (35/43)

36 Video file formats(2/3) Quicktime Originally on Apple, now on Windows as well Quicktime files (*.mov) contain video and audio Supports a broad range of functionality for programmers Allows new codecs Mostly use Cinepak or Indeo, now MPEG-4 (Quicktime 6) (36/43)

37 Video file formats(3/3) RealVideo Real Networks Satastream Bandwidth is shared between video and audio Lossy compression (codec RealVideo 8 ) Scalable (space and time) Resolutions from 176x132 up to 640x480 pixel Informations: guides/production8/htmfiles/video.htm (37/43)

38 Synchronized Multimedia Integration Lang. (SMIL)(1/6) smile Developed by W3C ( or http: // with RealNetworks, Microsoft, Netscape and Philips Create multimedia presentations Simple syntax (based on XML/HTML) SMIL-player interprets SMIL-document (38/43)

39 Synchronized Multimedia Integration Lang. (SMIL)(2/6) Example from Example: dreaming.html Source: sample2.smil Player: Soja (39/43)

40 Synchronized Multimedia Integration Lang. (SMIL)(3/6) Example of SMIL document (from archives/1998/09/bouthillier/: Header 1 <smil> 2 <head> 3 <layout> 4 <root-layout height="425" 5 width="450" 6 background-color="black"/> 7 8 <region id="title" 9 left="50" 10 top="150" 11 width="350" 12 height="200"/> 13 [...] 27 </layout> 28 </head> (40/43)

41 Synchronized Multimedia Integration Lang. (SMIL)(4/6) Body: 30 <body> 30 <seq> 31 <!-- This img tag displays the title screen --> 32 <text src="title.rt" 33 type="text/html" 34 region="title" 35 dur="20s"/> [...] (41/43)

42 Synchronized Multimedia Integration Lang. (SMIL)(5/6) 43 <!-- This section contains the video-annotated slideshow --> 44 <par> 45 <img src="slideshow.rp" 46 region="full" 47 fill="freeze"/> 48 <seq> 49 <video src="slide_narration_video1.rm" 50 region="video"/> 51 <audio src="slide_narration_audio1.ra"/> 52 <video src="slide_narration_video2.rm" 53 region="video"/> 54 </seq> 55 </par> 56 </seq> 57 </body> 58 </smil> (42/43)

43 Synchronized Multimedia Integration Lang. (SMIL)(6/6) Supports different types of media (text, video, audio,...) Supports <switch> for different files depending on resolution, bandwidth, etc. 1 <par> 2 <switch> 3 <img src="slideshow_hires.rp" region="full" fill="freeze" system-bit-rate="20000"/> 4 <img src="slideshow_lowres.rp" region="full" fill="freeze" system-bit-rate="45000"/> 5 </switch> 6 <audio src="video_narration.ra"/> 7 </par> Tutorial: (43/43)