COIT 475 Multimedia Network Technology Hossam M.J. Mustafa FCITR, KAU, Rabigh
Part I Introduction to Multimedia Networking COIT 475 Multimedia Network Technology 2
Overview What is Multimedia? Characteristics of multimedia Various media types What is Multimedia networking? User requirements of multimedia applications on the network Technologies associated with multimedia networking Overall structure of multimedia networking COIT 475 Multimedia Network Technology 3
Definition of multimedia What is multimedia? Hard to find a clear-cut definition In general, multimedia is an integration of text, graphics, still and moving images, animation, sounds, and any other medium where every type of information can be represented, stored, transmitted and processed digitally Characteristics of multimedia Digital key concept Integration of multiple media type, usually including video or/and audio May be interactive or non-interactive COIT 475 Multimedia Network Technology 4
Various Media Types Text, Graphics, image, video, animation, sound, etc. Classifications of various media types Captured vs. synthesized media Captured media (natural) : information captured from the real world Example: still image, video, audio Synthesized media (artificial) : information synthesize by the computer Example: text, graphics, animation Discrete vs. continuous media Discrete media: spaced-based, media involve the space dimension only (Text, Image, Graphics) Continuous media: time-based, media involves both the space and the time dimension (Video, Sound, Animation) COIT 475 Multimedia Network Technology 5
Classification of Media Type Sound Video Animation Continuous Continuous Image Text Graphics Discrete Discrete Captured From real world Synthesized By computer COIT 475 Multimedia Network Technology 6
Plain text Text Unformatted Characters coded in binary form ASCII code All characters have the same style and font Rich text Formatted Contains format information besides codes for characters No predominant standards Characters of various size, shape and style, e.g. bold, colorful COIT 475 Multimedia Network Technology 7
Plain Text vs. Rich Text An example of Plain text Example of Rich text COIT 475 Multimedia Network Technology 8
Graphics Revisable document that retains structural information Consists of objects such as lines, curves, circles, etc Usually generated by graphic editor of computer programs 10 5 Example of graphics (FIG file) 0-5 -10 4 2 0-2 -4-4 -2 0 2 4 COIT 475 Multimedia Network Technology 9
Images 2D matrix consisting of pixels Pixel smallest element of resolution of the image One pixel is represented by a number of bits Pixel depth the number of bits available to code the pixel Have no structural information Two categories: scanned vs. synthesized still image Digital still image Computer software Capture and A/D conversion Synthesized image Scanned image Camera COIT 475 Multimedia Network Technology 10
Images (cont.) Examples of images Binary image pixel depth 1 Gray-scale pixel depth 8 Color image pixel depth 24 Gray-scale image Binary image COIT 475 Multimedia Network Technology 11
Graphics Graphics vs. Image Revisable documents Document format retains structural information Semantic content is preserved in presentation Described as objects Images Not revisable Document format is unaware of any structural information Semantic content is NOT preserved Described as bitmaps formed of individual pixels COIT 475 Multimedia Network Technology 12
Video vs. Animation Both images and graphics can be displayed as a succession of view which create an impression of movement Video moving images or moving pictures Captured or Synthesized Consists of a series of bitmap images Each image is called a frame Frame rate: the speed to playback the video (frame per second) Animation moving graphics Generated by computer program (animation authoring tools) Consists of a set of objects The movements of the objects are calculated and the view is updated at playback COIT 475 Multimedia Network Technology 13
1-D time-based signal 0.2 Sound 0.15 0.1 0.05 0-0.05-0.1-0.15 Speech vs. non-speech sound -0.2 0 100 200 300 400 500 600 700 800 900 1000 Speech supports spoken language and has a semantic content Non-speech does not convey semantics in general Natural vs. structured sound Natural sound Recorded/generated sound wave represented as digital signal Example: Audio in CD, WAV files Structured sound Synthesize sound in a symbolic way Example: MIDI file COIT 475 Multimedia Network Technology 14
Networked Multimedia Local vs. networked multimedia Local: storage and presentation of multimedia information in standalone computers Sample applications: DVD Networked: involve transmission and distribution of multimedia information on the network Sample applications: videoconferencing, web video broadcasting, multimedia Email, etc. A scenario of multimedia networking Image server Video server Internet COIT 475 Multimedia Network Technology 15
Consideration of Networked Multimedia Characteristics of multimedia information Large data volume Exercise: What is the size of a video clip of 60 minutes if the frame size is 640*480, the pixel depth is 24, and the frame rate is 24 fps? Real-time property Continuous display Delay requirement of multimedia applications Properties of current Internet Limitation of bandwidth Best effort network, cannot guarantee quality of multimedia applications Heterogeneity Different user requirements Different user network conditions COIT 475 Multimedia Network Technology 16
Consideration of Networked Multimedia Requirements of multimedia applications on the network Delay requirement Quality requirement Satisfactory quality of media presentation Synchronization requirement Continuous requirement Can tolerant some degree of information loss Challenges of multimedia networking Conflict between media size and bandwidth limit of the network Conflict between the user requirement of multimedia application and the best-effort network How to meet different requirements of different users? COIT 475 Multimedia Network Technology 17
Technologies of Multimedia Networking Media compression reduce the data volume Address the1st challenge Image compression Video compression Audio compression Multimedia transmission technology Protocols for real-time transmission Rate / congestion control (traffic) Error control COIT 475 Multimedia Network Technology 18
Multimedia Networking Systems Live media transmission system Capture, compress, and transmit the media on the fly (example?) Send stored media across the network Media is pre-compressed and stored at the server. This system delivers the stored media to one or multiple receivers. (example?) Differences between the two systems For live media delivery: Real-time media capture, need hardware support Real-time compression speed is important Compression procedure can be adjusted based on network conditions For stored media delivery Offline compression better compression result is important Compression can not be adjusted during transmission COIT 475 Multimedia Network Technology 19
Generic Media Streaming System Input video Video Encoder Compressed Video Streaming Server Internet Error control, rate control can be done here to improve QoS Video Display Video Decoder Compressed Video Receiver Video Packets Error control, provide feedback to the sender COIT 475 Multimedia Network Technology 20