End-to-endDelayofVideoconferencingoverPacket

Transcription

1 End-to-endDelayofVideoconferencingoverPacket POBox218,YorktownHeights,NewYork1598 T.J.WatsonResearchCenter-IBM MarioBaldiandYoramOfek SwitchedNetworks calltointeractnaturally,theend-to-enddelayshouldbebelowhumanperception-about withtheobjectiveofunderstandinghowitcanbecontrolledandreduced. lossprobability,andboundeddelay.inorderfortheparticipantsinavideoconference 1ms.Weidentifythecomponentsoftheend-to-enddelayinvariouscongurations Therstcontributiontotheend-to-enddelaycomesfromtheprocessingperformed Videoconferencingapplicationsrequirefromthenetworkguaranteedbandwidth,small Abstract onpicturesbeforesendingandafterreceivingthem;wecallthisprocessingdelay.the secondcontributionisthenetworkdelay.sincepicturesmustbedisplayedatthesame paceatwhichtheyhadbeencaptured,anyvariationintheprocessingandnetwork switchingwithtimedrivenpriority,and(iv)asynchronouspacketswitching.thestudy results,forvideoconferencingwithmpeg,independentofthenetworkload. delaymustbecompensatedbeforepicturesaredisplayed.thiscompensationisdoneby MPEGvideo(VBRandCBR)over:(i)dedicatedlinks,(ii)circuitswitching,(iii)packet showsthatacommontimereferenceusedwithtimedrivenprioritycanprovideadequate throughanumberofcongurations.westudythetransmissionofbothrawvideoand addingresynchronizationdelay,whichisthethirdcomponentoftheend-to-enddelay. Wedevisethesebounds,andhencetherelatedend-to-enddelay,goingstep-by-step VistitingstudentfromDipartimentodiAutomaticaeInformatica,PolitecnicodiTorino,phone

2 2TheModel Contents 1Introduction 1.1ScopeoftheWork:::::::::::::::::::::::::::::::::5 2.2RequirementsforVisualQuality::::::::::::::::::::::::::8 2.1RequirementsforInteraction:::::::::::::::::::::::::::: BueringIssues::::::::::::::::::::::::::::::9 65 3TransmissionofRawVideo 2.3TheSystem:::::::::::::::::::::::::::::::::::::1 3.1DedicatedLinkBetweenSenderandReceiver::::::::::::::::::12 3.3PacketSwitchingwithTimeDrivenPriority:::::::::::::::::::16 3.2CircuitSwitching:::::::::::::::::::::::::::::::::: BasicPrinciplesofTimeDrivenPriority::::::::::::::::: TransmissionofVideoandScheduling:::::::::::::::::: MultipleVideoconferenceCallsontheSameNetwork:::::::::: TransmissionofVBRMPEGVideo 4.1MPEGOverview::::::::::::::::::::::::::::::::::29 3.4AsynchronousPacketSwitching:::::::::::::::::::::::::: NetworkResynchronizationDelay::::::::::::::::::::: TracShapingatNetworkBoundaries::::::::::::::::::26 4.3PacketSwitchingwithTimeDrivenPriority:::::::::::::::::::38 4.2DedicatedLinkBetweenSenderandReceiver:::::::::::::::::: BitGenerationRateandQuantization:::::::::::::::::: PredictiveCoding:::::::::::::::::::::::::::::: Intra-frameCoding:::::::::::::::::::::::::::::29 4.4AsynchronousPacketSwitching:::::::::::::::::::::::::: TracShapingatNetworkBoundaries:::::::::::::::::: SynchronizationbetweenEncoderandNetwork::::::::::::: ComplexScheduling:::::::::::::::::::::::::::: ChoosingaBoundonPictureDimension:::::::::::::::: AdaptingtheEncodedVideoStreamtotheNetwork:::::::::: ReducingDecodingTime:::::::::::::::::::::::::: ControllingDimensionofEncodedPictures::::::::::::::::41 4.5CircuitSwitching::::::::::::::::::::::::::::::::::59 2

3 5TransmissionofCBRMPEGVideo 5.1Intra-frameCodingOnly:::::::::::::::::::::::::::::: CodingShapingDelay::::::::::::::::::::::::::: RateControlFunction::::::::::::::::::::::::::: DimensionofEncodedPictures:::::::::::::::::::::: StartupShapingDelay::::::::::::::::::::::::::: ShapingDelayImplementation:::::::::::::::::::::: ExperimentalData:::::::::::::::::::::::::::::::::81 5.2Intra-frameandPredictiveCoding:::::::::::::::::::::::: CodingShapingDelayatConstantTargetRate::::::::::::: ExperimentalData::::::::::::::::::::::::::::: BueringIssuesRelatedtotheCodingShapingDelay::::::::::75 5.6PacketSwitchingwithTimeDrivenPriority:::::::::::::::::::95 5.5CircuitSwitching::::::::::::::::::::::::::::::::::95 5.4DedicatedLinkbetweenSenderandReceiver::::::::::::::::::: CodingShapingDelayatConstantVisualQuality:::::::::::: StartupShapingDelay:::::::::::::::::::::::::::8 5.7AsynchronousPacketSwitching:::::::::::::::::::::::::: NetworkShapingDelay::::::::::::::::::::::::::: End-to-endDelay:::::::::::::::::::::::::::::: ExcessResynchronizationDelay::::::::::::::::::::::97 AListofAchronymsandSymbols 6Summary 5.7.2Multi-HopConguration:::::::::::::::::::::::::: Single-HopPacketNetwork::::::::::::::::::::::::11 BTheMPEGEncodingStandard 5.7.3PacketizationandStartupShapingDelay:::::::::::::::::15 B.2Intra-codedPictures::::::::::::::::::::::::::::::::112 B.1GeneralPrinciples::::::::::::::::::::::::::::::::::111 B.2.1DiscreteCosineTransformation:::::::::::::::::::::: B.3Predictive-codedPictures::::::::::::::::::::::::::::::116 B.2.4ControllingI-frameDimension:::::::::::::::::::::::115 B.2.3EntropyEncoding::::::::::::::::::::::::::::::114 B.2.2Quantization::::::::::::::::::::::::::::::::113 B.3.3DCT:::::::::::::::::::::::::::::::::::::117 B.3.1MotionEstimation:::::::::::::::::::::::::::::116 B.3.2MotionCompensation:::::::::::::::::::::::::::117 3

4 B.5PacketizationandStreaming::::::::::::::::::::::::::::119 B.4SignaltoNoiseRatio::::::::::::::::::::::::::::::::119 B.3.6ControllingP-frameDimension::::::::::::::::::::::118 B.3.5EntropyEncoding::::::::::::::::::::::::::::::118 B.3.4Quantization::::::::::::::::::::::::::::::::118 4

5 1Introduction TheMovingPictureExpertGroup(MPEG)encodingwasdesignedforstorageandone-way mainobjectivehasbeentoreducethecommunicationresourcesrequirementswhilemaintaininghighvideoquality. (playback)transmissionofqualityvideowhereend-to-enddelayisnotamajorconcern.the (i)forhearingtheend-to-end(sendertoreceiver)delayshouldbebelow1ms, (ii)thevideostreamshouldbesynchronizedwiththeaudiostream(a.k.a.lip-synchronization), videocommunications: avoidvariouspitfalls,asshowninthisreport. qualityofservice(qos)parametershasnotbeendone.suchastudyisneededinorderto UsingMPEGforvideoconferencingisnotnew.However,asystematicstudyofvarious (iii)themedia(audioandvideo)replayatthereceivingsideshouldbecontinuous(which impliesthatthereplaybuerisneveroverowedorunderowed). whichimpliesthatthevideoend-to-enddelayshouldbekeptbelow1msaswell,and Wedenethefollowingrequirementsforinteractive("face-to-face")real-timeaudioand thatthepersonontheothersideisnexttous.ifwehadveryhighquality3d(dimension) nexttous-thisisalsoknownastele-presenceorvirtualreality.sotheresultspresentedin video,thenwithoureyesopenwecouldthinkthatthepersonontheothersideisactually thisworkcanhavefurtherconsequencesonthedesignofvirtualrealitysystems. Themainconsequenceoftheaboverequirementsisthatwecan"closeoureyes"andthink qualityofeachpicture.weconcentrateontheminimizationoftheend-to-endprocessing eachofthemthusdecreasingtheend-to-enddelay.whilereducingtheend-to-enddelayit isimportanttomaintainthevideoqualitybyavoidingpacketlossandbymaintainingthe howitisinuencedbythetransmissionandnetworkqueueingdelay. delay,i.e.,weconsiderthedelayintroducedbythecombinationofencoding/decodingand Weidentifythecomponentsofthedelayandstudytheirnaturewiththegoalofreducing investigationintoseveralsteps.ineachstepweincreasethecomplexityofthesystemwe 1.1ScopeoftheWork Inordertohaveanin-depthunderstandingofthevideoconferencingproblemwedivideour analyze.bydoingsowecanisolateandunderstandthevariousdelaycomponents. types: 1.Dedicatedlink,whichallowsthevideostreamtousetheentirecapacityofthelink, Weconsidertwovariables:thenetworktypeandthevideotype.Therearefournetwork thanaveragevideostreamrate. possiblyinaburstymanner.thus,inthiscasethelinkcapacitycanbemuchhigher 5

6 2.Circuitswitching,whichallocatesafractionofthecapacitytoeachvideostream.In 3.Timedrivenpriority[15],whichgivesthereal-timetracperiodicpriorityoverother thiscasetheassumptionisthatthecircuitcapacityequalstheaveragevideostream rate. 4.Asynchronouspacketswitching,whichdoesnothavecommontimereferenceandeven ifpriorityisusedcongestioncanoccur.asaresult,thedelayjittercanbelargeand trac(bothreal-timeandnonreal-time).sincealltheswitchingelementsareusing packetlosscanoccur. timeornonreal-time)trac,i.e.,thereal-timetracisprotectedbya\re-wall",and thedelayjittercanbeboundedbyaconstant. network.asaresult,thereal-timetraccannotbeadverselyaectedbyother(real- acommontimereference,timedrivenpriorityfacilitatespipelineforwardinginsidethe eectofthevideoprocessingontheend-to-enddelay. 2.Variablebitrate(VBR)MPEG.Eachpictureisencodedandimmediatelysenttothe 1.Rawvideo.Inthiscasetheprocessinginminimal:picturesarecapturedandsent Wealsoinvestigatethreetypesofvideostreamsinordertobetterunderstandtheactual 3.Constantbitrate(CBR)MPEG.Thevideoencoderincludesabuerandratecontroller networkinterface.inthiscasethevideoencoderincludessmallornobuer. directlytothenetworkinterface. boththevideostreamandnetworktypes.theentriesoftable1showthestructureof thisworkbyidentifyingthesectioninwhichthecorrespondingcongurationisstudied.in Section6wesummarizetheobtainedresults. ThevariouscongurationsareshowninTable1.Eachcongurationischaracterizedby inordertoensurethatthebitrateprovidedtothenetworkinterfaceisconstant.inthis caseanencodedpicturemaybedelayedforseveralpictureperiodsbeforeitisforwarded tothenetworkinterface. 2TheModel Inthissectionweintroducethekeyelementsofthesystemdealtwithinthiswork.Wefocus Dealingwiththequalityofvideoandaudioencodingisbeyondthescopeofthiswork.In fact,wemodelthedelayasbeingend-to-end,suchthat,itincludesthejittercompensation areconcernwiththedelayandjitter,thesender-receiversynchronization,andpacketloss. andmediasynchronization.ingeneral,synchronizationisneededforcontinuousplayofaudio onqualityofservice(qos)issuesinadigitalvideoconferencingsystem.inparticularwe andvideostreamsatthereceivingside. 6

7 RawVideoSection3.1 Dedicated Link Section3.2 Switching Circuit TimeDriven Section3.3 Priority PacketSwitching Asynchronous VBRMPEGSection4.2 CBRMPEGSection5.4 (Section3) (Section4) Section4.5 Section5.5 Section4.3 Section5.6 Section3.4 Section RequirementsforInteraction (Section5) Table1:CongurationsConsideredinthisWork. Section5.7 Peopleparticipatinginavideoconferenceshouldfeellikebeingface-to-faceinthesamemeeting room.anecessaryrequirementinordertoachievethisgoalistokeeptheend-to-enddelay sinceitissubjecttophysicallimitation,suchas,thespeedoflight... quality,3d,etc..however,theend-to-enddelayis,inourview,themostessentialrequirement forinteractivereal-timeapplications.thedelayrequirementisalsotheleastobvioustosolve, thatthelimitbelowwhichthedelayisnotperceptibleis1ms[13]. (justthinkofhowannoyingarephonecallsroutedthroughsatellitelinks);theassumptionis belowhumanperception.thehumanearissensitivetothedelayresponseinaconversation Requirement1Videoandaudiostreamsaresynchronized. Ofcourse,thedelayrequirementisnotsucient,thereareotherrequirementslikepicture constraintsappliestovideoaswell.thissynchronizationisessentialsincepeopleareusedto Requirement2Theend-to-enddelayoftheaudioandvideostreamsshouldbekeptbelow seeevenbeforehearing(consider,forexample,aspectatorwhoisattendingaconcertbutis farfromthestage).wefocusontheend-to-enddelayintransmissionofvideobecauseitis thehardertokeepbelowthe1mstarget. Sincethevideostreamshouldbesynchronizedwiththeaudiothesameend-to-enddelay 1ms. 7

8 theaimofreducingthemandkeeptheend-to-enddelaybelowthe1msbound: 2.Network:timetakentomovedatacontainingtheaudioandvideoinformationfrom 1.Processing:itisneededtotransformtheaudioandvideosignalsinaformatsuitable Theend-to-enddelayhasthreemaincomponentsthatwewanttostudyseparatelywith 3.Resynchronization:digitalizationandplayingofbothaudioandvideorequirestricttime fortransmissiononadigitalnetwork.processingmayincludealsothecompressionand decompressionoftheaudioandvideosignals. thesourcetotheotherparticipant(s).italsoencompassesprotocolprocessinginboth senderandreceiver(s). Digitaltransmissionofvideorequiresavideocameracapturepicturesataxedpace(video 2.2RequirementsforVisualQuality addingfurtherdelay;thisisdescribedmoreindetailinthenextsection. andnetworkdelaysvariable,theyintroduceasynchronythathastobecompensatedby synchronizationbetweencaptureofthesignalandreplayingit.beingtheprocessing framerate)andaframegrabberboarddigitalizethem;thetimebetweentwosubsequent capturesiscalledvideoframeperiod.theproducedbitsaresenttothereceiver(s)where theyareconvertedbacktopicturesanddisplayedonamonitoratthesameregularpaceat whichtheyhavebeencaptured.ifthereceiverdoesnotdisplaypicturesatthissamevideo framerate,thereconstructedvideosceneresultscanbeannoying.thisisasortofquality degradationwhichisnotacceptableinthiswork,i.e., Requirement3Thereceivercontinuouslydisplaypicturesattheaverageratetheyhavebeen captured. delayofeachpicturemustbeconstant,i.e.,senderandreceivermustbesynchronized. variability(a.k.a.networkdelayjitter)byintroducinganetworkresynchronizationdelay. correspondingpicture.asshowninfigure1,thebuerusedtocompensatethenetwork Thisisachievedbypossiblybueringthereceivedbitsanddelayingthepresentationofthe jitterisusuallycalledreplaybuer.theoveralleectofthereplaybuer,isthateachpicture Inordertohavecontinuousplayingofthevideoframesonthereceiverside,theend-to-end takesthesametimetotransitfromthesender(pointainfigure1)totheoutputofthe buer(pointbinfigure1);theshorterthenetworkdelayexperienced,thelongerthetime Sincethenetworkdelaycomponentisvariable,thereceiverhastocompensateforthis inmovingfromthepointatothepointbinfigure1isxed.ifapictureexperiencesanetwork thepicturespendsinthereplaybuer(i.e.,thenetworkresynchronizationdelay). Afterthesystem(inparticularthereplaybuer)isdimensioned,thetimeapicturespends 8

9 End to end Delay becausetheresultingimagequalitydegradationispropagatedtothefollowingvideoframes. atoolargedelay,itcannotbeusedwhendisplayingapicture.thisisnotacceptable,especiallywhenthevideoiscompressedbyeliminatingthetemporalredundancybetweenpictures delaylargerthanthistime,requirement3isnotsatised.ifonlypartofapictureexperiences E A B D andthusitisnotacceptable.hence, Figure1:SynchronizationinVideoTransmission. Network Replay Buffer capture display Ifthisrequirementissatised,thereplaybuercanbedimensionedlargeenoughtocompensateforthenetworkjitterandguaranteesynchronizationmissionveryexpensive,compressionisperformedbeforetransmission.Thistakesavariable amountoftimeaswellastheirdecompression;theencoder/decodersystemintroducesapro- timeisclearlylargerthanthemaximumprocessingdelayexperiencedbyapicture. ofthetimeeachpicturespendsinblockeandinblockddepictedinfigure1isconstant;this cessingresynchronizationdelaythatcompensatesforthisvariability.inotherwords,thesum Sincetheamountofbitsobtainedbydigitalizingapictureisverylargeandtheirtrans- Requirement4Thenetworkdelayisbounded. time time se.infact,awiseresynchronizationmakestheend-to-enddelaybethesumofthemaximum belowaperceptiblebound,inthefollowingwedonotconsiderresynchronizationdelaysper processingdelayandthemaximumnetworkdelayexperiencedbyanypacket.thus,wefocus onthesetwomaximumvaluesinvariousdierentscenarios. tioncomponentsofthesystem.sincetheaimofthisworkiskeepingtheend-to-enddelay Knowledgeoftheresynchronizationdelayisthebasisfordimensioningtheresynchroniza- canbemergedwiththedecoderbuer.intheformercase,thecompensationcanbemore ecient(i.e.,thenetworkresynchronizationdelaybeminimum)becauseitcanexploittiming becompensatedeitherinthenetworkinterfaceorinthedecoderitself,i.e.,thereplaybuer 2.2.1BueringIssues Figure1depictsthereplaybuerasaseparatefunction.Actually,networkdelayvariationcan 9

10 informationaboutthenetwork.instead,mergingthetwobuersprovidemorerobustness End to end Delay cansometimescompensateeachother. againstmisdimensioningofthebuersbecausethevariationsofnetworkandprocessingdelay E D display,asshowninfigure2.buerscanbepresentinthenetworkinterface(e.g.,ifapacket Figure2:SynchronizationJustBeforeDisplaying. Network switchednetworkisusedincomingbitsmustbeassembledinpacketsbeforetheheaderis Inprinciple,synchronizationcanbeperformedbyasinglebuerinsertedjustbeforethe Buffer capture intheprocessingandnetworkdelays. processed)andinthedecodertostorearrivingdatauntilthereisenoughdatatobeprocessed andwhilethedecoderisbusy.anywaythisbuersarenotusedforcompensatingvariations time fordisplaying,andtaggedwiththedisplayingtime;whenthistimehascome,thepicture beforedecodingthepictures. 2.3TheSystem isdisplayed.thesynchronizationbuerislargerthatthebuersusedforsynchronization Encodedpicturesaredecodedassoonastheyarereceived,bueredintheformatsuitable Figure3.Notallofthemarepresentinanycongurationtakenintoconsiderationinthis Inourview,avideoconferencingsystemiscomposedofthefunctionalblocksdepictedin thevideocamera;thistimecanbeneglectedwithrespecttotheobjective1msend-to-end itintroduces.inthissectiononlycapturecardanddisplayarediscussed;inthefollowing Sectionstheotherblocksaredescribedastheyarerstintroduced. decoderfunctionalitiesarenotincludedinthesystem. work.forexample,whentransmissionofrawvideoisconsidered,boththeencoderand delaybound. Whenapictureisavailableonthereceiversideinaformatsuitablefordisplaying,itis Acapturecard,orframegrabber,takesaround3mstodigitalizeeachpicturecapturedby Inthisworkeachofthesefunctionalblocksisconsideredfromtheviewpointofthedelay insertedintothevideoframebuer.thevideoadaptorperiodicallyscansthevideoframe 1

11 Encoder Buffer Shaping & Frame Encoder Grabber Packetization buerandtracestheimageonthescreenaccordingtotheinformationstoredforeachpixel. Figure3:HighLevelModelofaVideoconferencingSystem. Network Network Decoder Adaptation Adaptor Theboundonthepresentationdelayis17msforRf=6Hz.Pd=isobtainedby Thedelayintroducedbythedisplaydependsonthevideorefreshingfrequency.Current synchronizingthevideoadaptorwithothercomponentsofthereceiver:thecomponentto monitorsworkatrefreshingfrequenciesrfbetween6hzand1hz,thusintroducinga Pd2";1Rf# Decoder Buffer besynchronizedwillbeidentiedforeachofthesystemcongurationspresentedthroughout Inordertobetteranalyzeeachcomponentoftheend-to-enddelayandshowinwhichcomponentofthesystemitisintroduced,anincrementalapproachisused.Westartfromthe delayisaected. 3TransmissionofRawVideo videoframebuerandwhenthebuerisscannedforrefreshingthedisplay. accordingtotheinstantaneoustimerelationshipbetweenwhenapictureisinsertedinthe thiswork.whennosynchronizationisprovided,thevalueofpdvariesinthegiveninterval Sections4and5weaddfunctionalblockstothecongurationandshowhowtheend-to-end congurationofthevideoconferencingsystemwhichrequiresthesmallestnumberoffunctionalblocksandhasthesmallestnumberofcomponentsintheend-to-enddelay.then,in 11

12 networkcongurations.althoughvideoisnottransmittedinarawformatduetoitshigh Inthisrstapproachweconsiderthetransmissionofrawvideooveranumberofdierent Figure4:RawVideooveraDedicatedLink Frame Network Network Video aredirectlyconnectedbyalinkhavingcapacityc.thefrbitsofapicturearemadeavailable Figure4showstheblockdiagramofavideoconferencingsysteminwhichsenderandreceiver bandwidthrequirements,itisinstructivetostudythiscase.sinceinthiscasewecanstudy 3.1DedicatedLinkBetweenSenderandReceiver inisolationthenetworkdelayandthepresentationdelay1. Grabber Interface Interface Adaptor dimensionthenumberofbitsencodingapicture. tothenetworkinterface,fromthecapturecard,inafewmilliseconds.thepictureisthen sentwithoutanyadditionalprocessingatthelinkspeed,asshowninfigure5.wecallpicture gatherthebitswhiletheyarebeingreceivedfromthenetwork;assoonasthelastbitofa eachbitexperiencesinthesenderandreceiverbuersanoveralldelaywhichisequaltothe theframegrabber,tothelinkspeedc.abuerisalsoneededonthereceiversideto transmissiondelayoffrbits(dimensionofarawpicture)overalinkofcapacityc,i.e.,fr=c: pictureisreceived,thepicturecanbemovedintothevideoframebuerfordisplay.thus, therstbitspendsthetimeonlyinthereceiverbuer,whilethelastoneonlyinthesender Abuerisneededonthesendersidetomatchtherateatwhichbitsareproducedby buer;intermediatebitsspendtimepartlyinthesenderbuerandpartlyinthereceiverone. wherefristhedimensionofarawpicture,pisthepropagationdelay,andpdisthepresentationdelaywhichdependsontherefreshingfrequencyofthemonitorandthesynchronization relationshipbetweencapturecardandvideoadaptor.pdisnullifthevideoadaptoronthe Thepropagationdelayexistsinallthecongurationspresentedinthiswork,eventhough Ded Raw=Fr C+P+Pd; Ifthetimeneededfordigitizingpicturesisneglected,theend-to-enddelayisgivenby receiverissynchronizedwiththecapturecardofthesender. insomeofthemitisconcealedinsidesomeothercomponentoftheend-to-enddelay.the propagationdelayisaphysicallowerboundontheend-to-enddelayofthesystem;ifsender andreceiver(s)aresofarawaysuchthatp>1ms2,theboundontheend-to-enddelay 1Thetimerequiredfordigitalizing(capture)apictureisconsideredrelativelysmall. 2Propagationdelayof1msinberisabout2,kilometersorhalfoftheEarthcircumference. 12

13 Bits T 2T 3T Time Rate cannotbesatised(i.e.,face-to-face-likeinteractionisnotpossible),independentlyofthe C congurationusedforthevideoconferencingsystem. thenumberofbitsusedtoencodeeachpixel.forexample,aqcif(quartercommonimage Format)picturehasadenitionof176x144pixels;assumingthateachpixelisencodedon Figure5:GenerationandTransmissionofRawPicturesoveraDedicatedLink. T 2T 3T Time pictureona1mb/slinkis1.98ms:evenconsideringtheworstcaseforthepresentation Frdependsontheresolutionofthepictures(i.e.,thenumberofpixelperpicture)and Production transmission delay,8mscanbespentinpropagationdelayoverthelink,whilekeepingtheend-to-end 8bits,Fr=198kb.InsteadaHighDenitionTeleVision(HDTV)videoframeisdigitalized using192x18pixels,i.e.,162kb.thus,forexample,thetransmissiondelayofaqcif capacity delayunderthe1msbound.instead,anhdtvpicturetakes162mstobetransmitted overa1mb/slink;thisisabovethe1msboundrequiredforinteractionandmoreover itdoesnotallowforreal-timevideo,asshowninthefollowing. Requirement5EachpictureistransmittedwithinthevideoframeperiodT, andthevideoframerate,thecapacityofthelinkmustbelargeenoughtosatisfy Real-timevideotransmissionrequiresthat i.e.,thetransmissiondelaymustbesmallerthant;thus,giventheresolutionofpictures CFr 13T: (1)

14 Bits T 2T 3T Time Forexample,theminimumlinkcapacityneededtotransmitvideosequencesatHDTV Figure6:GenerationandTransmissionofRawPicturesoveraSlowDedicatedLink. Rate C T 2T 3T Time propagationdelayislarge,thelowerboundonlinkspeedisnotgivenby(1),butbythebound capacitylinkisexploited,thegenerationandtransmissionofpicturesshowsthebehavior depictedinfigure6. mainparameterindeterminingtheend-to-enddelayisthelinkspeed.thehigherthepicture denitionandthevideoframerate,thefasterthelinks.ifthevideoframerateisloworthe denitionis486mb/ssincethehdtvframerateis3picturespersecond.whenaminimum Thus,inavideoconferencingsystemthattransmitsrawvideooveradedicatedlink,the Production ontheend-to-enddelay.thisleadstohighcostforthevideoconferencingsystem,especially videoconferenceisusuallyneeded). whenhighdenitionisrequiredandsenderandreceiverarefar(thatisthesituationinwhich oftrac;inthefollowingotherschemesarestudiedwithrespecttotheresultsshowninthis video.accordingtoothertransmissionschemes,linksaresharedamongvariouscallsortypes particularlyoverlongdistances;nevertheless,giventhespeedoflinks,thecongurationdescribedinthissectionprovidesalowerboundintheend-to-enddelayfortransmissionofraw Thus,dedicatingthewholecapacityofalinktoavideoconferencecallisnotcosteective, Highspeedlinksarenowavailablealsoonlongdistances,buttheyareveryexpensive. Section. 14

15 isassignedtoeachconnectionusingtimedivisionmultiplexing.thearchitectureofthe Incircuitswitchinglinkcapacityissharedamongdierentcalls.Axedamountofcapacity 3.2CircuitSwitching Figure7:ModelofaVideoconferencingSystemExploitingaCircuitSwitchedNetwork. wereconnectedbyadedicatedlink,eventhoughitsbandwidthisusuallysmallerthanthe videoconferencingsystemisdepictedinfigure7;senderandreceivercommunicateasthey physicalspeedofthelinksonwhichdataactuallyow. Frame Interface spondingfractionofthecapacityofeachlinkonthepathbetweensenderandreceiver.the AvideoconferencecallisallocatedacircuitwithbandwidthBbyreservingthecorre- Grabber Interface Adaptor bitsencodingeachpicturearesentthroughthecircuitatratebasshowninfigure8.this introducesanetworkshapingdelaygivenby: wherepisthepropagationdelayandswistheswitchingdelay.thisend-to-enddelayis (almost)constantbecauseb,p,andswarealmostconstant;thus,networkresynchronization Theend-to-enddelayisgivenby CS Raw=Sn+P+Sw+Pd; Sn=Fr isnotneededinthereceiver. AccordingtoRequirement5forvideoconferencingapplicationSnT,i.e., BFr (3) B,thelargertheamountofbandwidthiswastedbecausethecircuitisbusyonlyforatime SnineachvideoframeperiodT.DuringtheremainingtimeT?Sn,nootherconnectionor Ȯnonehand,thelargerB,thesmallertheendtoenddelay;ontheotherhand,thelarger classoftraccanexploitthisunusedfractionoflinkscapacity. Ifminimumbandwidthisallocatedtothevideoconferencecall,theend-to-enddelayis T largerthanthevideoframeperiod;thelowerthevideoframerate,thelargertheend-to-end 3Mb/s,buttheresultingshapingdelayis67ms.Nevertheless,ifmorebandwidthisallocated delay.forexample,theminimumbandwidthrequiredtosendqcifpicturesat15fps,is B (2)

16 Bits T 2T 3T Time switchingistosharelinkcapaityinaexibleandecientmanner.however,somepacket becausethecircuitisidleforhalfofthevideoframeperiod.onofthemotivationsforpacket SwitchedConnection. todecreasethenetworkshapingdelayto3ms,5%oftheallocatedbandwidthiswasted Figure8:AmountofBitsandRateinRawVideoEncodingandTransmissionoveraCircuit switchingschemesmayrequiretomaintainlowbandwidthutilizationwhentryingtosatisfy real-timetracrequirements. B T 2T 3T Time 3.3PacketSwitchingwithTimeDrivenPriority Production time:thehigherthelinkspeed,thelongertheidletime.inthissectionweshowhowthelink Figure5showsthatthetransmissionofavideoframecanrequireuseofthelinkforashort canbeusedfortransmissionofotherclassesoftrac(eitherrequiringguaranteesonservice qualityor"besteort")duringitsidleperiods BasicPrinciplesofTimeDrivenPriority Timedrivenpriority[15]isamultiplexingschemeaimedatsharinglinkcapacitywhileguaranteeingsourcesfromuncontrolleddelays(orevenlosses)duetocontentioninaccessinglinks. andapacketistransmittedduringasingletf. IneachTFaxedamountofbitsTfCcanbesentonalink:dataaregroupedinpackets Thetimeisdividedintimeframes(TFs)ofxeddurationTf(atypicalchoiceisTf=125s). 16

17 Rate Figure9:GenerationofRawPicturesandMultiplexingwithTimeDrivenPriority. T 2T 3T 4T 5T 6T Rate C sentinasingletf.forthesakeofsimplicity,inthisworkwedonotconsidertheprocessing Figure1:ModelofaVideoconferencingSystemExploitingaPacketSwitchedNetwork. Apictureisdividedintooneormorepartswhichareinsertedintopackets.Eachpacketis T f Frame Packetization Packetization connectedtointermediatesystems,whichinturnareinterconnectedinageneraltopology, introducedbythepacketheaders.inthetfsbetweenthetransmissionoftwosubsequent requiredforpacketizationineitherthesenderorthereceiverandthetransmissionoverhead transmitted(seefigure9),i.e.,tracmultiplexingisperformedonthelinks. picturesofthesamesession(andintheremainingtimeinthesametf),otherdatacanbe Senderandreceiverarenotdirectlyconnectedthroughadedicatedlink;instead,theyare Grabber Adaptor directlyconnected;thepacketheaderidentiesthereceiversothattheintermediatenodes asshowninfigure1.thesendersendsitspacketstotheintermediatenodetowhichitis canrouteittothedestination. mediatelyfollowingtheoneinwhichitwasreceived3.toguaranteethatitisalwayspossible, 3AnodecanhappentorequireanumberofTFstoprocessthepacket;thebestTFfortheforwardingis ThenetworkshowsthebestperformancewhenapacketisforwardedduringtheTFim- 17

18 BufferFigure11:RISC-likePacketForwarding. C 3 B Output 4 callisnottobeforwarded,theallocatedbitscanbeusedanywaytosendotherdata. othertfs,andeventhenon-reservedbitsinthereservedtfs,canbeusedtosenddatanot TFsarereservedonthelinksonthepathbetweensourceanddestinationandthesource transmitsdataduringthetfsreservedonitsoutgoinglink.atfisreservedforavideoconferencecallwhenafractionofthectfbitswhichcanbeusedduringitiskeptavailablefor forwardingapacket(belongingtothevideoconferencecall)receivedintheprevioustf.the belongingtothevideoconferencecall.moreover,ifapacketbelongingtothevideoconference AreservationofTFsallowsnodestoperformRISClikeforwarding[14]:packetsare A bueredinintermediatenodesforaxedtimeandeachpackettakesaxednumberoftfs Time tomovefromtheoutputbuerofanintermediatenodetotheoutputbuerofthefollowing tomoveapacketfromsourcetodestinationdependsonlyonthephysicaltopologyofthe oneonthepathtothedestination,asshowninfigure11.asaconsequencethetimeneeded pathandhasadelayvariationorjitterof2tf[15].thisnetworkjitterisxedanddoes notdependontracloadorthepathlengthfromsourcetodestination.asaconsequence, apacketswitchednetworkwithtimedrivenpriorityandresourcereservationcompileswith Requirement4becausenetworkdelayhasapredenedbound.Thebueringtimeineach destination:theend-to-enddelayinthevideoconferencingsystemis 3.3.2TransmissionofVideoandScheduling networkisnotsubjecttocongestionandtherelatedloss. Giventhepaththroughthenetwork,apackettakesatimeLTftotravelfromsourceto nodeisxedatreservationtime:keepingitsmallenoughtoavoidoverowingofbuers,the therstafterthisprocessingtime. 18

19 canbeeliminatedbysynchronizingthevideoadaptorandthereceivernetworkinterfaceso wherepd2[;1=rf]isthepresentationdelayandrfistherefreshingrateofthedisplay.pd TDP chronizationdelay,i.e.,noreplaybuerisneededonthereceiverside.ldependsonthe continuousplayingrequirement3.hence,thereisnoneedforintroducinganetworkresyn- thatthedisplayisrefreshedjustafterthetfinwhichthepictureisreceived. Thevariationofthenetworkdelayissmallenoughtobetolerated,notwithstandingthe Raw=LTf+Pd ordertohaveintermediatenodesperformingrisc-likeforwarding,thetfsonalinkmustbe numberoftfs:allthenodessharethesameknowledgeoftheordinalpositionofthecurrent TFinsidethetimecycle.Bandwidthisallocatedtoasender/receiverpair,byreservinga numberofhopsonthepath(itisatleastequaltothenumberofhops),theprocessingdelay propernumberoftfspertimecycleoneachlinkonthepathbetweensenderandreceiver.in insideeachnode,andthepropagationdelaybetweeneachpairofintermediatesystems. totransitfromtheoutputbueroftheupstreamnodetotheoutputbueroftheconsidered chosenaccordingtothetfsreservedontheupstreamlinkandthetimeneededforapacket Resourcereservationisbasedonthedenitionofatimecyclewhichencompassesaxed node. thetimecyclebeingequaltothevideoframeperiod;theobtainedresultscanbeextendedto thegeneralcasebytakingintotheconsiderationsmadeinsection3.3.3.atfinthetime cycleisreservedtothevideoconferencecall,asshowninthelowerpartoffigure12.the choiceofthetftobereservedwithinthetimecycleoneachinvolvedlinkiscalledscheduling mustbeanintegermultipleofthevideoframeperiod.forthesakeofsimplicityweconsider anditmustimpactsthemaximumnumberofreal-timeconnectionsconcurrentlysupported Sincerawvideohasanaturalpacingdrivenbythepicturerate,thetimecycleduration bythenetwork[14].inthefollowingtheimpactofschedulingontheend-to-enddelayofthe videoconferencecallispresented;anyway,asschedulingisoutofthescopeofthisworkand itwillnotbeaddressedindetailsinthiswork. thenetworkandtheupperpartshowsthetimingoftheframegrabberdigitalizingpictures.if theframegrabberisnotsynchronizedtothenetworkinterface,thebitsproducedbytheformer arebueredforatimesas ThelowerpartofFigure12depictstheTFsallocatedonthelinkconnectingthesenderto two,sas synchronizationcomponentofthenetworkshapingdelaysinceitisusedtoadaptthetiming delineatetfs,sas atwhichpicturesarecapturedissynchronizedtotheoneusedbythenetworkinterfaceto characteristicsoftheapplicationtotheonesofthenetwork.iftheclockthatdrivesthepace nslowlyincreasesordecreases;whensas nisconstantduringavideoconferencecall.ifthereisadriftbetweenthe n2[;t]whilewaitingtobetransmitted;thisistheapplication overwrittenbyanewone(whensas andwrapsaround,atfcanbeleftunused(whensas nchangesfromtto),andhenceitisnottransmitted. 19 nreachestheboundaryofitsvariationinterval nchangesfromtot)orapicture

20 Rate Theendtoenddelayisgivenby: Figure12:TransmissionwithTimeDrivenPriority. T 2T 3T 4T 5T 6T Time Rate C S t Time T f TDP Raw=SAS n+ltf+pd (4) EvenwithQCIFformatpictures,capacityoflinksmustbelargerthan1.5Gb/sinorderto CTf.Thiscanbesatisedwhenthelinksarefastenoughsothat Thegivenexpressionholdswhenapicturecanbeincludedinasinglepacket,i.e.,Fr CFr Time Cycle rawvideoframesusingpacketswitchingwithtimedrivenpriorityisgivenby: (e.g.,infigure13ns=2).thus,theend-to-enddelayinavideoconferencingsystemsending sendawholepictureduringasingletf(hdtvdenitionrequireslinkspeedlargerthan 13Gb/s.Wheresuchlinksarenotavailable,apicturecanbesentduringNsTFs,where TDP Raw=SAS n+(nr?1)tf+ltf+pd Ns=&Fr TfC' wherenrnsisthenumberoftfsbetweentherstandthelasttfreservedforthe transmissionofapicture.infact,itcanhappenthatnsconsecutiveavailabletfscannotbe 2

21 CFigure13:TransmissionwithTimeDrivenPriority. Rate T 2T 3T 4T 5T 6T Time reservedandtheend-to-enddelayisincreased. Rate ThecomponentLTfdependsonthetopologicalandphysicalcharacteristicsofthesystem. foundoneachlinkintheproperpositioninsidethetimecycle;thus,nonconsecutivetfsare WecallnetworkshapingdelaySn=SAS Theend-to-enddelaycanthenbewrittenas4 Time T f TDP Raw=Sn+LTf+Pd (5) Time Cycle generated,tonetworkandtransmissionschemeexploitedtotransmitthem.sndependson thedelayintroducedinordertoadapttherateatwhichthebitsencodingeachpictureare thelinkspeed,thedenitionofpictures,andthescheduling. (whencapturecardandnetworkinterfacearesynchronizedandanoptimalscheduleisfound) UsingQCIFformatat15picturespersecond,Ns=16andSncanvaryfrom1.8ms n+(nr?1)tf problemspresentedinsection3.3.3arise. to68.8ms.consideringascenarioinwhichthepathfromsendertoreceiverincludes3 hops(withnegligiblepropagationandprocessingdelay,i.e.,l=3),sas (C=1Mb/sandPisnegligible)theendtoenddelayis1.98ms,aswasshownin theend-to-enddelayis2.175ms.inthecongurationwithdirectlinkinsimilarconditions 4Ifthetimecycledoesnotcoincidewiththevideoframeperiodandanoptimalscheduleisnotfound,the 21 n=andpd=,

22 dedicatedlinkcapacityisleftunused. Section3.1.Thus,theend-to-enddelayiscomparableinthetwocases,but97%ofthe turecardandnetworkinterfacewillbemorelikelyandthesas Asequipmentforvideoconferencingapplicationswillevolve,synchronizationbetweencap- sub-optimalschedulecanbeacceptedprovidedthatitcompileswiththe1msend-to-end schedule.thus,iftheroutefromsendertoreceiverischaracterizedbyasmallvalueofl,a apictureissentimmediatelyafterithasbeendigitalized(providedthatanoptimalschedule theloweristhevideoframerate.eventhoughcapturecardandnetworkinterfacearesynchronized,alargenetworkshapingdelaycanbeduetotheimpossibilityofndinganoptimal doesexist).thelackofthissynchronizationcanhaveaworseimpactontheend-to-enddelay ncomponentwillbenullbecause InSection3.3.2isshownhowbandwidthisallocatedtoavideoconferencecallbyreserving ontfinatimecyclewhosedurationequalsthevideoframeperiodt.ifmorethanone 3.3.3MultipleVideoconferenceCallsontheSameNetwork callissubmitted;thisincreasestheprobabilityforthecalltobeblocked. delaybound.iflislarge,anoptimalscheduleisrequiredtothenetworkwhentheconference videoconferencingsystemisexploitingthesamenetworkandtheydonotusethesamevideo framerate,thetimecycleischosenastheminimumcommonmultiplierofallthevideoframe multiplemofitsownvideoframeperiodt.assumingthateachpictureissentwithina periodsinuse. singletf,bandwidthisallocatedbyreservingmtfswithinthetimecycle.ifthetfsare ist,everythinghasbeenstatedaboutthecasem=1,stillholds. evenlydistributedwithinthetimecycle,i.e.,thetimedistancebetweentwosubsequenttf Fromthepointofviewofeachofthevideoconferencecalls,thetimecycleisaninteger ofthepictureinterarrivaltime.ifthisvariationistoolargeaccordingtothecontinuous ofthenetworkdelay;thelessevenlydistributedarethetfs,thelargertheresultingvariation playingrequirement3,thereceivercompensatesitthroughareplaybuerandtheend-toenddelayisaccordinglyincreasedwithrespecttotheonegivenin(4).ifmorethanonetfis usedtosendeachpicture,thechoiceofthetfstobereservedimpactstheend-to-enddelay; IfthereservedTFsarenotevenlydistributed,theirrelativetimedistanceisnotTanymore andthereceiverdoesnotreceivepicturesataconstantpace.thisisequivalenttoavariation videoconferencingapplications.thetwosometimesimposeconictingoptimalitycriteriafor requiringabandwidthreservationtothenetwork,canaskforaschedulethatminimizesthe end-to-enddelayanddoesnotrequirearesynchronizationcapablereceiver,attheriskof schedulingandthusatrade-omustbefoundamongthem[1].avideoconferenceapplication gettingtherequestrefused.alternatively,ifthereceiverisequippedwithareplaybuer adetailedanalysisoftheyieldedend-to-enddelayisoutsidethescopeofthiswork. tocompensatethevariationofthenetworkdelayintroducedbyscheduling,itcanaccepta Ingeneral,schedulingisimportanttobothnetworkperformanceandend-to-enddelayof 22

23 schedulethatincreasestheend-to-enddelay. multiplesoneoftheother.thelargestvideoframeperiodcanbechosenastimecycleinthe keepmsmall,videoconferencingapplicationshouldchoosevideoframeratethatareinteger network. 3.4AsynchronousPacketSwitching ThelargerM,theharderisfortheschedulingtosatisfytheoptimalitycriteria.Inorderto Thevideoconferencingsystemcanbebuiltonatraditionalasynchronouspacketswitching networkwhichstatisticallymultiplexdataonthelinks.thefrbitsencodingapictureare Assumption1AboundQMonthequeueingdelaydoesexist. insertedinpacketsof(maximum)dimensionpswhicharesentintothenetworkatthefull linkspeed.inthenetworkpacketsexperiencexeddelayduetotransmission(fr=c)and Packetsarequeuedinnetworknodeswhiletheyarecontendingforabusylink;thus,the propagation(p),andvariabledelayduetoqueueing. describedinsection2.2.accordingtorequirement4,thenetworkdelaymustbebound,i.e., queueingdelaydependsontheloadinthenetworkanditdoesnothaveadeterministic Queueingintroducesanetworkjitterthatmustbecompensatedbyareplaybueras 3.4.1NetworkResynchronizationDelay queuedimension;forthispurposetracisshapedattheboundariesofthenetwork. bound.aboundcanbeguessedbuttohavealowprobabilityofbreakingit,itmustbelarge. ontheend-to-enddelay. Moreover,thequeueingdelaycanbekeptsmallerbyavoidingburstsofpacketsthatincrease GiventheboundQMonthequeueingdelay,eachpacketthathasexperiencedashorter queueingdelayisdelayedinthereplaybuer.thisisshowninfigure14:theupperdiagram InthisSectionweanalyzethecontributionofnetworkresynchronizationandtracshaping todeterminethequeueingdelayexperiencedbyapacket.thus,therstpacketreceivedfor showsthearrivaltimeofpicturestothereplaybuer,whilethelowershowstheexittime. Theresultingend-to-enddelayofpicturesis foratime thevideoconferencecallisbueredforatimethatallowsresynchronizationintheworstcase: itisassumedthatthepackethasexperiencedtheminimumqueueingdelayqmandisbuered Ifthenetworkinterfacesofsenderandreceiverarenotsynchronized,thelatterisnotable Async Raw=Fr C+P+QM+Pd: (6) 23

24 F F r Q r + P + Q + P + Q C M C M+ T Figure14:NetworkResynchronizationDelaytoCompensateQueueingDelayVariation. F Time F F r r + P + Q C m C + P + Q m + T F r r + P + Q C M+ T C + P + Q m + 2T Q Q M m F delay F r r + P + Q + P + Q C M C M+ T followingpacketsduetoqueueingandresynchronizationisbetweenqmandqm+q.ifthe packet,delaysitbyqaccordingto(7).asaconsequence,theoveralldelayexperiencedby Thefollowingpacketsareresynchronizedaccordingly,asshowninFigure14,becausetheyare exitingincasethereceiverdoesnotknowtheactualqueueingdelayexperiencedbytherst experiencesanactualqueueingdelayqm.themiddlediagramshowsthetimingofpackets retrievedfromthereplaybuerattheconstantpaceatwhichpicturesaredisplayed. TheupperdiagramofFigure15showsaresynchronizationexamplewhentherstpacket Time receiverknowstheactualdelayexperiencedbytherstpacket(e.g.,senderandreceiverhave acommontimereferenceandthepacketcontainsatimestampindicatingwhenitwassent), thelowerdiagraminfigure15. doesnotdelayitatallandtheexitingtimefromthereplaybueraretheonesdepictedin callanddependsonthequeueingdelayexperiencedbytherstpacketreceived,er=when whereer2[;q]istheexcessresynchronizationdelay;itisconstantoveravideoconference senderandreceivernetworkinterfacesaresynchronized. Insummary,theend-to-enddelayisgivenby Async Raw=Fr C+P+QM+Er+Pd 24 Q=QM?Qm (7)

25 Q F r C + P + QM F r C + P + Q M+ T F r C + P + Q m F r C + P + Q m + T F r C F r + P + Q M+ T C + P + Q m + 2T Figure15:NetworkResynchronizationDelaytoCompensateQueueingDelayVariation. Time F r C + P + Q M+ Q Q Q M m Q F r + P + Q C M F r C F r C + P + Q M+ T 25 + P + Q M+ T + Q Resynchronization delay Resynchronization delay Time Time

26 Figure16:ModelofaVideoconferencingSystemExploitingaPacketSwitchedNetworkwith TracShapingatNetworkBoundaries TracShapingatNetworkBoundaries ResourcesmustbeallocatedintothenetworkinordertoguaranteetheboundQMonthe Sincetransmissionoflargeamountsofdataatlinkspeed(bursts)causessuddengrowthof queuingdelay(andpossiblylimittheamountofdatalostinthenetworkduetocongestion). queues,largeamountofresourcesisreservedforburstysources:limitsqueuesgrowthby keepinglinkutilizationlow. Frame Packetization Packetization Grabber Adaptor widthbetweensenderandreceiverwhilekeepingtheburstinessbelowapredenedlevel. Assumption2Tracshapingisusedatthenetworkboundarytoprovideanaverageband- Atokenisremovedfromthebucketforeachdataunitsent.Weassumetoexploitabuered asaleakybucket[2],asshowninfigure16.theburstytracgeneratedbyasourceisshaped longasthenumberoftokensinthebucketislargerthanthepacketsdimension(indataunits). byaleakybucketintoaowhavingaverageratebandcontrolledburstiness.abucketis packetsarepresentedtothetracshaper,theyaresentintothenetworkatlinkspeed5as assumedtocontainatmostatokensanditislledattherateofbtokenspersecond.when Thesourceshapesthetracitisgeneratingbyusingatracshapingmechanismssuch leakybucketwhichismodeledasshowninfigure17.whenthebucketisempty,packetsmay delay.accordingtorequirement5,aswearedealingwithreal-timevideotransmission,the andtheresolution,thetraccharacteristicsofarawvideosequenceareknowninadvance, shapedtracattheexpensesofanetworkshapingdelay.since,giventhevideoframerate thebuercanbedimensionedproperlysothatitneveroverows. thebuckettosendthepacket.thisbueradaptsthesourceratetothecharacteristicsofthe bedroppedorbuered(alsoknownasbueredleakybucket)untilenoughtokensarepresentin timetakentorellthetokenbucketmustbesmallerthanthevideoframeperiod,i.e., Thechoiceoftheparametersofthetracshaperdeterminesitseectsontheend-to-end Suchatracshapercanbebuiltbycascadingtwosimpleleakybuckets. asitiscaptured,asshowninfigure18wherea=ps.tracshapingdoesnothaveanyeect 5Thereisavariantcalleddualleakybucketthatsendsburstofdataataratelowerthanthelinkspeed. IfAFrthewholepictureisinsertedinpacketswhicharesentatthelinkspeedassoon BFr 26 T: (8)

27 andtheaveragerateisfr=t,asdeterminedbythevideoframerate(1=t)anddimension (Fr),andtheend-to-enddelayisgivenby(6). Figure17:ModelofaLeakyBucket. B A frame): Fr?Atokens(since,thanksto(8)thetokenbucketisfullbeforestartingtransmittinga showninfigure19wherea=ps;afterwards,thetimeneededtogeneratepstokens(i.e., Ps=B)iswaited.Thus,thenetworkshapingdelaySnisgivenbythetimeneededtogenerate IfA<Fr,A=PspacketsofdimensionPsaresentassoonasthepictureisdigitalized,as Unshaped Shaped Traffic Sn=Fr?A Traffic siondelayofapacket(ps=c),theend-to-enddelayisgivenby: ThetimeneededtotransmitapicturebeingthenetworkshapingdelaySnplusthetransmis- wherepisthepropagationdelayonthelinksonthepathbetweensenderandreceiver,qm maximum,thenetworkshapingdelayisnotpresent.nevertheless,thelackoftracshaping isthemaximumqueueingdelay,ander2[;q]istheexcessresynchronizationdelay. atnetworkboundariesresultsislowerlinkutilizationorlargermaximumqueueingdelayqm. Comparing(6)and(1)themaindierenceisthatintheformer,wheretheburstinessis Async?Sh Raw=Sn+Ps C+P+QM+Er+Pd (1) queueingpolicyusedbynodes.forexample,ifnetworknodesareperformingweightedfair queueing,theboundisinverselyproportionaltotheallocatedbandwidth[3]: Themaximumqueueingdelaydeliveredbythenetworkdependsonthereservationand QM=o1B 27 B (9)

28 Bits Figure18:HighBurstinessTransmissionofRawVideooverPacketSwitchedNetworks. T 2T 3T Time Rate C B T 2T 3T Time Production Bits Figure19:LowBurstinessTransmissionofRawVideooverPacketSwitchedNetworks. T 2T 3T Time Rate A 28 A C B C B T 2T 3T Time A A Production B C

29 ofbitsneededtoencodeeachpicturebyexploitingspatialandpossiblytemporalredundancy Transmissionofrawvideorequireslargeamountofbandwidth,particularlyiftheend-to-end delayhastobekeptunderthe1msbound.thus,compressionisusedtoreducetheamount 4TransmissionofVBRMPEGVideo TheMotionPictureExpertGroup(MPEG)[11,25,8]videoencodingstandardwasdesignedfordigitalstorageofqualityvideoforlaterre-playing.Adigitalizeduncompressed 4.1MPEGOverview turesaresentoverthenetworktothereceiverwheretheyaredecodedanddisplay. presentinsidepicturesandbetweensubsequentpictures,respectively.thecompressedpic Intra-frameCoding digitalizedpicturesandencodeseachoftheminoneoftwodierentways6. (n=2)x(m=2)pixelsofchrominanceinformation.theencoderreceivesininputasequenceof imageiscomposedofonematrixofnxmpixelsofluminanceinformationandtwomatricesof luminanceandthetwochrominancecomponents)isdividedin8x8pixelblocks.oneach coecientsofthespatialfrequenciespresentintheblock. blockadiscretecosinetransform(dct)isperformed;ityieldsan8x8matrixwiththe Spatialredundancyinsidethepictureisexploitedtogaincompression.Thepicture(the neededtorepresentitthankstothefactthatmanycoecients(particularlythehighfrequency moresensitivetohighspatialfrequencies[26]:highfrequencycoecientsarequantizedmore Actually,adierentquantizationstepsizeisusedforeachcoecientbecausehumaneyeis coarselythanlowfrequencyones,buttheperceivedvisualqualitydoesnotdegrade.thus, eachcoecientismultipliedbythecorrespondingelementofan8x8quantizationmatrix. Eachcoecientisquantizedbyintegerdividingitbyanintegercalledquantizationstepsize. ones)arezero.finally,therun-lengthencodedsymbolsequenceishumanencoded.the 4.1.2PredictiveCoding pictureencodedinthisfashioniscalledi-frame. Theresultofthequantizationisrun-lengthencodedinordertominimizethenumberofbits withrespecttoi-frames.thepictureisdividedintomacroblocks(mbs),eachcomposedof Temporalredundancyiseliminatedtoreducethenumberofbitsneededtoencodethepicture, periodsthatweconsidernotacceptableiftheend-to-enddelaybekeptbelow1ms.thus,thiscompression techniqueisnotconsidered. becoded,areferencesubsequentpicturemustbecapturedandcoded.thisintroducesadelayofsomeframe 6Actually,athirdtypeofencoding,calledbidirectionalpredictivecoding,exists.Inorderforapictureto 29

30 thetwochrominancecomponents.givenambtoencode,motionestimationisperformed, a16x16pixelmatrixofluminanceinformationandthetwocorresponding8x8pixelblocksof timessmallerthani-frames.thesmallerthedierencebetweenthembbeingcodedand DCT,quantization,run-lengthencoding,andHumanencoding.Ifa\similar"MBisnot found,eachblockofthembisencodedlikeablockinani-frame. dierencebetweentheactualmbandthereferenceoneiscalculatedandcodedbyperforming i.e.,thepreviouspictureissearchedfora\similar"mb.ifthismbisfound,thepixelbypixel thereferenceone,thehigherthecompressiongainedthroughdctandrun-length/human periodtotheother. encoding.themoresimilartwosubsequentpictures,thehigherprobabilityofndingamb \similar"totheonebeingcodedandthusyieldingbettercompression;subsequentpictures aresimilarifthesceneisslowmovingbecauseifdoesnotchangemuchfromavideoframe PicturescodedinthisfashionarecalledP-framesandtheirdimensionisfrom2to4 AvideosequenceiscompressedbyencodingapictureoutofNasI-frame,andtheremaining 4.1.3BitGenerationRateandQuantization N?1asP-frames;thesequencestartingwithoneI-frameandterminatingwiththelast consecutivep-framesiscalledagroupofpictures(gop).thelargern,thesmallerthe overallamountofbitsusedtoencodeascene.nevertheless,ifthenetworkintroducesan toerrorsthestream. isencodedwithinthevideoframeperiod,butap-frameisfrom2to4timessmallerthan picturenotaectedbytheerrorandstopsitspropagation,i.e.,thelargernthemoresensitive errorinanencodedpicture,itpropagatesinthefollowingones;thenexti-frameistherst videosequenceswitharesolutionof72x48pixels;animagefromeachsequenceisshownin ani-frame.moreover,evenvideoframesofthesametypehavedierentdimension:the ofbitsproducedbythesoftwarempegencoderdvdenc[17]duringtheencodingoffour dimensionofani-framedependsonthespatialredundancypresentintheimage,whilethepframeonedependsalsoontheamountoftemporalredundancy.figure2showstheamount Therateofthebitstreamproducedbytheencoderhashighvariability:eachpicture (motion)redundancyineachscene. atallfrequenciesandcoarsequantizationoflowfrequenciesismorenoticeabletohumaneye Figure21.Theoverallnumberofbitsencodingeachsequenceandthewayitisdistributed frameorpredictive)beingperformedonit.intra-framecodedmbstypicallycontainenergy thanhighfrequencies.instead,predictivecodedmbshavehigherenergyathighfrequencies amongthepicturesaredierentduetothedierentamountofspatial(detail)andtemporal zones[26].thus,theactualquantizationstepsizeisobtainedbymultiplyingthequantization andhumaneyeislesssensitivetocoarsequantizationofpredictivecodedmbs. AdierentquantizationmatrixisusedforaMB,dependingonthetypeofcoding(intra- Moreover,humaneyeismoresensitivetoquantizationerrorinatareas,thanverydetailed 3

31 Cheerleaders Hockey 5 5 Picture dimension (Kb) Picture dimension (Kb) Bikes 5 Picture dimension (Kb) Figure2:NaturalDimensionofMPEGEncodedPictures. Table tennis Picture dimension (Kb)

32 Figure21:ImagesfromtheFourVideoSequencesUsedintheExperiments. 32

33 matrixbyamblocalactivityfactorthatchangesonambbymbbasiswiththegoalof bitsproduced.thewayinwhichthemblocalactivityfactoriscalculatedisdependenton theimplementationoftheencoderandisbasedontheestimationofthe\activity"levelof themb[9][26]. basis,providesoverallcontrolonpicturedimension.theproductofthemblocalactivity keepingconstantthevisualqualitythroughoutthepicturewhileminimizingthenumberof factorpmbandtheglobaldistortionlevelfactorggivesthequantizationparameterqmbfor MBmb,i.e., Moreover,aglobaldistortionlevelfactorthatpossiblychangesonapicturebypicture wherecmb Insummary,quantizationisperformedaccordingtothefollowingformula: i;jisthe(i;j)dctcoecientofmbmb,^cmb i;j=cmb Qmb=pmbG qfijpgi;j i;jqmb thereisasmalldierencebetweenthedimensionofi-framesandp-frames.infact,ononeside, keptconstantoverthewholevideosequence.thedimensionyieldedforthepicturesisplotted infigure22:thelargerg,thelargerthepictures.forsmallvaluesofg(nerquantization), mb,andqfijpg accordingtothetypeofencodingbeingperformedonthecurrentmb. FoursequenceswereencodedusingfourvaluesfortheglobaldistortionlevelG,whichwas i;jiselement(i;j)ofthequantizationmatrixusedfori-frameorp-framecoding, i;jisitsquantizedvalue,qmbistheformb arenotreducedtozerobythenequantization.ontheothersidethehighfrequencydct P-framesdimensiongrowsbecausethesmallDCTcoecientsofdierentiallyencodedMBs coecientsofintra-framecodedmbsarecoarselyquantizedbythecorrespondingelements ofthequantizationmatrix.thisisconrmedbytheplotsinfigure23whichshowthatfor frames.figure24plotstheaveragequantizationparameterusedoneachpicture;eventhough constant.thevariationislargerforlargervaluesofgbecausegampliesbmbvariations. G=1theSignaltoNoiseRatio(SNR)duetotheencoding7issmallerforI-framesthanforP- itchangesduetothecharacteristicsofthepictures(i.e.,becausebmbchanges),itisalmost qualitybecauseitdoesnottakeintoaccountthecharacteristicsofthehumanvisualsystem.thus,even thoughcoarsequantizationofhighfrequencycomponentsintroducesanerror,itdoesnotnecessarilydegrade theperceivedvisualquality. mathematicalmeasureofthequalityofapicture,butitdoesnotexactlycorrespondtotheperceivedvisual intoaccount. VBRMPEGcompression;theimpactofdierenttransmissionschemesonthedelayistaken 7TheSNRiscalculatedaccordingtotheformulareportedinSectionB.4.TheSNRisusedtoprovidea Inthefollowingwestudytheend-to-enddelayinavideoconferencingsystemthatuses 33

34 3 Cheerleaders 3 Hockey Picture Dimension (Kb) Picture Dimension (Kb) Picture Dimension (Kb) G = 1 G = 1 G = 15 Bikes G = 2 Table Tennis 3 Figure22:DimensionofPicturesCodedaccordingtoMPEGStandard number number Picture Dimension (Kb)

35 Cheerleaders Hockey 5 5 SNR (db) SNR (db) G = 1 G = 1 G = 15 Bikes G = Table Tennis SNR (db) Figure23:SNRofVBRMPEGEncodedVideoSequences SNR (db)

36 Average Quantization Parameter Cheerleaders Average Quantization Parameter G = 1 G = 1 G = 15 Bikes G = 2 Table Tennis 2 2 Figure24:AverageQuantizationParameterUsedforVBREncodingaccordingtoMPEG Standard Average Quantization Parameter Average Quantization Parameter Hockey

37 Figure25:VideoconferencingSystemCongurationwithDedicatedLinkbetweenSenderand Receiver. Frame Network Network Encoder Decoder Grabber Interface Interface Adaptor 4.2DedicatedLinkBetweenSenderandReceiver Processed Encoder Output link,asshowninfigure25.theencoderprocessespicturesandproducesthebitsencoding Minimumdelaycanbeobtainedifsenderandreceiveraredirectlyconnectedbyadedicated Bits themasi-framesorasp-frameswiththetimingshowninfigure26:thetimetakentoencode Figure26:BitProductionofaNaturalVBRMPEGEncoder. apictureisnotconstantaswellastheamountofbitsproduced. suitableforthevideoadaptor. bitscanbesentoverthededicatedlinkassoonastheyareproduced.afterapropagation delayp,bitsgettothedecoderwhichdecodesthestreamgeneratingpicturesintheformat Inprinciple,ifthelinkspeedishigherthanthenaturalinstantaneousrateoftheencoder8, (I) T 1 (P) 2T 2 (P) 3T 3 (P) 4T 4 (I) 5T Time eventhoughitismorepredictableanditsvariationsaresmaller.thedecoderkeepsconstant oftheencodinganddecodingtimeandintroducestheprocessingresynchronizationdelay. playing)bybueringtheencodedordecodedpictures.thiscompensatesforthevariability thedelaysynchronizationbetweentheinputoftheencoderanditsownoutput(continuous Thetimeneededtoencodeapictureisnotconstant;alsodecodingtimeisnotconstant insteadofd. bitsitisreceiving.forthepurposeofthisworkweconsiderconstant9thedecodingdelayd. availabilityofspecialpurposehardware,andonhowearlythedecoderstartsprocessingthe 8ThisislikelybecausethepeakrateoftheencoderisoftheorderoffewtensofMb/s. 9Ifthedecodingdelayisnotconstantthedecodercompensatesitsvariationsand(11)holdswithmaxD Thedecodingtimestronglydependsonthedecoderimplementation:i.e.,bothonthe 37

38 wherecmisthemaximumcodingdelay;whenaframetakeslessthancmtobeencoded,it dealingwithreal-timevideo Theend-to-enddelayisgivenby Ded Requirement6Apictureshouldbeencodedwithinthevideoframeperiod, isdelayedbytheremainingtimeinthedecoderbuer. Thefastertheencoderanddecoder,thesmallertheend-to-enddelay;infact,sinceweare VBR=CM+P+D+Pd (11) tothehighestframeratetheencodersupports.thus,ifasceneiscapturedat3frames i.e.,cmt.themaximumencodingdelayisusuallythevideoframeperiodcorresponding persecondandthepropagationdelaybetweensenderandreceiveris2ms(e.g.,theyare Theyieldedcompressionislowerandalargernumberofbitsisneededtoencodeavideo connectedthroughatransoceaniclink),theend-to-enddelayisaround6ms,i.e.,acceptable toguaranteeinteractivitybetweentheparticipantsinavideoconference. consequentlytheend-to-enddelay)canbereducedbyusingonlyintra-encoding(i-frames). sequence. to-enddelayobtainedinavideoconferencingsystemexploitingmpegcompression.never- theless,itisnotecientgiventheexpensivenessofbandwidth:infact,thecapacityofthe ThesystemcongurationdiscussedinthisSectionprovidesalowerboundontheend- Motionestimationisthemosttimeconsumingpartoftheencodingprocess:EM(and howtheend-to-enddelayisaectedbytheusageofmultiplexingschemestomoreeciently exploitbandwidthoflinks. linkconnectingsenderandreceiverisonlypartiallyused.inthefollowingsectionswestudy 4.3PacketSwitchingwithTimeDrivenPriority wherelisthenumberoftfsapackettakestotravelfromsendertoreceiverandpdisthe ingresslink,asshowninfigure28.theend-to-enddelayofthesystemisgivenby producedbytheencoder,theyareinsertedintoapacketsandsentatthefullspeedofthe encodedpictures,asshowninfigure27.assoonasallthebitsencodingapictureare Apacketswitchednetworkwithtimedrivenprioritycanbeexploitedfortransmissionof presentationdelay.forthesakeofsimplicity,inthisworkweconsiderthattheprocessing requiredforpacketizationdoesnotintroducedelay.expression(12)istheactualend-to-end delayonlyifthenodesonthepathfromsendertoreceiverareabletoperformrisc-like forwardingofpackets(seesection3.3.1).38 TDP VBR=CM+LTf+D+Pd

39 SenderandReceiver. Figure27:VideoconferencingSystemCongurationwithPacketSwitchedNetworkbetween Frame Packetization Packetization Grabber Encoder Decoder Adaptor Bits Rate Figure28:TimeDrivenPrioritywithoutReservation. T 2T 3T 4T 5T 6T T f 39 Time Time

40 mustbeallocatedinthenetworkandvideoframessentduringreservedtfs.toreserve mensionofthepicturebeingsent.aswasshowninfigure2,thenumberofbitsencodinga resourcesinpacketswitchednetworkswithtimedrivenpriority,theamountofdatatobe capacitycanbereservedduringthepropertfs. sentandtheirtimingmustbeknownatreservationtimesothattheproperfractionoflink ToguaranteethexedboundLTfonthenetworkdelayandlossfreedelivery,resources tioncannotbeaccurate.ifduringatfausersendsmorebitsthantheamountreservedto picturedependsonthetypeofcodingusedforthecorrespondingpicture,andthespatialand possiblytemporalredundancypresentintheimage.sincetheamountofredundancydepends onthescenebeingencoded,picturedimensionisnotknowninadvanceandresourcereserva- it,thenetworkdoesnotprovideanyguaranteeonthedeliveryoftheexcessbits.if,onthe Theamountofbitsreservedshouldbelargerthan(andascloseaspossibleto)thedi- otherhand,thevideoconferencingapplicationusesonlyafractionofthereservedcapacity, theleftoverbandwidthcanbeusedbybesteorttracandisnotwasted.eventhoughthis isacceptablefromthenetworkpointofview,thesolutionisnotoptimalfortheuserthatis forbothtypesofpicturesgiventhebandwidthtobeallocatedtoavideoconferencecall.then, possiblypayingfortheallocatedbandwidthandwouldliketouseitallbyhimself. theend-to-enddelay.eventhoughsomecongurationscandeliverunacceptableend-to-end andnetworkinterface,andthescheduling(i.e.,thechoiceofthetfstobereserved)impact weshowhowtheencodingprocesscanbetunedinordertocontrolpicturedimensionso thatthevideoconferencingsystemneverusesmorebandwidththantheamountallocatedand exploitsasmuchofitaspossible.lastly,weanalyzehowthesynchronizationbetweenencoder InthefollowingofthisSectionweshowhowtodeterminetheamountofbitstobereserved delay,ifthesystemisadequatelyequippedandoperated,itsperformanceisactuallygiven by(12) ChoosingaBoundonPictureDimension andp-frames.theseamountsdeterminethebandwidthreservedtothevideoconferencecall AsexplainedinSection4.1.2,theslowerthemotioninthescenebeingencoded,thelargerthe as slowmovingscenes,weproposetoreservedierentamountsofbitsfortransmissionofi-frames dimensionofi-frameswithrespecttop-frames.sincevideoconferencesareexpectedtobe isthenumberofpicturespergop,andntisthegopduration. anaturalmpegencodedependsontheamountofmotioninthescene.thepictureratio wherefiandfparetheamountofbitsreservedfori-framesandp-frames,respectively,n AsdiscussedinSection4.1.3,therelativedimensionofI-framesandP-framesyieldedby Btdp=FI+(N?1)FP (13) 4

41 encodedandtransmitted.thisisingeneraladiculttask,butintheparticularcaseof mustthenbechosenwiselydependingtotheamountofmotionexpectedinthescenetobe Figure29:MPEGEncoderforControllingDimensionofFrames. Base Bounded Encoder Picture videoconferencingapplication,scenesarelikelytobeslow. beexpressedasfunctionofthebandwidthtobeallocatedas CombiningEquations(13)and(14),theamountofbitstobereservedtoeachframecan =FI Quantization I frame parameter Rate Bound (FP=BNT P frame Bound 4.3.2ControllingDimensionofEncodedPictures Thereservedbandwidthisusedmoreecientlyiftheencodingprocessiscontrolledandthe numberofbitsencodingeachpictureiskeptbelowtheamountofbitsreservedinthetf duringwhichthepictureissent.thisamountsareupperboundpicturedimensionandcan N+?1 mostsuitedtothispurpose.forexample,figure22showsthatthedimensionofbothi-frames encoderisextendedwitharatecontrolfunction.ittunestheparametersofthebasicmpeg codingprocesssothatthedimensionofeachpictureisbestttedtothetargetassociatedto beusedtodeviseatargetdimensionforeachtypeofvideoframe.asshowninfigure29,the Nevertheless,Gisnotsuitableforourpurposeofcontrollingpicturedimensionbecauseits itstype. andp-framesisstronglydependentonthevalueoftheglobaldistortionlevelg(whichisoneof thecomponentsofthequantizationparameter):thelarger,thesmallertheencodedpictures. valuemustbechosenbeforestartingtheencodingofthepicture.thereexistproposalsto predicttheamountofbitsanencodingprocessisgoingtoproducebasedontherawscene[22] AmongtheparametersoftheMPEGencodingprocessthequantizationparameteristhe orontheportionofstreamalreadyproduced[12].however,togetserviceguaranteesfrom thenetwork,thetargetmustneverbeexceededandanadaptiveapproachbasedonfeedback fromtheoutputoftheencodermustbeexploited. 41 FP (14) FI=FP (15)

42 produced.theratecontrolalgorithmalsoproposeshowtore-quantizethepictureifthe Thiscanleadtocodingtimesnotacceptableforreal-timeencodingasrequiredinvideoconferencingapplications(Requirement6).[5]proposestoexploitarate-quantizationmodelto yieldeddimensionisnotcompliantwiththetarget. chooseg.themodelistunedaccordingtothecharacteristicsoftheencodedstreamalready Otherauthorsproposeiterativeapproaches[28,16]todetermineasuitablevalueofG. parameterforeachmbas bmbcanbechangedonambbymbbasisinordertoadaptivelytunethequantization Alternatively,theMBlevelratecontrolfactorbmbisusedforcalculatingthequantization quality.manyapproacheshavebeenproposedintheliteratureforuniformlychoosingthe 17].Varyingthegranularityofquantizationthroughoutthepicturedeliversnonuniformvisual accordingtothetargetandthefeedbackobtainedbymonitoringthestreambeingproduced[9, Qmb=Gpmbbmb Thus,wehaveperformedsomeexperimentstoprovethatpicturedimensioncanbecontrolled ratecontrolfunctionwhichimplementsthealgorithmshowninfigure3. visualsystem[26]. astheproposednetworkrequires.thesoftwareencoderdvdenchasbeenexploited;ithasa quantizationparameter[1]possiblytakingintoaccountthecharacteristicsofthehuman Theforegoingapproacheshavebeenproposedandanalyzedinscenariosdierentfromours. (lack)ofbitsandthetoleranceonthetarget.theaverageofthemblevelcontrolfactor andthetotalnumberofmbsmb.theincrement(decrement)isproportionaltotheexcess exceeds(isbelow)theexpectednumberofbits.thisiscalculatedaccordingtothetargetft isincremented(decremented)withrespecttobmb?1iftheamountofbitsfproducedsofar overthewholepictureisusedtocalculatetheglobaldistortionlevelforthenextpictureof Ftandatoleranceonit(intermsofmaximumandminimumacceptabledimensions).bmb ThefunctionwhichcalculatestheMBlevelratecontrolfactorbmbisprovidedwithatarget ofyieldinguniformvisualqualitythroughoutpictures.nevertheless,theobjectivehereisto nottodeviseanoptimalmethodtocontrolthedimensionofframes. thesametype.theexperimentaldatashowninfigures3136areobtainedusingg=15for showthatthedimensionofframescanbekeptbelow(andverycloseto)apredenedbound, boththersti-frameandtherstp-frame.thisalgorithmisnotwisefromthepointofview pictureratios(oneineachrowofthefigures). theshownbounds(fminthealgorithminfigure3):theboundsareneverexceeded.the boundsaredeterminedaccordingtoequation(15)assumingthreebandwidthamountsand twocomplementaryreasons: AsshownbyFigures33and34,theaveragequantizationparameterhashighvariationfor Figures31and32plotthepicturedimensionobtainedencodingfourdierentsceneswith 42

43 accordingtothetypeofthepicturetobeencoded,define: F=numberofbitsencodingMB1withquantizationparameterQ1=p1GfIjPgb1 formb=2tototalnumberofmbsmb b1=1 choosegfijpgaccordingtothetype(i-frameorp-frame)ofcoding amaximumdimensionfm(reservednumberofbits) atargetdimensionft(e.g.,5%belowreservednumberofbits) aminimumdimensionfm(e.g.,1%belowreservednumberofbits) elseiff<ft iff>ft elsebmb=decr(f;mb;fm;ft;mb) iff?ft bmb=maxb Mbmbthen Mbmb>FM?Ftthen Qmb=pmbGfIjPgbmb elsebmb=bmb?1 elsebmb=incr(f;mb;fm;ft;mb) ifft Mbmb?F>Ft?Fmthen bmb=minb Mbmbthen GfIjPg=GfIjPg F=F+numberofbitsencodingMBmbwithQmb MbPMb Figure3:VideoFrameDimensionControlAlgorithm. mb=1bmbaccordingtothetypeoftheencodedpicture 43

44 18 Cheerleaders - Picture ratio 2 18 Hockey - Picture ratio 2 Picture dimension (Kb) Picture dimension (Kb) Picture dimension (Kb) Mb/s 8 Mb/s Cheerleaders - Picture ratio 3 16 Mb/s 18 Picture dimension (Kb) Hockey - Picture ratio 3 Picture dimension (Kb) I-frame bound Cheerleaders - Picture ratio 4 P-frame bound Hockey - Picture ratio 4 18 Figure31:DimensionofPicturesinControlledMPEGEncoding number number Picture dimension (Kb)

45 18 Bikes - Picture ratio 2 18 Table Tennis - Picture ratio 2 Picture dimension (Kb) Picture dimension (Kb) Picture dimension (Kb) Mb/s 8 Mb/s Bikes - Picture ratio 3 16 Mb/s 18 Picture dimension (Kb) Table Tennis - Picture ratio 3 Picture dimension (Kb) I-frame bound Bikes - Picture ratio 4 P-frame bound Table Tennis - Picture ratio 4 18 Figure32:DimensionofPicturesinControlledMPEGEncoding number number Picture dimension (Kb)

46 1.Thevariationofpicturedimensionislimitedbytheparametersprovidedtotherate controlfunctionintheexperiments:thetarget(ftinthealgorithmshowninfigure3) SNRplottedinFigures35and36.Theratefunctioncontrolisnotdesignedtokeepthe 2.Theratecontrolfunctiondoesnothavetheobjectiveofdeliveringuniformquality Asaresultthequalityofpicturesisnotuniformthroughouteachscene,asshownbythe (Fm)1%smallerthanthetarget. hasbeenset5%belowtheupperbound(fm)andthelowerboundofpicturedimension qualityuniforminsidepicturesaswell.asaconsequence,themblevelratecontrolfactor hashighvariabilitywithineachframe. Thescenesusedintheseexperimentshavepicturesmoredetailedandahigheramount throughoutthescene. functionwoulddelivermoreuniformqualitywhendealingwithwithtypicalvideoconferencing ofmotionthantypicalvideoconferencingheadandshoulderscenes.thesamesimplecontrol choiceofthetfsinwhichthebitsarereservedaectstheend-to-enddelay;theconditions underwhichitcanbereducedtoexpression(12)arealsopresented. scenes.moreover,enhancingtheratecontrolfunctiontoreducethequalityvariationinthis scenarioisnothard SynchronizationbetweenEncoderandNetwork allocatedbyreservingxedamountofbitsforthetransmissionofpicturesofthesametype. TheamountofbitsiscalculatedusingEquation(15).Inthefollowingweanalyzehowthe Havingtheratecontrolfunctionboundingpicturedimension,bandwidthiseciently pictureeverytseconds. videoconferencingsystemsmustexploitreal-timeencodersthatguaranteethedeliveryofa byconsideringthenatureoftheapplicationgeneratingtrac.accordingtorequirement6, savescheduling,i.e.,thechoiceofthetfstobereservedtoavideoconferencecall,issimplied considerm=1keepinginmindthatthewhatisdescribedinsection3.3.3appliestothe generalcase.ineachtimecycle,fibitsareallocatedduringonetf. siderthatonlyintra-framecodingisperformed.underthisassumption,resourcereservation mustbeanintegermultiplemofthevideoframeperiod;forthesakeofsimplicity,wenow isperformedasdescribedinsection3.3fortransmissionofrawvideo.thetimecycleduration ThebitsencodingapicturearebueredfromwhentheyareproducedtowhentheTF Tobetterunderstandtheimplicationsofschedulingontheend-to-enddelay,werstcon- codingtimecm,eachpicturespendsinthepacketizationfunctionbuertheapplication AsexplainedinSection3.3.2,iftheclockdrivingthecapturecard(andhencethepaceatwhich synchronizationcomponentofthenetworkshapingdelaysas packetizationfunction.temporarilyassumingthatencodingeachpicturetakesthemaximum reservedfortheirtransmissionisscheduled.thisbueringcantakeplace,forexample,inthe 46 n2[;t],asshowninfigure37.

47 Cheerleaders - Picture ratio 2 Hockey - Picture ratio 2 Average Quantization Parameter Average Quantization Parameter Mb/s 8 Mb/s 16 Mb/s Cheerleaders - Picture ratio Hockey - Picture ratio 3 Average Quantization Parameter Average Quantization Parameter Cheerleaders - Picture ratio 4 Average Quantization Parameter Figure33:AverageQuantizationParameterinControlledMPEGEncoding. 47 Hockey - Picture ratio Average Quantization Parameter

48 Bikes - Picture ratio 2 Table Tennis - Picture ratio 2 Average Quantization Parameter Average Quantization Parameter Mb/s 8 Mb/s 16 Mb/s Bikes - Picture ratio Table Tennis - Picture ratio 3 Average Quantization Parameter Average Quantization Parameter Bikes - Picture ratio 4 Average Quantization Parameter Figure34:AverageQuantizationParameterinControlledMPEGEncoding. 48 Table Tennis - Picture ratio Average Quantization Parameter

49 5 Cheerleaders - Picture ratio 2 5 Hockey - Picture ratio SNR (db) 4 35 SNR (db) Mb/s 8 Mb/s 16 Mb/s Cheerleaders - Picture ratio Hockey - Picture ratio SNR (db) 4 35 SNR (db) SNR (db) Figure35:SNRinControlledMPEGEncoding. Cheerleaders - Picture ratio SNR (db) Hockey - Picture ratio

50 5 Bikes - Picture ratio 2 5 Table Tennis - Picture ratio SNR (db) 4 35 SNR (db) Mb/s 8 Mb/s 16 Mb/s Bikes - Picture ratio Table Tennis - Picture ratio SNR (db) 4 35 SNR (db) SNR (db) Figure36:SNRinControlledMPEGEncoding. Bikes - Picture ratio SNR (db) Table Tennis - Picture ratio

51 Rate encodedpicturesexittheencoder)isnotsynchronizedwiththenetworkinterface,sas T 2T 3T 4T 5T 6T Rate Otherwise,theend-to-enddelayisgivenby variesinthegiveninterval.ifthetwocapturecardandnetworkinterfacearesynchronized, nisconstantandthetimeinwhichpicturesarecapturedcanbechosensothatsas Figure37:I-frameOnlyCodingandBandwidthReservationwithTimeDrivenPriority C S M n nslowly T sentassoonasencoded,itwouldbedelayedbythedecodertocompensatetheprocessing functionlongerthansas IftheencodingtimeofapictureislessthanCM,itisbueredbythepacketization n.thisdoesnotaecttheend-to-enddelaybecauseifthepicturewas f TDP VBR?I=CM+SAS n+ltf+d+pd n=. (16) Cycle delayvariation.thus,thiscongurationallowstosimplifythereceiverthatdoesnotneedto TFsintendedforsendingI-framesandthoseforP-frames.Thetimecyclemustbesettoan forthetwodierenttypesofframes:dierentamountsofbitsshouldbereservedinthe Whenbothintra-frameandpredictivecodingareexploited,twodierentboundsareimposed 4.3.4ComplexScheduling compensatenetworkjitterandprocessingdelayvariation. integermultiplemofthegopperiodandnmtfsmustbereservedwithinthetimecycle. Figure38showsasamplereservationwithM=1andN=4;theupperdiagramshowsthe amountofbitsgeneratedtoencodeeachpictureassumingthattheyareinstantlyavailable 51

52 Rate afterthemaximumencodingdelaycmfromthecapture1.thelowerdiagramshowsthe Figure38:TimeDrivenPriorityandComplexScheduling. T 2T 3T 4T 5T 6T Time Rate C AS delayisgivenby(16)asinthecaseinwhichonlyintra-framecodingisexploited.nevertheless, choiceofthetfstobereservedoneachlinkonthepathbetweensenderandreceiveriscalled amountofbitsreservedinthetfs:fiinonetfandfpinthefollowingn?1tfs.the M Sn Time T dierentvariationinterval;form=1,sas ofthevideoconferencecall. inthiscasetheapplicationsynchronizationcomponentofthenetworkshapingdelayhasa complexscheduling.thechoiceofthetfsimpactsbothnetworkperformance(intermsof maximumnumberofreal-timeconnectionsconcurrentlysupported)andtheend-to-enddelay AssuminguniformdistributionwithinthetimecycleofthereservedTFs,theend-to-end f Cycle where,duetotheslowmotioninthevideosequence,itisconvenienttousealonggop(large ofthegopperiodisdenitelynotacceptableespeciallyinvideoconferencingapplications N).Thus,synchronizationbetweenvideoencoderandnetworkinterfaceisessentialtokeep theend-to-enddelaybelowthe1msboundandthebuerofthepacketizationfunction small. n2[;nt].anetworkshapingdelayoftheorder alimitofthesystem,butinsteadallowsthedecodertobesimplied. Assumption3Thecapturecard(aswellastheencoder)issynchronizedwiththenetwork interface. 1Ithasbeenalreadyshowninthecaseofintra-frameonlycodingthatthisassumptiondoesnotconstitute IftheTFsreservedtoavideoconferencecallarenotuniformlydistributedwithinthetime 52

53 Bits I frames Bound P frames Bound cycletheinterarrivaltimebetweenvideoframesisnotconstant.thereceivercompensateto thisvariabilitybybueringvideoframesreceivedbeforescheduleanddelayingthewholevideo T 2T 3T 4T 5T 6T Time stream.sincethisdelayisduetotheadaptationofthetransmissiontimetothecharacteristics Rate ofthenetwork,wecategorizethisdelayasaschedulingcomponentofthenetworkshaping delaysschedfigure39:timedrivenpriorityandcomplexscheduling. AS S wheressched byawiseschedulinginthenetwork. n.ifassumption3holds,theend-to-enddelaycanbewrittenas TDP?CxSc n VBR Time T ThedecodingtimeDcanbereducedifthedecoderdoesnothavetowaitthewholepicture 4.3.5ReducingDecodingTime n2[;nt]:itisxedwhenthevideoconferencecallisplacedandiskeptsmall f =CM+SSched n+ltf+d+pd ofthem(figure39),thantoallocateasingletfduringwhichawholeencodedpictureis beforestartingprocessingit.thiscanbeobtainedbyinsertingencodedvideoframesin sent(figure38).choosingthistfsencompassestwonontrivialissues:determiningtheir smallerpacketsthataresentassoonasthecorrespondingbitshavebeenproducedbythe outlined,butasolutionisnotproposedbecauseitisoutofthescopeofthiswork. Inthefollowingtheseproblemsaredescribedandthetrade-osbetweendierentchoicesare positioninsidethetimecycleandtheirnumbersothattheend-to-enddelaybeminimized. encoder.thisdoesnotaectanyothercomponentoftheend-to-enddelaygivenby(16). ItisthusbettertoallocatemorethanoneTFperpictureandsendapacketineach 53

54 forthepictureoverallthetfs,theremainingreservedtfswillnothaveenoughcapacity tocarrytheotherbitsthathastobesentinthescheduledtf.ontheotherhand,ifthe bitsactuallyproducedtoencodethepictureisveryclosetotheglobalamountofbitsreserved underutilized(itcanbeexploitedbybesteorttrac),butbecause,ifthetotalamountof bytheencoder,otherwisetheamountoftransmitteddataissmallerthantheonereserved. Thisiscalledencoderunderowanditcriticalnotjustbecausetheallocatedbandwidthis WhenareservedTFisscheduledonthesenderlink,enoughbitsmusthavebeenproduced TFsreservedforapicturearechosenlater(withrespecttothebeginningofthevideoframe period),thebenetofusingmoretfstotransmitapictureisreduced,i.e.,theend-to-end delayisincreased. imagecomplexityandmotion,thanthescenesusedintheseexperiments,thusshowingamore andfigure41whenencodingframeswithboundeddimension.thecurvesshowthatthe productionrateisquiteconstant,i.e.,thepositionoftfscanbeestimatedwithasimple linearmodelwhichintroducessomesafetymargin(byshiftingthetfstowardstheendof probabilityofencoderunderow.moreover,avideoconferencescenehasusuallymoreuniform thevideoframeperiod).figures4and41showthatasafetymarginoffewmsyieldslow ThesoftwareencoderthathasbeenusedinthisworkshowsthetimingplottedinFigure4 regularproductionrate. bythemodel,thebitsthatcannotbetinthetfsreservedforthepicturecanbesentas besteort;ifthenetworkiscongested,theydonottimelygettothereceiverandthevisual thehigherthenumberofreservedtfs,thehighertheprobabilityofencoderunderow(the qualityofthereconstructedscenetemporarilydegrades. thesmallerthetransmissionunit,theearlierthedecodercanstartdecoding.nevertheless, Iftheinstantaneousamountofbitsproducedbytheencoderisbelowtheamountpredicted thedimensionofthetransmissionunit. modelaveragestherealproductionrateonashorterinterval).moreover,eventhoughinthis workpacketheaderoverheadisbeingneglected,ithastobetakenintoaccountwhenchoosing ThenumberofthereservedTFsperpictureshouldbekeptashighaspossiblebecause 4.4AsynchronousPacketSwitching Avideoconferencingsystemcanbeconstructedoveranasynchronouspacketswitchednetwork. andthetransmissionoverhead,ontheotherhand. thereductioninthedecodingdelay,ononehand,andthepossibilityofencoderunderow TheamountofbitstobereservedineachTFmustbechosenndingatrade-obetween AssoonastheencoderproducesenoughbitstoassembleapacketofdimensionPs,thepacket issentintothenetworkwhereitisexperienceavariablequeueingdelay.asshowninfigure42, thereceivermustexploitareplaybuertocompensatethenetworkdelayvariation.theend toenddelayisthusgivenby 54

55 Number of bits Number of bits Number of bits Cheerleaders Video frame priod Cheerleaders Video frame priod Cheerleaders Video frame priod Cheerleaders Figure4:ProductionofBitswhenEncodingthe\Cheerleaders"Sequence Video frame priod Video frame priod Number of bits Number of bits

56 Number of bits Number of bits Number of bits Hockey Video frame priod Hockey Video frame priod Hockey Video frame priod Hockey Number of bits Number of bits Figure41:ProductionofBitswhenEncodingthe\Hockey"Sequence Video frame priod Video frame priod

57 SenderandReceiver.Async Figure42:VideoconferencingSystemCongurationwithPacketSwitchedNetworkbetween wherecmisthemaximumtimerequiredtoencodeapicture,qmisthemaximumqueueing Disthedecodingdelay.Usingsmallpacketsandadecoderwhichstartsdecodingvideoframes Frame Packetization Packetization assoonasdataarereceived,canreduced.anyway,theend-to-enddelayisdominatedby delay,er2[;q]istheexcessresynchronizationdelayintroducedbythereplaybuer,and VBR=CM+Ps C+P+QM+Er+D+Pd Grabber Encoder Decoder Adaptor boundariesofthenetwork,eventhoughitintroducesanetworkshapingdelay.resources insection3.4.2,resourcereservationismoreecientiftracshapingisperformedatthe Resourcesarereservedinthenetworkinordertoboundthequeueingdelay.Asdiscussed 4.4.1TracShapingatNetworkBoundaries CMandQM. thedescriptiongivenatresourcereservationtime. averagerate,correspondingtotheshapedtrac;thenetworkguaranteesthequalityonthe arereservedinthenetworkbasedonthetracdescription,givenintermsofburstinessand networkshapingdelaysts characteristicsoftheshapedtrac.thereceiverhastocompensateitbymeansofareplay service(i.e.,theboundqmonthequeueingdelay)onlyiftheactualtraciscompliantwith buerwhichintroducesanetworkresynchronizationdelay.thus,eachpacketexperiencesa naturalbitgenerationrateoftheencoder,theimplementationofthetracshaper,andthe Thedelaygloballyexperiencedbyapictureduetothetracshaperdependsonthe caseofvbrmpegencodedvideothesametaskisharderanditisnotthegoalofthiswork, wedonotanalyzests isgivenby andtheimpactontheend-to-enddelayinthecaseoftransmissionofrawvideo.sinceinthe Section3.4.2,wehavediscussedtheconstraintsonthechoiceoftheleakybucketparameters ninmoredetail.theend-to-enddelayofthevideoconferencingsystem npartlyinthetracshaperandpartlyinthereplaybuer.in Async?TS VBR=CM+STS n+ps 57 C+P+QM+Er+D+Pd

58 4.4.2AdaptingtheEncodedVideoStreamtotheNetwork andtokenpoolsizeadeterminetheburstinessandaveragerateoftheshapedtrac.ifthe SincethetracpatterngeneratedbyanaturalVBRMPEGencoderisnotknowninadvance, discardedbyeitherthetracshaperitself,oratracpolicingfunctioninsidethenetwork[21]. itcanbeincompatiblewiththeshapedtracdescription.thenoncompliantpacketscanbe Forexample,ifaleakybucketisexploitedtoshapethetrac,thetokengenerationrateB whenthegopislarge.eventhoughtechniqueshavebeenproposedtolimittheeectof theexcesstracmustbeeitherdiscardedorsentinthenetworkwith"besteort"service. characteristicsoftheencodedvideoarenotcompatiblewiththevalueschosenforbanda Eventhoughabuerisinsertedbeforetheleakybuckettoadaptthevideostreamtothe loss[6],itshouldbebetterforthevideoconferencingsystemtoavoidlossinordertodeliver tracdescription,itcanoverowifthetwoaretoodierent. thehighestpossiblequality. ThelossofpacketsisnotacceptableinthetransmissionofMPEGencodedvideo,especially tracdescription. controlfunctiontunestheparametersofthebasicmpegencoderaccordingtothetrac MPEGencoders,thisapproachdoesnotguaranteeagainstpacketsnotcompliantwiththe sendsasbesteorttracpacketsthatcannotbeadaptedtothetracdescription.arate descriptionusedtodrivethetracshaper[24,23].duetotheunpredictableoutputof Alternatively,theratecontrolfunctioncantunetheMPEGencoderparametersbasedon TheMPEGencodingprocesscanbecontrolledtoavoidthatthetracshaperdiscardsor leakybucket)[2,4],asshowninfigure43.ifthisrequirestosignicantlydegradethevisual feedbackinformationreceivedfromthetracshaper(e.g.,thefullnessofthebuerprecedinga qualityofpictures,theresourceallocationisrenegotiatedaccordingtoarate-quantization modelthatistunedastheencodingprogresses. reconstructedonthereceiverwithacceptablequality.theencodedstreamcanbepartitioned twoseparatelayers,thehighprioritylayer-whichcontainsthemostcriticaldataaccording complytothetracdescriptionbecauseitneedsserviceguarantees,whilethelatterissent streamtotheserviceprovidedbythenetwork.thebitsencodingeachframearedividedinto tothesensitivityofthehumanvisualsystem-andthelowprioritylayer.theformermust asbesteort.eventhoughthelowprioritycomponentgetslost,thevideosequencecanbe Hierarchicalorlayeredencoding[19]providesanothermeanstoadapttheencodedvideo correspondingstreambettertothetracdescription. dynamicallyaccordingtofeedbackinformationfromthetracshaper(e.g.,thefullnessof thetokenbucket)[18].thefractionofdatasentatthehighprioritylayertobettertthe availablebandwidthwhenthenetworkisnotoverloaded,andgettingreasonablevisualquality whenitiscongested.onthecountrary,theapproachbasedoncontrollingtherateofthe MPEGencodervariesthequalityofimageswiththeobjectiveofttingtheencodedvideo Hierarchicalencodingcangetthebestfromapacketswitchednetworkbyusingallthe 58

59 B A Target Base Rate Encoder streamputsaharmfulburdenonthenetworkwhenitisoverloaded.duringcongestion, Figure43:MPEGEncoderControlledusingFeedbackfromaTracShaper. Quantization Rate nation.evenworse,lowprioritypacketswasteresourcesuntiltheyarediscarded.thewaste Parameter isevenlargerifpacketsmaketheirwaytothedestinationandtheretheyarediscardeddue thoughthenetworkcouldpossiblysupportahighrate(i.e.,higherquality). streamintothetracdescription.thus,visulalqualitycansometimesbedecreasedeven sendinglowprioritypaketsdoesnotyieldanyadvantagesincetheydonotgettothedesti- toexcessivedelay11. Nevertheless,thehierarchicalencodingapproachisnotecientbecausethelowpriority Model of a System Target Decoder Buffer AcircuitswitchednetworkcanbeusedforthetransmissionofVBRMPEGencodedvideo. 4.5CircuitSwitching whereswistheswitchingdelayandpthepropagationdelayinthenetwork.thisexpression issimilarto(11)giveninsection4.2fortheend-to-enddelayofavideoconferencingsystem Ifthevideoconferencecallisallocatedacircuitwithbandwidthlargerthantheinstantaneous rateoftheencoderandtheend-to-enddelayis thecapacitycoflinksisdedicatedtothevideoconferencecall. thatusesvbrmpegencodingandadirectlinkbetweensenderandreceiver.whena circuitswitchednetworkisexploitedtheswitchingdelayhastobeadded;moreover,notall CS VBR=CM+Sw+P+D+Pd; 11Takeintoaccountthatthedelayexperiencedbybesteorttrachasalargeboundandhighvariability. 59

60 showninfigure44.itsmoothestherateoftheencodertotitinthecircuitbandwidthatthe Figure44:VideoconferencingSystemTransmittingVBRMPEGvideooveraCircuitSwitched expensesofanetworkshapingdelayscs belowthepeakrateofthempegstreamifabuerisusedbetweenencoderandnetwork,as Network. Section4.4.1aboutthenetworkshapingdelaySTS encoder,alargepartofitsbandwidthiswasted.thebandwidthofthecircuitcanbereduced packetswitching,appliesalsotoscs SincethecircuithasbeendimensionedtosupporttheburstinessontheVBRMPEG Frame Network Interface andthecircuitswitchednetworkisequivalenttotheoneduetoatracshaperwithaverage n.infact,theshapingperformedbythesmoothingbuer naddedtotheend-to-enddelay.whatisdiscussedin nintroducedbyatracshaperwhenusing Grabber Encoder Network Interface Decoder Adaptor bandwidthbandmaximumburstiness1bit(e.g.,aleakybucketwithtokengenerationrate totheend-to-enddelaywhichisgivenby receiversidecompensatesthevariationofthistimethusyieldingthexedcontributionscs ontherateofthempegencoderandthebandwidthofthecircuit.areplaybueronthe BandtokenpoolsizeA=1).Picturesarebueredinthesenderforatimethatdepends aleakybucket,whichhasbeendescribedinsection3.4.2;whenitoverowspartoftheencoded streamisdiscarded.dimensioningthebuerandthecircuitbandwidthsothatdiscarding Thebuerbetweenencoderandnetworkhasthesamepurposeasthebuerattheingressof CS?TS VBR=CM+SCS n+p+sw+d+pd n ofpacketsisavoidedisnotpossiblebecausethebitrateproducedbythempegencoderis notknowninadvance.thus,thebitgenerationrateoftheencodermustbemonitoredand regulatedbyaratecontrolfunctionwhichexploitsthebuerstatusasfeedbackinformation. isactuallyacbrmpegencoderwhichisthesubjectofthenextsection. ThesystemcomposedbythebasicMPEGencoder,thebuer,andtheratecontrolfunction 5TransmissionofCBRMPEGVideo ThebasicMPEGcodingprocessproducesavariablebitratestream.Ashasbeenshownin theprevioussection,topreventloss,thenaturalvariabilityofthebitgenerationratemustbe controlledandadaptedtothetransmissionservice.ifthetransmissionservicerequiresdata tobesentatconstantbitrate(cbr),acbrmpegencoderisthechoiceforbuildingthe videoconferencingsystem. 6

61 Figure45:BasicMPEGEncoderandEncoderBuer. Rate Rate B Encoder Buffer alsopredictivecodingareconsiderednext.lastly,thetransmissionwithdierentschemesis viewofdelaywhenonlyintra-framecodingisused;theimplicationsonthedelaywhenusing thatinuencethem.first,thempegencoding/decodingsystemisanalyzedfromthepointof Inthefollowing,thetimeneededtoencodeandtransmitCBRMPEGvideoisstudiedfor furtherstepswiththeaimofhighlightingthedierentcomponentsandthesystemparameters transmissionschemeadoptedandwidelyvariesaccordingtotheparametersoftheencoder. ACBRMPEGencoder/decodersystemintroducesadelaythatisindependentofthe Time Time analyzedinordertodevisetheend-to-enddelayinvariouscongurations. Base Encoder iscomposedonlyofi-frames)isrstconsidered;basically,thisisthecompressionprocess Thecaseinwhichonlyspatialredundancyiseliminated(i.e.,theencodedvideosequence 5.1Intra-frameCodingOnly ACBRMPEGencoderisbuiltbysmoothingthenaturalrateofabasicMPEGencoderinto aconstanttargetrateb.asshowninfigure45,thesmoothingisperformedbytheencoder 5.1.1CodingShapingDelay alsoapplytoavideoconferencingsystemwhichexploitjpegtocompressvideoframes. recommendedbymovingjpeg[7][25]andthusthebasicprinciplesdiscussedinthissection bueredinthedecoderbueruntilawholepictureisreceivedandcanbedecode.apicture thatvideoframesarecontinuouslydisplayedataxedrate.thecompositionoftheprocessing buerdelayseachbitdependingonthenaturalbitrateandthetargetbitrate.also,bitsare ThisistheprocessingdelayoftheCBRencoderthatdependsonthepicturedimension.Since itnotconstant,thedecodermustcompensateittocomplywithrequirement3whichimpose encodedwithfbitstakesatimef=btoexittheencoderbuerandenterthedecoderone. buer,whichislledatthenaturalbitrateandemptiedatthetargetbitrateb.theencoder delayoftheencoder/decodersystemandtheprocessingresynchronizationdelayintroduced bythedecoderisaconstantcodingshapingdelaysc. 61

62 Natural Buffer Target Base wheremaxseqfisthemaximumnumberofbitsusedtoencodeapictureovertheallsequence. Figure46:BasicEncoderandRateControlFunction. (17)guaranteesthatinthetimebetweenwhenapictureisgrabbed,towhenitisscheduled Quantization fordisplaying,uptomaxseqfbitscanexittheencoderbueratrateb.inorderforthe SincebitsexittheencoderbueratrateB, Parameter encodedvideostreamtobecontinuous(i.e.,actuallyhavingaconstantrate), ScmaxseqF B; Model a System Target Decoder Buffer areproduced.(18)showsthattheend-to-enddelayofavideoconferencingsystemexploiting otherwisetherewouldbeatimeintervalbetweentwosubsequentframesduringwhichnobits acbrmpegencoderislargerthanthevideoframeperiod. itisanalogoustothenetworkshapingdelaysts toavbrmpegencoder(seesection4.4.1).ifabasicmpegencoderfreelyllsanite dimensionbuerwhichisemptiedaccordingtoaxedtracdescription,thebuercan Eventhoughthecodingshapingdelayisnotrelatedtothetransmissionoveranetwork, undergoestoaverystricttracdescription(beingitconstant).thus,aratecontrolfunction possiblyoverow.insection4.4.2wesurveyedsomeproposalstocontrolthebasicmpeg todiscardpackets.inthecaseofacbrencoder,therateatwhichthebuerisemptied encodingbasedonfeedforwardorfeedbackinformation,inordertoavoidatracshaper nintroducedbyatracshapercascaded mustbeintroducedtocontrolthebasicmpegencoderinordertoavoidtheencoderbuer tochangethenaturalbitrateandavoidthebuertounderoworoverow. Figure46showsthecomponentsofaCBRMPEGencoder.Inprincipletheratecontrol functionmonitorsthefullnessoftheencoderbuerandadjuststhequantizationparameter tooveroworunderow RateControlFunction 62 ScT; (18)

63 asystemtargetdecoderwiththestreamdoesnotoveroworunderowitsbuer,whose dimensionisincludedinthempegstream[11].thempegstandardspeciesthatthe systemtargetdecoderisfedattheconstantrateindicatedinthestreamanditretrievesdata fromitsbueroneaccessunitatatime.theaccessunitisawholepictureandallthebits timespeciedinthempegstream.thebuerofthesystemtargetdecoderunderowsifa encodingavideoframeareremovedwhenitsdecodingisstartedaccordingtothedecoding ThecomplianceofanencodedvideostreamwiththeMPEGstandardrequiresthatfeeding wholepictureisnotpresentinthebuerwhenitsdecodingisscheduled. timetoencodeapictureisassumedtobenull,thefullnesslevelsofthetwobuersarestrictly updatedmodelofthebuerofasystemtargetdecoder,asshowninfigure46.itislledat sothatthestreamcomplianceisgranted. asshowninfigure47.thequantizationparameter(andthusthenaturalbitrate)istuned thetargetrateb,andemptiedbyonepictureattheinstantinwhichdecodingisscheduled, Actually,iftheencoderbuerhasthesamesizeofthesystemtargetdecoderoneandthe Thus,besidemonitoringtheencoderbuer,theratecontrolfunctionshouldkeepan tothedimensionofpicturesandthetargetrateb.ifthevirtualbuerdoesnotunder- MPEGstandardprovidesguidelinesonhowtomanagethevirtualbuerandmodifythe oworunderows,neithertheencoderbuernorthesystemtargetdecoderbuerdo.the decoderoneunderows(overows). related.inparticulartheencoderbueroverows(underows)ifandonlyifthesystemtarget quantizationparameter[9];anyway,eachcbrmpegencoderimplementationcanuseits ownapproach[28,26,5,23]. Inpractice,theratecontrolfunctioncanmodelvirtualbuerwhichisupdatedaccording thesoftwareencoderdvdencwiththeobjectiveofdevisingitsimpactontheend-to-enddelay. accordingto(17)themaximumnumberofbitsusedtoencodeapictureprovidesalowerbound thedimensionofthesystemtargetdecoderbuerandisdrainedattheconstantbitrate onthecodingshapingdelay,weanalyzeinmoredetailthevirtualbuerimplementationof B.Beforeencodingapicture,theratecontrolfunctionsetsthemaximumandminimum Thisthevirtualbuerconstrainsthenumberofbitsusedtoencodeeachpicture.Since thatthebuerisnotemptiedbeforetheendofthevideoframeperiod.thevbvisthen updatedaccordingtotheactualnumberofbitsproduced.inthefollowingsection,thelower dimensionsofthepicture.themaximumdimensionissetbyconservativelyassumingthat itmustnotexceedthespaceavailableinthebuer12.theminimumdimensionguarantees Theratecontrolfunctionmodelsabuer,calledVideoBuerVerier(VBV),thathas wouldnotoverow. andupperboundsonthedimensionofpicturesinancbrmpegstreamaregiveninterms ofthesizeofthevbv. 12ThisconformstoconservativeassumptionthatevenifallthebitswereinjectedintheVBVinstantly,it 63

64 Figure47:StandardDecoderBuerModelKeptbytheControlFunction. Rate Rate Rate ofimageswouldnotbeuniform.insteadpicturedimensionisallowedtovaryandiscontrolled B accordingtothevbv. encodeeachpicture.eventhoughthiswouldguaranteeaconstantbitrate,thevisualquality InthevideoframeperiodT,aCBRMPEGencoderworkingatthetargetrateBproduces BTbits.TheratecontrolfunctioncanbedesignedinordertouseexactlyBTbitsto 5.1.3DimensionofEncodedPictures Time Encoder Buffer Time Buffer T Time Claim1ThedimensionoftheVBVistheupperboundonthedimensionofpicturesinan Base Encoder wherevsisthedimensionofthevbvandfisthedimensionofapicture. MPEGcompliantstream: ProofTheVBVmustbelargerthanthelargestencodedpicturebecauseitmustcontaina wholepicturebeforeitisstartedbeingdecoded.moreover,asthevbvoverowmust beavoidedtoguaranteecompliancewiththempegstandard,theratecontrolfunction Vsmax seqf showhowthislowerboundisdetermined,werstanalyzehowpicturedimensionaectsthe thisisnotarestrictionbecauseifthevbvdoesnotunderow,theencoderbuernever encoderbuerlevel.weassumethattheencoderbuerandthevbvasthesamedimension; ThelowerboundonencodedpicturedimensionisalsorelatedtotheVBVdimension.To controlsthebasicncodersothatthenumberofbitsusedtoencodeapictureisnotarger containsmorebitsthanthevbvdimension. thanthevbvdimension. BTbits.Ifsomebacklogispresentinthebueratthebeginningofthevideoframeperiod, ratebduringthetimetbetweentwopictures,apictureshouldbeencodedwithatleast thenumberofbitsusedforencodingthepicturecanbesmaller.thus,thelowerboundfm Inordertohaveenoughbitsintheencoderbuersothatitisemptiedattheconstant 64

65 Bits T 2T 3T Time largestbacklog. exempliedinfigure48.theupperdiagramshowsthattheencoderprocessesthebitsofeach rawimageduringasinglevideoframeperiod(thindashedline).thediagramalsoplotsthe Figure48:BitProductionRateandEncoderBuerFullness. T 2T 3T Time line);thetotalnumberofbitsperpicturevariesfordierentpictures.thecbrencoder onpicturedimensionistheminimumallowedpicturedimensionwhenthebuercontainsthe numberofbitsgeneratedbythebasicencoderversustimeforeachpicture(thickcontinuous Processed emitsbitsatconstantrateb(asshownbythethickdashedlineinthelowerdiagram),thus Thebacklogisduetobitsencodingthepreviouspicturethathasnotbeensentoutyet,as Natural Target sendingoutthesameamountofbits(bt)ineachvideoframeperiod(lastpointofthe Buffer Level thickdashedlineintheupperdiagram).ifthedimensionofapictureexceedsbt(e.g.,the rstpictureinfigure48),excessbitsaretemporarilybueredintheencoderbuer(whose levelisdepictedbythethincontinuousline)untiltheyhavebeenallsentout(aftertheend ofthevideoframeperiod). encodingapicturetheratecontrolfunctionchecksthevbvandallowsthepicturetobelarge enoughtollit.ifthemaximumallowedamountofbitsisproduced,thelevelofthevbv (andtheencoderbuer)attheendofthevideoframeperiodis ThelargestbacklogVbremainsinthebuerafterapicturehaslledit.Beforestarting Vb=Vs?BT 65

66 becauseduringthevideoframeperiodbtbitshavebeendrainedfromthebuer.the numberofbitsthatmustbeinsertedinthevbvsothatitcontainsatleastbtbits,i.e., ratecontrolfunctiondeterminestheminimumalloweddimensionofthenextpictureasthe theamountofbitstobedrainedduringthevideoframeperiod: Thisisthelowerboundonthepicturedimension. apicture.moreover,theminimumnumberofbitsusabletoencodeapictureisbounded bythevbvdimension,thevideoframerate,andthetargetrateoftheencoder.sincethe naturaldimensionofmpegencodedpicturesishighlyvariable(seesection4.1.3,themore Insummary,thedimensionoftheVBVisthemaximumnumberofbitsusabletoencode Fm=BT?Vb=2BT?Vs thedimensionofpicturesisallowedtovary,themoreuniformistheirvisualquality.the maximumpossiblevariationofpicturedimensionisgivenby isfundamentaltokeepthevisualqualityuniformthroughoutpicturesandscenes.thetask ismademoredicultbythefactthat,underassumption3ofcontinuousplayingmadein function.theratecontrolfunctionmustusewiselytheallowedvariabilityinpicturedimension pictures. ThelargertheVBVdimension,thelargerthepossiblevariationofdimensionofencoded Theactualnumberofbitsusedtoencodeeachpictureisdeterminedbytheratecontrol FM=Vs?Fm=2(Vs?BT) (19) picturemustbesmallerthanbt. Claim2Ifapicturehasitsmaximumalloweddimension(i.e.,itllstheVBV),thefollowing ProofWhenapictureisencodedwiththemaximumallowednumberofbits,attheendof intheprecedingones.thisisstatedbythefollowingclaim. Section2.2,theamountofbitsusedtoencodeapictureisconstrainedbytheamountsused 5.1.4StartupShapingDelay thevideoframeperiodinwhichithasbeencaptured,bthasbeendrainedfromthe VBV.BeingthisthenumberofbitsneededtolltheVBV,itprovidesthemaximum Sectionweshowthat,evenifthedimensionofapictureislargerthantheminimumallowed TheratecontrolfunctionusestheVBVtoguaranteethattheMPEGstreamcomplieswith thempegstandardandthattheencoderbuerdoesnotoveroworunderow.inthis alloweddimensionforthefollowingpicture. bythevbv,theencoderbuercanunderow.wealsodescribehowthecbrencodercopes withthissituation. 66

67 Bits theupperdiagramshowstheamountofbitsprocessedversusthetime.thisislargerthan theamountofbitsproducedbythebasicencoder(continuousline)becauseoftheyielded compression;thepunctualslopeofthelineistheinstantaneousnaturalrate.theobjective ofthecbrencoderistoproducebitsattheconstanttargetrateb,asshownbythethick Figure49:I-framesOnlyEncoding. dashedline;theencoderbuerisusedtosmooththevariationsofthenaturalbitrate,as Figure48showsthetimingassociatedwithintra-framecoding.Thethindashedlinein T 2T 3T Time exempliedbythelowerdiagramoffigure48. Processed Natural Rate levellowers,asshown,forexample,intheupperdiagramoffigure48duringthesecondvideo Whentheinstantaneousnaturalbitrateissmallerthanthetargetrate,theencoderbuer Target Rate situationdepictedinfigure49:atthebeginningoftheencodingofascene,thenaturalrate islowerthanthetargetrate.thiscanbepreventedbywaitingastartupshapingdelaywhen theencodingofascenestarts,beforestartingsendingbitsoutoftheencoderbuer.the whenthebuerisempty,theencoderbuerunderows.thishappens,forexample,inthe frameperiodbythethincontinuousline.ifthenaturalrateissmallerthanthetargetrate startupshapingdelayisobtainedbyxingafullnessthresholdinthebuerandwaitingitto becrossedbeforethempegstreamstartsowingoutofthecbrencoder. targetrate.sincethenaturalrateiscontrolledbytheratecontrolfunction,thethreshold canbeaslowasdesired,providedthattheratecontrolisaggressiveenoughtoincrementthe smallertheareaofthepictureoverwhichthenaturalproductionratecanbesmallerthan visualqualityofimagesisnotuniformbecausethequantizationparameterchangessuddenly. naturalratesothatthebuerdoesnotunderows.nevertheless,thelowerthethreshold,the thetargetrate,i.e.,theareaoverwhichtheratecontrolfunctionmustintervene.thus,the Thethresholdrepresentsthetoleranceofthesystemtoanaturalbitratelowerthanthe Ontheotherhand,thehigherthethreshold,thelargerthestartupshapingdelay. quality(andaggressivecontrolfunction)islefttothespecicencoderimplementation.the Thechoiceofthetradeobetweenlongstartupshapingdelayandnon-uniformvisual 67

68 Bits chosenvalueofthestartupshapingdelayisnotverycriticalbecause,iftheencoderispowerful Thismeansthatthemainaimoftheencoderbuerisnotreallysmoothingthevariationof enoughthetargetrateismuchlowerthantheaveragenaturalrate,asshowninfigure5. Figure5:FastI-frameOnlyEncoding. T 2T 3T Time Processed Natural Rate 5.1.5ShapingDelayImplementation delaytocopewithactualbitproductionratevariation,i.e.,thefractionofbuertobelled beforestartingtransmittingshouldnotbelargebecausethebuerisemptiedmuchslower thenaturalrate,butloweringit.thus,itisnotnecessarytointroducealargestartupshaping thanitislledontheaverage. Target Rate Afterhavingdescribedwhicharetheboundsonthedimensionofpictures,weanalyzetheir impactonthedelayandhowitisintroducedbythecbrencoder/decodersystem. partlyinthedecoderone.nevertheless,thevalueofthecodingshapingdelayischosenby bueruntilthebuerlevelcrossesapredenedthreshold.inordertokeepthestartupshaping crossedwithinapredenedamountoftime. delaysmall,theratecontrolfunctionmustincreasethenaturalrateifthethresholdisnot introducesthestartupshapingdelaywhenitstartsencodingascene:nobitexitsfromits Thecodingshapingdelayisexperiencedbypicturespartlyintheencoderbuerand Picturesexperiencethestartupshapingdelayentirelyintheencoderbuer.Theencoder stream[11].thisdeterminestheamountoftimespentbyeachpictureinthedecoderbuer sothattheoveralldelaybesc. theencoderthataccordinglysetsthedecodingandpresentationtimestampsinthempeg CBRencoderisdeterminedbytheVBVdimension,asstatedbyClaim1.Asaconsequence, resultingstream.infact,accordingtoequation(17),giventhecodingshapingdelay,picture delay,i.e.,setthetimestamps;nevertheless,thischoiceiscrucialtothevisualqualityofthe dimensioncannotexceedscb.theupperboundonthedimensionofpicturesencodedbya TheMPEGstandarddoesnotstatehowtheencodershouldchoosethecodingshaping 68

69 to oncethecodingshapingdelayhasbeenset,theencodermustsetthevbvdimensionaccording theend-to-enddelayofthesystem,butdeliversnonconstantquality.theimplementationof ity,andhencevisualqualityoftheencodedstream.thus,asmallcodingshapingdelayreduces InSection5.1.3ithasbeenshownthatVBVdimensionaectspicturedimension,itsvariabil- theratecontrolfunctionmustndatradeo. accordingly),theproductionofasuitablempegstreamisguaranteedbytheratecontrol Oncethestartupandcodingshapingdelayshasbeenchosen(andtheVBVdimensionset VsScB (2) possibletodecreasethebitproductionrate,atworstbystoppingencodingandusinga particularcodeinthempegstreamthatrepresentsambinwhichallthedctcoecients naturalbitratebecauseunencodedpicturesaremuchlargerthanencodedones.itisalways functionavoidingthevbvtooveroworunderow.itisalwayspossibletoincreasethe arenull ExperimentalData ThedvdencsoftwareMPEGencoderusedtoprovidetheexperimentaldatashowninthis disks.thus,itisnotoptimallydesignedforareal-timevideoconferencingsystem.asa consequence,thecontrolfunctionofthecbrencoderisnotaggressiveenoughtoproperly workhasbeenimplementedformpegcompressionofvideosequencestobestoredonvideo function,thedimensionofthebuermustbechosenquitelargerthatthisminimumandthus thetargetrateforawholevideoframeperiod.duetothelimitedaggressivenessthecontrol limitpicturedimensionwhenthevbvissmall. deliverpicturessmallerthanbt;inthiscasetheencoderwouldnotbeabletoemitbitsat itsimpactonpicturedimensionisnotclearlyvisible. TheminimumVBVdimensionisBT.AccordingtoClaim1,asmallerdimensionwould (averagedonthewholeframe)decreases,asshowninfigure52.accordingly,thequalityof congurationsdeliverthesamecodingshapingdelay. throughoutthiswork.threedierenttargetratesareusedfortheencodingofeachsequence variation.theratiobetweenvbvdimensionandtargetrateiskeptconstantsothatallthe inordertoshowhowtheparameterinuencesthenumberofbitsperframeproducedandits Figure51showsthenumberofbitsproducedbyintra-framecodingthefoursequencesused theencodedsequencebecomeshigher,asshownbythesnrplottedinfigure53. accordingtoequation(19),morevariabilityisallowedforpicturedimensionasthetarget rateincreases.thisisconrmedbytheplotsinfigure51. Astherateincreasesthedimensionofframesgrowslargerandthequantizationparameter SinceintheexperimentstheratiobetweenVBVdimensionandtargetrateisconstant, 69

70 Cheerleaders Hockey 7 7 Picture dimension (Kb) Picture dimension (Kb) Picture dimension (Kb) Mb/s - Buff 192 kb 8 Mb/s - Buff 384 kb Bikes 16 Mb/s - Buff 768 kb Table Tennis Figure51:FrameDimensioninIntra-frameOnlyCBREncoding Picture dimension (Kb)

71 Average Quantization Parameter Average Quantization Parameter Cheerleaders Average Quantization Parameter Mb/s - Buff 192 kb 8 Mb/s - Buff 384 kb Bikes 16 Mb/s - Buff 768 kb Table Tennis 1 1 Figure52:AverageQuantizationParameterinIntra-frameOnlyCBREncoding Average Quantization Parameter Hockey

72 5 Cheerleaders 5 Hockey SNR (db) SNR (db) SNR (db) Mb/s - Buff 192 kb 8 Mb/s - Buff 384 kb Bikes 16 Mb/s - Buff 768 kb Table Tennis 5 Figure53:SNRinIntra-frameOnlyCBREncoding SNR (db) 2

73 VBV fullness (%) VBV fullness (%) Cheerleaders VBV fullness (%) Hockey Mb/s - Buff 192 kb 8 Mb/s - Buff 384 kb Bikes 16 Mb/s - Buff 768 kb Table Tennis 1 Figure54:VBVFullnessinIntra-frameOnlyCBREncoding VBV fullness (%)

74 tothesmallvalueofthequantizationparameter(seefigure52).theglobaldistortionlevel Inthe\hockey"sceneat16Mb/sthedimensionofadjacentpicturesvariesverymuchdue Figure55:AmountofBitsEncodingEachPicture. T 2T 3T 4T 5T 6T Time I Frame Figure55andconrmedbytheexperimentalresultsshowninFigure2.Thus,ifpredictive Gisanintegerandisvariedby1;duetothelowvalueofG,therelativevariationislarge Predictivecodingyieldsbettercompressionthanintra-framecoding,asqualitativelyshownin andisreectedinalargevariationofpicturedimension. 5.2Intra-frameandPredictiveCoding P Frame codingisusedtoencodeascenewithagivenvisualquality,thewholenumberofbitsproduced issmallerthanifonlyintra-framecodingisused;thelargerthegop(i.e.,thenumberofpframesperi-frame),thesmallertheamountofbitsproduced13.moreover,theslowerthe predictivecodingmustbeusedtoincreasecompression. scene,thesmallerthedimensionofp-frames;sincevideoconferencingscenesarelikelyslow, thecodingshapingdelayislargerthanwhenusingonlyintra-framecoding.then,weshow slowerthesequence,thehigherthevisualqualityoftheencodedsequence.nevertheless, showthatwhenacbrmpegencoderusespredictivecoding,giventhetargetrate,the ofcodingshapingdelay,werstshowwherepicturesexperienceit.insection5.2.2we unsuitableforvideoconferencingapplications.inordertoprovideabetterunderstanding isstudied.weshowthatcodingshapingdelaymakescbrmpegwithpredictivecoding InthefollowingofthisSection,theimpactofpredictivecodingoncodingshapingdelay howpredictivecodingincreasesthecodingshapingdelaywhenencodingatconstantquality. notdecreasetheamountofbitgeneratedbecausethepredictionerroraccumulatesandrequiresmanybitsto Lastly,theimpactofpredictivecodingonthestartupshapingdelayisconsidered. 13Actually,asshownin[19],thereisamaximumGOPdimensionbeyondwhichenlargingtheGOPdoes beencoded. 74

75 Processed Basic Encoder Output CBR Encoder Output Bits Figure56:NaturalandTargetGenerationofBitsandEncoderandDecoderBuerFullness. (I) T 1 (P) 2T 2 (P) 3T 3 (P) 4T 4 (I) 5T Time (I) 2 (P) 3 (P) 1 (P) buer,asexplainedindetailinthissection. T 2T S c 3T S c +T 4T S c +2T 5T Time byacbrencoder/decodersystemusingbothintra-frameandpredictivecoding.theupper 5.2.1BueringIssuesRelatedtotheCodingShapingDelay diagramqualitativelyshowsthati-framesarelargerthanp-frames.thenaturalrateaveraged amountofbits.picturesexperiencepartofitintheencoderbuerandpartinthedecoder InSection5.1.1ithasbeenshownthataCBRMPEGencoder/decodersystemintroducesa codingshapingdelayinordertoallowpicturestobeencodedatconstantratewithvariable Figure56showshowthebitsofanencodedvideostreamareproducedandbuered Buffer onavideoframeperiodwhilethebasicencoderisintra-framecoding,islargerthanthetarget Buffer rate;thenaturalratewhilepredictivecodingissmallerthanthetargetrate. theirdimensionisthemaximumallowedbythesystem.forexample,picture,whichis assoonastheyarrivebecausetheyexperiencetheprocessingresynchronizationdelayunless correspondingtoaframe.encodedpicturesarenotretrievedfromthebueranddecoded bitsproducedbythebasicencoderforthepreviouspicturesandstillnotsentoutbythecbr encoder.thedecoderbuerlevelisincrementedlinearlyanddecrementedbyretrievalunits (dashedline)buer.p-framesndabacklogintheencoderbuerduetothehighnumberof Thelowerdiagramplotsthefullnessleveloftheencoder(continuousline)andthedecoder 75

76 butitisnotdecodeduntiltheinstantsc,i.e.,untilthecodingshapingdelayiselapsedfrom itscapture CodingShapingDelayatConstantTargetRate Weconsidertheencoderoperatingataxedtargetrateandstudytheeectofthecoding intra-framecoded,fullyentersthedecoderbuer(exitstheencoder)atthetimetagged(i), ordertohaveasimple(eventhoughnotaccurate)measureofthevisualqualitywhichallows dierentcongurationtobecompared,wemakethefollowingassumption. aresmallandmorebitsoutofthegopbudgetareusedinencodingi-frames.ifascene MPEGencodersusuallyhavetheirxedtargetrateandchangetheratiobetweenI-frameand isfast,p-framesrequiremorebitsandthedimensionofi-framesdecreasesaccordingly.in P-framedimensionaccordingtothemotioninthescenebeingencoded.Ifitisslow,P-frames shapingdelayonthevisualqualityoftheencodedstream.thisisarealisticscenarioascbr Assumption4Ifthesamesceneisencodedmorethanoncewithdierentencodingparameters,thevisualqualityoftheencodedstreamsobtainedwiththedierentexperimentscanbe comparedbycomparingthedimensionofi-frames:largeri-framesmeanbettervisualquality. AccordingtothisAssumptionandtowhatstatedabove,giventhetargetrate,thefastmoving encoderproduceslargei-frames.bydoingthisitintroducesalargecodingshapingdelay whichislowerboundbythedimensionofthelargestpictureaccordingtodisequation(17). isascene,thelowerthequality. dimensionandotherparametersofacbrencoderthatexploitspredictivecoding.weassume Inavideoconferencingscenario,scenesareusuallyslowandthus,givenatargetrate,the videoconferencingapplicationsshouldbetternotexploitpredictivecodinginordertoreduce thattheencoderusesaconstantnumberofbitstoencodeeachgop:itmustproducethese bitsinagopperiod,i.e.,eachgopisencodedwithntbbits.actualcbrdonot shapingdelayofanyparticularencoder.nevertheless,itgivestheavorthatcbrencodersfor usuallycomplytothisassumption;thus,theexpressionwedevisedoesnotprovidethecoding WenowdeviseanexpressionthatgivesthecodingshapingdelayasfunctionofP-frame theend-to-enddelay. wherefiisthedimensionofthei-frameinthegop,fp ByextractingFIfromEquation(21),weobtain,foreachGOP, Accordingtotheforegoingassumption,thefollowingequationholdsforeachGOP FI+N?1 Xi=1FP i=ntb 76 iisthedimensionoftheithp-frame. (21)

77 ThesmallerthenumberofbitsusedtoencodeP-frames,thelargerthedimensionofthe I-frameintheGOP. FI=NTB?N?1 Xi=1FP codingisexploited,thelargestpictureisthei-frameinthegopwiththesmallestp-frames. maxseqfisthedimensionofthelargestencodedpictureinthesequence.whenpredictive Equation(18)inSection5.1.1givesthecodingshapingdelaySc=maxseqF=B,where i (22) dimension,i.e.,maxseqf=iasgivenbyequation(22);substitutinginequation(18): Forthesakeofsimplicity(alsointhenotation),weassumethateachI-framehasthesame theencoding/decodingsystemmustintroduce.iftheencodedsceneiscompletelystatic,the PictureDimensionThesmallertheP-frames,thelongerthecodingshapingdelaythat P-framesareencodedwithaverysmallamountofbits;inprinciple.Thus,theI-frame Sc=NTB?PN?1 B i=1fp i=nt?pn?1 i=1fp B i toacodingshapingdelayequaltothegopperiodnt. growsusingallthebitsintendedfortheencodingofthegop,i.e.,ntb;thiscorresponds (23) bitsusedtoencodepicturesisthevbvdimensionvs;adimensionequaltontballows slowlymovingandwouldallowalargeratio.thekeyelementindeterminingthenumberof theratiobetweenthedimensionofi-framesandp-frames,eventhoughthescenetoencodeis theencodertotakefulladvantageofpredictivecodingwhenthesceneisparticularlyslowand tousealargeamountofbitstoencodei-frames.alargerdimensionallowstheencoderto deliverevenmoreuniformqualitybysmoothingsuddenincreaseinthecomplexityofpictures Thus,aCBRencoderwhichaimsatyieldingashortcodingshapingdelaymustbound andinthemotionovermorethanagopperiod.nevertheless,thisrequireslargercoding shapingdelayandbuersinbothencoderanddecoder. P-frames.Thisisparticularlyusefulwhenaparticularlycomplexpicture(i.e.,withlowspatial usedintheencodingofeachgopandapreferredratiobetweendimensionofi-framesand redundancy)isbeingintra-framecoded;iftheencoderusedonlythevbvtodeterminethe bitsusedtoencodepictures.forexample,thesoftwareencoderdvdencbeforeencodinga targetforthepicturedimension,inordertodelivergoodquality,itwouldusethemaximum picturesetsatargetdimension;thisischosenaccordingtoapredenedamountofbitstobe BesidetheVBV,CBRencodersbetterhaveothermeansforcontrollingthenumberof numberofbitsallowedbythesystemtoencodeapicture,i.e.,itwouldllupthevbv. AccordingtoClaim2,thedimensionoffollowingpicture(s)wouldbeboundbyBT;this boundcanbetoosmall,particularlyifthescenebecomessuddenlyfast.infact,p-framesare encodedwithanumberofbitsthatisnotsucienttoshowthesamevisualqualityofthe I-framecorrespondingI-frameandthusthequalityoftheGOPislowered. 77

78 Nevertheless,Equation(22)showsthatincreasingthevideoframeperiodalsoimprovesthe theencodedscene,itshouldbeavoidedtokeepthedelaysmaller14. thetargetrate;eventhoughaccordingtoassumption4thisincreasesthevisualqualityof GOPSizeIncreasingtheGOPsize(i.e.,N)increasesthecodingshapingdelaybecausethe qualityoftheencodedscenebecausei-framesbecomelarger. byequation(23);thelongerthevideoframeperiodt,thelargerthecodingshapingdelay. percentageofsmallpictures(thep-frames)increasesandsoi-framesaremadelargertokeep Conclusion1GiventhetargetrateofaCBRMPEGencoder,predictivecodingincreasesthe visualquality.nevertheless,everyparametersettingaimedatqualityimprovementincreases Usingalowvideoframerate(i.e.,largeT)increasesthecodingshapingdelay,asshown qualitythecbrencodermustdeliver.weassumetoencodethesamescenemorethanonce Wenowstudytheimpactofpredictivecodingonthecodingshapingdelay,giventhevisual thecodingshapingdelay CodingShapingDelayatConstantVisualQuality thani-framesandasmuchgopsizenislarger. I-frames,accordingtoAssumption4;weneedawaytocomparetheeectivenessofpredictive coding.theexploitationofpredictivecodingisasmoreeectiveasmuchp-framesaresmaller exploitedandthevisualqualitydeliveredisthesame. theparametersoftheencoderarechangedsothatpredictivecodingismoreorlessheavily withdierentamountsofcompressionduetopredictivecoding.i.e.,foreachexperiment usingadierentnumberofp-framespergop,thei-framesproducedineachexperimenthave thesamedimensionfi.assumption4isaroughapproximation,butitisclosetorealitywhen AccordingtoAssumption4,ifthesamesceneisencodedwithsamevisualquality,but Thevisualqualityoftheencodedstreamsiscomparedbycomparingthedimensionof withconstantamountofbitspandexploitasmeasureofyieldedcompressiontheratio thecontextoftheanalysiscarriedoninthissection,weassumethatp-framesareencoded dealingwithsimpleslowmovingscenes(suchasatypicalvideoconference);moreover,theaim ofthissectionisonlytogivetheavoroftheimpactofpredictivecodingonthecodingshaping delay,nottoprovideageneralanalyticalrelationshipbetweenthetwo.insection4.1.2we statedthatusuallythedimensionofi-framesisfrom2to4timesthedimensionofp-frames; theslowertheencodedscene,thelargertheratiobetweeni-frameandp-framedimension.in 14IncreasingtheGOPsizealsorenderstheencodedstreammoresensitivetoerrorsandlosses. Therateoftheencodedstreamcanberoughlyexpressedas =FI 78FP

79 wherenisthenumberofpicturespergopandfpisp-framedimension,whichweassume tobeconstantforallthep-frames.equation(24)canberewrittenas BN=FI+(N?1)FP BN=N+?1 NTFI (24) Bymergingthisequationwith(25),thelattercanbewrittenas Tobetterhighlighttheeectofpredictivecodingontherateoftheencoder,wealsoprovide theencoderratewhenonlyintra-framecodingisused: BI=FI T (26) (25) DuetotheapproximationsmadetodevisethisEquation,itdoesnotexactlyreectthereal behavior,butitgivestheavorthatthemoreeectivepredictivecoding(i.e.,theratio),the smallerthetargetrateneededtodeliverthesamevisualquality.also,thelargerthenumber ofp-framespergop(n),thesmallerthetargetrate. BN=BIN+?1 Equation(18)inSection5.1.1givesthecodingshapingdelayasSc=maxseqF=B,where Wenowanalyzetheeectofpredictivecodingatconstantqualityoncodingshapingdelay. N (27) BeingFIconstantwhenthequalityisconstant,thetargetrateoftheencoderBistheonly maxseqfisthemaximumdimensionofframesandbisthetargetrateofthecbrencoder. thevalueofbwithbngivenbyequation(25) factorwhichdeterminesthecodingshapingdelayindierentcongurations.bysubstituting AccordingtoAssumption4,maxseqF=FIandthecodingshapingdelayisexpressedas Sc=NT Sc=FI N+?1 B Themorepredictivecodingiseective(i.e.,),thelargerthecodingshapingdelay.Typical.Inavideoconferencingapplication,itislikelytohave=4;withN=15at15fps, valuesofarebetween2and4;theslowerthemotioninthescene,thelargerthevalueof largerthanthe1msboundessentialforinteraction.similarconsiderationsapplyalsoto Sc=222ms,i.e.,theCBRencoderaloneintroducesinthevideoconferencingsystemadelay theincreaseofn. 79

80 Processed Basic Encoder Output CBR Encoder Output Bits Conclusion2UsingaCBRMPEGencoderwithpredictivecodingtobuildavideoconferencingsystem,reducesthebandwidthrequirement,butitisnotacceptablefromthepointofview (I) T 1 (P) 2T 2 (P) 3T 3 (P) 4T 4 (I) 5T Time Bits (I) 2 (P) 3 (P) 5.2.4StartupShapingDelay oftheend-to-enddelay. Figure57:RealisticBitGeneration. 1 (P) AsshowninFigure57,atypicalMPEGencoderencodesI-frameswithalargeamountofbits T 2T S c 3T S c +T 4T S c +2T 5T Time inasmalltime,withrespecttop-frames.asaconsequence,itisunlikelythatthenatural ratebelowerthanthetargetratewhentheencodingofagopbegins.after,thebuerhas Encoder Buffer Decoder Buffer quantizationstepsizeinordertoincreasethenaturalrate)arevisibleonasmallpercentage thehighestunderowprobabilityisafterthewhenthebuergetsemptyattheendofa GOPwithsmallP-frames.Duringtherestofthetimeabacklogislikelytobepresentin delayisunderdimensioned,theeects(nonuniformqualityduetoasuddendecreaseofthe theencoderbuer,ascanbeseeninfigure56.thus,eventhoughthestartupshaping somebacklogduetothelargeamountofbitsencodingthei-frame. ofthepictures. TheencoderbuercanhappentounderowonlywhenP-framesareparticularlysmall; 8

81 oftheratecontrolfunction.thegopdimensionissetton=15,i.e.,eachi-frameisfollowed valuethatguaranteesthevbvnottobeoverowed,notwithstandingthelowaggressiveness threedierentrates.theratiobetweenthevbvdimensionandtherateiskeptconstanttoa Figures58-61showtheexperimentaldataobtainedbyencodingthefoursamplesequencesat by14p-frames. 5.3ExperimentalData dimensionofi-framesandp-frames.thisisduetothefactthatthecameraispanningover thehockeycourttofollowthegame.picturesaresimple(mostofthescreenisthewhiteice andthedimensionofbothi-framesandp-framesisvariable.especiallyatlowbitratesthe dimensionofi-framesisdoubleormorethandoublethedimensionofp-frames:inorderfor thistobepossiblethecodingshapingdelaymustbelarge. Inthe\hockey"sequenceencodedat16Mb/sthereisalmostnodierencebetweenthe Figure58showsthedimensionofeachpicture.I-framesaretypicallylargerthanP-frames isdecreased;beingitsmall,decreasingitbyoneunithasasignicanteectonthepicture dimension:thesecondp-frameismuchsmallerthantherstone.thiscausesthevbv smallerthanthei-frame.inordertoreducep-framedimensionthequantizationparameter leveltodecrease(figure61)andthequantizationparametertobereducedagaintoavoid thewholeimagesandp-framesdimensionislarge:therstp-frameinthegopisnotmuch asmallquantizationparameter(figure59).sincethecameraispanning,motionconcerns ofthecourt)sothati-framescanbeencodedwithfewbits;thep-framesarequantizedwith fullandemptiesmoreslowly(bottomrightplotinfigure65). underow.thisbehavioriskeptforthewholegopthusleadingtotheoscillationinpframesdimensionandpicturequality(seesnrinfigure6).figure59showstheaveragenceisusuallycharacterizedbylowmotionandpictureshavinglowcomplexity.thus,when whichi-framedimensionincreases(bottomrightplotinfigure62)andthevbvgetsmore arequitesimilartotheoneobtainedwithn=15exceptforthe\tabletennis"sequencein quantizationparametervalueusedoneachframe. compressingthemwithacbrmpegencoderafairlysmallamountofbitscanbesucient Thescenesusedintheseexperimentsarenotvideoconferencingsequences;avideoconfer- Figures62-65showtheexperimentaldataobtainedusingaGOPof3pictures.Theresults toencodei-frameswithgoodquality;whenencodingap-frametheerrorbetweenthemb thatisbeingencodedandthereferencembissmall,andthustheamountofbitsrequiredis lowinordertoemphasizethisbehaviorofacbrmpegencoder,weexploitedtoencodea applicationif,forexample,thecameraispointedatawhiteboardwhilesomebodyisexplaining itscontent.weobtainedtwostaticscenesbyrepeating12timesthesameimageofthe staticscene.thisisanextremecaseofslowmovingscenebutislikelyforavideoconferencing \cheerleaders"and\hockey"scenes,respectively. Figure66showsthedimensionofframesforvarioustargetratesandVBVdimensions. 81

82 Cheerleaders Hockey Picture dimension (Kb) Picture dimension (Kb) Mb/s - Buff 2.4 Mb 8 Mb/s - Buff 4.8 Mb 16 Mb/s - Buff 9.6 Mb Bikes Table Tennis Picture dimension (Kb) Figure58:DimensionofPicturesinCBRMPEGEncodingwithN= number number Picture dimension (Kb)

83 Average Quantization Parameter Average Quantization Parameter Cheerleaders Average Quantization Parameter Mb/s - Buff 2.4 Mb 8 Mb/s - Buff 4.8 Mb Bikes 16 Mb/s - Buff 9.6 Mb Table Tennis 8 Figure59:AverageQuantizationParameterinCBRMPEGEncodingwithN= Average Quantization Parameter Hockey

84 5 Cheerleaders 5 Hockey SNR (db) 4 35 SNR (db) SNR (db) Mb/s - Buff 2.4 Mb 8 Mb/s - Buff 4.8 Mb Bikes 16 Mb/s - Buff 9.6 Mb Table Tennis 3 5 Figure6:SNRinCBRMPEGEncodingwithN= SNR (db)

85 35 Cheerleaders 35 Hockey 3 3 VBV Fullness (%) VBV Fullness (%) VBV Fullness (%) Mb/s - Buff 2.4 Mb 8 Mb/s - Buff 4.8 Mb Bikes 16 Mb/s - Buff 9.6 Mb Table Tennis 35 Figure61:VBVFullnessinCBRMPEGEncodingwithN= VBV Fullness (%)

86 Picture dimension (Kb) Picture dimension (Kb) Cheerleaders Picture dimension (Kb) Mb/s - Buff 2.4 Mb 8 Mb/s - Buff 4.8 Mb Bikes 16 Mb/s - Buff 9.6 Mb Table Tennis 16 Figure62:DimensionofPicturesinCBRMPEGEncodingwithN= number number Picture dimension (Kb) Hockey

87 Average Quantization Parameter Average Quantization Parameter Cheerleaders Average Quantization Parameter Mb/s - Buff 2.4 Mb 8 Mb/s - Buff 4.8 Mb Bikes 16 Mb/s - Buff 9.6 Mb Table Tennis 8 Figure63:AverageQuantizationParameterinCBRMPEGEncodingwithN= Average Quantization Parameter Hockey

88 5 Cheerleaders 5 Hockey SNR (db) 4 35 SNR (db) SNR (db) Mb/s - Buff 2.4 Mb 8 Mb/s - Buff 4.8 Mb Bikes 16 Mb/s - Buff 9.6 Mb Table Tennis 3 5 Figure64:SNRinCBRMPEGEncodingwithN= SNR (db)

89 VBV Fullness (%) VBV Fullness (%) Cheerleaders VBV Fullness (%) Hockey Mb/s - Buff 2.4 Mb 8 Mb/s - Buff 4.8 Mb Bikes 16 Mb/s - Buff 9.6 Mb Table Tennis 7 Figure65:VBVFullnessinCBRMPEGEncodingwithN= VBV Fullness (%)

90 25 Cheerleaders - VBV 1.6 Mb 25 Hockey - VBV 1.6 Mb Picture dimension (Kb) Picture dimension (Kb) Mb/s 8 Mb/s Cheerleaders - 16 Mb/s 16 Mb/s Hockey - 16 Mb/s Picture dimension (Kb) Picture dimension (Kb) Picture dimension (Kb) VBV 1.6 Mb VBV 3.2 Mb VBV 6.4 Mb 4 Mb/s - VBV 1.6 Mb 8 Mb/s - VBV 3.2 Mb Figure66:DimensionofPicturesinCBRMPEGEncodingofaStaticScene Cheerleaders 16 Mb/s - VBV 6.4 Mb Hockey number number Picture dimension (Kb)

91 spondingnonstaticsceneisencodedatthesametargetrate.p-framesaresmallbecausethey mustonlyencodethedierencebetweentheirreferencepictureandtherawpicturethatis duetothequantization.thus,thesnrdeliveredbyp-framesishigh,asshowninfigure68. InanyplottheratiobetweenI-frameandP-framedimensionislargerthanwhenthecorre- parameterhastheminimumallowedvalueandcannotbefurtherreduced(seefigure67). dimensions(secondrow,rstcolumninfigure66)showsthatthelargeristhevbvdimension, allowalargeramountofbitstobeproduced,i-framesdonotgrowbecausethequantization thelargestarethei-frames.thisdoesnotholdforthe\hockey"scene(secondrow,second column)because,beingthepictureslessdetailed,eventhoughtheratecontrolfunctionwould Theplotobtainedbyencodingthe\cheerleaders"sceneat16Mb/swithvariousVBV thefullnesslevelisoftheorderofthe2%.nevertheless,whendealingwithreal-time inadvance.ifthevideoconferencingsystemmustbeabletodeliveragoodquality,especially encoding(suchasinvideoconferencingapplications),thescenetobeencodedisnotknown whenthecameraisstaticallysetonsomeimage,thevbvmustbelargeenoughtoallow I-framestogrowandcapturethedetailsoftheimage15(e.g.,secondrow,rstcolumnin Figure69).Whenpicturesarenotdetailed(e.g.,secondrow,secondcolumninFigure69)or Figure69showsthatinmanycasestheVBVis(almost)completelylled,whileinothers themotionishigher(figure61)onlyafractionofthevbvislled.inanycase,thecoding shapingdelayislargebecauseitmustbesetaccordingtothevbvdimensioninorderto 5.4DedicatedLinkbetweenSenderandReceiver guaranteecontinuousplaying. Forthesakeofcompletenessofthisstudy,thesystemcongurationinwhichsenderand receiverareconnectedthroughadedicatedlink(figure7)istakenintoaccount.havingthe advantages.onlyafractionofthelinkcapacity,correspondingtothetargetratebofthe encoderisactuallyexploitedandthecodingshapingdelayprovidesasignicantcontribution totheend-to-enddelay. wholelinkcapacityatdisposalforsendingthecbrmpegstreamdoesnotprovideparticular systemexploitingcbrmpegencoding. theend-to-enddelayofthesystemis lay.thisrepresentalowerboundintheend-to-enddelayobtainablefromavideoconferencing wheredisthedecodingdelay,scisthecodingshapingdelay,andssisthestartupshapingde- GiventhepropagationdelayP,whichdependsonthedistancebetweensenderandreceiver, 15Notethatasthesceneisnotmoving,thehumaneyeismoresensitivetoerrors. Ded CBR=Sc+Ss+P+D+Pd (28) 91

92 Cheerleaders - VBV 1.6 Mb Hockey - VBV 1.6 Mb Average Quantization Parameter Average Quantization Parameter Mb/s 8 Mb/s Cheerleaders - 16 Mb/s 16 Mb/s Hockey - 16 Mb/s Average Quantization Parameter Average Quantization Parameter VBV 1.6 Mb VBV 3.2 Mb VBV 6.4 Mb 4 Mb/s - VBV 1.6 Mb 8 Mb/s - VBV 3.2 Mb Figure67:AverageQuantizationParameterinCBRMPEGEncodingofaStaticScene Cheerleaders 16 Mb/s - VBV 6.4 Mb Hockey Average Quantization Parameter Average Quantization Parameter

93 Cheerleaders - VBV 1.6 Mb Hockey - VBV 1.6 Mb 5 5 SNR (db) 45 4 SNR (db) Mb/s 8 Mb/s Cheerleaders - 16 Mb/s 16 Mb/s Hockey - 16 Mb/s 5 5 SNR (db) 45 4 SNR (db) SNR (db) VBV 1.6 Mb VBV 3.2 Mb VBV 6.4 Mb 4 Mb/s - VBV 1.6 Mb 8 Mb/s - VBV 3.2 Mb Figure68:SNRinCBRMPEGEncodingofaStaticScene Cheerleaders 16 Mb/s - VBV 6.4 Mb Hockey SNR (db)

94 VBV Fullness (%) VBV Fullness (%) VBV Fullness (%) Cheerleaders - VBV 1.6 Mb VBV Fullness (%) Mb/s 8 Mb/s 16 Mb/s Cheerleaders - 16 Mb/s VBV Fullness (%) Hockey - VBV 1.6 Mb Hockey - 16 Mb/s VBV 1.6 Mb VBV 3.2 Mb VBV 6.4 Mb 4 Mb/s - VBV 1.6 Mb 8 Mb/s - VBV 3.2 Mb Figure69:VBVFullnessinCBRMPEGEncodingofaStaticScene Cheerleaders 16 Mb/s Mb Hockey VBV Fullness (%)

95 Figure7:ArchitectureofaVideoconferencingSystemExploitingaCBRMPEGEncoder Figure71:ArchitectureofaVideoconferencingSystemExploitingaCBRMPEGEncoder andadedicatedlinkbetweensenderandreceiver. Frame Network Network Encoder Decoder Grabber Interface Interface Adaptor rateoftheencoder.thecontributionofthenetworktotheend-to-enddelayistheswitching MPEGencoderisusedtobuildavideoconferencingsystem(Figure71),themosteectiveway toconnectsenderandreceiveristhroughacircuitthathasabandwidthequaltothetarget CBRMPEGencodingissuitedfortransmissionovercircuitswitchednetworks:whenaCBR andacircuitswitchednetwork. Frame Encoder receiver.theend-to-enddelayofthesystemis 5.5CircuitSwitching delayswandthepropagationdelaypwhichdependsonthedistancebetweensenderand Decoder Interface Network Grabber Interface Adaptor systemexploitingcbrmpegencoding. wheredisthedecodingdelay,scisthecodingshapingdelay,andssisthestartupshaping representsaswellalowerboundintheend-to-enddelayobtainablefromavideoconferencing receiver:theonlyfurthercontributionistheswitchingdelay.thus,itcanbesaidthatit delay.thisdelayalmostequalstheoneobtainedwithadedicatedlinkbetweensenderand CS CBR=Sc+Ss+Sw+P+D+Pd (29) 5.6PacketSwitchingwithTimeDrivenPriority CBRMPEGencodingisbestsuitedfortransmissionovercircuitswitchednetworks.Nevertheless,inthefollowingtwoSectionsweconsidertheend-to-enddelayobtainedwhenCBR encodedvideoissentthroughapacketswitchednetwork.inthissection,timedrivenpriority senderandreceiverisanalyzedrst;then,theend-to-enddelayisconsidered. isconsideredtobeexploitedinthenetwork.thenetworkshapingdelayintroducedinthe 95

96 Frame Grabber Video Adaptor Encoder Decoder Packetization Packetization Figure72:ArchitectureofaVideoconferencingSystemExploitingaCBRMPEGEncoder andapacketswitchednetworkwithtimedrivenpriority NetworkShapingDelay Whentimedrivenpriorityisusedinthenetwork,thetimecycleissettoTc,thevideoframe period,andbtcbitsmustbereservedineachtimecycle.thebitsexitingthecbrmpeg encoderarebueredinthebuerofthepacketizationfunctiondepictedinfigure72,untilthey canbeincludedinapacket.eachbitexperiencesadierentdelay,butduetorequirement3 ofcontinuousplaying,anetworkresynchronizationdelaymustbeintroducedonthereceiver sidesothateachbitexperiencesthesamewholedelayspack nwhichiscalledthepacketization componentofthenetworkshapingdelay. Thepacketizationfunctiononthereceiversideintroducesthenetworkresynchronization delay;thepacketizationfunctionknowsthedelayexperiencedinthebuerofthesender packetizationfunctionbythebitsofeachreceivedpacketandcancompensateit.thecompensationisdonebybueringthebitsandprovidingtothedecoderabitowattheconstant rateb.ifthetfsreservedtothevideoconferencecallareevenlydistributedinthetime cycle,nonetworkresynchronizationisrequired. ThedimensionofthebuerofbothpacketizationfunctionsmustbeatleastBSPack n. PPack nisthemaximumtimeabitspendsinthesenderbuerbeforebeingretrievedand insertedinapacket;thebuer,whichislledatconstantrateb,doesnotoverowifit canstorethebspack nbitsgeneratedbythedecoderinthemeantime.spack nisaswellthe maximumtimespentbyabitinthereceiverbuer;thebuercannotcontainmorethan BSPack nbitsotherwisethelastonewouldtakemorethanspack ntoexitattheconstantrate B.TheactualvalueofSPack ndependsonhowthetfshavebeenreservedwithinthetime cycleforthevideoconference;thus,itisxedatresourcereservationtimeandisconstant overthewholevideoconferencecall(oruntilthebandwidthisreallocated).if,forexample, asingletfisreserved,spack n=tcbecausetherstbitexitingthecbrencoderaftera packethasbeenassembledandsenthastowaitawholetimecyclebeforebeinginsertedina packet.intuitively,thesmallerandthemoreuniformlydistributedinsidethetimecycleare thepackets,theshorterthenetworkshapingdelayspack n. Exploitationofsmallanduniformlydistributedpacketsisencouragedalsobyotherarguments.InSection4.3.5ithasbeenshownthat,ifthedecoderstartsdecodingassoonassome ofthebitsencodingapictureenteritsbuer,thedecodingtimeisreducedbyusingsmall packets.in[14]isshownthatreservingasmallamountofbitsinmanytfsisconvenient 96

97 provideslowerblockingprobability.usingsmallpacketshasafurtheradvantagewhentime drivenpriorityisexploitedinthetransmissionofcbrmpegencodedvideo. alsofromthepointofviewofthenetworkbecausebalancingtheloadalongthetimecycle encodingasinglepicture.thenetworkshapingdelaycanbeminimizedifthetfsarereserved streamiscomposedofpacketshavingxeddimensionof188bytes;eachpacketcontainsbits ordertomakemorerobustitstransmissionoverpacketswitchednetworks.thetransport producedbyacbrmpegencoderbestructuredasatransportstream(seesectionb.5)in sothatassoonasanmpegpacketisproduced,areservedtfisscheduledandthepacketis MPEGstreammustbetakenintoaccount.TheMPEGstandardspeciesthatthebitstream Toidentifytheschedulethatminimizesthenetworkshapingdelay,thestructureofthe sent16.thisrequirestoreserve188bytesintfsequallyspacedby(1888)=bandhavethe encodersynchronizedwiththenetworkinterfacesothatthelastbitofeachmpegpacketis emittedjustbeforeareservedtfisscheduled17.iftheencoderisnotsynchronizedwiththe networkinterface,thereisanapplicationsynchronizationcomponentofthenetworkshaping delaysas components,i.e., Inthegeneralcase,thenetworkshapingdelayisgivenbythecompositionofthetwo n2[;(1888)=b] ExcessResynchronizationDelay ofthereceiver. anditisexperiencedpartlyinthesenderandpartlyinthenetworkresynchronizationfunction Sn=SAS n+spack Inwhathasbeendiscussedsofar,thenetworkresynchronizationdelayisassumedtobe n introducedbyanadhocfunctionwhichcanbeimplemented,forexample,inthesoftware layersbelowtheapplication.nevertheless,itcanalsobeintroducedinthedecoderbuer; mustbeenlarged.ifthedecoderissynchronizedwiththenetworkinterface,thebuermust bueringisnotrequiredinthereceiversoftwarebelowthedecoder18andthedecoderbuer end-to-enddelaybecauseitisalreadytakenintoaccountbythecodingshapingdelay:thebitswouldbe untilallthe188byteshavebeensentoutoftheencoder.however,thistimedoesnotcontributetothe onlyvariablecomponentofthenetworkdelay. isexploitedinthenetwork,thepacketizationcomponentofthenetworkshapingdelayisthe beenlargedby,atleast,bspack 17Actually,eveninthissituation,thebitsofeachpackethastobebueredinthepacketizationfunction 16BreakinganMPEGpacketandsendingitduringtwoTFsdecreasesrobustnesstolossesanderrors. n.thisstemsfromthefactthatwhentimedrivenpriority anywaykeptinthedecoderbuertointroducetheprocessingresynchronizationdelay.theonlyimpacton theend-to-enddelayisapossibleincreaseofthedecodingtimeifthedecoderhasanaccessunitsmallerthan thempegpacket. notrelevanttothiswork. 18Actually,abuerisneededonlytobuereachincomingpacketwhiletheheaderisprocessed,butthisis 97

98 andthedecoderbuermustbeenlargedby,atleast,bspack partlyinthesenderpacketizationfunction,partlyinthedecoderbuerisactuallyspack accordingly19.iftheassumptionistrue,thenetworkshapingdelayexperiencedbyeachbit thebitshaveexperiencedtheminimumdelayandthedecodingofpicturesmustbedelayed receivedthedecoderdoesnotknowhowmuchtimethebitshavespentinthebuerof thesenderpacketizationfunction.thus,thedecoderusestheconservativeassumptionthat Ifthedecoderisnotsynchronizedwiththenetworkinterface,whentherstpacketis true,aexcessresynchronizationdelayerisintroduced,i.e.,bitsarebueredlongerthan receivedpackethasalreadyexperiencedadelayspack totheexcessbueringtimeer.inthefollowing,themaximumvalueoferandtheminimum necessary;thedecoderbuermustbedimensionedaccordinglysothatitdoesn'toverowdue dimensionofthebuerarecalculated. Thelargestexcessresynchronizationdelayisintroducedwhentherstbitintherst n.iftheassumptionisnot n function.thedecodershouldnotintroduceanynetworkresynchronizationdelay;insteadthe buer,itisnecessarytocomputetheminimumdelayexperiencedinthesenderpacketization decodingisdelayedbecauseitisassumedthatthebitsexperiencedminimumdelayinthe packetizationfunction.inordertocalculatethetimespentbytherstpacketinthedecoder Bitsexperienceminimumdelayduetopacketization,ifapacketofdimensionPfissentas ninthebuerofthesenderpacketization whentherstbitwasproducedbythecbrencoder,thenetworkresynchronizationdelayexperiencedinthedecoderbuer(whichisalsotheupperboundontheexcessresynchronization delayer)is soonasthecbrencoderemitsthebitnumberpf.inthiscase,therstbitinthepackethad beeninthebuerofthesenderpacketizationfunctionforthetimethecbrencodertakes n?pf B nfrom toproducepfbits,i.e.,pf=b.sincedecodingoftherstpacketmuststartafterspack where andexcessresynchronizationdelayisthusgivenby Theoveralltimespentbybitsinthedecoderbuerduetonetworkresynchronizationdelay n+er (3) networks. resynchronizationdelaytocompensatethevariationofthequeueingdelayinasynchronouspacketswitched 19AsexplainedinSection3.4.1,thereplaybuershowsthesamebehaviorwhenintroducingthenetwork Er2;SPack n?pf B 98

99 itmustbeenlargedby thenetworkresynchronizationdelayandberobustagainsttheexcessresynchronizationdelay, Thedecoderbuermustbeoversizedaccordingtotheworstcase,i.e.,inordertointroduce call.thus,spack decoderdoesnotknowtheactualvalueofspack TheactualvalueofSPack n2[;tc]dependsonthetfsreservedforthevideoconference 2SPack casespack withamechanismtocommunicateittothedecoder.ifsuchamechanismisnotavailable,the n=tc;i.e.,decodingisstartedafter nisknownatreservationtimebythenetworkinterface;thiscanbeprovided n?pf nanddelaysdecodingaccordingtotheworst BB excessdelayisintroducedwhenthebitsintherstpackethasexperiencedminimumdelay fromthereceptionoftherstpacket.iftheactualpacketizationcomponentofthenetwork excessresynchronizationdelay.inparticular(31)isthemaximumervalue;theminimum shapingdelayisnull(becausetfshadbeenallocatedoptimally)andtherstpackethas experiencedspack ninthesender,itisunusefultodelaydecodingby(31),i.e.,thisvalueisthe Tc?Pf B anddecodingshouldbedelayedby(3): Thedecoderbuermustthusbeenlargedby Er2Tc?SPack SPack n+tc?pf n;tc?pf BBB tothenetworkinterface(i.e.,itdoesnothaveinformationaboutthescheduleofthereserved TFsandtheTFinwhicheachpacketisreceived),theexcessresynchronizationdelayislarger. eliminatestheneedforanadhocbuer.nevertheless,ifthedecoderisnottightlyconnected 5.6.3End-to-endDelay Thisshowsthatnetworkresynchronizationcanbeperformedinsidethedecoder;this where Thegeneralexpressionfortheend-to-enddelayis TDP?SH CBR=Sc+Ss+LTf+Sn+Er+D+Pd 99 (32)

100 Scisthecodingshapingdelayintroducedbytheencoder/decodersystemtoproduce theencodedstreamattheconstanttargetrate; Ssisthestartupshapingdelayintroducedbytheencodertoavoidtheencoderbuer tooverrunwhenthenaturalrateissmallerthanthetargetrate; LTfisthetimetakenbyapackettotravelthroughthenetworkfromsourcefrom destination,; Sn=SPack n+sas n: {SPack n2[;tc]dependsonthenumber,dimension,andpositionofthetfsreserved tothevideoconferencecallandisconstantoverthedurationofthecall, {SAS n2[;tc]isduetothelackofsynchronizationbetweenencoderandnetwork interface;becauseoftherelativedriftofthetwotimingreferences,thisdelaycan varyinsidethegivenintervalduringthevideoconferencecall; Er2[;Tc?Pf=B]isnotnullwhenthenetworkresynchronizationdelayisintroduced bythedecoderanditisnotsynchronizedwiththenetworkinterface. Ifresourcereservationisoptimal(SPack n=pf=b),theencoderissynchronizedwiththe networkinterface(sas n=),andnetworkresynchronizationisperformedwithinformation fromthenetworkinterface(er=),(32)isverysimilarto(29).thetermssw+pappearing inthelatteraresubstitutedbythetermltfintheformer;allofthesetermsdependon thepropagationdelayandthenumberofnodesonthepathbetweensenderandreceiver. Whenthesenderistopologicallyclosetothereceiver,thecodingshapingdelayScisthemain contributioninbothcs CBRandTDP CBR.Forexample,ifScisaround3Tat3framesper second,andthepropagationdelaypandswitchingdelayswarenegligible,cs CBR'1ms. Ifapacketswitchednetworkwithtimedrivenpriorityisexploitedinthesamescenario,itis likelythatpacketsareforwardedbyonerouter,i.e.,l=3.moreover,ifsmallpacketsare used(pf=188bytes)andtherateofthecbrencoderis1.5mb/s, TDP CBR=CS CBR+SPack n+3tf'1+1+:375'cs CBR Thus,theshortdistancetransmissionofCBRMPEGvideooverapacketswitchednetwork withtimedrivenpriorityyieldsalmostthesameend-to-enddelayasthetransmissionovera circuitswitchednetwork. Ifsenderandreceiverarenotclose(i.e.,thepropagationdelayisnotnegligible),the dierencebetweentdp CBRandCS CBRdependsonthespeedandthenumberofintermediate nodesonthepath.thefasterthepacketswitches,thesmallerthedierence. AsuboptimalscheduleoftheTFsreservedforthevideoconferencecallincreasestheendto-enddelay;nevertheless,suboptimalschedulescanbeacceptedinordertoreducetheprobabilitythatcallsbeblocked. 1

101 Figure73:MPEGEncoderandPacketizationFunction. Encoder Buffer Packetization Depacketization Decoder Base Encoder Network betweensenderandreceiveristakenintoaccount. areconnectedbyadirectlinkisrstconsidered.then,amulti-hoppacketswitchednetwork theissuesrelatedtopacketization,inthissectionacongurationinwhichsenderandreceiver besegmentedinpacketsandthenreconstructedinthereceiver.inordertobetterfocuson 5.7AsynchronousPacketSwitching TransmissionofaCBRMPEGstreamoverapacketswitchednetworkrequiresthestreamto Control 5.7.1Single-HopPacketNetwork Encoder WiththesystemcongurationdepictedinFigure73,senderandreceiverareconnected throughadirectdedicatedlinkonwhichtheysenddatainpackets,asshownbythepacketizationfunctioninfigure73.thiscongurationisnotnecessarilyusedinrealapplications, Assumption5Transmissionofapacketcannotstartuntilallthebitsthataretobeputin butallowsustofocusonthedelayduetopacketization. packetizationfunction.givenapacketsizeps,whenthebuercontainspsbitsitisemptied inthesystem.thebitsproducedbytheencoderarestoredinthebuerofthesender payload.nevertheless,duetothisassumptionthepacketizationfunctionintroducesadelay headerofthepacketachecksequencethatcanbecalculatedonlyfromthewholepacket itspayloadhasbeenproducedbytheencoder. (asshowninfigure74)andapacketissentoverthenetwork. ThisAssumptionconformswithrealitysincemanyexistingprotocolsneedtoinsertinthe thestructureofthempegtransportstreamsothatasinglempegpacketisnotsentinto twopackets.moreover,weassumethatapacketissenteachtimethelastbitencodinga rateoftheencoderandtokenbucketsizea=ps.packetsizemustbechosenaccordingto senderatregularintervalsps=b,asshowninfigure74.thegeneratedtrachasthesame characteristicsofdataexitingaleakybucketwithtokengenerationratebequaltothetarget pictureexitstheencoder,independentlyfromitsdimension: SincebitsareproducedbytheCBRMPEGencoderatconstantrateB,packetsleavethe 11

102 Figure74:FullnessoftheBuerofthePacketizationFunction. Bits P s Time 2 P P s 3 s B B B Encoder Output Transmission Assumption6Apacketcontainsbitsencodingonlyonepicture. Thus,therealbehaviorofthepacketizationbuercanbedierentfromtheonedepictedin Figure74,butthebasicprinciplesdiscussedinthisSectionstillhold. Figure75:GenerationandTransmissionofBits;FullnessLeveloftheDecoderBuer. sender.eachbitofapacketexperiencesadierentdelayfromwhenitisgeneratedbythe P s C NoBueringintheDepacketizationFunctionWerstassumethatpacketsgetdi- CBRencodertowhentheyitisputonthelink. AsshownbyFigure75,packetsaresentatthefullspeedCofthelinkconnectedtothe T 2T 3T Time rectlytothedecoderbueruponarrivalatthereceiver,asshownbyfigure76;thedepacke- 12

103 Decoder Buffer Fullness Bits (I) tizationfunctionhastheonlytaskofstrippingothepacketheader2. P s C 1 (P) togetherwhendecodingstarts.accordingtoassumption6,thelastbitofapictureisthe interestedinthedelayexperiencedbythewholepicture.thesystemtargetdecoderspecied bythempegstandard,thebitsencodingapictureareremovedfromthedecoderbuerall Figure76:FullnessLeveloftheDecoderBuer. P 2 (P) P lastbitinapacket;intheworstcaseatimeps=celapsesfromwhenthislastbitexitsthe s B CBRMPEGencodertowhenitentersthedecoderbuer.Thus,decodingisdelayedbythe packetizationdelaydpthatisthetransmissiondelayofapacket,i.e.: Eachbitofapacketexperiencesadierentdelayduetopacketization;neverthelessweare s T S c +D p 2T S c +D p +T 3T Time buermustnotbeoversized.consideringalsothedelayintroducedbytheencoder/decoder system(sc,ss,andd)andthepropagationdelay,theend-to-enddelayisgivenby Maximumdimensionpacketsexperiencethepacketizationdelayentirelyinthesenderpacketizationfunction;packetsofdimensionP<Psspend(Ps?P)=Cinthedecoderbuer.Since Dp=Ps nofurtherdataarriveduringthistimeandafteritthewholepictureisretrieved,thedecoder ordertopossiblychecktheerrorcontrolcode.thisisnotrelevanttowhatisexplainedinthefollowing:it meansthatfigure76shouldbemodiedbyhavingeachpacketenterthedecoderbuerallatonce. 2Thisassumptionisnotrealisticasthepacketizationfunctionusuallyneedstobuerthewholepacketin Thepacketsizeshallbechosenassmallaspossible(possiblyequaltothedimensionof Async CBR=Sc+Ss+Dp+P+D+Pd (34) 13 C (33)

104 Encoder Output Transmission Figure77:GenerationandTransmissionofBitsandOutputoftheReceiverPacketization Decoder Input Function. MPEGpackets)inordertodecreasethepacketizationdelayandpossiblythedecodingtime ifthedecoderprocessesbitsassoonastheyenteritsbuer. P s B bitsarefeededtothedecoderonlyafterthewholepackethasbeenreceived.thus,bits bueringincomingpacketsandfeedingthedecoderbuerattheconstantrateb.received functioninthereceiverreconstructsthecbrmpegstream,asshowninfigure77,by BueringintheDepacketizationFunctionWeassumenowthatthedepacketization P P s s T 2T 3T Time exitingthecbrmpegencoderarebueredinthebuerofthepacketization(sender)and B C onegivenbyequation(33).thus,ifthedepacketizationfunctionreconstructsacbrmpeg depacketization(receiver)functionsforanoveralltime isdroppedandthelastbitofapicturemustnotbethelastofthepacketinwhichitis stream,theend-to-enddelayisincreased.theps=bincrementisnecessaryifassumption6 sent.infact,thesenderpacketizationfunctiondelaystherstbitofeachpacketbyps=b,as ThepacketizationdelayintroducedinthissystemcongurationisPs=Blargerthanthe Dp=Ps B+Ps underowswhen,forexample,thelastbitofapicturehappenstobetherstinapacket. showninfigure77;ifdecodingisnotdelayedbythesameamountoftime,thedecoderbuer 14 C (35)

105 Figure78:ArchitectureofaVideoconferencingSystemExploitingaCBRMPEGEncoder delay,thenetworkresynchronizationdelay,andtheexcessresynchronizationdelayertothe andanasynchronouspacketswitchednetwork. end-to-enddelaygivenbyequation(34): 5.7.2Multi-HopConguration Figure78showsthearchitectureofavideoconferencingsystemthatexploitsaCBRMPEG encoderandanasynchronouspacketswitchednetwork.thiscongurationaddsthequeueing Async Frame Encoder CBR=Sc+Ss+Dp+P+QM+Er+D+Pd Packetization Packetization Decoder (36) Grabber Adaptor wherepisthepropagationdelayonthelinksonthepathfromsourcetodestination,qm excessresynchronizationdelayintroducedbythereplaybuer(beingitinthepacketization isthemaximumqueueingdelayexperiencedbypacketsinthenetwork,er2[;q]isthe functionorinthedecoder)duetolackofsynchronizationbetweensenderandreceivernetwork delaydpisgivenbyequation(33)orequation(35)dependingonthebehaviorofthedepacketizationfunction(seesection5.7.1). dimensionofthereplaybuermustbe,atleast, Thisexpressionprovidesalsotheincrementrequiredinthedimensionofthedecoderbuer incaseitisusedtocompensatethenetworkdelayvariation. SincebitsareproducedattheconstantrateBandpacketsaresentatconstantpace,the interfaces(seesection3.4.1),qisthevariationofthequeueingdelay.thepacketization Thepacketizationfunctioncanimpactthestartupshapingdelay.Inmanypracticalcases 5.7.3PacketizationandStartupShapingDelay 2QB canbesucienttoproduceapacketworthofbits. timeps instantaneouslowbitproductionrate.ifthepacketizationfunctionisincludedintheclosed controlloopoftheencoder,thestartupshapingdelaycanbereducedoreveneliminated.the themeanbitproductionrateishigherthanthetargetrate;thusthestartupshapingdelay isessentialonlywhileencodingtherstmbsofapictureinordertocompensateapossible controlfunctiontakesintoaccountthatbitsarenotremovedfromtheencoderbuerfora B.Thus,eveniftheinitialbitproductionrateislow,theaverageoverthistimeinterval 15

106 RawVideoDed Raw=Fr Dedicated Link C+P+PdCS Raw=Sn+P+Sw+PdTDP Switching Circuit Sn=SAS Raw=Sn+LTf+Pd TimeDriven Priority n+(nr?1)tfpacketswitching Asynchronous Er+PdEr2[;Q] Async?Sh Raw=Fr Raw=Sn+Ps C+P+QM+ VBRMPEGDed VBR=CM+P+D+PdCS VBR=CM+Sw+P+ P+QM+Er+Pd P+Sw+D+Pd CS?TS VBR=CM+SCS n+ TDP?CxSc SSched n+ltf+d+pd VBR?I=CM+SAS SSched n2[;nt]async n2[;t] =CM+ D+PdC+P+QM+Er+ STS Async?TS QM+Er+D+Pd n+ps VBR=CM+Ps CBRMPEGDed D+Pd CBR=Sc+Ss+P+ P+D+Pd CS CBR=Sc+Ss+Sw+ Sn+Er+D+Pd CBR=Sc+Ss+LTf+ Er2[;Tc?Pf=B]Async Table2:SummaryoftheCongurationsConsideredinthisWork. Sn=SPack SPack SAS n2[;tc] n+sas np+qm+er+d+pd CBR=Sc+Ss+Dp+ Dp=Ps B+Ps C 16

107 keeptheend-to-enddelaybelow1ms.thisrequirementcomesfromthehumanhearing 6Summary Inthisworkweanalyzedtheend-to-enddelayofvideoconferencingoverpacketswitched networks.ourkeyndingsaresummarizedintable2.themaindesignobjectiveisto sensitivityfordelayslargerthan1msandtherequirementforlip-synchronization,i.e.,the needfortheaudioandvideotobesynchronized. (ii)networkand(iii)receiver.thecontrolofthebuersizescanhaveadverseconsequences: 1.Decreasingthebuersinsidethenetworkcanincreasethepacketlossinsidethenetwork Controllingthedelayalsorequirestocontroltheamountofbuersusedatthe(i)sender, Thus,whatwefoundandformulated,inthisstudy,aresometradeobetweentheperception qualityduetodelayandthereceivedpicturequalityduetolossandcompression. 2.LimitingtheMPEGencoderbuersizewilllimitthemaximumI-framesizeanddegrade Wefoundinterestingresultssomeofwhicharecounterintuitive.Thisinturnillustrates thecompressedmpegvideoquality. anddegradethequalityofthereceivedvideo. theimportanceofthissortofstudy. 1.Transmissionofrawvideodoesnotnecessarilyprovidetheshorterdelay.Thisisbecause 2.MPEGCBRencodingofaxedsceneintroduceslongdelay.ThisisbecausetheinterframecodingofP-framesrequiresfewbits,andtherefore,asingleI-framecanuseall thecbrcapacityallocatedtothegroupofpicture(gop).asaresult,thei-frame transmissionwilllasttheentiregopperiod. 3.Usingasynchronouspacketswitchingwithstatisticalmultiplexingischallenging.This ofthetransmissiontimeneededforlargenumberofbitsofhighdenitionpictures. 4.Ifthecapturecardandthedisplaybuerareusingthesamereferenceclockthedelay replaybueranddistortionofthevideoviewedbytheuseratthereceiver. largedelay,andsmallreplaybuerwouldresultinoccasionalunderow/overowofthe isbecausethedistributionofthedelayvariationorjitterinsidethenetworkcanbelarge reductionof1msandfor3framespersecondthedelayreductionis33ms. canbedecreasedbyonevideoframeperiod.for1framespersecondthismeansdelay andwithheavytail.thus,conservativedesignwithlargereplaybuerwouldresultin 5.MPEGVBRvideooverasynchronouspacketswitchednetworkrequireshighequivalent capacityoreectivebandwidth,whichcanbetoomuchtobewastedoverwidearea examplemotion-jpeg. links.therefore,insomecaseitmaybebettertouseonlyi-frames,i.e.,tousefor 17

108 end-to-enddelaybelowhumanperception,therearefourrequirements: 1.Thecapture(framegrabber)cardanddisplay(video)adaptershouldhaveacommon 2.Thepicturesshouldbesentrightafterthecompressioniscompletedasavariablebit Inconclusion,MPEG-basedvideoconferencingispossible,however,inordertokeepthe timereferenceclock. havethefollowingpropertiesforvbrmpeg: 1.Boundof25secondsonthejitter. 3.Thenetworkjittershouldbecontrolledwithawelldenedbound. Wealsoshowedthatwithtimedrivenpriorityandcomplexschedulingitispossibleto rate(vbr)stream. 2.Nolossevenifthelinkisfullyutilized. 3.Theend-to-enddelayisdominatedbythepropagationdelayplusL125seconds(L videoframeperiod.ifthesceneisslow,thisdelaycanbeaslargeasthedurationofa encodermustbedeployedwhichintroducesacodingshapingdelaylargerthanthe obtainedoveracircuitswitchednetwork.onacircuitswitchedconnectionacbr groupofpictures. dependsonthenumberofhops).thisdelayisshorterthanthedelaythatcanbe Acknowledgments WethankPrasoonTiwariforprovidinguswiththesoftwareMPEGencoderdvdencand PeterWesterinkforhiskindandusefulhelpinunderstanding,modifying,andoperatingthe encoder. AListofAchronymsandSymbols bmbmacroblocklevelratecontrolfactorofmacroblockmb. BtargetrateforMPEGCBRencoderorrateofcircuitswitchedconnectionsusedtosend Adimensionofthetokenbucketinaleakybucketshaper. CMmaximumcodingdelay. Ccapacityofphysicallinks. rawandencodedvideo. 18

109 CBRConstantBitRate. DCTDiscreteCosineTransform. ^cmb i;jdctcoecient(i;j)ofmbmb. DTidelayexperiencedinthenetworkbyframei i;jquantizeddctcoecient(i;j)ofmbmb. Ddecodingtimeordecoderdelay. DRMmaximumdelayexperiencedinthereplaybuerormaximumtransmissionsynchronizationdelay. DTmminimumdelayexperiencedbyaframeinthenetwork. DRidelayexperiencedinthereplaybuerortransmissionsynchronizationdelayexperienced byvideoframei. DTMmaximumdelayexperiencedbyaframeinthenetwork. ErExcessresynchronizationdelay Fnumberofbitsencodingaframe; Fmlowerboundonpicturedimension, GOPGroupOfPictures. Gglobaldistortionlevelthatischosenonapicturebypicturebasis FPdimensionofaP-frame. FIdimensionofanI-frame, Frnumberofbitsencodingarawpicture, MBMacroBlock. Mnumberofvideoframeperiodsinthetimecycle. Jnnetworkdelayjitter. Nnumberofvideoframesinagroupofpictures. Lnumberoftimeframesneededtotransferapacketfromsendertoreceiverwithtime drivenpriority. 19

110 Nsnumberoftimeframesinwhichbitshavebeenallocatedtosendaraworencodedvideo pmbmblocalactivityfactorofmbmb. NrnumberoftimeframesbetweentherstandthelastTFreservedforthetransmission Ppropagationdelay. ofapicture. framewithtimedrivenpriority. Pssizeofapacket. Qqueueingdelayinthenetwork; qfijpg i;jelement(i;j)ofquantizationmatrixfori-framesorp-frames. QoSQualityOfService. Qmbquantizationparameterofmacroblockmb. QMmaximumqueueingdelay. Qmminimumqueueingdelay, SCS nnetworkshapingdelayduetolackofsynchronizationbetweenapplicationandnetwork Snnetworkshapingdelay. Sccodingshapingdelay. STS nnetworkshapingdelayduetocircuitbandwidth. nnetworkshapingdelayduetotracshapingattheboundariesofasynchronouspacket interface. Swswitchingdelay. Ssstartupshapingdelay; switchednetworks. Tctimecycleduration. Tvideoframeperiod. Smsminimumstartupshapingdelay. TFTimeFrame. 11

111 VBVVideoBuerVerier. Tfdurationofatimeframe. BTheMPEGEncodingStandard Avideosequenceisdigitalizedbyconvertingintodigitalformpictures(orvideoframes)ata VssizeoftheVBV. Vbmaximumbacklogintheencoderbuer. framespersecondintheusand25framespersecondineurope. bitsusedtoencodeeachpixeldeterminesthepictureresolution.picturesaredescribedby threecomponentswhichareeithertheintensityofthered,green,andblue(rgbformat)or theyhadbeencaptured.forexample,goodqualityvideoisdigitalizedanddisplayedat3 xedpace.digitalvideoisreplayedbydisplayingpicturesatthesamexedpaceatwhich theintensityofluminanceandchrominance(yuvformat)ofthepixels. Eachpictureisdividedinpixels;thenumberofpixelsineachpictureandthenumberof broadcasting(e.g.,tvbroadcastingandvideoondemand). (1)storageandsubsequentretrievalanddisplay(e.g.,ondigitalvideodisks),and(2)oneway todigitallyencodeandcompressvideoandrelatedaudiomainlyfortwoapplicationelds: schemeshavebeenproposedwhichexploittheredundancyinside(spatialredundancy)and among(temporalredundancy)picturesinordertoreducethenumberofbitsneededtoencode avideosequence.themovingpicturesexpertgroupcodingstandard[11,25,8]specieshow Thiscodingformatrequiresahugeamountofbitstoencodesequences.Manycompression ferencingpurposes,eventhoughtheencodingalgorithmiscomputationallyintensiveandit ingandexplainingthecharacteristicswhichareparticularlyrelevanttothiswork. andoutputbitratesthroughthesamecodingprinciples. wasnotdesignedforreal-timeoperation.thempegstandardsupportsvariousresolutions ThisAppendixdescribesthebasicprinciplesofMPEGstandardwiththeaimofhighlight- Nevertheless,theMPEGstandardhasbeenproposedforbeingappliedalsoforvideocon- Compressionschemesarebasedonreductionoftemporalandspatialredundancyandquantizationofvaluesusedtocodeimages.Whileredundancyreductionisalosslessprocess, referenceone.thisintroducesstrongdependencyamongpicturesandpicturescodedby dierencecannotbedecodedanddisplayedpriortodecodingthereferenceimage.thisturns Temporalredundancyiseliminatedbycodingthedierencebetweenthepictureanda B.1GeneralPrinciples quantizationdegradesimagequality. 111

112 outtobeaproblemwhentransmittingacodedsequencebecausethelossof(partof)a referencepicturecausesallthereferencingimagestobecorrupted. temporalredundancyiseliminated: TheMPEGstandarddenesthreecategoriesofpicturesaccordingtowhetherspatialor Figure79:BlockDiagramoftheMPEGEncodingAlgorithmforI-Frames. I-frames(Intra-codedframes)arecodedbyeliminatingspatialredundancyonly.Hence Sequence Entropy of Raw DCT Quantization Images Encoding Stream P-frames(Predictive-codedframes)areencodedwithreferencetothepreviousimage. theyaredecodedbythemselves,withoutreferencetoanyotherpicture. Quantizer B-frames(Bidirectionallypredictive-codedframes)arecodedwithreferencetothepreviousandfollowingI-frameorP-frame.Thisallowssimilaritieswithafutureimage (possiblynotpresentwithinapreviouspicture)tobeexploitedtoreducetemporal redundancy.thiscodingyieldsverygoodcompression,butintroducesalargedelay rstimageisintra-codedandthefollowingpicturesinthesamegoparepredictive-coded21. thelowertheframerate,thelargerthedelayintroduced.sinceinthisworkweare becauseab-framecannotbecodeduntilthefuturereferencepicturehasbeencaptured; B.2Intra-codedPictures Whenavideosequenceisencoded,imagesaregatheredinGroupsofPictures(GOPs):the codingisnotexploitedandfurtherexplained. consideringinteractiveapplicationswhichrequirelowdelays,bidirectionalpredictive RawpictureswhicharetobeencodedwiththeMPEGstandardaredigitalizedinformatsimilartoYUV.Eachimageconsistsofaluminancecomponentandtwochrominancecomponents. Theformerhastwiceasmanypixelsasthelattertwocomponents. B.2.1DiscreteCosineTransformation basicstepsofintra-codingarebrieydescribed. Intra-framecodingyieldscompressionbyeliminatingspatialredundancy.InthisSectionthe 21Ingeneral,theGOPcontainsapatternofB-framesinterleavedbyP-frames,e.g.,IBBPBBPBBPBB. 112

113 inthefollowing)areidentied.thediscretecosinetransformation(dct)isperformed independentlyoneachblockandtherepresentationoftheblockinthefrequencydomainis obtained.thedcttakesasinput648-bitintegervaluessx;yrepresentingthevalueofeach pixelintheblockandproduces6412-bitintegersci;jwhicharetherepresentationoftheblock inthefrequencydomain[8]:ci;j=dct(sx;y)i;j;x;y2[;7] Oneachofthethreecomponentsofapicture8x8blocksofpixels(namedsimplyblocks block.theothercoecients,ac-coecients,representtheseparationoflinesineitheror bothdirections.e.g.,c7;isthehighestfrequencyappearingintheverticaldirectionand representstheclosestseparationofverticallinesintheblock.ci;iisproportionaltothe amountofsquaresintheblockbecausetheyarecharacterizedbythesamespatialfrequency inbothdirections.e.g.,thelargerthenumberofisolatedpixels,thehighestthevalueofc7;7. Thec;coecientiscalledDC-coecientandrepresentsthefundamentalcolorinthe compressionasthevaluestobeencodedarenotuniformlydistributed. groupedaccordingtothelevelofdetailintheblock.lowspatialfrequencycoecientscontaintheinformationontheoverallaspectoftheblock;highfrequencycoecientscontain theinformationconcerningthedetailsoftheblock.lowfrequencycoecientsusuallyhave largervaluesthanhighfrequencyones.theencodingstep(seesectionb.2.3)providesbetter Moreover,frequencydomaincoecientscanbecodedwithdierentprecisionaccording TheDCTprovidesenergycompactionbecausetheinformationcontainedinablockis tohowsensitivethehumaneyeistoeachofthem.lowfrequencyonesaremoreimportant indeterminingtheperceivedqualityofanimage,i.e.,theymustbeencodedwithminimal B.2.2Quantization EachDCTcoecientisquantizedwithadierentquantizerbecauseeachoneofthemweighs quality,andthustheycanbeencodedwithlessaccuracyandhighercompressionratio(coarser informationloss(nequantization).highfrequencycomponentsarelesscriticalfortheimage elementsoftheblockbythecorrespondingelementofthequantizationmatrix.thus,the higheristhevalueofquantizationelements,thecoarseristhequantization,i.e.,thegreater gerscalledquantizationmatrix.eachelementofthematrixindicatestheaggressivenessof quantizationonthecorrespondingdctcoecient:quantizationisperformedbydividingthe dierentlyontheimagequality.thequantizationisdrivenbyan8x8matrixof8-bitinte- isincludedatthebeginningofthempegvideostream. istheyieldedcompression.theresultofthisstepisa1-bitintegervalue[8]. Thequantizationmatrixischosenaccordingtothenatureofthescenetobeencodedand 113

114 c, c Thisstepproducesacompressedstreamofbitsthroughtwosubsteps[7,27]: B.2.3EntropyEncoding 1.Run-length.Thecoecientsobtainedbythequantizationarerun-lengthencodedwith Figure8:EncodingOrderforQuantizedDCTCoecients. asequenceofcouplesofsymbols: therstsymbol,isdividedintwoparts:,7 c 7, c Asaconsequence,foreachnon-zerocoecientanumberofbitsbetween8to18isused. thesecondsymbolisthecoecientvalueexpressedonsizebits;thevaluezero doesnothavetobecodedasitisrun-lengthencoded. (b)size(4bits)isthenumberofbits(betweenand1)usedtoencodethevalue (a)run-length(4bits)representsthenumberofcoecientswithvaluezero, ofthenextnonzerocoecient; 7,7 TheDC-coecientc;isencodedusingapredictivecodingtechnique,i.e.,thedierence oneisencoded.asthetwoquantizedcoecientsarebothrepresentedon1bits,the betweenthevalueofthedc-coecientofthecurrentblockandtheoneoftheprevious resultisencodedonamaximumof11bits. 2.Humanencoding.Theintermediatesymbolsequenceproducedbytherun-lengthencoderistransformedbyreplacingtherstsymbolineachcoupleofsymbolswitha similarbigvalues,whilehighfrequencyoneshavesmallervalues;theprobabilityofzero togetherallowsrun-lengthencodingtodeliverbettercompression. valuedcoecientsishigheramongthehighfrequencyonesandhavingthemgrouped beingconsideredinincreasingfrequencyorder.lowfrequencycoecientstypicallyhave TheAC-coecientsareencodedaccordingtothezig-zagordershowninFigure1,thus 114

115 ApplicationoftheMPEGencodingschemeoftenrequirespicturedimensiontobecontrolled: AsshowninSection4.1(seeFigure2)dimensionofencodedpicturesishighlyvariable. B.2.4ControllingI-frameDimension variablelengthcodeobtainedfromahumantablethathasbeenspeciedforeach e.g.,cbrmpegencoding(section5)andtransmissionofvbrmpegencodingthrough componentofthepicture. whenintra-codingapicture(i-framedimension)dependsonthefollowingfactors: packetswitchednetworkswithtimedrivenprioritysection4.3).thenumberofbitsobtained Resolutionofthedigitalizedimage:itisdenedforeachconferenceanditisnotusually Contentofthedigitalizedimage:iftheimagehasfewdetails,manyhighfrequencyDCT Quantizer:thehigherthequantizationstepsize,thesmallerthevaluesofthequantized changedontheyduringthesession. Varianceoftherstsymbolsproducedbyrun-lengthencoding:Humanencodingisas coecientshavesmallvalues.quantizationreducessmallvaluestozeroandrun-length encodingdelivershighcompression. Amongtheforegoingfactors,onlythequantizercanbechangedonpurposeanddynamicallytocontrolthecompressionratioandhenceI-framedimension.Thequantizationstepsize quantizationisregulatedseparatelyforeachcoecientinablock,thusallowinganetuning overawiderangeofcompressionratios(andimagequalities). ofzerovaluesyieldevenhighercompressionthankstorun-lengthencoding. symbolhasahighoccurrenceprobability. moreeectiveasthenumberofdierentrstsymbolsinthesequenceislowandeach coecients.smallerquantizedvaluesarehumanencodedwithfewerbits;sequences thatallowstheintendednumberofbitstobeproduced,whilekeepingimagequalityuniform. canbepossiblymodiedonablockbyblockbasis.sinceaquantizationmatrixisexploited, ofiterationstheintendeddimensionisobtainedforthei-frame. Theproblemissometimesaddressedwithreiterativeapproachesthatquantizeapicturemore thecoecientsarere-quantizedwithacoarser(ner)stepandre-encoded.afteranumber Ateachiterationthepictureisencoded;iftheyieldedamountofbitsistoobig(small),all thanoncewithdierentquantizersinordertoidentifythemostsuitablequantizationstepsize. OneofthemostdebatedtopicsincontrollingI-framesdimensionisthechoiceofaquantizer bitsthataregoingtobeproducedbycompressingani-framewithouthavingtoperformthe codingprocessuptotheend. beingsuitedtoreal-timeencoding.someheuristicscanbeusedtopredicttheamountof Thisapproachiscomputationallyhardanditdoesnotboundtheencodingtime,thusnot 115

116 1.TheDCTisperformedonthewholepicturebeforestartingquantizingandencoding 2.Duringquantizationsomeverysimpleandcomputationallylightalgorithmisrunin thecoecients.byaswiftanalysisoftheyieldedcoecientsthecompressionfactor providedbyhumanencodingcanberoughlyestimatedandasensiblequantization ordertogetanestimateoftheamountofbitsexpectedasoutcomeoftheencoding stepsizecanbechosen. 3.BlocksaregroupedinsetsandanequalshareofthetargetI-framedimensionisassigned toeachset.afteralltheblocksinasethavebeenprocessed,theyieldednumberofbits systemhasmorethanoneprocessor,morethanonequantizationprocesses,eachwith dierentstepsize,canberuninparallel. withalargerquantizationstepsizebeforeencodingtheoutcomeofquantization.ifthe step.ifthei-frameisexpectedtogrowtoolarge,thecoecientsarequantizedagain whoseblocksareencodedwithasmallamountofbits;morecomplexpartsarecharacterizedbyblocksrequiringahugeamountofbits.duetothepseudo-randomorder iscomparedagainsttheassignedshare.ifitissignicantlygrater(smaller)thanthe Blockgroupingmustbepseudo-randombecauseanimagecanhaveverysimpleparts withwhichblocksareencoded,theimagesendingcannotstartbeforethewholepicture hasbeenencoded. target,thecoecientsarere-quantizedwithalarger(smaller)quantizerandre-encoded. B.3Predictive-codedPictures Constraintsmustbeimposedonthevariabilityofthequantizationstepsizewithineachimage inordertopreventnon-uniformquality. Predictive-codingreducesbothspatialandtemporalredundancy.Picturesareencodedby Theproposedheuristicscanbeusedsingularlyorincompositiontogetbetterresults. thersttwoallowtemporalredundancytobereducedbeforegoingthroughtheverystepsof referencepictureinthefollowing.thissectiondescribesthebasicstepsofpredictive-coding; intra-coding. B.3.1MotionEstimation referencetothepreviousone(havingitbeenintra-orpredictive-coded)whichiscalledthe thetwochrominancecomponents).thereferencepictureissearchedforamb\similar"to (MBs)whichconsistof6correspondingblocks(4ontheluminancecomponentandoneon composedof4blocks;oneblockoneachofthechrominancecomponentscorrespondstothe 16x16-pixelsquareontheluminancecomponent.Motionestimationoperatesonmacroblocks Ontheluminancecomponentofthedigitalimage,16x16-pixelsquaresareidentied,each 116

117 theonebeingencoded;thepossiblyfoundmbiscalledapredictor.thedierencebetween thegivenmbandthepredictorisencodedasarepresentationoftheformer. theareaofthereferencepicturearoundthelocationcorrespondingtothembbeingencoded. Thealgorithmbehavioriscontrolledbytwoparameterswhichdetermineitsperformancein termsofrunningtimeandcontributiontotheoverallpicturecompression. 1.ThesearchrangeidentiestheareaaroundthecurrentMBpositioninwhichthereferencepictureissearchedforthepredictor.Thelargerthesearcharea,thehigherthe Analgorithm,notspeciedbytheMPEGstandard,isruninordertoidentifyapredictorin 2.ThesimilaritycriteriaaimsatchoosingaspredictortheMBwhichisgoingtoprovide complexthesimilaritycriteria,thebetterthecompressionobtained,buttheharderthe thehighestcompressionwhenthedierencefromtheactualmbisencoded.themore thesearch.thesearchrangecanbechangedonapicturebypicturebasis. probabilityofndingaverywellmatchingmb,thusyieldinggoodcompression.onthe computation. otherhand,thelargeristhesearcharea,thelongerittakestothealgorithmtocomplete wholeencodingprocess. B.3.2MotionCompensation WhenapredictorisfoundfortheMB,thedierencebetweeneachpixelandthecorresponding oneinthepredictoriscomputed.sincetwo8-bitintegersaresubtracted,theresultisa9-bit Motionestimationisthemostcomputationallyintensiveandtimeconsumingstepofthe block,i.e.,themotioncompensationstepisskipped. integer.ifambsimilarenoughisnotfound,eachblockofthembisencodedasani-frame sentationwhichallowsthepredictortobeidentied.motionisrepresentedthroughamotion previousmb. B.3.3DCT MBbeingencoded.Themotionvectorisencodedasdierencefromthemotionvectorofthe vector,i.e.,thebidimensionalosetofthepredictorfromthepositioninthepictureofthe ThecompleteencodingofamotioncompensatedMBencompassesalsoamotionrepre- predictor.ifnopredictorhasbeenfound,thedctisappliedonthepixelsofthembitself. thedctisperformedonthedierencesbetweenthepixelsofthegivenmbandthoseofthe TheDCTisperformedontheoutputofthepreviousstep.Ifapredictorhasbeenfound, 117

118 thembhasbeenfound)thembisnofurtherprocessedanditisrepresentedinthempeg besmallerthanthequantizationstepsize. thequantizermusthavenodeadzone,i.e.,therangeofvaluesthatarequantizedtozeromust andintra-codedmbs.themotioncompensatedmbslikelyhavesmallcoecientsandthus B.3.4Quantization TheresultoftheDCTisquantizedusingdierentquantizersformotioncompensatedMBs AllthecoecientsofmotioncompensatedMBsareencodedusingrun-lengthencodingand B.3.5EntropyEncoding streambyaspecial6-bitcode. IfthecoecientsofamotioncompensatedMBareallzero(i.e.,apredictoridenticalto Humanencoding(i.e.,theDC-coecientisnottreateddierently).Blocksofnonmotion compensatedmbsareencodedaccordingtotheintra-codingprocess. Thenumberofbitsobtainedwhenpredictive-codingapicture(P-framedimension)depends onthefollowingfactors: B.3.6ControllingP-frameDimension Dynamicsofscenes:ifscenesarestatic,subsequentpicturesdonotchangetoomuch, Searchrangeandsimilaritycriteriausedtondthepredictor.Bothparameterscouldbe Resolutionofimages:seeSectionB.2.4. motioncompensationishighlyeective,andp-framesaresmall. Quantizer:seeSectionB.2.4. usedtocontrolthecompressionratio,buttheyareemployedtooearlyinthecompression Varianceoftherstsymbolsproducedbyrun-lengthencoding:seesectionB.2.4. process.thus,ononeside,reiterationswithdierentvalueswouldbeimpracticabledue tothelargeamountofcomputationrequiredbymotionestimation.ontheotherside, ThesameactionsproposedinSectionB.2.4forcontrollingthesizeofI-frames,canbe isnottrivial. therelationshipbetweenchangingtheseparametersandthenumberofgeneratedbits motioncompensatedmbs,itispossibletoroughlyestimatetheamountofbitsthataregoing tobeproducedandconsequentlychoosethesearchrangeandthequantizerstobeused. bemotioncompensatedcanberoughlyestimatedbyestimatingtheoverallsimilarityofthe twoimages.havingsomeknowledgeorstatisticsaboutthenumberofbitsneededtoencode takenforp-frames.inaddition,beforeencodingap-frame,thepercentageofmbswhichcan 118

119 B.4SignaltoNoiseRatio reconstructedfromthempegstream.thesignaltonoiseratioisscaledaccordingtothe wheresiisthevalueoftheithpixelbeforeencoding,while^siisthevalueofthesamepixel Thelossduetotheencodingprocesscanbequantitativelymeasuredbythepeak-to-peak maximumvalueofthepixel(peak-to-peak)whichis255. SignaltoNoiseRatio(SNR)whichiscalculatedas SNR=1log12552 B.5PacketizationandStreamingPi(si?^si)2 (37) stampandadecodingtimestampwhichindicatethetimeatwhichtherstaccessunit22 streamtowhichthepacketbelongs.moreover,theheadercancontainapresentationtime TheMPEGstandard[11]denesanelementarystreamforcarryingeachmedia:video,audio, elementarystreamwiththempegstandardischeckedbyfeedingasystemtargetdecoder (STD)withthestreamandverifyingthatitsbuerdoesnotoveroworunderow. inthepacketisintendedtobepresentedanddecoded,respectively.thecomplianceofan ordata.asystemstreamisdesignedtocombineanumberofelementarystreams. Eachelementarystreamispacketized;eachpackethasaheaderidentifyingtheelementary fromthesystemstream. Thebueroverowsifthereisnotenoughspacetostoreawholepacketwhenitisextracted astheygettothedecoder.thebuerisemptiedbyretrievingawholeaccessunitatthe underowsifthecompleteaccessunitisnotyetinthebuerbeforeitsretrievalisscheduled. timespeciedbythedecodingtimestampcontainedinthepacketheader.thestdbuer Packets,possiblybelongingtodierentstreams,aregroupedinpacks,eachprecededby TheSTDbuerislledbythepacketsofthecorrespondingelementarystreamassoon atthedecoder. timeofarrivalofthelastbyteofthetimestampitself.itisusedtorecovertheencoderclock repeatedinsomeotherpacks.thesystemheadercarriesinformationthatisrelevantto apackheaderwhichcontainsinformationabouttherateofthestream.moreover,thepack elementarystream.thecurrentsizeofthestdbueroptionallyincludedinpacketheaders headercontainsatimestamp,calledthesystemclockreference,whichindicatestheintended thewholestreamandcontainsthemaximumsizeofthestdbuertobeusedwitheach mustbesmallerthanthemaximumvaluepresentinthesystemheader. 22Forexample,thetheaccessunitinavideostreamisapicture. Therstpackofthesystemstreamcontainsasystemheaderthatcanbeoptionally TheMPEG-2standardprovidestwokindsofsystemstream: 119

120 References 1.TheProgramStreamisdesignedforerror-freeenvironments,e.g.,storageandretrieval 2.TheTransportStreamisintendedforexploitationinlossyenvironments(e.g.,transmissionovercomputernetworks)andpacketshavexedlength(188bytes). ofvideoscenesfromdigitalvideodisks.itusesvariablelengthpackets. [1]M.Baldi,Y.Ofek,andB.Yener.Adaptiverealtimegroupmulticast.Submittedto [2]M.Butto,E.Cavallero,andA.Tonietti.Eectivenessofthe`leakybucket'policing [3]A.Demers,S.Keshav,andS.Shenker.Analysisandsimulationofafairqueueingalgorithm.ACMComputerCommunicationReview(SIGCOMM'89),pages3{12,1989. mechanisminatmnetworks.ieeejournalonselectedareasincommunications, IEEEINFOCOM'97. 9(3):355{342,April1991. [5]W.DingandB.Liu.RatecontrolofMPEGvideocodingandrecordingbyratequantizationmodeling.IEEETransactionsonCircuitsandSystemsforVideoTechnology, [4]W.DingandB.Liu.JointencoderandchannelratecontrolofVBRvideooverATM networks.invisualcommunicationsandimageprocessing,volume2668,pages392{ 47,1996. [7]B.Fufht.Asurveyofmultimediacompressiontechniquesandstandards.PartI:JPEG [6]J.Feng,K.-T.Lo,H.Mehrpour,andA.E.Karbowiak.Celllossconcealmentmethodfor standard.realtimeimaging,(1):49{67,1995. MPEGvideoinATMnetworks.InGLOBECOM'95,pages192{1924, (1):12{2,February1996. [1]C.HorneandA.Puri.Videocodingwithadaptivequantizationandratecontrol.In [8]D.LeGall.MPEG:Avideocompressionstandardformultimediaapplications.CommunicationsoftheACM,34(4):47{58,April1991. [9]MovingPicturesExpertsGroup. [11]ISO/IEC.Informationtechnology-Codingofmovingpicturesandassociatedaudiofor JTC1/SC29/WG11/N4,April1993. VisualCommunicationsandImageProcessing,volume1818,pages798{86,1992. digitalstoragemediauptoabout1.5mbit/s.internationalorganizationforstandardization,1993. MPEG-2testmodel5. 12

121 [12]S.JungandJ.S.Meditch.AdaptivepredictionandsmoothingofMPEGvideoinATM [13]G.Karlsson.Asynchronoustransferofvideo.IEEECommunicationsMagazine,pages [14]C-S.Li,Y.Ofek,A.Segall,andK.Sohraby.Pseudo-isochronouscellswitchinginATM networks.inieeeinternationalconferenceoncommunications,pages832{836,1995. [15]C-SLi,Y.Ofek,andM.Yung.\Time-drivenpriority"owcontrolforreal-timeheterogeneousinternetworking.InIEEEINFOCOM'96, {126,August1996. networks.inieeeinfocom'94,pages428{437,1994. [17]E.Linzer.ArobustMPEG-2ratecontrolalgorithm.Unpublishedtechnicalreport,IBM- [16]L.-J.Lin,A.Ortega,andC.-C.J.Kuo.Agradientbasedratecontrolalgorithmwith [18]W.LuoandM.ElZarki.AdaptivedatapartitioningforMPEG-2videotransmission applicationstompegvideo.pages392{395,1995. [19]P.PanchaandM.ElZarki.MPEGcodingforvariablebitratevideotransmission.IEEE T.J.WatsonResearchCenter,department924A. [2]D.Reininger,G.Ramamurthy,andD.Raychaudhuri.VBRMPEGvideocodingwithdynamicbandwidthrenegotiation.InIEEEInternationalConferenceonCommunications, pages1773{1777,june1995. overatmbasednetworks.ininternationalconferenceonimageprocessing,volume1, pages17{2,october1995. [21]I.E.G.RichardsonandM.J.Riley.UsageparametercontrolcelllosseectsMPEG CommunicationsMagazine,pages54{66,May1994. [22]R.M.Rodriguez-Dagnino,M.R.K.Khansari,andA.Leno-Garcia.Predictionofbitrate [23]M.Simon,P.Villegas,J.Caballero,andM.Roser.Ageneralapproachtooutputrate 9(3):35{313,April sequancesofencodedvideosignals.ieeejournalonselectedareasincommunications, video.inieeeinternationalconferenceoncommunications,pages97{974,june [24]S.SinghandS-S.Chan.Amulti-levelapproachtothetransportofMPEG-codedvideo controlinvideocoding.invisualcommunicationsandimageprocessing,volume193, pages246{254,1993. overatmandsomeexperiments.inieeeglobecom'95,pages192{1924,

122 [27]G.K.Wallace.TheJPEGstillpicturescompressionstandard.Communicationsofthe [25]R.SteinmetzandK.Nahrstedt.Multimedia:computing,communications&applications. [26]A.SultanandH.A.Latchman.AdaptivequantizationschemeforMPEGvideocoders volume2668,pages181{188,1996. PrenticeHall,UpperSaddleRiver,NJ7458,1995. [28]L.Wang.RatecontrolforMPEGvideocoding.InVisualCommunicationsandImage Processing,volume251,pages53{63,1995. ACM,34(4):3{44,April1991. basedonhvs(humanvisualsystem.invisualcommunicationsandimageprocessing, 122