FacultyofComputingandInformationTechnology DepartmentofRoboticsandDigitalTechnology TechnicalReport93-11

FacultyofComputingandInformationTechnology DepartmentofRoboticsandDigitalTechnology TechnicalReport93-11 TheTheoryofCCITTRecommendationH.261, p64kbit/s"andreviewofsuchacodec \VideoCodecforAudiovisualServicesat December16,1993 AndrewMoore Enquiries:- ClaytonVIC3168 Australia tr.coord@rdt.monash.edu.au MonashUniversity RoboticsandDigitalTechnology TechnicalReportCoordinator +6135653402

AbstractandKeywords Contents 2TheH.261Standard 1Introduction 2.3VideoCodingAlgorithm::::::::::::::::::::::::::::9 2.2VideoFormat::::::::::::::::::::::::::::::::::6 2.1Background:::::::::::::::::::::::::::::::::::6 4 2.3.1SourceCoder/Decoder:::::::::::::::::::::::::9 65 2.4OthercomponentsinH.261::::::::::::::::::::::::::15 2.3.3DierentialPulseCodeModulated/DiscreteCosineTransformHybrid(DPCM/DCT)::::::::::::::::::::::::::14 2.3.2DiscreteCosineTransform(DCT):::::::::::::::::::12 2.6Entropyencoding::::::::::::::::::::::::::::::::23 2.5EncodedImageDataStructures::::::::::::::::::::::::21 2.4.1MotionCompensation:::::::::::::::::::::::::15 2.4.2Filter:::::::::::::::::::::::::::::::::::19 3AnH.261implementation 3.6TransmissionofpacketvideoviaTCP/IP:::::::::::::::::::30 3.5TheDCTandQuantization::::::::::::::::::::::::::26 3.4MovementDetection::::::::::::::::::::::::::::::26 3.3Inter/Intracodingdecisions::::::::::::::::::::::::::26 3.2VideoPiximaginghardware::::::::::::::::::::::::::25 3.1Overview:::::::::::::::::::::::::::::::::::::25 4Conclusion 3.7Summary::::::::::::::::::::::::::::::::::::33 3.6.1Locatingtheboundarybetweenimages::::::::::::::::31 3.6.2ControllingtheRateofFlow::::::::::::::::::::::31 Bibliography 5Acknowledgements 3.6.3ChoiceofQuantizer:::::::::::::::::::::::::::31 35 36 34 1

ListofTables 3.1EectsoncodingduetonumberofDCTcoecientsused::::::::::30 2.1ParametersfortheCCITTVideoFormats::::::::::::::::::7 2

ListofFigures 2.1StructureofaMacro-BlockshowingsamplingstructureofLuminanceand 2.4PeriodicnatureofDCTandDFTtransforms:::::::::::::::::13 2.6Complexmotionexample::::::::::::::::::::::::::::17 2.5Simplemotionexample:::::::::::::::::::::::::::::16 2.3BlockdiagramoftheSourcecoder::::::::::::::::::::::11 2.2Blockdiagramofthecompletevideocodec::::::::::::::::::10 2.7Complexmotion,withobjectrevelation::::::::::::::::::::18 Chrominance::::::::::::::::::::::::::::::::::8 2.10ArrangementofGOBsinanimage::::::::::::::::::::::22 2.9Hierarchicalnatureofthedatastructuresofthecoder::::::::::::21 2.8Non-integermotion:::::::::::::::::::::::::::::::20 3.1UsinglessDCTsamples::::::::::::::::::::::::::::29 2.12ArrangementofblocksinaMacroBlock:::::::::::::::::::22 2.11ArrangementofMacroBlocksinaGOB:::::::::::::::::::22 2.14Humancodingexample::::::::::::::::::::::::::::24 3.2TheeectofQuantizerstepondatarate:::::::::::::::::::32 2.13Zig-zagsampling::::::::::::::::::::::::::::::::23 3

Abstract ThispapergivesinformationaboutH.261,includingbackgroundonthedesignofthe H.261specication.TheimplementationcoveredisanH.261codecdesignedtorunusing Keywords H.261,codec,Internet,video,videoconference,DCT includesinformationaboutandresolutionofproblemsthatresultfromusingtheinternet protocolsasatrasmissionsystem. theinternetprotocolsasitstransmissionsystem.thediscussionoftheimplementation 4

Introduction Chapter1 Objective ThispaperdescribestheoperationaltheoryoftheCCITTRecommendationH.261[3,16] discussedinadditiontoinformationaboutandresolutionofproblemspresentbecauseof ponentsoftheh.261specication,nallycoveringparticularsonthenalcodinginthe andthenproceedstocoveranimplementationofacodecsatisfyingtheh.261standard. theuseofaninternetprotocolbasisforcommunications. sectioninformationonthebasicmakeupofasoftwareimplementationofh.261willbe willreviewanimplementationoftheh.261asaninternetprotocolbasedcodec.inthis H.261standard.Indealingwiththeimplementation[26],thesecondpartofthispaper ThispaperwillcoverthebackgroundoftheH.261,itsdesign,informationoncom- 5

TheH.261Standard Chapter2 2.1Background Videocodinghasahistorythatstartedinthe1940swhenC.E.Shannonandothersrst postulatedtheprobabilisticviewofinformationanditsrepresentation,transmissionand compression[9,25,7].thegrowthintheimagecompressioneldhasbeensteady,driven byitscontributiontoothereldssuchasmultimediacomputing.asaresult,there Atalaterstageinthestudyperiod,astandardforn64kbit/s(n=1,2,:::,5)wasalso recommendavideocodingstandardfortransmissionatm384kbit/s(m=1,2,:::,5). provideauniversalstandardforvideocodingandtransmissionusingintegratedservices DigitalNetwork(ISDN)[10].Atthistime,theCCITTestablishedaWorkingGroupto techniques. havebeenmanydierentandvarieddevelopmentsinthevideocodingandcompression becameclearthatsinglestandard,p64kbit/s(p=1,2,:::,30),couldcovertheentire considered.withcontinueddevelopmentsinvideocodingandcompressiontechniquesit TheH.261recommendation[3]cametobeaftertheCCITTrecognizedaneedto videoconferencing.therefore,therecommendedvideocodingalgorithmhastobeableto operateinrealtimewithaminimumofdelay.whenp=1or2,becauseoftheseverely limitedbandwidthavailable,onlydesktop,face-to-face,visualcommunication(videophone)isappropriate.forvaluesofpgreaterthan5,withtheadditionalavailablebit Themainapplicationsintendedforthisinternationalstandardareforvideophoneand InDecember1990,theCCITTcompletedandapprovedRecommendationH.261, VideoCodecforAudiovisualServicesatp64kbit/s. ISDNchannelcapacity. CIF(QCIF)asthevideoformatsforvisualtelephony.Thesizesinadditiontothe makesthesehigherdataratesmoresuitableforapplicationssuchasvideoconferencing. 2.2VideoFormat TheCCITTdecidedontheusesoftheCommonIntermediateFormat(CIF)andQuarter- rate,morecompleximagescanbetransmittedwithbetterquality.thisincreaseinquality sizesoftheseformatsarelistedintable2.1.thespecicationofh.261statesthatall codecsmustbeabletooperatewithqcifformatimages.theuseofcifformat(size) luminanceandcolorcodingvaluesareasdenedinccirrecommendation601[2].the 6

thecodedpixels. Table2.1:ParametersfortheCCITTVideoFormats.Thevaluesinbracketsrepresent Luminance(Y) Chrominance(CB) Chrominance(CR) Lines/FramePixels/LineLines/FramePixels/Line 288360(352) 144180(176) CIF 144180(176) 7290(88) QCIF frameisoptional.themaximumdataratespeciedforbothformatsis30000/1001( 29.97)frames/second.Thereisadditionalprovisionfor1,2or3framestobedroppedby theencoderbetweentransmittedframes,inthiswayperformingaspatialsubsamplingof theimagesequence. 101376829:97=24:31106 352288=101376 (2.1) ratereductionisrequiredtotransportsuchvideosignalsviaisdnchannels(p64kbit/s, QCIFat29.97frames/secondare24.31and6.08Mbit/srespectively.Agreatdealofbit- Equations2.2and2.4showthattheuncompresseddataratesfortransmittingCIFand 50688829:97=6:08106 176288=50688 (2.3) (2.2) Forp=1or2,QCIFisnormallyusedforapplicationssuchasdesktopvideophone.Even p1,2,:::,30).thechoiceofciforqcifdependsontheavailabilityofchannelcapacity. (2.4) ifqcifoperatedat10framespersecond,abit-ratereductionofapproximately92.7to1 isneededtotransportthatsignalusinga64kbit/schannel.suchcompressionisamajor videoconferencing. undertaking. picture.duetoitsincreasedresolution,cifisappropriateforapplicationssuchasfull thecbimageandoneforthecrimage.)fourblocksmakeupamacroblock[3]. Thehierarchicalstructureoftheimagedataisdiscussedingreaterdetaillater.The upof88luminancesamplesandapairof44chrominancesamples(apair:onefor TheBlockisanintegralpartofthehierarchicalencodingstructure,ablockismade Forpgreaterthan5,CIFmaybeusedsincetherearemorebitsavailabletocodea algorithm,whichwillbedescribedinfurtherdetaillater. elaboratehierarchicalblockstructureisessentialforthehigh-compressionvideocoding compositionofamacroblockisshowninfigure2.1. 88chrominance(CBandCR)Blocks.AGroupofBlocks(GOBs)iscomposedof3 11MacroBlocks.ACIFPicturehas12GOBswhileaQCIFhasonlythreeGOBs.This Inthisway,eachMacroBlockiscomposedoffour88luminance(Y)Blocksandtwo 7

Figure2.1:StructureofaMacro-BlockshowingsamplingstructureofLuminanceand Chrominance. - Chrominance (sample point) - (sample point) 8

compresseddata,therearetwosortsofvideocompression/codingschemes:thereissource codingandthereisentropycoding. removingredundantinformationcarriedinaparticularform.forvideoimages,likeother 2.3VideoCodingAlgorithm Thebasicobjectiveofcompressionofanysortistoreducetherequiredbitrateby techniques. resultswhicharelossy,i.e.,picturequalityisdegraded,whileentropycodingreliesonthe change,interframecodingisusedforsequencesofsimilarimages,includingthosecontainingmovingobjects.intraframecodingremovesredundantspatialinformationwithin animage,whileinterframecodinginadditiontoremovingredundantspatialinformation, alsoremovestemporalredundancybetweenimages. statisticalpropertiesofthesignalsand,intheory,islossless.h.261makesuseofboth traframecodingisusedfortherstpictureinasequenceandforimagesafterascene Sourcecodingcanbedividedintotwofurthertypes:intraframeandinterframe.In- Sourcecodingdealsdirectlywithcharacteristicsofthesourcematerialandyields betweentheoriginalframeandtherecreatedframewillbeprogressivelyaccumulated errorswillbebetweentheoriginalintraframeandthefollowinginterframes.sucherrors thattherewillbeerrortolerancesinthecreationofdierencesbetweentheimages,these systematicerrorincurredfromoneinterframetothenext.thiserrorisduetothefact Apartfromanytransmissionerrorseectingimagedata,therewillbeanaccumulated dierencesbetweenthatframeandthepreviousframe),erroraccumulationismorelikely. DuetothenatureofInterframeencoding(eachframeofdataisactuallyasetof Asaresult,intraframesareregularlysent(every30interframes,forexample1)tostop thisaccumulationoferrorsintheresultantimage. becauseeachnewinterframe(withitsownerrors)iscreatedfromthepreviousframe (interorintra)andnotasadierenceof(forexample)theoriginalintraframereference. correctioncode.theuseofthisoptionalerrorcorrectioncodewillnotbediscussedin additiontoencodingthereistheoptionaladditionofthebch2(511;493)forwarderror coderandsourcedecoderarebasedonlossyinter/intraframestylesourceencoding.in transmissioncoderandreceivingdecoderareentropystylesystems,whereasthesource wellastheinterandintravariationsofsourcecoding. Figure2.2givesadiagramaticoverviewoftheH.261codec.Inthisdiagramthe Undercertaincircumstances,H.261makesuseofbothsourceandentropycodingas thispaper. 2.3.1SourceCoder/Decoder tizedandthensenttothevideomultiplexcoder.thissamepictureisalsorecovered tion(dpcm)schemesincorporatingmotionestimation.inintraframecodingmode,the DPCMisnotoperative. AblockdiagramofthesourcedecoderforH.261isshowninFigure2.3.Thissystemis ahybridofthediscretecosinetransform(dct)andadierentialpulsecodemodula- intraframescanbesentmoreoften. 1TheH.261standard[3]denesthatanintraframemustbesentforevery132interframes.Although 2Bose,ChaudhuriandHoequenham(code)[18]9 Each88blockintheimageistransformedintoitsDCTcoecients,linearlyquan-

External Control Coding Control Video Signal Video multiplex Transmission Transmission BCH correction Source Coder oryforuseininterframecoding..(throughaninversequantizerandinversedcttransform)andstoredinanimagemem- Coder buffer coder code generator a) Video coder basedonmotionestimationbycomparingeverymacroblock(onlyluminanceblocksare Video Multiplex Receiving Receiving BCH forward error Source decoder decoder buffer decoder forthatparticularmacroblock,insteadonlyamotionvectorisencoded.otherwise, predictedmacroblocksislessthanaparticularthreshold,thennodataistransformed processed)ofthecurrentframewiththemacroblocksintheneighbourhoodofthecorrespondingmacroblockinthepreviousframe.ifthedierencebetweenthecurrentandthe TheDPCMstyleofcodingisusedintheinterframecodingmode.Thepredictionis b) Video decoder thedierenceistransformed(bydct),linearlyquantizedandthensenttothevideo onandotoimproveimagequalityberemovinghigh-frequencynoiseasrequired. multiplexcoderalongwiththemotionvectorinformation.aloopltercanbeswitched therebyincreasingtheamountofinformationgenerated(andtobetransmitted)bythe shouldthetransmissionbuerberelativelyempty,thequantizationstepcanbereduced, consistentamountofdatathroughput)isareductionintheimagequality.alternately, generated(andtobetransmitted)bytheencoding.theimmediateresult(apartfroma thetransmissionbueroftheencoder.whenthetransmissionbuerisclosetobeingfull, thequantizerstepsizecanbeincreased,therebydecreasingtheamountofinformation Thestepsizeofthelinearquantizercanbeadjusteddependentuponthefullnessof encoder. ofthestepsizeofthelinearquantizer.10 coecientsandvariousotherinformation. regulatetheowtoaconstantrateby(asmentionedabove)providingfeedbackforcontrol entropycodingisusedinthevideomultiplexcoderwhichimmediatelyfollowsthesource coderstage.therearevevariableword-lengthcodingtablesforthequantizeddct Theoutputofthevideomultiplexcoderissenttoatransmissionbuer,whichwill Inafurtherattempttoincreasetheeciencyofthecoding,variableword-length Figure2.2:Blockdiagramofthecompletevideocodeccode correction Coded bit stream

CC p t qz T Q q Video In - Q -1 To Video multiplex coder -1 T F P v f T. Q P F CC p t qz q v f Transform Quantizer Picture memory with motion compensated variable delay Loop Filter Coding control Figure2.3:BlockdiagramoftheSourcecoder Flag for INTRA/INTER Flag for transmitted or not Quantizer indication Quantizing index for transform coefficients 11 Motion vector Switching on/off of the loop filter

2.3.2DiscreteCosineTransform(DCT) TransformcodingsystemsbasedontheKarhunen-Loeve(KLT),discreteFourierTransform(DFT),discreteCosineTransform(DCT),Walsh-HadamardTransform(WHT),and anumberofothertransformshavebeenconstructedandstudiedextensively.thechoice ofaparticulartransforminagivenapplicationdependsuponthepropertiesoftheapplication(forexample,howmuchinter-elementcorrelationispresent),upontheamountof reconstructionerrorthatcanbetoleratedandthecomputationalresourcesavailable. duringthetransformationstep). Thevariablem(u;v)isaparticulartransformcoecientmaskingfunctiondesignedto Themean-squareerrorbetweensubimageF(ofsizenn)andapproximation^Fis Compressionisachievedduringthequantizationofthetransformedcoecients(not eliminatethebasisimagesthatmakethesmallestcontributiontothetotalpost m(u;v)=f0ift(u;v)satisfiesaspecifiedtruncationcriteria 1otherwise transformimages. givenby: Huv=264h(0;0;u;v) Huvisthearrayofbasisimagesofthetransformdenedas: h(0;1;u;v):::h(0;n?1;u;v) (2.5) HuvcanbeusedtocomputetheseriesexpansionweightingcoecientsT(u;v). ems=efkf?^fk2g h(n?1;0;u;v)h(n?1;1;u;v):::h(n?1;n?1;u;v) : : : 3 75(2.6) =Efkn?1 =n?1 Xu=0n?1 Xv=02T(u;v)[1?m(u;v)] Xu=0n?1 Xv=0T(u;v)Huv[1?m(u;v)]k2g Xv=0T(u;v)Huv?n?1 Xu=0n?1 Xv=0T(u;v)m(u;v)Huvk2g (2.9) (2.8) (2.7) abilitytothedftandwht,andthisisusuallythecaseformostnaturallygenerated wheret(u;v)isthe2-dtransformoftheimagef(x,y),kf?^fk,isthematrixnormof Itcanbeshown([7,5,20,31])thatwhileDCThasasuperiorinformationpacking (F?^F)and2T(u;v)isthevarianceofthecoecientattransformlocation(u;v). (2.10) images(i.e.notarandomwhitenoiseimage,)infact,theklt,notthedct,isthe optimaltransforminaninformationpackingsense.thatis,thekltminimizesthe foreachsubimage,ingeneral,isanon-trivialcomputationaltask.forthisreason,the mean-squareerror(equations2.8,2.10)foranyinputimagesandanynumberofretained coecients.however,becausethekltisdatadependent,obtainingthekltbasisimages 12

A Figure2.4:PeriodicnatureofDCTandDFTtransforms F (a) DFT A DFTwillcauseGibbsphenomenonattheblockboundaries,therebygivingerroneous packingabilityoftheoptimalklt. odicityofthedctlendsitselftobeinglesspronetoblockingartifacts3.whereasthe KLTisseldomusedinpractice.InsteadoftheKLT,atransformsuchasDCTisused. valuesatthediscontinuitiesandboundaries.thedct,ontheotherhand,reducesthis thecomputationrequired.additionally,thedctcloselyapproximatestheinformation TheDCTusesxedbasisimages(inputindependent)therebydramaticallyreducing GonzalezandWoods(1992)[7]reasonthatincomparisonwiththeDFT,theperi- F eectbecauseofitsimplicit2npointperiodicitydoesnotinherentlyproduceboundary (b) separabletwo-dimensional,88blocks,theseinturnaretransformedusingthedct. Equation2.11showstheforwardtransformationforthegenerationofadiscretecosine discontinuities.theperiodicnatureisshownincomparisontothedftinfigure2.4 computationalrequirements. thedct,whichprovidesagoodcompromisebetweeninformationpackingabilityandits 3Theresultsofboundariesbetweensubimagesbecomingoverlyobvious. Fortheparticularapplicationathand,eachtransmittedblockisrstprocessedinto Asaresultofthesefactors,mostpracticaltransformencodingsystemsarebasedon 13

transformoftheoriginal88imageblock. withu;v;x;y=0;1;2;:::;7 where, F(u;v)=1=4C(u)C(v)7Xx=07Xy=0f(x;y)cos[(2x+1)u=16]cos[(2y+1)v=16](2.11) Equation2.12showstheinversetransformationfortherecoveryofanapproximationof C(u)=1=p2foru=0,otherwise1 C(v)=1=p2forv=0,otherwise1 u;v=coordinatesinthetransformdomain x;y=spatialcoordinatesinthepixeldomain theoriginal88imageblock. withu;v;x;y=0;1;2;:::;7 where, f(x;y)=1=47xu=07xv=0c(u)c(v)f(u;v)cos[(2x+1)u=16]cos[(2y+1)v=16](2.12) x;y=spatialcoordinatesinthepixeldomain theleftandtopedgesoftheimage. Note:Withintheblockbeingtransformed,x=0andy=0refertothepixelnearest C(u)=1=p2foru=0,otherwise1 C(v)=1=p2forv=0,otherwise1 u;v=coordinatesinthetransformdomain 2.3.3DierentialPulseCodeModulated/DiscreteCosineTransformHybrid(DPCM/DCT) processing.intraframeencodinginvolvesonlyasingleframe(withnoreferencetothe DierentialPulseCodeModulationisonlyrelevantfortheinterframecaseinimage previousframe),theframeissubdividedandeachcomponentencodedindependently usingadctandquantizationstep. quantizerandaninversedct,thisisthenstoredinaframebuerforuseasfollows.in Ininterframeencoding,thepreviousframeisregeneratedusinganinverseofthe 14

theinterframemodeofsourcecoding,onlythedierencesbetweentwoparticularframes information.) needtobesent.(inasequenceofimages/framesthereisoftenmuchredundanttemporal inturnreducestheamountoftransmitteddata,therebygivingadditionalcompression. frameinmemory.nowthesedierencesarethenencodedandquantizedtoformthedata wherebyifpartsoftheimage(particular88blocks)havenotchangedbygreaterthan usethepreviousblockcontents.bynotsendingredundant,duplicatedinformation,this tobetransmitted.often(andinthecaseofh.261)thereisasimplicationprocess aparticularthresholdthesecomponentsarenottransmitted,ineectthedecoderwill Thisisdone,inthesimplestform,bysubtractingthenewframefromthereconstructed 2.4OthercomponentsinH.261 thesemotionvectorshaveintegervaluesnotexceeding15.thisvectormustberepresentativeofthewholemacroblockasitisusedforallfourluminanceblocksinamacro amotionvectorforeachmacroblock(mb.)bothhorizontalandverticalcomponentsof whileintheinterframeencodingmodeofoperation.inthismodeanencoderallocates sationisanadditionaltechniqueofremovingfurthertemporalredundancytransmitted 2.4.1MotionCompensation Block. MotionCompensationisanoptionintheimplementationofanencoder.Motioncompen- thatthepredictionisformedfrompixelsinthepreviousframewhicharespatiallytothe rightorbelowthepixelsbeingproduced. Apositivevalueofthehorizontalorverticalcomponentofthemotionvectorsignies canreducetheamountofinformationrequiredtobetransmitted.inthegure,(a)shows putationrequired,motionestimationisanoptionintheh.261coderimplementation. requiredtobetransmitted. SimpleMotionCompensationFigure2.5showshowmotioncompensationdetection Ifweuseanexample,wecanshowhowmotionestimationcanreduceimagedata Becauseofthenumerousamountofsamplingandthustheinherentamountofcom- identicalexceptforatranslation(oftwototherightandonedown,)thusitshould tothepreviousimage. redundantinformation(oneforeachimage),theencodedinformationshouldcontainthe theoriginal(orreference)frameand(b)showsthenextframe.thetwoimagesare secondcopyoftheimage,downtoonlyavectorconsistingoftwoparts.thisexample beintuitivethatinsteadoftheencodedinformationcontaintwocopiesofsimilarbut greatlyoversimpliestheadditionalinformationrequiredinthecodingprocess,including informationtoindicatewhethermotioncompensationisused,butingeneralshouldshow originalimageandthenthefollowingimagebemarkedtosayitmakesuseofmotion inthisexamplehowmotioncompensationwillreducetheamountofcodedinformation compensationandinthiscase,containsonlyavectorofthemotionofthisimagerelative required. Forthissimpleexample,thiswouldreducetheamountofinformationfromafull 15

Image Motion (a) (b) themotionasbrokendownintoitshorizontalandverticalcomponents. Figure2.5:ASimpleMotionexample:(a)showstheoriginal(orreference)frame,(b) showsthenextframe,(c)indicatesthemotionintheimageoftheobjectand(d)shows 2 to right (-2) 1 down (-1) (d) 16

Image Motion (a) (b) 2 to the left (+2) 1 down (-1) (c) (d) Figure2.6:AComplexMotionexample:(a)showstheoriginal(orreference)frame,(b) showsthenextframe,(c)indicatesthemotionintheimageoftheobject,(d)showsthe motionasbrokendownintoitshorizontalandverticalcomponents,(e,f)showsthedierencebetweentheoriginalandresultantimages(whenoverlaid)(e)showsthedierences shaded,while(f)showstheoverlayinatransparentway. 17 (e) (f)

(f)showthedierencesbetweentheimages.insub-image(e)theseareasareshaded Figure2.7:ASecondComplexMotionexample:(a)showstheoriginal(orreference) frame,(b)showsthenextframe.inthesecondframeanobject(thedonut)hasbeen ComplexMotionCompensationFigure2.6showsamorecomplexmotioncompensationexample.Inthisgure,(a)and(b)showconsecutiveframes.Figure2.6,(e)and revealedbythemotionoftheoriginalobject. thatneedstobeencoded. tical.inthissituation,motioncompensationcanstillsaveontheamountofinformation black. Itcanbeseenthatwhilethereismotion,andtheimagesaresimilar,theyarenotiden- informationwillstillbelessthanifthetwoimages(a)and(b)wereseparatelyencoded. example)andthedierencesbetweentheimagesareencodedaswell.thisamountof secondimage,itencodesamotionvectorforthepreviousimage(similartotheprevious thisexample,themotionofoneobjecthasaddedanobjecttothescene(thetorusshape, ordonut).suchrevealingofanobjectmeansthatinadditiontotheencodingofthemotion AnothercommoncomplexmotioncompensationsituationisshowninFigure2.7.In Forthisexample,theencodedinformationcontainstherstimageandinsteadofthe oftheimage(inthiscasetheashape)theencodedimagewillalsocontaininformation describingtherevealedobject. motionofanobjectrevealsotherobjects,obscuresotherobjectsormakeschangestothe background(forexample,introducingashadoworlter). Somesituationswherethemotioncompensationismadecomplexwilloccurwhenthe [13,22,17].Motionestimationrequiresenormousprocessingtosampleandcomparethe compensationisworthwhileand(b)generatethecodedinformationimages. encodedinformation.thedicultiesofmotioncompensationrelatetotheamountof imageprocessing(comparison,subtraction,etc.)requiredto(a)determineifmotion ispresent,motioncompensationwillgivesomeamountofreductionintheamountof Severaltechniquesforperformingthemotiondetectionandmotionestimationexist Whilesimplifyingtheprocess,thisshowsthatinallimageswhereanamountofmotion havebeendeveloped,thesimplestistosubsamplethetwoimagestocompare.this twoimagesbetweenwhichmotionistobedetected.asaresultofthis,anumber oftechniquestomaximizetheeciencyandminimizethetimerequiredforthistask 18

subsamplingtechniqueisuseintheh261encoderexaminedinthesecondpartofthis paper. itself). 2.4.2Filter Thepredictionprocessmaybemodiedthroughtheuseofatwodimensionspatiallter Block,anditactsuponalltheblocksinsideaparticularMB(likethemotionestimation thecaseofmotioncompensationthusthelterisswitchedon/oforaparticularmacro whichoperatesonpixelsinaparticular88block.thelterisusedparticularlyin up.equation2.13showsthetwoseparablecomponentsforverticalandhorizontal,aswell Bothofthesearenon-recursivewithcoecientsof14,12,14exceptatblockedgeswhere valuesonlyatthe2-dlteroutput.valueswhosefractionalpartisonehalfarerounded coecientsof0,1,0.fullarithmeticprecisionisretainedwithroundingto8bitinteger oneofthetapswouldfalloutsidetheblock.inthiscase,the1-dlterischangedtohave Thelterisabletobeseperatedintoone-dimensionalhorizontalandverticalfunctions. asthe33arrayofthe2-dlter. formotioncompensationthereisanassumptionofintegeramountsofmovementbythe 264141214375141214=2641 MacroBlock(from1to15pixelwidthmovesineitherup/downorleft/rightdirections.) Asaresult,whenanimagemovesbyless(ormore)thananintegeramount,themotion Thelterbecomesnecessary,especiallyinthecaseofmotioncompensation,because 1618116375 181418 (2.13) ofcausingjumpingbytheimage,withoutsmoothmovement. compensationroutineswillenforceanintegeramountofmovement.thishastheeect adjacentpixels(althoughitispredominantlyplacedinoneparticularset). motion.thedisadvantageofthelteristhatimageswillappearfuzzier. Figure2.8.Inthisgure,themotionwillplacetheimageinapositionwhereitoverlaps images(linesetc)willhavealessclearlydenededgeandappeartobemoresmoothin Theuseofthelterisnotrestrictedtouseincombinationwiththemotioncompensationsystem.Theltercanbeusedatanytime.Thisbecomesusefulwhenimage pictureinformation,itremovessomeoftheinformation(low-bandpass)andthisresults insmoother,softerimages. Acasewheretheimageasplacedandtherealimagedonotactuallyagreeisshownin Alterwillosetthisproblembycausinganimagetobelow-bandltered,asaresult detailsarenerthanthedisplayresolutionand,asaresult,motionwillappeartojumpor appearjagged.usingthelter,theimageoutlineswilltendtobeblurred,theimageswill appearsofter,asaresultmotionwillnotappearasjumpy.thelterdoesnotincreasethe 19

(a) (b) 1.75 to right (- 1.75) horizontalandverticalcomponents. Figure2.8:Non-integermotion:(a)showstheoriginal(orreference)frame,(b)shows integerinterpolationofthatmovementand(d)showsthemotionasbrokendownintoits thenextframe,(c)showsthenon-exactmatchbetweentherealimagemovementandthe 0.75 down (-0.75) 20

Picture Header GOB data Figure2.9:Hierarchicalnatureofthedatastructuresofthecoder GOB data Picture Layer GOB header MB data MB data Group of Block Layer MB header Block data Block data Macro Layer MacroBlocks(MB)andBlocks.Therelationshipbetweeneachdatastructureisshowin Figure2.9. ThevideocodingalgorithmdescribesamethodwherebytheCIFandQCIFimagesare 2.5EncodedImageDataStructures Variable Length Code TCOFF TCOFF EOB Layer imageheaderincludesa20-bitimagestartcodeinadditiontootherinformationsuch dividedintoahierarchicalblockstructureconsistingofpictures,groupsofblocks(gobs) astheformatofthepicture(ciforqcif),framenumber(temporalreference)etc. DataforeachimageconsistsofapictureheaderfollowedbydatafortheGOBs.The Fixed Length Code Figure2.10showsthearrangementofGOBsinanimage. lteron/o).thisinformationisinturnfollowedbythedataforblocks.figure2.12 forthembaddress,thevlcforthembtype(intra/inter,motionestimationon/o,loop positionofthisgob(intheimage),quantizerinformationforthegob(unlessoverwrittenbyanmbsownquantizerset),etc.thisisfollowedbydataforthembsthemselves. Figure2.11showsthearrangementofMacroBlocksinaGOB. TheMBlayerconsistsofanMBheader,containingthevariablelengthcode(VLC) TheGOBlayerconsistsofaGOBheader,containinga16-bitGOBstartcode,the QCIFformats. showsthearrangementofblocksinamacroblock;notethedierencebetweencifand becomesdependentonthespecicsofthembitself. noblockdatafordctcoecientsneedstobetransmitted.itshouldbenotedthatnot everymbinagobneedstobetransmittedifitcontainsnoinformationforthatpart ofthepicture. VarioussortsofvideosideinformationmayalsobeincludedinanMBheader,this lengthcode,eob,toindicatetheendofablock.thecoecientsarecodedusingtwo Whenmotionestimateinformationforaparticularblockisoftherequiredaccuracy, ifitcontainsnoadditionalinformation.21 dimensionalvlc. TheBlocklayercontainsDCTcoecients(TCOEFF)ofablockfollowedbyaxed Likeallotherlevelsinthehierarchy,noteveryblockinanMBneedstobetransmitted

1 2 3 4 1 Figure2.10:ArrangementofGOBsinanimage 5 6 2 7 8 3 9 QCIF 11 12 CIF Figure2.11:ArrangementofMacroBlocksinaGOB 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 Figure2.12:ArrangementofblocksinaMacroBlock 22 1 2 5 6 3 4 Y C C B R

0,0 0,7 Figure2.13:Zig-zagsamplingofa2dimensional88datablockintoa64elementlinear 2.6Entropyencoding array. Theentropycoderconsistsoftwomaincomponents: zig-zagreorganizationofsamples thehumanencoding Thezig-zagreorganizationofthesamplesinablock(showninFigure2.13)describes 7,0 7,7 themannerinwhichan88blockofsamplesisconvertedintoa64itemlineararray. ThisarrayrepresentstheresultsfromatwodimensionalDCT,where(0,0)istheDC componentand(7,7)wouldbethehighestfrequency[14]. encodingusingamodiedhumancodingscheme.thisresultsinavariablebitrate encodeforanyparticularblock.modiedhumanencodingisapracticalimplementation ofthehumanalgorithm.(fromjain(1989)[12]):::: Nowthattheblockisinalineararrayformat,theentropyencodercanperform coderepresentingthequotientq,followedbyaterminatorcode,whichisthesameasthe Humancodefortheremainderj;0jL1?1.::: TherstL1symbolsareHumancoded.Theremainingsymbolsarecodedbyaprex i=ql1+j;0qint[(l?1) L1];0jL1?1 (2.14) 23

Symbol Binary Code p Huffman code (HC) s 1 s 2 s 3 s 4 0 0 0 0 0 1 0 1 0 0 1 1 0.25 0.21 0.15 0.14 0 1 0 0 1 0.29 0.54 0 1.0 1 0.46 1 0 0 1 0 0 1 0 0 1 1 s 1 0 0 5 0.0625 0 1 1 0 0 0.125 0.25 s 1 0 1 0 6 0.0625 1 1 1 0 1 1 s 1 1 0 0.0625 0 1 1 1 0 7 0.125 beseenthewayinwhichaprobabilitytreeisbuiltupfromastatisticalanalysisofthe s 1 1 1 8 0.0625 1 1 1 1 1 Average Themorepopulartheitemofdata,thelessthenumberofbitsusedtorepresentthat dataandbitsareallocatedtothedataonthebasisofthisprobability. Figure2.14:Humancodingexample. code 3.0 2.781 (entropy) 2.79 data. ThismaybebetterexplainedbyFigure2.14(alsofromJain(1989))inwhichitcan length Code Humancompressiontablestobeusedateachcompressionstage(Block,MacroBlock, TheH.261standardtakestheuseofHumanonestepfurtherbypredeningthe ThekeytoHumancodingistheuseavariablenumberofbitstorepresentthedata. efficiency 92.7% 99.7% GroupofBlocks,andImage.)Suchprecalculationhastheeectofspeedingthroughput H/B inthisstageofthecompressionsequence. 24

AnH.261implementation Chapter3 3.1Overview TheimplementationdiscussedisauthoredbyThierryTurlettiofINRIAinFrance.Itis imagesoverthepacketbasednetworksmakinguptheinternet.naturallythishashad referredtoash261encode/h261decode[26]andhasbeenusedasinaninternetprotocol severalmajorimpactsontheimplementation;however,thefunctionalityofthecodec implementationstillstands. [6,11]basedvideoconferencingsystem[27]. thespecicationwasintendedfor[3],insteadthecodechasbeenwrittenfortransferof Theh261softwarecodecisdividedintotwoprogramcomponents: TheH.261codechasnotbeenwrittenforuseoverap64kbit/sdatachannelas useofthex-windowssystemforthepurposesofdisplayoftheimages[23,19]. h261encode-beinganimageencodersatisfyingtheh.261recommendations h261decode-beinganimagedecodercompatiblewithh261encodeandalsosatisfyingtheh.261recommendations imagelesasinput.thecodecthencompressestheimageinaccordancewiththeh.261 forsunmicrosystem'scomputers.although,theencoderalsoworksbytakingundecoded ThesoftwareisdesignedtooperatingundertheUNIXoperatingsystem[24,21]making TheencoderhasbeendesignedtofunctionwiththeVideoPixvideoframegrabber standardandeithertransmitstheimagetoadecoderviatcp/iporstorestheh261 encodedoutputinale. 3.2VideoPiximaginghardware advantageofreducingthecomplexityofthesoftwarecodec,whilestillachievingthe onlydealingwiththeluminancecomponentsofthesignals.)thishastheconsiderable desiredfunctionality. Thesoftwareencoder,atthisstage,onlysupportsBlackandWhiteimages(thereby WhilethecodechasbeendesignedspecicallywiththeVideoPixframegrabberinmind, basedcomputers,withoutmanymajorchanges.onedirectresultofbeingdesignedtouse thesoftwareshouldbeabletobeadaptedtomostotherframegrabbersrunningonunix 25

itappearedthatthemajorbottle-neckimpedingtheecienttransferofimagesisthe thevideopixhardwareisabletoreturnacifsizeimageinabout250ms.intesting, thevideopixhardwareisthatthesoftwareexpectsthevideoframegrabbertobeslow, transferoftheimageacrossthebus(ofthehostcomputer)tothemainmemory,where h261encodecanmanipulatetheimage[28]. stagethisonlyeectsthesizeoftheimagegrabbed. 3.3Inter/Intracodingdecisions Therstimageisprocessedinintramode.Asthereisnoimageprecedingthisone, anintermodecodingisnotpossible;asaresult,sourcecodingwillonlyreducespatial ThesoftwareissetuptotakeeitherPALorNTSCformatimagesasinput,atthis intraframe,theeliminationofredundanciesinthetemporaldomainisalsopossible redundanciesintheimage. 3.4MovementDetection throughtheuseofmotioncompensationroutines. (e.g.,unchangedpartsofthepicture).additionaltemporalredundanciescanberemoved Withtheintermodeofcoding,possibleafterthecoding(andtransmission)ofan marked.onlymarkedblocksareencodedintheintermode(therestarenotconsideredto Themovementdetectionalgorithmaimsatidentifyingtheblocksinanimagewhichare aremadeinordertodetectvariations.ifthesumofthefourabsolutedierencesexceeds sucientlydierentfromcorrespondingblocksinthepreviousimage.theseblocksare havechangedenoughtowarranttransmission.)ineachblock,foursuccessivecomparisons 44blockforeach88block(i.e.4pixelsperblock.) pixelsthataresucientlyscatteredtocoverthewholeblock.onepixelischosenper fromimagetoimage.thesecomparisonsaredoneforasignicantnumberofpixels,i.e. vectorisencodedforthemacroblock. athreshold,adecisionismadetoencodethemacroblock. Ifthesumofthefourabsolutedierencesdoesnotexceedthethreshold,amotion Itisinterestingtonote,asaresult,thatthechoiceofthresholdbecomesacompromisebetweenthetimetakentodotheencodingandthesensitivitytomovementthatis evengroupofblockslevel,theseimagecomponentsaresimplynottransmitted. 3.5TheDCTandQuantization Thecodingalgorithmtakesasinputasetofblocks(i.e.88)pixels,andgeneratesthe transformcoecientsfortheseblocks.inintermodecoding,theinputblocksareonly If,ininterframecodingmode,theencoderndsredundancyattheMacroBlockor approximately2-3timesaslongtocomputeassimplycodingtheinter-framedierence. Evenwiththissub-samplingoftheimage,detectionofmotioncompensationtakes Inordertotestallregionsoftheblock,thepixelstestedarenotfromthesamepoint required. thoseblocksmarkedbythemovementdetectionalgorithm. 26

mendationdoesnotspecifyatwhatpointtochooseinterorintrastyleencoding(thisis leftuptotheimplementer.)inthisimplementation,afullintramodecodingisusedfor therstimage,forchangeofsceneandeachmbthatisencodedusesintramodeafter30 interencodingmodeframes.ascenechangeissaidtohaveoccurredwhenthenumber ofblocksmarkedbythemovementdetectionalgorithmexceedssomethreshold. Inintramodecoding,theinputblocksarealltheblocksofanMB.TheH.261recom- ThenthealgorithmcalculatestheInverseDCT(IDCT)andreversesthequantization madeupofan88groupofsamples. processtorecoverwhatwillbecomethepreviousimageforthenextinterpass. follows: AspertheH.261recommendation,attheinputofthecoder,sourceimageblocksare Thecodingalgorithmcomputesforeachblockof88pixels,itsDCTandquantization. PerformH.261encodingofalldetectedblocks: Inthisway,theencodinganddecodingprocessesforthecodeccanbesummarisedas Forencoding: Performmovementdetectionforeachblockoftheimage {Interframe/IntraframedecisionIFInterframeTHEN: performaninversequantizationandinversedctthensumtheresultwith performadctofthisdierenceandthenquantizethedctresults computethedierencebetweenthepreviousblockandthenewblock {performh.261encodingoftheimage,groupofblocksandmacroblocksusing ELSEIFIntraframeTHEN: performaninversequantizationandinversedctthenstoretheresultas performadctoftheoriginalimageandthenquantizethedctresults thepreviousblockandstorethisasthenewpreviousblock Fordecoding: Ifintraframecoding: {Inversequantization,InverseDCTofallblocksintheMBthenstorethisas Humantables. Ifinterframecoding: OnaMacroBlockbyMacroBlockbasis: {IftheMacroBlockisencodedwithamotionvector,computeandapplythe thepreviousimage. Nowdisplaytheimage. {IftheMacroBlockisnotencodedwithamotionvector: motionvectortothemacroblockofthepreviousimage. performaninversequantizationandaninversedctoftheencodedblocks Sumtheresultswiththepreviouslystoredblock. inthemb; 27

PointsofInterest Thecalculationsareonlyeectivefortheblockswhichhaveneverbeensucientlymod- ofthetransmissionofcodeddatavariesasafunctionofthemovementintheparticular tionaltothequantityofnewinformationwhichistobecarried.additionally,thetime ied.inconsequence,thechangesnecessaryforcodingofanimagearedirectlypropor- sequenceofimages. conference/videotelephoney,forwhichtheoriginalstandardwasconceived).asaresult, thesoftwaremustattempttomeetthisrequirementofindependenceofthedurationof codingfromtheparticularimagetypecarried. Thissituationisparticularlyirritatingforthereal-timescenario(suchasavideo- Tomeetthisrequirementanumberofsolutionsareimplementable: reductionofthenumberofblockstocalculateperimage.forthistowork,itis Thisparameter(thenumberofblocksperimage)hasthemostdirecteectonthe durationofencodingrequired.ifsuchatechniqueisused,oneneedstomakeapass oftheentireimage,(preprocessingtheimage)inordertodetecttheblocksthathave necessarytolimitthedetectionofmovementinaccordancewiththetypeofimage. changingthequantizationstepswhenasignicantchangetotheimageoccurs.however,suchasolutionmaynothavethedesiredeectsincethisstage(quantization) Anothersolutionconsistsofonlytransmittinganumberofthecoecientsofthe changedmost.thissolutionwouldnotworkifmanyoftheblocksinanimagehad beenchangedsimultaneously,asforexampleinascenechange. isnecessarythatsuchchangesarerepresentativeofchangestothewholeblock,then coecientscanbechosenwhichencodethelowfrequenciesoftheimage.forthis, DCTinthesituationwhereonlysomeofthesampleshavechanged.Asaresult,it itisalsonecessarytoperformapreliminarypassovertheimage,butitisuselessto isperformedafterthecalculationoftheinter-imagedierenceandofthedct. Figure3.1showshowsmallerregionsofthe16x16DCTcoecientsblockisactually brightnesslevels.forthe22example,fourdctcoecientsaretransmitted,for trytouseblockdetectionsincethevaluesaretheresultsofpreviouscalculations. becausethisisthedccomponent,theresultisablockconsistingofonlytwo transmitted.forthe11caseonlytherstofthecoecientsistransmitted, the33,9aretransmitted.thefull88(64)dctcoecientsarealsotransmitted asareference. Usingthistechnique,astudytriedseveralsizetransforms:88,44,33,22 and11.theconsideredopinionofthestudywasthat11,22,and33did nottransferenoughinformationtobeofuse.table3.1showsresultsobtainedfor betweenthelengthofcodingandtherateofcompressionissought. stateofthecommunicationsnetwork.inthecaseofcongestion,thedecodercanorderthe mustbeaparameterofcodingmodiableonadecoderbydecoderbasis,basedonthe codertoreducetherateofowbetweenthecoderandit;inthiswaythebestcompromise Thelengthofthecoding,inadditiontothemaximumnumberofbitsforanimage thetestsequencemissamerica(consistingof71frames)[28]. 28

0,0 0,7 0,0 0,7 7,0 1x1 7,7 7,0 2x2 7,7 0,0 0,7 0,0 0,7 (1coecient),22(4coecients),33(9coecients)and44(16coecients) Figure3.1:Thisdiagramshowshowonlysomeofthe64DCTcoecientsareused.11 examplesareshown. 29 7,0 7,7 7,0 7,7 3x3 4x4

Table3.1:EectsoncodingduetonumberofDCTcoecientsused NumberofCoecients1x12x23x34x48x8 Meanrateoftransfer5.63.43.12.92.2 Meantimetoencode177289320342455 Meansizeofimage6801380189021303050 ReportedS/N (inimage/s) (inoctets) (inms) 3.6TransmissionofpacketvideoviaTCP/IP Theh261encode/decodecodecisdesignedtoexchangedataoveraTCP/IPbasednetwork;asaresult,severalextraconsiderationsmustbemadeforthepeculiaritiesofthe networkbeingused. (indb) 30.133.735.436.439.4 links.thisisinstarkcontrasttothetcp/ipnetworkwheretheactualdatarateis imposedonacommunicationslinkbyotherusers. carrierortcp/ipcouldbeaslowas9600bit/s,10mbit/sinperfectethernetupto100 TCP/IPrunsasfastasitcan,thisspeedisdynamicdependingupontheparticularload Mbit/sinFDDIandevenhigherratesintheWideAreaNetworksystems.Inthisway dependentonthecommunicationsmediacarryingthevideodata.thecommunications TheH.261recommendationisforthetransmissionofH.261overp64kbit/sdata (nisdnchannels)andtcp/ip(basedonethernet):isdnassumesasetrateavailable (64kbit/sperchannel),whereastherateinTCP/IPmightbeaslowasseveralkbit/s consumptionaccordinglysoasnottobecomeanetworkbandwidthpig,consumingall thenetworktothedetrimentofotherusers. orashighasseveralmbit/s;thisistotallydependentontheloadandcapacityofa particularchannel.asaresult,theh261decode/encodesoftwaremustadjustitsnetwork ThisistherstmajordierencebetweentheH.261assumedcommunicationsnetwork basedcommunicationsisthatisdncommunicationsprotocolsassumeasteadystream ensuredofcomingouttheotherend,unduplicated,unreplicatedandinorder. styleconnection,thatis,alldataputinoneendofthecommunicationsconnectionis ispacketizedandsentthroughthenetwork.dataisnotsentinacontinualstream,at themostbasiclevel,reliabilityisnotguaranteed,additionallythereremainsthechance ThesecondmajordierencebetweentheISDNstyleofcommunicationsandTCP/IP oflosingpacketsalltogetheraswellasreceivingthemoutoforderorevenreplicated. (TCP))givesareliablepseudo-streamofdata,althoughdatadelayisacertaintyona busynetworkwiththeworkrequiredtomakeaparticularstreamofdatareliable. Incomparison,thebasicInternetProtocolisbaseduponapacketnetworkwheredata TheprotocolsintheInternetnetwork(inthiscasetheTransmissionControlProtocol 30

imagesarecoded.atransmittedimagehasnotgotasetlength,infactitssizeis Humanencoding.Inaddition,inintermode,onlythedierencesbetweentwosuccessive octetalignmentofthedatatransmittedovertheinternet.asaresult,thedecoderonly recognizesinformationattheendofanimagewhenitreceivesthethenextframespicture Inordertoincreasetherateofthecompresseddatastream,H.261utilizesmodied notknownuntilthetransmissioniscomplete.thisisasadirectresultofthenon- 3.6.1Locatingtheboundarybetweenimages StartCode(PCS). willtransmitthedatastream. EvenpaddingoutdatatothesizeofabytewillnotguaranteethemomentatwhichTCP isdecoded,itisnecessarytocontainthepcsofthefollowingimageinthesamebyte. bitsof)remainsofthepreviousimageapushthroughthechanneltothedecoder. InTCP,thenotionofbitsdoesnotexistsinceprogramsworkwithastreamofoctets. AsaresultiftheincomingimageistobedisplayedassoonasthelastMacroBlock ThePCSislargeenoughtollgreaterthanoneoctetthusgivingthe(uptoseven userquitstheprogramoranotherimageisreceived). receivingasetsequenceofimagesthelastimagewillnotbedisplayed(either,untilthe 3.6.2ControllingtheRateofFlow Thereisaneedtocontroltherateofdataowoutofthecodectoenabletimetoreact Theresulttotheenduserisatleastoneframelag,aswellasthepossibilitythatafter principalfactors: incaseofcongestiononthenetworks.therateofowisafunctionofthefollowing Thenumberofallocatedbitsperimage,determinedby: Thefrequencyoftheimagesampling(adjustablebyunder-sampling) 3.6.3ChoiceofQuantizer {thesettingofthethresholdofblockchangewhichcontrolsthedecisionof {thestepsizeofthequantizer(thestepsizeofthequantizerisininverseproportionaltothequalityoftheimage); Thechoiceofthequantizerstepuseddependsuponthecoecientswhichhavetobe coded.itisobviousthatlargecoecientvaluesmaynotbeabletobecodedwitha quantizerthatusestoosmallastep;alargecoecientcorrespondingtoaquickchange whethertocodeornot. foranabnormallylongamountoftime(takesalongamountoftimetooperate),thestep thet.turlettipaperdescribingthecodec[28]. dierentquantizationvaluesusingtheturlettih.261codec.thisgureistakenfrom ofscene.itisforthisreasonahazeisobservedatthetimeofascenechange. However,asaresultofincreasingthestepsizetheimagequalitywillbedegraded.Using sizeofthequantizercouldbeincreased,therebyreducingtheamountofdatatotransmit. Fortherstimagethenetworkisseentobecongested,ifthesend()operationblocks Figure3.2showsthedatarateforthecodingoftheMissAmericasequenceforseveral 31

Figure3.2:TheeectofQuantizerstepondatarate.Source:T.Turletti(1992)[28] 32

3.7Summary arougherdct(fewercoecientsperblock)thiswillalsoincreasethroughputofthe Fromthisimplementationseveralitemsofinterestdevelop: balancebetweenimagequalityandimagerate. images;however,withacorrespondingloweringofreconstructedimagequality. Asaresult,inaneorttoachievethebestqualityimages,thetaskistondthebest Motioncompensationtakes2-3longerthanjustusinginter,thusifmotioncompensationistobeused,thethresholdchoice(onwhetherablockhaschangedsuciently Inordertoachievethebestqualitypictures,thereisabalancebetweenimage Thetimeoftransmissionvariesasafunctionofthemovementinaparticularimage sequence. torequireretransmission)isacompromisebetweenencodetimeandsensitivityto movement. theavailablebandwidth.)fortheprivilegingofrate,twochangescanbemade: quality.inthesecondselection,thesoftwareattemptstoselectthebestquantizer(for ofthecodec,theuserisonlyabletoselectbetweentheprivilegingofimagerateorimage ifthenetworkiscongested,thequantizationstepisincreasedtherebydecreasing Intheoperationoftheencoder,theuserdoesnothavecontrolofmanyofthefunctions qualityandimagerate. iftherearealotofblockstoencode,theuseofarougherdct,whilelowering theamountofdatatransferred; infrastructuresofatm(asynchronoustransfermode)communicationswhichhaveat Videooverpacketswitchnetworks(andformodicationstotheencodingsystemtobe requiredforadequateresults),willbecomemoreimportantwiththedeploymentoflarger ThiscoderisaninterestingapplicationoftheH.261systemandthebasisofsending imagequality,willalsolowerthenumberofcoecientsgeneratedperblock,and theirheartpacketswitchtechnology. thustheamountofdatatransferred. 33

Conclusion Chapter4 currentdevelopments;andassuch,h.261givesagoodinsightintootherdevelopments. andcompression.partsofthisrecommendationbeenincorporatedintomanyofthe TheH.261recommendationformsanexcellentintroductiontotheworldofvideocoding works.asaresult,theimplementationcoverswaysinwhichh.261canbemodiedto suitethissituation. TheimplementationisfascinatingbecauseithastoworkinwithInternetbasednet- 34

Acknowledgements Chapter5 IwouldliketoextendmythankstoDrWuforgivinghisstudentsenthusiasminthe subjectmaterialandforpresentingmanyinformativeanddynamicsessionsonvideo available,andhismakingavailableanumberofreferencepapersusedinthepreparation CodingandCompression.IwouldalsoliketoextendmythankstoThierryTurletti ofthispaper. forhisassistanceinmyattemptstounderstandthesoftwarehehasgraciouslymade 35

Bibliography [1]AngP.H.,RuetzP.A.,andAuldD.,\Videocompressionmakesbiggains",IEEE [2]CCIR,\Recommendation601-IDigitalMethodsofTransmittingTelevisionInformation",1986. Spectrum,vol.?,pp.16{19,October1991. [4]ChanS.C.andHoK.L.,\Anewtwo-dimensionalfastcosinetransformalgorithm", [3]CCITT,RecommendationH.261,Videocodecforaudiovisualservicesatp64 [5]ClarkeR.J.,TransformCodingofImages,AcademicPress,24{28OvalRd.,London kbit/s,ccitt,1990. [6]ComerD.,InternetworkingwithTCP/IPVolume1;Principles,ProtocolsandArchitecture,volume2,Prentice-HallInternational,Inc.,April1991. NW17DX,1985. IEEETransactionsonSignalProcessing,vol.39,pp.481{485,February1991. [8]HaqueM.A.,\Atwo-dimensionalfastcosinetransform",IEEETransactionson [7]GonzalezR.C.andWoodsR.E.,DigitalImageProcessing,Addison-Wesley,1992. [10]IEEE,\SpecialIssueonISDN",IEEECommunicationsMagazine,vol.28,,April [9]HorganJ.,\ClaudeE.Shannon",IEEESpectrum,vol.,pp.72{75,April1992. Acoustic,SpeechandSignalProcessing,vol.ASSP-33,pp.1532{1539,December [11]ISI,\Internetprotocol",RequestForComment:791,vol.?,,September1981, 1985. [12]JainA.K.,FundamentalsofDigitalImageProcesing,Prentice-Hall,1989. [13]JainJ.R.andJainA.K.,\DisplacementMeasurementandItsApplicationin 1990. WritterninassociationwithDARPA. [14]JayantN.,\SignalCompression:TechnologyTargetsandResearchDirections", [15]leGallD.,\Mpeg:Avideocompressionstandardformultimediaapplications", InterframeImageCoding",IEEETransactionsonCommunications,vol.COM-29, CommunicationsoftheACM,vol.34,pp.47{58,April1991. pp.1799{1808,1981. IEEEJournalonSelectedareasinCommunications,vol.10,pp.796{818,June1992. 36

[16]LiouM.,\Overviewofthep*64kbit/svideocodingstandard",Communicationof [17]NetravaliA.N.andStullerJ.O.,\Motioncompensatedtransformcoding",Bell [18]PetersonW.W.andWeldonJrE.J.,ErrorCorrectingCodes,MITPress,1972, theacm,vol.34,pp.60{63,april1991. [19]QuerciaV.andO'ReillyT.,XWindowSystemUsersGuideforX11R3andR4, AppendicesCandDareusefulforndingg(x)forthebinaryBCHcodes. SystemsTechnicalJournal,vol.88,pp.1703{1718,September1979. [22]RoccaF.andZanolettiS.,\Bandwidthreductionviamovementcompensationon [20]RaoK.R.andYipP.,DiscreteCosineTransform,AcademicPress,Inc,1990. [21]RitchieD.andThompsonK.,\TheUNIXTime-SharingSystem",Communications oftheacm,vol.17,pp.365{375,july1974. O'Reilly&Associates,Inc.,Sebastapol,CA,May1990. [23]ScheierR.andGettysJ.,\TheXWindowSystem",ACMTransactionsonGraphics,vol.5,pp.79{109,April1986. amodeloftherandomvideoprocess",ieeetransactionsoncommunications, [24]SobellM.G.,APracticalGuidetotheUNIXSystemV,BenjaminCummings, vol.com-20,pp.960{965,october1972. [25]StixG.,\Encodingthe\Neatness"ofOnesandZeros",ScienticAmerican,vol., [26]TurlettiT.,\H.261Codec",1992,Availableusinganonymousftpfrom MenloPark,CA,1985. pp.27{28,september1991. [28]TurlettiT.,\Logicieldecodeur-decodeurH.261",September1992,Paperasyet [27]TurlettiT.,\INRIAvideoconferencesystem",1992,Availableusinganonymousftp fromavahi.inria.fr:/pub/videoconference/ivs.tar.z. avahi.inria.fr:/pub/h261/h261.tar.z. [29]WuH.,\RDT4630VideoCodingandCompression",July1992,Syllabusguidelines [30]WuH.,\RDT4630VideoCodingandCompression:PracticalWorkInstruction onlypublishedinfrench,aspartofinriaprojectrodeo. Technology,FacultyofComputingandInformationSystems,MonashUniversity. forvideocodingandcompression(rdt4630),departmentofrobiticsanddigital [31]WuH.,\RDT4630VideoCodingandCompression:SubjectNotes",July1992, Notes",August1992,SubmissionguidelinesforVideoCodingandCompression (RDT4630),DepartmentofRobiticsandDigitalTechnology,FacultyofComputing andinformationsystems,monashuniversity. Coursenotesforthesubject:VideoCodingandCompression(RDT4630),DepartmentofRobiticsandDigitalTechnology,FacultyofComputingandInformation 37