Real-Time Opaque and Semi-Transparent TV Logos Detection

Transcription

1 Real-Tme Opaque and Sem-Transparent TV Logos Detecton Alex Res dos Santos, Hae Yong Km Escola Poltécnca, Unversdade de São Paulo, Av. Prof. Lucano Gualberto, trav. 3, 58, CEP 558-9, São Paulo, SP, Brazl Abstract - One of the most mportant strateges created by TV broadcast statons to clam and protect vdeo content ownershp s the TV logo. Wth dfferent colors, shapes and styles, TV logos dentfy the broadcast staton, and sometmes even the knd of the broadcasted program. Consequently, the detecton of TV logos s useful n applcatons rangng from detecton of TV commercals to audence measure. In ths paper, we propose an mproved edgebased template matchng to detect opaque, semtransparent and partally anmated logos. The proposed algorthm could be mplemented n a DSP for real-tme logo recognton, because t s economc n memory use and requres low processng power. We tested durng more than 4 hours our mplementaton to recognze logos of Brazlan and Amercan TV channels and acheved % of correct recognton rate. s by fndng TV logos out of the vdeo stream []. Another applcaton, proposed n [, 3], s to use the lack of logos to ndcate the presence of a televson commercal. The logo detecton s the frst step to remove t, usng npantng technques [4, 5, 6], where the vewng experence can be mproved wth the logo removed. TV logos can be classfed n three types: opaque, sem-transparent and (partall anmated. Fgure depcts some examples. On some Brazlan broadcast statons, the logo changes from sem-transparent to opaque when the content s beng transmtted alve, for example n Record channel (second row of fgure ). I. INTRODUCTION Dscovery (anmated) Dscovery (anmated) One of the most mportant strateges created by TV broadcast statons to clam and protect vdeo content ownershp s the TV logo. These logos can be consdered as a vsble watermark and dentfy the broadcast staton, and sometmes even the knd of the broadcasted program. For example, some channels change ther logo from sem-transparent to opaque to ndcate alve transmsson. Also n most channels, the logo dsappears durng transmsson of TV commercals. Consequently, a good way for audence survellance nsttutes to detect the channel selected by a TV vewer Record (sem-transparent) Globo Record (opaque) Gazeta Cultura MTV Fg. : Examples of captured TV logos: Dscovery logo s partally anmated (the terrestral globe spns). The authors would lke to express ther grattude to IBOPE and CNPq for the partal fnancal supports of ths work under grants 3565/3-3 and 47555/4-. The am of ths paper s to measure TV audence by fndng a logo n the vdeo stream. We search the vdeo

2 stream lookng for one (or more) logo out of a set of prevously stored TV logos. We propose a sngle algorthm to detect opaque, sem-transparent and partally anmated logos. The proposed algorthm s economc n the memory usage, and requres low processng power, so that t could be mplement n a portable DSP board. Opaque logos are easy to detect. Partally anmated logos can be regarded as opaque ones, because they can be detected through ther mmovable parts. the sub-vdeo F, and for SporTV logo only n the subvdeos F and F 4. We have bult a table contanng the nformaton of n whch sub-vdeos each logo can appear. Ths process accelerates the processng, essental for real-tme mplementaton. Sem-transparent logos are the most dffcult ones to detect. Our method frst dscards the color nformaton, because the hue and the color saturaton nformaton do not contrbute n the task of detectng sem-transparent logos, due the changng background. Then, we proceed wth tme averagng (as mentoned by Albol et al. n []). The tme averagng emphaszes the logos whle blurs out the background. Next, we dscard the absolute grayscale by extractng the edges, because the grayscale of sem-transparent logos vary wth the background, whle the edges do not. Even usng only the grayscale edge nformaton, we acheved % of detecton accuracy. Our result s n apparent contradcton wth Wang et al. s paper [7], where they state edge-based template matchng s weak for sem-transparent ones when ncomplete edges appear. Fg. : Extracton of the four sub-regons where TV logos can appear. II. Extractng edges A vdeo stream n PAL-M system (Brazlan broadcast standard) has pxels. Emprcally, we observed that the logos always appear nsde 4 rectangular regons located near the four corners. So, we extracted four subvdeos (say, F, F, F 3 and F 4 ) each one wth 5 pxels. Fgure depcts the four sub-regons and fgure 3 depcts a frame of the vdeo stream F obtaned by concatenatng the four sub-vdeos. For other TV systems wth dfferent pxel resolutons (such as NTSC), a slghtly dfferent set of sub-regons must be defned. Emprcally, we have observed that each TV logo can appear n only one or two sub-vdeos. For example, CNN logo always appears n the upper left sub-vdeo F, and SporTV logo always appears n the upper rght subvdeo F or n the lower rght sub-vdeo F 4. Consequently, t s necessary to search for CNN logo only n Fg. 3: The four sub-regons merged to form vdeo stream F. We dscarded the color nformaton, because the color of a sem-transparent logo changes wth the background varaton. Tme averagng s used to emphasze the pxels that ether do not vary through the tme or vary only a lttle. Ths flterng removes the nconstant background mages and emphaszes the logos. Only one out of t frames are

3 taken nto account n the tme averagng. t s approxmately 3 frames ( second), and the processor uses ths tme nterval to make the rest of the processng (edge extracton, logo searchng, etc.) Indeed, we have notced that t s not worth to use all frames n the tme averagng, because averagng smlar frames does not help to get rd of the background. Mathematcally: The convoluton of the tme-averaged mage F wth Prewtt operators (fgure 5) can be used to evaluate the two partal dervatves of the equaton above. Fgure 6 depcts some examples of the edge mages G. We have tested also detectng edges before performng the tme averagng, and smlar results were yelded. [ F ( t t) F( )] F ( t) = + t, t () where F ( t) s the tme-averaged vdeo stream at pxel ( and frame t. Note that the tme-averaged frame at tme t s a weghted average of the frame t (wth weght.5), the frame t t (wth weght.5), the frame t t (wth weght.5), and so on. The frst tme-averaged frame F ( ) can be defned equal to the frame F ( ) or as a completely black mage. Fgure 4 depcts a tme-averaged frame. After the tme averagng, most of the objects n the frame become blurred, except the logo and perhaps some other tmenvarant objects. Fg. 5: Prewtt operators Fg. 6: The edge mage G of a vdeo stream and the edge mage of a sem-transparent logo. III. Template Matchng Fg. 4: A tme-averaged frame. Now, the edges can be extracted. There are many dfferent edge-fndng methods n the lterature. We use the magntude G of the gradent of the tme-averaged mage F as the edge mage: F ( x, F ( x, G ( x, = F ( x, = + () x y In order to recognze logos n a vdeo stream, t s necessary to have a dataset of edge mages of logos, say L, L,..., L n, obtaned by pre-processng the sample vdeos as descrbed n the prevous secton. Assocated wth each logo, there must be a lst of one or two subvdeos where the logo can appear. Gven a vdeo stream, the edge mage G of the tme-averaged vdeo F s computed for each t frames. Then, the cross-correlaton s used to spatally localze the logo. Before the correlaton, the mages are frst mean-corrected, that s, the DC level s taken out: ~ G( = G( G ( (3)

4 ~ L ( = L ( L (, n (4) where G ( and L ( are the mean grayscale levels of the edge mage G (at frame t) and of the logo mage L. Fg. 7: Convoluton of the two mages of fgure 6. The matchng s at the brghtest pont. The cross-correlaton R between two real-valued mages F ~ and L ~ s defned [8, 9]: ~ ~ R ( m, n) = n). (5) F( L ( j + m, k + j k The cross-correlaton can be normalzed, by dvdng t by the length of vectors F ~ and L ~, and yeldng the correlaton coeffcent: ~ F ~ ( L ( j + m, k + n) j k γ( m, n) = (6) [ ~ (, )] [ ~ F j k L ( j + m, k + n) ] j k j k The correlaton coeffcent γ ( m, n) ranges from - to. Emprcally, we estmated the threshold level.73 that dd not produce any false alarms and found all logos n our vdeo streams. A more statstcally sound decson can be made by performng a hypothess test. To test the hypothess, the cross-correlaton s converted nto a Student s t-statstcs τ. Then, the hypothess the logo L ~ s located n edge mage G ~ at pxel (m, n) can be statstcally tested. The underlyng supposton s that the pxel values n G ~ are generated ndependently at random (ths supposton s not completely true). Let us denote the pxel values of mage G ~ scanned n some predefned order (lke raster order) as onedmensonal vector Y. Smlarly, let us denote the pxel values of mage L ~ translated to poston (m, n) and scanned n the same order as one-dmensonal vector X. Then, the objectve s to estmate parameter β that mnmzes error n the followng equaton: Y M Y X = M N X N β + M N (7) Ths equaton s usually wrtten n matrx notaton as: Y = Xβ +. (8) s the vector of resdual errors, whch are consdered ndependent dentcally dstrbuted normal varables. The parameters β that mnmzes the mean square value of error can be estmated by the least squares procedure: XY β =. (9) X The parameter β can be transformed nto the Student s t statstc τ by computng: τ = X β. () ( n ) For large n, the Student s t statstc can be approxmated by the normal statstc. The obtaned statstc τ s used to perform the hypothess test. Assumng that the null hypothess H ndcates no correlaton between Y (the edge mage G ~ ) and X (the logo mage L ~ translated to poston (m, n)), we would lke to know how lkely s our measure τ. The hypothess test allows us to perform a comparson between the obtaned value τ and the value τ α correspondng to the selected sgnfcance level α (the acceptable false postve rate), acceptng or rejectng

5 the null hypothess f τ < τα or τ τα, respectvely. The followng smple numercal example clarfes these deas: = β () The frst vector Y s the pxel values of the edge mage G ~ of the vdeo stream. The second vector X s pxel values of the edge mage L ~ of the logo shfted to poston (m, n). Estmatng the parameter β, we obtan.75, and estmatng the Student s t dstrbuton wth 7 degrees of freedom we get τ = Ths means that the logo mage was probably found at poston (m, n) of the vdeo stream. The null hypothess wll be rejected at α =. sgnfcance level τ α = 3. 4 for a one-tal t test. Expermentally, τ assumes values as hgh as 6 whenever there s a logo matchng. IV. Implementaton Detals and Expermental Results from broadcast statons. Fve logos were opaque (CNN, SporTV, AXN, Record and Unversal), four were sem-transparent (Record, Globo, Gazeta and Cultura) and one partally anmated (Dscover. After creatng the dataset wth the ten logos, new vdeos from these broadcast statons and other 5 vdeos that dd not contan any of the logos were captured. All logos were correctly detected, and all absences of logos were also correctly detected. However, t took dfferent tmes to detect the logos. Opaque logos were detected n average after second. Sem-transparent logos were detected n average after 5 seconds, dependng on the varaton of the background (the more varaton, the less tme takes to detect the logo). B. Implementaton n Embedded System (DSP) We have mplemented a complete embedded system envronment usng DSP (dgtal sgnal processor) n C and assembly language. We have used the development kt Blackfn, model EZ-KIT LITE BF533, from Analog Devces []. Ths development board contans all the hardware necessary for ths applcaton: volatle flash memory to store the dataset, non-volatle fast memory to compute data, a DSP processor wth clock up to 6MHz, a decoder of PAL-M vdeo (Brazlan color TV broadcast standard) and some seral ports to communcate wth a PC computer. Fgure 8 and 9 depcts our system mplemented n a Blackfn board. A. Implementaton n C++ Usng the mage-processng lbrary called ProEkon [], we have mplemented the proposed algorthm. Ths mplementaton does not work n real-tme. It was used only to test quckly the deas developed n the prevous sectons. We have used a TV Card named Play TV Pro Ultra by PC Vew [] to capture vdeos from broadcast statons and store them as AVI fles. Ten logos were created from the vdeo fles captured Fg. 8: Our system mplemented n Blackfn DSP board.

6 After creatng the dataset wth the logos, we have montored 5 broadcast statons for more than 4 hours: statons that corresponds to the logos, and 5 statons that do not correspond to the stored logos. All logos were correctly detected wthout false alarms. However, t took dfferent tmes to detect the logos. Opaque logos were detected n average after second. Semtransparent logos were detected n average after 5 seconds, and the tme delay depends on the varaton of the background (the more varaton, the less the tme dela. V. Concluson Fg. 9: The broadcastng staton dentfed n the PC montor. Our system downloads logos from a PC computer, makes acquston of the vdeo stream n real tme, makes vdeo and mage processng as descrbed n prevous sectons, and sends to PC the dentty of the detected logo through an asynchronous seral port. The vdeo s decoder receves a PAL-M sgnal, and generates a vdeo stream n ITU-656 format [3] n realtme wth 8 bts word. In ths applcaton we are nterested just n Y sgnals (lumnance) that represents grayscales. The logo searchng s made sequental and exhaustvely, logo-by-logo, n the sub-regons specfed n the dataset. Ths process s called logo detecton. When there s a correlaton coeffcent larger than a specfed threshold, the DSP nform t to the PC through a seral port. When a broadcast staton s dentfed, another process called logo trackng begns. It conssts n confrmng ths logo untl t changes or dsappears. Fgures 8 and 9 depct the entre applcaton n our laboratory. The envronment conssts of a televson, the development board and a PC that dsplays n ts montor the dentty of the logo found n the vdeo stream. We have used the same dataset as the mplementaton n C++. The same ten logos were used: fve opaque logos, four sem-transparent ones, and one partally anmated. In ths paper, we have presented a real-tme portable logo detecton system. The proposed algorthm s based on edge-based template matchng, and requres only small amount of memory and low processng power. It was mplemented on a DSP board. All three knds of logos (opaque, sem-transparent and partally anmated) could be detected. We have tested our system for more than 4 hours and all logos were correctly detected. REFERENCES [] A. Dvakaran,. Radhakrshnan, Logo Detecton and Classfcaton n a Sport Vdeo: Vdeo Indexng for Sponsorshp Revenue Control, Proc. Int. Soc. Optcal Engneerng (SPIE), pp ,. [] A. Albal, M. J. C. Fulà, A. Albal, and L. Torres, Detecton of TV commercals, Proc. ICASSP 4, vol. III, pp , May 4. [3] J. H. Yeh, J. C. Chen, J. H. Kuo, J.-L. Wu, TV Commercal Detecton n News Program Vdeo, Proc. Int. Sym. Crcuts and Systems (ISCAS), vol.5, pp , May 5. [4] W. Q. Yan and M. S. Kankanhall, Erasng Vdeo Logos Based on Image Inpantng, n Proc. Int. Conf. on Multmeda and Expo (ICME), Swtzerland, vol., pp. 5-54, Aug.. [5] W. Q. Yan, J. Wang, and M. S. Kankanhall, Automatc Vdeo Logo Detecton and Removal Multmeda Systems, (5), pp , July 5.

7 [6] K. Mesnger, T. Troeger, M. Zeller, and A. Kaup Automatc TV Logo Removal Usng Statstcal Based Logo Detecton and Frequency Selectve Inpantng Proc. European Sgnal Processng Conference, September 5. [7] J. Wang, L. Duan, Z. L, J. Lu, H. Lu, and J. S. Jn, A Robust Method for TV Logo Trackng n Vdeo Streams n Proc. IEEE Int. Conf. on Multmeda and Expo (ICME), pp. 4-44, 6. [8] K. R. Castleman, Dgtal Image Processng, Prentce-Hall, 996. [] accessed on September 6, 6. [] accessed on August, 6. [] accessed on August, 6. [3] accessed on August 5, 6. [9] R. C. Gonzalez, R. E. Woods, Dgtal Image Processng, second edton, Prentce-Hll,.