Context Aware Personalized Ad Insertion in an Interactive TV Environment

Context Aware Personalized Ad Insertion in an Interactive TV Environment Amit Thawani, Srividya Gopalan and Sridhar V. Applied Research Group Satyam Computer Services Limited #14, Langford Avenue, Lal Bagh Road, Bangalore, India 560025 Amit_Thawani@Satyam.com Abstract. The growth of digital television over satellite, cable as well as terrestrial networks has driven advertisers to better target users by local personalized advertisements, which generates qualified lead for future sales. Context awareness in ITV applications can be used as a powerful tool to deliver most relevant and personalized ad based on user s context. In this paper we have proposed architecture for context aware real time selection and insertion of advertisements into the live broadcast stream by taking into consideration the user s current and past contextual information. Our approach is based on aggregating a past sequence of individual contexts and associating the current user activity to those past contexts to determine the best ad to be delivered relevant to the current activity. Introduction The growth of digital television over satellite, cable as well as terrestrial networks has driven advertisers to better target users by local personalized advertisements, which generates qualified lead for future sales. The advent of Personal Video Recorders (PVRs) and Video on Demand (VOD) is viewed as a threat by advertisers as they create ad-skippers. As the viewers started evading advertisements, ad agencies compete on innovative ways to ensure their ads are viewed. There is a strong need to rework on TV advertising to create innovative features that can exploit advertisement opportunities. Personalization in interactive and future televisions aims in targeting content to individual users by adapting the content based on users likes and circumstances. [9] compares different types of addressable advertising and demonstrate superiority of home addressable advertising. [4] presents a life-style based approach for the delivery of personalized advertisements based on segmentation variation and similarity variation. In a related paper [7], we have proposed a Surfing Analysis (CSA) algorithm for predicting the user(s) in front of TV by a dynamic analysis of channel viewing characteristics. Such user identification can be used along with a Home Information System (HIS) to categorically decide about the nature of ads that are likely to have an impact. Context-aware and situation-aware services together enhance the perceived quality of the delivered information. Context awareness provides mechanisms for developing

adaptive applications, which is an important aspect of ITV applications. [3] presents a context-aware information retrieval system that analyzes the user s history to derive an enhanced context. [5] proposes to model the user actions from contextual information, in a way to ease its interaction with context-aware ITV applications. Advertisers create multiple versions of advertisements for a single TV spot in order to cater for different user communities. Given multiple versions of advertisements, and also information about user currently in front of the television, the challenge is to make use of and analyze the user s current and past contextual information to select and insert the most appropriate advertisement from the multiple versions of multiple advertisements at real time. Though research has been done for personalized ad insertion that considers users preferences and user profile information, the use of complete contextual information for personalization needs to be addressed. In this paper we have proposed an architecture for context aware real time selection and insertion of advertisements into the live broadcast stream by taking in to consideration the user s current and past contextual information. Our approach is based on aggregating past sequence of individual contexts i.e. past viewing patterns and associating the current user activity to those past contexts to determine the most appropriate ad to be delivered relevant to the current activity. The proposed architecture is realized by means of four distinct subsystems, CoD (Context Derivation), CaASI (Context-aware Ad Selection and Insertion), UId (User Identification), and BAR (Bulk Ad Retrieval). Context Context can be described as information that can be used for characterizing the status of an entity in one specific case. One entity can be a one person, one place or one object relevant for any type of interaction between user and application, including the user and the application itself [1]. Parameters such as: a) where is the user; b) who is the user; c) how the user works; d) when the activity is being done; and e) what the user is doing; are used to build a context aware environment. [1] defines the four entities viz. location, entity, activity and time to address the parameters for building a context aware environment. [6] discusses which type of contextual information could be used in an interactive environment. Fig. 1 describes the general context information comprising of location, identity, activity and time and more specifically context information for ITV applications. Location information of the user at home such as user is in living room or user is in bedroom can be used to determine the location context of the user. information in ITV environment for context can be characterized as user identity, device identity, content identity and event identity. User identity is specified by user profile as described in the HIS stored in the Set Top Box (STB) at home. User identity typically includes information such as user s name, age, occupation, gender, marital status, favorite channels, movie interests, music interests, ad interest, etc. Device identity specifies attributes such as resolution, features, connectivity, etc. of multimedia devices at home. Content identity is stored in the STB as Electronic Program Guides (EPG) after being retrieved from the broadcast. The EPG also consists of other pro-

gram related information such as category, channel type, genre, parental guidance, summary, etc. The Event identity is determined either by the broadcast or by other sources such as sensors and the network. Event information can be for national or local events that might have an impact on the user context or user s TV watching behavior. Context Location Time Presence in Living Room Presence in Kitchen Presence in Entertainm ent Room Device Content User Event Device User Past Pattern Group Morning Evening Weekend Weekday Fig. 1: Context Information information for context can be characterized for user activity, device activity and user/device history. User activity such as user watching TV with family, user watching TV while working from home, user watching TV while doing household jobs, etc. can be used to derive the user s context information for the activity entity. Device activity information typically includes the device usage information by the user. User and device history information typically includes users interaction with interactive controls of programs and ads in the past, his past purchases, websites and web pages visited in the past and channels and programs viewed in past. Time information for context consists of the current time as well as time information of the users history for channels/programs, purchases and interactivity. Time information gives cues for the context of the user as user s TV watching behavior changes for different time of the day as well as different days of the week. Our proposed approach considers context information as Current context (CCo) and Analyzed or Aggregated Context (ACo). CCo is defined by the current activity and the entities associated with it. CCo is the information available at hand at an instance of time from direct sources. In our proposed system past context sequences are analyzed and aggregated to derive usage trends and user preferences, in the form of rules which are defined as ACo. For the ITV application of personalized ad insertion, based on CCo and ACo classification discussed above, the CCo information contains identified user watching TV at that instance of time, the user s age, the current program schedule for all the channels derived from the EPG, the program type, the program category, the program genre and parental guidance of the programs derived from EPG, ad slots in the programs and also ad sponsors. Similarly, the ACo information would contain derived user preferences for the programs, program types, program category and program genre, user s demographic information, user s viewing behavior, user s remote usage pattern, user s viewing history, user s purchase history, user s interactivity history and local and national event information. The XML-based MPEG-7 (also called Multimedia Content Description Interface) Description Definition Language (DDL) [2] along with TV-Anytime Metadata Specification [8] can be used as a representation format for metadata to represent and de-

scribe components of a broadcast, ads and user profiles in the HIS. The representation format of the content enables determination of keywords describing the content. Similarly, keywords can be determined for user attribute information as described in the HIS. Since the context information is based on the content and user attribute information, a set of keywords can be determined to represent the context information. These determined context information keywords are matched against the keywords representing the ad content. The ads whose content description keywords have a match with the context information keywords are identified as the targeted ads to be inserted and presented to the user. System Architecture Multiple distinct networks are interconnected to deliver broadcast content to home users. The broadcast network broadcasts content to MSOs through radio channel and MSOs deliver content to home users via a cable network. From the point of view of ad insertion, a typical approach is to insert the ads before broadcast. This gives little opportunity to target ads to obtain a better return on investment. While content reaches MSO through radio, the related ads could reach the MSO through Internet or an IP network. In this case, the MSO stores these ads and could insert them at an appropriate place before forwarding the content to the home users. The third approach is to distribute the relevant ads directly to a home network where a set-top box would do the necessary ad insertion into the content before displaying the same. The last two approaches provides an opportunity to select and insert the most appropriate ads. The system level description of our proposed approach is depicted in Fig. 3. As described in Fig. 3., the architecture consists of four distinct subsystems, CoD, CaASI, UId, and BAR Functionality of The CoD subsystem involves determination of both the CCo and the ACo information. The ACo information is used to identify ad tags, which depict analyzed and aggregated context of all users at home, for requesting ads to be retrieved from the MSO. The ACo information, along with inputs from the UId subsystem, is also used to identify appropriate ads from the ad database to populate secondary ad storage (cache). The UId subsystem implemented on an STB is responsible for processing home users viewing characteristics. Information such as user s likes and dislikes (such as with respect to movies, sports and music), his/her recent purchases, typical viewing hours on weekdays and weekends, and typical channels viewed are used to analyze the received viewing characteristics to identify one or more users (joint viewing) who might be watching TV [7]. The BAR subsystem is responsible for requesting ads from the MSO based on the context information of the users at home. Considering the number of sponsors and various versions of ads for each product of the sponsor, it is difficult to store all the ads in the STB. This subsystem solves this issue by requesting ads based on analyzed and aggregated context of all the users at home. It receives the context information of all the users at home from the CoD subsystem module and identifies ad tags depicting the context information. It then requests MSO to send ads based on the ad tags thus determined. The CaASI subsystem is responsible for selection and insertion of ads in the broadcast content, based on the context information,

for presentation to the user. This subsystem determines points of ad insertion in the broadcast stream by decoding digital cue tone messages. Targeted ads are identified using CCo and the ACo information and further inserted in the broadcast stream in the STB. Home Home Home MSO Multiple s Content Ads Content Ads Multiple Insertion Bulk Ad selection Ad Tags Aggregation Ad Tags Transmit Contents Home CaASI Ad Cache Ad DB BAR Send Ad Tags Contents CC based Ad Selection Bulk Request processor Selection content insertion Context Derivation (CC and AC) AC based Ad Tag Selection Mixing History DB UI Surfing Analysis Decoding Display Home Info System Collect Viewing Characteristics Home Home Home Fig. 3: System Architecture for Ad Selection and Insertion Scenario Consider a home with four members consisting of a dad (John) and mom (Cindy) and 2 kids (Andy and Samantha). The User profile information stored in the STB depicts: John is an insurance agent and is a sports lover; John s favorite sport is basketball; John generally watches TV from 7 to 10 in the evening; Cindy works for a bank; Andy and Samantha are school going kids. History information stored in the STB depicts: John had been visiting websites searching for a basketball jersey; John had purchased tickets for the finals of the college league basketball; John has been watching ads showing sports jersey and had interactions with the ITV components of type contact me for these ads; Cindy had been looking for vacuum cleaners on websites; Andy and Samantha had been interacting with ads for back to school products. The Events database depicts: Final of college league basketball tournament is on the coming weekend; Wal-Mart has sale for home equipment on the coming weekend. The EPG information depicts: Nike, Wal-Mart and Honda are sponsors for live coverage of college league basketball tournament; Nike has scheduled to show ads for its new range of golf shoes; Wal-Mart has scheduled to show ads for its upcoming sale of home equipment. Consider a working day and the time 8:00 P.M. John comes back home and switches on the TV and starts watching ESPN channel which is showing live coverage of college league basketball championship. When the TV is switched on, the ACo component analyzes the history data and user profiles for all members of home. From the events database, the ACo component determines that the final of college league

basketball tournament is on the coming weekend. Similarly, the ACo component, based on user profiles and history information, determines that Cindy has been planning to buy a vacuum cleaner from Wal-Mart and Andy and Samantha had been planning to buy school bags from Wal-Mart. Based on the information determined, the ACo component determines rules and passes on the information to the BAR subsystem. Typically the rules that can be determined for the scenario under consideration are: Since John purchased tickets of the college league final match, he would like to wear Nike basketball jersey when he goes to watch the final match; Since Cindy had been looking for vacuum cleaners on websites, she would probably be interested in buying one from Wal-Mart sale on home equipment; Andy and Samantha would be interested in buying back to school products from Wal-Mart since they had been interacting with such ads. The BAR subsystem determines keywords such as <sports, jersey, basketball, college league tournament, vacuum cleaner, kids, schooling, school bag, Northface, Home equipment> and requests the MSO for ads matching the context information. The MSO, on receiving the request, sends the following ads to STB at home: Nike ad for recently launched golf shoes; Nike ad for Nike jersey with basketball theme; Nike ad for soccer shoes with football theme; Wal-Mart ad for the weekend sale on home equipment; Wal-Mart ad for always low prices ; Wal-Mart ad for back to school products; Wal-Mart ad for gardening equipment; Honda ad with F1 theme; Honda ad for new coup model. Further, the ACo component, based on user profile information, determines that only John and Cindy might be watching TV at this instance of time. Based on the rules determined, it then analyzes the received ads and caches Nike and Wal-Mart ads in secondary ad storage. On remote click, the UId subsystem analyses user profiles and remote usage pattern and determines that the most probable user watching TV at that time should be John. Based on inputs from the UId subsystem and also based on user profile and EPG information, CCo component of the CoD subsystem forms rules which determines the CCo. The CaASI subsystem analyses the CCo information and ads in the secondary ad storage and determines that the Nike ad with the theme Nike jersey with basketball theme is the most appropriate ad as compared to the scheduled ad for recently launched golf shoes. The CaASI subsystem then inserts the ad in the broadcast for presentation. Ads Requested Based on ACo Nike ad for recently launched golf shoes; Nike ad for Nike jersey with basketball theme; Nike ad for soccer shoes with football theme; Wal-Mart ad for the weekend sale on home equipment; Wal-Mart ad for always low prices ; Wal-Mart ad for back to school products; Wal-Mart ad for gardening equipment; Honda ad with F1 theme; Honda ad for new coup model. Probable Users at Home John and Cindy Ads Cached Based on ACo and Context Information History Nike ad for Nike jersey with basketball theme; Wal-Mart ad for the weekend sale on home equipment; Wal-Mart ad for always low prices ; Wal-Mart ad for back to school products Identified User at Home John Most Appropriate Based on CCo and Inputs from UId Nike ad for Nike jersey with basketball theme Similarly, if the identified user by the UId subsystem would had been Cindy and not John, the most appropriate ad determined by CCo component would be: John and Cindy Wal-Mart ad for the Cindy weekend sale on home equipment; Wal-Mart ad for always low prices

For the same day but at different time, say 5:00 P.M. in the evening, when Andy and Samantha are watching their favorite cartoon channel, the cached ads by the ACo component and most appropriate ads determined by CCo component would be: Andy and Samantha Wal-Mart ad for always low prices ; Wal-Mart ad for back to school products Andy and Samantha Wal-Mart ad for back to school products Summary In this paper we have proposed an architecture for context aware real time selection and insertion of advertisements into live broadcast stream by taking in to consideration the user s current and past contextual information. XML-based MPEG- 7 DDL along with the TV-Anytime Metadata Specification are used as a representation format for metadata to represent and describe components of the system. The proposed work is part of our ongoing work on developing an STB environment with enhanced functionalities and capabilities. The project also involves development of intelligent itv applications such as user identification, intent tracking, personalized ad insertion etc. We are currently working on a context sensitive cache conscious indexing technique to perform lookups for ad selection. References 1. A.K. Dey and G.D. Abowd, Towards a better understanding of context and contextawareness, Handheld and Ubiquitous Computing, (H.W. Gellersen, Ed.), Springer, Berlin, pp. 304-307, 1999. 2. ISO/MPEG N3445 MPEG-7, 2001: Multimedia Description Schemes XM (v.7.0). http://mpeg.telecomitalialab.com/working_documents.htm. 3. H. Kammanahalli, Srividya Gopalan, and Sridhar V., Context aware retrieval in web-based collaboration, Proc. Second IEEE Conference on Pervasive Computing and Communications Workshops. Orlando, FL, 2004. 4. G. Lekakos, and G.M. Giaglis, A lifestyle-based approach for delivering personalized advertisements in digital interactive television, Journal of Computer-Mediated Communications, 9(2), January 2004. 5. J.B.D. Santos, R Goularte, G.B. Faria, and E.D.S. Moreira, Modeling of user interaction in context-aware interactive television application on distributed environments, Proc. TV01 - Workshop on Personalization in Future TV in Conjunction with User Modeling 2001. Sonthofen, Germany, 2001. 6. J.B.D. Santos, R Goularte, G.B. Faria, and E.D.S. Moreira, The modeling of structured context-aware interactive environments, Transactions of the SDPS Journal of Integrated Design and Process Science, v. 5, n. 4, p. 77--93, December de 2001. 7. A. Thawani, Srividya Gopalan and Sridhar V., Viewing characteristics based personalized ad streaming in an interactive TV environment, Proc. 2004 IEEE Consumer Communications and Networking Conference, 2004. 8. TV-Anytime Forum. Specification Series (Metadata). 2003-August. http://www.tvanytime.org/. 9. V. Dureau, Addressable advertising on digital television, Proc. Second European Conference on Interactive Television, 2004.