MM4U - A framework for creating personalized multimedia content

Transcription

1 MM4U - A framework for creating personalized multimedia content Susanne Boll University of Oldenburg, Computer Science Department Multimedia and Internet Technologies Group, D Oldenburg, Germany Susanne.Boll@informatik.uni-oldenburg.de Abstract Multimedia content today can be considered as the composition of media elements into an interactive multimedia presentation. Personalization of such multimedia content means that the content reflects the users s context, their specific background and interest as well as the heterogeneous infrastructure to which the content is delivered and in which it is presented. To achieve personalized multimedia content a manual creation of many different documents for all the different user contexts, however, is not feasible let alone economical. Rather a dynamic process of selecting and assembling personalized multimedia content depending on the user context seems reasonable. Support for such a dynamic creation of personalized multimedia presentations, however, is lacking today. Existing multimedia authoring tools do not support dynamic content creation. Systems that already dynamically assemble personalized content, typically found in the Web, remain text-focused. First research approaches in the field of dynamic generation of personalized multimedia content are limited with regard to their expressiveness of the personalization task and the user features they can personalize to. Whenever the content personalization task is more complex, these systems need to employ additional programming. Consequently, with MM4U (short for multimedia for you ), we propose a generic and modular framework to support multimedia content personalization applications in this multimedia software development process. The framework s components provide generic components for typical tasks of the multimedia content personalization. In turn the development of multimedia applications becomes easier and much more efficient for different users with their different contexts. First sample applications we develop on the basis of our framework are a personalized multimedia picture gallery and a personalized sightseeing tour. Keywords: multimedia content personalization, user context driven multimedia authoring, software framework for personalized multimedia applications 1. Introduction Personalization is the shift from one size fits all to a very individual and personal, i.e., one to one treatment of customers. Though we are still at a hype, there is a clear awareness for personalized content and products, technologies are evolving and developing. In this field, the development of support for creating personalized multimedia content pertains to this effort. In this paper, we are concerned with providing a software framework to support dynamic creation of personalized multimedia content. Figure 1 illustrates the general process of dynamically generating a personalized multimedia presentation. Input to this process are media data, the associated meta data as well as user profile information. On a request, a personalization engine exploits the user profile information and meta data to select the most suitable media data. The selected media elements composed in time and space into a coherent, personalized multimedia presentation. As indicated in Figure 1 result can be, e.g., a personalized multimedia lesson in biology about the drosophila in an HTML format, a personalized sightseeing tour of the city of Oldenburg as a SMIL presentation, or a quick sightseeing info on a mobile phone about the Secession building in Vienna 1. Even though we find appreciated solutions in what one could call contributing technologies for the intended dynamic multimedia content personalization, i.e., collecting and representing user profile information, modeling meta data, storing and retrieving media data, etc., solutions for the actual personalization engine box as shown in Figure 1 are still lacking. The subsequent section shows in more detail where we have reached to far in research and industry with regard to the multimedia content personalization effort, where the problems and the challenges are, and how the proposed MM4U framework can provide a substantial progress in the field of personalized multimedia applications. With MM4U one simplifies and cheapens the de- 1 Pictures and content gratefully taken from and 1

2 The Horst-Janssen-Museum in Oldenburg shows Life and Work Horst Janssen in a comprehensive permanent Horst-Janssen Museum The Horst-Janssen-Museum in Oldenburg shows Life and Work Horst Janssen in a comprehensive permanent Personlization Engine Meta data Context dependent selection of content Context dependent multimedia composition Media data User Profile Figure 1. Multimedia Content Personalization velopment of end-user tools for any personalized multimedia experience like a personal picture gallery, personalized sightseeing guides, or personalized e-learning applications. The remainder of this paper is structured as follows: Section 2 reviews the related approaches in the field, shows their limitations, and motivates the development of our framework. The MM4U framework itself is presented in Section 3. An overview of the implementation and our sample applications is given in Section 4, before we conclude the paper in Section The limitations of existing approaches t Multimedia authoring tools like Macromedia Director [12] let the authors create very sophisticated multimedia presentations in a proprietary format. For this, the authors need to have high expertise in using the tool and though do create multimedia presentations only that are targeted at a specific user or user group. Everything personalizable would have to be costly programmed within the tool s programming language. Declarative descriptions of multimedia documents employ a declarative language to describe the temporal, spatial, and interactive features of a multimedia presentation. Addressing the aspect of personalization in multimedia documents, the declarative, XML-based approach SMIL [2] allows for the specification of adaptive multimedia presentations by the specification of so called presentation alternatives. The manual authoring of such documents that are adaptable to many different contexts is too complex; also the existing authoring tools for this are still tedious to handle. Consequently, we have been working on the approach in which a multimedia document is authored for one context and is then automatically enriched by the different presentation alternatives needed for the expected user contexts in which the document is to be viewed [4]. However, this approach is reasonable only for a limited number of presentation alternatives and limited presentation complexity in general. Approaches that already dynamically create personalized content are typically found on the Web, e.g., Amazon.com [1] and with adaptive hypermedia systems [8] [7]. However, they remain text-centric are not occupied with the complex composition of media data in time and space into real multimedia presentations. On the pathway to usercentered, on-demand generation of personalized multimedia content, we primarily find research approaches that address the personalization of single media like the semi-automatic home-video editor Hitchcock [11], or the personalized album MyPhotos [13]. In the area of dynamically generating (personalized) multimedia presentations there exist only a few research approaches: The Cuypers system [14] employs constraints for the description of the intended multimedia programming and logic programming for the generation of a multimedia document [10]. A generic architecture for the automated construction of multimedia presentations based on transformation sheets and constraints has been presented by [9]. These research solutions typically use a declarative description like style sheets, rules, constraints, configuration files and the like to express the dynamic, personalized multimedia content creation. However, they can solve only those presentation generation problems that can be covered by such a declarative approach; whenever a complex and application-specific personalization generation task is required, the systems find their limit and need additional programming to solve the problem. Also, the approaches we find usually rely on fixed data models for describing user profiles, structural presentation constraints, technical infrastructure, rhetorical structure etc. and use these data models as an input to their personalization engine. The latter evaluates the input data, retrieves the most suitable content, and tries to most intelligently compose the media into a coherent aesthetic multimedia presentation. A change of the input data models as well as an adaptation of the presentation to more complex presentation generation tasks is difficult if not unfeasible. Additionally, for these

3 approaches the border between the declarative descriptions for describing content personalization constraints and the additional programming needed is not clear and differs from solution to solution. Our implication here is that for the generation of complex personalized multimedia content, programming is unavoidable anyway. Consequently, we propose the software framework MM4U that relieves the developers of personalized multimedia applications from the central content personalization tasks and let them concentrate on their domainspecific content creation job. A framework which provides the necessary interfaces to develop personalized applications but at the same time a home for existing, specialized (research) approaches which can be embedded in the framework. HTML sequential 4 5 Multimedia Presentation Presentation Format Generators HTML+ TIME SMIL ZYX 3 Multimedia Composition Slide parallel Book show 2 User Profile Accessor 2 Media Pool Accessor 1 User Profile Connectors 1 Media Data Connectors 3. The Multimedia Personalization Framework MM4U CC/PP user profile DB image retrieval system video database This section presents MM4U, a generic software framework for the dynamic generation of personalized multimedia presentations. The framework approach in general has proven to be very successful in areas in which specific solutions are needed and in which general basic patterns can be reused, e.g., the Java Media Framework and the Java SWING library in the field of multimedia applications and graphical user interfaces, respectively. Figure 2 illustrates the architecture of our framework. It is divided in several layers which provide modular support for the different tasks of the personalization process as illustrated in Figure 1. The framework is designed to be extensible such by embedding additional modules via well-defined interfaces as indicated by the empty boxes with dots. In the following, we describe the framework s features along the different layers (indicated by the circled numbers in the framework) of the architecture. We begin from the bottom of the architecture which integrates the user profile, media and metadata information (1 and 2), then present the support for personalized multimedia composition (3), and finally the creation and rendering of the personalized content (4 and 5): 1. Connectors: The components that bring user profile data and media data into the framework are the User Profile Connectors and the Media Data Connectors. The tasks of the connectors is to integrate existing systems for user profile stores, media storage, and retrieval solutions. As there are many different systems and formats available for user profile information, the User Profile Connectors abstract from the actual access to and retrieval of user profile information and provides suitable interfaces to the profile information. With this component, the different formats and struc- Figure 2. Overview Multimedia Personalization Framework tures of user profile models can be made accessible via a unified interface. On the same level, the Media Data Connector abstracts from the access to media elements in different media storage and retrieval solutions that are available today. The different systems for storage and content-based retrieval of media data are interfaced by this component. 2. Accessors: On top of the Connector components, the Accessor components, i.e., the User Profile Accessor and the Media Pool Accessor, provide the internal data model of the user profiles and media data information within the system. Via these components the user profile information and media data needed for the desired content personalization is accessible and processable for the application. The Connectors and Accessors are designed such that the framework s components are not re-inventing existing systems for user modeling or multimedia content management. The rather provide a seamless integration of the systems by distinct interfaces and comprehensive data models. 3. Multimedia Composition: The Multimedia Composition component resides on top of the User Profile component and Media Pool component and employs this information for the composition task. This component comprises abstract operators in compliance with the composition capabilities of multimedia composition models like SMIL [15], Madeus [16], or ZYX

4 [12]. With these operators, an application can carry out the actual composition of the personalized multimedia presentation. The Multimedia Composition component is developed as such that it enables to seamlessly plug-in additional, possibly more complex or application specific composition modules. This allows the most complex personalized multimedia composition task to be just plugged into the system an be used by a multimedia application. 4. Presentation Format Generators: The composition results in an internal representation (data model) of the multimedia content. For its presentation either a presentation component works on this internal model or based on another (XML-based) multimedia document format. The presentation format s are modules that undertake this conversion from the internal composition into the presentation format like SMIL or HTML+TIME. 5. Multimedia Presentation: The Multimedia Presentation component on top of the framework realizes the interface for applications to actually play the presentation of different multimedia presentation formats. Within this component one can realize an own presentation module that directly presents the internal representation of the composition. But one can also integrate existing presentation components of the common multimedia presentation formats like SMIL or HTML+TIME which the corresponding underlying Presentation Format Generator produces. Besides this central components that support the actual personalized multimedia composition process, we are developing a set of utility components that arise in the context of such applications. 4. Implementation and Prototypical Applications Implementation of the Framework. We have been designing the first version of the framework based on an extensive study of related work in user profiles, media and meta data modeling as well as composition features of multimedia document models [5]. We also derived application requirements to the framework from first prototypes of personalized multimedia applications we developed in different fields such as a personalized sightseeing tour through Vienna [3], a personalized mobile paper chase game [6], and a personalized multimedia music newsletter. The framework, its components, classes and interfaces, are specified using the Unified Modeling Language. We have been implementing the framework in Java. The design and the development process for the framework is carried out as an iterative software development process. Consequently, we implemented a first prototype of the framework which undergoes constant review and re-design phases. This redesign is triggered by the actual experience of implementing the framework but also by employing the framework in several application scenarios. Sample Applications. Currently, we are implementing different application scenarios to prove the applicability of the framework in different application domains. They are the first stress test for the framework. At the same time the development of the sample applications gives us an important feedback about the comprehensiveness and the applicability of the framework. A personalized sightseeing tour creates a multimedia presentation depending on the specific sightseeing preferences of the user and her presentation device. On the basis of video streams and associated meta data, the sightseeing tour is automatically generated on the basis of region of the city the visitors wants to see and the type of sightseeing spots she is interested in like baroque buildings and juvenile style paintings. The output is generated as a SMIL file and presented in a SMIL presentation environment on a PC or a PDA. The personalized media gallery aims a solving the problem of accessing the huge amount of personal media data at home. Instead of manually organizing the content into fixed albums, a user only needs to annotate the media data. Exploiting the framework the personalized media gallery selects the most suitable pictures for the request and generates the personalized multimedia presentation. The application creates, on demand, multimedia presentations according in HTML+TIME or SMIL for the next reunion from the personal media data like me and my summer holidays in Italy, a short story of my life in pictures and videos, or my sweet kids. Smart Authoring. Besides the applications as sketched above that generate personalized content based on the framework, we currently design a smart authoring tool, i.e., a multimedia authoring tool for personalized multimedia content using the framework. Such a tool can be seen as a specialized application employing the framework to create personalized multimedia content. This authoring tool will provide support for a semi-automatic multimedia composition in cases in which an automated creation of multimedia content is not desired, e.g., a very specific content domain in which an expert is still wanted for the decision which content to compose. For example, in the e-learning context, one can imagine an author that designs a multimedia course in cardiac surgery and is supported in the different authoring steps for generating a personalized course for a targeted audience. For this, the Multimedia Composition

5 component can be extended by application specific modules for the composition to achieve a document structure that is suitable just for that content domain and the targeted audience. The Accessor components support the authoring tool in those parts in which it should let the author choose from only those media elements that are suitable for the intended user contexts and that can be adapted to the user s infrastructure. Using the presentation format the authoring tool, finally generates the presentations for the different system infrastructures of the targeted users. Thus the authoring process is guided and specialized with regard to selecting and composing personalized multimedia content. For the development of this authoring tool, the framework fulfils the same function in the process of creating personalized multimedia content in a multimedia application as described in the applications scenarios above. However, the creation of personalized content is not achieved at once but step by step during the authoring process. 5. Conclusion In this paper, we have been presenting a generic, modular framework that supports the development of personalized multimedia applications. The framework relieves application developers from the basic personalization tasks when developing new user-centered multimedia applications. Due to the framework s design, new data models, new composition operator etc. can be easily integrated into the system by implementing the respective new generic module only. The framework approach itself does not outrival other approaches in the field but rather embraces existing approaches in the field. The modular architecture of the framework allows to integrate and embed existing technology at al levels: new user profile models can be integrated by creating a new connector; the same applies to new media storage and retrieval solutions. New, complex or application specific composition capabilities like an intelligent constraint solving system or a specific rhetorical construction of a presentation within a specific composition operator can be integrated into the Multimedia Composition components. As the framework is independent of the presentation format a new or enhanced multimedia presentation format can be embedded with a new respective presentation format. The framework provides a shelf of generic building blocks for personalization which can be employed and adjusted to the specific needs. Consequently, providers of personalized multimedia software can profit from such a multimedia software framework as is supports the modular, stepwise advancements of their applications; newly developed parts of the framework might be reused in different product line. By this more efficient creation of user-centered generation of multimedia content applications can be achieved. References [1] Amazon, Inc., USA. amazon.com, URL: [2] J. Ayars, D. Bulterman, A. Cohen, K. Day, E. Hodge, P. Hoschka, et al. Synchronized Multimedia Integration Language (SMIL 2.0) Specification. W3C Recommendation, URL: Aug [3] S. Boll. Vienna 4 U - What Web Services can do for personalized multimedia applications. In Proceedings der Seventh Multi-Conference on Systemics Cybernetics and Informatics (SCI 2003), Orlando, Florida, USA, July [4] S. Boll, W. Klas, and J. Wandel. A Cross-Media Adaptation Strategy for Multimedia Presentations. In Proc. of the ACM Multimedia Conf. 99, Part 1, pages 37 46, Orlando, Florida, USA, November [5] S. Boll, W. Klas, and U. Westermann. Multimedia Document Formats Sealed Fate or Setting Out for New Shores? In Proc. of IEEE Int. Conf. on Multimedia Computing and Systems (ICMCS 99), Florence, Italy, June IEEE Computer Society. [6] S. Boll, J. Krösche, and C. Wegener. Paper chase revisited - a Real World Game meet Hypermedia (short paper). In Proc. der Intl. Conference on Hypertext (HT03), Nottingham, United Kingdom, August [7] P. D. Bra, P. Brusilovsky, and R. Conejo. Proc. of the Second Intl. Conf. for Adaptive Hypermedia and Adaptive Web- Based Systems. Springer LNCS 2347, Malaga, Spain, [8] P. Brusilovsky, A. Kobsa, and J. Vassileva. Adaptive Hypertext and Hypermedia. Kluwer Acad. Publ., Dordrecht, [9] M. J. F. Bes and F. Khantache. A Generic Architecture for Automated Construction of Multimedia Presentations. In Intl. Conf. on Multimedia Modeling (MMM 2001), Amsterdam, The Netherlands, Nov [10] J. Geurts, J. v.ossenbruggen, and L. Hardman. Applicationspecific contstraints for Multimedia Presentation Generation. In Intl. Conf. on Multimedia Modeling (MMM 2001), Amsterdam, The Netherlands, Nov [11] A. Girgehnsohn, S. Bly, F. Shipman, J. Boreczky, and L. Wilcox. Home Video Editing Made Easy - Balancing Automation and User Control. In Proc. of the Human- Computer Interaction, pages , Tokyo, Japan, [12] Macromedia, Inc. San Francisco, USA. Macromedia Director Shockwave 8, URL: [13] Y. Sun, H. Zhang, L. Zhan, and M. Li. MyPhotos - A System for Home Photo Management and Processing. In Proc. of ACM Multimedia Conf. (ACMMM 2002), Juan-les-Pins, France, Dec [14] J. van Ossenbruggen, F. Cornelissen, J. Geurts, L. Rutledge, and H. Hardman. Cuypers: a semi-automatic hypermedia presentation system. Techn. Rep. INS-R0025, CWI, The Netherlands, Dec