Exploiting Films and Multiple Subtitles Interaction for Casual Foreign Language Learning in the Living Room

Transcription

1 Exploiting Films and Multiple Subtitles Interaction for Casual Foreign Language Learning in the Living Room Victor Bayon Ambiente Research Division, Fraunhofer IPSI, Dolivostrasse 15, Darmstadt, Germany Abstract. Films at the cinema, at home or on the computer screen are a fundamental part of today s entertainment culture. The careful listening and watching of foreign films with or without subtitles is often used as an aid for the learning of foreign languages. This paper presents the ongoing development of an entertainment/learning computer based DVD subsystem for the home environment that has been extended with the aim of further enabling the learning of foreign languages while watching films. By providing a DualSubsView of subtitles of 2 languages (native and foreign) as the film plays, users can become more familiar with the vocabulary of the foreign language that they want to learn. With this approach, vocabulary in a foreign language is presented as extra information embedded into the film and learners can exploit the information directly or indirectly for entertainment and for learning. Potential users of the system were involved in the initial design stages and in the informal evaluation process of the initial prototypes. 1 Introduction Computers, in general, have a long and evolving history as tools or aids to support entertainment [5] and the learning of foreign languages [3]. As computers evolve, they are continuously adopting new form/factors and functionality, disappearing from their current appearances and contexts and reappearing into different ones with different form/factors and new application domains [4]. One of the areas where computers are converging and reshaping the home entertainment is towards their integration as multimedia hubs for the home [2], where computers can be shaped into Home Media Appliances to offer sophisticated entertainment. Such functionality can include capabilities for storing, organising, pre and post processing, replaying and interacting with all types of media and media sources, pushing fully featured computers into the living/entertainment spaces of households. Standard home Audio/Visual equipment (A/V), such as radio, broadcast TV, Video/DVD, interactive TV, etc, are commonplace in living rooms with the potential and the tradition of being used as an entertainment and learning tool [1][8]. DVD players, with their interactive and multi-track capabilities, can already, for example, display a film in multiple languages and with subtitles and can be used this way to practice listening and reading a foreign language.

2 As learning a foreign language can be a very long and complicated process where many skills have to be developed, intrinsic motivation for learning plays a very important part in the process [Nakanishi,2002]. Film watching is an entertaining, proactive and intrinsically motivating experience and with the integration of home media appliances into the household new applications to support foreign language learning can be developed. The paper describes the development of a system with the purpose of support the advanced used of subtitles in films for language learning. First of all, ideas and user requirements were gathered conducting an informal survey among learners of foreign languages. With this input, two integrated applications and one prototype have been initially developed: a DualSubsView DVD player where 2 subtitles in 2 different languages are shown, a movie recommender system that determines which films might be more suitable for the learner to watch by comparing the words/vocabulary used in film and the vocabulary used in the language course and a PDA based remote control application. Finally the applications have been evaluated using an expert review and user feedback in order to gather their initial impressions of the system as part of the user-centred design process. The paper concludes with a summary and ideas for future work based around the concepts presented in this paper. 2 Designing the System In order to design the system and to gather initial ideas and user requirements, an informal survey conducted via interviews and moderated discussions between the designers and members of a language course for mature students in a local college was conducted. The purpose of the survey was to find out what out-of-class materials and activities of structured (course materials) and un-structured nature (that are not part of the language course) students used to reinforce their structured learning at the college. The conversations among the designers and the mature students revealed that although DVD of foreign films with subtitles were commonly used as an un-structured learning aid (among many other activities), it presented several challenges for the students, especially among the ones at the beginner s level. Although the survey highlighted some issues that are typical and expected to foreign language students, it provided enough design input and discussions among the designers and students to generate the first ideas for the initial application prototypes and to try to tackle some of them within the context of film watching in the home DualSubsView The application improves on the current subtitles and sub-captioning of standard DVD technology by enabling the possibility of visualising subtitles in 2 languages at the same time (Fig 1). The initial contact with the students revealed the need for

3 shortening the gap between the language knowledge required to understand and the ability to follow a film with subtitles. With the inclusion of the 2 subtitles channels, the students can view a film with the option of seeing the subtitles in two different languages (the native and the target language) and try to follow and understand the dialogues, the subtitles, or both. In fig 1 the screenshot of the DualSubsView DVD player shows two channels for text, top text for the English subs and bottom text for the German subs. Students can follow several strategies in order to understand and learn from the conversation and find out ad-hoc which ones are more suitable for them depending on their foreign language skills. A typical strategy is matching the foreign audio/foreign track and rely on the native text to match the words that are unknown. Following this strategy, the students can reinforce the listening, the vocabulary that they already know and acquire new vocabulary. Fig. 1. DualSubsView. 2.2 Movie Recommender As all movies can differ in types of context, discussions, dialogue structures and vocabulary used, the task of selecting which movie to watch in order to practise listening and reading according to the student s language level can be a difficult task without external recommendation (such as the lecturer on the course or other students that know the films already). A simple movie recommender application has been built in order to provide basic advice and complexity awareness to the students of which movie might be more

4 suit-able for them to watch It focuses on the most basic feature of the films: the vocabulary used. The movie recommender holds a database of the vocabulary learned by the students in the course. The vocabulary is then compared with the vocabulary used in the films to determine how complex films are in relation to the course s vocabulary. A standard foreign language course can contain, for example, from 1200 basic words to many thousands. Fig. 2. Movie Recommender. Fig 2 shows the interface front-end of the recommender system, where the film The Big Lebowski is the film that contains most words or word repetitions (52%) from the user s dictionary and therefore has the highest recommendation percentage, and in principle the student could have more possibilities to see/hear the vocabulary that is part of his/her course. The column Words In Dict represents the number of matches (with repetitions) between dictionary words and those in the film.. The column Unique Words represents the number of words in the films (without repetition). The comparisons are made using an approximate/fuzzy word by word comparison. Although there are existing recommending approaches and movie recommenders based on advanced text categorisation techniques [Mark, 2003], the basic heuristics employed here can be sufficient for the purpose of helping the students to identify the elemental features of the vocabulary in each film and help them to make a decision. The main purpose of the recommender is to make the students aware of how complex the vocabulary in a film is in relation to the vocabulary on their course. 2.3 Remote Control The application, as it is designed for the living room, must have means of being controlled remotely. The PDA based Remote Control (RC) application includes the basic functions of a normal DVD player (start/stop/play, etc). This functionality is accessible via the hardware buttons on the PDA. The RC connects to the main application on the computer via a wireless network. Further functionality of the RC is in a mock-up/prototyping stage in order to provide further interaction with the subtitles via the RC.

5 Fig. 3. Remote Control mock-up interface. One more feature that is currently being prototyped, is to produce an interface that is able to display the subtitles on the PDA RC during the film. The aim of this interface is to enable the students to pause and to interact with the (foreign) text and read it as well from the PDA. For example, students can have the possibility of including the words that are not in the database (Fig 3) by selecting them with the PDA. As the film plays, the subtitles are displayed on the PDA. When a word is selected on the PDA, the DVD pauses and the word is looked up in a dictionary, its meaning is retrieved and the database gets updated with the word. This interaction modality allows the students to further explore the meanings of words, allowing to pause the film and reflect on the language used by the film. 3 Evaluation of The Initial Prototypes With the current stage of development, the prototypes have been evaluated using an expert walkthrough and user feedback. A lecturer reviewed the system indicating the potential of the system as a complementary learning tool. As both subtitles are shown, students can learn new words and reinforce the ones that they already know with the possibility of observing how grammatically and semantically the expressions in both languages are constructed. Another observation that the lecturer remarked upon is that as the spoken dialogues can differ from the subtitles, students could reflect on these differences and that the system has the potential to be used as a complementary activity and aid on top of a structured foreign language course. Students have also reviewed informally and used the system. The students commented that the system could have the potential of giving further use and bring a new dimension to their DVD collections as movies can be watched again not just for their entertainment value but also for their learning potential. However, students pointed out that having two subtitles displayed at the same time requires a lot of concentration in order to be able to follow the text and the audio track if there are many un-known words/sentences or if the movie is new. The students' suggestion was to watch the movie more than once in order to focus on different aspects of the movie first (story line, characters, etc) and then on the languages afterwards.

6 4 Summary, Future Work This short paper has described the initial design of a system for complementing the study of a foreign language course based on the concept of displaying subtitles in 2 different languages during the viewing of a DVD based movie with the objective of vocabulary acquisition and reinforcement, complimented with listening and reading skills. The system incorporates a recommender system that facilitates the task of determining the richness of the vocabulary of a film in relation to the vocabulary used on the course and a PDA based remote control application. The system has been developed using an iterative design process involving practitioners, users and designers with the goal of producing an application that can be used both for entertainment and for foreign language learning, and that can be integrated into living room and the lifestyle of the students. Although the system is currently in a prototype stage and undergoing development, the stakeholders involved have shown interest in the concept and in the prototypes. Future work will include the enhancement of the functionality and usability of the remote control and the recommender system and further evaluation of the system as a learning aid. This work has been funded under the ERCIM Fellowship Program. The author would like to thanks to all the members of Ambiente for their ideas, suggestions and comments on early drafts of this paper and to all the learning stakeholders that participated directly or indirectly in the research. References 1. BBC Learning Bell, G, Gemmell, J.: A Call For The Home Media Network. Communications of the ACM. 45(7), July 2002, Chapelle, C. A., Long, M. H., Richards, J. C.: Computer Applications in Second Language Acquisition. Cambridge University Press (2001) 4. The Disappearing Computer Initiative Holmquist, L.E., Bjrk, S., Falk, J., Jaksetic, P., Ljungstrand, P., Redstrm, J.: What About Fun and Games? In Ljungberg, F. (ed.): Informatics in the Next Millennium, Lund: Studentlit-teratur, Mak, H., Koprinska, I., Poon, J,: INTIMATE: A Web-Based Movie Recommender Using Text Categorization. IEEE/WIC International Conference on Web Intelligence (WI 03). October 13-17, Halifax, Canada 7. Nakanishi, T.: Critical Literature Review on Motivation. Journal of Language and Linguistics. Vol. 1 No. 3 (2002 ) Pemberton L The Potential of Interactive Television for Delivering Individualised Language Learning Future TV Workshop ITS 2002 conference in San Sebastian/Biarritz, June 2002