CLST Annual Report 2010

CLST Annual Report 2010 CLST is the Centre for Language and Speech Technology. CLST operates as a separate unit within the Faculty of Arts of the Radboud University Nijmegen. CLST is active in an impressive number of projects in various domains. In 2010 we participated in and acquired projects in the domains of: - ehumanities & datamining - Computer Assisted Language Learning - Communication in health care - Forensic speech processing Our projects are funded through grants from the EU, NWO (the Netherlands Organisation for Scientific Research), The Dutch language Union (STEVIN-programme), the Ministry of Economic affairs (IOP), STW (Technology Foundation; part of NWO), CLARIN-NL. A number of projects are carried out for companies (speech transcriptions, music detection). During 2010, twenty-eight researchers were actively involved in CLST projects. CLST can look back on a stable and fruitful year both with respect to its project port folio and to its financial situation, as the brief account below demonstrates. Board Board members in 2010 were: L. Boves (chairman), A. van Kemenade, D. Iskra, D. Kenyon-Jackson and N. Oostdijk. N. Schröder is advisor of the Board. Final responsibility lies with the Dean of the Faculty. CLST s executive director is Dr H. van den Heuvel. Projects Below we present an overview of the research topics and projects 1 in which CLST was involved in 2010. ehumanities & datamining: - BATS( Speaker tracking and topic detection in audio archives) - TM4IP (Text Mining for Intellectual Property) - LOHW (Living Oral History Workbench ASR-based access to 250 interviews with veterans & development of a web-based annotation tool) Improving automatic speech recognition: - Autonomata Too (recognition of names) - MIDAS (noise robust speech recognition) - SOMS (Second Opinion Music Recognition System as distinguished from speech) Computational modeling of language acquisition and processing: - ACMoLA (A Computational Model of Language Acquisition) Computer assisted second language learning and speech technology: - DISCO (ASR-based CALL for training oral proficiency In Dutch) - TST-AAP (a demosystem for pronunciation training of Dutch) 1 See for detailed information on all projects: http://www.ru.nl/clst 1

- FASOP (Corrective feedback and acquisition of syntax in oral proficiency) - My Pronunciation Coach (Development of a pronunciation trainer for English as second language) Communication in Health Care; - Optifox (Automatic optimization of the settings of a cochlear implant using automatic pronunciation assessment techniques) - FeetBack (combining body sensors and persuasive messages to incite patients to healthy behavior) Production & standardisation of language resources (corpora and tools): - SoNaR (a 500 M words text corpus of written Dutch) - AAM-LR (Demonstrator web application for annotation of audio recordings) - Adelheid (demonstrator web application for tagging historical Dutch texts) - INTER-VIEWs (curation of 250 interviews see LOHW and implementation of a search tool) - TQE (demonstrator web application for automatic quality evaluation of phonetic transcriptions in audio) CLST also participates in Training Networks funded by the EC in the Marie Curie Programme: - S2S (exploring how humans and computers understand speech using fine phonetic details) - SCALE (Speech Communication with Adaptive Learning) - Bbfor2 (Bayesian Biometrics for forensics) Five projects were completed in 2010: Autonomata Too, LOHW, SOMS, TM4IP, Veni-project Odette Scharenborg, Acquisition Our cooperation with the University s Language and Communication institute Radboud in to Languages 2 resulted in a successful application for a Valorisation Grant Phase 2 from STW for a developing a demonstrator web application focusing on pronunciation training for Dutch learners of English. This application uses speech technology to detect pronunciation errors. This should result in a spin-off company bringing the product (called My Pronunciation Coach) to the market by the end of 2011. The Sint Maartenskliniek acquired funding from ZonMW for a project together with CLST aiming at a digital poli webapplication in which language and speech technology will be developed to improve patients communication possibilities through chat-by-click, word prediction, speech synthesis, and speech recognition. Another new project in health care is OptiFox. This is an FP7 SME project from the EC. Aim of the project is automatic optimization of the settings of a cochlear implant using dedicated speech processing software in an application which supports the audiologist in a clinical setting. CLST provides algorithms and speech technology for automatic pronunciation assessment. Suzan Verberne acquired a Google Digital Humanities Award for a research project on extracting factoids from Dutch texts. 2 http://www.ru.nl/radboudintolanguagesuk/ 2

Participation in new initiatives and awareness CLST members are involved in European initiatives with potential impact on the landscape of language and speech technology: - CLARIN, which is a large-scale pan-european collaborative effort to create, coordinate and make language resources and technology available and readily usable in the Humanities and Social Sciences. 3 - FLaReNet, which aims at developing a common vision of the area of language resources and language technologies, and fostering a European strategy for consolidating the sector, thus enhancing competitiveness at EU level and worldwide. 4 At national level CLST keeps track of new trends and developments as well. To that end, CLST participates at Board level in NOTaS. This is a foundation in which national companies and knowledge centers cooperate to stimulate the development and application of language and speech technology in the Dutch language. Personnel Three temporary staff member left CLST in 2010 and found employment elsewhere. Two other temporary staff members had an appointment in a Marie Curie network project and moved to a new position in their location of origin. Three new employees were added to the team: one PhD student, a junior researcher and one post-doc. At the end of 2010 CLST consisted of three tenure staff members, five temporary staff members, ten PhD students, and three members with a temporary part-time posting from the Department of Linguistics, summing up to 19.1 fte. The institute s daily management is in the hands of a director with the support of a secretary. The table below shows a comprehensive overview of the fte distribution in CLST s personnel. NWO-like projects Other external Overhead funding Tenure staff 1.9 0.6 PhD students 5.4 4.8 Other temporary staff 1.6 4.0 Secondments Dept. 0.4 0.4 Linguistics Director & Secretary 0.4 Sum 9.3 9.8 0.4 Annual performance interviews were held with all staff members. These interviews serve to achieve a match between personal ambitions, job contents and career opportunities of individual employees. CLST reserves part of its budget for training and education of its personnel. Most staff members followed one or more courses, most of which were offered by the Radboud University as part of its career policy. 3 http://www.clarin.eu 4 http://www.flarenet.eu 3

Customer Satisfaction For most of its projects CLST is obliged to write progress reports. These progress reports follow the templates offered by the funding bodies (STEVIN, EU, CLARIN-NL). Customer satisfaction in other projects (typically with companies) is monitored by e-mail or telephone. Customer responses are almost without exception very positive. Public Relations Every CLST research employee visited at least one international conference or workshop. Attendance of workshops is, as a rule, restricted to conferences to which the employee contributes an accepted paper. In all contributions the affiliation to CLST and the Radboud University is mentioned. Further PR activities in 2010 were: o Website updates and improvement of its position in rank orders of search engines o Folders and brochures (Dutch, English) o Publicity in newspapers o Periodicals for a general or professional audience such as DIXIT, Kennislink, Onze Taal, Levende Talen Henk van den Heuvel, Director CLST H.vandenHeuvel@let.ru.nl http://www.ru.nl/clst 4

Publications Aimetti, G., Moore, R.K. & Bosch, L.F.M. ten (2010). Discovering an Optimal Set of Minimally Contrasting Acoustic Speech Units: A Point of Focus for Whole-Word Pattern Matching. In Proceedings of interspeech 2010 (pp. 310-313). Makuhari, Japan. Altosaar, T., Bosch, L.F.M. ten, Aimetti, G., Koniaris, Chr., Demuynck, K. & Heuvel, H. van den (2010). A Speech Corpus for Modeling Language Acquisition: CAREGIVER. In Proceedings of LREC 2010 (pp. CD). Malta. Beijer, L., Rietveld, T., Beers, M. van, Slangen, R., Heuvel, H. van den, Swart, B.J.M. de & Geurts, A.C.H. (2010). E-learning based Speech Therapy (EST): a web application for speech training. Telemedicine Journal and e-health, 16(2), 177-180. Bergmann, C., Gubian, M. & Boves, L.W.J. (2010). Modelling the effect of speaker familiarity and noise on infant word recognition. In Proceedings of Interspeech 2010 (pp. CD). Makuhari, Japan. Bergmann, C., Paulus, M.A. & Fikkert, J.P.M. (2010). A closer look at pronoun comprehension: comparing different methods. In J. Costa, A. Castro, M. Lobo & F. Pratas (Eds.), Language Acquisition and Development: Proceedings of GALA 2009 (pp. 53-61). Cambridge: Cambridge Scholars Publishing. Bosch, L.F.M. ten & Boves, L.W.J. (2010). Language Acquisition and Cross-Modal Associations: Computational Simulation of the Result of Infant Studies. In Proceedings of Interspeech 2010 (pp. 2926-2929). Makuhari, Japan. Bruijn, M. de, Bosch, L.F.M. ten, Kuik, D.J., Langendijk, J.A., Leemans, C.R. & Verdonck-de Leeuw, I.M. (2010). Neural network analysis to assess hypernasality in patients treated for oral or oropharyngeal cancer. In Proceedings 28th World Congress of the International Association of Logopedics and Phoniatrics (pp. 136-136). Athene, Griekenland. Bruijn, M. de, Bosch, L.F.M. ten, Kuik, D.J., Langendijk, J.A., Leemans, C.R. & Verdonck-de Leeuw, I.M. (2010). Objective assessment of speech quality in patients treated for a tumour in the oral cavity or oropharynx. In Proceedings 28th World Congress of the International Association of Logopedics and Phoniatrics (pp. 137-137). Athene, Griekenland. Cheng, C., Xu, Y. & Gubian, M. (2010). Exploring the Mechanism of Tonal Contraction in Taiwan Mandarin. In Proceedings of Interspeech 2010. Makuhari, Japan. Cucchiarini, C., Doremalen, J.J.H.C. van & Strik, H. (2010). Fluency in non-native read and spontaneous speech. In Proceedings of DiSS-LPSS Joint Workshop 2010 (pp. CD). Tokyo, Japan. Cucchiarini, C. (2010). Language resources, speech technology and language learning: how to establish a virtuous circle. CLARIN-EU Newsletter. [Online]. Available from: http://www.clarin.eu/newsletter [01-01-2010]. D'hondt, E.K.L. & Verberne, S. (2010). CLEF-IP 2010: Prior Art Retrieval using the different sections in patent documents. In Proceedings of the Conference on Multilingual and Multimodal Information Access Evaluation (CLEF 2010), CLEF-IP workshop. Padua, Italy. Available from: http://clef2010.org/index.php?page=pages/proceedings.php [22-09-2010]. D'hondt, E.K.L., Verberne, S., Oostdijk, N.H.J. & Boves, L.W.J. (2010). Re-ranking based on Syntactic Dependencies in Prior-Art Retrieval. In Proceedings of the Dutch-Belgium Information Retrieval workshop 2010 (DIR-2010) (pp. 23-30). Nijmegen. Doremalen, J.J.H.C. van, Cucchiarini, C. & Strik, H. (2010). Optimizing automatic speech recognition for low-proficient non-native speakers. EURASIP Journal on Audio, Speech and Music Processing. [Online]. Available from: http://www.hindawi.com/journals/asmp/2010/973954.html [05-09-2009]. Doremalen, J.J.H.C. van, Cucchiarini, C. & Strik, H. (2010). Phoneme Errors in Read and Spontaneous Non-Native Speech: Relevance for CAPT System Development. In Proceedings of the SLaTE-2010 workshop (pp. CD). Tokyo, Japan. Doremalen, J.J.H.C. van, Cucchiarini, C. & Strik, H. (2010). Using Non-Native Error Patterns to Improve Pronunciation Verification. In Proceedings of Interspeech 2010 (pp. CD). Tokyo, Japan. Doremalen, J.J.H.C. van, Strik, H. & Cucchiarini, C. (2010, mei 26). Automatic Speech Recognition in CALL: The Essential Role of Adaptation. Katholieke Universiteit Leuven, Campus Kortrijk (KULAK), Kortrijk, ITEC. 5

Gemmeke, J.F., Hamme, H. Van, Cranen, B. & Boves, L.W.J. (2010). Compressive Sensing for Missing Data Imputation in Noise Robust Speech Recognition. IEEE Journal of Selected Topics in Signal Processing, 4(2), 272-287. Gemmeke, J.F., Cranen, B. & Remes, U. (2010). Sparse imputation for large vocabulary noise robust {ASR}. Computer Speech & Language. [Online]. Available from: http://dx.doi.org/10.1016/j.csl.2010.06.004 [01-04-2010]. Gemmeke, J.F. & Virtanen, T. (2010). Artificial and online acquired noise dictionaries for noise robust {ASR}. In Proceedings of Interspeech 2010 (pp. 2082-2085). Makuhari, Japan. Gemmeke, J.F. & Virtanen, T. (2010). Noise robust exemplar-based connected digit recognition. In Proceedings ICASSP 2010 (pp. DVD). Dallas, Texas, USA. Gemmeke, J.F., Remes, U. & Palomäki, K.J. (2010). Observation uncertainty measures for sparse imputation. In Proceedings of Interspeech 2010 (pp. 2262-2265). Makuhari, Japan. Gubian, M., Cangemi, F. & Boves, L.W.J. (2010). Automatic and Data Driven Pitch Contour Manipulation with Functional Data Analysis. In M. Hasegawa-Johnson (Ed.), Proceedings of Speech Prosody (pp. 100231:1-4). Chicago, IL, USA. Gubian, M., Bergmann, C. & Boves, L.W.J. (2010). Investigating word learning processes in an artificial agent. In Development and Learning (ICDL), Proceedings of the Ninth IEEE International Conference (pp. 178-184). Available from: http://dx.doi.org/10.1109/devlrn.2010.5578847 [18-08-2010]. Halteren, H. van (2010). In bewerking. ACM Transactions on Speech and Language Processing (TSLP). Heijden, M van der, Hinne, M., Verberne, S., Hoenkamp, E.C.M., Weide, T. van der & Kraaij, W. (2010). When is a query a question? Reconstructing wh-requests from ad hoc-queries. In B. Croft (Ed.), Query Representation and Understanding : Workshop of the 33rd Annual International ACM in Information Retrieval on Research and Development SIGIR Conference (pp. 17-20). [S.l]: ACM. Heuvel, H. van den, Horik, R. van, Scagliola, S.I., Sanders, E.P. & Witkamp, P. (2010). The VeteranTapes: Research Corpus, Fragment Processing Tool, and Enhanced Publications for the e-humanities. In Proceedings of LREC (pp. CD). Malta. Heuvel, H. van den (2010). Interviews, geknipt voor onderzoek. DIXIT, 7(1), 20-21. Huijbregts, M.A.H. & Leeuwen, D.A. van (2010). Towards automatic speaker retrieval for large multimedia archives. In Proceedings AIEMPro. Firenze: ACM. Jongenelen, M.M., Hoeken, H. & Hendriks, B.C. (2010). Explicit and implicit messages in telehealth. Information Design Journal, 18(3), 198-204. Oostdijk, N.H.J., D'hondt, E.K.L., Halteren, H. van & Verberne, S. (2010). Genre and Domain in Patent Texts. In Proceedings of The 3rd International Workshop on Patent Information Retrieval at CIKM 2010 (pp. 39-46). Oostdijk, N.H.J., Verberne, S. & Koster, C.H.A. (2010). Constructing a broad coverage lexicon for text mining in the patent domain. In Proceedings of LREC 2010 (pp. 2292-2299). Malta: European Language Resources Association (ELRA). Penning de Vries, B.W.F., Cucchiarini, C., Strik, H. & Hout, R.W.N.M. van (2010). The Role of Corrective Feedback in Second Language Learning: New Research Possibilities by Combining CALL and Speech Technology'. In Proceedings of L2WS (pp. USB). Tokyo, Japan. Réveil, B., Martens, J-P. & Heuvel, H. van den (2010). Improving Proper Name Recognition by Adding Automatically Learned Pronunciation Variants to the Lexicon. In Proceedings of LREC 2010 (pp. CD). Malta. Reynaert, M., Oostdijk, N.H.J., Clercq, O. De, Heuvel, H. van den & Jong, F. de (2010). Balancing SoNaR: IPR versus Processing Issues in a 500-Million-Word Written Dutch Reference Corpus. In Proceedings of LREC 2010 (pp. 2693-2698). Malta: European Language Resources Association (ELRA). Ruiter, M., Rietveld, T., Cucchiarini, C., Krahmer, E. & Strik, H. (2010). Human Language Technology and communicative disabilities: Requirements and possibilities for the future. In Proceedings of LREC 2010 (pp. 2839-2846). Malta. Sanders, E.P. & Heuvel, H. van den (2010). Automatic Pronunciation Error Detection in Repetitor. In Proceedings L2WS 2010, Tokyo, Japan. Tokyo. Available from: http://www.gavo.t.u-tokyo.ac.jp/l2ws2010/papers/l2ws2010_o2-04.pdf [22-09-2010]. Scharenborg, O.E. & Boves, L.W.J. (2010). Computational modelling of spoken-word recognition processes: design choices and evaluation. Pragmatics and Cognition, 18(1), 136-164. 6

Strik, H., Colpaert, J., Doremalen, J.J.H.C. van & Cucchiarini, C. (2010). Language resources and CALL applications: speech data and speech technology in the DISCO project. In Proceedings E-learning Workshop 2010 (pp. CD). Valletta, Malta. Strik, H., Loo, J. van de, Doremalen, J.J.H.C. van & Cucchiarini, C. (2010). Practicing Syntax in Spoken Interaction: Automatic Detection of Syntactic Errors in Non-Native Utterances. In Proceedings of the SLaTE 2010 Workshop (pp. CD). Tokyo, Japan. Sun, Y., Bosch, L.F.M. ten & Boves, L.W.J. (2010). Hybrid HMM/BLSTM-RNN for Robust Speech Recogntion. In Proceedings of Text, Speech and Dialogue (TSD) 2010 (pp. 400-407). Brno, Czech Republic. Sun, Y., Gemmeke, J.F., Cranen, B., Bosch, L.F.M. ten & Boves, L.W.J. (2010). Using a DBN to Integrate Sparse Classification and GMM-Based ASR. In Proceedings of Interspeech 2010 (pp. 2098-2101). Makuhari, Japan. Verberne, S., Halteren, H. van, Raaijmakers, S., Theijssen, D.L. & Boves, L.W.J. (2010). Learning to Rank for Why-Question Answering. Information Retrieval, online. [Online]. Available from: http://dx.doi.org/10.1007/s10791-010-9136-6 [05-06-2010]. Verberne, S., Boves, L.W.J., Oostdijk, N.H.J. & Coppen, P.A.J.M. (2010). What is not in the Bag of Words for Why-QA? Computational Linguistics, 36(2), 229-245. Verberne, S., Hinne, M., Heijden, M van der, Hoenkamp, E., Kraaij, W. & Weide, T. van der (2010). How does the Library Searcher Behave? A Contrastive Study of Library Search against Ad-hoc Search. In Proceedings of the Conference on Multilingual and Multimodal Information Access Evaluation (CLEF 2010), logclef workshop (pp. 1-5). [S.l:: s.n]. Verberne, S., Vogel, Merijn & D'hondt, E.K.L. (2010). Patent classification experiments with the Linguistic Classification System LCS. In Proceedings of the Conference on Multilingual and Multimodal Information Access Evaluation (CLEF 2010), CLEF-IP workshop (pp. CD). Verberne, S., D'hondt, E.K.L., Oostdijk, N.H.J. & Koster, C.H.A. (2010). Quantifying the Challenges in Parsing Patent Claims. In Proceedings of the 1st International Workshop on Advances in Patent Information Retrieval at ECIR 2010 (pp. 14-21). [S.l]: [s.n]. Versteegh, M.H. (2010). Active Word Learning Under Uncertain Input Conditions. In Proceedings of Interspeech 2010 (pp. 2930-2933). Makuhari, Japan. Virtanen, T., Gemmeke, J.F. & Hurmalainen, A. (2010). State-based labelling for a sparse representation of speech and its application to robust speech recognition. In Proceedings of Interspeech 2010 (pp. 893-896). Makuhari, Japan. Zellers, M., Gubian, M. & Post, B. (2010). Redescribing Intonational Categories with Functional Data Analysis. In Proceedings of Interspeech 2010 (pp. CD). Makuhari, Japan. 7