Motion, Sound, Interaction Design, Machine Learning

Transcription

1 Dr. Caramiaux Baptiste b.caramiaux@gold.ac.uk web : http :// twitter tel : Department of Computing Goldsmiths, University of London New Cross, London, SE14 6NW UNITED KINGDOM Motion, Sound, Interaction Design, Machine Learning Research Associate Date and place of birth: 24 th September 1983, Soissons, France Citizenship: French Place of Residence: 143 Clifden Road, London E5 0LW, United Kingdom RESEARCH EXPERIENCE Present Goldsmiths College, University of London, UK Research Associate (European Research Council) with Prof. Atau Tanaka Topic: Machine Learning of Musical Gestures STMS Lab Ircam-CNRS-UPMC Research Associate (ANR funding) with Dr. Frédéric Bevilacqua Topic: Sensorimotor Learning in Gesture-Sound Interactive Systems University Paris 6, Ircam CNRS Research Assistant (PhD Student) Topic: Relationships between Gesture and Sound in Musical Performance McGill University, Montreal, Canada Visiting researcher (COST Actions) with Prof. Marcelo M. Wanderley Topic: Computational Models for the Analysis of Instrumentalists Gestures EDUCATION PhD, Computer Science University Paris 6 and Ircam Centre Pompidou, Paris, France Topic: Studies on the relationship between gesture and sound in musical performance MSc, Acoustics, Signal Processing, Computer Science, applied to Music University Paris 6, Telecom ParisTech and Ircam Centre Pompidou BSc, Mathematics Polytechnic University of Catalonia, Barcelona, Spain TEACHING EXPERIENCE 2014 March, April. Teaching a course in Machine Learning for Musical Performance at Goldsmiths University of London and SAE Institute in London 2012 April. Teaching a course a week master class in Gesture-based Interaction for Sound Design. At the Fine Arts School of Le Mans, France 2011 January to May. Course of Human Computer Interaction for Masters at the EFREI (Engineering school, Computer Science), Villejuif, France 2011 September to December. Course of imperative programming in C in bachelor classes INVITED TALKS & CONFERENCE PARTICIPATION Invited talks (selected) EARZOOM Festival organised by IRZU, Ljubljana, Slovenia. Presentation and Demo of the softwares developed for gesture recognition and variation tracking. Specific focus on

2 possible transfer to the Industry Lecture at SAE Institute, invited by Andy Farnell. Presenting possibilities of motion-based interactions to game designers and sound engineers Talk at the Open University, London, UK, Department of Computing. Overview of my research Talk at Queen Mary University, C4DM group. Overview of the applicative part of my research Talk at MIT Media Lab, invited by the Tangible Media group, led by Iroshi Ishii. Talk about how Music can inform on the design of innovative interfaces for creative and exploratory applications Talk at Parsons the New School of Design, New York, USA. I presented techniques for Sonic Interaction Design based on body motion Talk at the University of Edinburgh on machine learning of musical gestures Talk at SIGCHI Paris, University of Paris XI on techniques for recognizing gestures in realtime and applications in HCI. ORGANISATION OF CONFERENCES, SYMPOSIUMS, WORKSHOPS Organisation of International Conferences 2014 Scientific Chair of NIME (International Conference on New Interfaces for Musical Expression). As a scientific chair of the conference I set up a new review process in order to improve the quality of the scientific selection. More precisely I created an intermediate level of meta-reviews between members of the committee and chairs. The conference gathers 400 attendees. See Organisation of International and National Symposiums 2010 Core organisation team for the Journées des Jeunes Chercheurs en Audition, Acoustique Musicale et Signal Audio. The 2-day symposium is meant to be organised by PhD students with only PhD students attending. The event gathers about 100 attendees. The event was hosted at Ircam Organisation of the International Symposium GDIF / SpatDIF, a community-based symposium about sharing data and analysis tools for the analysis of sound and gesture. The symposium gathers 40 attendees at Ircam Organisation of Workshops 2014 Machine learning, Expressive Movement, Interaction Design, Creative Application organised during the AISB Convention at Goldsmiths, University of London 2013 Embodied Sonic Interaction Design III Ircam Paris France 2013 Embodied Sonic Interaction Design II Parsons The New School of Design, New York, USA 2013 Embodied Sonic Interaction Design I, Goldsmiths, London 2013 Machine Learning Playground, Goldsmiths, London 2012 Designing digital musical interaction based on everyday objects, Orsay, France 2012 Designing musical interaction with augmented objects, Le Mans, France Program committee member Journals Conferences Computer Music Journal Pattern Recognition Letters Journal of New Music Research ACM CHI (Computer-Human Interaction), DIS (Designing Interactive Systems), TEI (Tangible Embodied and Embedded Interaction)

3 SUPERVISION ACTIVITIES NIME (New Interfaces for Musical Expression), SMC (Sound and Music Computing), ICMC (International Conference in Music Computing), INTER-FACE (Symposium on Live Interfaces), MOCO (Motion and Computing) Since 2012 I have been co-supervising with Prof. Atau Tanaka two PhD students, Alessandro Altavilla and Marco Donnarumma, at Goldsmiths University of London I supervised two bachelor students for a short-term project Alejandro Van Zandt-Escobar (Princeton University, one term exchange) on a 4-month project and Eliza Pedrozo, also on a 4-month project I supervised two bachelor students for summer projects (3 months): Igor Correia and Tom Rushmore I supervised two MSc students from University of Pierre et Marie Curie for their Master thesis: Jules Françoise (now PhD student at Ircam in Paris) and Aymeric Masurelle (now PhD student at Telecom ParisTech). REALIZATIONS Software development Gesture Variation Follower (GVF) GVF is my main development activity. The GVF Library allows for real-time gesture recognition and gesture variation tracking. It can be interfaced in various creative coding softwares: openframeworks, MaxMSP and PureData. The software has received good reviews on social networks and it is currently in testing in different projects (performing arts, video games, interactive cinema). Gestural Sound Toolkit I am the main developer of this library that allows for quick prototyping of gestural sonic interactions. The toolkit has been used in several workshops but also in lectures. The Library is implemented in the MaxMSP software. Exhibitions 2014 Transport for London Festival, Musical Bus Stop, Regent Street, London 2014 Transmediale, Digital Arts Festival, Berlin, Germany 2012 Designing Interactive Systems, Art Track Newcastle UK 2012 Printemps de la Culture (Art-Science French Festival) Paris France AWARDS Young Researcher Prize in Science and Music, organised by IRISA and sponsored by AFIM and Foundation Rennes COST Action SID (Sonic Interaction Design) European Grant for short-term scientific mission, 1 month 2008 Grant for Doctoral Studies EDITE Paris, 3 years 2007 Grant of Excellence SKILLS Languages Programming France Ministry of Research, 1 year French mother tongue English very good level (spoken and written) Spanish good level (spoken and written) C++, Python, Java, C, MaxMSP, PureData, Matlab

4 Music Guitar (15 years), Clarinet (5 years), Ableton Live, Cubase PUBLICATIONS Journals B. Caramiaux, M. Donnarumma, A. Tanaka. Understanding Gesture Expressivity through Muscle Sensing. ACM Transactions on Computer-Human Interaction (ToCHI) (In Press) B. Caramiaux, N. Montecchio, A. Tanaka, F. Bevilacqua. Adaptive Gesture Recognition with Variation Estimation for Interactive Systems. ACM Transactions on Interactive Intelligent Systems (TiiS) (In Press will be published in the Volume 4, Number 4, December 2014) B. Caramiaux, J. Françoise, F. Bevilacqua, N. Schnell. Mapping Through Listening. Computer Music Journal, 38(3), pp B. Caramiaux, F. Bevilacqua, T. Bianco, N. Schnell, O. Houix, P. Susini. The role of sound source perception in gestural sound description. ACM Transactions on Applied Perception (TAP), 11(1), pp B. Caramiaux, M.M. Wanderley, F. Bevilacqua. Segmenting and Parsing Instrumentalists' Gestures. Journal of New Music Research, 41(1), pp S. Hahmann, G.-P. Bonneau, M. Cornillac, B. Caramiaux. Multiresolution morphing for planar curves. Computing, 79 (2-4), pp Book Chapters D. Schwarz, B. Caramiaux. Interactive Sound Texture Synthesis through Semi-Automatic User Annotations. Lecture Notes in Computer Science (In Press) B. Caramiaux, F. Bevilacqua, N. Schnell. Towards a Gesture-Sound Cross-Modal Analysis. In Embodied Communication and Human-Computer Interaction, volume 5934 of LNCS. pp Conferences Peer-reviewed (selected) B. Caramiaux. Motion Modelling for Expressive. MOCO 14 Proceedings of the International Workshop on Movement and Computing. pp.76-81, 2014 B. Caramiaux, A. Tanaka. Machine Learning of Musical Gestures. NIME 13 Proceedings of the International Conference on New Interfaces for Musical Expression. pp , 2013 A. Altavilla, B. Caramiaux, A. Tanaka. Towards Gestural Sonic Affordances. NIME 13 Proceedings of the International Conference on New Interfaces for Musical Expression. pp.61-64, 2013 B. Caramiaux, F. Bevilacqua, A. Tanaka. Beyond Recognition: Using Gesture Variation for Continuous Interaction. CHI 13 Alt.CHI Extended Abstracts on Human Factors in Computing Systems. pp , 2013 (acc. rate: 30.3%) S. Fdili Alaoui*, B. Caramiaux*, M. Serrano, F. Bevilacqua. Movement Qualities as Interaction Modality. DIS 12 Proceedings of the Designing Interactive Systems Conference. pp , 2012 (acc. rate: 19.8%) honorable mention (* equal contribution on the paper) J. Françoise, B. Caramiaux, F. Bevilacqua. A Hierarchical Approach for the Design of Gesture-to-Sound Mapping. SMC 12 Proceedings of the 9th Sound and Music Computing Conference. pp , 2012 (acc. rate: 54.9%) K. Nymoen, B. Caramiaux, M. Kozak, J. Torresen. Analyzing Sound Tracings A Multimodal Approach to Music Information Retrieval. ACM Multimedia MM'11, Workshop MIRUM, Scottsdale, Arizona, USA B. Caramiaux, P. Susini, T. Bianco, F. Bevilacqua, O. Houix, N. Schnell, N. Misdariis. Gestural Embodiment of Environmental Sounds: an Experimental Study. NIME'11, Oslo, Norway B. Caramiaux, F. Bevilacqua, N. Schnell. Sound Selection by Gestures. NIME 11 Proceedings of the International Conference on New Interfaces for Musical Expression. pp , 2011

5 S. Fdili Alaoui*, B. Caramiaux*, M. Serrano. From dance to touch: movement qualities for interaction design. CHI 11 Extended Abstracts on Human Factors in Computing Systems. pp , 2011 (* equal contribution on the paper) B. Caramiaux, F. Bevilacqua, N. Schnell. Analysing Gesture and Sound Similarities with a HMM-based Divergence Measure. SMC 10 Proceedings of the 6th Sound and Music Computing Conference Dissertation B. Caramiaux. Studies on the Relationship between Gesture and Sound in Musical Performance. PhD Dissertation, University Pierre et Marie Curie (Paris VI), Ircam - Centre Pompidou. 2012