Multimedia Environment for Technology Enhanced Music Education and Composition and Performance

Size: px

Start display at page:

Download "Multimedia Environment for Technology Enhanced Music Education and Composition and Performance"

Dominic Nichols
8 years ago
Views:

1 I-MAESTRO: Interactive Multimedia Environment for Technology Enhanced Music Education and Creative Collaborative C Composition and Performance Kia Ng, ICSRiM - University of Leeds, UK Paolo Nesi, DISIT-DSI - University of Florence, Italy p. 1

2 Motivations Response to requirements from music institutions and market Music learning and teaching inherently time consuming, expensive one-to-one, particularly for instrumental performances Increase training efficiency while reducing cost optimise effectiveness enhance attractiveness widening participation and accessibilities p. 2

3 Introduction Research and development on innovative technologyenhanced environment for music learning & teaching Background and Relevant R&D EC projects and others MUSICNETWORK ISO MPEG Symbolic Music Representation (MPEG-SMR) standard MOODS; WEDELMUSIC; IMUTUS; COST287-ConGAS; CANTATE; PLAY2; CUIDADO; MvM, and others Music Learning & Teaching Theory and Performance Focus on string instruments Expressions, gesture 6 th Framework Programme, Priority : Technologyenhanced Learning 3 years: 1 Oct 2005 to 30 Sept 2008 p. 3

4 I-MAESTRO Consortium p. 4

5 Main Aims Produce novel methods and tools for music education create computer-assisted tuitions with theory training: individual and group/collaborative explore innovative pedagogical paradigms with cooperative and interactive self-learning environments integrated digital media interactive multimedia environment take into account key factors, such as expressivity, interactivity, gesture controllability cooperative-working accessibilities performance training (for strings): solo and collaborative playing creative interfaces: virtual- and augmented-instruments p. 5

6 Main Technical Objectives and Challenges to support the creation of e-learning courses personalisable for different students to support traditional pedagogical paradigms for music training with innovative and technical solutions innovative pedagogical paradigms (such as cooperative work, self-learning and class work) Symbolic Training paradigms Practice Training paradigms for string instruments (non- MIDI-based) including interactive, gesture-based and creative tools p. 6

7 Main Technical Objectives (cont.) to explore an unified educational model bring together theory and practical aspects taking into account different methodologies, paradigms and scenarios to utilise different pedagogical methodologies to increase efficiency, motivation and interests in the learning processes to reduce the costs of music courseware production and customisation support teachers and students in the learning and teaching process by developing semi-automatic/automatic tools based on: pedagogical aspects of music education, enabling technologies for music education exploiting interactivity and cooperative work, pedagogical tools for music education p. 7

8 Main Technical Objectives (cont.) to further enhance the growing standard of the ISO MPEG SMR model to integrate it with educational aspects model to create a framework for technology-enhanced music educational models and tools to carry out basic research on new solutions and enabling technologies to improve participation and accessibilities to the musical knowledge and heritage p. 8

9 Results Expected outcomes include framework and tools methods enhanced standards (e.g. MPEG-SMR) exploitation plan to transfer the R&D into products and services Validation innovative tools for music courseware production interactive and creative tools for music training different environments: classrooms and cooperative work different devices: PC, PDA, etc assessment and pedagogical models for music training p. 9

10 I-MAESTRO Overview Sensors Interface Posture & Gesture Visual Rendering & Interaction Audio Rendering & Acquisition Accessible Interface Symbolic Training Processing Tools I-MAESTRO DB Access I-MAESTRO Music Training Exercise Processor Practice Training Processing Tools Other players & tools Cooperative Support Training I-MAESTRO Client Tool Music Editing & Scorefollowing Assessment Models & Tools Audio Processor: beat, pitch, etc. Individual student Individual student I-MAESTRO School Server Group of Pupils, Students, Users Authentications, Distributions and Publications Teacher Educational Content, Profiles, history and results Group of Pupils p. 10

11 General Scenario Teacher Student Scores Media Annotations Lessons and Exercises i-maestro Authoring Tools assessments, annotations, recordings etc i-maestro Client Tools National Institution Personal Syllabus Course Lesson Exercise Main Stages i-maestro Client Tools assignments i-maestro Client Tools Group of Pupils, Students, Users Authoring (lesson/exercise creation, ) Teaching (lesson/exercise assignment, assessment, ) Learning (lesson/exercise usage, cooperative activities, selfassessment, ) p. 11

12 Playing, Practising and Learning School Server Practise Record and save progress Multimedia Annotation Assessment tools Specific techniques (e.g. bowing ) Networ k Supporting Tools for Practising Metronome; Tuner; Scale generator Scorefollowing Auto accompaniment Creative Interface Multimedia control/playback, e.g. gesture to audio/video, mpeg navigation p. 12

13 Musical Gesture Performance analysis Musical gesture v.s. physical gesture music; body; instrument Better understand music playing enhanced mirror Gestural monitoring and analysis 3D motion acquisition and visualisation Relationships, joint angles, space, flow, Not imitating but understanding E.g. Bowing techniques exercises Interactive Multimedia Interface Multimedia Annotation Gesture-based multimedia control and feedback Trans-domain mapping; Sonification Audio-Visual accompaniment; augmented instruments Edutainment Meeting for technology -enhanced learning projects from IST call 4 w w w.i-maestro.net, w w w.i-maestro.org info@i-maestro.org p. 13

14 Events & Activities of Interests 14 March 2006, Rome, Italy Allegro con bit: Music for Europe and Europe for Music May 2006, Leeds, UK GIMS: 2nd International Symposium on Gesture Interfaces for Multimedia Systems co-locate with FUSE Leeds; supported by COST-287 ConGAS, MUSICNETWORK and I-MAESTRO 8 June 2006, Paris, France I-MAESTRO Workshop co-locate with NIME Dec 2006, Leeds, UK AXMEDIS: 2 nd International Conference on Automated Production of Cross Media Content for Multi-channel Distribution MUSICNETWORK Workshop Many more please see latest list on p. 14

15 I-MAESTRO Contacts: Dr. Kia Ng (i-maestro Project Coordinator) ICSRiM, University of Leeds, Schools of Computing & Music, Leeds LS2 9JT, UK Web: w w w.kcng.org, w w w.icsrim.org.uk Prof. Paolo Nesi (i-maestro Architecture Responsible) DISIT-DSI, University of Florence, Via S. Marta 3, Firenze, Italy nesi@dsi.unifi.it, p.nesi@computer.org Web: w w w.disit.dsi.unifi.it, w w w.dsi.unifi.it/~nesi p. 15

City Research Online. Permanent City Research Online URL: http://openaccess.city.ac.uk/2968/

City Research Online. Permanent City Research Online URL: http://openaccess.city.ac.uk/2968/ Weyde, T., Ng, K., Neubarth, K., Larkin, O., Koerselman, T. & Ong, B. (2007). A Systemic Approach to Music Performance Learning with Multimodal Technology. Paper presented at the Support E- Learning Conference,