Interactions et collaboration dans les Learning Games immersifs David.Panzoli@univ-jfc.fr
Parcours scientifique Thèse en informatique à l IRIT (Université de Toulouse) Environnements de réalité virtuelle, intelligence artificielle Post-doc au Serious Games Institute (Coventry University) Serious Games immersifs Post-doc au LIFL (Université de Lille) Serious Games, systèmes multi-agents Post-doc SGRL (CUFR Champollion) Serious Games, intelligence artificielle Roma Nova (SGI) Format-Store (LIFL) 3D VOR (JFC, IRIT) 2
Préambule : le GIS SGRN 3
Préambule : le GIS SGRN 4
Activités de recherche Constat : les jeux immersifs non métaphoriques sont privilégiés pour la formation professionnelle Reconstitution virtuelle immersive de l activité professionnelle, Travail sur les compétences interpersonnelles, Assimilation et réutilisation des acquis. 5
Activités de recherche Verrous Méthodologie de conception participative* De nombreux acteurs sont impliqués dans le cycle de conception d un LG immersif, encore plus que pour une simulation. Collaboration dans un environnement 3D Verrous techniques : cohérence de l environnement partagé, comportements des personnages non-joueurs Représentation naturelle de l action collaborative dans un contexte de privation de communication non verbale. Représentation (et traçabilité) de la communication Importance cruciale de la communication dans la formation et donc dans le jeu (débriefing) 6
Context 3D Virtual Operating Room (3D VOR) is a Learning Game set in an OR. Targets various defects caused by pre/per/post-operative miscommunication between the healthcare professionals Features Immersive Set in a 3D environment where each player re-enacts their job Collaborative Multiple players have too cooperate Real-time Dynamic environment, actions and decisions have immediate consequences Learning Game Educational scenarios are designed with respect to specific learning objectives 7
3D VOR prototype (clip) 8
3D VOR Academic prototype (clip) 9
3D VOR: A collaborative work Researchers from various disciplines and professionals are collaborating in 3D VOR 3D modellers Immersion, design of the environment Artificial intelligence researchers Collaborative game entails non playing characters Game designers 3D real time game engine Knowledge managers, training professionals, domain experts Scenarios, educational aspects, content 10
As a technical coordinator, how do I make this work? 11
Related projects Existing games in medical training Are set in immersive and dynamic environment Enable players to collaborate and communicate efficiently 3DiTeams Virtual Heroes/Duke University, 2011 Imperial College London s Virtual Hospital, 2009 Indiana Uni. Medical School Virtual Clinic, 2008 Clinispace Innovation in Learning, inc.,, 2011 12
Related projects Existing products apt for training, yet: Controlling non playing characters is arduous Scripting scenarios is either limited or out of reach of domain experts Proposition of a methodology that ticks all the boxes 13
Methodology Engineer a semantic environment Meaningful features about the environment are extracted from the scenarios modelled by the domain experts. and make it usable to the characters within the game The same features ground the actions used by the players and the NPCs to advance the scenario 14
Activity and scenarios: the BPMN diagrams The Business Process Modelling Notation (BPMN) allows for describing the human activity Sequences of actions for multiple actors Collaborative tasks or exclusive paths (choices) 15
Actions and changing states Every action is specified in terms of changes in the environment A change occur in the value of an attribute of an object. Only boolean attributes are allowed (design choice) An action can be undertaken by one or several roles/actors Preconditions indicate under which condition the action is available Post-conditions detail the effects of the action 16
The semantic environment The semantic environment Is composed by all the objects playing a role in the scenario, Is built from browsing the actions Can be represented as a vector of attributes The semantic environment can be organised using UML class diagram notation Superclasses and interfaces for factorizing 17
Interactive objects How to link an object s description from the semantic world with a 3D object? Semantic Env. Game Engine Patient bleeding:{true/false*} monitored:{true/false*} 18
Actions and affordances How are the actions made accessible to the players in the 3D environment? The answer is called affordances Action 1 Action 2 Action 3 Action 4 Action 5 Action 6 Action 7 Action 8 Action 9 (J. J. Gibson, 1977) filter + spread Action 2 Action 3 Action 6 19
Artificial intelligence: early results AI planning techniques are used for controlling every NPC at once Actions are assembled into plans which are evaluated The Monte Carlo Tree Search (MCTS) technique has yielded some interesting results (Sanselone et al. 2014) Tested evaluation criteria: The BPMN diagram of the scenario, as an observed and therefore desirable scenario The game objectives 20
Conclusion The methodology has been successfully applied to the game prototype of 3D VOR The scenario of the prototype was actually modelled by domain experts Transforming the semantic environment into a dynamic and interactive 3D environment proved to be unproblematic as expected The scenario can be run without any players at all. When humans are playing, the NPCs still cope with their unpredictable behaviour. Future work Boolean values for the attributes are limiting. More than one object per class (e.g. one instance) Gamification will allow for an objective-driven AI 21
Thank you for your attention! References S. Sanchez, O. Balet, H. Luga, and Y. Duthen, Autonomous Virtual Actors, in 2nd International Conference on Technologies for Interactive Digital Storytelling and Entertainment, 2004 C. E. Hughes and J. M. Moshell, Shared virtual worlds for education: the ExploreNet experiment, Multimedia Systems, 1997. F. Bellotti, R. Berta, A. De Gloria, and L. Primavera, Supporting authors in the development of task-based learning in serious virtual worlds, British Journal of Educational Technology, 2010. J. Orkin. Symbolic Representation of Game World State: Toward Real-Time Planning in Games. Proceedings of the AAAI Workshop on Challenges in Game AI, 2004 J. Porteous, M. Cavazza, and F. Charles, Applying planning to interactive storytelling: Narrative control using state constraints, ACM Transactions on Intelligent Systems and Technology, 2010. M. Sanselone, S. Sanchez, C. Sanza, D. Panzoli and Y. Duthen. Control of non-playing characters in a medical learning game with Monte Carlo Tree Search. IEEE Conference on Computational Intelligence and Games (CIG), 2014 Contact me : david.panzoli@univ-jf.fr