Statechart modelling of interactive gesture-based applications

Transcription

1 Statechart modelling of interactive gesture-based applications Romuald Deshayes Tom Mens Software Engineering Lab Deshayes Romuald UMONS 1 / 20

2 Table des matières 1 Problem Statement 2 Proof-of-concept application 3 Modeling interactive behaviour 4 The application framework 5 Statechart models 6 Conclusion and Future Work Deshayes Romuald UMONS 2 / 20

3 Problem Statement Outline : Problem Statement Problem Statement Deshayes Romuald UMONS 3 / 20

4 Problem Statement Problem Statement Challenge Develop intuitive interactive applications that are easy to maintain Why is it a challenge? Complex behaviour Nondeterministic user actions Same events lead to different actions Contributions Gestural user interface for 3D objects manipulation Modeling executable behaviour using a visual formalism Validation with an application framework Deshayes Romuald UMONS 4 / 20

5 Proof-of-concept application Outline : Proof-of-concept application Proof-of-concept application Deshayes Romuald UMONS 5 / 20

6 Proof-of-concept application Proof-of-concept application 3D visual drawing tool Uses gestures to create and manipulate 3D objects OpenGL graphical library Microsoft Kinect + NITE Deshayes Romuald UMONS 6 / 20

7 Proof-of-concept application Small video Deshayes Romuald UMONS 7 / 20

8 Modeling interactive behaviour Outline : Modeling interactive behaviour Modeling interactive behaviour Deshayes Romuald UMONS 8 / 20

9 Modeling interactive behaviour Modeling interactive behaviour Context Gesture-based interaction Highly reactive event-driven systems Proposed solutions Visual modeling language Statecharts Petri nets Amenable to formal analysis Easier to evolve Reduced complexity Deshayes Romuald UMONS 9 / 20

10 The application framework Outline : The application framework The application framework Deshayes Romuald UMONS 10 / 20

11 The application framework The application framework Framework for developing interactive applications Executable behaviour specified using statecharts Client-server architecture Deshayes Romuald UMONS 11 / 20

12 The application framework The application framework II Features Generic Abstract classes Observer design pattern Communication between statecharts Uses Java SwingStates library Deshayes Romuald UMONS 12 / 20

13 The application framework The application framework III Deshayes Romuald UMONS 13 / 20

14 Statechart models Outline : Statechart models Statechart models Deshayes Romuald UMONS 14 / 20

15 Statechart models Statechart models - Hand Deshayes Romuald UMONS 15 / 20

16 Statechart models Statechart models - Component Deshayes Romuald UMONS 16 / 20

17 Statechart models Statechart models - Scene Deshayes Romuald UMONS 17 / 20

18 Conclusion and Future Work Outline : Conclusion and Future Work Conclusion and Future Work Deshayes Romuald UMONS 18 / 20

19 Conclusion and Future Work Conclusion and Future Work Statecharts Appropriate and scalable formalism for modeling the interactive gesture-based behaviour? Improvements Externalise/decouple statechart represention from framework Compare statecharts with Petri nets Deshayes Romuald UMONS 19 / 20

20 Conclusion and Future Work Future Work II Apply our approach to other HCI applications Other types of natural interfaces Compare/combine with other UIML Controlled experiments, carry out user studies, empirical studies Deshayes Romuald UMONS 20 / 20