Broadening Educational Game Design using the World Wide Web

Transcription

1 Broadening Educational Game Design using the World Wide Web Doctoral Dissertation submitted to the Faculty of Informatics of the Università della Svizzera Italiana in partial fulfillment of the requirements for the degree of Doctor of Philosophy presented by Navid Ahmadi under the supervision of Prof. Mehdi Jazayeri December 2012

2

3 Dissertation Committee Prof. Fabio Crestani Prof. Marc Langheinrich Prof. Alexander Repenning Prof. Mary Beth Rosson Università della Svizzera Italiana, Switzerland Università della Svizzera Italiana, Switzerland University of Colorado at Boulder, USA Pennsylvania State University, USA Dissertation accepted on 20 December 2012 Prof. Mehdi Jazayeri Research Advisor Università della Svizzera Italiana, Switzerland Prof. Antonio Carzaniga PhD Program Director i

4 I certify that except where due acknowledgement has been given, the work presented in this thesis is that of the author alone; the work has not been submitted previously, in whole or in part, to qualify for any other academic award; and the content of the thesis is the result of work which has been carried out since the official commencement date of the approved research program. Navid Ahmadi Lugano, 20 December 2012 ii

5 iii To my parents...

6 iv

7 Abstract Educational game design has been increasingly used for teaching computational thinking skills from kindergarten to college. Learners deal with concepts such as objects, instances, behavior, and interaction and and gain programming skills while they are engaged in creating a video game. We identify two limitations in current educational game design environments. First, the learning activities are tied to the presence of a teacher who helps the learner to solve problems that require computational thinking. These activities often take place in an educational setting such as a classroom or after-school computer club. The learning opportunities are constrained to face-to-face communication in a physical location. Therefore, those with no access to an educational setting miss the learning opportunities. Second, Computer Science courses have been perceived as being hard, boring, and asocial among learners. Educational game design environments have employed end-user development techniques to cope with the hardness and put these techniques into the context of creating a game to cope with boredom. However, the asocial face of computing has been addressed far less. The World Wide Web has become a dominant platform for computer-supported cooperative work and social media. Cyberlearning platforms such as Massive Open Online Courses are increasingly employing the Web to deliver education to hundreds of thousands of learners across the world. These platforms rely on the participative culture of the Web, user-created content, social networking applications, and other social media to achieve their educational goals. However, game design environments have not leveraged such rich media to go beyond the classroom environment. The Web can be employed as a cyberlearning platform for educational game design. Such a platform broadens the scope of game design practices, allowing participation to those without access to a physical educational setting. Learners rely on peers and communities to gain computational thinking skills. Ultimately, such a platform can change the asocial face of computing. In this thesis, I explore the feasibility of employing the Web as a cyberlearning platform for educational game design, in two steps: 1) by taking the game design activities to the Web. I have created an open-web educational game design environment and demonstrated that the Web is a suitable hosting platform for game design activities including creating game objects, programming them, and designing the game scene, v

8 vi all with a visual interface. 2) by transferring the educational practices from the classroom to the Web. I have developed a cooperative Website around the game design environment, which allows users to share and explore games and create a community around their game design activities. My experiments with a group of novices validates their success in using the cyberlearning platform to gain computational thinking skills during the game design activities, while relying on the cooperative features of the Website to gain knowledge and learn computational concepts.

9 Acknowledgements This research has been a long journey for me. It is hard to acknowledge everybody who inspired, affected, or helped me in this work. I attempt my best and apologize for those whom I omit. Foremost, I would like to express my highest gratitude to my advisor, Mehdi Jazayeri, for encouraging and supporting me throughout this work. He constantly taught me the more objective way of thinking, and patiently let me find my own way. It has been a privilege for me to be your student. I extend my sincere gratitude to Alexander Repenning, who introduced me to the end-user programming and its educational values. His great care about computer science education in schools has highly affected my research. I am thankful to Nadia Repenning, who made me feel home during my stay in Boulder, CO. I am grateful to the members of Scalable Game Design project, in particular, Kyu Han Koh, Vicki Bennett, Burke Taft, and Sandra Wilder for providing me with opportunities to evaluate my system with novices. I am sincerely grateful to Gerhard Fischer for hosting me in the Center for Life-Long Learning and Design (L 3 D) during my visit to the University of Colorado at Boulder. I learned so much about human aspects of computing throughout many stimulating discussions and inspiring meeting in L 3 D. I would like to thank the members of L 3 D, in particular, Hal Eden and Holger Dick, for their friendly support. I am thankful to Monica Landoni, who helped me evaluate AgentWeb with the school teachers and students in Ticino, Switzerland. Without her great care and support, this would not have been possible. I would like to thank my office-mates, formerly Francesco Lelli and Sasa Nesic, and lately Domenico Bianculli, who inspired me by their commitment to research and brought fun to our office in Lugano. I would like to thank Julia Eberle, Zhu Li, and Ivan Vaghi for several stimulating discussions that we had on software design, information technology, and education during my stay in Boulder. I would like to thank my parents, Manouchehr and Ozra, for their never-ending love, encouragement, and support. I wish the words were not so limited to tell you how proud I am to be your son. Finally, I wish to thank my beloved girlfriend, Sadaf, for her love, care, and patience, from thousands of miles away. Life is more colorful when I think about you. vii

10 viii

11 Contents Contents List of Figures List of Tables ix xiii xv 1 Introduction Challenges in Educational Game Design Cyberlearning as a Solution Framework World-Wide Web as a Cyberlearning Platform Thesis Statement Research Method Thesis Structure Related Work End-user Development End-User Programming Collaboration practices among end users Collaborative and Social Software Engineering Educational Game Design Game-Based Learning Educational Game Design Environments State-of-the-art of support for collaboration in GDEs Computational Thinking Computational Thinking Patterns World-Wide Web Participative Web Web 2.0 for Education Web as a runtime platform Programmable Web and Mashups RIA Platforms vs. Open Web for Games Conclusions ix

12 x Contents 3 Building an Open-Web Game Design Environment System Requirements Choice of game design model and support for end users Choice of collaboration platform Choice of Web technology Challenges Feasibility Studies Ristretto Mobile: an AgentSheets to JavaScript Compiler Web-based Visual Programming of Physical Devices User Interface Architecture Visual Programming Language Language Specification Client-side Execution Performance Issues Code Compilation Differential Scene Rendering Performance Evaluation Usability Testing of the Game Design Environment Conclusion Transition to the Web as a Cyberlearning Platform A process to support design and computational thinking Identified Computational and Design Thinking Activities Evaluation Results Comments from Teachers: Towards Smarter GDEs Examples, Tutorials Computational Thinking Issues Design Thinking Issues Software Engineering Process Conclusions Building a Cooperative Game Design WebSite Issues with existing models of cooperation A Vision for Unobtrusive Cooperation Implementation Content Development Video Tutorials Forum Showcase Usability Evaluation of the Website

13 xi Contents 5.6 Conclusion Experimenting with Cooperation Research Questions Research Method Overview Data Collection Methods Computer Logs Procedure Sample Pre Test Game Design Session Post test Results Task completion and quiz assessment Cooperation Details of game design activities Conclusions Conclusions Research Contributions Open Issues Future Directions A AgentWeb s Visual Programming Language Specification 99 B Usability Evaluation Form 103 B.1 Intellectual task B.2 User Interface Task B.3 Computational Thinking Task B.3.1 Matching with the sheet B.3.2 Programming B.4 Computation Thinking Pattern Matching Sheet C Programming Tutorial Manuscript 109 C.1 List of video tutorials C.2 Introducing AgentWeb C.3 Creating a simple game with AgentWeb C.4 Using the Depiction Editor C.5 Drawing the game scene C.6 Programming D Evaluation Forms 117

14 xii Contents

15 Figures 2.1 Research Space Cost of learning versus Scope of EUD tools, adapted from [39] AgentSheets Game Design Environment Stagecast Creator Game Design Environment Squeak Etoys Game Design Environment Alice 3D Game Design Environment Game Maker Game Design Environment Scratch Game Design Environment Greenfoot Game Design Environment Kodu Game Design Environment AgentCubes Game Design Environment Ristretto Mobile compiles AgentSheets applications to HTML5 to be executed inside the Web browser A first-generation iphone running a Sokoban game inside Mobile Safari. The game was created by AgentSheets and compiled to HTML5 by Ristretto Mobile JavaScript-based visual rule-based programming language was created to rapidly compose the behavior of Lego MindStorms NXT The AgentWeb IDE, used by novice programmers to develop open-web games inside the browser The overview of AgentWeb architecture: life-cycle of agents, user interaction with the system, and flow of operations during the execution process are shown The control flow of a method in an agent program Main conditions and actions of AgentWeb Runtime system components and their relationship xiii

16 xiv Figures 3.9 On the left side (a,b,c,d), the display rate is measured from execution of games in a number of browsers. Each browser has been benchmarked with four different configurations. Configurations are ordered from the slowest with no optimization (N) on the left to the fastest with code compilation and differential scene rendering on the right (CD). Evaluations with the result of less than 3 frames per second are not visible; On the right side (e,f), the performance of the same browsers are measured using two browser benchmarking suites: e) In SunSpider, the faster browser has the lower benchmarking time. f) In V8, the faster browser has the higher score A sample usage pattern automatically recorded from end user s interaction with Web-based AgentSheets. X axis is the passage of time in minutes. Y axis is the section in which user has spent the time. The time periods of less than 10 seconds are considered as noise and removed Evaluated game design process combines the computational thinking tasks with tool usage tasks Mean of teachers rating of research questions based on cognitive jogthrough s scaling Mode of participants rating based on the likert scale The participant receives the description of tasks from inside the game design environment. The logging system records the flow of completing the tasks and the time spent on each task B.1 Computational Thinking Patterns, Page B.2 Computational Thinking Patterns, Page

17 Tables 3.1 Challenges of creating an educational game design environment using open-web technologies and corresponding proposed solutions Evaluated Games and Their Properties Comparison of Game Design 1.0 and Game Design 2.0 styles Results of the experiment, divided by the tasks A.1 Visual Data Types A.2 List of Conditions A.3 List of Actions C.1 List of video tutorials available on AgentWeb Website C.2 List of video tutorials of conditions and actions that are embedded into AgentWeb s programming environment xv

18 xvi Tables

19 Chapter 1 Introduction Computer science is a growing field in both academia and industry. With the ubiquity of digitized information around us, computational thinking is a must-have skill in the 21 st century [147]. The growth of number of end-users who are engaged in programming activities proves the increasing need to computational solutions [119]. However, gaining computational thinking skills is a challenge to novices. To learn computer science concepts, often students are engaged in programming activities which let them gradually gain the expected skills. Papert and others created Logo, the first educational programming language to support teaching programming and CS concepts in 1967 [94]. Since then, many programming languages and environments have been created and used in schools and universities. These environments follow constructivism [14] and constructionism [58] learning theories and practices and they have been influential on how programming languages are taught. Since early 90s, educational programming environments started to form the programming practices around game design activities, i.e., becoming an Educational Game Design Environment. In this thesis I use the acronym GDE to refer to these environments. In such an environment, the learner plays the role of a game designer/programmer which involves designing and programming graphical objects that interact with each other and with the player in a game-like scene. Educational Game Design may point to two different yet related definitions: 1. Teachers use educational games during teaching different courses. One field of research and practice focuses on how to design these educational games to best convey the teaching material while keeping the learner engaged [85]. 2. In the course of teaching, in particular in Science, Technology, Engineering, and Math (STEM) classes, the teacher engages her students in creating a video game or scientific simulation during which they learn the expected teaching material 1. 1 In several cases, students are engaged in creating interactive artifacts such as animations and stories, rather than a full game. For the sake of simplicity for the reader, I consider all of the interactive artifacts as games. 1

20 2 The first definition is a relevant but secondary topic to this dissertation. The focus of this dissertation is the second definition. I shall use the term Educational Game Design hereafter and throughout this dissertation to point to the second definition. The usual audience of educational game design environments are users with no or little programming skills who are known as novice programmers [61], or in short novices, who also are known as end users if they are not interested in programming per se [87]. In this thesis, I will use these terms interchangeably. Educational games and simulations are being increasingly adopted in public schools [54]. Students explore computer science and software engineering [25][121] concepts such as objects, agents, instances, behavior, abstraction, decomposition, and interaction through creating game characters, programming them, enacting and observing their behavior in the game scene. AgentSheets [101], Alice [27], Squeak Etoys [59], GameMaker [93], and Scratch [112] are examples of such educational game design environments. Kelleher and Pausch provide a comprehensive survey on these languages, their educational purposes, and their programming paradigm [61]. In an emerging view on the general benefits of computing knowledge, whether the students eventually pursue computer science or any other discipline, the term computational thinking has gained much attention [147]. Recently, game design courses with their focus on computational thinking have been designed, taught and reported. Accordingly, educational game design environments are used to help students gain computational thinking skills. The wider impact of gaining computational thinking skills, in addition to just learning how to program, have only recently been recognized. Educational game design environments have been around for more than two decades. As games have become more and more popular in education, they have been often used in STEM education [54],[128] [103],[93]. They have matured enough to let us know their characteristics: GDEs use a set of end-user programming tools and techniques to support novices. These techniques include visual languages, domain specific languages, and programming by example. Existing GDEs are solitary desktop-based applications. They have no or limited support for sharing and collaboration. The extent of collaboration has not gone beyond a supplementary Website which is used as a forum for communication and for sharing the games. Game design activities mostly take place in classrooms and after-school clubs, i.e., a physical location in the presence of a teacher. Most of the existing GDEs are available free, or at least have a free trial version, on Windows and Mac OS operating systems.

21 3 1.1 Challenges in Educational Game Design 1.1 Challenges in Educational Game Design There are a number of challenges regarding the scope and extent of use of existing game design environments in education. Here I review some of the well-known challenges that are relevant to this theis. With respect to the use of game design environments in education, we can divide the users into two groups: those with access to a teacher, and those with no access to a teacher. For those who have a teacher, the majority of teaching takes place in the context of a school course or after-school computer club. Even when GDEs are used in the schools, the outcomes are not clear. A comprehensive study on the computer science education across the United States [146] reveals: This study finds that current federal, state, and local government policies underpinning the K 12 education system are deeply confused, conflicted, or inadequate to teach engaging computer science as an academic subject. Quality instruction always depends on knowledgeable and wellprepared teachers, on instructional materials that are engaging and carefully developed to enable student learning, and adequate resources and infrastructure to support teachers and student learning. These goals must be supported by a policy framework that sustains teacher development; certification and continuing education; appropriate curriculum development; and student access and interest. When it comes to computer science education, this framework is failing. Lack of an assessment framework contributes to the failure of building a standard and promising curriculum [146] [141]: Assessments for computer science education are virtually non-existent." Furthermore, game design practices are tightly bound to the presence of a teacher, which mainly takes place in school and after-school settings in which a teacher is present. Despite free availability of educational game design software for download, those with no access to an educational context are hardly interested in or capable of challenging with computational abstractions and programming concepts on their own. Another challenge is the lack of leveraging social interactions among the learners. Lack of support for collaboration among learners in most of the educational contexts has undermined computer science education, whether it has been due to the structure of the course or the lack of tools for collaboration. For instance, computing fields are streotyped as being asocial, causing low participation of women in the field [1] [113]. In contrast to the misperception of computer science as an asocial area, it has been shown in several cases that enabling collaboration among students of an introductory computer science course has led to better educational outcomes. For instance, pair programming has improved learning outcomes in teaching computer science [144]. In another example, employing peer learning has led to better results among students

22 4 1.2 Cyberlearning as a Solution Framework [145]. Cooperative work among novices is a well-studied phenomenon. End users often seek help from more skilled peers in the workplace [87]. When it comes to using GDEs, support for communication and collaboration among the learners is missing. Existing game design environments are solitary desktop-based applications. They have no or very limited support for sharing and collaboration. Kelleher laments the lack of social support and considers it as the sociological barrier to programming among novices [61]. Even worse, with the growth of social media in the past decade, game design activities have not employed socio-technical systems for integrating into the learner s lifestyle, being left as an activity to be done, if ever, in the school. These challenges raise the following questions: Can we broaden computer-science education beyond the classroom, for instance through a Massive Open Online Course (MOOCs) [71] [65], to provide equal educational opportunity to those with no access to a formal educational setting or an appointed teacher? Can we leverage socio-technical systems in computer science education to change the asocial face of computing and integrate the game design practices into the social lifestyle of students? This thesis reflects my efforts to answer these questions. 1.2 Cyberlearning as a Solution Framework Cyberlearning is a term which refers to educational practices that leverage networked platforms. The opportunities provided by networked platforms are studied and suggested for learning [19]. I have explored the cyberlearning opportunities to address the aforementioned challenges. Relying on networked technologies opens a whole new dimension of possibilities for learning: Using a virtual learning environment, we can transcend some of the limitations associated with time and space. Learning can take place anywhere/anytime. In a networked environment, the lack of a formal teacher can be compensated by building a community of practitioners which help each other to gain expertise. Moreover, teachers can use the environment as a supporting tool in their classroom, or extend their classroom beyond a physical location in their school. In a networked environment, there will be opportunities to integrate educational game design into students and young people s daily use of social technology, enabling them to seamlessly work on their projects at home and school. Cooperative and social game design scenarios can change the asocial face of computer science, leading to an increase in participation of women in computer science [130].

23 5 1.3 World-Wide Web as a Cyberlearning Platform To narrow down my research, I raise two fundamental questions, to be answered in this thesis: How can a networked platform realize these goals? Would it be possible to replicate computer science education in a networked learning environment with no formal teacher? 1.3 World-Wide Web as a Cyberlearning Platform The Web has brought us an information universe which has changed how people live and work [16]. During the last decade, the Web has evolved from an information platform to a social platform for sharing, collaboration, participation, and education [136]. Above all the impact that the Web has made in many aspects of our lives, it has remained open and non-proprietary. The Web has evolved how software is developed and delivered to end users [92]. Moreover, the Web has become the main gateway to access the software as a service [133]. I believe that these benefits have the potential to address all the challenges that were discussed earlier regarding computer science education The Web is accessible from all devices and platforms for end users. A Web-based platform can reach its audience globally. At the moment, there are examples of Web-based educational platforms that have succeeded to reach hundreds of thousands of students per course. In the software-as-a-service paradigm, users enjoy software updates as soon as they are available. When game design takes place in the cloud", monitoring and collecting game design patterns is greatly facilitated. Mining such data can lead to developing comprehensive assessment frameworks. To the best of my knowledge, to date no information on user interaction with an education game design environment has been collected. Assuming that an assessment tool is developed, it can quickly reach the learners, track learners progress and provide immediate feedback on their progress. With the success of Websites that allow virtually any user to create content and share it globally with everyone on the Internet, the concept of Web 2.0 [91] and its participative culture [136] was formed. Soon, the potential of the participative Web applications to bring education to the masses was recognized [17]. The advent of participative Web has engendered social learning environments [8] and E-learning 2.0 [19]. By taking educational game design to the Web, we can leverage the participative and social culture of the Web to:

24 6 1.4 Thesis Statement Shift the game design towards a community-based practice which relies on cooperation, collaboration, and peer learning. Integrate the game design activities into the users social life style through online social media and social networking applications The intention of this thesis is to leverage the opportunities provided by the Web, its participative culture, and its service-oriented software development and delivery to the benefits of educational game design. However, no great benefit comes without challenges. The original Web was not intended to be used for building game and media-rich applications. Therefore, third-party proprietary technologies such as Flash [9] was used to deliver rich and interactive media to the Web. For educational purposes, it has been my intention to avoid any proprietary software. I rely on open-web technologies that are standardized by W3C consortium to deliver a rich-media game design platform to the Web. As software, an educational game design environment consists of components that make it challenging to build using open-web technologies, going beyond today s practices and patterns for building open-web applications. Such components include a visual domain-specific programming language and its compiler, a graphical scene authoring tool, a game engine and its graphic rendering engine, and putting them all together as an integrated development environment (IDE) suitable for novice programmers. A major part of this research has involved exploring how to leverage open-web technologies to build a media-rich environment beyond existing Web application practices. 1.4 Thesis Statement In this thesis, I verify and validate the following statements: The open Web is a suitable hosting platform for creating and executing educational games in a visual interface with the standard end-user development techniques that are used to lower the engagement barriers for novices. We can replicate game design practices right on the Web. We can replicate the classroom educational practices for delivering computational thinking skills in a Web-based game design platform. These two assertions establish the foundations needed to scale up computer science education beyond the classroom environment. 1.5 Research Method To verify the assertions, I have developed a fully Web-based educational game design environment, called AgentWeb, that provides instant access to the design and implementation of the games, enabling students to share and cooperate on exploring the