Controlling a Games Development Project

Controlling a Games Development Project Draft Proposals on Process Peter Warren Information Technology Programmes University of Natal Pietermaritzburg email:peterw@cs.unp.ac.za April 28, 2000 Abstract The Virtual Learning Spaces [1] project is an attempt to use games to teach material to university students. It is a large project following on from previous successful (but much smaller) games projects. Its participants comprise a heterogeneous collection of academics from diverse fields such as Music, History, Graphic Design, Biology and Computer Science and spread across a number of universities in KwaZulu Natal (KZN). It was clear that a more formal software development process would be needed than had been followed in the past. This work as undertaken to propose such a framework as follows:- A formal human computer interaction (HCI) and user centred approach to the development of the Virtual Learning Spaces software. See Figure 1. User studies start at the outset of the project leading to the creation of an imaginary user called a persona whose description can be used to guide the development process [2]. A series of prototypes are created starting with cheap paper based techniques (story boards) leading to a full blown prototype which will eventually become the released product. Usability goals are set and testing is done throughout the product development life cycle starting with the quicker analytical techniques, through just observing single users with the system using a note book, to setting up a usability laboratory, and finally field tests. 1 Introduction The Virtual Learning Spaces (VLS) [1] project is an initiative to use games as educational material for university students. It is a large project following on from previous successful (but much smaller) games projects. Its participants comprise a heterogeneous collection of academics from diverse fields such as Music, History, Biology and Computer Science, and spread across a number of universities in KwaZulu Natal (KZN). None of these people have been 1

involved a software project of this magnitude before. The project has three characteristics which make it risky, namely it is being undertaken by academics who are distributed throughout KZN and who are tackling a software venture much larger than they have attempted before. The characteristics that give rise to the concern are academics, distributed and larger. life cycle. Failure in a project of this nature is virtually assured unless a formal software development process is in place. And given that it is a games project, there is also a need to build in Human Computer Interaction (HCI) concerns throughout the development life cycle. This research was undertaken to propose such a process. The main purpose is to enhance the usability of the final product, and thus contribute to its eventual success. This kind of process is necessary if any meaningful testing (Tasks 4, 5 & 6 of the VLS proposal) is to be undertaken. Testing after the event is less than useful. The proposed approach is based on the work of Newman and Lamming [5]. It is founded upon the notion that quality is not only the result of attention to the product itself, but also to the procedures that generate the product. The major characteristic is the continual inspection of the usability aspects of the software. Evaluation steps were included in the VLS proposal. The nature of the evaluations are perfectly correct but the timing is not. They are essentially summative, that is they evaluate the quality of the actual delivered product. What is needed is an evaluation process that is formative, namely one which will lead to forming the eventual design. However care needs to be exercised. The VLS project is one designed to answer research questions about games in academic setting. A heavy handed evaluation process could well stifle that initiative. The tests must be quick and cheap to conduct, on going throughout the project and acceptable to all. 2 The Software Engineering Model If we are to get control of the process we may to look towards Software Engineering (SE) to provide the methods. Software starts its life with some vague idea about a situation in the world in need of remedial action (the situation of concern) and hopefully grows up into some form that can address that situation (the final product). The methodology is discussed in [5]. Like all growth, unless the process is controlled in some way, the final form is unpredictable, and certainly not what is desired. The classical SE approach is to develop a detailed specification at the outset so the final form is known exactly. The problem then becomes how to construct the software to meet that specification. The rationale is that it is much cheaper (by up to 100 times) to fix an error at the start of the process than at the end. The fallacy is that no such detailed specification is as yet possible with software and especially games. Many of the theories of software development attempt to resolve the conflict between the inevitable failure of a product to emerge when there is no process involved, to the wrong product emerging when too rigid a phased approach is adopted. However all agree that some middle way is needed to ensure that the project moves through a series of phases, but that each phase has a review process with known methods of evaluation which check that the product is on track, and which help form the eventual solution. These evaluations are part of the design process and not simply a policeman role. Software development can be likened to a feedback system. Too little feedback and the 2

process is not controlled. Too much feedback, on the other hand, and the process is unstable. Furthermore the feedback must nudge the system in the right direction. Lastly the feedback must not rob the system of too much power than could be used to move forward. The proposed approach is designed to tread the path between these extremes. Fortunately it is a fairly broad path. The proposal is an approach that is both phased and iterative. This is very much the middle position. It is based upon the work of Newman and Lamming [5]. It is compatible with the existing project plan, it ensures evaluation throughout the development cycle, and it is user focused. 3 Proposed development cycle 3.1 General Approach Figure 1 illustrates the general approach, which is compatible with the VLS proposal, but with two important additions. Firstly, we have added formal statements, using an existing methodology, which will allow the reviews to be undertaken. Secondly, we have added user studies. User studies and a formal process of review are particularly important to ensure a usable product. 3.2 Situation of concern The first step to ensure that the correct system is being constructed, is to articulate the situation of concern [5]. This describes the current unsatisfactory state of the world that is to be addressed. Without this statement it is impossible to determine if the project has been successful, and it needs to be written in sufficient detail to ensure that such a determination can be made. On the other hand, it must not describe the course of action that will actually resolve it. Whereas the situation of concern does not normally address technological issues, it can do. In this project, exploiting internet technology is an opportunity to be grasped, and that it is not currently being done, is indeed cause for concern. Articulating the situation of concern is one of the ways we can get control of the process as it provides a measure of the success of the project and a way to reason about later phases. Based on the VLS proposal, the following is formulated:- Situation of concern. South African students, in common with students all over the world, need a better understanding of their own artistic, cultural, scientific and technological environment. In particular they need to be able to exploit the revolution that has taken place in the communication and information technologies. There is also a need to foster an understanding of their own history, their place in the global society, and the functioning of democratic societies. A possible solution to this problem would be the development of computer based education but the traditional approach to this has proved a failure. 3

Figure 1: Proposed development process. 3.3 Problem statement The problem statement is the first step in addressing the situation of concern. In interactive computer systems it consists of four elements:- Who. Describes the users who will be exploiting the system. In our case the system will be used by South African university students in all faculties. Elsewhere in VLS proposal reference is made to school children. This issue needs to be clarified. It is suggested that a persona is created [2]. A persona is an imaginary but representative user who flows out of initial user studies. It has been claimed that this approach allows developers to visualize the user. It is part of the research objectives to observe this process. What. Describes the human activity that the system will be supporting. In this case the 4

activity is a learning process and the particular topics are specified in the VLS proposal. It also specifies that this learning process will take place through games. However from an HCI point of view, the learning objectives are too vague to allow adequate evaluation of the prototypes and this needs to be done before detailed story boards are developed. How. Describes the degree of support that the user will get in his activity. We will need to specify various usability goals including the time it takes to learn the system, the ease with which the system is remembered, the number of errors and the ability to recover from errors. With. Describes the technology to be used. As this is largely beyond the usability issue, and seems under control, it will not form part of this document. A number of action steps follow from the problem statement. 1. There needs to be a formal process to validate that the problem statement does indeed address the situation of concern. This will ensure that a deeper understanding is reached of both the problem to be solved, and the way it is to be solved. 2. A user study needs to be initiated to find out the characteristics of the user population. It has been recommended that a Persona is created which describes the characteristics of a fictitious user of the system. The developers of the software need to go out an talk to some real people. 3. What describes in classical HCI terms the human activity to be supported. This process is described in Task 2 : Story Development. However, from an HCI point of view, we would like to see the presentation of a simple, paper based prototype of these stories. These are tested against the Persona, the Situation of Concern and the Problem Statement. 3.4 Requirement document Knowing what system is to be built is important for all software projects. When the group is as amorphous as this one, it is absolutely crucial that a detailed requirements document is developed. It corresponds to story development in the VLS proposal, although some aspects belong with the paper prototypes described above. Without a requirements document experience shows that the programmers take control of the project, and that the final product is a loose collection of features that do not operate in unison or achieve the user goals [2]. The requirements document describes in detail the screens and interaction techniques to be used. It is the bridge between the lofty ideals of the problem statement and the reality of programming. The requirements document needs to be both valid and verifiable. Validity is the term used to describe that the correct product is being described. This is where the situation of concern and problem statement are so useful. Verifiability means that the statements in the requirements document can be checked. For instance it is not enough to say that a system must be easy to use without more detail on what that means. One can foresee a number of dangers involved which need to be avoided. 5

Uncontrolled development. Whereas the VLS proposal acknowledges the need for iterative development it makes no mention of the evaluation mechanisms. Iteration achieves nothing unless there is an appropriate feedback loop. Prototype becomes the product. Prototypes are built with all kinds of considerations that are not part of the final product. We should not allow them to become the final product. We should build prototypes as we would design experiments - that is to answer specific questions about the game we are developing and not as a pale shadow of things to come. They need not, and should not be fully functional programs. For this reason it is neither feasible or desirable to have user testing of these prototypes to any large degree. The process consumes too much resource. There is a danger that the development of the requirement s document and prototypes takes up too much of the resources of the project. It is therefore imperative that the purpose of the prototypes is identified and that cheap analytical techniques are used to assess them. Policeman mentality. Formal evaluations are part of an iterative design process and they need to be seen by participants as such. The purpose is to speed the convergence to a worthwhile outcome. There is a grave danger that they are seen as a hurdle to be overcome and this must be avoided if they are to be effective. The milestones (Table 3 of the VLS proposal) makes no explicit reference to a requirements document. We are suggesting that it be amended so that it does so. The details of the requirements document need to be specified and each requirement needs to be formally evaluated before it is accepted. Without these documents its will be very difficult to make any assessment of the products. These considerations lead to the proposal that the requirements document is developed in conjunction with a throw away prototype, probably paper based. The evaluation techniques to be used at this stage need to be Usability Inspection Methods as opposed to Usability Testing Techniques. The two standard techniques are Heuristic Evaluation which is an inspection against a set of standards [6] and Cognitive Walkthrough which is based on the theories of exploratory learning[5]. These techniques have been applied to traditional software where the objective is to make it as easy as possible. The application to games is an interesting challenge as there the objective is not to make it as easy as possible! 3.5 Specifications The specification section details the specific product to be built. Given the number of parallel processes to ensure that the right technology is used, and the innovative nature of the project, a radical prototyping approach can be used [3]. In this technique the prototype becomes the product. The only requirement is that there is a meaningful process of evaluation. Hence we would suggest that the protocols for this are drawn up. At this stage actual user testing needs to start and a usability laboratory needs to be built. It is tempting to think that a state of the art usability laboratory needs to be constructed. Since the articles by Gould and his collaborators (eg [4]) this is not the main issue; accessibility 6

to the testing equipment is. We are therefore proposing a simple portable system that will we available to passers-by but where we can take and annotate video and audio recordings [8]. 4 Expected outcomes. The Newman and Lamming [5] process is a traditional software engineering waterfall model with two important differences and enhancements. Firstly the process is user centred with user characteristics playing an important defining role. Secondly the evaluations are done as cheaply as possible, and at every stage in the development process. It will be part of the research to document whether a process of this nature is a viable way of producing games software a distributed project of this nature. The methodology will be action research where the investigator is a full participant rather than a disinterested bystander. The outcome is going to be descriptive. There is no research hypothesis to be validated. What we want is a description of what happened along with an interpretation of the outcomes. 5 Caveats and Conclusions It is one thing to propose a methodology, it is another to get the process into place. Early indications are not encouraging. Games software resembles in many ways the problems of virtual reality. There is a need for high performance in both disciplines and there is the same fixation with the widgets [7] that characterized HCI research of a number of years ago. There are also not quite the same concepts about what is to be achieved as there is in traditional HCI. Listening to musicians talk about dynamic composition, where there is little abstraction of the concepts they wish to convey, creates concerns that much of what we believe about software design is problematical in this situation. Form and content are not neatly separated in quite the same way computer scientists think about the world. Similar concepts can be found in [3]. Failure to make the distinction between what and how is going to make abstract and cheap prototypes difficult to build. However, if we have leaned anything from failed software products is that the failure is not just an act of God but comes from the failure of adequate planning. These problems notwithstanding there is a process of evaluation that has been proposed. Another experience of software development, especially in prototyping environments, is that some testing and early testing is needed and that it need not be elaborate or expensive. This project will test these ideas in a games and university setting and has implications for many other less formal software development projects. References [1] Alan Amory. Virtual learning spaces. Technical report, University of Natal, 1999. [2] Alan Cooper. The Inmates Are Running the Asylum. SAMS, 1999. [3] T. Winograd (Ed). Bringing design to software. Addison-Wesley : Reading, Mass., 1996. 7

[4] Gould J.D. How to design usable systems. Handbook of Human Computer Interaction, In emphhandbook of Human Computer Interaction (Helander M., ed):757, 1988. [5] W.M. Newman and M.G. Lamming. Interactive System Design. SAMS, 1995. [6] J. Nielsen and Eds. R Mack. Usability Inspection Methods. John Wiley and Sons, Inc., New York, NY, 1994. [7] Michael Rorke. Designing and implementing a virtual reality interaction framework. Master s thesis, Roodes University, 1999. [8] Jeffrey Rubin. Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests. John Wiley & Sons, 1994. 8