Measuring Entertainment and Automatic Generation of Entertaining Games

Transcription

1 Measuring Entertainment and Automatic Generation of Entertaining Games Zahid Halim Department of Computer Science National University of Computer and Emerging Sciences, Islamabad Note: This is a non camera ready copy of thesis

2 Measuring Entertainment and Automatic Generation of Entertaining Games A thesis submitted to Department of Computer Science FAST-NUCES For the Partial Fulfilment of the Requirement of PhD (CS) Degree Zahid Halim FAST-National University of Computer and Emerging Sciences Islamabad Campus November 2010 ii

3 Abstract With the advancement in technology and decrease in prices of electronic items, Personal Computers (PCs) are becoming common. This has resulted in PCs replacing many other electronic gadgets (televisions, play stations, etc.), as people are inclined to use these in their PCs through software. Games are not an exception from this electronic advancement. All this has increased the number of choices in computer games for the users. At the same time the quality of entertainment provided by these games has also decreased due to abundance of games in the market for PCs. On the other hand the task of game development for the developers is becoming tiresome, which requires scripting the game, modeling its contents and other such activities. Still it cannot be known how much the developed game is entertaining for the end users, as entertainment is a subjective term. What might be entertaining for one user may not be entertaining for others. Another issue from the point of view of game developers is the constant need of writing new games, requiring investment both in terms of time and resources. In order to address afore mentioned challenges. First task is to devise some metrics that can quantify the entertainment value of a game. Although creating a single quantitative measure for all genres of games is not trivial, but a separate metrics for each can be devised. Based upon the entertainment metrics some computational intelligence techniques can be used to create new and entertaining games. In this work we create a set of metrics for measuring entertainment in computer games. The genres we address are board based games and video games (predator/prey and platform games). The metrics are based theories of entertainment in computer games, taken from literature. Further we use Evolutionary Algorithm (EA) to generate new and entertaining games using the proposed entertainment metrics as the fitness function. The EA starts with a randomly initialized set of population and using genetic operators (guided by the proposed entertainment metrics) we reach a final set of population that is optimized against entertainment. For the purpose of verifying the entertainment value of the evolved games with that of the human we conduct a human user survey and experiment using the controller learning ability. iii

4 Acknowledgements I am grateful to all those who have provided me their support and guidance throughout my PhD research. This thesis is not a solo effort. Long hours were spent on research and documentation, and I could not have done it alone. I thank Allah for granting me the energy to complete my work. Many people deserve chunks of the credit. First and foremost, I would like to extend my gratitude to my supervisor Dr. A. Rauf Baig, for his able guidance. His constant support was a vital ingredient in this piece of work. Without his unending interest and insightful guidance this work might have remained incomplete. FAST National University of Computer and Emerging Sciences proved to be an excellent platform for conducting research. I would like to thank the cooperative faculty members, the director and our head of department (current and former), Dr. Aftab A. Maroof, Dr. Arshad Ali Shahid and Dr. Anwar M. Mirza for their constant support. I am endlessly grateful of my parents. My family s support gave me strength and a desire to learn. Lastly, I apologize to all individuals and organizations I may have overlooked and unintentionally missed in this acknowledgment. iv

5 Table of Contents Chapter 1: Introduction Motivation Questions and Objective Scientific Relevance List of Publications... Error! Bookmark not defined. 1.5 Thesis Outline Summary... 6 Chapter 2: Computational Intelligence and Games Game AI and Computer Games Types of Computer Games Board Based Games Video Games Predator/prey games Platform games First person shooter games Real time strategy (RTS) game Physically activation games Tools Evolutionary Computation Genetic Algorithm Chromosome Cross-over v

6 Mutation Evolutionary Strategy Artificial Neural Networks (ANN): Neuron ANN Learning Supervised learning Unsupervised learning Reinforcement learning Summary Chapter 3: Literature Survey Board Based Games Video Games Physical Activating Games Entertainment Theories Summary Chapter 4: Entertainment Metrics Board Based Games Duration of the Game Intelligence for Playing the Game Dynamism Exhibited By the Pieces Usability of the Play Area Combined Fitness Function Video Games (Predator/Prey and Platform) Duration of the Game vi

7 4.2.2 Appropriate Level of Challenge Diversity Usability Combined Fitness Function Summary Chapter 5: Board Based Games Search Space General Rules of Chess and Checkers Size of Play Area Types of Pieces, Their Quantities and Their Initial Positions Movement Direction of Each Piece & Step Size Capturing Logic Game Ending Logic & Rules Defining Wins & Draws Conversion to Pieces of Other Type after Reaching Last Row If a Piece is being Captured, Is It Mandatory To Capture It? Turn Passing Allowed? Structure of the Chromosome Software Agents for Playing the Games Random Agent Agent Using Min-Max with Rule Based Evaluation Function Experiments Evaluation of Checkers and Chess Evolution of New Games Analysis of Evolved Games vii

8 5.4 Learnability of Evolved Games User Survey on Entertainment Value of Evolved Games Hypothesis Testing Metrics versus Metrics of Chess Checkers and the Evolved Games Summary Chapter 6: Predator/Prey Games Search Space Chromosome Encoding The Controller Rule based controller Artificial Neural Network (ANN) Based Controller Experimentation and Results Duration of the Game Appropriate Level of Challenge Diversity Usability The Combined Fitness Function Verification of the results Human User Survey Controller Learning Ability Hypothesis Testing Summary Chapter 7: Platform Games Search Space viii

9 7.2 Chromosome Encoding The Controller Experimentation and Results Verification of the results Human User Survey Hypothesis testing Summary Chapter 8: Conclusion Answer to Problem Statements Contributions Limitations Future Work Summary Appendix A: Human User Survey A.1 Board Based Games A.2 Predator/Prey Game Appendix B: Evolved Games B.1 Board Based Games Board Game 1: Board Game 2: Board Game 3: Board Game 4: Board Game 5: Board Game 6: ix

10 Board Game 7: Board Game 8: B.2 Predator/Prey Games Predator/Prey Game 1: Predator/Prey Game 2: Predator/Prey Game 3: Predator/Prey Game 4: Predator/Prey Game 5: Predator/Prey Game 6: Predator/Prey Game 7: Predator/Prey Game 8: Predator/Prey Game 9: Predator/Prey Game 10: References x

11 List of Figures Figure 2.1: Component of a computer game... 9 Figure 2.2: Board configuration for the game of chess Figure 2.3: Board configuration for the game of monopoly Figure 2.4: Typical environment of Pac-Man game Figure 2.5: Typical environment of super Mario game Figure 2.6: Screenshot of Doom Figure 2.7: Screenshot of WarCraft III Figure2.8: Screen shot of playware platform Figure 2.9: A general GA Figure 2.10: A single point crossover scheme Figure 2.11: ES algorithm Figure 2.12: Artificial Neuron Figure 3.1: A child playing the Space Invaders game Figure 4.1: Scaling ranges for raw value for duration of game Figure 5.1: Structure of the chromosome Figure 5.2: Algorithm for the random playing agent Figure 5.3: Algorithm for evaluation of board positions Figure 5.4: Family 1 metrics values Figure 5.5: Family 2 metrics values Figure 5.10: Family 7 metrics values Figure 5.14: ES population change trend Figure 5.15: Rules of game 1 and pieces board positions Figure 5.16: Rules of game 2 and pieces board positions Figure 5.19: Learnability of evolved and random game Figure 5.20: Screenshot of the software used for human user survey Figure 5.21: Statistics of the human user survey Figure 5.22: Statistics collected considering Duration of game metrics for random vs. random, random vs. intelligent and intelligent vs. intelligent controller experiment xi

12 Figure 5.23: Statistics collected considering Intelligence metrics for random vs. random, random vs. intelligent and intelligent vs. intelligent controller experiment Figure 5.24: Statistics collected considering Dynamism metrics for random vs. random, random vs. intelligent and intelligent vs. intelligent controller experiment Figure 5.25: Statistics collected considering Usability metrics for random vs. random, random vs. intelligent and intelligent vs. intelligent controller experiment Figure 6.1: Typical environment of Pac-Man game Figure 6.2: Typical environment of the generic predator/prey game Figure 6.3: Chromosome encoding along with possible values a gene can have Figure 6.4: ANN architecture used to control agent Figure 6.6: The best chromosome evolved by optimizing duration of game. a) Evolved using rule based controller b) Evolved using ANN based controller Figure 6.7: The best chromosome evolved by optimizing appropriate level of challenge. a) Evolved using rule based controller b) Evolved using ANN based controller Figure 6.8: The best chromosome evolved by optimizing diversity. a) Evolved using rule based controller b) Evolved using ANN based controller Figure 6.9: The best chromosome evolved by optimizing usability. a) Evolved using rule based controller b) Evolved using ANN based controller Figure 6.10: The best chromosome evolved by optimizing the combined fitness function. a) Evolved using rule based controller b) Evolved using ANN based controller Figure 6.11: The interface for playing the evolved games for User Survey Figure 6.12: Randomly select game rules for user survey Figure 6.13: Pie chart representation of the human user survey using rule based controller.. 86 Figure 5.14: Pie chart representation of the human user survey using ANN based controller Figure 6.15: Comparison of the rule based and ANN based controller Figure 6.16: Learning ability of a controller Figure 7.1: Typical environment of the game Infinite Mario Bros Figure 7.2: Chromosome encoding for platform games Figure 7.3: Algorithm for rule based agent xii

13 Figure 7.4: Game evolved for duration Figure 7.5: Game evolved for challenge Figure 7.6: Game evolved for diversity Figure 7.7: Game evolved for usability Figure 7.8: Game evolved using final fitness rank Figure 7.9: Random initialized game Figure: A.1 Screenshot of the software for user survey for board games Figure: A.2 Screenshot of the software for user survey for predator/prey games xiii

14 List of Tables Table 5.1: Summary of the search space to evolve new board based games Table 5.2: Duration of Games (data generated by playing100 games) Table 5.3: Intelligence required (data generated by playing100 games) Table 5.4: Dynamism of pieces (data generated by playing100 games) Table 5.5: Usability of Play Area (data generated by playing100 games) Table 5.6: Fitness values of chromosomes in archive Table 5.7: Diversity count of evolved games Table 5.8: Demographics of the subject for user survey Table 5.9: Human user survey results Table 5.10: Answer to the question " Is game in row is more entertaining than game 4(random game)?" Table 5.11: p-values for each pair of game Table 5.12: Duration of Games (data generated by playing100 games) Table 5.13: Intelligence required (data generated by playing100 games) Table 6.1: Demographics of the subject for user survey Table 6.2: Human user survey results of rule based controller Table 6.3: Human user survey results of ANN based controller Table 6.4: Correlation between subject s value of entertainment and the one carried by the game Table 6.5: Correlation and p-values for each pair of game Table 7.1: Results of user survey Table 7.2: Answer to the question " Is game in row is more entertaining than game 4(random game)?" Table 7.3: p-values for each pair of game xiv

15 This page is intentionally left blank

16 Chapter 1 Introduction Humans have always been in search of ways to utilize their free time in activities that are entertaining. This search has led towards computer games. With the passage of time computer games industry has grown rapidly [1, 2] and this means of entertainment has become one of the major sources of relaxation and fun for children and adults [3, 4]. There can be many reasons for the popularity of such games, some of which include the higher amusement capacity of these games due to their graphics, increasing levels of challenge and choice. Still, at the same time, there is a need of ranking these games based on their value of entertainment. Furthermore, this measurement of entertainment combined together with computational intelligence techniques can be used towards automatic generation of games and/or its contents. Having said that, entertainment is of subjective nature which varies from person to person. In addition to entertainment in games being dependent on humans it does also depend upon the particular genre of game. Once a quantitative measure of entertainment is formulized, it would be interesting to create games on the fly using computational intelligence techniques. This chapter describes the motivation behind this thesis and identifies the research questions to be answered in this work along with its scientific relevance. 1.1 Motivation Nowadays computer games have became a major source of entertainment for all age groups, especially children. The reason for computer games being the primary source of entertainment could be many, including these being highly interactive, high resolution graphics, diverse level of choice and challenge. According to a survey in [5] on 1254 subjects (teenagers), only 80 were found playing non electronic games in the last 6 months. The results in [5] show the popularity of computer and video games in young generation. 2

17 Thinking from the point of view of game developers it has always been a challenge to measure the entertainment value of the human player. This is due to the fact that entertainment is very subjective. It also depends upon the genre of the game and contents of the game in addition to the subject (user) playing it. Keeping this fact in mind it would be very convenient for the game developers to develop entertaining games if they could somehow measure entertainment, the way other things like temperature, weight are measured. This would give a quantitative representation of the entertainment a game has as opposed to the subjective one nowadays. Still based upon the measurable entertainment, the responsibility of producing a game like today will be on the shoulders of game developers. Game developers at present have to define the complete game from start to end with each stage and its components and complexities. It would be very convenient that we could also produce the game automatically based upon the measurable entertainment. This would lead to interesting application in the area of computer game development. There has been some initial work started in the areas of measuring the entertainment of computer games and the automatic generation of computer games (Chapter 3 covers the survey in this regard) but still the idea is in its infancy. In this thesis we address the two issues in game development as mentioned earlier: (a) measuring entertainment value of a game and (b) automatic generation of entertaining games. We propose an entertainment metrics to quantitatively measure the entertainment value of the game. In this thesis we address two genres of games and a separate set of entertainment metrics is proposed for each. The genres of games addressed in this thesis include board based games and video games (predator/prey and platform games). We have further shown the utility of the proposed entertainment metrics by generating new and entertaining games through an evolutionary process which uses the proposed entertainment metrics as fitness function, guiding the evolution towards an entertaining game. In order to counter check the entertainment value of the evolved games we have conducted a human user survey to verify that the results correlate with those produced by the system. In contrast to the user survey, the entertainment value of the evolved games is also verified using the Schmidhuber s theory of artificial curiosity [6]. 3

18 1.2 Questions and Objective Based upon the motivation, the questions that need to be addressed, restricting ourselves in the domain of computer games, are as follows: 1. What is entertainment in computer games? 2. What are the factors that contribute towards entertainment? 3. Does entertainment depend upon a genre of game? 4. Are their any existing theories on measuring entertainment? 5. How can we measure entertainment? 6. If we can measure entertainment how can we verify it against human s actual perception of entertainment? 7. Can the proposed measure of entertainment be used in automatic generation of games which are entertaining? 8. If automatic entertaining games are generated, how can we verify that they are actually entertaining for humans or not? Based upon the questions mentioned above the thesis will focus on the following tasks 1. Defining entertainment in games. 2. Based upon different theories of entertainment, develop a quantitative measure of entertainment for each genre of game. 3. Use computational intelligence techniques to generate entertaining games, utilizing the proposed entertainment metrics. 4. Conduct a user survey and hypothesis testing to verify the evolved games against entertainment and to validate the user survey respectively. In order to identify the factors that contribute towards entertaining games we study the theories of entertainment/fun from literature (listed in Chapter 3). Using the identified factors of entertainment we propose a separate set of metrics of entertainment for each genre of game considered. Further we use evolutionary computation to evolve games which are entertaining based upon the proposed entertainment metrics. 4

19 1.3 Scientific Relevance Games have always been a valuable subject of research for social scientists but they may be seen as less worthy an application from a computer scientist s point of view. This impression is false, as games are the driving force behind the development of modern 2d and 3d graphics and animations [5]. In addition to this, games are a perfect test bed for the computational intelligence techniques. Computer games are a perfect challenge for testing and improving the Computational Intelligence (CI) techniques due to the following reasons: Reflection of the real world: Games are mostly simulations of real world scenarios and entities [7]. CI in games can capture the real world behaviours. Test bed for human like behaviour: CI in games involves mimicking human like behaviour for many purposes. Thus, contributing in understanding and developing systems having human like capabilities. Optimal utilization of resources: CI in games involves using techniques to optimally utilize the limited resources in order to develop games that capture the interest of users, free from any degradation. Variety: Games provide a variety of genre to test CI techniques in different scenarios and constraints. The above mentioned characteristics along with others make results achieved by applying CI in games widely applicable. These may be transferred to other domains having fewer constraints. 1.5 Thesis Outline The thesis is organized as follows: Chapter 1 introduces the problem we are going to address. Chapter 2 is dedicated to cover the computational intelligence in computer games 5

20 and different types of computer games. The chapter describes which computational intelligence techniques we will be applying to answer the research questions identified in chapter 1. Chapter 3 lists the previous work done in the context of measuring entertainment and automatic generation of game and/or its contents. We also list the different studies on the concept of entertainment in computer games in Chapter 3. Chapter 4 introduces the entertainment metrics proposed in this work which are based on the theories of entertainment listed in chapter 3. The entertainment metrics proposed in chapter 4 are for the board based and predator/prey genre of games. Chapter 5 lists the methodology of automatically generating entertaining board based games using the proposed entertainment metrics as a fitness function for the evolutionary algorithm. Chapter 6 covers the automatic generation of entertaining predator/prey games. Chapter 7 covers the automatic generation of entertaining platform game levels. Chapter 8 concludes the thesis with possible future directions. 1.6 Summary Inspired by the concept of measuring entertainment and using measured entertainment to produce computer games this thesis is focused on devising entertainment measures for computer games. The genres of games addressed are the board based and video games (predator/prey and platform). For the purpose of devising entertainment metrics different theories on entertainment in computer games have been used as bases. The proposed metrics consists of duration, usability, dynamism and intelligence as individual components for the board based genre of games. For video games entertainment metrics consists of duration, diversity, usability and appropriate level of challenge. These metrics are used by evolutionary algorithms (evolutionary strategy and genetic algorithm) as a fitness function to guide a population, representing game rules, towards more entertaining games. For the purpose of creating new and entertaining games we first define a search space for each of the addressed genre of game. The entertainment value of evolved games is verified by conducting a user survey and a controller learning experiment. The next chapter will cover computer games, its different genres and role/techniques of computational intelligence in games. 6

21 7

22 Chapter 2 Computational Intelligence and Games Games have been part of all cultures. With the advent of computers and advancement in technology computer games have gained popularity. Initially scripts were written for games defining it from start till end, along with hard coded rule set for the entities in it. With the emergence of artificial intelligence, entities in games were replaced by agents working intelligently and autonomous of any hard coded rule set. With the passage of time both, games and computational technologies used in games, evolved and have been improved. In this chapter we define a computer game and its basic components, both from the point of view of game developer and a user. We list the use of computational intelligence in games. It is important to know the different genres of computers games. This helps in understanding the complexity to create a single quantitative measure of entertainment addressing all these genres, major genres of computer games are listed in the chapter. A game may contain entities being controlled by computational intelligence techniques like artificial neural networks and the rules of the games may be represented by a chromosome in a genetic algorithm. The chapter also explains computational intelligence used to accomplish thesis objective. 2.1 Game A game is a set of rules that govern some entities in order to achieve an aim. Huizinga in his work [8, 9] defines game as follows: To play is an activity which proceeds within certain limits of time and space, in a visible order, according to rules freely accepted, and outside the sphere of necessity or material utility. The play-mood is one of joy and enthusiasm, and is sacred or festive in accordance with the occasion. A feeling of exaltation and tension accompanies the action, mirth and relaxation follow. 8

23 A dictionary defines game as "a competitive activity involving skill, chance, or endurance on the part of two or more persons who play according to a set of rules, usually for their own amusement or for that of spectators." [10]. A game has following components: Players, at least one A set of rules and Objective(s) to achieve Computer games are the one that are carried out with the help of a computer program. We can distinguish three roles for a computer program in a game [11]. Co-ordinating the game process Illustrating the situation Participating as a player Model Current State Analyzer Controller View Controller Logic View Controller Synthetic Player Driver Synthetic Rendered Script/rules Input Device Output Device Human Player Figure 2.1: Components of a computer game 9

24 Figure 2.1 shows the different components of a computer game. The model area consists of those software modules that are to coordinate by analysing current state of the game. The view area manages the pectoral role. A main-view provides an interface into the Model. It creates a synthetic view for a synthetic player and creates a rendering view to an output device. The synthetic view may not be suitable for a real player but it is placed in an appropriate data structure that can be later mapped to a rendered view. This helps in faster processing of game current state by the synthetic player (as the processing time on audio/visual effect is saved). Rendering includes visualization, audification and other forms of sensory feedback. The controller part includes the components for the participation of players. A human player s input is received through an input device. The configuration component provides instance data, which is used in generating the initial state for the game. The human player provides his input to the game by perceiving information from the output devices and giving actions to the input devices. There can be multiple players participating the data flow, each with their own output and input devices. A computer game may be distributed among several computers. A synthetic player is a computer controlled actor in the game. It can play the role of an opponent or a non-player character (NPC) which participates limitedly. Theoretically, there should be little or no difference between a human and synthetic player. If they are to operate at same level, both should ideally have the same powers of observation, capabilities and intelligence. 2.2 AI and Computer Games The main aim of AI in computer games is to make them more attractive for the users. This may be done in multiple ways, either by making the game AI challenging enough that 10

25 the human user faces a tough time to beat the AI. On the one hand games may be more attractive by making the NPC's in the game more diverse or interesting [12, 19]. In this regard the first computer program ever written for computer games was by Christopher Strachey for checkers and Dietrich Prinz chess [20, 21]. Arthur Samuel wrote a checkers software in mid 1950's with some capability to beat an amateur [22]. The first video games developed in were implemented on discreet logic strictly based on competition of two players with no AI. Games with single player mode were introduced in late 1970s. The first in this series is Atari games Qwak [23] and Pursuit [24]. Games like Hunt the Wumpus [25] and Star Trek [26] had enemies and their movement was based on stored patterns. The game Pac-Man [27] in 1980 applied these patterns to maze games along with personalities for each enemy. As new game genres emerged in the 1990s, formal AI tools were incorporated. During the last ten years there has been a great stride in game AI mainly because of raw computing power and memory capacities of modern computers. Driving games like Nascar 2002 [28] have computer controlled drivers with their own personalities and driving styles. Strategy games like Warcraft 3 [29] ask the users to develop an army to defeat one or more computer-controlled villages with armies. The game AI is continuously improving, using different techniques such as genetic algorithms, swarm intelligence and other such, with the aim to both make the game more entertaining/attractive for the user and there be no difference between computer and human players. 2.3 Types of Computer Games Computer games can be divided into multiple types based upon different categorization schemes. The most common type of computer game include board based games, video games, real time strategy games, and 11

26 physically activating games. Each of the above mentioned categories may contain further subcategories Board Based Games Board based games are the ones that are played using some fixed size game boards with pieces or cards for the player. There may or may not be dice rolling involved in these games. The most popular board based games include chess, checkers and monopoly. Games like chess and checkers comes under the category of perfect information games. The perfect information games are those in which all the available information is known by all the players during the game and there is no hidden factor to influence the game state. Both chess and checkers are played on a fixed 8x8 board of black and white colour. Figure 2.2 displays the board configuration for the game of chess. Each piece in the game has its own rules for movement and capturing opponent's pieces. Figure 2.2: Board configuration for the game of chess 1 The game of monopoly belongs to the category of imperfect information games. In imperfect information games parts of the information is hidden from some or all of the players. In case of monopoly the rolling of a dice is an addition probabilistic factor that also plays a role of hidden information in the game. Figure 2.3 shows the board configuration of monopoly. Each player is represented by a small metal token that is moved around the edge of the board. Other popular board based games includes the games of Tic Tac Toe, Othello, Go, Risk and Sudoku Puzzles. 1 Image taken from free online version of chess 12

27 2.3.2 Video Games Figure 2.3: Board configuration for the game of monopoly 2 Video games involve interaction with a user interface to generate visual feedback on a video device. The input device used to manipulate video games is called a game controller. These vary across platforms which include arrow button and/or a joystick. Popular games like Pac man, dome and super Mario belongs to the category of video games. These games can further be classified in three sub categories of predator/prey game, platform games and first person shooter games. Predator/prey games: In these games human player is supposed to be the prey and the rest of the entities in the game are predators, other than the walls and pills, who are suppose to kill the prey. The classical example of a predator/prey type game is Pac-Man. It was developed by Namco, a Japanese corporation, and was first released in Japan on May 22, Figure 2.4 show a typical environment of Pac-Man. The human player controls Pac-Man through arrow keys for eating pills/dots. Figure 2.4: Typical environment of Pac-Man game 3 2 Image taken from free online version of monopoly 3 Image taken from free online version of Pac-Man 13

28 Platform games: A platform game, also known as platformer, covers the jumping to and from suspended platforms or over obstacles. A human player controls these jumps and to fall from platforms or miss jumps. The most common operation of these games is jump over other features include swinging from extendable arms or bouncing. Super Mario is one of the most popular platform games today. Figure 2.5 displays a screen shot of the game. Other platform games include: Aladdin, Beauty and the Beast and Blackthorne [30]. Figure 2.5: Typical environment of super Mario game 4 First person shooter games: A first-person shooter game centres the game play around gun and projectile weapon-based combat through the first person perspective. The player experiences the action through the eyes of a protagonist. The player moves across the play area killing enemies and gaining pills, guns and other such items. The movement around the play area is generally guided by a map. Doom is one of the popular games of this category. Figure 2.6 displays the screenshot of doom. Figure 2.6: Screenshot of Doom 5 Real time strategy (RTS) games: In these games the participants position and manoeuvre various units and structures under their control to secure areas of the map and/or destroy their opponents' assets. Typically in these games it is possible to create additional units and structures during the game. This is generally limited by a requirement to expend accumulated 4 Image taken from Super Mario 4 5 Image taken from 14

29 resources. The tasks a player must perform to succeed at an RTS can be very demanding, and complex user interfaces have evolved to cope with the challenge. The most popular RTS games include WarCraft III Expansion, Age of Mythology and Rise of Nations. Figure 2.7 shows a screenshot of the popular RTS game WarCraft III. Figure 2.7: Screenshot of WarCraft III 6 Physically activation games: These are played on playgrounds requiring electronic gadgetry (e.g. electronic tiles, laser beams) [31]. Physically activation games are a hybrid of the computer games and the real world outdoor games. These games provide a good platform for game play in limited physical area and limited players. Different games can be created with different arrangement of electronic gadgets. Figure 2.8 display a child using one such game know as playware. Figure2.8: Screen shot of playware platform 7 6 Image taken from 7 Image taken from 15

30 2.4 Tools and Techniques This section is dedicated to introduce the computational intelligence tools used to accomplish the thesis objectives Evolutionary Computation Evolutionary Computation (EC) is inspired from the natural process of evolution. The concept behind evolutionary computation is survival of the fittest individuals (solutions) and death of the weak ones. In natural evolution two parents produce a child or in many cases more than one child. The same concept is inherited here were an offspring is produced from two parents which contains genetic material from both of its parents with the hope that the best features of the parents are in the offspring. Those offspring which inherit bad qualities of the parents are weak and cannot survive the battle against the strong ones. In EC we first represent the candidate solution in the shape of an individual known as the chromosome, which defines the characteristics (known as gene) of an individual. We then create a population of chromosomes. The value that can be assigned to each gene is called allele. In each generation of EC chromosomes produce offsprings. These offsprings are reproduced using two generic operators called crossover and mutation. In crossover the offspring is generated by combining parts of the parents. Mutation alters some of the alleles of the chromosome. The strength of each individual is measured using a function known as the fitness function. The fitness function depends upon the problem at hand. At the end of each generation of EC individuals may undergo culling. In addition to these basic features some other types of EC algorithms have been developed, major variants of EC are listed below: Genetic algorithms (GA): A GA models the genetic evolution. Genetic programming (GP): GP is based on GA but the individuals in the population are the programs represented most of the times as trees. 16

31 Evolution Strategies (ES): ES model the strategic parameters that control variation in evolution. Differential Evolution (DE): It is based on GA but differing in the reproduction mechanism used. Cultural Evolution (CE): It models the evolution of culture of a population and how this influences the genetic evolution of individuals. Co-evolution: Here initially random individuals evolve in competition with one another, acquiring the necessary characteristics to survive Genetic Algorithm GA was introduced by John Holland in 1975 [32]. The original GAs developed by Holland had following components: Bit string representation Proportional selection and Cross-over as the primary genetic operator. Initialize generation count G c to 0 Initialize data for generation C g Do until population converges Calculate the fitness value of each individual in population Increment generation count G c by 1 Select parents Create offspring from selected parents using crossover Mutate the offspring Select the new population using previous one and the offspring Figure 2.9: A general GA [32] Chromosome A binary vector of fixed length (but not necessarily to be of fixed length) is used to represent an individual. If the search space if of size M then the length of the chromosome is equal to M, for variables of binary nature. In the case of nominal-valued variables, each nominal value can be encoded as a D-dimensional bit vector, where 2D is the total number of discrete nominal values for that variable. Each D-bit string represents a different nominal value. In the case of continuous-valued variables, each variable should be mapped to a D- dimensional bit vector. There are some versions of GA that use integer or real-valued 17

32 representations [33-35, 36]. In addition to this the length of a chromosome is not necessarily fixed [37]. Cross-over Cross-over produces an offspring from two parents (selected using a selection process). It is not necessary that each group of parents produces offspring. There is a probability associated with cross-over, called the cross-over rate, which controls the frequency of crossover operation being performed. Figure 2.10 illustrates the concept of cross-over using two parents. There are three main types of cross-over schemes. Parent 1 Cross over point Parent 2 Child Figure 2.10: A single point crossover scheme Uniform cross-over: For uniform cross-over the mask of length L is created at random for each pair of individuals selected for reproduction. One point cross-over: A single bit position is randomly selected and the bit substrings after that point are swapped between the two chromosomes (as shown in Figure 2.10). Two point cross-over: In this case two bit positions are randomly selected, and the bit substrings between these points are swapped. Mutation The major purpose of mutation is to introduce new genetic material into an existing individual that are not inherited by the individual from its parents. Mutation supports crossover to explore the full range of allele values in the search space. Mutation also occurs at a certain probability called mutation rate. Normally the mutation rate is kept lower to ensure 18

33 that good solutions are not distorted too much. However, research has shown [36] that an initial large mutation rate that decreases exponentially as a function of the number of generations improves convergence speed and accuracy Evolution Strategy Evolution Strategy (ES), introduced by Rechenberg in the 1960s [38] and further explored by Schwefel [39], based on the concept of the assessment of evolution. Each individual is represented by its genetic building blocks and a set of strategy parameters that models the behaviour of that individual in its environment. Evolution is done by evolving the genetic characteristics and the strategy parameters. The evolution of the genetic characteristics is controlled by the strategy parameters. Changes based on mutation in EC are accepted only in case the mutated child is better in fitness as compared to its parent. In ES offspring can also be produced from more than two parents. Figure 2.11 lists the ES algorithm. 1. Create N parents containing m parameters X = (x1, x2,, x m ) Each parameter chosen randomly satisfying problem constraints 2. For each parent create a new offspring X by mutation 3. Calculate the fitness for X and X. Choose the best one for next generation 4. Repeat steps 2 and 3 until convergence occurs or the computation time ends Figure 2.11: ES algorithm Artificial Neural Networks (ANN): Inspired from the human brain an artificial neural networks (NN) consists of the four components, namely: a neuron, an activation function that is part of a neuron, a set of input, output and hidden layers, and a mechanism for learning [40, 42]. Neuron A neuron does a nonlinear mapping from I dimensional input to a range of [0, 1] or [-1,1], depending on the activation function used. To each input signal x i is associated a weight w i to strengthen or weaken the input signal. The neuron computes the net input signal, and uses an activation function to compute the output signal y given the net input. The 19

34 strength of the output signal is further influenced by a threshold value theta called bias. Figure 2.12 show a typical structure of an artificial neuron. X 1 w 1 X 2 w 2 Activation Function y X n w n Figure 2.12: Artificial Neuron The net input signal to a neuron is usually computed as the weighted sum of all input signals, listed in Equation 2.1. Net_add =X 1 W 1 + X 2 W 2 + X 3 W X m W m (2.1) An alternative to compute the net input signal is to use product units. Net_mult =X 1.W 1 X X 2.W 2 X X 3.W 3... X X m.w m (2.2) Product units allow higher-order combinations of inputs, having the advantage of increased information capacity. The activation function, the net input signal and bias determine the output of the neuron. Different types of activation functions can be used. The most common types of functions used are: Linear function Step function Ramp function Sigmoid function Hyperbolic tangent Gaussian function 20

35 ANN Learning There are three major learning paradigms namely supervised learning, unsupervised learning and reinforcement learning [36]. Usually any given type of network architecture can be employed in any of these tasks. Supervised learning In supervised learning, we are given a set of example pairs and the aim is to find a function that matches the examples. We wish to infer the mapping implied by the data; the cost function is related to the mismatch between our mapping and the data and it implicitly contains prior knowledge about the problem domain [36]. A commonly used cost is the mean-squared error which tries to minimize the average squared error between the network's output, f(x), and the target value y over all the example pairs. Unsupervised learning In unsupervised learning we are given some data x and the cost function that is to be minimized, that can be any function of the data x and the network's output, f. The cost function is dependent on the task and our a priori assumptions. Reinforcement learning In reinforcement learning, data x are usually not given, but generated by an agent's interactions with the environment. At each point in time t, the agent performs an action y t and the environment generates an observation x t and an instantaneous cost c t, according to some dynamics. The aim is to discover a policy for selecting actions that minimizes some measure of a long-term cost. 2.5 Summary A computer game is a combination of entities governed by a set of rules to be manipulated by an external entity (user) for the purpose of accomplishing a goal. Computer games have evolved from rule based games to modern games with dynamic scripting and contents. Major aim of computational intelligence in games is to make them more attractive for users. There are many genres of computer games including: board based game, 21

36 predator/prey, platform, real time strategy and physically activating games being the most popular. The games addressed in this thesis are the board based and video games (predator/prey and platform). For the purpose of achieving the objective of this thesis, measuring entertainment and automatic generation of entertaining games, we use computational intelligence tools of evolution strategy, genetic algorithm for the purpose of representing a population of game rules. This population of rules is evolved and optimized for entertainment using the proposed entertainment metrics (listed in Chapter 4) as fitness function. During evolution each individual of the population representing one complete game needs to be played multiple times this is done by using a rule based controller and an artificial neural network based controller. Next chapter lists the literature survey and different theories on entertainment in computer games. 22

37 Chapter 3 Literature Survey The concept of measuring entertainment and automatic generation of games and/or its contents is fresh. This chapter is dedicated to the work done in the domain of measuring entertainment in computer games and their automatic generation. The genres we have covered in this chapter include board based games, video games (including predator/prey and platform games) and the physically activating games. In addition to work done by computer scientists there exist some theories that address the theoretical concept of entertainment in computer games. We have listed these theories in the chapter which prove to be the bases of our proposed set of entertainment metrics in Chapter 4. We have studied the work done in this regard by different researchers and listed it here. The chapter is organized genre-wise. 3.1 Board Based Games Iida [43], in 2003, has proposed a measure of entertainment for games and used it to analyze the evolution of game of chess over the centuries. This measure is considered to be the pioneer in quantification of entertainment. Even though Iida s work is limited to chess variants, the measure of entertainment can be easily applied to other board games. According to this measure, the entertainment value of a game is equal to the length of the game (duration) divided by the average number of moves considered by a player on his turn. The game is more entertaining if the value of this measure is low. The main idea is that the player should have many choices (moves) on the average and the length of the game (duration) should not be large. Long games with few choices per move are boring. The authors differentiate between possible moves and the moves considered by a player. If the set of considered moves is smaller than the set of possible moves and the metric is based on the moves considered by a player. 23

38 In [44] the authors introduce the uncertainty of game outcome as a metric of entertainment. If the outcome is known at an early stage then there is not much interest in playing it. Similarly if it is found at the last move then it is probably probabilistic. The outcome should be unknown for a large duration of the game and should become known in the last few moves of the game. Authors state that it is easy to create new board games and variants of classical games but to make a game attractive to the human user, is challenging. In [44] a simple technique based on synchronism and stochastic elements is used to refine the game of Hex. Authors prove that the game's attraction has increased by conducting experiments to show an increment of the outcome uncertainty. In [45] Symeon uses board games for e-learning. He proposes an e-learning board game that adopts the basic elements of a racing board game and cultivates in students skills like creativity, problem-solving, and imagination, as students try to reach the end by improving their performance in a variety of learning activities. The work done in [45] addresses the issues of measuring entertainment value of the games. Except for Iida s work, all other research has been in context of computer (video) games and physically interactive games. 3.2 Video Games Togelius [46] has presented an approach to evolve entertaining car racing tracks for a video game. Tracks were represented as b-splines and the fitness of a track depended on how an evolved neural network based controller (modeled after a player) performed on the track. The objectives were for the car to have made maximum progress in a limited number of time steps (high average speed), high maximum speed (so that at least one section of the track is such that high speeds can be achieved), and high variability in performance (as measured by the final progression made) between trials (so that the track is challenging: neither too easy nor too hard). The game model used for experimentations in [46] is simple both graphically and physically (being 2D). 24