Video Game Design Chapter 5 Game Evaluation

Video Game Design Chapter 5 Game Evaluation Witcha Feungchan

How can we know is a game good or bad? Game industry reviews Usually based on expert reviews (opinions) Reviews are done after games are released Game scores are used to reflect user satisfaction and fun

GamePro 100 GameSpot 90 GameInformer 88 fantastic single- and multiplayer modes... amazing visuals, an intense campaign... the best-looking shooter on the PS3...

GamePro 100 GameSpot 90 GameInformer 88 fantastic single- and multiplayer modes... amazing visuals, an intense campaign... the best-looking shooter on the PS3...

GamePro 100 GameSpot 90 GameInformer 88 fantastic single- and multiplayer modes... amazing visuals, an intense campaign... the best-looking shooter on the PS3...

How can we know is a game good or bad? Formal analysis Analysis can be done in every stages of game development Provide only factors for game design Lack of evaluation method

Academic References (excerpt) [1] M. A. Federoff, "Heuristics and usability guidelines for the creation and evaluation of fun in video games," 2002, pp. viii, 86 leaves. [2] R. M. Ryan, C. S. Rigby, and A. Przybylski, "The Motivational Pull of Video Games: A Self-Determination Theory Approach," Motivation and Emotion, vol. 30, pp. 344-360, 2006. [3] S. Penelope and W. Peta, "GameFlow: a model for evaluating player enjoyment in games," Comput. Entertain., vol. 3, pp. 3-3, 2005. [4] S. Cornett, "The usability of massively multiplayer online roleplaying games: designing for new users," in Proceedings of the SIGCHI conference on Human factors in computing systems. Vienna, Austria: ACM Press, 2004. [5] P. Sweetser and P. Wyeth, "GameFlow: a model for evaluating player enjoyment in games," Computer in Entertainment (CIE), vol. 3, pp. 3-3, 2005. [6] M. B. Oliver and R. L. Nabi, "Exploring the Concept of Media Enjoyment: An Introduction to the Special Issue," Commun Theory, vol. 14, pp. 285-287, 2004. [7] Metacritic.com, "About Metascores," vol. 2006: CNET Networks <http://www.metacritic.com/about/scoring.shtml>, 2006. [8] ISO9241-11, "Ergonomic Requirements for Office Work with Visual Display Terminals (VDTs) Part 11: Guidance on Usability," International Organization for Standardization, International Standard March 15 1998. [9] B. FOX, Game interface design. Boston, Mass.: Thomson Course Technology, 2005. [10] W. G. Cochran, Sampling techniques, 3d ed. New York: Wiley, 1977. [11] J. E. Bartlett_II, J. W. Kotrlik, and C. C. Higgins, "Organizational Research: Determining Appropriate Sample Size in Survey Research," Information Technology, Learning, and Performance, vol. 19, pp. 43-50, 2001. [12] W. Mendenhall and T. Sincich, Statistics for engineering and the sciences, 4th ed. Upper Saddle River: Prentice Hall, 1995. [13] T. H. Apperley, "Genre and game studies: Toward a critical approach to video game genres," Simulation and Gaming, vol. 37, pp. 6-23, 2006. [14] Wikipedia, "Computer and video games," vol. 2005: Wikipedia <http://en.wikipedia.org/wiki/computer_and_video_games>, 2005. [15] SPSS Inc., SPSS Base 9.0 applications guide. Chicago, IL: SPSS Inc., 1999. [16] N. R. Draper and H. Smith, Applied regression analysis, 3rd ed. New York: Wiley, 1998. [17] B. Shelley, "Guidelines for Developing Successful Games," vol. 2006: Gamasutra <URL: http://www.gamasutra.com/features/20010815/shelley_01.htm>, 2001. [18] D. S.-C. Dalmau, "Learn Faster to Play Better: How to Shorten the Learning Cycle," vol. 2006: Gamasutra <http://www.gamasutra.com/features/19991108/ dalmau_01.htm>, 1999. [19] A. Rollings and E. Adams, Andrew Rollings and Ernest Adams on game design, 1st ed. Indianapolis, Ind.: New Riders, 2003. [20] T. W. Malone, "Heuristics for designing enjoyable user interfaces: Lessons from computer games," in Proceedings of the 1982 conference on Human factors in computing systems. Gaithersburg, Maryland, United States: ACM Press, 1982. [21] E. Brown and P. Cairns, "A grounded investigation of game immersion," in CHI '04 extended abstracts on Human factors in computing systems. Vienna, Austria: ACM Press, 2004.

Game Evaluation There are a lot of game evaluation systems and reviews from game industry but little research in academia. We created our game evaluation model from collected information of both industry and academia. An experiment was conducted to find an evaluation model that can be used to justify and distinguish between good and bad games.

Evaluation Analysis 59 evaluation factors From game industry and game academia From 5 groups (game interface, gameplay, game mechanic, immersion, and social interaction) 60 games as the sample From various game genres (action, adventure, role-playing (RPGs), simulation, strategy, and puzzle games) From various game platforms (game consoles, portable gaming devices, computers, PDAs, and mobile phones)

Evaluation Analysis (cont d) Internal evaluation One expert evaluated and gave the scores for all samples. External evaluation The overall game scores for all games were taken from a reliable source (www.metacritic.com).

www.metacritic.com

Evaluation Analysis (cont d) We apply statistical methods to explore the key factors for game evaluation. Descriptive analysis Principal component analysis Regression Analysis (model creation)

Regression Analysis External evaluation scores are used as the independent variables 36 factors are used in the regression Using backward regression technique to find the game evaluation model Adjusted R 2 = 0.997 Standard error of estimation is 4.26 (<10%) Do these 10 variables reliably predict the evaluation score?, ANOVA test showed that the model p-value (0.0000) is smaller than the alpha level (0.05); we can conclude, Yes, these 10 variables reliably predict the evaluation score

Regression Analysis (cont d)

Game Evaluation Model Pr edictedscore = 5.5A + 1.9B +1.6C + 2.4D +1.9E + 4.5F + 4.8G + 6.6H + 3.3I + 3.3J Factors Descriptions A B C D E F G H I J Minimize control options Follow the trends set by the gaming community to shorten the learning curve Players should be able to save games in different states Intermediate goals should be clear and presented at the appropriate times Make game replayable Should use visual and audio effects to arouse interest Players should become less aware of their surroundings Game should increase the players' skill at the appropriate pace as they progress through the game Allow player to build content Build as though the world is going on whether your character is there or not

Case Study To test the evaluation model, we used the model to evaluate 2 games to see how accurate of the prediction scores compare with the industrial score. We picked 2 games to evaluate with our model; Half Life 2 (PC), and The Elder Scrolls IV: Oblivion (Playstation 3). Games Predicted score Industrial score Half Life 2 98.4 96 The Elder Scrolls IV: Oblivion 96.5 93

Questions?