# Playing and Solving Games

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Playing and Solving Games Zero-sum games with perfect information R&N 6 Definitions Game evaluation Optimal solutions Minimax Alpha-beta pruning Approximations Heuristic evaluation functions Cutoffs Endgames Non-deterministic games (first take) 1

2 Types of Games (informal) Deterministic Chance Perfect Information Imperfect Information Chess, Checkers Go Battleship Backgammon, Monopoly Bridge, Poker, Scrabble, Nuclear war Types of Games (informal) Deterministic Chance Perfect Information Chess, Checkers Go Backgammon, Monopoly Imperfect Information Battleship Bridge, Poker, Scrabble, Nuclear war Note: This initial material uses the common definition of what a game is. More interesting is the generalization of the theory to scenarios that are far more useful to a wide range of decision making problems. Stay tuned. 2

3 Definitions Two-player game: Player A and B. Player A starts. Deterministic: None of the moves/states are subject to chance (no random draws). Perfect information: Both players see all the states and decisions. Each decision is made sequentially. Zero-sum: Player s A gain is exactly equal to player B s loss. One of the player s must win or there is a draw (both gains are equal). Example Initially a stack of pennies stands between two players Each player divides one of the current stacks into two unequal stacks. The game ends when every stack contains one or two pennies The first player who cannot play loses A B 3

4 7 A s turn 6, 1 5, 2 4, 3 B s turn 5, 1, 1 4, 2, 1 3, 2, 2 3, 3, 1 A s turn 4, 1, 1, 1 3, 2, 1, 1 3, 1, 1, 1, 1 2, 1, 1, 1, 1, 1 B Loses 2, 2, 1, 1, 1 A Loses 2, 2, 2, 1 B Loses B s turn A s turn B s turn Search Problem States: Board configuration + next player to move Successor: List of states that can be reached from the current state through of legal moves Terminal state: States at which the games ends Payoff/Utility: Numerical value assigned to each terminal state. Example: U(s) = +1 for A win, -1 for B win, 0 for draw Game value: The value of a terminal that will be reached assuming optimal strategies from both players (minimax value) Search: Find move that maximizes game value from current state 4

5 U = +1 U = -1 2, 2, 2, 1 U = +1 2, 2, 1, 1, 1 2, 1, 1, 1, 1, 1 Optimal (minimax) Strategies Search the game tree such that: A s turn to move find the move that yields maximum payoff from the corresponding subtree This is the move most favorable to A B s turn to move find the move that yields minimum payoff (best for B) from the corresponding subtree This is the move most favorable to B 5

6 Minimax Minimax (s) If s is terminal Return U(s) If next move is A ( s' ) Return max Minimax s' Succs( s) Else min Minimax s' Return s' Succs( s) ( ) A 3 = max(3,2,2) B 3 = min(3,12,8)

7 Minimax Properties Complete: If finite game Optimal: If opponent plays optimally Complexity: Essentially DFS, so: Time: O(B m ) Space: O(Bm) B = number of possible moves from any state (branching factor) m = depth of search (length of game) Pruning B 8 A A cutoff The value at A move is 8 (so far) If A moves right, the value there is 1 (so far) B will never increase the value at this node; it will always be less than 8 B can ignore the remaining nodes

8 Pruning A 3 =3 =2 B = X X Maintain: αβ Pruning α = Best value found so far at A nodes, including those at current node β = Best value found so far at B nodes, including those at current node If at a B node: No need to expand this node any further if α >= β because there is no way that a descendant of the current node can yield a better value 8

9 Minimax (s,α,β) If s is terminal Return U(s) If A node For each s in Succs(s) α = Max(α, Minimax(s,α,β) If (α >= β) Return β Return α If B node For each s in Succs(s) β = Min(β, Minimax(s,α,β) If (α >= β) Return α Return β Same as default minimax Minimax (s,α,β) If s is terminal Return U(s) Prune if no If A node better For each s in Succs(s) solution can α = Max(α, Minimax(s,α,β) be found If (α >= β) Return β Return α If B node For each s in Succs(s) β = Min(β, Minimax(s,α,β) If (α >= β) Return α Return β 9

10 Properties Guaranteed to find same solution O(B m/2 ) with proper ordering of the nodes At A node, the successor are in order from high to low score Use heuristic evaluation functions to cut off search early Example: Weighted sum of number of pieces (material value of state) Stop search based on cutoff test (e.g., maximum depth) Iterative deepening search to limit DFS Solve by brute-force dynamic programming when the number of states is small Choice of Value? Absolute game value is different in the two cases Minimax solution is the same Only the relative ordering of values matters, not the absolute values ordinal utility values True only for deterministic games Evaluation functions can be any function that preserves the ordering of the utility values 10

11 Non-Deterministic Games A Non-Deterministic Games Chance B 11

12 Chance Non-Deterministic Games Includes states where neither player makes a A choice. A random decision is made (e.g., rolling dice) Use expected value of successors at chance nodes: s' Succs( s) p( s') MiniMax( s') B Choice of Utility Values Different utility values may yield radically different result even though the order is the same Absolute utility values do matter Utility should be proportional to actual payoff, it is not sufficient to follow the same order Think of choosing between 2 lotteries with same odds but radically different payoff distributions Implication: Evaluation functions must be linear positive functions of utility Kind of obvious but important consideration for later developments 12

13 Non-Deterministic Minimax Minimax (s) If s is terminal Return U(s) max Minimax If next move is A: Return If next move is B Return min s' Succs( s) Minimax s' Succs( s) ( s' ) ( s' ) If chance node Return p s' Succs( s) ( s' ) Minimax( s' ) Properties α-β pruning can be extended provided that the utility values are bounded We don t need to evaluate all the children of a chance node to bound the average Less effective Different outcomes depending on exact values of utility, not just ordering 13

14 Definitions Game evaluation Optimal solutions Minimax Alpha-beta pruning Approximations Heuristic evaluation functions Cutoffs Endgames Non-deterministic games 14

### COMP-424: Artificial intelligence. Lecture 7: Game Playing

COMP 424 - Artificial Intelligence Lecture 7: Game Playing Instructor: Joelle Pineau (jpineau@cs.mcgill.ca) Class web page: www.cs.mcgill.ca/~jpineau/comp424 Unless otherwise noted, all material posted

### Game playing. Chapter 6. Chapter 6 1

Game playing Chapter 6 Chapter 6 1 Outline Games Perfect play minimax decisions α β pruning Resource limits and approximate evaluation Games of chance Games of imperfect information Chapter 6 2 Games vs.

### Game-Playing & Adversarial Search MiniMax, Search Cut-off, Heuristic Evaluation

Game-Playing & Adversarial Search MiniMax, Search Cut-off, Heuristic Evaluation This lecture topic: Game-Playing & Adversarial Search (MiniMax, Search Cut-off, Heuristic Evaluation) Read Chapter 5.1-5.2,

### Adversarial Search and Game- Playing. C H A P T E R 5 C M P T 3 1 0 : S u m m e r 2 0 1 1 O l i v e r S c h u l t e

Adversarial Search and Game- Playing C H A P T E R 5 C M P T 3 1 0 : S u m m e r 2 0 1 1 O l i v e r S c h u l t e Environment Type Discussed In this Lecture 2 Fully Observable yes Turn-taking: Semi-dynamic

### AI and computer game playing. Artificial Intelligence. Partial game tree for Tic-Tac-Toe. Game tree search. Minimax Algorithm

AI and computer game playing Artificial Intelligence Introduction to game-playing search imax algorithm, evaluation functions, alpha-beta pruning, advanced techniques Game playing (especially chess and

### Minimax Strategies. Minimax Strategies. Zero Sum Games. Why Zero Sum Games? An Example. An Example

Everyone who has studied a game like poker knows the importance of mixing strategies With a bad hand, you often fold But you must bluff sometimes Lectures in Microeconomics-Charles W Upton Zero Sum Games

### Laboratory work in AI: First steps in Poker Playing Agents and Opponent Modeling

Laboratory work in AI: First steps in Poker Playing Agents and Opponent Modeling Avram Golbert 01574669 agolbert@gmail.com Abstract: While Artificial Intelligence research has shown great success in deterministic

### CS91.543 MidTerm Exam 4/1/2004 Name: KEY. Page Max Score 1 18 2 11 3 30 4 15 5 45 6 20 Total 139

CS91.543 MidTerm Exam 4/1/2004 Name: KEY Page Max Score 1 18 2 11 3 30 4 15 5 45 6 20 Total 139 % INTRODUCTION, AI HISTORY AND AGENTS 1. [4 pts. ea.] Briefly describe the following important AI programs.

### Chess Algorithms Theory and Practice. Rune Djurhuus Chess Grandmaster runed@ifi.uio.no / runedj@microsoft.com October 3, 2012

Chess Algorithms Theory and Practice Rune Djurhuus Chess Grandmaster runed@ifi.uio.no / runedj@microsoft.com October 3, 2012 1 Content Complexity of a chess game History of computer chess Search trees

### Sequential lmove Games. Using Backward Induction (Rollback) to Find Equilibrium

Sequential lmove Games Using Backward Induction (Rollback) to Find Equilibrium Sequential Move Class Game: Century Mark Played by fixed pairs of players taking turns. At each turn, each player chooses

### Monte-Carlo Methods. Timo Nolle

Monte-Carlo Methods Timo Nolle Outline Minimax Expected Outcome Monte-Carlo Monte-Carlo Simulation Monte-Carlo Tree Search UCT AMAF RAVE MC-RAVE UCT-RAVE Playing Games Go, Bridge, Scrabble Problems with

### Chess Algorithms Theory and Practice. Rune Djurhuus Chess Grandmaster runed@ifi.uio.no / runedj@microsoft.com September 23, 2014

Chess Algorithms Theory and Practice Rune Djurhuus Chess Grandmaster runed@ifi.uio.no / runedj@microsoft.com September 23, 2014 1 Content Complexity of a chess game Solving chess, is it a myth? History

### Techniques to Parallelize Chess Matthew Guidry, Charles McClendon

Techniques to Parallelize Chess Matthew Guidry, Charles McClendon Abstract Computer chess has, over the years, served as a metric for showing the progress of computer science. It was a momentous occasion

### Backward Induction and Subgame Perfection

Backward Induction and Subgame Perfection In extensive-form games, we can have a Nash equilibrium profile of strategies where player 2 s strategy is a best response to player 1 s strategy, but where she

### Acknowledgements I would like to thank both Dr. Carsten Furhmann, and Dr. Daniel Richardson for providing me with wonderful supervision throughout my

Abstract Game orientated application programming interfaces (API), built using Java, are intended to ease the development of Java games, on multiple platforms. Existing Java game APIs, tend to focus on

### Nash Equilibrium. Ichiro Obara. January 11, 2012 UCLA. Obara (UCLA) Nash Equilibrium January 11, 2012 1 / 31

Nash Equilibrium Ichiro Obara UCLA January 11, 2012 Obara (UCLA) Nash Equilibrium January 11, 2012 1 / 31 Best Response and Nash Equilibrium In many games, there is no obvious choice (i.e. dominant action).

### Playing around with Risks

Playing around with Risks Jurgen Cleuren April 19th 2012 2011 CTG, Inc. Introduction Projects are done in a probabilistic environment Incomplete information Parameters change over time What is true in

### Monte Carlo Tree Search and Opponent Modeling through Player Clustering in no-limit Texas Hold em Poker

Monte Carlo Tree Search and Opponent Modeling through Player Clustering in no-limit Texas Hold em Poker A.A.J. van der Kleij August 2010 Master thesis Artificial Intelligence University of Groningen, The

### The Taxman Game. Robert K. Moniot September 5, 2003

The Taxman Game Robert K. Moniot September 5, 2003 1 Introduction Want to know how to beat the taxman? Legally, that is? Read on, and we will explore this cute little mathematical game. The taxman game

### Search methods motivation 1

Suppose you are an independent software developer, and your software package Windows Defeater R, widely available on sourceforge under a GNU GPL license, is getting an international attention and acclaim.

### Optimization in ICT and Physical Systems

27. OKTOBER 2010 in ICT and Physical Systems @ Aarhus University, Course outline, formal stuff Prerequisite Lectures Homework Textbook, Homepage and CampusNet, http://kurser.iha.dk/ee-ict-master/tiopti/

### Game theory and AI: a unified approach to poker games

Game theory and AI: a unified approach to poker games Thesis for graduation as Master of Artificial Intelligence University of Amsterdam Frans Oliehoek 2 September 2005 ii Abstract This thesis focuses

### 10. Machine Learning in Games

Machine Learning and Data Mining 10. Machine Learning in Games Luc De Raedt Thanks to Johannes Fuernkranz for his slides Contents Game playing What can machine learning do? What is (still) hard? Various

### COMP30191 The Theory of Games and Game Models

COMP30191 The Theory of Games and Game Models Andrea Schalk A.Schalk@cs.man.ac.uk School of Computer Science University of Manchester August 7, 2008 About this course This is an introduction into the theory

### 1 Representation of Games. Kerschbamer: Commitment and Information in Games

1 epresentation of Games Kerschbamer: Commitment and Information in Games Game-Theoretic Description of Interactive Decision Situations This lecture deals with the process of translating an informal description

### Example: Tree Diagram

Foul Or Not? Strategy is certainly crucial to any sport or any situation where we have a confrontation or conflict. In this section, we will see how probability and expected value can help us make better

### Mastering Quoridor. Lisa Glendenning THESIS. Submitted in Partial Fulfillment of the Requirements for the Degree of

Mastering Quoridor by Lisa Glendenning THESIS Submitted in Partial Fulfillment of the Requirements for the Degree of Bachelor of Science Computer Science The University of New Mexico Albuquerque, New Mexico

### Implementation of a Chess Playing Artificial. Intelligence. Sean Stanek.

Implementation of a Chess Playing Artificial Intelligence Sean Stanek vulture@cs.iastate.edu Overview Chess has always been a hobby of mine, and so have computers. In general, I like creating computer

### Game Theory. Themes. 1. Introduction to Game Theory 2. Sequential Games 3. Simultaneous Games 4. Conclusion. Introduction to Game Theory

Game Theory Themes 1. Introduction to Game Theory 2. Sequential Games 3. Simultaneous Games 4. Conclusion Introduction to Game Theory Game theory is the branch of decision theory concerned with interdependent

### AI: A Modern Approach, Chpts. 3-4 Russell and Norvig

AI: A Modern Approach, Chpts. 3-4 Russell and Norvig Sequential Decision Making in Robotics CS 599 Geoffrey Hollinger and Gaurav Sukhatme (Some slide content from Stuart Russell and HweeTou Ng) Spring,

### FIRST EXPERIMENTAL RESULTS OF PROBCUT APPLIED TO CHESS

FIRST EXPERIMENTAL RESULTS OF PROBCUT APPLIED TO CHESS A.X. Jiang Department of Computer Science, University of British Columbia, Vancouver, Canada albertjiang@yahoo.com M. Buro Department of Computing

### Lab 11. Simulations. The Concept

Lab 11 Simulations In this lab you ll learn how to create simulations to provide approximate answers to probability questions. We ll make use of a particular kind of structure, called a box model, that

### One pile, two pile, three piles

CHAPTER 4 One pile, two pile, three piles 1. One pile Rules: One pile is a two-player game. Place a small handful of stones in the middle. At every turn, the player decided whether to take one, two, or

### Decision Theory. 36.1 Rational prospecting

36 Decision Theory Decision theory is trivial, apart from computational details (just like playing chess!). You have a choice of various actions, a. The world may be in one of many states x; which one

### Game Theory 1. Introduction

Game Theory 1. Introduction Dmitry Potapov CERN What is Game Theory? Game theory is about interactions among agents that are self-interested I ll use agent and player synonymously Self-interested: Each

### 6.042/18.062J Mathematics for Computer Science. Expected Value I

6.42/8.62J Mathematics for Computer Science Srini Devadas and Eric Lehman May 3, 25 Lecture otes Expected Value I The expectation or expected value of a random variable is a single number that tells you

### Introduction Solvability Rules Computer Solution Implementation. Connect Four. March 9, 2010. Connect Four

March 9, 2010 is a tic-tac-toe like game in which two players drop discs into a 7x6 board. The first player to get four in a row (either vertically, horizontally, or diagonally) wins. The game was first

### Roulette Wheel Selection Game Player

Macalester College DigitalCommons@Macalester College Honors Projects Mathematics, Statistics, and Computer Science 5-1-2013 Roulette Wheel Selection Game Player Scott Tong Macalester College, stong101@gmail.com

### How to Win Texas Hold em Poker

How to Win Texas Hold em Poker Richard Mealing Machine Learning and Optimisation Group School of Computer Science University of Manchester / 44 How to Play Texas Hold em Poker Deal private cards per player

### Contemporary Mathematics- MAT 130. Probability. a) What is the probability of obtaining a number less than 4?

Contemporary Mathematics- MAT 30 Solve the following problems:. A fair die is tossed. What is the probability of obtaining a number less than 4? What is the probability of obtaining a number less than

Load Balancing Backtracking, branch & bound and alpha-beta pruning: how to assign work to idle processes without much communication? Additionally for alpha-beta pruning: implementing the young-brothers-wait

### Game Playing in the Real World. Next time: Knowledge Representation Reading: Chapter 7.1-7.3

Game Playing in the Real World Next time: Knowledge Representation Reading: Chapter 7.1-7.3 1 What matters? Speed? Knowledge? Intelligence? (And what counts as intelligence?) Human vs. machine characteristics

### AP Stats - Probability Review

AP Stats - Probability Review Multiple Choice Identify the choice that best completes the statement or answers the question. 1. I toss a penny and observe whether it lands heads up or tails up. Suppose

### Bayesian Nash Equilibrium

. Bayesian Nash Equilibrium . In the final two weeks: Goals Understand what a game of incomplete information (Bayesian game) is Understand how to model static Bayesian games Be able to apply Bayes Nash

Advanced Game Theory Herve Moulin Baker Hall 263; moulin@rice.edu Rice University ECO 440 Spring 2007 1 Two person zero sum games 1.1 Introduction: strategic interdependency In this section we study games

### Lesson 13: Games of Chance and Expected Value

Student Outcomes Students analyze simple games of chance. Students calculate expected payoff for simple games of chance. Students interpret expected payoff in context. esson Notes When students are presented

### Measuring the Performance of an Agent

25 Measuring the Performance of an Agent The rational agent that we are aiming at should be successful in the task it is performing To assess the success we need to have a performance measure What is rational

### ECON 459 Game Theory. Lecture Notes Auctions. Luca Anderlini Spring 2015

ECON 459 Game Theory Lecture Notes Auctions Luca Anderlini Spring 2015 These notes have been used before. If you can still spot any errors or have any suggestions for improvement, please let me know. 1

### Week 5: Expected value and Betting systems

Week 5: Expected value and Betting systems Random variable A random variable represents a measurement in a random experiment. We usually denote random variable with capital letter X, Y,. If S is the sample

### University of Alberta. Library Release Form

University of Alberta Library Release Form Name of Author: Michael Bradley Johanson Title of Thesis: Robust Strategies and Counter-Strategies: Building a Champion Level Computer Poker Player Degree: Master

### Decision Making under Uncertainty

6.825 Techniques in Artificial Intelligence Decision Making under Uncertainty How to make one decision in the face of uncertainty Lecture 19 1 In the next two lectures, we ll look at the question of how

### Avoiding the Consolation Prize: The Mathematics of Game Shows

Avoiding the Consolation Prize: The Mathematics of Game Shows STUART GLUCK, Ph.D. CENTER FOR TALENTED YOUTH JOHNS HOPKINS UNIVERSITY STU@JHU.EDU CARLOS RODRIGUEZ CENTER FOR TALENTED YOUTH JOHNS HOPKINS

### This is a game for two players and one referee. You will need a deck of cards (Ace 10).

PLAYER SALUTE This is a game for two players and one referee. You will need a deck of cards (Ace 10). Each player draws a card and, without looking, holds the card on his/her forehead facing outward. The

### Stat 20: Intro to Probability and Statistics

Stat 20: Intro to Probability and Statistics Lecture 16: More Box Models Tessa L. Childers-Day UC Berkeley 22 July 2014 By the end of this lecture... You will be able to: Determine what we expect the sum

### Probabilistic Strategies: Solutions

Probability Victor Xu Probabilistic Strategies: Solutions Western PA ARML Practice April 3, 2016 1 Problems 1. You roll two 6-sided dice. What s the probability of rolling at least one 6? There is a 1

### Models for Discrete Variables

Probability Models for Discrete Variables Our study of probability begins much as any data analysis does: What is the distribution of the data? Histograms, boxplots, percentiles, means, standard deviations

### Elementary Statistics and Inference. Elementary Statistics and Inference. 16 The Law of Averages (cont.) 22S:025 or 7P:025.

Elementary Statistics and Inference 22S:025 or 7P:025 Lecture 20 1 Elementary Statistics and Inference 22S:025 or 7P:025 Chapter 16 (cont.) 2 D. Making a Box Model Key Questions regarding box What numbers

### 1. (a) Multiply by negative one to make the problem into a min: 170A 170B 172A 172B 172C Antonovics Foster Groves 80 88

Econ 172A, W2001: Final Examination, Possible Answers There were 480 possible points. The first question was worth 80 points (40,30,10); the second question was worth 60 points (10 points for the first

### Hoover High School Math League. Counting and Probability

Hoover High School Math League Counting and Probability Problems. At a sandwich shop there are 2 kinds of bread, 5 kinds of cold cuts, 3 kinds of cheese, and 2 kinds of dressing. How many different sandwiches

### 10/13/11 Solution: Minimax with Alpha-Beta Pruning and Progressive Deepening

10/1/11 Solution: Minimax with Alpha-Beta Pruning and Progressive Deepening When answering the question in Parts C.1 and C. below, assume you have already applied minimax with alpha-beta pruning and progressive

### Ch. 13.2: Mathematical Expectation

Ch. 13.2: Mathematical Expectation Random Variables Very often, we are interested in sample spaces in which the outcomes are distinct real numbers. For example, in the experiment of rolling two dice, we

### 15-251: Great Theoretical Ideas in Computer Science Anupam Gupta Notes on Combinatorial Games (draft!!) January 29, 2012

15-251: Great Theoretical Ideas in Computer Science Anupam Gupta Notes on Combinatorial Games (draft!!) January 29, 2012 1 A Take-Away Game Consider the following game: there are 21 chips on the table.

### Classification/Decision Trees (II)

Classification/Decision Trees (II) Department of Statistics The Pennsylvania State University Email: jiali@stat.psu.edu Right Sized Trees Let the expected misclassification rate of a tree T be R (T ).

### TomVerhoeff. Technische Universiteit Eindhoven Faculty of Math. & Computing Science Software Construction

Optimal Solitaire Yahtzee Strategies TomVerhoeff Technische Universiteit Eindhoven Faculty of Math. & Computing Science Software Construction T.Verhoeff@TUE.NL http://www.win.tue.nl/~wstomv/yahtzee/ *)

### .4 120 +.1 80 +.5 100 = 48 + 8 + 50 = 106.

Chapter 16. Risk and Uncertainty Part A 2009, Kwan Choi Expected Value X i = outcome i, p i = probability of X i EV = pix For instance, suppose a person has an idle fund, \$100, for one month, and is considering

### Generalized Widening

Generalized Widening Tristan Cazenave Abstract. We present a new threat based search algorithm that outperforms other threat based search algorithms and selective knowledge-based for open life and death

### Alpha-Beta with Sibling Prediction Pruning in Chess

Alpha-Beta with Sibling Prediction Pruning in Chess Jeroen W.T. Carolus Masters thesis Computer Science Supervisor: Prof. dr. Paul Klint Faculteit der Natuurwetenschappen, Wiskunde en Informatica University

### Expected Value. 24 February 2014. Expected Value 24 February 2014 1/19

Expected Value 24 February 2014 Expected Value 24 February 2014 1/19 This week we discuss the notion of expected value and how it applies to probability situations, including the various New Mexico Lottery

### The Next Move. Some Theory and Practice with Impartial Games. Saturday Morning Math Group. March 1, 2014

The Next Move Some Theory and Practice with Impartial Games Saturday Morning Math Group March 1, 2014 Combinatorial games are two-player games with the following characteristics: Two players alternate

### 6.254 : Game Theory with Engineering Applications Lecture 2: Strategic Form Games

6.254 : Game Theory with Engineering Applications Lecture 2: Strategic Form Games Asu Ozdaglar MIT February 4, 2009 1 Introduction Outline Decisions, utility maximization Strategic form games Best responses

### Creating a NL Texas Hold em Bot

Creating a NL Texas Hold em Bot Introduction Poker is an easy game to learn by very tough to master. One of the things that is hard to do is controlling emotions. Due to frustration, many have made the

### The Cross-Platform Implementation

The Cross-Platform Implementation Of Draughts Game Valery Kirkizh State University of Aerospace Instrumentation Saint-Petersburg, Russia Introduction Age-old Russian game Draughts; Cross-platform implementation,

### A Least-Certainty Heuristic for Selective Search 1

A Least-Certainty Heuristic for Selective Search 1 Paul E. Utgoff utgoff@cs.umass.edu Department of Computer and Information Science, University of Massachusetts, Amherst, MA 01003 Richard P. Cochran cochran@cs.umass.edu

### 1.3. Knights. Training Forward Thinking

1.3 Knights Hippogonal 2,1 movement. Captures by landing on the same square as an enemy piece. Can jump over other pieces Count one, two, turn. There has always been a horse in chess and the chess variants

### Decision Making Under Uncertainty. Professor Peter Cramton Economics 300

Decision Making Under Uncertainty Professor Peter Cramton Economics 300 Uncertainty Consumers and firms are usually uncertain about the payoffs from their choices Example 1: A farmer chooses to cultivate

### Parallelizing a Simple Chess Program

Parallelizing a Simple Chess Program Brian Greskamp ECE412, Fall 2003 E-mail: greskamp@crhc.uiuc.edu Abstract In a closely watched match in 1997, the parallel chess supercomputer Deep Blue defeated then

### INTRODUCTION 4 1 OVERALL LOOK ON CHESS PROGRAMMING 8

Table of contents INTRODUCTION 4 AIM OF THE WORK 4 LAYOUT 4 RANGE OF THE WORK 5 RESEARCH PROBLEM 5 RESEARCH QUESTIONS 6 METHODS AND TOOLS 6 1 OVERALL LOOK ON CHESS PROGRAMMING 8 1.1 THE VERY BEGINNING

### Quiz CHAPTER 16 NAME: UNDERSTANDING PROBABILITY AND LONG- TERM EXPECTATIONS

Quiz CHAPTER 16 NAME: UNDERSTANDING PROBABILITY AND LONG- TERM EXPECTATIONS 1. Give two examples of ways that we speak about probability in our every day lives. NY REASONABLE ANSWER OK. EXAMPLES: 1) WHAT

### V. RANDOM VARIABLES, PROBABILITY DISTRIBUTIONS, EXPECTED VALUE

V. RANDOM VARIABLES, PROBABILITY DISTRIBUTIONS, EXPETED VALUE A game of chance featured at an amusement park is played as follows: You pay \$ to play. A penny and a nickel are flipped. You win \$ if either

### We { can see that if U = 2, 3, 7, 11, or 12 then the round is decided on the first cast, U = V, and W if U = 7, 11 X = L if U = 2, 3, 12.

How to Play Craps: Craps is a dice game that is played at most casinos. We will describe here the most common rules of the game with the intention of understanding the game well enough to analyze the probability

### We employed reinforcement learning, with a goal of maximizing the expected value. Our bot learns to play better by repeated training against itself.

Date: 12/14/07 Project Members: Elizabeth Lingg Alec Go Bharadwaj Srinivasan Title: Machine Learning Applied to Texas Hold 'Em Poker Introduction Part I For the first part of our project, we created a

### State space representation. General scheme of searching the state space. Blind and informed strategies. Short review

Suppose you are an independent software developer, and your software package Windows Defeater R, widely available on sourceforge under a GNU GPL license, is getting an international attention and acclaim.

### Using Probabilistic Knowledge and Simulation to Play Poker

Using Probabilistic Knowledge and Simulation to Play Poker Darse Billings, Lourdes Peña, Jonathan Schaeffer, Duane Szafron Department of Computing Science, University of Alberta Edmonton, Alberta Canada

### Mathematical goals. Starting points. Materials required. Time needed

Level S2 of challenge: B/C S2 Mathematical goals Starting points Materials required Time needed Evaluating probability statements To help learners to: discuss and clarify some common misconceptions about

### Abstracting Reusable Cases from Reinforcement Learning

Abstracting Reusable Cases from Reinforcement Learning Andreas von Hessling and Ashok K. Goel College of Computing Georgia Institute of Technology Atlanta, GA 30318 {avh, goel}@cc.gatech.edu Abstract.

### Linear Programming: Chapter 11 Game Theory

Linear Programming: Chapter 11 Game Theory Robert J. Vanderbei October 17, 2007 Operations Research and Financial Engineering Princeton University Princeton, NJ 08544 http://www.princeton.edu/ rvdb Rock-Paper-Scissors

### Lecture V: Mixed Strategies

Lecture V: Mixed Strategies Markus M. Möbius February 26, 2008 Osborne, chapter 4 Gibbons, sections 1.3-1.3.A 1 The Advantage of Mixed Strategies Consider the following Rock-Paper-Scissors game: Note that

### Clock Arithmetic and Modular Systems Clock Arithmetic The introduction to Chapter 4 described a mathematical system

CHAPTER Number Theory FIGURE FIGURE FIGURE Plus hours Plus hours Plus hours + = + = + = FIGURE. Clock Arithmetic and Modular Systems Clock Arithmetic The introduction to Chapter described a mathematical

### cs171 HW 1 - Solutions

1. (Exercise 2.3 from RN) For each of the following assertions, say whether it is true or false and support your answer with examples or counterexamples where appropriate. (a) An agent that senses only

### Best-response play in partially observable card games

Best-response play in partially observable card games Frans Oliehoek Matthijs T. J. Spaan Nikos Vlassis Informatics Institute, Faculty of Science, University of Amsterdam, Kruislaan 43, 98 SJ Amsterdam,

### Feature Construction for Reinforcement Learning in Hearts

Feature Construction for Reinforcement Learning in Hearts Nathan R. Sturtevant and Adam M. White Department of Computer Science University of Alberta Edmonton, AB, Canada T6G 2E8 {nathanst, awhite}@cs.ualberta.ca

### Game Theory and Algorithms Lecture 10: Extensive Games: Critiques and Extensions

Game Theory and Algorithms Lecture 0: Extensive Games: Critiques and Extensions March 3, 0 Summary: We discuss a game called the centipede game, a simple extensive game where the prediction made by backwards

### The New Mexico Lottery

The New Mexico Lottery 26 February 2014 Lotteries 26 February 2014 1/27 Today we will discuss the various New Mexico Lottery games and look at odds of winning and the expected value of playing the various

### Gamesman: A Graphical Game Analysis System

Gamesman: A Graphical Game Analysis System Dan Garcia Abstract We present Gamesman, a graphical system for implementing, learning, analyzing and playing small finite two-person

### TD-Gammon, A Self-Teaching Backgammon Program, Achieves Master-Level Play

TD-Gammon, A Self-Teaching Backgammon Program, Achieves Master-Level Play Gerald Tesauro IBM Thomas J. Watson Research Center P. O. Box 704 Yorktown Heights, NY 10598 (tesauro@watson.ibm.com) Abstract.

### Learning to Play Board Games using Temporal Difference Methods

Learning to Play Board Games using Temporal Difference Methods Marco A. Wiering Jan Peter Patist Henk Mannen institute of information and computing sciences, utrecht university technical report UU-CS-2005-048

### Equilibrium computation: Part 1

Equilibrium computation: Part 1 Nicola Gatti 1 Troels Bjerre Sorensen 2 1 Politecnico di Milano, Italy 2 Duke University, USA Nicola Gatti and Troels Bjerre Sørensen ( Politecnico di Milano, Italy, Equilibrium