# COMP-424: Artificial intelligence. Lecture 7: Game Playing

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 COMP Artificial Intelligence Lecture 7: Game Playing Instructor: Joelle Pineau Class web page: Unless otherwise noted, all material posted for this course are copyright of the instructor, and cannot be reused or reposted without the i nstructor s written permission. Quick recap Standard assumptions (except for the last lecture) Discrete (vs continuous) state space. Deterministic (vs stochastic) environment. Observable (vs unobservable) environment. Static (vs changing) environment. A single AI agent. 2 1

2 Overview of today s lecture Adversarial search => games with 2 players Minimax search Evaluation functions Alpha-beta pruning State-of-the-art game playing programs 3 Game playing One of the oldest, most well-studied domains in AI! Why? People like them! And are good at playing them. Many games are hard: State spaces are very large and complicated. E.g. Chess has branching factor of 35, games go to 50 moves per player, so nodes in search tree! Sometimes there is stochasticity and imperfect information. Real-time constraints (e.g. fixed amount of time between moves). Clear, clean description of the environment. Easy performance indicator. Winning is everything! 4 2

3 Types of games Perfect vs Imperfect information Perfect: See the exact state of the game Imperfect: Information is hidden 5 Types of games Perfect vs Imperfect information Perfect: See the exact state of the game Imperfect: Information is hidden Deterministic vs Stochastic Deterministic: Change in state is fully determined by player move. Stochastic: Change in state is partially determined by chance. 6 3

4 What type of game? Yes Perfect information? No Deterministic Checkers Othello Chess Go Mastermind Stochastic Backgammon Scrabble Bridge Poker Most card games 7 Game playing as search Consider 2-player, perfect information, deterministic games. Can we formulate them as a search problem? State: Operators: Goal: Cost: We want to find a strategy (i.e. way of picking moves) that maximizes utility (i.e. max prob. of win or min cost). 8 4

5 Game playing as search Consider 2-player, perfect information, deterministic games. Can we formulate them as a search problem? State: State of the board, player turn Operators: Legal moves Goal: States in which the game is won/lost/drawn Cost: We want to find a strategy (i.e. way of picking moves) that maximizes utility (i.e. max prob. of win or min cost). 9 Game playing as search Consider 2-player, perfect information, deterministic games. Can we formulate them as a search problem? State: State of the board, player turn Operators: Legal moves Goal: States in which the game is won/lost/drawn Cost: Simple case: +1 for winning, -1 for loosing, 0 for a draw. More complex cases: points won, money, We want to find a strategy (i.e. way of picking moves) that maximizes utility (i.e. max prob. of win or min cost). 10 5

6 Game search challenge Not quite the same as simple searching. There is an opponent! The opponent is malicious! Opponent is trying to make things good for itself, and bad for us. We have to simulate the opponent s decisions. Key idea: Define a max player (who wants to maximize its utility) And a min player (who wants to minimize it.) 11 Example: Game Tree for Tic-Tac-Toe 12 6

7 Minimax search Expand complete search tree, until terminal states have been reached and their utilities computed. Go back up from leaves towards the current state of the game. At each min node: backup the worst value among the children. At each max node: backup the best value among the children. 13 Minimax search Expand complete search tree, until terminal states have been reached and their utilities computed. Go back up from leaves towards the current state of the game. At each min node: backup the worst value among the children. At each max node: backup the best value among the children. 14 7

8 Minimax Algorithm Operator MinimaxDecision() For each legal operator o: Apply the operator o and obtain the new game state s. Value[o] = MinimaxValue(s) Return the operator with the highest value Value[o]. Double MinimaxValue(s) If isterminal(s), return Utility(s). For each state s in Successors(s) Let Value(s ) = MinimaxValue(s ). If Max is to move in s, return max s Value(s ). If Min is to move in s, return min s Value(s ). 15 Properties of Minimax search Complete? Optimal? Time complexity? Space complexity? 16 8

9 Properties of Minimax search Complete? If the game tree is finite. Optimal? Against an optimal opponent (otherwise don t know.) Time complexity? O(b m ) Space complexity? O(bm) if we use DFS Why not use Minimax to solve chess? In chess: b 35, m 100, so an exact solution is impossible! 17 Coping with resource limitations Suppose we have 100 seconds to make a move, and we can search 10 4 nodes per second. Can only search 10 6 nodes. (Or even fewer, if we spend time deciding which nodes to search.) Possible approach: Use a cutoff test (e.g. based on depth limit) Use an evaluation function for the nodes where we cutoff the search. Need to think about real-time search. 18 9

10 Evaluation functions An evaluation function v(s) represents the goodness of a board state (e.g. chance of winning from that position). If the features of the board can be evaluated independently, use a weighted linear function: v(s) = w 1 f 1 (s) + w 2 f 2 (s) + + w n f n (s) (where s is board state) This function can be given by the designer or learned from experience. 19 Example: Chess Black to move White slightly better White to move Black winning Linear evaluation function: v(s) = w 1 f 1 (s) + w 2 f 2 (s) w 1 = 9 f 1 (s) = (# white queens) - (# black queens) w 2 = 3 f 2 (s) = (# white pawns) - (# black pawns) 20 10

11 How precise should the evaluation fn be? Evaluation function is only approximate, and is usually better if we are close to the end of the game. Move chosen is the same if we apply a monotonic transformation to the evaluation function. 21 How precise should the evaluation fn be? Evaluation function is only approximate, and is usually better if we are close to the end of the game. Move chosen is the same if we apply a monotonic transformation to the evaluation function. Only the order of the numbers matter: payoffs in deterministic games act as an ordinal utility function

12 Minimax with an evaluation function Use evaluation function to evaluate non-terminal nodes. Helps us make a decision without searching until the end of the game. Minimax cutoff algorithm: Same as standard Minimax, except stop at some maximum depth m and use the evaluation function on those nodes. 23 Minimax cutoff in Chess How many moves ahead can we search in Chess? >> 10 6 nodes with b=35 allows us to search 4 moves ahead! Is this useful? 4-moves ahead novice player 8-moves ahead human master, typical PC 12-moves ahead Deep Blue, Kasparov Key idea: Search few lines of play, but search them deeply.need pruning! 24 12

13 α-β Pruning example α-β Pruning example

14 α-β Pruning example X X 27 α-β Pruning example X X 28 14

15 α-β Pruning example X X 29 α-β Pruning example X X

16 α-β Pruning Simple idea: if a path looks worse than what we already have, discard it. If the best move at a node cannot change (regardless of what we would find by searching) then no need to search further! Algorithm is like Minimax, but keeps track of best leaf value for our player (α) and best one for the opponent (β). Standard technique for deterministic, perfect information games. 31 α-β algorithm Instead of MinimaxValue, we have two functions. Double MaxValue(s,α,β) If cutoff(s), return Evaluation(s). For each state s in Successors(s) Let α = max { α, MinValues(s,α,β) }. If α β, return β. Return α. Double MinValue(s,α,β) If cutoff(s), return Evaluation(s). For each state s in Successors(s) Let β = min { β, MinValues(s,α,β) }. If α β, return α. Return β

17 Example 33 Example 34 17

18 Example 35 Example 36 18

19 Example 37 Example 38 19

20 Example 39 Example 40 20

21 Example 41 Example 42 21

22 Example 43 Important lessons of α-β pruning Pruning can greatly increase efficiency!! Pruning does not affect the final result. Best moves are same as returned by Minimax (assuming the opponent is optimal and the evaluation function is correct.) 44 22

23 Important lessons of α-β pruning Pruning can greatly increase efficiency!! Pruning does not affect the final result. Best moves are same as returned by Minimax (assuming the opponent is optimal and the evaluation function is correct.) Order matters for search efficiency! 45 α-β Pruning example X X On the middle branch, nodes were ordered well and we pruned a lot. On the right branch, the order was bad and there was no pruning

24 Important lessons of α-β pruning Pruning can greatly increase efficiency!! Pruning does not affect the final result. Best moves are same as returned by Minimax (assuming the opponent is optimal and the evaluation function is correct.) Order matters for search efficiency! The α-β pruning demonstrates the value of reasoning about which computations are important! 47 Efficiency of α-β pruning With bad move ordering, time complexity is O(b m ) Means nothing was pruned. With perfect ordering, time complexity is O(b m/2 ) Means double the search depth, for same resources. In chess: this is difference between novice and expert player. On average, O(b 3m/4 ), if we expect to find the max/min after b/2 expansions. Randomizing the move ordering can achieve the average case. Evaluation function can be used to order the nodes

25 Forward pruning Simple idea (for domains with large branching factor): Only explore n best moves for current state (according to the evaluation function). Unlike α-β pruning, this can lead to sub-optimal solution. Can be very efficient with a good evaluation function. 49 Drawbacks of α-β If the branching factor is really big, search depth is still too limited. E.g. in Go, where branching factor b~300. More on this next time! Optimal play is guaranteed against an optimal opponent if search proceeds to the end of the game. But the opponent may not be optimal! If heuristics are used, this assumption turns into the opponent playing optimally according to the same heuristic function as the player. This is a very big assumption! What to do if the opponent plays very differently? 50 25

26 Chinook (Schaeffer et al., U. of Alberta) Best checkers player (since 1990 s). Plain α-β search, performed on standard PCs. Evaluation function based on expert features of the board. Opening database. HUGE endgame database! Chinook has perfect information for all checkers positions involving 8 or fewer pieces on the board (a total of 443,748,401,247 positions). Only a few moves in middle of the game are actually searched. They have now done an exhaustive search for checkers, and through optimal play can force at least a draw. 51 Deep Blue (IBM) Specialized chess processor, special-purpose memory architecture. Very sophisticated evaluation function (expert features, tuned weights). Database of standard openings/closings. Uses a version of α-β pruning (with undisclosed improvements) Can search up to 40-deep in some branches. Can search over 200 billion positions per second! Overall, an impressive engineering feat. Now, several computer programs running on regular hardware are on par with human champions (e.g. Fritz)

27 53 Some possible strategies for AI in games Design a compact state representation. Search using iterative deepening to get anytime real-time play. Use alpha-beta pruning with an evaluation function. Order moves using the evaluation function. Tune the evaluation function using domain knowledge, trial-and-error, learning. Searching deeper is often more important than having a good evaluation function. Consider using different strategies for opening, middle and endgame. Use randomization to break ties. Consider that the opponent might not be optimal. It is crucial to decide where to spend the computation effort! 54 27

28 Summary Understand the different types of games. Understand Minimax and alpha-beta pruning, what they compute, how to implement, and common methods to make them work (e.g. node ordering). Understand the role of the evaluation function. Be familiar with state-of-the-art for solving some common games

Artificial Intelligence Adversarial Search Andrea Torsello Games Vs. Planning Problems In many problems you re are pitted against an opponent certain operators are beyond your control: you cannot control

### Playing and Solving Games

Playing and Solving Games Zero-sum games with perfect information R&N 6 Definitions Game evaluation Optimal solutions Minimax Alpha-beta pruning Approximations Heuristic evaluation functions Cutoffs Endgames

### Introduction to Artificial Intelligence. Game Playing

Introduction to Artificial Intelligence Game Playing Bernhard Beckert UNIVERSITÄT KOBLENZ-LANDAU Summer Term 2003 B. Beckert: Einführung in die KI / KI für IM p.1 Outline Perfect play Resource limits α-β

### Administrativia. Assignment 1 due tuesday 9/24/2002 BEFORE midnight. Midterm exam 10/10/2002. CS 561, Sessions 8-9 1

Administrativia Assignment 1 due tuesday 9/24/2002 BEFORE midnight Midterm exam 10/10/2002 CS 561, Sessions 8-9 1 Last time: search strategies Uninformed: Use only information available in the problem

### Game playing. Chapter 6. Chapter 6 1

Game playing Chapter 6 Chapter 6 1 Outline Games Perfect play minimax decisions α β pruning Resource limits and approximate evaluation Games of chance Games of imperfect information Chapter 6 2 Games vs.

### AI and computer game playing. Artificial Intelligence. Partial game tree for Tic-Tac-Toe. Game tree search. Minimax Algorithm

AI and computer game playing Artificial Intelligence Introduction to game-playing search imax algorithm, evaluation functions, alpha-beta pruning, advanced techniques Game playing (especially chess and

### Game-Playing & Adversarial Search MiniMax, Search Cut-off, Heuristic Evaluation

Game-Playing & Adversarial Search MiniMax, Search Cut-off, Heuristic Evaluation This lecture topic: Game-Playing & Adversarial Search (MiniMax, Search Cut-off, Heuristic Evaluation) Read Chapter 5.1-5.2,

### Adversarial Search and Game- Playing. C H A P T E R 5 C M P T 3 1 0 : S u m m e r 2 0 1 1 O l i v e r S c h u l t e

Adversarial Search and Game- Playing C H A P T E R 5 C M P T 3 1 0 : S u m m e r 2 0 1 1 O l i v e r S c h u l t e Environment Type Discussed In this Lecture 2 Fully Observable yes Turn-taking: Semi-dynamic

### ! HW5 Due Monday 11:59 pm. ! no randomness, e.g., dice. ! Perfect information. ! No hidden cards or hidden Chess pieces. ! Finite

!! Game Trees and Heuristics Announcements! HW5 Due Monday 11:59 pm 15-211: Fundamental Data Structures and Algorithms Margaret Reid-Miller 7 April 2005 2 X O X O X O X O O Games Game Properties!!Two!

### Artificial Intelligence in Chess

Artificial Intelligence in Chess IA in Chess is quite complex Even the fastest computer cannot solve the chess game (cannot analyze every possible situation) There are many ways of reducing the need of

### Chess Algorithms Theory and Practice. Rune Djurhuus Chess Grandmaster runed@ifi.uio.no / runedj@microsoft.com October 3, 2012

Chess Algorithms Theory and Practice Rune Djurhuus Chess Grandmaster runed@ifi.uio.no / runedj@microsoft.com October 3, 2012 1 Content Complexity of a chess game History of computer chess Search trees

### CS182 Lecture 5: Games and Adversarial Search

CS182 Lecture 5: Games and Adversarial Search Agenda EOLQ: uninformed and informed search Games The difference an adversary makes Minimax Alpha-beta pruning Static evaluation functions (for depth-limited

### 13. *-Minimax. Introduction. Stochastic Games. Deep Search!? Perfect information deterministic games? Imperfect information games?

13. *-Minimax Introduction Perfect information deterministic games? Lots of success Imperfect information games? Hard because of missing information Subject of active research Bridge, poker Non-deterministic/stochastic

### Games Computers Play. Bart Massey Computer Science Dept. Portland State University

Games Computers Play Bart Massey Computer Science Dept. Portland State University bart@cs.pdx.edu http://www.cs.pdx.edu/~bart 35 Years Ago... Computer world chess champion: "soon" Widespread agreement

### Chess Algorithms Theory and Practice. Rune Djurhuus Chess Grandmaster runed@ifi.uio.no / runedj@microsoft.com September 23, 2014

Chess Algorithms Theory and Practice Rune Djurhuus Chess Grandmaster runed@ifi.uio.no / runedj@microsoft.com September 23, 2014 1 Content Complexity of a chess game Solving chess, is it a myth? History

### Games Computers Play. Bart Massey Computer Science Dept. Portland State University

Games Computers Play Bart Massey Computer Science Dept. Portland State University bart@cs.pdx.edu http://www.cs.pdx.edu/~bart 35 Years Ago... Computer world chess champion: "soon" Widespread agreement

### Tactical Analysis Board Games

DM842 Computer Game Programming: AI Lecture 9 Tactical Analysis Board Games Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark Outline 1. Tactical Pathfinding

### Game Playing in the Real World. Next time: Knowledge Representation Reading: Chapter 7.1-7.3

Game Playing in the Real World Next time: Knowledge Representation Reading: Chapter 7.1-7.3 1 What matters? Speed? Knowledge? Intelligence? (And what counts as intelligence?) Human vs. machine characteristics

### FUNDAMENTALS OF ARTIFICIAL INTELLIGENCE USING HEURISTICS IN GAMES

Riga Technical University Faculty of Computer Science and Information Technology Department of Systems Theory and Design FUNDAMENTALS OF ARTIFICIAL INTELLIGENCE Lecture 6 USING HEURISTICS IN GAMES Dr.habil.sc.ing.,

### Computer Simulation (2) Game Playing

Computer Simulation (2) Game Playing Different Types of Games How many players? One Two Element of chance? Deterministic Nondeterministic Outcome Zero sum (what one player wins, the other loses) Non-zero

### State Space Representation and Search

1. Introduction In this section we examine the concept of a state space and the different searches that can be used to explore the search space in order to find a solution. Before an AI problem can be

### Lecture 2. Uninformed search

Lecture 2 Uninformed search " Reference: Preparing for week 1 Reading: Chapters 1 and 2.1, 2.2, 2.5, 3.1, 3.2, 3.3, 3.4, 3.5 Assignment 1 has now been posted on the course LEARN site Uses MATLAB (a tutorial

### CS188 Spring 2011 Section 3: Game Trees

CS188 Spring 2011 Section 3: Game Trees 1 Warm-Up: Column-Row You have a 3x3 matrix of values like the one below. In a somewhat boring game, player A first selects a row, and then player B selects a column.

### Solutions. 1) Is the following Boolean function linearly separable? Explain.

CS/ES 480, CES 510 Mid-term # 1 Practice Problems Solutions 1) Is the following Boolean function linearly separable? Explain. f ( x, y,z ) ( x y z ) ( x y z ) ( x y z ) Solution: Geometric representation

### 10. Machine Learning in Games

Machine Learning and Data Mining 10. Machine Learning in Games Luc De Raedt Thanks to Johannes Fuernkranz for his slides Contents Game playing What can machine learning do? What is (still) hard? Various

### Introduction to Artificial Intelligence (G51IAI) Dr Rong Qu. Blind Searches - Introduction

Introduction to Artificial Intelligence (G51IAI) Dr Rong u Blind Searches - Introduction Aim of This Section (2 hours) Introduce the blind searches on search tree Specifically, the general search pseudo-code

### Learning to play chess using TD(λ)-learning with database games

Learning to play chess using TD(λ)-learning with database games Henk Mannen & Marco Wiering Cognitive Artificial Intelligence University of Utrecht henk.mannen@phil.uu.nl & marco@cs.uu.nl Abstract In this

### Laboratory work in AI: First steps in Poker Playing Agents and Opponent Modeling

Laboratory work in AI: First steps in Poker Playing Agents and Opponent Modeling Avram Golbert 01574669 agolbert@gmail.com Abstract: While Artificial Intelligence research has shown great success in deterministic

### CPSC 322 Midterm Practice Questions

CPSC 322 Midterm Practice Questions The midterm will contain some questions similar to the ones in the list below. Some of these questions may actually come verbatim from the list below. Unless otherwise

### Implementation of a Chess Playing Artificial. Intelligence. Sean Stanek.

Implementation of a Chess Playing Artificial Intelligence Sean Stanek vulture@cs.iastate.edu Overview Chess has always been a hobby of mine, and so have computers. In general, I like creating computer

### Applications of Artificial Intelligence and Machine Learning in Othello

Applications of Artificial Intelligence and Machine Learning in Othello Jack Chen tjhsst Computer Systems Lab 2009 2010 Abstract This project explores Artificial Intelligence techniques in the board game

### 3.6.2 Generating admissible heuristics from relaxed problems

82 Heuristic h 2 gives a stricter estimate than h 1 : h 2 (n) h 1 (n) We say that the former dominates the latter A* expands every node for which f(n) < C* h(n) < C* g(n) Hence, a stricter heuristic estimate

### Techniques to Parallelize Chess Matthew Guidry, Charles McClendon

Techniques to Parallelize Chess Matthew Guidry, Charles McClendon Abstract Computer chess has, over the years, served as a metric for showing the progress of computer science. It was a momentous occasion

### Module 2. Problem Solving using Search- (Single agent search) Version 1 CSE IIT, Kharagpur

Module 2 Problem Solving using Search- (Single agent search) Lesson 6 Informed Search Strategies-II 3.3 Iterative-Deepening A* 3.3.1 IDA* Algorithm Iterative deepening A* or IDA* is similar to iterative-deepening

### Introduction to AI Techniques

Introduction to AI Techniques Game Search, Minimax, and Alpha Beta Pruning June 8, 2009 Introduction One of the biggest areas of research in modern Artificial Intelligence is in making computer players

### LECTURE 10: GAMES IN EXTENSIVE FORM

LECTURE 10: GAMES IN EXTENSIVE FORM Sequential Move Games 2 so far, we have only dealt with simultaneous games (players make the decisions at the same time, or simply without knowing what the action of

### 3D Tic Tac Toe Adina Rubinoff CSC 242 Febuary

3D Tic Tac Toe Adina Rubinoff CSC 242 Febuary 7 2010 The goal of this project is to provide a Java program that plays a game of 3D 4x4 Tic Tac Toe. The program uses a minimax algorithm with optional alpha-beta

### Implementing a Computer Player for Abalone using Alpha-Beta and Monte-Carlo Search

Master Thesis Implementing a Computer Player for Abalone using Alpha-Beta and Monte-Carlo Search Pascal Chorus Master Thesis DKE 09-13 Thesis submitted in partial fulfillment of the requirements for the

### Roulette Wheel Selection Game Player

Macalester College DigitalCommons@Macalester College Honors Projects Mathematics, Statistics, and Computer Science 5-1-2013 Roulette Wheel Selection Game Player Scott Tong Macalester College, stong101@gmail.com

### Hierarchical Heuristic Search Techniques for Empire- Based Games

Hierarchical Heuristic Search Techniques for Empire- Based Games Kenrick Mock University of Alaska Anchorage Dept. of Mathematical Sciences, Computer Science 3211 Providence Dr. Anchorage, AK 99508 kenrick@uaa.alaska.edu

### Artificial Intelligence Lecture 2

Artificial Intelligence Lecture 2 Representation From AI admirers to AI programmers. Step 1: Represent the problem so that it is computerfriendly. Step 2: Code the problem in a programming language. Step

### (Refer Slide Time: 01:35)

Artificial Intelligence Prof. Sudeshna Sarkar Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur Lecture - 6 Informed Search -2 Today we start our 6 th lecture. This

### Artificial Intelligence Search I

Content: Search I Artificial Intelligence Search I Lecture 3 Introduction Knowledge, Problem Types and Problem Formulation Search Strategies General Search Problem 4 Criteria for Evaluating Search Strategies

### Principles of AI Planning

Principles of Dr. Jussi Rintanen Albert-Ludwigs-Universität Freiburg Summer term 2005 Course: Principles of Lecturer Dr Jussi Rintanen (rintanen@informatik.uni-freiburg.de) Lecture Monday 2-4pm, Wednesday

### Measuring the Performance of an Agent

25 Measuring the Performance of an Agent The rational agent that we are aiming at should be successful in the task it is performing To assess the success we need to have a performance measure What is rational

### Informed search algorithms

CmpE 540 Principles of Artificial Intelligence Informed search algorithms Pınar Yolum pinar.yolum@boun.edu.tr Department of Computer Engineering Boğaziçi University Chapter 4 (Sections 1 3) (Based mostly

### Script for How to Win a Chinese Chess Game

Script for How to Win a Chinese Chess Game Jinzhong Niu April 25, 2004 1 Presentation Set Up Chinese chess is played on a 10 9 board, on which either player has 16 pieces. Pieces are located at the intersections

### Module 2. Problem Solving using Search- (Single agent search) Version 2 CSE IIT, Kharagpur

Module 2 Problem Solving using Search- (Single agent search) 2.1 Instructional Objective The students should understand the state space representation, and gain familiarity with some common problems formulated

### Uninformed Search. Chapter 3. (Based on slides by Stuart Russell, Dan Weld, Oren Etzioni, Henry Kautz, and other UW-AI faculty)

Uninformed Search Chapter 3 (Based on slides by Stuart Russell, Dan Weld, Oren Etzioni, Henry Kautz, and other UW-AI faculty) What is Search? Search is a class of techniques for systematically finding

### Acknowledgements I would like to thank both Dr. Carsten Furhmann, and Dr. Daniel Richardson for providing me with wonderful supervision throughout my

Abstract Game orientated application programming interfaces (API), built using Java, are intended to ease the development of Java games, on multiple platforms. Existing Java game APIs, tend to focus on

### CS91.543 MidTerm Exam 4/1/2004 Name: KEY. Page Max Score 1 18 2 11 3 30 4 15 5 45 6 20 Total 139

CS91.543 MidTerm Exam 4/1/2004 Name: KEY Page Max Score 1 18 2 11 3 30 4 15 5 45 6 20 Total 139 % INTRODUCTION, AI HISTORY AND AGENTS 1. [4 pts. ea.] Briefly describe the following important AI programs.

### Comparison of TDLeaf(λ) and TD(λ) Learning in Game Playing Domain

Comparison of TDLeaf(λ) and TD(λ) Learning in Game Playing Domain Daniel Osman and Jacek Mańdziuk Faculty of Mathematics and Information Science, Warsaw University of Technology, Plac Politechniki 1, -1

### ITEC2620 Introduction to Data Structures

ITEC2620 Introduction to Data Structures Lecture 8b Search Trees Traversals Typical covered as graph traversals in data structures courses I ve never seen a graph traversal in practice! AI background Tree

### IMPLEMENTING A COMPUTER PLAYER FOR CARCASSONNE. Cathleen Heyden

IMPLEMENTING A COMPUTER PLAYER FOR CARCASSONNE Master Thesis DKE 09-15 Thesis submitted in partial fulfillment of the requirements for the degree of Master of Science of Artificial Intelligence at the

### Monte-Carlo Methods. Timo Nolle

Monte-Carlo Methods Timo Nolle Outline Minimax Expected Outcome Monte-Carlo Monte-Carlo Simulation Monte-Carlo Tree Search UCT AMAF RAVE MC-RAVE UCT-RAVE Playing Games Go, Bridge, Scrabble Problems with

### Evolutionary Computation: introduction

Evolutionary Computation: introduction Dirk Thierens Universiteit Utrecht The Netherlands Dirk Thierens (Universiteit Utrecht) EC Introduction 1 / 44 What? Evolutionary Computation Evolutionary Computation

### An Analysis of UCT in Multi-Player Games

An Analysis of UCT in Multi-Player Games Nathan R. Sturtevant Department of Computing Science, University of Alberta, Edmonton, AB, Canada, T6G 2E8 nathanst@cs.ualberta.ca Abstract. The UCT algorithm has

### Algorithmic Game Theory and Applications. Lecture 1: What is game theory?

Algorithmic Game Theory and Applications Lecture : What is game theory? Kousha Etessami Basic course information Lecturer: Kousha Etessami; Office: IF 5.2; Phone: 65 597. Lecture times: Monday&Thursday,

### Knowledge engineering in chess programming- Evaluation of a chess position-criteria and procedure

Knowledge engineering in chess programming- Evaluation of a chess position-criteria and procedure malugu_satyajit@yahoo.com vidyuth_koniki@yahoo.co.in Authors: M.Satyajit ¾ Cse-A Gitam K.B.N.Vidyuth ¾

### (Refer Slide Time: 01:04)

Artificial Intelligence Prof. Sudeshna Sarkar Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur Lecture - 3 State Space Search Today we will start the third lecture

### Heuristic (Informed) Search (Where we try to choose smartly)

Heuristic (Informed) Search (Where we try to choose smartly) R&: Chap., Sect.. Recall that the ordering of FRIGE defines the search strategy Search Algorithm # SEARCH#. ISERT(initial-node,FRIGE). Repeat:

### CSC384h: Intro to Artificial Intelligence

Instructor Hojjat Ghaderi CSC384h: Intro to Artificial Intelligence hojjat@cdf.toronto.edu, SF3209 Office Hours: SF3207 Wed/Fri 11-12:30. Lectures: Monday, Wednesday, & Friday 10-11 SS2127. Tutorials Usually

### FIRST EXPERIMENTAL RESULTS OF PROBCUT APPLIED TO CHESS

FIRST EXPERIMENTAL RESULTS OF PROBCUT APPLIED TO CHESS A.X. Jiang Department of Computer Science, University of British Columbia, Vancouver, Canada albertjiang@yahoo.com M. Buro Department of Computing

### AI Principles, Semester 2, Week 1, Lecture 3, Search

AI Principles, Semester 2, Week 1, Lecture 3, Search Examples of problems which are solved by search Setting the scene: your Romanian holiday search problem How to be systematic in your search Uninformed

### Search methods motivation 1

Suppose you are an independent software developer, and your software package Windows Defeater R, widely available on sourceforge under a GNU GPL license, is getting an international attention and acclaim.

### Mastering Quoridor. Lisa Glendenning THESIS. Submitted in Partial Fulfillment of the Requirements for the Degree of

Mastering Quoridor by Lisa Glendenning THESIS Submitted in Partial Fulfillment of the Requirements for the Degree of Bachelor of Science Computer Science The University of New Mexico Albuquerque, New Mexico

### A problem consists of. Artificial Intelligence. Example: 8-puzzle. Example: 8-puzzle. Example: 8-puzzle. Expanding 8-puzzle

A problem consists of Artificial Intelligence Uninformed search Chapter 3, AIMA An initial state, θ(0) A list of possible actions, α, for the agent A goal test (there can be many goal states) A path cost

### CMPT 301 Artificial Intelligence Fall 2014 Homework #4

CMPT 301 Artificial Intelligence Fall 2014 Homework #4 75 Total Points Due Date: No Later than 5:30 PM on Monday, October 6, 2014. You may either hand-in this homework written or submit it to the drop

### 10/13/11 Solution: Minimax with Alpha-Beta Pruning and Progressive Deepening

10/1/11 Solution: Minimax with Alpha-Beta Pruning and Progressive Deepening When answering the question in Parts C.1 and C. below, assume you have already applied minimax with alpha-beta pruning and progressive

### Course: CS40002 Instructor: Dr. Pallab Dasgupta Department of Computer Science & Engineering Indian Institute of Technology Kharagpur

Problem Solving by Search Course: CS40002 Instructor: Dr. Pallab Dasgupta Department of Computer Science & Engineering Indian Institute of Technology Kharagpur Search Frameworks State space search Uninformed

### Problems, Problem Spaces and Search

Problems, Problem Spaces and Search Contents Defining the problem as a State Space Search Production Systems Control Strategies Breadth First Search Depth First Search Heuristic Search Problem Characteristics

### CS181 Lecture 5 Reinforcement Learning

CS181 Lecture 5 Reinforcement Learning The topic for today is reinforcement learning. We will look at several methods, and then discuss two important issues: the exploration-exploitation tradeoff and the

### CS 151: Intelligent Agents, Problem Formulation and Search

CS 151: Intelligent Agents, Problem Formulation and Search How do we make a vacuum "smart?" Vacuum, clean the house! Um OK?? This one's got no chance How do we represent this task? Hmmm where to begin?

### Created by Peter McOwan and Paul Curzon of Queen Mary, University of London with support

Winning games: the perfect tictac-toe player Created by Peter McOwan and Paul Curzon of Queen Mary, University of London with support from EPSRC and Google Winning games: the perfect tic-tac-toe player

### The exam is closed book, closed calculator, and closed notes except your one-page crib sheet.

CS 188 Fall 2015 Introduction to rtificial Intelligence Midterm You have approximately 80 minutes. The exam is closed book, closed calculator, and closed notes except your one-page crib sheet. Mark your

### Last time: search strategies

Last time: search strategies Uninfrmed: Use nly infrmatin available in the prblem frmulatin Breadth-first Unifrm-cst Depth-first Depth-limited Iterative deepening Infrmed: Use heuristics t guide the search

### Monte Carlo Tree Search and Opponent Modeling through Player Clustering in no-limit Texas Hold em Poker

Monte Carlo Tree Search and Opponent Modeling through Player Clustering in no-limit Texas Hold em Poker A.A.J. van der Kleij August 2010 Master thesis Artificial Intelligence University of Groningen, The

### TIC TAC TOE GAME BETWEEN COMPUTERS: A COMPUTATIONAL INTELIGENCE APPROACH

TIC TAC TOE GAME BETWEEN COMPUTERS: A COMPUTATIONAL INTELIGENCE APPROACH Ashutosh Kumar Sahu, Parthasarathi Palita*, Anupam Mohanty asu.ku.sahu@gmail.com parthapalita107@gmail.com anupam.iter@gmail.com

### Created by Peter McOwan and Paul Curzon of Queen Mary, University of London with support

The sweet computer: machines that learn Created by Peter McOwan and Paul Curzon of Queen Mary, University of London with support from EPSRC and Google The sweet computer: machines that learn Age group:

### BEYOND CLASSICAL SEARCH: SEARCHING WITH NON-DETERMINISTIC ACTIONS SEARCHING WITH PARTIAL OBSERVATIONS ONLINE SEARCH & UNKNOWN ENVIRONMENTS

BEYOND CLASSICAL SEARCH: SEARCHING WITH NON-DETERMINISTIC ACTIONS SEARCHING WITH PARTIAL OBSERVATIONS ONLINE SEARCH & UNKNOWN ENVIRONMENTS Chapter 4, Sections 4.3-4.5 Machines will be capable, within twenty

### Creating a NL Texas Hold em Bot

Creating a NL Texas Hold em Bot Introduction Poker is an easy game to learn by very tough to master. One of the things that is hard to do is controlling emotions. Due to frustration, many have made the

### Midterm Examination CS540: Introduction to Artificial Intelligence

Midterm Examination CS540: Introduction to Artificial Intelligence October 27, 2010 LAST NAME: SOLUTION FIRST NAME: SECTION (1=Zhu, 2=Dyer): Problem Score Max Score 1 15 2 16 3 7 4 15 5 12 6 10 7 15 8

### Problem Solving and Search 2. Problem Solving and Search. Let s have Holiday in Romania. What is the Problem? Is there more?

Let s have Holiday in Romania On holiday in Romania; currently in Arad. 2 Problem Solving and Search Flight leaves tomorrow from Bucharest at 1:00. Let s configure this to be an AI problem. 1 2 What is

### BEST-REPLY SEARCH IN MULTI-PLAYER CHESS. Markus Esser. Master Thesis 12-13

BEST-REPLY SEARCH IN MULTI-PLAYER CHESS Markus Esser Master Thesis 12-13 Thesis submitted in partial fulfilment of the requirements for the degree of Master of Science of Artificial Intelligence at the

### Learning: An Effective Approach in Endgame Chess Board Evaluation

Sixth International Conference on Machine Learning and Applications Learning: An Effective Approach in Endgame Chess Board Evaluation Mehdi Samadi, Zohreh Azimifar, Mansour Zolghadri Jahromi Department

### AI: A Modern Approach, Chpts. 3-4 Russell and Norvig

AI: A Modern Approach, Chpts. 3-4 Russell and Norvig Sequential Decision Making in Robotics CS 599 Geoffrey Hollinger and Gaurav Sukhatme (Some slide content from Stuart Russell and HweeTou Ng) Spring,

### Understanding Proactive vs. Reactive Methods for Fighting Spam. June 2003

Understanding Proactive vs. Reactive Methods for Fighting Spam June 2003 Introduction Intent-Based Filtering represents a true technological breakthrough in the proper identification of unwanted junk email,

### 6.231 Dynamic Programming and Stochastic Control Fall 2008

MIT OpenCourseWare http://ocw.mit.edu 6.231 Dynamic Programming and Stochastic Control Fall 2008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms. 6.231

### Heuristics and A* Search

1 Hal Daumé III (me@hal3.name) Heuristics and A* Search Hal Daumé III Computer Science University of Maryland me@hal3.name CS 421: Introduction to Artificial Intelligence 2 Feb 2012 Many slides courtesy

### CSPs: Arc Consistency

CSPs: Arc Consistency CPSC 322 CSPs 3 Textbook 4.5 CSPs: Arc Consistency CPSC 322 CSPs 3, Slide 1 Lecture Overview 1 Recap 2 Consistency 3 Arc Consistency CSPs: Arc Consistency CPSC 322 CSPs 3, Slide 2

### Using Probabilistic Knowledge and Simulation to Play Poker

Using Probabilistic Knowledge and Simulation to Play Poker Darse Billings, Lourdes Peña, Jonathan Schaeffer, Duane Szafron Department of Computing Science, University of Alberta Edmonton, Alberta Canada

### COMP 551 Applied Machine Learning Lecture 6: Performance evaluation. Model assessment and selection.

COMP 551 Applied Machine Learning Lecture 6: Performance evaluation. Model assessment and selection. Instructor: (jpineau@cs.mcgill.ca) Class web page: www.cs.mcgill.ca/~jpineau/comp551 Unless otherwise

### 573 Intelligent Systems

Sri Lanka Institute of Information Technology 3ed July, 2005 Course Outline Course Information Course Outline Reading Material Course Setup Intelligent agents. Agent types, agent environments and characteristics.

### Learning. Artificial Intelligence. Learning. Types of Learning. Inductive Learning Method. Inductive Learning. Learning.

Learning Learning is essential for unknown environments, i.e., when designer lacks omniscience Artificial Intelligence Learning Chapter 8 Learning is useful as a system construction method, i.e., expose

### Artificial Intelligence

Introduction to Artificial Intelligence Lecture 1 Introduction CS/CNS/EE 154 Andreas Krause 2 What is AI? The science and engineering of making intelligent machines (McCarthy, 56) What does intelligence

### CSC384 Intro to Artificial Intelligence

CSC384 Intro to Artificial Intelligence Artificial Intelligence A branch of Computer Science. Examines how we can achieve intelligent behaviour through computation. What is intelligence? Are these Intelligent?

### G52APT AI Programming Techniques

G52APT AI Programming Techniques Lecture 8: Depth first search in Prolog Brian Logan School of Computer Science bsl@cs.nott.ac.uk Outline of this lecture recap: definition of a search problem state spaces

### Decision Making under Uncertainty

6.825 Techniques in Artificial Intelligence Decision Making under Uncertainty How to make one decision in the face of uncertainty Lecture 19 1 In the next two lectures, we ll look at the question of how