Last time: search strategies

Last time: search strategies Uninfrmed: Use nly infrmatin available in the prblem frmulatin Breadth-first Unifrm-cst Depth-first Depth-limited Iterative deepening Infrmed: Use heuristics t guide the search Best first: Greedy search queue first ndes that maimize heuristic desirability based n estimated path cst frm current nde t gal; A* search queue first ndes that maimize sum f path cst s far and estimated path cst t gal. Iterative imprvement keep n memry f path; wrk n a single current state and iteratively imprve its value. Hill climbing select as new current state the successr state which maimizes value. Simulated annealing refinement n hill climbing by which bad mves are permitted, but with decreasing size and frequency. Will find glbal etremum. CS 460, Sessins 8-9 1

Eercise: Search Algrithms The fllwing figure shws a prtin f a partially epanded search tree. Each arc between ndes is labeled with the cst f the crrespnding peratr, and the leaves are labeled with the value f the heuristic functin, h. Which nde (use the nde s letter) will be epanded net by each f the fllwing search algrithms? (a) Depth-first search (b) Breadth-first search (c) Unifrm-cst search (d) Greedy search (e) A* search 3 5 19 B h=14 C D 6 h=18 4 5 5 h=15 E F G H A h=10 h=12 h=8 h=10 h=20 CS 460, Sessins 8-9 2

Depth-first search Nde queue: initializatin # state depth path cst parent # 1 A 0 0 -- CS 460, Sessins 8-9 3

Depth-first search Nde queue: add successrs t queue frnt; empty queue frm tp # state depth path cst parent # 2 B 1 3 1 3 C 1 19 1 4 D 1 5 1 1 A 0 0 -- CS 460, Sessins 8-9 4

Depth-first search Nde queue: add successrs t queue frnt; empty queue frm tp # state depth path cst parent # 5 E 2 7 2 6 F 2 8 2 7 G 2 8 2 8 H 2 9 2 2 B 1 3 1 3 C 1 19 1 4 D 1 5 1 1 A 0 0 -- CS 460, Sessins 8-9 5

Depth-first search Nde queue: add successrs t queue frnt; empty queue frm tp # state depth path cst parent # 5 E 2 7 2 6 F 2 8 2 7 G 2 8 2 8 H 2 9 2 2 B 1 3 1 3 C 1 19 1 4 D 1 5 1 1 A 0 0 -- CS 460, Sessins 8-9 6

Breadth-first search Nde queue: initializatin # state depth path cst parent # 1 A 0 0 -- CS 460, Sessins 8-9 8

Breadth-first search Nde queue: add successrs t queue end; empty queue frm tp # state depth path cst parent # 1 A 0 0 -- 2 B 1 3 1 3 C 1 19 1 4 D 1 5 1 CS 460, Sessins 8-9 9

Breadth-first search Nde queue: add successrs t queue end; empty queue frm tp # state depth path cst parent # 1 A 0 0 -- 2 B 1 3 1 3 C 1 19 1 4 D 1 5 1 5 E 2 7 2 6 F 2 8 2 7 G 2 8 2 8 H 2 9 2 CS 460, Sessins 8-9 10

Breadth-first search Nde queue: add successrs t queue end; empty queue frm tp # state depth path cst parent # 1 A 0 0 -- 2 B 1 3 1 3 C 1 19 1 4 D 1 5 1 5 E 2 7 2 6 F 2 8 2 7 G 2 8 2 8 H 2 9 2 CS 460, Sessins 8-9 11

Unifrm-cst search Nde queue: initializatin # state depth path cst parent # 1 A 0 0 -- CS 460, Sessins 8-9 13

Unifrm-cst search Nde queue: add successrs t queue s that entire queue is srted by path cst s far; empty queue frm tp # state depth path cst parent # 1 A 0 0 -- 2 B 1 3 1 3 D 1 5 1 4 C 1 19 1 CS 460, Sessins 8-9 14

Unifrm-cst search Nde queue: add successrs t queue s that entire queue is srted by path cst s far; empty queue frm tp # state depth path cst parent # 1 A 0 0 -- 2 B 1 3 1 3 D 1 5 1 5 E 2 7 2 6 F 2 8 2 7 G 2 8 2 8 H 2 9 2 4 C 1 19 1 CS 460, Sessins 8-9 15

Unifrm-cst search Nde queue: add successrs t queue s that entire queue is srted by path cst s far; empty queue frm tp # state depth path cst parent # 1 A 0 0 -- 2 B 1 3 1 3 D 1 5 1 5 E 2 7 2 6 F 2 8 2 7 G 2 8 2 8 H 2 9 2 4 C 1 19 1 CS 460, Sessins 8-9 16

Greedy search Nde queue: initializatin # state depth path cst ttal parent # cst t gal cst 1 A 0 0 20 20 -- CS 460, Sessins 8-9 18

Greedy search Nde queue: Add successrs t queue, srted by cst t gal. # state depth path cst ttal parent # cst t gal cst 1 A 0 0 20 20 -- 2 B 1 3 14 17 1 3 D 1 5 15 20 1 4 C 1 19 18 37 1 Srt key CS 460, Sessins 8-9 19

Greedy search Nde queue: Add successrs t queue, srted by cst t gal. # state depth path cst ttal parent # cst t gal cst 1 A 0 0 20 20 -- 2 B 1 3 14 17 1 5 G 2 8 8 16 2 7 E 2 7 10 17 2 6 H 2 9 10 19 2 8 F 2 8 12 20 2 3 D 1 5 15 20 1 4 C 1 19 18 37 1 CS 460, Sessins 8-9 20

Greedy search Nde queue: Add successrs t queue, srted by cst t gal. # state depth path cst ttal parent # cst t gal cst 1 A 0 0 20 20 -- 2 B 1 3 14 17 1 5 G 2 8 8 16 2 7 E 2 7 10 17 2 6 H 2 9 10 19 2 8 F 2 8 12 20 2 3 D 1 5 15 20 1 4 C 1 19 18 37 1 CS 460, Sessins 8-9 21

A* search Nde queue: initializatin # state depth path cst ttal parent # cst t gal cst 1 A 0 0 20 20 -- CS 460, Sessins 8-9 23

A* search Nde queue: Add successrs t queue, srted by ttal cst. # state depth path cst ttal parent # cst t gal cst 1 A 0 0 20 20 -- 2 B 1 3 14 17 1 3 D 1 5 15 20 1 4 C 1 19 18 37 1 Srt key CS 460, Sessins 8-9 24

A* search Nde queue: Add successrs t queue frnt, srted by ttal cst. # state depth path cst ttal parent # cst t gal cst 1 A 0 0 20 20 -- 2 B 1 3 14 17 1 5 G 2 8 8 16 2 6 E 2 7 10 17 2 7 H 2 9 10 19 2 3 D 1 5 15 20 1 8 F 2 8 12 20 2 4 C 1 19 18 37 1 CS 460, Sessins 8-9 25

A* search Nde queue: Add successrs t queue frnt, srted by ttal cst. # state depth path cst ttal parent # cst t gal cst 1 A 0 0 20 20 -- 2 B 1 3 14 17 1 5 G 2 8 8 16 2 6 E 2 7 10 17 2 7 H 2 9 10 19 2 3 D 1 5 15 20 1 8 F 2 8 12 20 2 4 C 1 19 18 37 1 CS 460, Sessins 8-9 26

Last time: Simulated annealing algrithm Idea: Escape lcal etrema by allwing bad mves, but gradually decrease their size and frequency. - Nte: gal here is t maimize E. CS 460, Sessins 8-9 28

Last time: Simulated annealing algrithm Idea: Escape lcal etrema by allwing bad mves, but gradually decrease their size and frequency. < - - Algrithm when gal is t minimize E. CS 460, Sessins 8-9 29

This time: Outline Game playing The minima algrithm Resurce limitatins alpha-beta pruning Elements f chance CS 460, Sessins 8-9 30

What kind f games? Abstractin: T describe a game we must capture every relevant aspect f the game. Such as: Chess Tic-tac-te Accessible envirnments: Such games are characterized by perfect infrmatin Search: game-playing then cnsists f a search thrugh pssible game psitins Unpredictable ppnent: intrduces uncertainty thus game-playing must deal with cntingency prblems CS 460, Sessins 8-9 31

Searching fr the net mve Cmpleity: many games have a huge search space Chess: b = 35, m=100 ndes = 35 100 if each nde takes abut 1 ns t eplre then each mve will take abut 10 50 millennia t calculate. Resurce (e.g., time, memry) limit: ptimal slutin nt feasible/pssible, thus must apprimate 1. Pruning: makes the search mre efficient by discarding prtins f the search tree that cannt imprve quality result. 2. Evaluatin functins: heuristics t evaluate utility f a state withut ehaustive search. CS 460, Sessins 8-9 32

Tw-player games A game frmulated as a search prblem: Initial state:? Operatrs:? Terminal state:? Utility functin:? CS 460, Sessins 8-9 33

Tw-player games A game frmulated as a search prblem: Initial state: Operatrs: Terminal state: Utility functin: bard psitin and turn definitin f legal mves cnditins fr when game is ver a numeric value that describes the utcme f the game. E.g., -1, 0, 1 fr lss, draw, win. (AKA payff functin) CS 460, Sessins 8-9 34

Game vs. search prblem CS 460, Sessins 8-9 35

Eample: Tic-Tac-Te CS 460, Sessins 8-9 36

Type f games CS 460, Sessins 8-9 37

Type f games CS 460, Sessins 8-9 38

The minima algrithm Perfect play fr deterministic envirnments with perfect infrmatin Basic idea: chse mve with highest minima value = best achievable payff against best play Algrithm: 1. Generate game tree cmpletely 2. Determine utility f each terminal state 3. Prpagate the utility values upward in the three by applying MIN and MAX peratrs n the ndes in the current level 4. At the rt nde use minima decisin t select the mve with the ma (f the min) utility value Steps 2 and 3 in the algrithm assume that the ppnent will play perfectly. CS 460, Sessins 8-9 39

Generate Game Tree CS 460, Sessins 8-9 40

Generate Game Tree 1 ply 1 mve CS 460, Sessins 8-9 43

CS 460, Sessins 8-9 44 A subtree win lse draw

CS 460, Sessins 8-9 45 What is a gd mve? win lse draw

Minima 3 12 8 2 4 6 14 5 2 Minimize ppnent s chance Maimize yur chance CS 460, Sessins 8-9 46

Minima MIN 3 2 2 3 12 8 2 4 6 14 5 2 Minimize ppnent s chance Maimize yur chance CS 460, Sessins 8-9 47

Minima MAX MIN 3 3 2 2 3 12 8 2 4 6 14 5 2 Minimize ppnent s chance Maimize yur chance CS 460, Sessins 8-9 48

Minima MAX MIN 3 3 2 2 3 12 8 2 4 6 14 5 2 Minimize ppnent s chance Maimize yur chance CS 460, Sessins 8-9 49

minima = maimum f the minimum 1 st ply 2 nd ply CS 460, Sessins 8-9 50

Minima: Recursive implementatin Cmplete:? Optimal:? Time cmpleity:? Space cmpleity:? CS 460, Sessins 8-9 51

Minima: Recursive implementatin Cmplete: Yes, fr finite state-space Optimal: Yes Time cmpleity: O(b m ) Space cmpleity: O(bm) (= DFS Des nt keep all ndes in memry.) CS 460, Sessins 8-9 52

1. Mve evaluatin withut cmplete search Cmplete search is t cmple and impractical Evaluatin functin: evaluates value f state using heuristics and cuts ff search New MINIMAX: CUTOFF-TEST: cutff test t replace the terminatin cnditin (e.g., deadline, depth-limit, etc.) EVAL: evaluatin functin t replace utility functin (e.g., number f chess pieces taken) CS 460, Sessins 8-9 53

Evaluatin functins Weighted linear evaluatin functin: t cmbine n heuristics f = w 1 f 1 + w 2 f 2 + + w n f n E.g, w s culd be the values f pieces (1 fr prawn, 3 fr bishp etc.) f s culd be the number f type f pieces n the bard CS 460, Sessins 8-9 54

Nte: eact values d nt matter CS 460, Sessins 8-9 55

Minima with cutff: viable algrithm? Assume we have 100 secnds, evaluate 10 4 ndes/s; can evaluate 10 6 ndes/mve CS 460, Sessins 8-9 56

2. α-β pruning: search cutff Pruning: eliminating a branch f the search tree frm cnsideratin withut ehaustive eaminatin f each nde α-β pruning: the basic idea is t prune prtins f the search tree that cannt imprve the utility value f the ma r min nde, by just cnsidering the values f ndes seen s far. Des it wrk? Yes, in rughly cuts the branching factr frm b t b resulting in duble as far lk-ahead than pure minima CS 460, Sessins 8-9 57

α-β pruning: eample MAX 6 MIN 6 6 12 8 CS 460, Sessins 8-9 58

α-β pruning: eample MAX 6 MIN 6 2 6 12 8 2 CS 460, Sessins 8-9 59

α-β pruning: eample MAX 6 MIN 6 2 5 6 12 8 2 5 CS 460, Sessins 8-9 60

α-β pruning: eample MAX 6 Selected mve MIN 6 2 5 6 12 8 2 5 CS 460, Sessins 8-9 61

α-β pruning: general principle Player Oppnent m α If α > v then MAX will chse m s prune tree under n Player Similar fr β fr MIN Oppnent n v CS 460, Sessins 8-9 62

Prperties f α-β CS 460, Sessins 8-9 63

The α-β algrithm: CS 460, Sessins 8-9 64

Mre n the α-β algrithm Same basic idea as minima, but prune (cut away) branches f the tree that we knw will nt cntain the slutin. CS 460, Sessins 8-9 65

Mre n the α-β algrithm: start frm Minima CS 460, Sessins 8-9 66

Remember: Minima: Recursive implementatin Cmplete: Yes, fr finite state-space Optimal: Yes Time cmpleity: O(b m ) Space cmpleity: O(bm) (= DFS Des nt keep all ndes in memry.) CS 460, Sessins 8-9 67

Mre n the α-β algrithm Same basic idea as minima, but prune (cut away) branches f the tree that we knw will nt cntain the slutin. Because minima is depth-first, let s cnsider ndes alng a given path in the tree. Then, as we g alng this path, we keep track f: α : Best chice s far fr MAX β : Best chice s far fr MIN CS 460, Sessins 8-9 68

Mre n the α-β algrithm: start frm Minima Nte: These are bth Lcal variables. At the Start f the algrithm, We initialize them t α = - and β = + CS 460, Sessins 8-9 69

Mre n the α-β algrithm In Min-Value: MAX α = - β = + MIN Ma-Value lps ver these MAX Min-Value lps ver these 5 10 6 2 8 7 α = - β = 5 α = - β = 5 α = - β = 5 CS 460, Sessins 8-9 70

Mre n the α-β algrithm In Ma-Value: MAX α = - β = + MIN Ma-Value lps ver these α = 5 β = + MAX 5 10 6 2 8 7 α = - β = 5 α = - β = 5 α = - β = 5 CS 460, Sessins 8-9 71

Mre n the α-β algrithm In Min-Value: MAX MIN α = 5 β = + α = - β = + MAX Min-Value lps ver these 5 10 6 2 8 7 α = - β = 5 α = - β = 5 α = - β = 5 α = 5 β = 2 End lp and return 5 CS 460, Sessins 8-9 72

Mre n the α-β algrithm In Ma-Value: MAX α = - β = + MIN Ma-Value lps ver these α = 5 β = + α = 5 β = + MAX 5 10 6 2 8 7 α = - β = 5 α = - β = 5 α = - β = 5 α = 5 β = 2 End lp and return 5 CS 460, Sessins 8-9 73

Anther way t understand the algrithm Frm: http://yda.cis.temple.edu:8080/ugaiwww/lectures95/search/alpha-beta.html Fr a given nde N, α is the value f N t MAX β is the value f N t MIN CS 460, Sessins 8-9 74

Eample CS 460, Sessins 8-9 75

α-β algrithm: CS 460, Sessins 8-9 76

Slutin NODE TYPE ALPHA BETA SCORE A Ma -I +I B Min -I +I C Ma -I +I D Min -I +I E Ma 10 10 10 D Min -I 10 F Ma 11 11 11 D Min -I 10 10 C Ma 10 +I G Min 10 +I H Ma 9 9 9 G Min 10 9 9 C Ma 10 +I 10 B Min -I 10 J Ma -I 10 K Min -I 10 L Ma 14 14 14 K Min -I 10 10 NODE TYPE ALPHA BETA SCORE J Ma 10 10 10 B Min -I 10 10 A Ma 10 +I Q Min 10 +I R Ma 10 +I S Min 10 +I T Ma 5 5 5 S Min 10 5 5 R Ma 10 +I V Min 10 +I W Ma 4 4 4 V Min 10 4 4 R Ma 10 +I 10 Q Min 10 10 10 A Ma 10 10 10 CS 460, Sessins 8-9 77

State-f-the-art fr deterministic games CS 460, Sessins 8-9 78

Nndeterministic games CS 460, Sessins 8-9 79

Algrithm fr nndeterministic games CS 460, Sessins 8-9 80

Remember: Minima algrithm CS 460, Sessins 8-9 81

Nndeterministic games: the element f chance epectima and epectimin, epected values ver all pssible utcmes CHANCE? 0.5 0.5 3?? 8 17 8 CS 460, Sessins 8-9 82

Nndeterministic games: the element f chance Epectima CHANCE 4 = 0.5*3 + 0.5*5 0.5 0.5 3 5 Epectimin 5 8 17 8 CS 460, Sessins 8-9 83

Evaluatin functins: Eact values DO matter Order-preserving transfrmatin d nt necessarily behave the same! CS 460, Sessins 8-9 84

State-f-the-art fr nndeterministic games CS 460, Sessins 8-9 85

Summary CS 460, Sessins 8-9 86

Eercise: Game Playing Cnsider the fllwing game tree in which the evaluatin functin values are shwn belw each leaf nde. Assume that the rt nde crrespnds t the maimizing player. Assume the search always visits children left-t-right. (a) Cmpute the backed-up values cmputed by the minima algrithm. Shw yur answer by writing values at the apprpriate ndes in the abve tree. (b) Cmpute the backed-up values cmputed by the alpha-beta algrithm. What ndes will nt be eamined by the alpha-beta pruning algrithm? (c) What mve shuld Ma chse nce the values have been backed-up all the way? B E F G H I J K C L M N O P Q R S T U V W X Y 2 3 8 5 7 6 0 1 5 2 8 4 10 2 A D Ma Min Ma Min CS 460, Sessins 8-9 87