A Brief Survey of Chess AI: Strategy and Implementation Mark S. Montoya University of New Mexico CS 427 Fall 2012 Introduction Chess AI seems to be a less popular area of research since the 1997 defeat of Chess grandmaster Garry Kasparov by IBM's Deep Blue, but the decades-long quest to build a chess playing automata such as Deep Blue produced a strong foundation and helped form many of the strategies used in AI applications nowadays. Currently, there is research done in Chess AI studying possibilities through techniques including machine learning, neural networks, and genetic algorithms. Chess AI development is truly a cornerstone of early interest in AI techniques and one of the fundamental problems which helped build modern game-playing AI. Chess was popular enough for people to care about creating a chess AI, simple enough to allow represention of the game board and rule system, yet complicated enough for it to require ingenuity, strategy, and innovation. Long-Term Planning The branching factor of a typical chessboard configuration is considered nontrivial by today's computing capabilities, and for the pioneers of game-playing AI (specifically chess), it was a major limitation. A basic chess AI suffers from a phenomenon called myopia, the culprit of its superficial and predictable play. This problem is found in any tree-searching, game-playing program and it stems from an inability to fully represent the search space of the game. When building a tree with ply n (n levels deep), there is a chance that the n+1 level could contain a drastic, game-altering state which the program would critically avoid if it had knowledge of the state. Even use of techniques such as alpha-beta pruning, iterative deepening, and transposition tables are a band-aid on the root of the problem they simply don't help enough to produce a championship AI. Like chess grandmasters, the program must think long-term. This can be achieved by including some expert knowledge, or some type of pre-processer which evaluates the board and decides on a long-term strategy. The idea here is to first pass the current game-board into a long-term planning program, and 1
this program will decide on a suitable strategy for playing the game, well outside of the scope of the search tree. Then, with this long-term strategy in mind, the short-term, n-ply search tree could be used to decide on appropriate moves for the short term. To influence the short-term planner, the long-term planning program could devise a new set of heuristics for each long-term strategy. For example, if the goal is to achieve checkmate with a queen and a rook using a ladder checkmate, the long-term planner could place high value on those pieces, strongly discouraging the short-term planner from sacrificing those pieces. The long-term planning method is similar to how a grandmaster might initially approach a move decision. First, the overall status of the board must be considered and a long term strategy is devised. Only then, after deciding on how to act in the long term, should we consider the optimal move in the short term which also moves the game in the planned long-term direction. Implementing this in a program could probably best be described as an expert knowledge implementation. The overall state of the board is assessed, and matched as closely as possible to some type of general board configuration for which there exists a known strategy. One way to define these strategies is on the current depth of the game. We can think of opening strategies, mid-game play, and finales methods for achieving checkmate. Opening strategies are based on any number of opening-play strategies which can be found online. These are not difficult to encode and are highly effective at producing a good opening-board situation. Determining a mid or end-game strategy, however, is more difficult. These choices rely heavily on well-implemented, innovative, and properly abstracted expert knowledge. Hans Berliner, a pioneer of chess AI at CMU claims that Chess programs may be viewed as highly tuned expert systems (Berliner et al, 338), and defines what he calls a taxonomy for defining moves and board positions which could be effective in a long-term planning program. Another advantage of this long-term planning approach is that the program can be thought of as two distinct parts, the short-term and the long-term planner, which can be implemented separately and with minimal dependency between them. Arguably, however, the long-term planning interface can be thought of as far more important than the short-term planner as it defines the direction of game-play. 2
Man vs. Machine In 1988, a computer designed by F. H. Hsu named Deep Thought defeated a human with the grandmaster title for the first time. One year later, Hsu joined IBM and they began development on the next iteration of this machine, named Deep Blue. Deep Blue was to be the first chess program using massively-parallel computation, and was completed in 1995. It was a successful program, but it was not until 1997 that the latest version of Deep Blue defeated a slightly-bitter Garry Kasparov, marking the first defeat of a renowned world-champion by a computer in match play. Although Deep Blue was encoded with top-tier expert knowledge and highly optimized search algorithms, the main advantage of Deep Blue is its computation ability. Deep Blue employed specifically designed search algorithms which allowed for parallel search. Using the super-computing capabilities of Deep Blue, the algorithm speedup allowed Deep Blue to calculate unprecedented numbers of positions in short-term play, sometimes extending search from the current game state as deep as 40 ply. Deep blue was an expensive device implemented with an exclusive purpose in mind. Its hardware was centered around playing chess, and after its defeat of Garry Kasparov in 1997 the program was ended and Deep Blue dismantled. Today, due to the steep level of growth in computing power, the capabilities of Deep Blue can be nearly attained in a PC. Deep Blue, however, will remain a landmark in not only chess-playing AI, but in the capabilities of game-playing automata over even the greatest of human players. The well-publicized victory of Deep Blue prompted public interest in AI, documentaries, and books about the event. Modern Techniques: Genetic Algorithms & Evolving Strategy Over time, brute-force algorithms such as those described above have been a successful means of attacking the chess problem. They have improved significantly over the years, to the point where they can pose a formidable challenge to human chess grandmasters. These methods, however, seem to lack something that makes them truly intelligent. The creator of a program wielding such a method knows its strengths and its weaknesses he is the one who encoded the expert knowledge, and the methods of decision. The program is an extension of the programmer, and it will compute the next best move based on quantitative information with which it has been encoded. The automata does not know why it is 3
choosing to take one move over the other in any qualitative sense which might evoke the idea of intelligence. It is useful to consider how a human would play chess. The human player will not need to search to a large depth to realize that an obviously poor move should not be considered. Humans have the ability to focus on a select few moves without having to explore the entire search space of possibilities. This is accomplished by systematic pattern recognition and learning. How then, can we create chess programs that learn like humans do? There are many conventional learning methods which have been applied to chess, such as reinforcement learning and temporal difference learning. These methods have typically produced poor results even after being given extensive learning time. Such methods have successfully been applied to games such as checkers, but with chess the problem is best described as an optimization problem. The search space is massive, with about 10^46 possible states in a chess game. Further, the search space is not smooth or unimodal that is, it won't be matched with a heuristic which gives it nice, gradient slopes. Instead, the space is erratic and unpredictable. Genetic algorithms, on the other hand, typically perform well in large, non-unimodal search spaces. Genetic algorithms are simple enough to explain at a high level, and can be extremely powerful in many situations. The idea came from the evolution and learning of life itself it is a direct application of survival of the fittest to machines. In life, there are occasional mutations in our DNA structure which, over large periods of time, prove to be beneficial mutations or harmful. If they are beneficial, this mutation will eventually propagate throughout the population, while harmful mutations will die out. The amount of time this process takes to produce a noticeable effect is proportional to the length of one generation. In machines, this can be an extremely short amount of time, allowing evolutionary algorithms to produce outstanding results in relatively short time. The two essential components are: a means to create new, different generations, and a method to determine if a particular specimen is worthy of further consideration typically called the fitness function. Genetic algorithms can be applied to chess in many ways. A simple way would be to have each generation, say with population size 100, play games against each other member of the population and tally the score of each at the end of this round-robin tournament. The top-scorers are considered for the next round of breeding or generation, and this continues until a superior chess-playing AI is produced. Unfortunately, the amount of time required for this tempting method makes it unfeasible. 4
A better idea is to combine supervised and unsupervised learning to produce chess AI with genetic algorithms. In the first supervised phase, the programs are taught to mimic the play of human grandmasters, comparing the way they play to a large database of previously recorded games played by human grandmasters. The fitness function can be thought of as how close the prospective AI's decision was to a human grandmaster's decision in a similar game. Since this is a learning phase, the candidate can perform a search of low ply with a very simple evaluation function to determine its next move the main goal here is to evolve candidate AI which act like grandmasters. A strong advantage of this method is that it doesn't require an already robust evaluation function for determining the next move, but rather uses a simple evaluation function and is trimmed by the fitness function. In the unsupervised phase, the candidates will play against each other and co-evolve. Now that they are already made to play like grandmasters, they will compete in a long series of successive tournaments and breeding cycles to produce a superior chess AI. Experimentally, this method has been shown as an effective means of producing chess AI through the work of David-Tabibi, Jaap van den Herik, et al in 2009. It should also be noted that in the breeding phase, inbreeding is something to avoid this occurs when too many similar candidates are in the pool, and eventually all candidates are essentially the same with only minor differences setting them apart. When this happens, the final result may not be a suitable AI as it has not been tested against a well-represented group of candidates with various play styles and strategies. Conclusion Over time, developing AI for game playing has proven to be an important testbed and starting point for the creation of AI techniques across the board, including search, planning, and learning. Chess was considered as early as 1950 as a potentially worthwhile candidate for developing AI, and has been challenging the field ever since. The popularity and complexity of chess make it a fascinating and sought-after problem in AI, while the simplicity of representing the board and the rule-set makes it an accessible problem to anyone, including those without advanced training in computer science. Truly, it will be interesting to see how far chess AI will develop, and at what point we will consider the game to be beaten. Will there ever reign an undefeated, insurmountable chess AI, and how long will it take? 5
References Berliner, H., Kopec, D., & Northam, E. (1990). A taxonomy of concepts for evaluating chess strength. Supercomputing '90 Proceedings of the 1990 ACM/IEEE conference on Supercomputing, 336-343. David-Tabibi, O., Jaap van den Herik, H., Hoppel, M., & Netanyahu, N. (2009). Simulating human grandmasters: evolution and coevolution of evaluation functions. GECCO '09 Proceedings of the 11th Annual conference on Genetic and evolutionary computation, 1483-1490. Lassabe, N., Sanchez, S., Luga, H., & Duthen, Y. (2006). Genetically programmed strategies for chess endgame. GECCO '06 Proceedings of the 8th annual conference on Genetic and evolutionary computation, 831-838. Marshland, T. A., & Rushton, P. G. (1973). Mechanisms for comparing chess programs. ACM '73 Proceedings of the ACM annual conference, 202-205. Schaeffer, J. (1983). Long-range planning in computer chess. ACM '83 Proceedings of the 1983 annual conference on Computers : Extending the human resource, 170-179. Tan, C. J. (1995). Deep blue: Computer chess and massively parallel systems. ICS '95 Proceedings of the 9th international conference on Supercomputing, 237-239. 6