Poker Vision: Playing Cards and Chips Identification based on Image Processing

Transcription

1 Poker Vision: Playing Cards and Chips Identification based on Image Processing Paulo Martins 1, Luís Paulo Reis 2, and Luís Teófilo 2 1 DEEC Electrical Engineering Department 2 LIACC Artificial Intelligence and Computer Science Lab. Faculdade de Engenharia da Universidade do Porto, Rua Dr. Roberto Frias, 4200 Porto, Portugal {[email protected],[email protected],[email protected]} Abstract. This paper presents an approach to the identification of playing cards and counting of chips in a poker game environment, using an entry-level webcam and computer vision methodologies. Most of the previous works on playing cards identification rely on optimal camera position and controlled environment. The presented approach is intended to suit a real and uncontrolled environment along with its constraints. The recognition of playing cards lies on template matching, while the counting of chips is based on colour segmentation combined with the Hough Circles Transform. With the proposed approach it is possible to identify the cards and chips in the table correctly. The overall accuracy of the rank identification achieved is around 94%. Keywords: Poker, playing cards identification, image processing, template matching, colour segmentation, chips counting 1 Introduction To build an autonomous agent of Poker it is necessary to consider two very distinct modules: the intelligence of the agent and the agent s interaction with the real world. There is a lot of research work in Poker agent intelligence. The most renowned work is Darse Billings PhD thesis [1] where were distinguished and discussed several possible poker agent architectures. Building Poker agents requires the use of opponent modelling techniques, because the agent is not aware of the full game state. Several articles were wrote about this matter, such as [2] or [3]. Most opponent modelling techniques are based on David Slansky publications, such as [4]. This work focused on the interaction of the agent with the real world, more specifically collecting information about the state of the poker game. There has been relatively little work on playing cards or chips recognition based on vision [5 8]. The perception layer designed and implemented relies completely on image acquisition and processing. Besides the tasks of recognition of cards and chips, it also features the detection of players in the game at the beginning of each hand and the perception of the dealers position.

2 2 Paulo Martins 1, Luís Paulo Reis 2, and Luís Teófilo 2 2 Playing Cards Identification 2.1 Find Cards Finding the cards present on the table relies on the great contrast between the poker table and the cards lying on it, where the former features dark colours, e.g. dark green, and the latter is always white coloured. The greater the contrast between the cards and the poker table, the stronger the edges between both will be, i.e. the border between the card and the table Fig. 1. Detect cards contours algorithm overview The image captured of the table overview is first submitted to the preprocessing block. This block consists of low level image processing functions, where the image is first converted to gray scale, followed by smoothing with a Gaussian filter and finally the contrast between the card and the poker table is enhanced by a linear stretch to the histogram. The image is subject of edge detection, more specifically Canny edge detector. The resulting output from the Canny detector is scanned for external contours, thus all the first level contours found are kept while the remaining are discarded. The first level contours correspond to the border between the table and the cards. Finally, the algorithm approximates all the contours remaining by polygons and selects those which feature four vertices, correspondent to the four vertices of a card, thus remaining only trapezoids. Since playing cards have the shape of a rectangle, i.e. four right angles, the algorithm filters the trapezoids by their inner angles. All the trapezoids which inner angles are close to 90 are selected as cards, while the remaining are discarded. Accepting inner angles within a range around 90 makes the system robust enough to be in any angle relatively to the cards. If all the inner angles had to be exactly 90, it would be required to place the camera perpendicular to the cards on the table. 2.2 Cards Extraction The algorithm, shown in Fig. 2, is used to isolate each one of the trapezoid (cards) found so it is possible to process them individually. When the tripod is setup in one of the seats around the table, the position relative to the cards is not calibrated. Therefore the PokerVision does not know in advance in which angle it is relative to the community cards. In order to make this system robust enough to be positioned in any place around the poker table, the position of each card is computed using the coordinates information of its corners. This

3 PokerVision: Perception layer for a Poker Game 3 computation is based on the position of the corners relative to each other and to the global coordinate system. The reverse perspective block is intended to reverse Fig. 2. Cards extraction algorithm the perspective view of the card, outputting the image of the card with the correct size proportions, as a regular playing card, and without the perspective view effect,. It computes the matrix of perspective transform, based on the starting points, i.e. corners of the card on the original image, and destination points, i.e. the corners of the new rectangle shaped. The former are based on the corners coordinates of each card combined with the cards position/rotation determined previously. Since enlargement occurs, the final image will have some loss of quality. Fig. 3. Extraction of playing card reversing the perspective view 2.3 Rank Extraction This block aims to extract the rank of the card and prepare it for the template matching. By performing this, the algorithm restricts the region to be subject of recognition, benefiting the rank identification reliability. Not only prevents possible misidentifications with something else drew on the card than the rank itself, but also enables the identification to compute much faster since the image area to be analyzed is reduced significantly. The position and size of the rank are both the same across cards of the same deck, therefore the size and position of the crop was pre defined for the cards

4 4 Paulo Martins 1, Luís Paulo Reis 2, and Luís Teófilo 2 Fig. 4. Rank extraction algorithm overview used along the work. After the region of interest is extracted, the algorithm removes unwanted noise by smoothing it with a Gaussian filter. Thereafter binarizes the resultant image with an Adaptive Threshold. The resulting image after binarization is eroded in order to remove some remaining thin noise. In order to normalize the position of the rank, along with the filtering of remaining noise, the image is submitted to the isolate rank block. This block was implemented in order to tightly isolate the rank of the card from the rest of the crop. It seeks for all the elements, i.e. contours, present on the binarized image and encloses them in a box. Afterwards, the algorithm analyses the sizes of the enclosed contours as well as its centroids and discards all the contours which feature a non reasonable size, followed by selecting from the remaining, the one which centroid is the closest to the centre of the image. The resulting image is one featuring only the rank tightly isolated, Fig. 5 d). Fig. 5. Part of the rank extraction process The rank 10 is constituted by two characters instead of one, which means the algorithm when iterating through the conditions referred above, discards one of the characters. The work around for this issue consists in drawing one horizontal line 1pixel thick, wide enough to superimpose the characters 1 and 0. This misleads the algorithm, which searches for contours, to interpret both characters as the same one. 2.4 Rank Identification The approach used here, template matching [2], is a pattern recognition technique that allows detecting the presence of a specific object in an image. It can produce reliable results but needs to know exactly what to look for since it is based on the comparison of the image against a pre-defined template.

5 PokerVision: Perception layer for a Poker Game 5 Fig. 6. Card Identification algorithm The method is performed by sliding a template, Fig. 6, over the input image (card rank) while it calculates the normalized square differences between both. The best match corresponds to the one closest to 0. A total of thirteen templates were prepared previously and based on the cards used, corresponding each one of them to a rank of a card, Fig. 7. To prevent false matches, it was defined empirically that the scores above 0.45 are not considered as a valid match. Fig. 7. The thirteen templates used It is worth mentioning that, preceding the template matching, both ranks depicted in the card are extracted and subject of recognition. This provides the Template Matching with more samples to identify, i.e. ranks, extracted from the same card. Furthermore, since the samples come from different corners of the card, overcomes situations where illumination reflectance affects one of the corners, thus blinding the camera in that region, but not the other. From both resulting scores, prevails the lowest score, i.e. the best match. In order to make the matching more consistent, it was implemented a loop of successive capturing and extraction succeeded by matching along N-frames, during which the best matches found are stored. This loop is followed by a decision function which takes the stored matches and outputs the result of the identification. The decision function used is the weighted mean, which considers the identified ranks and the number of times each one occurred. The card rank which match is the closest to the weighted mean result, is the best match found for the correspondent card. 2.5 Suit Extraction and Identification Both the suit extraction and the suit identification algorithms are similar to the rank extraction and the rank identification, respectively. It is worth mentioning that the playing cards, on which this algorithm is based, have one suit under the

6 6 Paulo Martins 1, Luís Paulo Reis 2, and Luís Teófilo 2 rank. Therefore the size and position of the region to crop are compatible with the size and position of the suit. 3 Chips Identification In order to build a complete humanoid poker player, the information about the chips on the table is essential since it will have a role on the decision during game play. Fig. 8. Chips counting - algorithm overview The remove cards block performs the removal of the community cards present on the captured poker table overview. This procedure aims to prevent any misidentification between chips and playing cards, since the latter features elements the same colour as the chips used, such as the suit. The removal of the cards is achieved with a mask which distinguishes between the playing cards region and the rest. The colour segmentation aims to separate chips of different values on the game, which are only distinguishable by their colours. Within the colour segmentation block, the captured image undergoes a smoothing filter with the purpose of neutralizing high variations on the pixel values. The smoothing is followed by colour segmentation based on the RGB vector of each colour of chips. The threshold of each RGB component is within a fixed range in order to cover more points that can belong to the same chip. If one thinks of the RGB color space, this defined range can be seen as a cube centered in the RGB vector which represents the colour of the chip. This range overcomes the different illumination inherent to the environment. The resulting binarized image allows defining the regions of interest. The previous procedure spots the chips as well as other kinds of objects of the same colour. In case the latter are present on the captured image, these must be discarded. Moreover, it is intended to design the algorithm as robust as possible by making the detection of partially superimposed chips, since it is common to happen during the game play. All these requirements are achieved using the Hough Circles Transform. The Hough Circles Transform outputs better results, the better the contrast between the chips and the background. Therefore, preceding it, it is computed the absolute difference between the gray image of the chips with itself. Only difference is that the first has the brightness inverted, while the second does not. This procedure results in a great contrast between the chips and the table.

7 4 Experimental Results PokerVision: Perception layer for a Poker Game 7 In order to test the reliability of the algorithm, the webcam was placed on a tripod 81 cm high relative to the poker table. The tripod was placed in one of the seats of an octagonal poker table. In each round of the tests, three cards were dealt and placed along the table, such that the cards were 88cm, 103cm and 119cm away from the webcam. The resolution used was 800x600. Fig. 9. a) Webcam setup relative to the three cards b) perspective view of the webcam To determine the rank identification accuracy, it was registered how many times, for one specific rank, the identification was correct, incorrect or if the rank didnt meet the pre-requisites for the decision function to output it, which happens when the template match score is above For the suit the same procedure applies. A total of 265 cards were analyzed with an equally distributed number of ranks and suits. The results for the card rank identification are shown in the table above. Easily the 10 can be identified as being the one with the worst results. This relies on the fact that it is constituted by 2 digits, 1 and 0, and when the card is in the furthest position relative to the camera, it happens that in the preprocessing both of the digits get merged into one, making difficult the template matching. Just for reference, most of the times the 10 was identified as a 2. Concerning the suits, Spades features the worst accuracy, where most of the times was misidentified as Diamonds.

8 8 Paulo Martins 1, Luís Paulo Reis 2, and Luís Teófilo 2 5 Conclusion and Future Work Regarding the rank identification, the method used here should not be discarded, since it achieves an overall accuracy of 94,4%. Concerning the chips identification, the presented algorithm enables to count chips accurately if these are not occluded. Otherwise, when more than 30% occluded, it presents errors on the counting. These errors are higher, the further the chips are from the capturing device. Some improvements should be performed in order to make the algorithm more reliable. The first concerns the distinction between a red card and a black card. This would immediately prevent the misidentification between red suits and black suits, as it happens frequently with the spades being misidentified as diamonds. Moreover, the upgrade of the video source to one with a higher resolution would also improve the reliability of the system. The study and implementation of a stereo vision system would be of interest, in order to improve the counting of chips. 6 References References 1. Billings D (2006) Algorithms and Assessment in Computer Poker 2. Felix D, Reis LP An Experimental Approach to Online Opponent Modeling in Texas Hold em Poker. In: Springer-Verlag (ed) SBIA 08 Proceedings of the 19th Brazilian Symposium on Artificial Intelligence: Advances in Artificial Intelligence Salvador, Brazil, Van G, Kurt B, Ramon DJ Monte-Carlo Tree Search in Poker using Expected Reward Distributions. In: 1st Asian Conference on Machine Learning: Advances in Machine Learning, Nanjing, China, pp Sklansky D (2002) The Theory of Poker: A Professional Poker Player Teaches You How to Think Like One. Two Plus Two 5. Zheng C, Green R (December 2007) Playing Card Recognition Using Rotational Invariant Template Matching. University of Canterbury, Christchurch, New Zealand) 6. Zutis K, Hoey J (2009) Who s Counting?: Real-Time Blackjack Monitoring for Card Counting Detection. University of Dundee 7. Hollinger G, Ward N (2003) Introducing Computers to Blackjack: Implementation of a Card Recognition System using Computer Vision Techniques. Colby College, Waterville, Maine 8. Chen W-Y, Chung C-H (May 2010) Robust poker image recognition scheme in playing card machine using Hotelling transform, DCT and run-length techniques. In: Digital Signal Processing, vol edn., pp