How ToPlay Oce Pools If You Must: Maximizing Expected Gain In Tournament Models

Transcription

1 Abstract How ToPlay Oce Pools If You Must: Maximizing Expected Gain In Tournament Models David Breiter and Bradley P. Carlin University of Minnesota August 26, 1996 The increasing attention paid in recent years to the annual NCAA men's basketball tournament seems due at least in part to the proliferation of oce pools which follow tournament outcomes. Such pools typically assign a point value to each game in the tournament the winner of the pool is then the person who amasses the most points. Picking the winner of each individual game can be done using a variety of strategies, but nding one that is well-documented, performs well, and is relatively easy to use is often dicult. This paper presents such a method, where the primary goal is maximizing expected point total (hence winnings) for a given scoring system. The method employs Monte Carlo simulation, and makes use of predicted win probabilities determined using external information concerning the relative strengths of the teams, as well as the point spreads available at the start of the tournament for the rst round games. After investigating our approach for a small four-team tournament (where exact calculations are possible), we use Monte Carlo methods to expand to 16 teams, illustrating with data from the 1996 NCAA men's basketball tournament. 1 Introduction A common way of selecting the best member of a class of competitors is through a tournament, a series of contests matching two competitors at a time in some predetermined order, 1

2 with the losers eliminated from the competition and the winners advancing. Such contests are most common in the world of sport, but also nd application in many business and biomedical settings. For example, a drug company with a limited research budget may wish to select the one or two best drugs from a larger collection of promising therapies, with the winners advancing to a full-scale (Phase III) clinical trial (Whitehead, 1985 Brunier and Whitehead, 1994). In such a case, drug company executives would likely be concerned not only with the probabilities of various drugs advancing in the tournament, but also the expected gain to the company should a particular treatment emerge as the winner. In sports tournaments, the expected gain of interest is usually the nancial rewardtoa gambler betting on the various outcomes en route to the eventual winner. While investigation of such designs may not seem a particularly noble use of our statistical prowess, it is certainly one that will nd practical application. For perhaps as long as athletes have teamed together to participate in sporting events, gamblers have gathered to bet on the outcomes. In America today, gambling and sports seem to go almost hand in hand. Every NFL pregame show has had an oddsmaker since the days of Jimmy \The Greek" Snyder. Pick up a newspaper and you can read the latest point spreads on just about any game in a wide array of professional and college sports. Fantasy football and baseball leagues are soaring in popularity, and even have their own magazines. Still, the nation's most universally gambled upon event is probably the annual NCAA men's basketball tournament. Even those who know very little or nothing at all about the tournament and the teams competing will risk a few dollars and pick their favorites in a friendly oce pool. Whether they pick by an informed, calculated, learned strategy, by regional or other (e.g., alma mater) favoritism, or by the colors of the teams' jerseys, everyone has a method, as well as high 2

3 hopes of winning the pot. The popularity of such pools is probably due in part to a perception that winning them is mostly a matter of luck, so specialized knowledge of the teams is not required. Indeed, even a sportswriter or other professional follower of college basketball would be unlikely to have knowledge of all the strengths and weaknesses of each of the 64 tournament teams. Still, it seems clear that some strategies ought to do better than others. What is needed is a method to maximize the probability of winning a tournament pool that does not require expert knowledge of each team, but instead depends only on easily calculated team win probabilities that are based on readily accessible measures of team strength. Inanutshell, the NCAA men's basketball tournament is composed of 64 teams divided into four regional tournaments (East, West, Southeast, and Midwest) of 16 teams each. Teams reach the tournamentby one of two methods: they may receive an \at-large" bid, meaning they have beeninvited by the tournament selection committee to compete, or they may also gain an automatic bid by winning their own conference tournament. The selection committee then seeds the teams 1 through 16 in each region, according to their relative strength as perceived by the committee. The best team in the region will be seeded number 1, the next best number 2, and so on through the weakest team, seeded number 16. It is also the duty of the committee to try to keep the average team strength approximately equal across regions. Play within each region is single elimination (see Figure 1), with eight rst round games, four second round, two third round, and one regional nal. The winner of each region then goes on to the \Final Four," a four-team nal tournament to determine a single national champion. As seen in Figure 1, the games are organized so that the advantage to advance to the next round is given to the stronger teams. In the rst round seed 1 plays seed 16, seed 2 plays seed 15, and so on down to seed 8 playing seed 9. In the second round the winner of seed 1 and 3

4 Figure 1: Diagram of an NCAA Regional Basketball Tournament seed 16 plays the winner of seed 8 and seed 9 (the weakest available seed assuming all lower seeds advance). Thus, provided the quality of teams does indeed decrease with seed order, the number 1 seed has the easiest road to the Final Four, followed by number 2, and so on. 2 Determination of Win Probabilities As stated in the introduction, a method for maximizing expected gain relies upon individual game win probabilities, and is therefore only as good as the method used to determine those probabilities. Since Figure 1 shows that any pairing of seeds is possible at some point inthe tournament, we require a matrix of win probabilities P having entries for each potential pairing, where P (i j) =1; P (j i) fori 6= j, andp (i i) is left undened (a team cannot play against itself). The two methods for calculating these win probabilities considered in this paper were presented by Schwertman et al. (1991) and Carlin (1996), respectively. The former method is based 4

5 solely on team seedings. In its simplest version, for any two teams in the region seeded i and j where i<j(i.e., seed i is the stronger team), the probability P (i j) that seed i beats seed j is P (i j)=j=(i+j). The resulting P matrix is shown in Table 1. These crude estimates are a good start, but tend to overestimate the strength of the weaker teams. More sophisticated models for P (i j) are considered in this paper and a subsequent one by Schwertman et al. (1996), but all result in win probability matrices which are identical for each region and tournamentyear. i\j Table 1: P matrix computing using the simple method of Schwertman et al. (1991) The 64 teams selected to participate in a given tournament are unlikely to be the best 64 teams in the NCAA (recall the automatic conference bids), nor is it necessary that they will be ranked in the proper order by the selection committee. As such, the dierence in strength between each of the seeds will be unique to a given region and year. Reacting to this, Carlin (1996) models the win probabilities as Yij P (i j) (1) 5

6 where () denotes the cumulative distribution function of the standard normal distribution, Y ij is the point spread for game (i j), and is an appropriately chosen standard deviation. A point spread is a predicted amount by which one team (the \favorite") will defeat the other (the \underdog"), oered by casinos and sports wagering services and often printed in major daily newspapers. Equation (1) is based on the work of Stern (1991), who, working with data from professional football, demonstrated the appropriateness of the normal approximation and the \fairness" of the point spread (i.e., that it does in fact represent the average game outcome). For pro football, Stern suggested the standard deviation =13:86 in subsequent unpublished work with pro basketball data, = 11:5 emerged as most appropriate. For the lower-scoring game of college basketball, an even smaller value seems in order. Using data from the rst four rounds of the 1994 NCAA tournament, Carlin (1996) obtained the estimate ^ = 8:83. In what follows, we adopt the value = 11, a somewhat conservative choice that places a bit less faith in the point spreads, and also slightly increases the win probabilities for the underdogs. At the time the tournament begins, point spreads will only be available for the rst round games. To complete the P matrix, Carlin (1996) instead estimates the point spread as ^Y ij = [S(i) ; S(j)] (2) where S(k) is the pre-tournament Sagarin rating of team k, and is a xed constant. Sagarin ratings are numerical measures of team strength, published every Monday in the newspaper USA Today as well as on the world wide web at the URL http : ==www:usatoday:com=sports=basketba=skm=skmjs:htm 6

7 where they can be freely downloaded. These ratings incorporate several important variables (team record, opponents' records, strength of schedule, and so on) monitored over the course of the season. The ratings are designed to produce hypothetical point spreads when dierenced, but in a regression analysis of 1994 tournament data, Carlin (1996) shows these dierences are a slight underestimate in games matching teams of widely diering strengths. This explains the presence of the \blowout ination factor" in equation (2). Carlin (1996) suggested the value =1:16 we again adopt a more conservative choice, namely =1:05. i\j Table 2: P matrix computing using the method of Carlin (1996) Looking at the resulting probabilities for the East region in Table 2, we see that the win probabilities associated with the weaker teams are smaller than those in Table 1, giving a more accurate depiction of actual tournament play. The method can also account for anomalies in the seeding order for example, note that P (6 3) P(6 4), and P (6 5) are all bigger than.5, suggesting that the #6 seed (North Carolina) is actually better than the #3, 4, and 5 seeds in this region. 7

8 3 Determination of Expected Gain Throughout this paper, the term \strategy" is used in reference to the set of winners picked for the games in the tournament. The term \expected gain" refers to the expected point total associated with each strategy. (Recall that maximizing monetary winnings in tournament pools is done by maximizing point total.) The expected gain for each strategy is calculated by multiplying the probability of each possible outcome by the point total accumulated for the strategy given the outcome, and summing the results. We use a progressive scoring system typical of pools of this type: ve points awarded for correctly picking the winner of a rst round game, eight points for a second round game, 15 for a third, and 25 for the fourth round game. Our scoring system also rewards risk-taking by awarding an \upset bonus" anytime a higher (poorer) seed defeats a lower seed. The upset bonus we adopt is (j ; i) (round number + 1) : That is, the bonus is twice the seed dierence for rst round upsets, increasing to ve times the seed dierence in the fourth (regional nal) round. 3.1 Exact Calculation Looking rst at a simple four-team tournament allows us to compute the expected gain exactly. As shown in Figure 2, there are only three games, giving us 2 3 = 8 possible outcomes. Since we are looking at a hypothetical situation knowing nothing about the past performance of these teams, we will illustrate the calculation using Schwertman's simple method of computing win probabilities based solely on seedings. 8

9 Figure 2: Diagram of a hypothetical four-team tournament Table 3 displays the exact expected gain for each possible strategy, where strategies are written in the form (a b c) wherea is the winner of the game between seeds 1 and 4, b is the winner of the game between seeds 2 and 3, and c is the winner of the game between a and b. For this four-team tournament, picking the favorites in each game turns out to be the best strategy, though this result could be easily modied by changing either the win probabilities or the upset bonus structure. For example, a slight increase in the latter causes (1 3 1) to emerge as best. Strategy Expected Gain (1,2,1) (1,3,1) (1,2,2) 9.40 (1,3,3) 8.29 (4,2,2) 7.60 (4,3,3) 6.49 (4,2,4) 6.14 (4,3,4) 5.88 Table 3: Exact Expected Point Totals by Strategy for the Four-Team Tournament 9

10 3.2 Monte Carlo Simulation of Expected Gain Exact calculations are feasible when the number of possible outcomes is relatively low. However, if we now return to our 16-team tournament, we see that there are 15 games played, giving us 2 15 = possible outcomes. Calculating the exact expected gains for each strategy would be extremely tedious to program, and likely take an enormous amount of computer time. Therefore, a reasonable estimate via Monte Carlo methods seems preferable. A Fortran program was created to generate 15 random Uniform(0,1) probabilities per draw, one for each game played. The program compares each of these randomly generated probabilities to the win probabilities in the P matrix. For each game, if the generated probability is greater than the corresponding win probability then an upset occurs otherwise the favored team wins (see the computer code in Appendix A). The gain g i is then computed based on the strategy under consideration. For each simulation, N = draws were used to produce an average point total g and corresponding estimated standard deviation sd(g). c The average point totals are simply the sample means of the g i the standard deviations were calculated using the usual Monte Carlo formula, csd(g) = vu u t 1 N(N ; 1) NX i=1 (P! N gi 2 ; i=1 g i ) 2 : N In the next section, we apply this approach to data from the 1996 NCAA basketball tournament. 4 Numerical Illustrations Since the NCAA tournament itself is composed of teams of widely varying strength, Carlin's method of determining win probabilities will be most appropriate. The illustrations in this section used win probabilities calculated using the best information available just prior to the 10

11 1996 tournament (rst round point spreads from a prominent Las Vegas oddsmaker and pretournament Sagarin ratings), as described in Section Comparison of promising strategies We begin by comparing the performance of a few commonly used strategies. One obvious choice (which we label \Seed Favorites") is to always pick the lower (favored) seed for every game, working through the rounds from 1 to 4. This strategy tries to maximize the chance of getting the usual round points, but forfeits any chance at upset bonus (since no upsets are predicted). A second possibility (which we label \Big Four") is the same as above, but replacing the rst round victories by seeds 5{8 with upsets by seeds 9{12, who are all then eliminated in the second round (hence the strategy name). Here the idea is to take advantage of the nearparity of the middle 8 teams in the bracket, hoping that the potential upset bonus in the rst round will more than oset the slight underdog status of seeds 9{12. Regional score (sd) Strategy Round Picks E W MW SE Seed Favorites (0.23) (0.21) (0.20) (0.23) Big Four (0.27) (0.26) (0.24) (0.26) Table 4: Simulated expected point totals (and associated Monte Carlo standard errors) for 1996 NCAA regional tournaments, two seed-based strategies Table 4 shows that in each region, the \Big Four" strategy does indeed perform better than simply picking seed favorites. In the 1996 tournament, several upsets were predicted by the 11

12 rst-round point spreads, and 8 of the 16 such games matching middle (5{12) seeds did in fact end in upsets. It thus seems that aiming for some sort of upset bonus is indeed a good idea. Regional score Strategy (Region) Round Picks (sd) Trial Best (E) (0.30) Trial Best (W) (0.29) Trial Best (MW) (0.24) Trial Best (SE) (0.24) Table 5: Simulated expected point totals (and associated Monte Carlo standard errors) for 1996 NCAA regional tournaments, best strategies obtained via trial and error Of course, we could continue this line of thinking indenitely, creating more and more promising strategies to investigate. One obvious such choice might be the \Sagarin Favorites" strategy, where we always choose the team not with the higher seed, but with the higher Sagarin rating (thus picking upsets only when the seeding committee has made a mistake and the \upset" is actually more likely than not). Rather than develop a collection of rules of this type, we instead simply began looking for good strategies for our data by trial and error. Looking at table 5 we see that the best strategies found by trial and error (which we label \Trial Best") perform substantially better than the Seed Favorites, but only marginally better than Big Four (in fact, the two are equivalent in the Midwest and Southeast regions). Similar to Big Four, the Trial 12

13 Best strategies pick 10 middle seed upsets in the 16 rst round games, plus four more upsets in later rounds. Regional score Total Strategy E W MW SE Score Seed Favorites Big Four Trial Best Table 6: Comparison of actual performance of strategies, 1996 NCAA tournament Finally, Table 6 shows how the three strategies considered thus far would have actually fared had they been used in the 1996 pool. The table shows points earned in each region, as well as the sum over all regions. It is comforting to see that the ordering in performance of the three strategies is the same as that predicted by Tables 4 and 5, with Trial Best emerging as superior. Note that most of the dierence in scores can be attributed to performance in the East and West regions. In the East, Trial Best correctly chose the 12th seed (Arkansas) to win two games, something not chosen by the other two strategies and richly rewarded with upset bonus points. In the West, Trial Best correctly picked #8 Georgia to upset #1 Purdue in the second round, and correctly picked #4 Syracuse to win its regional seminal (though not the nal, which Syracuse also went on to win). None of these picks were made by the other two less exible strategies, suggesting that, like good win probability estimates, strategies must be region- and year-specic, in order to earn maximal upset bonus points and thus do well overall. 4.2 Algorithm for checking all possible strategies Even though many of the 32,768 possible strategies are easy to eliminate, the trial-anderror method seems unlikely to produce the absolute best strategy in each region with respect 13

14 to expected gain. For one thing, it may be dicult to guess whether to risk picking many upsets in the rst round, even when the expected rst-round upset bonus is high, since this by denition eliminates favorites that may be better able to persevere to the later rounds, where point rewards are higher (and thus crucial to a winning strategy). Rather than continue with an ad hoc approach, the natural solution is to search all possible strategies, evaluating each via Monte Carlo simulation as before. The Trial Best strategy can serve as a reference point in this search, in that we canagany strategy having simulated expected gain better than that of the Trial Best strategy minus two standard errors (so that, to the order of accuracy in our simulation, each agged strategy would be no worse than the Trial Best). Regional score Strategy (Region) Round Picks (sd) Best of All (E) (0.30) Best of All (W) (0.29) Best of All (MW) (0.28) Best of All (SE) (0.25) Table 7: Simulated expected point totals (and associated Monte Carlo standard errors) for 1996 NCAA regional tournaments, best of all possible strategies We used a Fortran program (see Appendix B) to create a matrix of all the possible strategies, which we then ran through a modied version of our rst program (see Appendix 14

15 C) to obtain the simulated scores. The entire process took roughly 8 hours of CPU time { a seemingly small price to pay forabettershotatbraggingrights in the oce pool. The best strategies found (which we label \Best of All") are displayed in Table 7. The table shows that once again, many upsets were picked in the rst round games between 5 through 12 seeds (11 of 16 games). The only dierences between Best of All and Trial Best are in the former's choice of three more upsets in the second round and one more in the third, showing once again the lure of the upset bonus. Had this stategy been used in the 1996 tournament, the actual point total would have been 433, as shown in Table 8. Comparing Tables 8 and 6 we note that Best of All, the strategy having larger pre-tournament expected gain than any other, did not actually perform as well as our Trial Best strategy when used with the 1996 tournament data, since the upsets picked by the former did not pan out. While Best of All cashed in on #12 Arkansas' two victories in the East, in the Midwest its choice of #3 Villanova to win three games was o by two. In addition, West #6 Iowa and Southeast #11 Boston College both failed to win their second games as predicted by Best of All. Regional score Total Strategy E W MW SE Score Best of All Table 8: Best of All Possible Strategies in 1996 Tournament 15

16 5 Discussion This paper oers a promising method for maximizing expected gain in a tournament model via Monte Carlo simulation. The information used to compute the win probabilities (point spreads and Sagarin ratings) is readily accessible, and the calculation of these probabilities requires only rudimentary statistical concepts and software (more specically, a normal cumulative distribution function routine). The method itself employs a collection of Fortran programs, all freely available over the world wide web at http : ==www:biostat:umn:edu=~brad= the second author's home page. The win probability determination method of Carlin (1996) that we employ depends heavily on pre-tournament team strength ratings. The probabilities are thus based on past performance, and cannot account for sudden changes, such as injuries to key players, coaching changes, etc. An example of this from the 1996 tournament would seem to be the Southeast region, where no strategy using our win probabilities (and indeed, few expert observers at the time) predicted #13 Princeton's remarkable victory over #4 (and defending national champion) UCLA, or #5 Mississippi State's surprising rise to the Final Four. In the former case, emotional players won in the nal appearance of a beloved retiring coach in the latter, a key player returned from injury that had caused him to miss much of the regular season. Both of these cases suggest additional information not included in our P matrix for this region. As shown in Section 4, the simulation method for evaluating strategies provided expected gain estimates having reasonably small standard errors (i.e., at most 0.30) using a modest number of 16

17 Monte Carlo draws (N =10 000). However, it is important to remember that, even if all our win probabilities are correctly specied, this standard error is associated with the expected gain, not the gain that will actually occur in the pool. After we settle on our pool picks, we get only one trial, for which the standard error will be sd(g)= p Nsd(g) (3) or 100 times the standard errors shown in our Section 4 tables. This means a standard error in the 25 to 30 point range { larger than the amount by which the winner of a 10- or 20-person pool might be expected to win! Finally, the choice of whether to choose a strategy by trial and error, or actually search over all possible strategies to nd the best one certainly depends on one's available computer time, but also to some extent on personal preference. In our example, the Best of All strategy did not perform as well the Trial Best strategy. Note that, while the former did indeed have the higher expected value, it was also slightly riskier (use equation (3) with the sd(g) values in Tables 5 and 7). Apparently in this particular tournament, this additional risk was not worth the payo. We suspect that this scenario (risky \best" strategies that predict upsets in the second and later rounds) is common when searching over all possible strategies. Perhaps a solution is to print out the top 5 or so strategies found by the computer, and select the least risky of these. Alternatively, one could use the time-honored method of multiple pool entries (say, one found by trial and error, and another generated by the computer). But no matter how one chooses to bet, our nal recommendation must echo the now-famous refrain of Dubins and Savage (1965): It is probably best not to play at all, since you are very probably going to walk away empty-handed. 17

18 References Brunier, H.C. and Whitehead, J. (1994), \Sample Sizes For Phase II Clinical Trials Derived From Bayesian Decision Theory," Statistics In Medicine, 13, 2493{2502. Carlin, B.P. (1996), \Improved NCAA Basketball Tournament Modeling via Point Spread and Team Strength Information," The American Statistician, 50, 39{43. Dubins, L.E. and Savage, L.J. (1965), HowtoGambleifYou Must Inequalities for Stochastic Processes, New York: McGraw-Hill. Schwertman, N.C., McCready, T.A. and Howard, L. (1991), \Probability Models for the NCAA Regional Basketball Tournaments," The American Statistician, 45, 35{38. Schwertman, N.C., Schenk, K.L. and Holbrook, B.C. (1996), \More Probability Models for the NCAA Regional Basketball Tournaments," The American Statistician, 50, 34{38. Stern, H. (1991), \On the Probability of Winning a Football Game," The American Statistician, 45, 179{183. Whitehead, J. (1985), \Designing Phase II Studies in the Context of a Programme of Clinical Research," Biometrics, 41, 373{