Sports Forecasting: A Comparison of the Forecast Accuracy of Prediction Markets, Betting Odds and Tipsters

Transcription

1 Sport Forecating: A Comparion of the Forecat Accuracy of Prediction Market, Betting Odd and Tipter Martin Spann 1 and Bernd Skiera 2 Thi i a preprint of an Article accepted for publication in the Journal of Forecating 2008 John Wiley & Son, Ltd. Short title: Sport Forecating: Prediction Market, Betting Odd and Tipter Contact Information: 1 School of Buine and Economic, Univerity of Paau, Inntr. 27, Paau, Germany, Phone: , Fax: , pann@pann.de 2 School of Buine and Economic, Johann Wolfgang Goethe-Univerity, Mertontr. 17, Frankfurt am Main, Germany, Phone: , Fax , kiera@kiera.de. i

2 Sport Forecating: A Comparion of the Forecat Accuracy of Prediction Market, Betting Odd and Tipter Abtract Thi article compare the forecat accuracy of different method, namely, prediction market, tipter and betting odd, and aee the ability of prediction market and tipter to generate profit ytematically in a betting market. We preent the reult of an empirical tudy that ue data from 678 to 837 game of three eaon of the German premier occer league. Prediction market and betting odd perform equally well in term of forecating accuracy, but both method trongly outperform tipter. A weighting-baed combination of the forecat of thee method lead to a lightly higher forecat accuracy, wherea a rule-baed combination improve forecat accuracy ubtantially. However, none of the forecat lead to ytematic monetary gain in betting market becaue of the high fee (25%) charged by the tate-owned bookmaker in Germany. Lower fee (e.g., approximately 12% or 0%) would provide ytematic profit if punter exploited the information from prediction market and bet only on a elected number of game. Keyword: Sport Forecating, Prediction Market, Combined Forecat, Rule-baed Forecat ii

3 1 Introduction Sport forecating i important for port fan, team manager, ponor, the media and the growing number of punter who bet on online platform (Vlataki et al., 2007). Widepread demand for profeional advice regarding the reult of porting event i met by a variety of expert forecat, uually in the form of recommendation from tipter (Forret & Simmon, 2000). In addition, betting odd offer a type of predictor and ource of expert advice regarding port outcome. Wherea fixed odd reflect the (expert) prediction of bookmaker (Pope & Peel, 1989), the odd in parimutual betting market indicate the combined expectation of all punter, which implie an aggregated expert prediction (Plott et al., 2003). Prediction market (PM), firt applied to forecat political election reult (Forythe et al., 1992; Forythe et al., 1999) and later buine outcome (Dahan et al., 2006; Elbere, 2007; Elbere & Eliahberg, 2003; Gruca et al., 2003; Jank & Foutz, 2007; Pennock et al., 2001; Spann & Skiera, 2003), increaingly attempt to predict porting event (Luckner & Weinhardt, 2007; Servan-Schreiber et al., 2004; Wolfer & Zitzewitz, 2006), which ugget they provide an additional method of port forecating. In eence, PM bring a group of participant together via the Internet and let them trade hare of virtual tock that repreent bet on the outcome of future market ituation; the tock value depend on the realization of thoe choen market ituation. When an outcome aociated with a pecific market ituation occur, each hare of virtual tock receive a cah dividend (payoff) (e.g., $1 if the predicted team win, $0 otherwie). In a PM, each participant contribute hi or her knowledge to the market by trading, o the tock price repreent participant' aggregated knowledge and thu the PM prediction (Hahn & Tetlock, 2006; Spann & Skiera, 2003). The availability of multiple forecating method raie quetion about their effective ue. Previou tudie conider the performance of betting odd and tipter (for recent ummarie, ee 1

4 Forret et al. (2005) or Anderon et al. (2005)), repectively betting odd and prediction market (Servan-Schreiber et al., 2004) for port forecating, but knowledge about their comparative performance veru PM remain carce, becaue no tudie compare all three forecating method (Anderon et al., 2005; Boulier et al., 2006; Chen et al., 2005; Forret et al., 2005; Goddard & Aimakopoulo, 2004; Paton & Vaughan William, 2005). Furthermore, little i known about the potential imilarity of forecat acro method, their performance or their ability to improve forecat accuracy if ued in a weighting-baed or rule-baed combination. However, uch knowledge i important becaue it might allow punter to ytematically earn money on thoe market. In addition, it provide recommendation for port and betting companie on how to improve their forecat. Therefore, thi article empirically compare the forecat accuracy of PM, tipter and betting odd, a well a weighting- and rule-baed combination of thoe forecat. We preent the reult of an empirical tudy that ue data from 837 game acro three eaon of the German premier occer league. In conideration of the vat um of money at take in betting market, we alo determine whether the forecat of the three method or their combination enable ytematic profit. Thu, we contribute to the port forecating literature by providing the firt large-cale empirical tudy of the three forecating method and their combination in term of their forecating accuracy and ability to enable profit for punter in betting market. The remainder of thi article i tructured a follow: In Section 2, we decribe the three forecating method, then ue Section 3 to decribe the data, the performance meaure and the calculation required for the three expert forecating method. In Section 4, we compare the forecat accuracy of the three forecating method, a well a of their combination. The final ection conclude our paper. 2

5 2 Decription of Three Forecating Method 2.1 Prediction Market The fundamental concept behind PM ugget that market can olve information problem (Hayek, 1945). Related, the efficient market hypothei poit that price alway reflect all available information (Fama, 1970). A competitive market achieve market efficiency through the price mechanim, the mot efficient intrument for aggregating aymmetrically dipered information poeed by market participant (Hayek, 1945; Smith, 1982). Therefore, price in a competitive market offer an aggregate reflection of all participant public and private information and thu erve a a good predictor (Spann & Skiera, 2003). A a reult, market poe the poitive characteritic of information elicitation and aggregation, immediate reaction to new information and calability with repect to the number of participant (Dahan & Hauer, 2002; Oliven & Rietz, 2004). Thee characteritic make them a potentially promiing method for olving information problem (Spann et al., 2007; Tzirali & Tatiopoulo, 2007). In the variou PM for porting event (e.g., port.u.newfuture.com, participant trade virtual tock related to future market ituation, namely, the outcome of porting event. The cah dividend (payoff) of thee hare of virtual tock depend on the actual outcome of the event; therefore, the price of one hare of a virtual tock hould correpond to the PM' aggregate expectation of the event outcome and, in turn, the (dicounted) expected cah dividend of a hare of tock. Participant in the PM ue their (individual) expectation of the outcome to derive an (individual) expectation of the cah dividend of the related hare of virtual tock. Accordingly, they compare their expected cah dividend with the PM' aggregate expectation, which i a function of the tock price, a a mean to trade their individual expectation. For example, if a participant anticipate that the L.A. Laker will core 100 point in a pecific game, the cah 3

6 dividend of the related hare of virtual tock would be $100, and each point would correpond to $1. In the cae of a current tock price of $95 ($105) that i, an expectation of 95 (105) point the tock i undervalued (overvalued), according to the etimate of thi participant, who therefore could try to attain an expected profit of $5 by buying (elling). If the potential gain in the virtual portfolio value create a ufficiently high incentive for participant to perform well in the PM, it become their bet trategy to engage in tranaction on the bai of their bet individual expectation. Thu, the participant reveal their true expectation of future market ituation through their buying and elling activitie (Oliven & Rietz, 2004; Spann & Skiera, 2003). By making individual expectation tradable, a PM create a market for prediction about future market ituation, in which participant compete according to their individual expectation. Thu, the tock price reflect the participant aggregated information. Extenive tudie uing both empirical data and laboratory experiment upport the informational efficiency of uch market (ee overview by Fama (1970, 1991)), a do the powerful reult of political tock market (Forythe et al., 1999). 2.2 Tipter Expert forecat of port outcome often come from o-called "tipter", whoe prediction appear in port journal or daily newpaper. Tipter are uually independent expert who do not apply a formal model but rather derive their prediction from their experience or intuition (Forret & Simmon, 2000). They generally provide forecat for only a pecific election of game, often related to betting. No immediate financial conequence reult from the prediction of tipter. Empirical evidence regarding the forecat accuracy of tipter how that their ability i limited. Forret & Simmon (2000) how that tipter perform better than random forecating 4

7 method but wore than a forecating method that alway predict a home win (three tipter correctly predicted 41.09%, 42.56% and 42.86%; win by home team occurred 47.5% of the time). Anderon et al. (2005) alo reveal, paradoxically, that occer expert fail to predict more accurately than people with limited knowledge of the game. Thee author ugget their finding indicate the expert' inefficient ue of information, a well a layperon effective ue of fat, frugal heuritic. Their reult alo mirror reearch that found poor forecating abilitie of tock market expert (e.g., Törngren & Montgomery (2004)) and economit for buine trend (e.g., Mill & Pepper (1999)). 2.3 Betting Odd Extenive analye in economic and buine literature ugget that betting odd provide an efficient forecating intrument (Gandar et al., 1998; Pope & Peel, 1989; ee alo the recent pecial iue of Applied Economic on the Economic of Betting Market). For a recent ummary of the hitory of port wagering, ee Vlataki et al. (2007). Bookmaker determine fixed betting odd according to their expectation of game outcome probabilitie, and once they are publihed, fixed odd rarely change. Thee fixed odd therefore repreent expert prediction by bookmaker (Pope & Peel, 1989). Anderon et al. (2005) compare the performance of expert and laypeople in predicting the outcome of the occer World Cup 2002; the difference in their performance i not tatitically ignificant. Forret et al. (2005) alo compare the forecating performance of everal Britih bookmaking companie for the outcome of Englih occer game over a five-year period and find that the forecating performance increae over time. However, Goddard & Aimakopoulo (2004) reveal, in the context of Englih occer league matche during the eaon, that conidering additional information, uch a previou outcome, team quality indicator and geographical ditance between the team, lead to betting trategie with a poitive 5

8 gro return of +8%. Boulier et al. (2006) imilarly analyze the forecating performance of betting market pread for outcome during in the American National Football League (NFL) but how that no information beyond the point pread explain outcome ignificantly better. Finally, Paton & Vaughan William (2005), alo in the context of Englih premier occer league game, indicate that information about initial bid offer pread of four major U.K. port pread betting companie improve prediction, making them lightly better than the prediction of individual betting companie. Their reult ugget betting odd provided by betting companie have a rather high forecating accuracy, which i plauible, becaue betting companie with inefficient odd would not urvive. However, depite thee variou analye and conideration, no previou tudie compare their reult with thoe of PM or tipter. 3 Decription of the Data 3.1 Data Set We forecat the outcome of game in Germany' premier occer league over three eaon: , and Germany' premier occer league include 18 team that each play twice in a eaon, which equal 34 tournament round with 9 game each, or 306 game per eaon, and thu 918 game in all three eaon. In the and eaon, a tournament round had the following tructure: two game on Friday, five game on Saturday and two game on Sunday. In the eaon, a tournament round intead meant even game on Saturday and two game on Sunday. In contrat to many other port, epecially in the United State, a occer game ha three poible outcome: win, loe or draw. In the cae of a draw, each team receive one league point; in cae of a win, the winning team receive three league point and the loer none. For each tournament round, we collected game outcome, tock price on a PM ( for prediction of thoe game outcome, tipter prediction (win, 6

9 draw, loe) of the mot popular German port journal (Sport Bild) and the fixed betting odd of the larget German tate-owned bookmaker (Oddet). The PM provided prediction for 91.18% of all game (N PM = 837 game), o we collect betting odd for the ame ample of game (N BET = 837). However, we have fewer obervation of tipter prediction, becaue the port journal did not publih prediction for the two game on Friday during the and eaon. In addition, the journal arbitrarily ignored prediction for game in ome week, which leave u with N TIP = 721 prediction by tipter. Therefore, prediction aociated with all three method are available for 678 game. While the number of obervation i maller than thoe of tudie that analyze the forecat accuracy of betting market (e.g., Pope & Peel (1989): 1,291 matche, Cain et al. (2000): 2,855 matche, Dixon & Pope (2004): 6,629 matche, Vlataki et al. (2007): 12,841 matche, Graham & Stott (2008): 11,000 matche), it i ubtantially larger than thoe of tudie analyzing the forecat accuracy of prediction market (e.g., Jank & Foutz (2007): 262, Pennock et al. (2001): 161, Servan-Schreiber et al. (2004): 208, Spann & Skiera (2003): 152). Table 1 provide decriptive tatitic regarding the number of obervation and proportion of actual home victorie, draw and away victorie in each ample and eaon. Thee reult roughly match Englih league occer outcome; a Goddard & Aimakopoulo (2004) report, home team win in 45.3% of game, away win occur in 28.0% and draw happen in 26.7% of all game. == Pleae inert Table 1 about here == 3.2 Calculation of the Prediction Market Forecat The PM we invetigate, the Soccer Market, attracted approximately 10,000 total regitered uer, with an average of 1,500 active participant for each tournament round. It uually opened on Thurday at 6:00 p.m., and trading ended five hour later, at 11:00 p.m. on 7

10 Thurday. On Friday, Saturday and Sunday, the Soccer Market remained open for five hour each day during the game, then cloed each tournament round at the end of the lat game on Sunday. The payoff function of each hare of virtual tock depend on the number of league point a occer team gain in one tournament round. In the eaon, the ultimate payoff of a hare of tock of the loing team wa $100; of a drawing team $200, and of the winning team wa $400. The minimum $100 payoff for a lo erve to avoid "penny tock": Win Draw (1) dhome ( Away), g, r, ZHome ( Away), g, r, d Lo d d = $400 = $100 + $100 = = $200, = $100 ( S, r R, g G r, ) for S = 1999/2000. The payoff-rule changed for later eaon. Each hare of tock of a loing team wa $0, of a drawing team $1 and of the winner wa $3. d = $3 Win Draw (2) dhome ( Away), g, r, ZHome ( Away), g, r, d Lo d = $1 = = $1, = $0 ( S, r R, g G r, ) for S = 2000/2001&2001/2002, where d Home ( Away), g, r, : cah dividend of a hare of tock that model the number of league point the home (away) team gain in the g th game in the r th tournament round of the th eaon, Z Home ( Away), g, r, : number of league point the home (away) team gain in the g th game in the r th tournament round of the th eaon, Draw ( Win / Lo ) d : cah dividend of a hare of tock in the cae of a draw (win/lo) in the th eaon, G r, : index et of game in the r th tournament round of th eaon, R : index et of tournament round of th eaon, and S: index et of eaon. 8

11 In each tournament round of the eaon, all participant of the Soccer Market tart with the ame aet: 1,000 hare of each type of team tock and $500,000 (virtual), with the poibility of a maximum virtual loan of $500,000 at a 1% weekly interet rate. For the and eaon, the endowment in each tournament round conited of 1,000 hare of each type of team tock and $5,000 (virtual) cah, with no loan poible. Participant are treated alike, regardle of when they enter the Soccer Market. A participant can trade hare according to hi or her etimation of the game outcome by elling hare of a preumably overvalued team tock or buying hare of a preumably undervalued team tock. Portfolio value from one tournament round do not tranfer to the following tournament round; intead, the incentive involve monetary reward for each round. At the end of each tournament round, the participant with the highet (virtual) portfolio value receive $150 (real), the peron with the econd highet value receive $100, and the third-ranking participant receive $50. There i no rik of actual financial lo. Table 2 follow the recommendation of Spann & Skiera (2003) to decribe the deign of the PM. == Pleae inert Table 2 about here == To determine outcome prediction from the PM, we ue the tock price of the team tock at the end of trading on the firt day, that i, the earliet poible end-of-trading point before the firt game of a tournament round to predict all game of that round. Equation (3) repreent the expected league point of a team in a pecific tournament round in the eaon; Equation (4) decribe the and eaon according to the current tock price: 9

12 (3) Zˆ PM Home ( Away), g, r,, t ( phome ( Away), g, r,, t 100) = 100, ( S, r R, g G r,, t < T ) for S = 1999/2000, and (4) Z ˆ PM Home ( Away), g, r,, t phome ( Away), g, r,, t =, ( S, r R, g G r,, t < T ) for S = 2000/2001&2001/2002, where: ˆ PM Home ( Away), g, r,, t Z : expected gain of league point according to the PM for the home (away) team at the t th point in time in the g th game in the r th tournament round of the th eaon, p : price of a hare of the home (away) team tock at the t th point in Home ( Away), g, r,, t time in the g th game in the r th tournament round of the th eaon, and T: point of time at the end of the game of the home (away) team in the g th game in the r th tournament round of the th eaon. Prediction for game outcome reflect the difference in the tock price of two competing team; we predict a win for the team with the higher tock price. We predict a draw a the game outcome only when the two competing team achieve identical tock price. 1 After determining the price of the home and away team in a pecific game 2 and given that all outcome probabilitie um to 1, we can calculate the pecific outcome probabilitie PR Z for a pecific game (for detail, ee the Appendix): ( / ) ( Draw Home Away gr,, ) Home (5) PR( Z,, ) = ( d d ) ( 2 d d d ) gr Win Lo Draw Win Lo φ, 1 2 We alo ued a le trict definition for the prediction of a draw by alo allowing mall difference in tock price a predicton of a draw. Such variation had very little influence on the reult. To equalize the difference in tock price of competing team in the and eaon with thoe for the eaon, we multiply them by a caling factor of 100. For example, aume the PM expect a home team to gain 2.5 league point and the away team to earn 1.8 league point. In the eaon, the tock price difference would be ( ) ( ) = = 70. However, in the and eaon, the ame prediction yield =.7, which we then multiply by 100 to equal to the difference in the eaon. 10

13 with φ = ( price Lo ) ( Draw Win,,, ) ( Win,,, ) ( Draw Lo Homegr d d d priceawaygr d d d ) Draw Win Lo ( S, r R, g G r,, 2 d d + d, d Win d ). Lo Draw (6) PR( Z,, ) = price d Lo Home, g, r, Draw Win Lo gr Draw Lo d d φ ( 2 d d d ), Draw Win Lo ( S, r R, g G r, ), 2 d d + d, d Win d ). Lo Away Draw Home (7) PR( Z gr,, ) 1 ( PR( Z gr,, ) PR( Z gr,, )) = +, ( S, r R, g G r, ). 3.3 Calculation of the Betting Market Forecat We ue the fixed betting odd of the larget German tate-owned bookmaker (Oddet), which employ decimal odd and charge a fee of 25%, included in the odd. 3 That fee i ubtantially higher than the average margin of approximately 12% in mot European (non tateowned) bookmaker (Vlataki et al., 2007) or the 5% in peron-to-peron betting on betting exchange uch a Betfair (Smith et al., 2006). We derive the bookmaker' forecat from the betting odd by retrieving the implied probability of the different game outcome and tandardizing the probabilitie to 1: (8) b u Draw( Home/ Away) Draw( Home / Away) Draw( Home / Away) gr,, gr,, gr,, = = Draw Home Away u gr,, + ugr,, + ugr,, + + Draw Home Away qg, r, qgr,, qgr,, q 1 ( S, r R, g G r, ), where: Draw( Home / Away) b gr,, : tandardized probability derived from betting odd of a draw (home team win/away team win) in the g th game in the r th tournament round of the th eaon, Draw( Home / Away) u gr,, : untandardized probability derived from betting odd of a draw (home team win/away team win) in the g th game in the r th tournament round of the th eaon, and 3 The data howed that the numerator in (8) i alway equal to 1.25, which indicate a margin of 25%. 11

14 q : betting odd of a draw (home team win/away team win) in the g th Draw( Home / Away) gr,, game in the r th tournament round of the th eaon. Therefore, if the decimal odd of a home win, draw and away win are, repectively, 1.7, 2.8 and 3.3, the tandardized probabilitie are 47.1%, 28.6% and 24.3%. The highet probability determine the forecat for the game outcome. Our reult how that the bookmaker never aign a draw with the highet probability. Furthermore, we calculate the expected gain of league point by the home and away team in a game on the bai of the tandardized probabilitie for each of the three poible game outcome: (9) Zˆ = 3 b + 1 b + 0 b, Odd Home( Away) Draw Away ( Home) Home ( Away), g, r, g, r, g, r, g, r, ( S, r R, g G r,, t < T ). In Table 3, we diplay the hare of outcome predicted by each method for each eaon and all three eaon together. == Pleae inert Table 3 about here == 4 Forecat Accuracy of Three Method 4.1 Evaluation Criteria Our criteria for evaluating and comparing the three forecating method are a follow: 1. We calculate the percentage of hit for each method, that i, the number of correctly predicted game relative to the total number of predicted game. 2. We calculate the root mean quared error (RMSE) for the deviation between the expected and actual gain of league point for each of the two team in every game (N: total number of game in ample): (10) RMSE = S r R g Gr, ( Zˆ,,,,,,) ( ˆ Home g r ZHome g r + ZAway, g, r, ZAway, g, r, ) N

15 3. We calculate the amount of money the prediction of each forecating method would have won on the betting market for three poible fee cenario: (a) with the 25% fee of the (tate-owned) betting company, (b) with a fee of 12%, which i common for mot European (non tate-owned) betting companie and (c) with no fee. The calculated profit in all three cenario indicate the value of each forecating method. Specifically, the winning without a fee (0%) how whether forecating performance i better than the betting odd. The amount after ubtracting the betting company' margin denote whether punter can ue the information to make money in a real-world betting market. The 12% fee reveal whether punter could earn money in a (competitive betting market) ituation with a fee below the monopolitic fee (25%) of the tate-owned betting market in Germany. In addition, we compare the forecat of the three method with thoe of a naïve model and a pure random draw model. The naïve model alway predict a home win, which i the mot frequent game outcome (i.e., the naïve model i not trictly naïve, becaue it ue thi information; Forret & Simmon, 2000). The pure random draw model randomly predict one of the three event with overall probabilitie, which provide a forecating accuracy of (h 2 + d 2 + a 2 ), in which h, d and a are the proportion of home victorie, away victorie and draw in our data et (Forret & Simmon, 2000, p. 321). 4.2 Forecat Accuracy of Each Method In Table 4, we compare the hit rate of the PM, the naïve model, random pick and betting odd for the whole ample of 837 game. The PM yield a hit rate of 52.69%, greater than the total number of home victorie (50.42%) and pure random pick (37.73%). Betting odd have a lightly higher hit rate of 52.93% and a lightly lower RMSE, but lead to lower profit. Difference between the PM and betting odd are inignificant, indicating that the forecat accuracy i comparable. Both method outperform the naïve model of home win. == Pleae inert Table 4 about here == 13

16 Table 5 diplay the hit rate of the PM, betting odd and tipter for the overlapping ample of 678 game. Thi time, the PM achieve a higher hit rate and profit than the betting odd, but alo a higher RMSE. Again, difference between the PM and betting odd are not ignificant and both method ignificantly outperform the tipter and the naïve model. The tipter' prediction are notably poor; even the naïve model clearly outperform them. Thu, the forecating accuracy of the PM and the betting odd i comparable and much better than thoe of the tipter or the naïve model. == Pleae inert Table 5 about here == Thee reult fall in line with the correlation of the prediction (Table 6), for which we code the forecat of a home win a "1", a draw a "0" and an away win a " 1". The correlation between the prediction of the PM and the tipter i.436; that between the PM and naïve model i.216. Therefore, the prediction of thee method differ ubtantially. In contrat, the forecat of the PM and betting odd correlate at.844, indicating their relatively cloe imilaritie. However, the forecat are far from being equal, which indicate that we might be able to exploit thee difference by combining the reult of the different method. == Pleae inert Table 6 about here == 4.3 Forecat Accuracy of Combination of the Method Several tudie how that combining the reult of different forecating method can improve forecating accuracy (Armtrong, 2001; Batchelor & Dua, 1995; Blattberg & Hoch, 1990). Therefore, we tet the forecat accuracy of a weighting-baed combination of forecat (Blattberg & Hoch, 1990), a well a variou rule-baed combination of forecat. 14

17 4.3.1 Accuracy of Weighting-Baed Combined Forecat We follow Blattberg & Hoch (1990), who ugget a 50:50 weighting, thu a imply averaging of forecat in a different etting. Therefore, we averaged the predicted number of league point for the home and away team from the PM and betting odd (ee Equation (11)). We exclude the tipter, which doe not provide a forecat for the expected league point and offer fairly poor prediction. (11) where: Zˆ = 0.5 Zˆ Zˆ, ( S, r R, g G r, ), Comb. PM Odd Home( Away), g, r, Home( Away), g, r, Home( Away), g, r, ˆ Comb.( PM / Odd) Home( Away), g, r, Z : forecat of the weighting-baed combination (PM/betting odd) for the expected league point of the home (away) team of the g th game in the r th tournament round of the th eaon. We ue the difference in the expected league point to predict a win for the team with more expected league point; we predict a draw when the team have identical expected league point. Thi weighting-baed combination etablihe a forecat accuracy of 52.69% (N = 837), equal to the hit rate of the forecat of the PM for all 837 game. It alo yield profit on betting market with 25%, 12% and 0% fee of 13.12%, 0.59% and 11.47%, repectively. Neither the hit rate (one-tailed binomial tet, p >.5) nor the profit (two-tailed paired t-tet, p >.6) differ ignificantly from the forecat of the PM or the betting odd for the ame ample of 837 game (compare Table 4 with Table 7). However, the RMSE of the weighting-baed combination i lower than that of the PM, though higher than that of the betting odd. The reult for the ample of the 678 game are very imilar (compare Table 5, econd row, with Table 7, lat column): Neither the hit rate (one-tailed binomial tet, p >.4) nor the profit (two-tailed paired t-tet, p >.3) lead to ignificantly different reult. Therefore, we conclude that our weighting-baed combination of forecat doe improve the forecat of the PM or the betting odd notably. 15

18 4.3.2 Accuracy of Rule-Baed Combined Forecat Thu far, we have forecat all game, but we might improve forecat accuracy by concentrating on elected game. Thi ituation more accurately reflect the real world; punter can uually deliberately bet on only a elected number of game. Therefore, we analyze the quality of the following rule to elect the game that we want to forecat: 1. Only forecat if the forecat of PM and betting odd are the ame. 2. Only forecat if the forecat of PM and the tipter are the ame. 3. Only forecat if the forecat of betting odd and the tipter are the ame. 4. Only forecat if the forecat of PM, betting odd and the tipter are the ame. Table 7 how the reult. The rule-baed forecat elect between 380 (56.0%) and 778 (93.0%) game in each ample and increae the hit rate to 53.98% (rule 1), 56.85% (rule 2), 56.52% (rule 3) and 57.11% (rule 4). Thu, rule-baed combined forecat increae the hit rate, but none of the improved hit rate i ignificantly different (one-tailed binomial tet) from the hit rate of the PM, that i, 54.28% for the ample of 678 game (Table 5). Rule 4 achieve the highet hit rate (57.11%) but elect the fewet game. Betting $100 on each game would lead to winning of $5,267 (13.42%) if the betting companie do not charge fee. Thi amount i much le than thoe realized for the rule that elect more game and ignificantly le than the total profit of the weighting-baed combined forecat. Total profit are highet for the PM forecat ($10,295 for all 837 game, $10,984 for the overlapping 678 game). Thi total profit i greater than that achieved through weighting-baed combined forecat or relying on betting companie ($9,977 for all 837 game, $9,146 for the overlapping 678 game). Hence, thi reult upport the high forecat accuracy of PM and betting odd. == Pleae inert Table 7 about here == 16

19 5 Summary and Concluion We compare the forecat accuracy of different method, namely, prediction market, tipter and betting odd, a well a weighting-baed and rule-baed combination of thoe forecat. The reult indicate that PM and betting odd yield a comparable and good forecat accuracy. PM would allow punter to make more money on the betting market if the betting company doe not charge fee or at leat doe not charge monopolitic fee. In contrat, tipter forecat are poor, in upport of the reult of previou tudie (Forret & Simmon, 2000). Our finding alo upport reult cited by, among other, Forret et al. (2005), Boulier et al. (2006) and Spann & Skiera (2003), who how that betting odd and prediction market provide very good forecat. Thee previou tudie do not, however, compare the forecating accuracy of thoe method and tipter, wherea we how that PM and betting odd forecat equally well and clearly outperform tipter. Interetingly, PM forecat could yield profit from betting if the betting market charged moderate fee. A weighting-baed combination of the forecat of PM and betting odd lead to a lightly higher forecat accuracy, wherea rule-baed combined forecat improve forecat accuracy ubtantially. However, the latter come at a cot: It predict the reult of fewer game. Still, our reult how that PM can enhance the accuracy of port forecating. 17

20 6 Appendix follow: The price of a hare of tock depend on the probability of the outcome of the game, a (12) (13) where ( ) ( ) Draw Draw ( gr,, ) d Home Win ( gr,, ) d price = 1 PR Z PR Z d + PR Z Draw Home Lo Home, g, r, g, r, g, r, + PR Z ( ) ( ) Draw Draw ( gr,, ) d Home Lo ( gr,, ) d price = 1 PR Z PR Z d + PR Z Draw Home Win Away, g, r, g, r, g, r, + PR Z, ( S, r R, g G r, ), and, ( S, r R, g G r, ), price Home ( Away), g, r, : price of a hare of tock of the home (away) team in the g th game in the r th tournament round of the th eaon, Draw ( / ) d : cah dividend of a hare of tock in cae of a draw (win/lo) in the th eaon, G : index et of game in the r th tournament round of th eaon, r, Draw( Home/ Away) PR( Z gr,, ): Probability of the outcome of the g th game in the r th tournament round of the th eaon (i.e., draw, home win or away win), R : index et of tournament round of th eaon, and S: index et of eaon. Knowing the price price Home, g, r, and price Away, g, r, enable u to calculate the correponding probabilitie of the PM for the outcome PR Z Draw( Home/ Away) ( gr,, ). Thu, we aume PR( Z ) + PR( Z ) + PR( Z ) = 1, and olving Equation (12) and (13) for the two Draw Home Away gr,, gr,, gr,, probabilitie PR( Z Draw g, r, ) and PR( Z Home g, r, ) lead to: (14) price Lo ( Draw ) ( Draw Lo,,,,, ) ( Home,, ) ( Win Lo Home g r d PR Z g r d d PR Z g r d d ) = +, ( S, r R, g G r, ) and 18

21 (15) price Win ( Draw ) ( Draw Win,,,,, ) ( Home,, ) ( Lo Win Away g r d PR Z g r d d PR Z g r d d ) = +, ( S, r R, g G r, ). Solving both equation for the probability PR( Z Draw g, r, ) yield: Draw (16) PR( Z,, ) ( ) ( ) price d PR Z d d = Lo Home Win Lo Home, g, r, g, r, gr Draw Lo d d, ( S, r R, g G r,, d Draw d Lo ) and Draw (17) PR( Z,, ) ( ) ( ) price d PR Z d d Win Home Lo Win Away, g, r, g, r, gr = Draw Win d d, Then, if we ubtract Equation (17) from Equation (16), we obtain: (18) ( S, r R, g G r,, d Draw d ). Win ( ) ( ) ( ) ( ) price d PR Z d d price d PR Z d d = d d d d Lo Home Win Lo Win Home Lo Win Home, g, r, g, r, Away, g, r, g, r, Draw Lo Draw Win Draw Lo d d, ( S, r R, g G r,, which, when rearranged, equal:, d Draw d ), Win (19) Lo Draw Win Win Draw Lo ( pricehome, g, r, d ) ( d d ) ( priceaway, g, r, d ) ( d d ) Home Win Lo Draw Win Lo = PR( Z gr,, ) ( d d ) ( 2 d d d ), ( S, r R, g G r, ). Home Solving for the probability PR( Z g, r, ) yield: Home (20) PR( Z,, ) = Lo Draw Win Win Draw Lo ( pricehomegr,,, d ) ( d d ) ( priceawaygr,,, d ) ( d d ) ( d d ) ( 2 d d d ) gr Win Lo Draw Win Lo, Draw Win Lo ( S, r R, g G r,, 2 d d + d, d Win d ). Lo Then, if we ubtitute Equation (20) into Equation (16), we achieve the probability PR( Z Draw g, r, ): 19

22 Draw (21) PR( Z,, ) = price d Lo Draw Win ( pricehome, g, r, d ) ( d d ) Win Draw Lo ( priceaway, g, r, d ) ( d d ) ( 2 d d d ) Lo Home, g, r, Draw Win Lo gr Draw Lo d d, Draw Win Lo ( S, r R, g G r, ), 2 d d + d, d Win d ). Lo 20

23 Reference Anderon P, Edman J, Ekman M. Predicting the World Cup 2002 in Soccer: Performance and Confidence of Expert and Non-Expert. International Journal of Forecating 2005; 21: Armtrong JS (Ed.) Principle of Forecating. Kluwer Academic Publiher: Dordrecht, Batchelor R, Dua P. Forecater Diverity and the Benefit of Combining Forecat. Management Science 1995; 41: Blattberg RC, Hoch S. Databae Model and Managerial Intuition: 50% Model and 50% Manager. Management Science 1990; 36: Boulier BL, Stekler HO, Amundon S. Teting the Efficiency of the National Football League Betting Market. Applied Economic 2006; 38: Cain M, Law D, Peel D. The Favourite-Longhot Bia and Market Efficiency in UK Football Betting. Scottih Journal of Political Economy 2000; 47: Chen Y, Chu C-H, Mullen T, Pennock DM. Information Market v. Opinion Poll. Paper preented to the Reearch Paper Proceeding of the 6th ACM conference on Electronic Commerce, Vancouver, Canada, Dahan E, Hauer JR. The Virtual Cutomer. Journal of Product Innovation Management 2002; 19: Dahan E, Lo AW, Poggio T, Chan NT, Kim A. Securitie Trading of Concept (STOC). UCLA, Dixon M, Pope P. The Value of Statitical Forecat in the UK Aociation Football Betting Market. International Journal of Forecating 2004; 20: Elbere A. The Power of Star: Do Star Actor Drive the Succe of Movie? Journal of Marketing 2007; 71: Elbere A, Eliahberg J. Demand and Supply Dynamic for Sequentially Releaed Product in International Market. The Cae of Motion Picture. Marketing Science 2003; 22: Fama EF. Efficient Capital Market: A Review of Theory and Empirical Work. Journal of Finance 1970; 25: Fama EF. Efficient Capital Market: II. Journal of Finance 1991; 46: Forret D, Goddard J, Simmon R. Odd-etter a Forecater: The Cae of Englih Football. International Journal of Forecating 2005; 21: Forret D, Simmon R. Forecating Sport: the Bahaviour and Performance of Football Tipter. International Journal of Forecating 2000; 16: Forythe R, Nelon F, Neumann GR, Wright J. Anatomy of an Experimental Political Stock Market. American Economic Review 1992; 82: Forythe R, Rietz TA, Ro TW. Wihe, Expectation and Action: A Survey on Price Formation in Election Stock Market. Journal of Economic Behavior & Organization 1999; 39: Gandar JM, Dare WH, Brown CR, Zuber RA. Informed Trader and Price Variation in the Betting Market for Profeional Baketball Game. Journal of Finance 1998; 53: Goddard J, Aimakopoulo I. Forecating Football Reult and the Efficiency of Fixed-odd Betting. Journal of Forecating 2004; 23: Graham I, Stott H. Predicting Bookmaker Odd and Efficiency for UK Football. Applied Economic 2008; 40:

24 Gruca TS, Berg JE, Cipriano M. The Effect of Electronic Market on Forecat of New Product Succe. Information Sytem Frontier 2003; 5: Hahn B, Tetlock P (Ed.) Information Market: A New Way of Making Deciion. AEI-Brooking Pre, Hayek FAv. The Ue of Knowledge in Society. American Economic Review 1945; 35: Jank W, Foutz N. Uing Virtual Stock Exchange to Forecat Box-Office Revenue via Functional Shape Analyi. Paper preented to the Second Workhop on Prediction Market, San Diego, California, Luckner S, Weinhardt C. How to Pay Trader in Information Market: Reult from a Field Experiment. Journal of Prediction Market 2007; 1: Mill TC, Pepper GT. Aeing the Forecat: An Analyi of Forecating Record of the Treaury, The London Buine School and the National Intitute. International Journal of Forecating 1999; 15: Oliven K, Rietz TA. Sucker Are Born but Market Are Made: Individual Rationality, Arbitrage, and Market Efficiency on an Electronic Future Market. Management Science 2004; 50: Paton D, Vaughan William L. Forecating Outcome in Spread Betting Market: Can Bettor Ue Quarb to Beat the Book? Journal of Forecating 2005; 24: Pennock DM, Lawrence S, Gile LC, Nielen FA. The Real Power of Artificial Market. Science 2001; 291: Plott CR, Wit J, Yang WC. Parimutuel Betting Market a Information Aggregation Device: Experimental Reult. Economic Theory 2003; 22: Pope PF, Peel DA. Information, Price and Efficiency in a Fixed-Odd Betting Market. Economica 1989; 56: Servan-Schreiber E, Pennock DM, Wolfer J, Galebach B. Prediction Market: Doe Money Matter? Electronic Market 2004; 14: Smith MA, Paton D, Vaughan William L. Market Efficiency in Peron-to-Peron Betting. Economica 2006; 73: Smith VL. Microeconomic Sytem a an Experimental Science. American Economic Review 1982; 72: Spann M, Ernt H, Skiera B, Soll JH. Identification of Lead Uer for Conumer Product via Virtual Stock Market. Journal of Product Innovation Management 2007; forthcoming. Spann M, Skiera B. Internet-Baed Virtual Stock Market for Buine Forecating. Management Science 2003; 49: Törngren G, Montgomery H. Wore than Chance? Performance and Confidence among Profeional and Laypeople in the Stock Market. Journal of Behavioral Finance 2004; 5: Tzirali G, Tatiopoulo I. Prediction Market: An Extended Literature Review. Journal of Prediction Market 2007; 1: Vlataki N, Doti G, Markello RN. How Efficient i the European Football Betting Market? Evidence from Arbitrage and Trading Strategie. Athen Univerity of Economic and Buine Working Paper, Wolfer J, Zitzewitz E. Interpreting Prediction Market Price a Probabilitie. Wharton, Univerity of Pennylvania,

25 Table 1: Data ample (actual outcome in each ample) Seaon Sample No. Ob. % home % draw % away 1999/2000 Game predicted by PM and betting odd % 28.13% 25.69% Game predicted by tipter % 27.59% 23.15% Game predicted by PM, betting odd and tipter % 27.59% 23.15% 2000/2001 Game predicted by PM and betting odd % 21.35% 25.84% Game predicted by tipter % 23.65% 27.39% Game predicted by PM, betting odd and tipter % 22.71% 28.02% 2001/2002 Game predicted by PM and betting odd % 21.99% 25.53% Game predicted by tipter % 20.58% 25.99% Game predicted by PM, betting odd and tipter % 20.90% 25.75% All 3 Game predicted by PM and betting odd % 23.89% 25.69% (1999 Game predicted by tipter % 23.58% 25.66% 2002) Game predicted by PM, betting odd and tipter % 23.45% 25.66% 23

26 Table 2: Step Choice of forecating goal Deign of incentive for information revelation Financial market deign Deign of the prediction market Deciion Forecating of occer game outcome in the German premier league Payoff function: gain of league point of home (away) team in a tournament round of the German premier occer league (Equation (1) for eaon and Equation (2) for and eaon) Public acce, poible to join at any time Compoition of Initial Portfolio/Endowment: Endowment of 1,000 hare of each type of team tock and $500,000 [$5,000] (virtual) (each tournament round for every participant) in [ and ] eaon Proviion of loan up to $500,000 (virtual) at a 1% weekly interet rate in eaon; no loan in later eaon Remuneration/Incentive Mechanim: Monetary reward Rank-order tournament; participant with highet increae in (virtual) portfolio value receive $150 in cah, econd highet $100 and third highet $50 Time interval: Rank-order tournament for each tournament round No nonperformance baed incentive Double auction trading mechanim with competitive market maker and open order book Each tournament round: 5 hour of daily trading from Thurday to Sunday No hort trading Order type: limit and market without temporal retriction No poition or price limit No trade fee 24

27 Table 3: Outcome predicted by the three method (actual outcome in each ample) Seaon Sample No. Ob. % home % draw % away 1999/2000 Outcome predicted by PM % 1.74% 22.22% Outcome predicted by betting odd % 0.00% 20.14% Outcome predicted by tipter % 30.54% 18.23% 2000/2001 Outcome predicted by PM % 1.12% 20.97% Outcome predicted by betting odd % 0.00% 15.36% Outcome predicted by tipter % 33.20% 16.18% 2001/2002 Outcome predicted by PM % 2.13% 22.70% Outcome predicted by betting odd % 0.00% 20.92% Outcome predicted by tipter % 28.16% 17.33% All 3 Outcome predicted by PM % 1.67% 21.98% (1999 Outcome predicted by betting odd % 0.00% 18.88% 2002) Outcome predicted by tipter % 30.51% 17.20% 25

28 Table 4: Intrument Comparion of forecating accuracy of prediction market and betting odd Hit rate % improve (p-value) a) RMSE b) Profit c) (25% fee) Profit c) (12% fee) Profit c) (0% fee) Prediction market 52.69% %.27% 12.30% Betting odd 52.93% -.45% (.462) % -.07% 11.92% Naïve model: Home win 50.42% 4.50% (.099) % -.18% 11.79% Random draw model 37.73% 39.65% (.000) n.a. n.a. n.a. n.a. No. Ob. = 837 a) Percentage of improvement of PM over alternative method = [hit rate PM hit rate of alternative method]/hit rate of alternative method (one-tailed binomial tet for difference of hit rate of PM). b) Root mean quared error for the deviation between the expected and actual gain of league point for both team in every game. The naïve model only provide an outcome prediction (home win), from which we derive the expected gain in league point. Thu, the comparability of the RMSE of the naïve model i limited to the RMSE of PM and betting odd prediction, which provide eparate prediction for the expected gain of league point for each team. c) Profit meaured a the (relative) return on betting. 26

29 Table 5: Intrument Comparion of forecat accuracy of different method Hit rate % improve (p-value) b) RMSE c) Profit d) (25% fee) Profit d) (12% fee) Profit d) (0% fee) Prediction market 54.28% % 3.75% 16.20% Betting odd 53.69% 1.10% (.389) % 1.33% 13.49% Tipter 42.63% 27.33% (.000) % % -0.19% Naïve model: Home win 50.88% 6.68% (.041) %.39% 12.44% Random draw model 37.98% 42.92% (.000) n.a. n.a. n.a. n.a. No. Ob. = 678 a) The port journal did not predict all game; therefore, the comparion with the PM and betting odd depend on the ame election of game. b) Percentage of improvement of PM over alternative method = [hit rate PM hit rate of alternative method]/hit rate of alternative method (one-tailed binomial tet for difference to hit rate of PM). c) Root mean quared error for the deviation between the expected and actual gain of league point for both team in every game. However, the tipter and naïve model only provide an outcome prediction (home win, draw or away win), from which we derive the expected gain in league point. Thu, the comparability of the RMSE of the tipter and naïve model i limited to the RMSE of PM and betting odd prediction, which provide eparate prediction for the expected gain of league point for each team. d) Profit meaured a the (relative) return on betting. 27

30 Table 6: Correlation between forecat of different forecating method Intrument PM Betting Odd Tipter Prediction market Betting odd Tipter Home win Actual outcome No. Ob.= 678 a), all correlation ignificant at p <.001 a) The port journal did not predict all game; the comparion with PM and betting odd i baed on the ame election of game. 28

31 Table 7: Intrument Rule-baed and weighting-baed combined forecat i) PM and Betting Odd Agree ii) PM and Tipter Agree Rule-Baed Forecating iii) Betting Odd and Tipter Agree iv) PM, Betting Odd and Tipter Agree Weighting-Baed Forecating 50:50 forecat [N=837] 50:50 forecat [N=678] Sample overlap a) Number of forecat a) Forecat: Home win b) Forecat: Away win b) Forecat: Draw b) Hit rate (%) 53.98% 56.85% 56.52% 57.11% 52.69% 53.98% RMSE c) Profit d) (25% fee) -9.86% -7.64% -9.08% -8.72% % -8.15% Profit d) (12% fee).39% 2.87% 1.26% 1.66% -.59% 2.28% Profit d) (0% fee) 12.44% 15.21% 13.42% 13.86% 11.47% 14.55% Total profit (0%) e) 9,678 $ 5,993 $ 5,247 $ 5,267 $ 10,013 $ 9,865 $ a) Number of game. b) c) d) e) In cae intrument agree, number of game that predict that outcome. Root mean quared error for the deviation between the expected and actual gain of league point for both team in every game. The combination method only provide an outcome prediction (home win, draw or away win), from which we derive the expected gain in league point. Thu, the comparability of the RMSE of the combination method i limited to the RMSE of direct PM and betting odd prediction, which provide eparate prediction for the expected gain of league point for each team. Profit meaured a the (relative) return on betting. Total profit (0%) equal the profit realized by betting $100 on each elected game. 29