1 Predctng Fnancal Markets: Comparng Survey, News, Twtter and Search Engne Data Huna Mao, Indana Unversty-Bloomngton, Scott Counts, Mcrosoft Research, and Johan Bollen, Indana Unversty-Bloomngton arxv:1112.1051v1 [q-fn.st] 5 Dec 2011 Abstract Fnancal market predcton on the bass of onlne sentment trackng has drawn a lot of attenton recently. However, most results n ths emergng doman rely on a unque, partcular combnaton of data sets and sentment trackng tools. Ths makes t dffcult to dsambguate measurement and nstrument effects from factors that are actually nvolved n the apparent relaton between onlne sentment and market values. In ths paper, we survey a range of onlne data sets (Twtter feeds, news headlnes, and volumes of Google search queres) and sentment trackng methods (Twtter Investor Sentment, Negatve News Sentment and Tweet & Google Search volumes of fnancal terms), and compare ther value for fnancal predcton of market ndces such as the Dow Jones Industral Average, tradng volumes, and market volatlty (VIX), as well as gold prces. We also compare the predctve power of tradtonal nvestor sentment survey data,.e. Investor Intellgence and Daly Sentment Index, aganst those of the mentoned set of onlne sentment ndcators. Our results show that tradtonal surveys of Investor Intellgence are laggng ndcators of the fnancal markets. However, weekly Google Insght Search volumes on fnancal search queres do have predctve value. An ndcator of Twtter Investor Sentment and the frequency of occurrence of fnancal terms on Twtter n the prevous 1-2 days are also found to be very statstcally sgnfcant predctors of daly market log return. Survey sentment ndcators are however found not to be statstcally sgnfcant predctors of fnancal market values, once we control for all other mood ndcators as well as the VIX. Index Terms Fnancal predcton, behavoral fnance, sentment analyss, nvestor sentment, Twtter mood, search engne, news meda. I. INTRODUCTION THE effcent market hypothess (EMH) asserts that fnancal market valuatons ncorporate all exstng, new, and even hdden nformaton, snce nvestors act as ratonal agents who seek to maxmze profts. Behavoral fnance [13] has challenged ths noton by emphaszng the mportant role of behavoral and emotonal factors, ncludng socal mood [17], n fnancal decson-makng. As a consequence, measurng nvestor and socal mood has become a key research ssue n fnancal predcton. Tradtonally, publc and nvestor mood are measured by surveys. For example, the Gallup Lfe Evaluaton Index measures the general well-beng of the US publc on a daly bass by conductng a survey across a representatve sample of the US populaton. Investor mood s lkewse assessed by surveys, n whch nvestors or newsletter wrters rate ther current stance on the market, e.g. Daly Investor Sentment 1 and Investor Intellgence 2. In spte of ther popularty, surveys are, 1 www.trade-futures.com 2 http://www.nvestorsntellgence.com/x/us advsors sentment.html however, resource ntensve and thus expensve to conduct, and can be subject to problems related to responder truthfulness [9], [20], ndvdual bases, socal bases, and group thnk. In recent years, researchers have explored a varety of methods to compute ndcators of the publc s sentment and mood state from large-scale onlne data. Ths approach holds consderable promse. Frst, computatonal analyss of publc sentment or mood may be more rapd, accurate and costeffectve to conduct than large-scale surveys. Second, there now exsts consderable support for the clam that the resultng publc mood and sentment ndcators are ndeed vald measurements of publc sentment and mood, even to the degree that they have been found to predct a varety of socoeconomc phenomena, ncludng presdental electons [23], commercal sales [7], [16], and nfluenza epdemcs [8]. It s of consderable nterest to behavoral fnance that a respectable and growng amount of lterature n ths area has shown that computatonal ndcators of publc sentment may also have predctve value wth respect to fnancal market movements [1], [4], [5], [9], [10], [18]. To the best of our knowledge, three dstnct classes of onlne data sources have been nvestgated for fnancal predcton. Frst, news meda content has been shown to be an mportant factor shapng nvestor sentment. For nstance, Tetlock found that hgh levels of pessmsm n the Wall Street precede lower market returns the followng day [21]. Ths effect has also been observed at the level of ndvdual frms, wth hgh negatve sentment forecastng lower frm earnngs [22]. In [19] t was shown that addng textual features of news to a stock predcton system can mprove the forecastng accuracy. Second, web search (query) data has been shown to be related to and even predctve of market fluctuatons. Search volumes of stock names reveal nvestor attenton and nterest, and hgh search volumes thus predct hgher stock prces n the short-term, and prce reversals n the long-term [9]. Also, search volumes of stocks correlate hghly wth tradng volumes of the correspondng stocks, wth peaks of search volume antcpatng peaks of tradng volume by one day or more [5]. Smlar phenomena have been found at the weekly level [18]. Thrd, socal meda feeds are becomng an mportant source of data to support the measurement of nvestor and socal mood extracton. In an early study, Internet stock message boards were studed to predct market volatlty and tradng volumes [1]. In past couple of years, publc mood ndcators extracted from socal networks such as Facebook [14], LveJournal [11] and Twtter [4] have been used to predct stock market fluctuatons.
2 Together these results are hghly suggestve that a varety of web-scale data sources may provde predctve power n fnancal analytcs. However, each of the mentoned nvestgatons uses dfferent types of web data to predct dfferent fnancal ndcators. It s not clear whch mood ndcators constructed from partcular data sources most effectvely capture nvestor mood-related sgnals and thereby provde the best predctve power. In ths paper, we therefore collect multple data sources,.e. surveys, news headlnes, search engne data and Twtter feeds, from whch we defne a varety of sentment ndcators,.e. Survey Investor Sentment, Negatve News Sentment, Google search volumes of fnancal terms, Twtter Investor Sentment and Tweet volumes of fnancal terms. Subsequently, we determne the predctve value of these sentment ndcators over a range of fnancal ndcators,.e. Dow Jones Industral Average prce, tradng volumes, market volatlty (VIX) and the prce of gold. II. DATA COLLECTION AND SENTIMENT ANALYSIS In ths secton we outlne our data collecton methods, and how we computed nvestor sentment ndcators from Twtter, news, and search engne data. A. Survey Data Surveys are the most drect and common method for collectng nvestor sentment. Investor Intellgence (II), publshed by an nvestment servces company, determnes whether opnon n over one hundred ndependent market newsletters ponts towards a bullsh, bearsh or correcton market. II has been avalable at a weekly level datng to 1964. Daly Sentment Index (DSI) provdes daly market sentment readngs on all actve US markets daly snce 1987, and s one of the most popular short-term market sentment ndces for futures traders. Hgh vs. low DSI values of respectvely above 90% or below 10%, suggests that a short-term top or bottom s ether developng, or has been acheved. B. News Meda We chose eght news meda outlets to collect our news data from: Wall Street Journal, Bloomberg, Forbes.com, Reuters Busness& Fnance, BusnessWeek, Fnancal Tmes, CNN- Money and CNBC. These are the top news sources for fnancal traders and nvestors. In order to track recent and featured news from these sources, we followed ther respectve Twtter accounts ( wsjusnews, wsjbreakngnews, wsjmarkets, bloombergnews, bloombergnow, bloomberg, forbes, BusnessWeek, Reuters Busness, reuters bz, fnancaltmes, FnancalTmes, CNNMoney, CNBC ). We then extracted and parsed the URLs from these tweets, savng the story headlnes as our news corpus. Ths approach of usng headlnes s based on prevous research that studed stock prce reacton to news headlnes [6]. Prevous research has demonstrated that negatve mood seems to be more predctve of fnancal market values than postve mood [21]. There are two well-accepted fnancal Frequency 200 150 100 50 0 aganst arrested bad bankruptcy concern crss cut declnes default defct downgrade downgraded downgrades erred fear fears hurt loses loss losses lost recalls recesson Word shutdown slow unemployment weak worry worse worst Fg. 1. Frequency of negatve terms n News headlnes from July 31st to August 9th 2011. lexcons for negatve word dentfcaton. One s the Harvard IV-4 dctonary 3 as used n [21], [22]. The other 4 s developed by Loughran and McDonald n [15], whch s shown to better reflect the tone of fnancal text than the Harvard IV-dctonary. In our paper, we apply the latter fnancal negatve lexcon to our news headlnes. We count the total number of words n a news headlne and take the rato of the number of negatve sentment words to the total number of words n the headlne. Then, we sum the emotonal rato and dvde by the total number of news artcles on the same day, yeldng our Negatve News Sentment score. Fg. 1 shows the example of top negatve fnancal terms of the news headlnes from July 31st to August 9th 2011, when the DJIA dropped whle market volatlty ncreased. As a result words such as downgrade, cut, crss and losses frequently occur n news headlnes n that perod. C. Search Engne Data Prevous research has shown that search volume tself can be a mood ndcator for fnancal market [5], [9], [10], [18]. In [9], t has been shown that the more people search on economc negatve terms such as recesson and bankruptcy, the more pessmstc people feel about the economy. To create a search query-based ndcator of fnancal mood, we took the followng steps. Frst, we downloaded the weekly search volume data for a set of seed queres ncludng dow jones, stock market, stock to buy, stock, bullsh, bearsh, fnancal news and wall street from Google Insghts for Search () 5. s a Google servce that provdes search volume data from January 2004 to the present. Second, to more fully capture search actvty related to the fnancal markets we expanded these seed keywords wth those terms that are top relevant search terms as recommend by. Ths procedure resulted n a lexcon of about 26 fnancal search terms for whch we agan retreved search frequency 3 http://www.wjh.harvard.edu/ nqurer/ 4 http://www.nd.edu/ mcdonald/word Lsts.html 5 http://www.google.com/nsghts/search/
3 ndces, resultng n a tme seres of frequences for all searches contanng those 26 terms as shown n Table I. TABLE I 26 SEARCH TERMS DJIA, Dow,Dow Jones, Dow Jones Industral Average, bearsh, bear market,best stock, bullsh, bull market, fnance, fnance news, fnancal news, fnancal market, long stock, SP500, stock, stock market, stock declne, stock fall, stock market crash, stock market news, stock market today, stock prce, stock to buy, wall street, wall street news today fnancal search terms from July 2010 to September 2011. Second, we calculate the weekly mean over the daly volumes of tweets. Ths step s necessary to compare Twtter (daly) and (weekly) at the same tme scale. Thrd, and fnally, we take the average of the separate, weekly tme seres generated for each ndvdual term, whch yelds a and Tweet volume tme seres over 66 weeks, for the combnaton of all the fnancal search terms. Fg. 2 shows these two tme seres. D. Socal Meda Data The enormous amount of socal meda data that has become avalable n recent years has provded sgnfcant research opportuntes for socal scentsts and computer scentsts. In fact, Twtter, whch s now one of the most popular mcrobloggng servces, has been extensvely used for real tme sentment trackng and publc mood modelng [3], [12]. And ts fnancal predctve power has also been explored. In [2], t has been shown that Twtter content and sentment can be used to forecast box-offce revenues of moves. In [24], the correlaton between emotonal tweets and fnancal market ndcators are studed, ndcatng that the percentage of emotonal tweets s sgnfcantly negatvely correlated wth Dow Jones, NASDAQ and S&P500 values, but postvely correlated wth VIX values. Moreover, n [4] a sx-dmensonal model of publc emotons s derved from Twtter (Calm, Alert, Sure, Vtal, Knd and Happy) and found to have sgnfcant predctve power wth respect to DJIA fluctuatons. In ths paper, we use a 15%-30% random sample of all publc tweets posted every day from July 2010 to September 2011. From ths collecton, we defne two Twtter-based fnancal mood ndcators: Twtter Investor Sentment (TIS) and Tweet volumes of fnancal search terms (TV-FST). These are dscussed n greater detal below. 1) Twtter Investor Sentment: We smply defne a tweet as bullsh f t contans the term bullsh, and bearsh f t contans the bearsh. On the bass of the number of Bearsh and Bullsh tweets on a gven day, we defne the nvestor sentment score, Twtter Investor Sentment (T IS) on day t, denoted T IS t as follows: T IS t = N bull N bull + N bear (1) where N bull s the number of bullsh tweets on day t and N bear s the number of bearsh tweets on day t. 2) Tweet Volumes of Fnancal Search Terms (TV-FST): As mentoned n Secton II-C, search query volume of stock names and varous fnancal/economc terms has been used n prevous research as proxes of publc and nvestor mood. Our proposal s to apply a smlar approach to defne our Tweet Volumes of Fnancal Search Terms ndcator (TV-FST). We want to compare Tweet volumes and Search volumes of the same search queres. To do so, we use the followng procedure for data processng: Frst, we compute both the weekly Google search volumes () and daly Tweet volumes of those 26 TV FST 100 200 300 400 500 600 Fg. 2. TV FST Jul 03 10 Sep 25 10 Dec 18 10 Mar 12 11 Jun 04 11 Aug 27 11 Weekly TV-FST vs.. Date A correlaton analyss over all weekly values of the two tme seres reveals a statstcally sgnfcant Pearson correlaton coeffcent of 0.62 (p < 0.01). To see whether these two ndcators sgnal notable movement n the fnancal market, we marked the tme perod from July 23rd to August 20th 2011 n a shaded rectangle as shown n Fg. 2. Durng ths perod, the stock market had a huge declne (.e., the DJIA declned 1864 ponts between July 22nd and August 19th 2011.) We can see that from June 4th, 2011 (at the frst vertcal lne), TV-FST values started to ncrease, whle 5 weeks later, on July 9th 2011 (at the second vertcal lne), followed. Ths suggests that may be less effcent than Twtter n revealng publc/nvestor negatve sentment. E. Economc and Fnancal Market Data We collected daly and weekly Dow Jones Industral Average, tradng volume, Volatlty (VIX) from Yahoo! Fnance. In addton, we calculate the market log returns R of stock prces S(t) over a tme nterval t as follows: R t = logs(t + t) logs(t) (2) Here t = 1. Addtonally, we also retreved the prce of gold 6 over the same perod of tme. Table II summarzes the correspondng tme range and daly/weekly scale for all the data we obtaned. 6 http://www.gold.org/nvestment/statstcs/gold prce chart/ 20 30 40 50 60
4 TABLE II TIME-RANGE COVERAGE OF DIFFERENT DATA SOURCES. Data Daly(mm/dd/yy) Weekly(mm/yy) DSI (Daly Sentment Index) 07/01/10-09/05/11 / II (Investor Intellgence) / 01/08-09/11 TIS (Twtter Investor Sentment) 07/01/10 09/29/11 / TV-FST (Tweet volumes of fnancal search terms) 07/01/10 09/29/11 / NNS(Negatve News Sentment) 07/01/10 09/29/11 / (Google Insght Search) / 01/08-09/11 DJIA/VIX/Volume/Gold 07/01/10 09/29/11 01/08-09/11 III. SEARCH VOLUME ()-BASED PREDICTION OF FINANCIAL INDICATORS A. Search Volume and Fnancal Indcator Correlatons In ths secton, we compare the tme seres (search query volume of 26 fnancal search terms) wth the DJIA prce, volume, and the prce of gold from January 2008 to September 2011, roughly 196 weeks. Ths perod was punctuated by sgnfcant market volatlty, as well as sgnfcant bear and bull markets, thus allowng us to perform our analyss under a varety of market condtons. We frst compute the par-wse correlaton between our 26 tme seres of search terms and the fnancal tme seres. All tme seres are transformed to log scale for analyss. The results are summarzed n Table III. Due to the space lmtatons, we only lst the correlatons of 10 search terms. TABLE III PEARSON CORRELATION COEFFICIENTS BETWEEN AND VIX, DJIA, TRADING VOLUME. Search Query VIX DJIA Volume DJIA 0.88-0.76 0.69 Dow Jones 0.84-0.69 0.68 Dow 0.83-0.67 0.68 Dow Jones Industral Average 0.78-0.77 0.65 Stock market news 0.77-0.37 0.59 Fnance 0.71-0.50 0.70 Stock market today 0.69-0.62 0.51 Fnancal news 0.68-0.43 0.57 Stock 0.66-0.38 0.57 SP500 0.65-0.34 0.49 We fnd relatvely strong correlatons n most cases, especally for what seem to be DJIA-relevant search terms such as DJIA, Dow Jones, etc. The tme seres has a postve correlaton wth the VIX and tradng volumes, but negatve correlatons wth DJIA, whch may ndcate that as more people search on fnancal terms, the market wll be more volatle (.e. hgh VIX), and tradng volumes wll be hgher, whle DJIA prces wll move lower. For further testng, we keep the top search term whose search volume has the hghest correlaton wth the correspondng fnancal ndex for each tme seres. In Fg. 3, we overlad the resultng tme seres wth the mentoned fnancal ndcators to vsually examne the occurrence of any partcular trend. The top panels of Fg. 3 show the actual tme seres whereas the lower panels show the scatter plot of values vs. fnancal ndcator values n log-log scale. A smple vsual nspecton of the top panels reveal a clear correlaton between search term volumes and the fnancal ndcator tme seres; peaks n values generally co-occur wth those of VIX and Volume values, and n some cases even precede the peaks of the varous fnancal tme seres (DJIA, Gold). The scatter plots n Fg. 3 show that search volumes exhbt a hgh postve correlaton wth VIX and tradng volume (γ = 0.88, γ = 0.70), and a hgh negatve correlaton wth DJIA prce (γ = 0.77). The correlaton between gold prce and search volumes on gold s also satsfactory (γ = 0.45). Ths correlaton value may n fact be an underestmaton due to nonlnear patterns n how the two varables relate. For log(gold prces) > 7.0 we do observe a lnear pattern of correlaton. Below that value there seems to be lttle to no correlaton. Ths pattern s confrmed by the trend plot at the upper rght of Fg. 3: from md-2010 to the end, at hgher gold prces, we ndeed observe a strong postve correlaton, and n fact two spkes of search volumes appear before the gold prce reached ts peak n early September 2011. z score z score VIX DJIAClose 2 0 2 4 2 0 2 4 6 3.0 3.5 4.0 8.8 9.0 9.2 9.4 VIX VIX (DJIA) 2008 2009 2010 2011 DJIA DJIA (dow jones ndustral average) 2008 2009 2010 2011 cor=0.88 2.0 2.5 3.0 3.5 4.0 4.5 cor= 0.77 1.5 2.0 2.5 3.0 3.5 4.0 4.5 z score z score Gold Prce 2 0 2 4 2 0 2 4 6.6 7.0 7.4 Gold Gold Prce (gold) 2008 2009 2010 2011 Tradng Volume DJIA Volume (fnance) 2008 2009 2010 2011 cor=0.45 3.0 3.5 4.0 4.5 cor=0.70 3.8 4.0 4.2 4.4 4.6 Fg. 3. Trend analyss and log scale scatter plots of tme seres vs. fnancal ndcators such as VIX, DJIA closng values, gold prce and DJIA tradng volume. (Search query terms are nsde the brackets). VIX s a wdely used measure of market rsk and s often referred to as the nvestor fear gauge. Our results show that search volumes of fnancal terms reflect VIX fluctuatons, mplyng that search volume for key fnancal terms may be a computatonal gauge of nvestor fear. To evaluate tme-lag correlatons between search volume and fnancal tme seres, we compute ther cross-correlaton. In order to compare the effectveness of search volumes wth the survey data wth respect to how well they predct the fnancal markets, we also nclude the Investor Intellgence (II) tme seres n our analyss. DJIA Volume 21.5 22.5
5 Consder two seres x = {x 1,..., x n } and y = {y 1,..., y n }, the cross correlaton γ at lag k s then defned as: γ = (x +k) x)(y ȳ) (x +k) x) 2 (y (3) ȳ) 2 where x and ȳ are the sample mean values of the x and y, respectvely. We use the cross-correlaton functon provded n ccf, an R statstcs package. For example, where ccf(x, y) estmates the correlaton between x[t + k] and y[t], t means that we keep y stll, but move x forward or backward n tme by a lag of k. Where k > 0, t means y antcpates x, and vce versa. As can be seen n Fg. 4, DJIA values and (search volume) exhbt the hghest correlaton and partcularly so on the rght sde of the graph where lag values are postve,.e. k > 0, and, n other words, values lead DJIA values. A smlar effect can be observed for vs. VIX values, especally where k = [+1, +3] weeks. In contrast, as shown n Fg. 4, the cross correlaton between II and VIX seems to work n the opposte drecton, ndcatng that VIX leads changes n II values. The correlaton coeffcents at both sdes seem to be roughly balanced for tradng volume. The search query tme seres for gold exhbts the opposte effect of other search query tme seres: search volumes on gold do not lead gold prces. Ths runs counter to our earler observaton (n Fg. 3) that spkes of gold search volumes precede spkes n gold prces, ndcatng that gold may yet have predctve value under certan condtons. We speculate ths may be due to a non-lnear nteracton wth absolute gold prce levels, but we leave ths for future exploraton. that X(t) does not help predct,.e. Granger-cause, Y (t). The alternatve hypothess s that addng X(t) does help predct Y (t). An F-test s conducted to examne f the null hypothess can be rejected. We cauton that Granger causalty analyss mght establsh that the lagged value of X(t) exhbts a statstcally sgnfcant correlaton wth Y (t). However, correlaton does not prove causaton. In other words, Granger causalty testng does not establsh actual causalty, merely a statstcal pattern of lagged correlaton. Ths s smlar to the observaton that cloud cover may precede ran and may thus be used to predct ran, but does not tself actually cause ran. Table IV presents the results of applyng the Granger causalty test n two drectons,.e. wth postve and negatve lags, reflectng the hypothess that each tme seres may Granger cause the other. TABLE IV STATISTICAL SIGNIFICANCE (P-VALUES) OF GRANGER CAUSALITY ANALYSIS BETWEEN SEARCH VOLUMES/ II AND FINANCIAL INDICATORS OVER LAGS OF 1, 2, AND 3 WEEKS. 1 2 3 VIX 0.0051 0.0004 0.0010 VIX 0.0025 0.0202 0.0091 VIX II 8.04e-05 3.63e-07 9.98e-08 II VIX 0.398 0.726 0.849 DJIA 0.207 0.040 0.096 DJIA 7.85e-04 1.48e-03 9.31e-04 Volume 0.409 0.705 0.843 Volume 0.020 0.028 0.101 Gold 0.055 0.104 0.082 Gold 0.139 0.00036 0.0013 (p value < 0.01:, p value < 0.05:, p value < 0.1: ) Correlaton Coreffcent 0.2 0.4 0.6 0.8 Fnancal Value leads ccf(djia,) ccf(volume,) ccf(vix,) ccf(gold,) ccf(vix,ii) 10 5 0 5 10 Lag (weeks) leads Fnancal Value The values n the frst column of Table IV represent the partcular hypothess under consderaton. For example, VIX represents the null hypothess that addng VIX does not help predct. As can be seen from the lsted p- values, ths partcular null-hypothess s rejected wth a hgh level of confdence. In the row below, we observe that addng can also help predct VIX. However, the Granger causalty between Investor Intellgence (II) and VIX runs n only one drecton,.e. VIX II: addng survey data (II) does not help predct VIX. In addton, the null hypothess that addng does not help predct DJIA, s strongly rejected at a hgh level of confdence level. Smlarly, we fnd a very sgnfcant p- value for Gold at lag 2 and 3 weeks. of the prevous 1 to 2 weeks sgnfcantly Granger-cause tradng volume. Fg. 4. Cross correlaton analyss between fnancal tme seres and search volume () tme seres. B. Granger Causalty Analyss We further refne the observatons dscussed above by a Granger causalty test, a technque that s wdely used to analyze the relatons between economc tme seres. The Granger causalty test s a statstcal hypothess test to determne whether a tme seres X(t) s useful n forecastng another tme seres Y (t) by attemptng to reject the null hypothess C. Forecastng Analyss Can search volumes predct future values of fnancal ndcators? As a further valdaton, we conduct a 1-step ahead predcton over 20 weeks based on a baselne model, denoted M 0, and an advanced model, denoted M 1. Here Y represents the partcular fnancal ndex (.e. DJIA, tradng volumes or VIX) and X represents a sentment ndcator. In ths secton we wll focus on n partcular. M 0 : Y t = α + β Y t + ɛ t (4) =1
6 M 1 : Y t = α + β Y t + =1 γ X t + ɛ t (5) Forecastng accuracy s measured n terms of the Mean Absolute Percentage Error (MAPE) and the drecton accuracy. The MAPE s defned as follows: =1 Predcton Error n Percent 12 Model 1 (MAPE=4.15%) Model 0 (MAPE=4.56%) 10 8 6 4 2 0 VIX Predcton Error n MAP E = y ŷ y 100 (6) n where ŷ s the predcted value and y s the actual value. Drecton accuracy s measured smply n terms of whether ( y,t+1 ˆ y,t ) (y,t+1 y,t ) > 0. In other words, f the dfference between today s and yesterday s predcted value has the same sgn as the dfference between today s vs. yesterday s observed value, we conclude that the drecton of the change was predcted accurately for that day. Our search volume and fnancal ndcator tme seres are avalable from January 2008 to September 2011. There are 196 weeks n total and we use the last 20 weeks,.e. May 21st 2011 to October 1st 2011, as the predctng perod. Each forecast uses only the nformaton avalable up to the tme the forecast s made. The raw data are transformed to log scale before predcton. For VIX and DJIA predcton, the lag n s chosen to be 3 weeks. However, accordng to the Granger test analyss shown n Table IV, the p value s not sgnfcant for lags > 2 weeks n the case of vs. tradng volume. We therefore chose n = 2 n Eq. 4 and Eq. 5 for tradng volume predcton. Fg. 5 shows the predcton errors for these 20 forecastng weeks. Table V shows the forecastng errors expressed as MAPE and drecton accuracy. TABLE V FORECASTING ACCURACY OF USING WEEKLY SEARCH VOLUMES TO PREDICT FINANCIAL INDICATORS (DJIA, VOLUME AND VIX). DJIA Volume VIX Model MAPE Drecton Model 0 0.253 0.55 Model 1 0.244 0.70 Model 0 0.386 0.55 Model 1 0.366 0.55 Model 0 4.560 0.55 Model 1 4.148 0.65 From these results t appears that addng search volumes (1) reduces the MAPE predcton error for VIX, DJIA and tradng volumes predctons, and (2) mproves the drecton accuracy for DJIA and VIX forecastng, but not for tradng volumes. Fg. 5 furthermore shows that durng several weeks the baselne model output outperformed the advanced model. Ths agan hghlghts the dffculty of fnancal market predcton, even usng data that has been shown to have statstcally sgnfcant Granger causalty wth the partcular fnancal ndcators. We offer the observaton that on August 15th 2011 (hghlghted wth a yellow bar), the predcton error of the advanced model (red) dropped well below that of the baselne model (blue). In that perod (August 15th -19th) the weekly VIX reached a hgh value of 43.05, the DJIA decreased over 450 ponts, and tradng volumes ncreased sgnfcantly compared to the prevous week. Ths s suggestve that search volumes of fnancal terms may be partcularly useful for predcton when Predcton Error n Percent Predcton Error n Percent 0.7 Model 1 (MAPE=0.24%) Model 0 (MAPE=0.25%) 0.6 0.5 0.4 0.3 0.2 0.1 0.0 1.2 1.0 0.8 0.6 0.4 0.2 0.0 Fg. 5. 1.4 Model 1 (MAPE=0.37%) Model 0 (MAPE=0.39%) 2011 05 16 2011 05 23 2011 05 31 2011 06 06 2011 06 13 2011 06 20 Predcton Error Plot. 2011 06 27 2011 07 05 DJIA Predcton Error DJIA Volume Predcton Error the market experences hgh degrees of volatlty, sgnfcant changes n values and hgh tradng volumes. IV. TWITTER, SEARCH ENGINE, NEWS MEDIA AND SURVEY-BASED PREDICTION OF FINANCIAL INDICATORS A. Correlaton Analyss In prevous sectons we focused on weekly analyss due to data avalablty. However, our Twtter data and the Daly Sentment Index (DSI) were recorded daly from July 1st 2010 to September 29th 2011, for a total of 456 days. Gven the avalablty of daly data, n ths secton our analyss wll focus on daly tme seres, rather than weekly. Agan, Google Insght Search () does not provde daly volume search data. We therefore do not use search volumes n our daly analyses, and nstead use the Tweet volumes of fnancal search terms (TV-FST), as ntroduced n Secton II-D2. In total, we examne four daly sentment ndcators,.e. Twtter Investor Sentment (TIS), Tweet Volume of Fnancal Search Terms (TV-FST), Negatve News Sentment (NNS) and Daly Sentment Index (DSI). Usng the same defnton as shown n Secton II-D2, the TV-FST s calculated as the average of Tweet volumes of all these fnancal search terms. Table VI dsplays the Pearson correlaton values observed between these sentment ndcators. Survey data, DSI (percentage of bullsh readngs), has a postve correlaton wth TIS, but negatve correlatons wth the other two sentment ndcators: TV-FST and NNS. TV-FST exhbts a negatve correlaton wth DSI and TIS, but a postve correlaton wth NNS, whch suggests that TV-FST may be a 2011 07 11 2011 07 18 2011 07 25 2011 08 01 2011 08 08 2011 08 15 2011 08 22 2011 08 29 2011 09 06 2011 09 12 2011 09 19 2011 09 26
7 TABLE VI TIS, NNS, TV-FST, AND DSI CORRELATIONS. TIS NNS TV-FST DSI TIS 1 NNS -0.237 1 TV-FST -0.304 0.225 1 DSI 0.431-0.322-0.202 1 DJIA 10000 11000 12000 bearsh/negatve sentment ndcator. All lsted correlatons are statstcally sgnfcant wth p value < 0.01. After lnearly extrapolatng fnancal ndcators values mssng on weekends (because of markets closng), we analyze the correlaton between these sentment ndcators and fnancal market ndexes. The results are shown n Table VII. TABLE VII CORRELATIONS BETWEEN SENTIMENT AND FINANCIAL INDICATORS. DJIA Log return Volume VIX TIS -0.071 0.267-0.127-0.314 NNS 0.147-0.147 0.039 0.237 TV-FST 0.449-0.091 0.096 0.183 DSI 0.277 0.181-0.341-0.832 ( ndcates p value < 0.01) News Sentment 0.016 0.018 0.020 0.022 0.024 TIS 0.75 0.70 0.65 0.60 TIS(30 day MA) DSI(30 day MA) NNS(30 day MA) TV FST(30 day MA) 100 200 300 400 500 TV FST 80 60 DSI 40 20 We observe that TIS s postvely correlated wth market log returns (cf. Eq. 2) and negatvely correlated wth VIX. DSI s postvely correlated wth DJIA closng values, as well as log return, but negatvely correlated wth tradng volume and VIX. VIX reflects perceved market rsk, wth hgher VIX values potentally ndcatng greater levels of nvestor fear. Its negatve correlaton wth DSI and TIS may therefore ndcate that the latter correspond to postve sentment, or a lower percepton of rsk or fear among nvestors. Conversely, the postve correlaton between VIX vs. NNS and TV-FST may ndcate that these are ndeed ndcators of fear or negatve sentment. To better vew the correlaton between the sentment ndcators and fnancal market, we plot the tme seres of DJIA and four sentment ndcators, n Fg. 6. In Fg. 6, the tme seres n the top panel shows the daly DJIA closng value from July 1st 2010 to September 29th 2011. The four tme seres n the lower panel represent TIS, DSI, NNS and TV-FST durng the same tme perod and they are smoothed over the past 30 days. We nvert the TIS and DSI to make them consstent wth the drectonalty of the other two negatve market ndcators (.e. NNS and TV-FST). As such, up means negatve sentment, whle down ndcates postve sentment. We marked fve tme perods n the lower panel of Fg. 6 wth rectangle bars to ndcate when DJIA prces fell n August and November 2010, and March, June and August 2011. Before DJIA prces fell n August 2010 (ndcated by the frst rectangle bar), t can be seen that the TIS and NNS graphs moved upwards (.e. a rse n negatve sentment), whle DSI dropped (.e. postve). Before the second bar (November 2010), we see TIS and TV-FST trendng upward. Before the fall n DJIA prces n March 2011 (thrd bar) we observe a clear and long-term ncrease of TV-FST, NNS and TIS values. TIS and TV-FST values are trendng upwards before the fourth Fg. 6. Jul 01 Aug 22 Oct 15 Dec 08 Jan 31 Mar 26 May 21 Jul 12 Sep 02 Date Tme seres of DJIA and TIS, DSI, NNS as well as TV-FST. bar that marks June 2011. All four sentment ndcators trend upwards before the last bar that makes August 2011, but the up trend of DSI seem to lag the up trend of NNS and TV-FST. In concluson, though there s consderable nose n the daly data, the non-survey sentment ndcators, especally TIS and TV-FST, do show sgnfcant ncreases n negatve sentment that clearly precede perods of fallng DJIA prces. B. Granger Causalty Analyss The predcton of stock market returns s a matter of consderable nterest. To determne whether any of our sentment ndcators are useful to predct daly DJIA log returns, we conduct a Granger causalty analyss smlar to Secton III. Accordng to Table VII, the correlaton coeffcent between TV-FST and log returns s statstcally nsgnfcant (γ = 0.09). To determne whch of our set of search terms are most effectve to predct log returns, we conduct a correlaton analyss between the search volumes of each fnancal term ndvdually and log returns. Then, we select the search terms 7 whose search volumes exhbt the most sgnfcant correlatons wth log returns, and take the average of ther tme seres to be the TV-FST. The correlaton coeffcent between the resultng TV-FST and daly log returns s -0.30 wth a p-value < 0.01. Table VIII lsts the p-values for a number of b-drectonal Granger causalty tests of log returns vs. our sentment ndcators. We fnd statstcally sgnfcant Granger causaton n both drectons between log returns and TIS, NNS, and TV-FST, 7 DJIA, dow, Dow Jones, Dow Jones Industral Average, SP500, stock(s) fall(s), stocks declne, fnancal market.
8 wth the exceptons of lag = 1, TV-FST Return, and lag = 3, 5, Return TV-FST. No statstcally sgnfcant Granger causaton was observed between DSI and log returns. These results ndcate that sentment ndcators extracted from Twtter (TIS and TV-FST ) and News headlnes (NNS) are predctve to the DJIA log return, but DSI s not predctve. C. Multple Regresson Analyss In ths secton, we conduct a multple regresson for daly log returns obtaned accordng to Eq. 2. The regresson nputs are our four sentment ndcators and the past fnancal values of log return. As an addtonal control, we nclude VIX, snce t s a well-accepted predctor for market return. The multple regresson model s shown n Eq. 7, where n = 7 days, and Y represents the daly log return. In order to mantan a common scale, we normalzed all data to standard scores. Y t = α + β Y t + φ T V F ST t + χ T IS t + γ DSI t + δ NNS t + η V IX t + ɛ t Table IX provdes the summary statstcs of the multple regresson. Compared wth the baselne model, the adjusted R 2 mproves from 0.092 to 0.200. Ths means that an addtonal 11% of the varaton n log returns s accounted for by addng these sentment ndcators. After controllng for all other varables, we fnd that DSI s not a statstcally sgnfcant predctor. The two sentment ndcators extracted from Twtter,.e. TIS and TV-FST, are however very sgnfcant predctors at a lag of 1 to 2 days. Here, we observe a reversal effect, namely that daly log returns are postvely assocated wth TIS and TV-FST on the prevous day, but negatvely correlated wth those on the lag of 2 days. VIX values at lags of 2 days are hghly statstcally sgnfcant predctors of log return. NNS s also a statstcally sgnfcant predctor at lags rangng from 1 or 4 days, but wth much less lower coeffcents, e.g. at a lag= 1 we fnd that the p value = 0.08, and the coeffcent s -0.087, whch means we expect to see a log return decrease of only 0.087 standard devatons for each one standard devaton ncrease of NNS. D. Forecastng analyss To further test the hypothess that addng sentment ndcators can help predct fnancal ndcators such as the DJIA, tradng volumes, and VIX, we conduct a 1-step forecastng test over 30 days,.e. from August 31st 2011 to September 29th 2011. As wth the weekly predcton n Secton III-C, the baselne model s based on ts own hstorcal fnancal values (cf. Model 0 n Eq. 4) whereas the advanced model (cf. Model 1 n Eq.5) adds the hstorcal values of the sentment ndcators TIS, NNS, TV-FST and DSI. Here we assume n = 7. The forecastng accuracy s measured n terms of the Mean Absolute Percentage Error (MAPE) and the drecton accuracy. Results are shown n Table X. (7) TABLE X FORECASTING ACCURACY OF USING TIS, NNS, TV-FST AND DSI TO PREDICT FINANCIAL INDICATORS (DJIA, TRADING VOLUME AND VIX). DJIA Volume VIX Model MAPE Drecton Model 0 1.00 0.5 Model 1 0.97 0.63 Model 0 7.24 0.47 Model 1 7.56 0.60 Model 0 4.00 0.6 Model 1 3.88 0.67 We fnd mprovements n the drecton accuracy and MAPE of the forecastng accuracy for DJIA, VIX and volume predcton, wth the excepton of the MAPE for volume predcton. However, the mprovement s not hghly sgnfcant. The extremely hgh volatlty n the fnancal markets durng our tranng and testng perods, especally n August and September 2011, may account for ths. In addton, we used relatvely smple lnear models n ths paper that may not be suted to model the complex nteractons of factors nvolved n shapng fnancal market values. Further research wll need to focus on the development of more accurate and more advanced lnear/non-lnear predcton models. V. CONCLUSION Behavoral fnance challenges the Effcent Market Hypothess by emphaszng the mportant role that human emoton, sentment and mood play n fnancal decson-makng. Thus places the accurate measurement of sentment and mood at the heart of a dscusson over how to best model and predct the behavor of the fnancal markets. Prevous research n ths doman has reled manly on surveys or news analyss to obtan nvestor sentment. Research has recently started to leverage very large-scale web data, ncludng search engne and socal meda data, to assess publc as well as nvestor sentment. However, most exstng work adopts only a sngle data source (survey, socal meda or search engne data) as a proxy to publc and nvestor sentment, and then uses t to computer a partcular fnancal ndex. To the best of our knowledge, no work has been done to perform a detaled survey of a varety of dfferent classes of mood ndcators extracted from a varety of classes of data sources. Studyng the relatons between dfferent mood ndcators and ther predctve relatonshps to dfferent fnancal ndexes s necessary to unravel the causal relatons sentment and mood relate to the fnancal markets, and thus crucal n mprove fnancal forecastng models. Our paper s a frst, prelmnary contrbuton of such a comparson to the rapdly emergng doman of computatonal behavoral fnance. In ths paper, we collect sx sentment ndcators from nvestor sentment surveys (II and DSI), socal meda (Twtter), news meda servces, as well as search engne (Google). Those nclude DSI bullsh percentage, Investor Intellgence (II), Twtter Investor Sentment (TIS), Tweet volumes of fnancal search terms (TV-FST), Negatve News Sentment (NNS) and Google search volumes of fnancal search terms (). Frst, n a weekly analyss, we fnd a sgnfcant correlaton between weekly of fnancal terms wth DJIA closng
9 TABLE VIII STATISTICAL SIGNIFICANCE (P-VALUES) OF GRANGER CAUSALITY ANALYSIS BETWEEN DAILY LOG RETURN AND TIS, NNS, DSI AND TV-FST. 1 2 3 4 5 TIS Return <0.001 0.0086 0.035 0.028 0.021 Return <0.001 <0.001 <0.001 <0.001 <0.001 NNS Return 0.017 0.030 0.011 0.005 0.004 Return NNS 0.014 0.013 0.031 0.030 0.055 DSI Return 0.523 0.138 0.203 0.308 0.377 Return DSI 0.267 0.174 0.647 0.377 0.059 TV-FST Return 0.413 <0.001 <0.001 <0.001 <0.001 Return TV-FST 0.0025 0.019 0.151 0.071 0.140 (p value < 0.01:, p value < 0.05:, p value < 0.1: ) TABLE IX SUMMARY STATISTICS OF MULTIPLE REGRESSION. Lag Return TIS NNS DSI VIX TV-FST Coeff. p-value Coeff. p-value Coeff. p-value Coeff. p-value Coeff. p-value Coeff. p-value 1 0.282 0.004 0.170 0.008-0.087 0.080 0.389 0.385 0.494 0.182 0.235 0.0007 2-0.139 0.175-0.164 0.018-0.066 0.191-0.239 0.709-1.161 0.029-0.324 3.73e-05 3 0.0006 0.995 0.069 0.316 0.048 0.349-0.265 0.678 0.641 0.235 0.059 0.486 4-0.115 0.275-0.088 0.208-0.097 0.058 0.730 0.251 0.384 0.479-0.059 0.490 5-0.0212 0.837 0.152 0.031-0.017 0.740 0.177 0.780 0.502 0.351 0.171 0.045 6 0.071 0.472-0.132 0.062 0.057 0.257-0.514 0.405-0.490 0.341-0.096 0.261 7-0.117 0.040 0.005 0.935 0.008 0.874-0.115 0.789-0.204 0.559-0.024 0.743 (p value < 0.01:, p value < 0.05:, p value < 0.1: ) Resdual standard error: 0.893 on 406 degrees of freedom. Multple R-squared: 0.2841, Adjusted R-squared: 0.200 (baselne model (cf. Equaton 4, Y s daly log return here): Adjusted R-squared 0.092). F -statstc: 3.67 on 42 and 406 DF, p-value: 5.523e-12 values, tradng volume, and VIX values. Granger causalty tests confrm that s ndeed predctve of fnancal ndcators, but surveys of nvestor sentment (.e. II) are not. Weekly forecastng accuracy was mproved by addng search volumes of fnancal terms, notably so when the DJIA was trendng downward and the VIX was ndcatng hgh volatlty, such as n August 2011. Second, n our daly analyss, all mood ndcators exhbted a sgnfcant correlaton wth log returns and VIX values. Controllng for other mood ndcators ncludng the VIX, we fnd that TIS and the TV-FST values of the prevous 1-2 days are very statstcally sgnfcant predctors of daly market returns, whle DSI s not. NNS s also found to be a statstcally sgnfcant predctor, however, compared to the TIS and TV- FST, we fnd less sgnfcant predctablty of log return. Ths fndng ndcates that the predctve power of Twtter s two sentment ndcators outperformed survey sentment as well as news meda analyss. Moreover, we found that before the hghly downward movement of DJIA n the end of July and August 2011, Tweet volumes of fnancal terms started to ncrease several weeks earler than Google volumes dd. Ths ndcates a potental effcency gan of Twtter over. Studyng the predctve power of the onlne web data s stll n ts nfancy. The varous correlatons and lmtatons of these dfferent data sources, dfferent sentment measures, and ts general predcton applcablty to dfferent domans reman unclear. Our work s the frst attempt to extract a range of sentment ndcators from several popular data sources (Twtter, search engne and news) and use varous sentment ndcators to predct dfferent fnancal ndexes (DJIA, tradng volumes, VIX and gold) n both daly and weekly scale. Contnued research s needed to deepen our understandng of why and how these dfferent mood ndcators relate to and predct soco-economc phenomena such as the fnancal markets. ACKNOWLEDGEMENTS We would lke to express our grattude to trade-futures and Investor Intellgence for makng ther sentment data avalable to our research program. We also thank the Soconomcs Insttute for ther knd advce on a range of research questons, and ther assstance n obtanng research data that was crucal n conductng the research presented n ths paper. REFERENCES [1] W. Antweler and M. Frank. Is all that talk just nose? the nformaton content of nternet stock message boards. The Journal of Fnance, LIX(3):1259 1294, June 2004. [2] S. Asur and B. Huberman. Predctng the future wth socal meda. http://arxv.org/abs/1003.5699/, 2010. [3] J. Bollen, H. Mao, and A. Pete. Modelng publc mood and emoton: Twtter sentment and soco-economc phenomena. In ICWSM, 2011. [4] J. Bollen, H. Mao, and X.-J. Zeng. Twtter mood predct the stock market. Journal of Computatonal Scence, 2011. [5] I. Bordno, S. Battston, G. Caldarell, M. Crstell, A. Ukkonen, and I. Weber. Web search queres can predct stock market volumes. http://arxv.org/abs/1110.4784. [6] W. Chan. Stock prce reacton to news and to no-news- drft and reversal after headlnes. Journal of Fnancal Economcs, 70:223 236, 2003. [7] H. Cho and H. Varan. Predctng the present wth google trends. Techncal report, Google Inc, 2009. [8] A. Culotta. Towards detectng nfluenza epdemcs by analyzng twtter messages. In 1st Workshop on Socal Meda Analytcs (SOMA 10), Washngton, DC, USA, 2010. [9] Z. Da, J. Engelberand, and P. Gao. The sum of all fears: nvestor sentment and asset prces. http://ssrn.com/abstract=1509162, 2010. [10] Z. Da, J. Engelberand, and P. Gao. In search of attenton. Journal of Fnance, 66:1461 1499, 2011. [11] E. Glbert and K. Karahalos. Wdespread worry and the stock market. In Internatonal AAAI Conference on Weblogs and Socal Meda, 2010. [12] S. Golder and M. Macy. Durnal and seasonal mood vary wth work, sleep, and daylght across dverse cultures. Scence, pages 1814 1815, 2011.
10 [13] D. Kahneman and A. Tversky. Prospect Theory: An Analyss of Decson under Rsk. Econometrca, 47(2):263 291, 1979. [14] Y. Karabulut. Can facebook predct stock market actvty? Standford, CA., 2011. Natonal Bureau of Economc Research, Behavoral Fnance Meetng. [15] T. Loughran and B. McDonald. When s a lablty not a lablty? textual analyss, dctonares, and 10-ks. Journal of Fnance, 66(1):67 97, 2011. [16] G. Mshne and N. Glance. Predctng move sales from blogger sentment. In AAAI 2006 Sprng Symposum on Computatonal Approaches to Analysng Weblogs, 2006. [17] R. Prechter. The Wave Prncple of Human Socal Behavor and the New Scence of Soconomcs,. New Classcs Lbrary, Ganesvlle,GA., 1999. [18] T. Pres, D. Reth, and H. E. Stanley. Complex dynamcs of our economc lfe on dfferent scales: nsghts from search engne query data. Phlos Transact A Math Phys Eng Sc, 368(1933):5707 19, 2010. [19] R. Schumaker and H. Chen. Textual analyss of stock market predcton usng breakng fnancal news: Theazfntext system. ACM Transactons on Informaton Systems, 27(2), 2009. [20] E. Snger. The use of ncentves to reduce nonresponse n household surveys. Survey Nonresponse edted by Robert M. Groves, et al., 2002. [21] P. C. Tetlock. Gvng content to nvestor sentment: The role of meda n the stock market. Journal of Fnance, 62(3):1139 1168, 2007. [22] P. C. Tetlock, M. Saar-Tsechansky, and S. Macskassy. More than words: Quantfyng language to measure frms fundamentals. Journal of Fnance, 63:1437 1467, 2008. [23] A. Tumasjan, T. Sprenger, P. Sandner, and I. Welpe. Predctng electons wth twtter: What 140 characters reveal about poltcal sentment. In Internatonal AAAI Conference on Weblogs and Socal Meda, Washngton, D.C., 2010. [24] H. F. Xue Zhang and P. Gloor. Predctng stock market ndcators through twtter hope t s not as bad as fear. Socal and behavoral scences, 2010. Huna Mao School of Informatcs and Computng, Indana Unversty-Bloomngton, Unted States. Emal: hunmao@ndana.edu Scott Counts Mcrosoft Research, Redmond, Washngton, Unted States. Emal: counts@mcrosoft.com Johan Bollen School of Informatcs and Computng, Indana Unversty-Bloomngton, Unted States. Emal:jbollen@ndana.edu