How To Find Out What Political Sentiment Is On Twitter

Transcription

1 Predicting Elections with Twitter What 140 Characters Reveal about Political Sentiment Andranik Tumasjan, Timm O. Sprenger, Philipp G. Sandner, Isabell M. Welpe Workshop Election Forecasting 15 July 2013 TUM School of Management Lehrstuhl für BWL Strategie und Organisation

2 Agenda Introduction and related research Data set and methodology Results and implications 2

3 The successful use of social media in the last presidential campaigns has established Twitter as an integral part of the political campaign toolbox The increasing use of Twitter as means of political communication has triggered attempts to better understand and aggregate this information 3

4 The goal of our study was to explore 3 research questions Research questions 1 Deliberation Does Twitter provide a platform for political deliberation online? 2 Sentiment How accurately can Twitter inform us about the electorate's political sentiment? 3 Prediction Can Twitter serve as a predictor of the election result? 4

5 Existing research related to our research questions and resulting research gaps we try to address Research questions Related research Research gap 1 Deliberation Does Twitter provide a platform for political deliberation online? Twitter is not only used for one-way communication, but 31% of all tweets direct a specific addressee (Honeycut & Herring, 2009) Political internet discussion boards found to be dominated by a small number of heavy users (Koop & Jansen, 2009) Many contexts largely unexplored, e.g. the political debate online Unclear whether findings apply to microblogging forums 2 Sentiment How accurately can Twitter inform us about the electorate's political sentiment? 19% of all tweets contain mentions of a brand or product and statistically significant differences of customer sentiment can be extracted (Jansen et al., 2009) Pessimism toward the ability of blogs to aggregate dispersed bits of information (Sunstein, 2008) Limited application to political sentiment Few empirical studies to explore information aggregation in social media 3 Prediction Can Twitter serve as a predictor of the election result? Some studies explore the reflection of the political landscape in "traditional" weblogs and social media (e.g., number of Facebook users a valid indicator of electoral success, Williams & Gulati, 2008) Count of candidate mentions in the press can be a better predictor of election results than official election polls (Véronis, 2007) Unclear whether findings apply to microblogging forums 5

6 Agenda Introduction and related research Data set and methodology Results and implications 6

7 We examined more than 100,000 tweets and extracted their sentiment using LIWC Data set 104,003 political tweets Published between August 13th and September 19th, 2009 (one week prior to the election) Collected all tweets containing the name of either At least one of the 6 major parties Selected prominent politicians Methodology Linguistic Inquiry and Word Count (by James Pennebaker et al.) Text analysis software developed to assess emotional, cognitive, and structural components of text samples using a psychometrically validated dictionary Calculates the share of words in a text belonging to empirically defined psychological and structural dimensions LIWC has been used widely in psychology and linguistics including to Measure the sentiment levels in US Senatorial (Yu et al., 2008) Profile politicians Twitter messages 7

8 Agenda Introduction and related research Data set and methodology Results and implications 8

9 1 While Twitter is used as a forum for political deliberation on substantive issues, this forum is dominated by heavy users Two widely accepted indicators of blog-based deliberation The exchange of substantive issues Equality of participation Party Sample tweet* Users Messages CDU CDU wants strict rules for internet User group Total Share Total Share CSU CSU continues attacks on partner of choice FDP One-time users Light (2-5) 7,064 4, % 32.9% 7,064 13, % 19.3% FDP Grüne Whoever wants civil rights must choose FDP! After the crisis only Green can help GREEN+ Medium (6-20) Heavy (21-79) Very heavy (80+) 1, % 3.3% 0.6% 18,191 15,990 14, % 23.1% 21.2% SPD Only a matter of time until the SPD dissolves Total 14, % 69, % Die Linke Society for Human Rights recommends: No government partication for LINKE 31% of all messages contain "@"-sign 19% of all messages are retweets While the distribution of users across user groups is almost identical with the one found on internet message boards, we find even less equality of participation for the political debate on Twitter Additional analyses have shown users to exhibit a party-bias in the volume and sentiment of their messages * Examples shortened for citation (e.g. omission of hyperlinks) 9

10 2 The online sentiment in tweets reflects nuanced offline differences between the politicians in our sample Leading candidates LIWC profiles* Other politicians Very similar profile for all leading candidates Only polarizing political characters, such as liberal leader Westerwelle and socialist Lafontaine, deviate in line with their roles as opposition leaders Messages mentioning Steinmeier, who was sending mixed signals regarding potential coalition partners, are the most tentative * We focused on the 12 dimensions which a priori seemed best suited to profile sentiment and political issues) Positive outweigh negative emotions, except in the case of CSU leader Seehofer who in addition is associated the most with anger (he irritated many voters with his attacks on desired coalition partner FDP) For Steinbrück and zu Guttenberg, the issues money and work, reflect their roles as finance and economics minister 10

11 2 The similarity of profiles is a plausible reflection of the political proximity between the parties Similarity of LIWC profiles Group Politicians Distance* Key findings Distance measure to quantify the similarity of sentiment profiles All politicians Governing coalition Right coalition Left coalition Candidates for chancellor Leading candidates High convergence of the leading candidates More divergence among politicians of the governing grand coalition than among those of a potential right wing coalition The similar profiles of Merkel and Steinmeier mirror the consensusdriven style of their grand coalition Other candidates 0.24 Parties All parties Governing coalition Right coalition Left coalition Union The fit of a potential right-wing coalition is almost as good as the fit in the governing coalition Greatest divergence among parties on the left Tight fit between sister parties CDU and CSU * Average distance from the mean profile per category across all 12 dimensions in percentage points 11

12 3 The activity on Twitter prior to the election seems to validly reflect the election outcome The share of tweets can be considered a plausible reflection of the election results and joint party mentions accurately reflect the political ties between parties All mentions Election results Relative frequency of joint mentions** Party Total Share Vote share Error CDU CSU SPD FDP Linke CDU 30, % 29.0% 1.0% CSU 1.25* CSU 5, % 6.9% 1.3% SPD 1.23* 0.71* SPD 27, % 24.5% 2.2% FDP 1.04* * FDP 17, % 15.5% 1.7% Die Linke 0.81* 0.79* 1.04* 0.97 Die Linke 12, % 12.7% 0.3% Grüne 0.84* 0.79* * 1.18* Grüne 8, % 11.4% 3.3% MAE = 1.65% Research institute Forsa Forschungsgruppe Wahlen GMS Infratest/dimap MAE (last poll) 0.84% 1.04% 1.48% 1.40% An analysis of messages surrounding the TV debate between the main candidates has shown that tweets can also reflect the sentiment over time * Significant at the.05-level ** Measures how often two parties are mentioned together relative to the random probability 12

13 Our findings suggest the use of social media information content to complement insights regarding the public's political sentiment Research questions 1 Deliberation Does Twitter provide a platform for political deliberation online? Conclusions While we find evidence of a lively political debate on Twitter, this discussion is dominated by a small number of users: only 4% of all users account for more than 40% of the messages 2 Sentiment How accurately can Twitter inform us about the electorate's political sentiment? Sentiment profiles plausibly reflect many nuances of the election campaign Politicians evoke a more diverse set of profiles than parties Similarity of profiles is indicative of the parties' proximity with respect to political issues 3 Prediction Can Twitter serve as a predictor of the election result? In contrast with previous studies of political message boards, we find that the mere number of messages reflects the election results and even comes close to traditional election polls Joint party mentions mirror closeness on political issues and likely coalitions 13

14 Summary and discussion Aftermath Currently ~ 350 citations (since 2010) Several attempts to replicate or extend / enhance our approach in other electoral contexts Countries Time intervals Election types (e.g, primaries) Constituencies (e.g., counties) Mention types (e.g., candiates) Analytical methods (e.g., senitment) Preliminary result of own literature survey (depending on aspiration level) 11 rather positive papers 7 rather negative papers National level results tend to be more supporting of our initial findings than other election types Longer time frames more accurate Party mentions tend to be more accurate than candidate mentions Open questions and challenges Sampling time frame Constantly changing user number and demographics in Twitter Type of mentions (candidates, party, ) Keyword selection (full names, abbreviations ) Type of analysis (simple counts, sentiment, algorithms, input data ) Type of elections (primaries, parliament, ), constituencies, and political systems Trustworthiness of tweets File drawer problem Aspiration level (replace or complement other forecasting methods) Real replications hardly possible Partly based on Gayo-Avello (2012) 14

15 hank you for your attention! 15