How To Kill A Black Person

Transcription

1 Do big data hide or reveal stories? Recording history in the NewsReader project Funded by the EU - FP7 Work Programme Call FP7-ICT Research theme Information and Communication Technologies, challenge Area Intelligent Information Management as project ICT Piek Vossen Faculteit der Letteren VU Universiteit Amsterdam Seminar on Reference Cultures and Identity formation NIAS, Wassenaar, April 25th 2013

2 Do big data hide or discover stories? Big data and the financial crisis What is the impact of the economic crisis on the car industry? Lexis Nexis estimates that their archive contains 10 million English news documents on car industry for the last 10 years. 2

3 3

4 4

5 How news accumulates over time Fall of Srebrenica Lehman Brothers Bankrupcy Project X Haren ? Facebook Direct report Later in time Moslim women and deportation children are loaded genocide Intentions, responsible, strategy, judgment into trucks and transported Moslim men are shot Dijksterhuislezing, 19 april 2013, VU Universiteit Amsterdam Grexit 5

6 Big data asks for new solutions Search results are too plentiful and rich to browse through result lists, even for faceted browsing Results contain too many duplicates, overlap, repetitions: how do you distinguish old from new information? Every single result is incomplete, only tells you a small part of the story Results are inconsistent and contradict each other News is opiniated and coloured Most information is not authorised: who tells the right story? 6

7 NewsReader-ICT Develop technology to automatically process massive streams of daily news from sources in 4 different languages: what happened, where, when and who is involved which temporal and causal relations hold between extracted events and what do they tells us about people's intentions (the why) place the cumulated result in a knowledge store that can handle the dynamic growth of information on events over time, ultimately recording the reported history: a history recorder. Organise and visualise massive amounts of information as stories, scripts, plots to provide more effecient access Partners: Netherlands (VU, LexisNexis, Synerscope), Spain (Basque University), UK (ScraperWiki) and Italy (Federation Bruno Kessler, Trento) Januari 2013 December

8 8

9 From text to structure 9

10 Part production Polo transferred from Spain to Vorst Wat happened in the Belgium place Vorst in 2006? A spokesman of Volkswagen in Germany commented that it is possible that the production at Volkswagen Vorst will be increased. part of the current production of Golf in Vorst will be transferred to Wolfsburg, without social consequences. 10

11 Jobs at Vorst April 2006: Juli 2006: Less Golfs produced in Vorst, maybe more Polos. If not, we have a problem, says a union representative...chances that Vorst will not make any Polos next year are minimal, because the factory invested this year in a special new welding installation specific for Polo cars. November 2006: Polo production in Vorst, no jobs lost in Spain but extra jobs in Belgium. Augustus 2006: production of Polo from Spain to Eastern-Europe because of social problems in Pamplona and maybe to Vorst in Belgium Volkswagen stops the production of Golf in Vorst: 3,500 jobs are lost November 2009: Audi plant in Vorst stops the production of Polo: 300 jobs lost 11

12 Grounded Annotation Framework Sources report on events mentions of events Sources: texts, databases on payment transactions, social-economic data, telephone usage, internet usage, etc, sensors such as cameras. Most events take place in reality and not in the source instances of events defined by a URI Linked Open Data (Semantic Web) GAF links all mentions of the same event to a unique instance URI and combines all information from each mention New information from mentions in future sources (after the event date) is continuously merged with the information around that single instance: reinterpretation through historical coreference Published in NAACL-2013 workshop (Fokkens et al.) 12

13 Bron: Bron: gen-fabriek-belgie-moet-stoppen-met-de-golf hp?id=2868 Tijd: om 15:07 Tekst: Volkswagen fabriek België moet stoppen met de Golf. stop with the Golf. Er gingen namelijk geruchten over zware saneringen, van tot ontslagen. disappear sure jobs Action: Participants: Time: Location: There were rumours about heavy cuts, of up to fired Bron: Tijd: :26 Tekst: Bij de Volkswagen fabriek in Vorst verdwijnen zeker banen jobs gone in Vorst Plant kanaalid=261&pageid=15 loses Golf Tijd: Di 21/11/2006 Tekst: Tot banen weg in Vorst Fabriek verliest Golf BRUSSEL - Boze arbeiders blokkeren de poorten van de Volkswagenfabriek in Vorst, nadat bekend raakte dat zeker van de banen verdwijnen. sure 3500 of the 5300 jobs disappear DECREASE? NR OF JOBS, VW PLANT Vorst, Belgium Database unemployment figures Bron: Tijd: 13 mrt 2013 om 13:24 Tekst: Op 21 november 2006 kwam het nieuws dat Volkswagen Vorst de productie van de Golf verliest en er meer dan 3500 van de 5000 banen geschrapt zouden worden. production of the Golf lost and there more than 3500 of the 5000 jobs canceled should be 13

14 How to determine mentions WORD SEQUENCE. Bij de Volkswagen fabriek in Vorst verdwijnen zeker banen... CONCEPT SEQUENCE....PLANT...DISAPPEAR JOBS RELATIONS & IMPLICATIONS TIJD, PLACE (DISAPPEAR, JOBS) (OWNS, VOLKSWAGEN, PLANT) (OPERATE, PLANT, JOBS) PROVENANCE autointernationaal SOURCE Language Technology makes STATEMENT CERTAIN 14

15 What has been said about 2006? VOLKSWAGEN VORST VORST, BELGIUM VOLKSWAGEN LOCATED-IN OWNS HAS-NAME autoblog.nl autointernationaal.nl standaard.be nl.wikipedia.org annual GOLF annual AUDI annual PRODUCES PLANT November 2006 POLO PRODUCES PRODUCES FUTURE (DISAPPEAR) JOBS HAS JOBS 5300 PATIENT 5000 DISAPPEAR November 2006 March

16 5 years later: 2011 AUDI BRUSSEL VORST, BELGIUM HAS-NAME VOLKSWAGEN LOCATED-IN OWNS nl.wikipedia.org PLANT PRODUCES HAS HAS March 2013 AUDI ROBOTS 450 JOBS annual

17 From structure to stories What to do with billions of events from millions of news articles? 17

18 Plots Human motivation, intentions, responsible (Bremond 1980, Brooks 1992, Ryan 1991): become market leader increase profit by transferring production to Eastern Europe Dramatic effect: pressure rises: Porsche buys shares in Volkswagen climax: speculation on takeover of Volkswagen resolution: Volkswagen takes over Porsche Wiedeking sued by investment companies 18

19 19

20 Scenario-1: Connecting co-occurring clusters Pyongyang (25 milion hits on Google) Mahmoud Ahmadinejad (21 million hits on Google)

21 Scenario-1: Connecting co-occurring clusters How to find the needle in the haystack that connects two trendy topics? 21

22 Scenario-1: Connecting co-occurring clusters 22

23 Scenario-2 Connecting disjoint cluster Ontslagen na overname Ontslagen na overname management transfer

24 Scenario-3 Cumulation to a critical point - unemployment - price of wheat - oil price - production of cars - market shares - measurements with impact - people, organisations that may have impact

25 Determining the value of a connection impact of participants trendy: persons that frequently occur in the news, e.g. Kim Young-Nam strength of opinions on participants: sentiment analyse role and function (decision maker with power) past: participants with a 'backpack', involved in a previous event with impact impact of events impact potential of the participants of an event: everything Obama does... trendy: frequency in news strength of opinions on events type of event (disasters with impact, fraude, corruption, bankrupcies) critical states of important factors, e.g oil price, price of wheat, market shares, monopolies 25

26 Impact Anything done by an important person is potentially important: Barack Obama has a cold. If an event has impacts then all its participants are potentially important: Belgium and Dutch boys that fight in Syria 26

27 Searching for the why? The internet has a wealth of opinions on possible reasons behind and explanations of events: conspiracy theories opinions of ordinary citizins views and opinions of research journalists and scientist 27

28 Dijksterhuislezing, 19 april 2013, VU Universiteit Amsterdam 28

29 as reason for Crisis used as reason for massive firing of people Dijksterhuislezing, 19 april 2013, VU Universiteit Amsterdam 29

30 A view of the past, gives... 30

31 \ Visionairs 31