Speaking of Data: Computational Language Analysis

Size: px

Start display at page:

Download "Speaking of Data: Computational Language Analysis"

Scot Dorsey
8 years ago
Views:

1 NP-SBJ PRP it S VBD had VP VBN traded VP S-ADV NP-SBJ ADJP-PRD -NONE- RB * over-the-counter TOP h 1 INDEX e 2 def q rel bark v rel prpstn m rel LBL h 4 dog n rel LBL h RELS LBL h 1 ARG0 x 5 LBL h 9 8 ARG0 e MARG h 3 RSTR h 6 ARG0 x 2 5 ARG1 x BODY h 5 7 HCONS h 3 = q h 9, h 6 = q h 8 Speaking of Data: Computational Language Analysis (In About Twenty Five Minutes) Stephan Oepen Universitetet i Oslo, Department of Informatics oe@ifi.uio.no

h 6 ARG0 x 2 5 ARG1 x BODY h 5 7 HCONS h 3 = q h 9, h 6 = q h 8 Speaking of Data: Computational Language

2 So, What Actually is Language Technology? (2001: A Space Odyssey; HAL 9000; 1968) Speaking of Data: Computational Language Analysis (2)

3 So, What Actually is Language Technology? (2001: A Space Odyssey; HAL 9000; 1968) Speaking of Data: Computational Language Analysis (2)

4 So, What Actually is Language Technology? (IBM Watson beats long-time Jeopardy! champions; 2011) Speaking of Data: Computational Language Analysis (2)

5 So, What Actually is Language Technology? (young) interdisciplinary science: language, computing, cognition; (again) culturally and commercially relevant for knowledge society. Speaking of Data: Computational Language Analysis (2)

cognition; (again) culturally and commercially relevant

6 What Makes Natural Language a Hard Problem? < Den andre veien mot Bergen er kort x 30 x 25 = 25 > The other path towards Bergen is short. {0.58} (1:1:0). > The other road towards Bergen is short. {0.56} (1:0:0). > The second road towards Bergen is short. {0.55} (2:0:0). > That other path towards Bergen is a card. {0.54} (0:1:0). > That other road towards Bergen is a card. {0.54} (0:0:0). > The second path towards Bergen is short. {0.51} (2:1:0). > The other road against Bergen is short. {0.48} (1:2:0). > The second road against Bergen is short. {0.48} (2:2:0). > Short is the other street towards Bergen. {0.33} (1:4:0). > Short is the second street towards Bergen. {0.33} (2:4:0). Speaking of Data: Computational Language Analysis (3)

> That other road towards Bergen is a card. {0.54} (0:0:0). > The second path towards Bergen is short. {0.51} (2:1:0). > The other road against Bergen is short. {0.48} (1:2:0).

7 What Makes Natural Language a Hard Problem? < Den andre veien mot Bergen er kort x 30 x 25 = 25 > The other path towards Bergen is short. {0.58} (1:1:0). > The other road towards Bergen is short. {0.56} (1:0:0). > The second road towards Bergen is short. {0.55} (2:0:0). > That other path towards Bergen is a card. {0.54} (0:1:0). > That other road towards Bergen is a card. {0.54} (0:0:0). > The second path towards Bergen is short. {0.51} (2:1:0). > The other road against Bergen is short. {0.48} (1:2:0). > The second road against Bergen is short. {0.48} (2:2:0). Scraped Off the Internet The other way to Bergen is short. > Short is the other street towards Bergen. {0.33} (1:4:0). > Short isthe theroad second to street the other towards bergen Bergen. is short {0.33}. (2:4:0). Den other roads against Boron Gene are short. Other one autobahn against Mountains am abrupt. Speaking of Data: Computational Language Analysis (3)

8 Some Examples: The Google Books Corpus (1 of 2) We constructed a corpus of digitized texts containing about 4% of all books ever printed. Analysis of this corpus enables us to investigate cultural trends quantitatively. We survey the vast terrain of culturomics, focusing on linguistic and cultural phenomena that were reflected in the English language between 1800 and We show how this approach can provide insights about fields as diverse as lexicography, the evolution of grammar, collective memory, the adoption of technology, the pursuit of fame, censorship, and historical epidemiology. [Jean-Baptiste Michel et al., 2010; Science] Speaking of Data: Computational Language Analysis (4)

We survey the vast terrain of culturomics, focusing on linguistic and cultural phenomena that were reflected in the English language between 1800 and 2000.

9 Some Examples: The Google Books Corpus (2 of 2) Speaking of Data: Computational Language Analysis (5)

10 Some Examples: The Google Books Corpus (2 of 2) The corpus cannot be read by a human. If you tried to read only the entries from the year 2000 alone, at the reasonable pace of 200 words per minute, without interruptions for food or sleep, it would take eighty years. The sequence of letters is a thousand times longer than the human genome. Speaking of Data: Computational Language Analysis (5)

words per minute, without interruptions for food or sleep, it would take eighty years.

11 Some Examples: Statistical Machine Translation Rundt 200 forskere og informatikere fra hele verden kommer til Universitetet i Oslo og Ole Johan Dahls hus for å delta på foredrag og workshops om tungregning i forskningssammenheng. Konferansen har høy profil, og etter to dager med innledende workshops åpnes selve konferansen av viserektor Doris Jorde 26. mai. Some 200 researchers and computer scientist from all over the world stream to Oslo and the new Computer Science Building (Ole Johan Dahls hus) to participate in presentations and workshops on highperformance computing. The conference keeps a top-tier international profile. Following two days of introductory workshops, the main conference is opened on May 26 by vice dean Doris Jorde. Speaking of Data: Computational Language Analysis (6)

Some 200 researchers and computer scientist from all over the world stream to Oslo and the new Computer Science Building (Ole Johan Dahls hus) to participate in presentations and workshops on

12 Some Examples: Statistical Machine Translation Rundt 200 forskere og informatikere fra hele verden kommer til Universitetet i Oslo og Ole Johan Dahls hus for å delta på foredrag og workshops om tungregning i forskningssammenheng. Konferansen har høy profil, og etter to dager med innledende workshops åpnes selve konferansen av viserektor Doris Purely statistical approach: based on probabilities of (a) word-to-word or phrase-to-phrase translations and (b) strings of just the target language words (n-grams). Some 200 researchers and computer scientist from all over the world stream to Oslo and the new Computer Science Building (Ole Johan Dahls hus) to participate in presentations and workshops on highperformance computing. The conference keeps a top-tier international profile. Following two days of introductory work- Jorde 26. mai. Statistics estimated shops, from: the main conference is (a) many millions of words of opened parallelon textmay and 26 by vice (b) trillions of words of target dean language Doris Jorde. text. Speaking of Data: Computational Language Analysis (6)

Konferansen har høy profil, og etter to dager med innledende workshops åpnes selve konferansen av viserektor Doris Purely statistical approach: based on probabilities of (a) word-to-word or

13 Closer to Home: Web-Scale N-Grams Google Web 1T N-Gram Corpus Frequency counts for sequences of between one and five words; reflecting one trillion tokens of English Web content (around 2006); royalty-free for research use; available for a dozen of languages; about to become important component in translation, parsing, et al. Technological Requirements 25 gbytes compressed ( 300 gbytes uncompressed); 14 million keys; hundreds to tens of thousands of queries in analysing one sentence. Speaking of Data: Computational Language Analysis (7)

become important component in translation, parsing, et al.

14 Closer to Home: Web-Scale N-Grams Google Web 1T N-Gram Corpus Frequency counts for sequences of between one and five words; reflecting one trillion tokens of English Web content (around 2006); royalty-free for research use; available for a dozen of languages; about to become important component in translation, parsing, et al. Relatively typical use pattern: indexing and searching large, unstructured or structured data; predominantly random access. Technological Requirements 25 gbytes compressed ( 300 gbytes uncompressed); 14 million keys; hundreds to tens of thousands of queries in analyzing one sentence. Speaking of Data: Computational Language Analysis (7)

Relatively typical use pattern: indexing and searching large, unstructured or structured data; predominantly random access.

15 Our Current (Modest) Utlization of NorStore Speaking of Data: Computational Language Analysis (8)

16 Our Current (Modest) Utlization of NorStore Bring on-line a collection of standard language resources; used in teaching and research, across disciplines and faculties; should in principle be shared with other (Norwegian) universities. (To date, only textual resources; audio and video coming.) Speaking of Data: Computational Language Analysis (8)

should in principle be shared with other (Norwegian) universities.

17 Language Analysis: Some High-Level Predictions A Family of Disciplines Going Computational Imminent paradigm shift in linguistics, lexicography, philologies, et al. no scientific computing tradition: diversity in backgrounds and needs. Storage and Processing Requirements Emerging on-line repositories of language data (including metadata); standardize services for user identity and access rights management; batch processing: refinement of (mostly unstructured) language data; interactive processing: spontaneous search and browsing services; needs tighter coupling of storage and (scalable) processing services. Speaking of Data: Computational Language Analysis (9)

Storage and Processing Requirements Emerging on-line repositories of language data (including metadata); standardize services for user identity and access rights

18 Language Analysis: Some High-Level Predictions A Family of Disciplines Going Computational Imminent paradigm shift in linguistics, lexicography, philologies, et al. no scientific computing tradition: diversity in backgrounds and needs. Storage and Processing Requirements Emerging on-line repositories of language data (including metadata); standardize services for user identity and access rights management; batch processing: refinement of (mostly unstructured) language data; interactive processing: spontaneous search and browsing services; needs tighter coupling of storage and (scalable) processing services. Language Analysis is an inherently data-intensive science. Speaking of Data: Computational Language Analysis (9)

19 More Concretely: Short-Term Initiatives Language Analysis einfrastructure User Group UniNett Sigma (Σ) looking to establish discipline-specific user groups; group-internal functions: exchange experience, coordinate activities; interface function to Σ: give feedback on user experience and needs; maybe one annual meeting contact if interested. Birds of a Feather Meeting Later Today Informal exchange of experiences, among language analysis users: 13:30 15:00: meeting room AWK (third floor). Speaking of Data: Computational Language Analysis (10)

feedback on user experience and needs; maybe one annual meeting contact oe@ifi.uio.no, if interested.

20 Credits NoTur and NorStore (via UniNett Sigma); The UiO Scientific Computation Group; The Norwegian Taxpayer.

Guidelines for Resource Allocation on the National e-infrastructure.

Guidelines for Resource Allocation on the National e-infrastructure. Abbreviations: Administration: Committee: Partners/Metacenter: UNINETT Sigma2 Resource Allocation Committee (Ressursfordelingskomité)