DataStorm: Large-Scale Data Management in Cloud Environments

Transcription

1 DataStorm: Large-Scale Data Management in Cloud Environments INESC-ID Data Management & Information Retrieval Group 1st DataStorm Workshop DataStorm W01:

2 Outline Task H1 1 Task H1: Data Acquisition and Information Extraction 2 Task V4: Cultural Data Resources and Data Processing Infrastructure DataStorm W01:

3 Outline Task H1 1 Task H1: Data Acquisition and Information Extraction 2 Task V4: Cultural Data Resources and Data Processing Infrastructure DataStorm W01:

4 Task H1: Data Acquisition and Information Extraction Goals: Exploit textual information present in digital media I.e., extracting structured data from natural language text Do this in the Terabyte scale And across all vertical tasks DataStorm W01:

5 Methodology Task H1 Focus on effectiveness Deal with the tradeoff between complexity and quality Leverage on the amount of data available Focus on efficiency Explore automatic optimization of extraction tasks Explore massive parallelization of extraction tasks DataStorm W01:

6 Participants Task H1 Pável Pereira Calado; Bruno Emanuel da Graça Martins; Helena Isabel de Jesus Galhardas; Helena Sofia Andrade Nunes Pereira Pinto; José Luis Brinquete Borbinha; Mário Jorge Costa Gaspar Silva; Paula Cristina Quaresma da Fonseca Carvalho; Paulo Jorge Fernandes Carreira Gonçalo Fernandes Simões; Ivo Miguel da Quinta Anastácio; Luís Miguel Gomes dos Santos Reis Leitão + 1 BIM + 2 BIC DataStorm W01:

7 Schedule Task H1 Horizontal task: spans the entire project DataStorm W01:

8 Outline Task H1 Task V4 1 Task H1: Data Acquisition and Information Extraction 2 Task V4: Cultural Data Resources and Data Processing Infrastructure DataStorm W01:

9 Task V4 Task V4: Cultural Data Resources and Data Processing Infrastructure Goal: Focus: Large-scale data analytics on Web archive collections Detecting, resolving and tracking named entities in Web document Extracting contextual information Retrieval and visualization of information DataStorm W01:

10 Task V4 Open Problems in Task V4 (and H1) Information Extraction and Retrieval (in general) Improvements to current IE/IR techniques Application of IE to a diverse environment such as the Web Large-scale Information Extraction and Retrieval Application to large and dynamic data repositories Parallelization/optimization of current IE/IR algorithms DataStorm W01:

11 Task V4 Available Tools and Techniques Starting points: The plethora of existing IE/IR solutions Many existing large-scale parallelization solutions Work within the team: Optimization of IE execution plans Application of IE/IR in several contexts (social networks, geographic information, bibliography, etc.) Work on Web data extraction DataStorm W01:

12 Areas of Research Task V4 Continuing work on optimization of IE execution plans On-line machine learning algorithms (applied to IE/IR) Parallelization of IE algorithms Large-scale data visualization Large-scale data analysis DataStorm W01:

13 Beyond the Web... and beyond task V4? Task H1 Task V4 Digital libraries Geographic information systems Social networks Messages and communication DataStorm W01:

14 In sum... Task H1: Data Acquisition and Information Extraction Task H4: Cultural Data Resources and Data Processing Infrastructure Challenges and opportunities: Application of IE/IR techniques to cultural data resources Adaptation of IE/IR techniques to large-scale data problems DataStorm W01:

15 Questions? DataStorm W01: