Societal Data Resources and Data Processing Infrastructure

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Societal Data Resources and Data Processing Infrastructure"

Transcription

1 Societal Data Resources and Data Processing Infrastructure Bruno Martins INESC-ID & Instituto Superior Técnico 1

2 DATASTORM Task on Societal Data Project vision : Build infrastructure for large scale social data analysis and processing, of interest to areas such as the social sciences and economics, to be hosted at the National Foundation for Scientific computing (FCCN). Study of large real social networks Data acquisition, storage and processing Knowledge extraction Understand user activity and behavior Analyze social behavior Entity disambiguation Analyze information diffusion and influence patterns Mining of network communities Deep articulation with horizontal tasks (e.g., H1, H2, H3 and others) 2

3 Challenges in Using Societal Data Several heterogeneous sources of relevant information Traditional sources (small scale datasets) Large sets of statistical series on various areas Scientific data repositories from the social sciences Repositories with geographic and territorial information Particular focus : Social media and news (? big data? ) Articles in traditional media (i.e., online newspapers) and comments to these articles Data on Web archives Data from the social Web (e.g., Twitter, FourSquare, Facebook, etc.) Project vision : Deep (automated) study of societal issues involves integrating these sources, and analyzing the resulting datasets by combining techniques for processing textual information and networks 3

4 Statistical Series and Scientific Data Repositories PORDATA features statistical series about Portugal, Portuguese municipalities and Europe, organized into themes (e.g., population, health, family income and expenditure, education, employment, etc.) dados.gov.pt is a open-data initiative/portal that aggregates and publishes information produced by the Portuguese public administration (e.g., public expenditure, electoral results, etc.) Arquivo Português de Inf. Social (now hosted on RCCAP/FCCN) aggregates information collected about the Portuguese society in the context of academic studies (e.g., surveys and opinion pools) 4

5 News Articles, Coments over News Articles, and Social Media Contents Society is reflected in the media that is presented to us Text mining on news articles and on comments to news articles Twitter and Facebook continue to become a more and more important collective, global media voice, and is thus an important story in itself worthy of scientific analysis Science 30 September 2011: Vol. 333 no

6 Horizontal Tasks from DATASTORM Horizontal Task 1 : Data aquisition and extraction Task Leader : Pavel Calado Data crawling data from social media platforms and from the Web Mining textual contents Horizontal Task 2 : Data representation and query validation Task Leader : Alexandre Francisco Building graph-based representations for the data Data structures and algorithms for efficient storage and processing Horizontal Task 3 : Knowledge Discovery Task Leader : Ana Teresa Freitas Algorithms for knowledge discovery from societal data 6

7 Some Previous Work on the Area Previous work in the area 7

8 Using the Social Web for Analyzing Social Change Online social networks are now a fundamental organizing mechanism in recent country-wide social movements Characterizing the networks associated to these movements Discovering communities and influential individuals Examine the temporal and geospatial evolution of the communication activity Comprehend modern societal dynamics! The Digital Evolution of Occupy Wall Street. PLoS ONE 8(5) The Geospatial Characteristics of a Social Movement Communication Network. PLoS ONE 8(3) Structural and Dynamical Patterns on Online Social Networks: The Spanish May 15th Movement. PLoS ONE 6(8) 8

9 Location, Social Ties, and Human Mobility Using location-based social networks (LBSNs) to investigate human mobility and the relationship between social ties between people and co-occurrences in time and space Predicting which individuals may become friends on social networks, based on visited places Geolocating users with basis on their social links Human mobility follows surprisingly regular patterns Understanding individual human mobility patterns Nature 453, (2008) Friendship and Mobility: User Movement In Location-Based Social Networks Proceedings of the 2010 ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 9

10 Quantifying Happiness and Public Well-Being hedonometer.org is an instrument that measures the happiness of large populations in real time Uses Twitter s gardenhose feed (e.g. about 10% of all messages, 100GB of JSON per day). Messages written in English are assigned a happiness score based on the average happiness score of the words contained within (words manually accessed through crowdsourcing). This measure of happiness correlates very well with traditional surveys of well-being. Temporal Patterns of Happiness and Information in a Global-Scale Social Network: Hedonometrics and Twitter. PLoS ONE, 6, e26752,

11 The Hedonometer and Quantifying Hapiness and Public Well-Being 11

12 Text Driven Forecasting and Predicting the Real World from Text Regression models from sparse/noisy word features Often using methods that promote sparse models (e.g., Lasso regression) Many possible applications Box office results Elections and opinion pools Stock value Investment risk Restaurant menu prices Whitepaper from Noah Smith in 2009 : Text-Driven Forecasting Word Salad: Relating Food Prices and Descriptions Proceedings of the 2012 Conference on Empirical Methods in Natural Language Processing and Natural Language Learning 12

13 Interfaces to other vertical tasks - Cultoromics and Epidemics - and the cultural Harvard Using n-grams extracted by Google from books Type in a word or phrase in one of seven languages and see how its usage has been changing throughout the last few centuries Studying epidemics with Twitter Use information embedded in the Twitter stream to: Track rapidly-evolving public concern with respect to H1N1 or swine flu Track and measure actual disease activity. Quantitative analysis of culture using millions of digitized books. Science, 2010 The Use of Twitter to Track Levels of Disease Activity and Public Concern in the U.S. during the Influenza A H1N1 Pandemic. PLoS ONE 6(5): e19467 The Expression of Emotions in 20th Century Books. PLoS ONE 8(3): e

14 Interfaces to other vertical tasks - Cultoromics and Epidemics - and the cultural Harvard Using n-grams extracted by Google from books Type in a word or phrase in one of seven languages and see how its usage has been changing throughout the last few centuries Studying epidemics with Twitter Use information embedded in the Twitter stream to: Track rapidly-evolving public concern with respect to H1N1 or swine flu Track and measure actual disease activity. Quantitative analysis of culture using millions of digitized books. Science, 2010 The Use of Twitter to Track Levels of Disease Activity and Public Concern in the U.S. during the Influenza A H1N1 Pandemic. PLoS ONE 6(5): e19467 The Expression of Emotions in 20th Century Books. PLoS ONE 8(3): e

15 Some National Projects as Well 15

16 The rest of this session at the DATASTORM workshop Silvio Moreira, INESC-ID The REACTION and POPSTAR projects João Vasconcelos, AMA Introducing the dados.gov.pt portal Paula Carvalho, INESC-ID Natural language processing for the social sciences Hopefully, some discussion afterwards 16

DataStorm: Large-Scale Data Management in Cloud Environments

DataStorm: Large-Scale Data Management in Cloud Environments DataStorm: Large-Scale Data Management in Cloud Environments INESC-ID Data Management & Information Retrieval Group 1st DataStorm Workshop DataStorm W01: Outline Task H1 1 Task H1: Data Acquisition and

More information

REACTION Workshop 2013.07.31 Overview Porto, FEUP. Mário J. Silva IST/INESC-ID, Portugal REACTION

REACTION Workshop 2013.07.31 Overview Porto, FEUP. Mário J. Silva IST/INESC-ID, Portugal REACTION Workshop 2013.07.31 Overview Porto, FEUP Mário J. Silva IST/INESC-ID, Portugal Agenda 11:30 Welcome + Quick progress report and status summary 11:45 Task leaders summarize ongoing activities (10 min each

More information

Technical Presentations. Arian Pasquali, FEUP, REACTION Data Collection Plataform David Batista, INESC-ID, Sematic Relations Extraction REACTION

Technical Presentations. Arian Pasquali, FEUP, REACTION Data Collection Plataform David Batista, INESC-ID, Sematic Relations Extraction REACTION Agenda 11:30 Welcome + Quick progress report and status summary 11:45 Task leaders summarize ongoing activities (10 min each max) 12:30 Break. 14:00 Technical Presentations 15:00 Break 16:00 Short Technical

More information

(Big) Data Analytics: From Word Counts to Population Opinions

(Big) Data Analytics: From Word Counts to Population Opinions (Big) Data Analytics: From Word Counts to Population Opinions Mark Keane Insight@University College Dublin October 2014 ~ RSS ~ Edinburgh September 2014/EPIC 2 September 2014/EPIC 3 September 2014/EPIC

More information

Digital Collections as Big Data. Leslie Johnston, Library of Congress Digital Preservation 2012

Digital Collections as Big Data. Leslie Johnston, Library of Congress Digital Preservation 2012 Digital Collections as Big Data Leslie Johnston, Library of Congress Digital Preservation 2012 Data is not just generated by satellites, identified during experiments, or collected during surveys. Datasets

More information

Using Text and Data Mining Techniques to extract Stock Market Sentiment from Live News Streams

Using Text and Data Mining Techniques to extract Stock Market Sentiment from Live News Streams 2012 International Conference on Computer Technology and Science (ICCTS 2012) IPCSIT vol. XX (2012) (2012) IACSIT Press, Singapore Using Text and Data Mining Techniques to extract Stock Market Sentiment

More information

Task 3 Web Community Sensing & Task 6 Query and Visualization

Task 3 Web Community Sensing & Task 6 Query and Visualization Task 3 Web Community Sensing & Task 6 Query and Visualization REACTION Workshop January 31 th, 2013 Summary of on-going activities Team update WP3 & WP6 progress reports Resources & publications Team update

More information

Healthcare data analytics. Da-Wei Wang Institute of Information Science wdw@iis.sinica.edu.tw

Healthcare data analytics. Da-Wei Wang Institute of Information Science wdw@iis.sinica.edu.tw Healthcare data analytics Da-Wei Wang Institute of Information Science wdw@iis.sinica.edu.tw Outline Data Science Enabling technologies Grand goals Issues Google flu trend Privacy Conclusion Analytics

More information

Ethnography and Big Data

Ethnography and Big Data Ethnography and Big Data Ethnography Workshop #1 16.6.2014, Doctoral Training Center, Department Of Computer Science, University Of Trento, Italy Outline 1. Introductions 2. Outline of the workshop 3.

More information

Text Mining - Scope and Applications

Text Mining - Scope and Applications Journal of Computer Science and Applications. ISSN 2231-1270 Volume 5, Number 2 (2013), pp. 51-55 International Research Publication House http://www.irphouse.com Text Mining - Scope and Applications Miss

More information

DataStorm 2013 Workshop on Large-Scale Data Management

DataStorm 2013 Workshop on Large-Scale Data Management Domain Specific Languages for Large- Scale-Data Applications DataStorm 203 Workshop on Large-Scale Data Management 6/7/203 Alberto Rodrigues da Silva (on behalf of the Information Systems Group, INESC-ID)

More information

A Platform for Supporting Data Analytics on Twitter: Challenges and Objectives 1

A Platform for Supporting Data Analytics on Twitter: Challenges and Objectives 1 A Platform for Supporting Data Analytics on Twitter: Challenges and Objectives 1 Yannis Stavrakas Vassilis Plachouras IMIS / RC ATHENA Athens, Greece {yannis, vplachouras}@imis.athena-innovation.gr Abstract.

More information

Large Scale Repository Auditing to ISO 16363. José Carvalho jcarvalho@sdum.uminho.pt

Large Scale Repository Auditing to ISO 16363. José Carvalho jcarvalho@sdum.uminho.pt Large Scale Repository Auditing to ISO 16363 José Carvalho jcarvalho@sdum.uminho.pt Topics RCAAP Project ISO 16363 Methodology Results (preliminary audit) Future steps 2 Authors Eloy Rodrigues José Carvalho

More information

Cloud Thinking. Simplifying Big Data Processing. Rui L. Aguiar, Diogo Gomes Universidade de Aveiro - Portugal

Cloud Thinking. Simplifying Big Data Processing. Rui L. Aguiar, Diogo Gomes Universidade de Aveiro - Portugal Cloud Thinking Simplifying Big Data Processing Rui L. Aguiar, Diogo Gomes Universidade de Aveiro - Portugal Problem A connected world of Information Systems and Electronic Devices produces terabytes of

More information

AN INTEGRATED APPROACH TO E-GOVERNANCE, E-PARTICIPATION AND POLICY MODELLING

AN INTEGRATED APPROACH TO E-GOVERNANCE, E-PARTICIPATION AND POLICY MODELLING AN INTEGRATED APPROACH TO E-GOVERNANCE, E-PARTICIPATION AND POLICY MODELLING www.fupol.eu www.facebook.com/fupol www.youtube.com/fupoltv 1 Content 1. FUTURE POLICY MODELLING (FUPOL) IN BRIEF... 4 2. FUPOL

More information

Building a SDI for small countries the Portuguese example

Building a SDI for small countries the Portuguese example Building a SDI for small countries the Portuguese example Rui Pedro Julião Instituto Geográfico Português Deputy Director-General rpj@igeo.pt Abstract Portugal was one of the SDI pioneers in the beginning

More information

Complex, true real-time analytics on massive, changing datasets.

Complex, true real-time analytics on massive, changing datasets. Complex, true real-time analytics on massive, changing datasets. A NoSQL, all in-memory enabling platform technology from: Better Questions Come Before Better Answers FinchDB is a NoSQL, all in-memory

More information

María Elena Alvarado gnoss.com* elenaalvarado@gnoss.com Susana López-Sola gnoss.com* susanalopez@gnoss.com

María Elena Alvarado gnoss.com* elenaalvarado@gnoss.com Susana López-Sola gnoss.com* susanalopez@gnoss.com Linked Data based applications for Learning Analytics Research: faceted searches, enriched contexts, graph browsing and dynamic graphic visualisation of data Ricardo Alonso Maturana gnoss.com *Piqueras

More information

Big Data, Official Statistics and Social Science Research: Emerging Data Challenges

Big Data, Official Statistics and Social Science Research: Emerging Data Challenges Big Data, Official Statistics and Social Science Research: Emerging Data Challenges Professor Paul Cheung Director, United Nations Statistics Division Building the Global Information System Elements of

More information

Data Warehouse: Introduction

Data Warehouse: Introduction Base and Mining Group of Base and Mining Group of Base and Mining Group of Base and Mining Group of Base and Mining Group of Base and Mining Group of Base and Mining Group of base and data mining group,

More information

Predicting the NFL Using Twitter. Shiladitya Sinha, Chris Dyer, Kevin Gimpel, Noah Smith

Predicting the NFL Using Twitter. Shiladitya Sinha, Chris Dyer, Kevin Gimpel, Noah Smith Predicting the NFL Using Twitter Shiladitya Sinha, Chris Dyer, Kevin Gimpel, Noah Smith Disclaimer This talk will not teach you how to become a successful sports bettor. Questions What is the NFL and what

More information

Exploring Big Data in Social Networks

Exploring Big Data in Social Networks Exploring Big Data in Social Networks virgilio@dcc.ufmg.br (meira@dcc.ufmg.br) INWEB National Science and Technology Institute for Web Federal University of Minas Gerais - UFMG May 2013 Some thoughts about

More information

Ethnography and Big Data: A Rapprochement?

Ethnography and Big Data: A Rapprochement? Ethnography and Big Data: A Rapprochement? Ethnography Workshop #2 4.6.2013, DISI/Design Thinking Center, University Of Trento, Italy Outline 1. Introductions 2. Outline of the workshop 3. Definitions

More information

Enhanced Information Access to Social Streams. Enhanced Word Clouds with Entity Grouping

Enhanced Information Access to Social Streams. Enhanced Word Clouds with Entity Grouping Enhanced Information Access to Social Streams through Word Clouds with Entity Grouping Martin Leginus 1, Leon Derczynski 2 and Peter Dolog 1 1 Department of Computer Science, Aalborg University Selma Lagerlofs

More information

NESSI Summit 2014 The European Data Market. Gabriella Cattaneo, IDC Europe May 27, 2014 Brussels

NESSI Summit 2014 The European Data Market. Gabriella Cattaneo, IDC Europe May 27, 2014 Brussels NESSI Summit 2014 The European Data Market Gabriella Cattaneo, IDC Europe May 27, 2014 Brussels Content The Big Data Market main trends The European Data Market gaining speed Measuring the European Data

More information

A GENERAL TAXONOMY FOR VISUALIZATION OF PREDICTIVE SOCIAL MEDIA ANALYTICS

A GENERAL TAXONOMY FOR VISUALIZATION OF PREDICTIVE SOCIAL MEDIA ANALYTICS A GENERAL TAXONOMY FOR VISUALIZATION OF PREDICTIVE SOCIAL MEDIA ANALYTICS Stacey Franklin Jones, D.Sc. ProTech Global Solutions Annapolis, MD Abstract The use of Social Media as a resource to characterize

More information

Economic Commentaries

Economic Commentaries n Economic Commentaries Data and statistics are a cornerstone of the Riksbank s work. In recent years, the supply of data has increased dramatically and this trend is set to continue as an ever-greater

More information

Reading News with Maps p.1/13

Reading News with Maps p.1/13 Reading News with Maps Hanan Samet hjs@cs.umd.edu Department of Computer Science University of Maryland College Park, MD 20742, USA Reading News with Maps p.1/13 Extend GIS Notions to Textually Specified

More information

UNIVERSITY OF INFINITE AMBITIONS. MASTER OF SCIENCE COMPUTER SCIENCE DATA SCIENCE AND SMART SERVICES

UNIVERSITY OF INFINITE AMBITIONS. MASTER OF SCIENCE COMPUTER SCIENCE DATA SCIENCE AND SMART SERVICES UNIVERSITY OF INFINITE AMBITIONS. MASTER OF SCIENCE COMPUTER SCIENCE DATA SCIENCE AND SMART SERVICES MASTER S PROGRAMME COMPUTER SCIENCE - DATA SCIENCE AND SMART SERVICES (DS3) This is a specialization

More information

Search and Data Mining: Techniques. Applications Anya Yarygina Boris Novikov

Search and Data Mining: Techniques. Applications Anya Yarygina Boris Novikov Search and Data Mining: Techniques Applications Anya Yarygina Boris Novikov Introduction Data mining applications Data mining system products and research prototypes Additional themes on data mining Social

More information

Privacy: Legal Aspects of Big Data and Information Security

Privacy: Legal Aspects of Big Data and Information Security Privacy: Legal Aspects of Big Data and Information Security Presentation at the 2 nd National Open Access Workshop 21-22 October, 2013 Izmir, Turkey John N. Gathegi University of South Florida, Tampa,

More information

Sentiment Analysis. D. Skrepetos 1. University of Waterloo. NLP Presenation, 06/17/2015

Sentiment Analysis. D. Skrepetos 1. University of Waterloo. NLP Presenation, 06/17/2015 Sentiment Analysis D. Skrepetos 1 1 Department of Computer Science University of Waterloo NLP Presenation, 06/17/2015 D. Skrepetos (University of Waterloo) Sentiment Analysis NLP Presenation, 06/17/2015

More information

Big Data Analytics of Multi-Relationship Online Social Network Based on Multi-Subnet Composited Complex Network

Big Data Analytics of Multi-Relationship Online Social Network Based on Multi-Subnet Composited Complex Network , pp.273-284 http://dx.doi.org/10.14257/ijdta.2015.8.5.24 Big Data Analytics of Multi-Relationship Online Social Network Based on Multi-Subnet Composited Complex Network Gengxin Sun 1, Sheng Bin 2 and

More information

News media analysis at Lab SAPO UPorto. Jorge Teixeira

News media analysis at Lab SAPO UPorto. Jorge Teixeira News media analysis at Lab SAPO UPorto Jorge Teixeira Past deliverables and visualization prototypes Twitómetro Twitteuro Mundo Visto Daqui interativo (MVDi) On-going work Mundo Numa Rede Sapo Notícias

More information

An Automated Guided Model For Integrating News Into Stock Trading Strategies Pallavi Parshuram Katke 1, Ass.Prof. B.R.Solunke 2

An Automated Guided Model For Integrating News Into Stock Trading Strategies Pallavi Parshuram Katke 1, Ass.Prof. B.R.Solunke 2 www.ijecs.in International Journal Of Engineering And Computer Science ISSN:2319-7242 Volume 4 Issue - 12 December, 2015 Page No. 15312-15316 An Automated Guided Model For Integrating News Into Stock Trading

More information

What is Big Data? The three(or four) Vs in Big Data In 2013 the total amount of stored information is estimated to be Volume.

What is Big Data? The three(or four) Vs in Big Data In 2013 the total amount of stored information is estimated to be Volume. 8/26/2014 CS581 Big Data - Fall 2014 1 8/26/2014 CS581 Big Data - Fall 2014 2 CS535/CS581A BIG DATA What is Big Data? PART 0. INTRODUCTION 1. INTRODUCTION TO BIG DATA 2. COURSE INTRODUCTION PART 0. INTRODUCTION

More information

JamiQ Social Media Monitoring Software

JamiQ Social Media Monitoring Software JamiQ Social Media Monitoring Software JamiQ's multilingual social media monitoring software helps businesses listen, measure, and gain insights from conversations taking place online. JamiQ makes cutting-edge

More information

Certification In SAS Programming. Introduction to SAS Program

Certification In SAS Programming. Introduction to SAS Program Certification In SAS Programming Introduction to SAS Program What Lies Ahead In this session, you will gain answers to: Overview of Analytics Careers in Analytics Why Use SAS? Introduction to SAS System

More information

Social Influence Analysis in Social Networking Big Data: Opportunities and Challenges. Presenter: Sancheng Peng Zhaoqing University

Social Influence Analysis in Social Networking Big Data: Opportunities and Challenges. Presenter: Sancheng Peng Zhaoqing University Social Influence Analysis in Social Networking Big Data: Opportunities and Challenges Presenter: Sancheng Peng Zhaoqing University 1 2 3 4 35 46 7 Contents Introduction Relationship between SIA and BD

More information

Web Archiving and Scholarly Use of Web Archives

Web Archiving and Scholarly Use of Web Archives Web Archiving and Scholarly Use of Web Archives Helen Hockx-Yu Head of Web Archiving British Library 15 April 2013 Overview 1. Introduction 2. Access and usage: UK Web Archive 3. Scholarly feedback on

More information

Research trends relevant to data warehousing and OLAP include [Cuzzocrea et al.]: Combining the benefits of RDBMS and NoSQL database systems

Research trends relevant to data warehousing and OLAP include [Cuzzocrea et al.]: Combining the benefits of RDBMS and NoSQL database systems DATA WAREHOUSING RESEARCH TRENDS Research trends relevant to data warehousing and OLAP include [Cuzzocrea et al.]: Data source heterogeneity and incongruence Filtering out uncorrelated data Strongly unstructured

More information

the USPSTF report on prostate cancer

the USPSTF report on prostate cancer Using Social Media to Gauge Reaction to the USPSTF report on prostate cancer screening: Twitter as an investigative tool. Vinay Prabhu New York University School of Medicine New York University Cancer

More information

Science 2.0 & Big Data Science 2.0 Conference, Hamburg, March 25, 2015

Science 2.0 & Big Data Science 2.0 Conference, Hamburg, March 25, 2015 Science 2.0 & Big Data Science 2.0 Conference, Hamburg, March 25, 2015 b Prof. Dr. Stefanie Lindstaedt b www.know-center.at Know-Center GmbH Know-Center Austria s Research Center for Data-driven Business

More information

Big Data: Opportunities & Challenges, Myths & Truths 資 料 來 源 : 台 大 廖 世 偉 教 授 課 程 資 料

Big Data: Opportunities & Challenges, Myths & Truths 資 料 來 源 : 台 大 廖 世 偉 教 授 課 程 資 料 Big Data: Opportunities & Challenges, Myths & Truths 資 料 來 源 : 台 大 廖 世 偉 教 授 課 程 資 料 美 國 13 歲 學 生 用 Big Data 找 出 霸 淩 熱 點 Puri 架 設 網 站 Bullyvention, 藉 由 分 析 Twitter 上 找 出 提 到 跟 霸 凌 相 關 的 詞, 搭 配 地 理 位 置

More information

Knowledge Discovery from Data Bases Proposal for a MAP-I UC

Knowledge Discovery from Data Bases Proposal for a MAP-I UC Knowledge Discovery from Data Bases Proposal for a MAP-I UC P. Brazdil 1, João Gama 1, P. Azevedo 2 1 Universidade do Porto; 2 Universidade do Minho; 1 Knowledge Discovery from Data Bases We are deluged

More information

Network Big Data: Facing and Tackling the Complexities Xiaolong Jin

Network Big Data: Facing and Tackling the Complexities Xiaolong Jin Network Big Data: Facing and Tackling the Complexities Xiaolong Jin CAS Key Laboratory of Network Data Science & Technology Institute of Computing Technology Chinese Academy of Sciences (CAS) 2015-08-10

More information

Data Warehousing and Data Mining in Business Applications

Data Warehousing and Data Mining in Business Applications 133 Data Warehousing and Data Mining in Business Applications Eesha Goel CSE Deptt. GZS-PTU Campus, Bathinda. Abstract Information technology is now required in all aspect of our lives that helps in business

More information

Data Analytics for Healthcare: Creating understanding from big data

Data Analytics for Healthcare: Creating understanding from big data Data Analytics for Healthcare: Creating understanding from big data Data Analytics for Healthcare Data analytics is an essential resource for any profession. This collection of data and information is

More information

Development of Framework System for Managing the Big Data from Scientific and Technological Text Archives

Development of Framework System for Managing the Big Data from Scientific and Technological Text Archives Development of Framework System for Managing the Big Data from Scientific and Technological Text Archives Mi-Nyeong Hwang 1, Myunggwon Hwang 1, Ha-Neul Yeom 1,4, Kwang-Young Kim 2, Su-Mi Shin 3, Taehong

More information

The Challenges of Geospatial Analytics in the Era of Big Data

The Challenges of Geospatial Analytics in the Era of Big Data The Challenges of Geospatial Analytics in the Era of Big Data Dr Noordin Ahmad National Space Agency of Malaysia (ANGKASA) CITA 2015: 4-5 August 2015 Kuching, Sarawak Big datais an all-encompassing term

More information

Dirk Helbing (ETH Zurich) A New Approach to Sustainability Calling for Novel Science, Technology and Arts

Dirk Helbing (ETH Zurich) A New Approach to Sustainability Calling for Novel Science, Technology and Arts Dirk Helbing (ETH Zurich) A New Approach to Sustainability Calling for Novel Science, Technology and Arts Dirk Helbing (ETH Zurich) Economics 2.0: Towards A Self- Regulating, Participatory Market Society

More information

WHITE PAPER ON. Operational Analytics. HTC Global Services Inc. Do not copy or distribute. www.htcinc.com

WHITE PAPER ON. Operational Analytics. HTC Global Services Inc. Do not copy or distribute. www.htcinc.com WHITE PAPER ON Operational Analytics www.htcinc.com Contents Introduction... 2 Industry 4.0 Standard... 3 Data Streams... 3 Big Data Age... 4 Analytics... 5 Operational Analytics... 6 IT Operations Analytics...

More information

Software Engineering for Big Data. CS846 Paulo Alencar David R. Cheriton School of Computer Science University of Waterloo

Software Engineering for Big Data. CS846 Paulo Alencar David R. Cheriton School of Computer Science University of Waterloo Software Engineering for Big Data CS846 Paulo Alencar David R. Cheriton School of Computer Science University of Waterloo Big Data Big data technologies describe a new generation of technologies that aim

More information

MLg. Big Data and Its Implication to Research Methodologies and Funding. Cornelia Caragea TARDIS 2014. November 7, 2014. Machine Learning Group

MLg. Big Data and Its Implication to Research Methodologies and Funding. Cornelia Caragea TARDIS 2014. November 7, 2014. Machine Learning Group Big Data and Its Implication to Research Methodologies and Funding Cornelia Caragea TARDIS 2014 November 7, 2014 UNT Computer Science and Engineering Data Everywhere Lots of data is being collected and

More information

Collaborations between Official Statistics and Academia in the Era of Big Data

Collaborations between Official Statistics and Academia in the Era of Big Data Collaborations between Official Statistics and Academia in the Era of Big Data World Statistics Day October 20-21, 2015 Budapest Vijay Nair University of Michigan Past-President of ISI vnn@umich.edu What

More information

Big Data (Adv. Analytics) in 15 Mins. Peter LePine Managing Director Sales Support IM & BI Practice

Big Data (Adv. Analytics) in 15 Mins. Peter LePine Managing Director Sales Support IM & BI Practice Big Data (Adv. Analytics) in 15 Mins. Peter LePine Managing Director Sales Support IM & BI Practice Agenda Big Data in 15 Mins. Goal: Provide a basic understanding of; What is Big Data; Why it s important

More information

HOW TO MAKE SENSE OF BIG DATA TO BETTER DRIVE BUSINESS PROCESSES, IMPROVE DECISION-MAKING, AND SUCCESSFULLY COMPETE IN TODAY S MARKETS.

HOW TO MAKE SENSE OF BIG DATA TO BETTER DRIVE BUSINESS PROCESSES, IMPROVE DECISION-MAKING, AND SUCCESSFULLY COMPETE IN TODAY S MARKETS. HOW TO MAKE SENSE OF BIG DATA TO BETTER DRIVE BUSINESS PROCESSES, IMPROVE DECISION-MAKING, AND SUCCESSFULLY COMPETE IN TODAY S MARKETS. ALTILIA turns Big Data into Smart Data and enables businesses to

More information

Introduction. A. Bellaachia Page: 1

Introduction. A. Bellaachia Page: 1 Introduction 1. Objectives... 3 2. What is Data Mining?... 4 3. Knowledge Discovery Process... 5 4. KD Process Example... 7 5. Typical Data Mining Architecture... 8 6. Database vs. Data Mining... 9 7.

More information

Research Article 2015. International Journal of Emerging Research in Management &Technology ISSN: 2278-9359 (Volume-4, Issue-4) Abstract-

Research Article 2015. International Journal of Emerging Research in Management &Technology ISSN: 2278-9359 (Volume-4, Issue-4) Abstract- International Journal of Emerging Research in Management &Technology Research Article April 2015 Enterprising Social Network Using Google Analytics- A Review Nethravathi B S, H Venugopal, M Siddappa Dept.

More information

SOCIAL MEDIA AND MARKETS: The New Frontier

SOCIAL MEDIA AND MARKETS: The New Frontier WHITEPAPER Gnip, Inc. www.gnip.com 888.777.7405 trading@gnip.com @gnip SOCIAL MEDIA AND MARKETS: The New Frontier Source: Gnip, Inc. For the first time in history, access to the observations, wisdom and

More information

Exploration and Visualization of Post-Market Data

Exploration and Visualization of Post-Market Data Exploration and Visualization of Post-Market Data Jianying Hu, PhD Joint work with David Gotz, Shahram Ebadollahi, Jimeng Sun, Fei Wang, Marianthi Markatou Healthcare Analytics Research IBM T.J. Watson

More information

NOMAD Demonstration of NOMAD tools. 22 November 2013 1 st Pilot workshop, HEP, Athens

NOMAD Demonstration of NOMAD tools. 22 November 2013 1 st Pilot workshop, HEP, Athens NOMAD Demonstration of NOMAD tools 22 November 2013 1 st Pilot workshop, HEP, Athens NOMAD Consortium Project Coordinator: University of the Aegean(Greece) Project Partners: Athens Technology Center -

More information

Neural Networks for Sentiment Detection in Financial Text

Neural Networks for Sentiment Detection in Financial Text Neural Networks for Sentiment Detection in Financial Text Caslav Bozic* and Detlef Seese* With a rise of algorithmic trading volume in recent years, the need for automatic analysis of financial news emerged.

More information

Cleaned Data. Recommendations

Cleaned Data. Recommendations Call Center Data Analysis Megaputer Case Study in Text Mining Merete Hvalshagen www.megaputer.com Megaputer Intelligence, Inc. 120 West Seventh Street, Suite 10 Bloomington, IN 47404, USA +1 812-0-0110

More information

Combining Social Data and Semantic Content Analysis for L Aquila Social Urban Network

Combining Social Data and Semantic Content Analysis for L Aquila Social Urban Network I-CiTies 2015 2015 CINI Annual Workshop on ICT for Smart Cities and Communities Palermo (Italy) - October 29-30, 2015 Combining Social Data and Semantic Content Analysis for L Aquila Social Urban Network

More information

THE ENTERPRISE GAMING COOKBOOK

THE ENTERPRISE GAMING COOKBOOK THE ENTERPRISE GAMING COOKBOOK Learn how game studios in our Ecosystem are using Bluemix to build the world s most advanced serious games We break down the web services needed to develop a variety of experiences

More information

Advancing Sustainability with Geospatial Steven Hagan, Vice President, Server Technologies João Paiva, Ph.D. Spatial Information and Science

Advancing Sustainability with Geospatial Steven Hagan, Vice President, Server Technologies João Paiva, Ph.D. Spatial Information and Science Advancing Sustainability with Geospatial Steven Hagan, Vice President, Server Technologies João Paiva, Ph.D. Spatial Information and Science Engineering 1 Copyright 2011, Oracle and/or its affiliates.

More information

Helix Nebula, the Science Cloud

Helix Nebula, the Science Cloud Helix Nebula, the Science Cloud e-irg strategy workshop 11 & 12 June 2012 Maryline Lengert, ESA From Requirement Collection to Strategic Plan & Proof of Concept End 2010: ESA started collecting Cloud Computing

More information

Social Media Analytics

Social Media Analytics Social Media Analytics Raghu Krishnapuram and Jitendra Ajmera IBM Research - India 2011 IBM Corporation Convergence of Social and Analytic Technologies Transform the Way the World Operates Socially Synergistic

More information

Measuring your Social Media Efforts

Measuring your Social Media Efforts Measuring your Social Media Efforts Measuring your Social Media Efforts Author PR & Social Media Consultant, Sarah Michelle Willis (MCIPR), is a passionate senior strategic PR & Communications professional

More information

DIGITS CENTER FOR DIGITAL INNOVATION, TECHNOLOGY, AND STRATEGY THOUGHT LEADERSHIP FOR THE DIGITAL AGE

DIGITS CENTER FOR DIGITAL INNOVATION, TECHNOLOGY, AND STRATEGY THOUGHT LEADERSHIP FOR THE DIGITAL AGE DIGITS CENTER FOR DIGITAL INNOVATION, TECHNOLOGY, AND STRATEGY THOUGHT LEADERSHIP FOR THE DIGITAL AGE INTRODUCTION RESEARCH IN PRACTICE PAPER SERIES, FALL 2011. BUSINESS INTELLIGENCE AND PREDICTIVE ANALYTICS

More information

Online Content Optimization Using Hadoop. Jyoti Ahuja Dec 20 2011

Online Content Optimization Using Hadoop. Jyoti Ahuja Dec 20 2011 Online Content Optimization Using Hadoop Jyoti Ahuja Dec 20 2011 What do we do? Deliver right CONTENT to the right USER at the right TIME o Effectively and pro-actively learn from user interactions with

More information

Tracking the flu pandemic by monitoring the Social Web

Tracking the flu pandemic by monitoring the Social Web Tracking the flu pandemic by monitoring the Social Web Vasileios Lampos, Nello Cristianini Intelligent Systems Laboratory Faculty of Engineering University of Bristol, UK {bill.lampos, nello.cristianini}@bristol.ac.uk

More information

International Journal of Scientific & Engineering Research, Volume 5, Issue 4, April-2014 442 ISSN 2229-5518

International Journal of Scientific & Engineering Research, Volume 5, Issue 4, April-2014 442 ISSN 2229-5518 International Journal of Scientific & Engineering Research, Volume 5, Issue 4, April-2014 442 Over viewing issues of data mining with highlights of data warehousing Rushabh H. Baldaniya, Prof H.J.Baldaniya,

More information

Curriculum Vitae Ruben Sipos

Curriculum Vitae Ruben Sipos Curriculum Vitae Ruben Sipos Mailing Address: 349 Gates Hall Cornell University Ithaca, NY 14853 USA Mobile Phone: +1 607-229-0872 Date of Birth: 8 October 1985 E-mail: rs@cs.cornell.edu Web: http://www.cs.cornell.edu/~rs/

More information

Cloud-based business innovation, transformation, and the future of enterprise IT

Cloud-based business innovation, transformation, and the future of enterprise IT Cloud-based business innovation, transformation, and the future of enterprise IT Professor Marco Iansiti David Sarnoff Professor of Business Administration Harvard Business School May 3, 2011 Copyright

More information

Taking Data Analytics to the Next Level

Taking Data Analytics to the Next Level Taking Data Analytics to the Next Level Implementing and Supporting Big Data Initiatives What Is Big Data and How Is It Applicable to Anti-Fraud Efforts? 2 of 20 Definition Gartner: Big data is high-volume,

More information

Standards for Big Data in the Cloud

Standards for Big Data in the Cloud Standards for Big Data in the Cloud International Cloud Symposium 15/10/2013 Carola Carstens (Project Officer) DG CONNECT, Unit G3 Data Value Chain European Commission Outline 1) Data Value Chain Unit

More information

PUBLIC HEALTH MEETS SOCIAL MEDIA: MINING HEALTH INFO FROM TWITTER

PUBLIC HEALTH MEETS SOCIAL MEDIA: MINING HEALTH INFO FROM TWITTER PUBLIC HEALTH MEETS SOCIAL MEDIA: MINING HEALTH INFO FROM TWITTER Michael Paul (@mjp39) Johns Hopkins University Crowdsourcing and Human Computation Lecture 18 Learning about the real world through Twitter

More information

Information Management course

Information Management course Università degli Studi di Milano Master Degree in Computer Science Information Management course Teacher: Alberto Ceselli Lecture 01 : 06/10/2015 Practical informations: Teacher: Alberto Ceselli (alberto.ceselli@unimi.it)

More information

The 2006 IEEE / WIC / ACM International Conference on Web Intelligence Hong Kong, China

The 2006 IEEE / WIC / ACM International Conference on Web Intelligence Hong Kong, China WISE: Hierarchical Soft Clustering of Web Page Search based on Web Content Mining Techniques Ricardo Campos 1, 2 Gaël Dias 2 Célia Nunes 2 1 Instituto Politécnico de Tomar Tomar, Portugal 2 Centre of Human

More information

Big Data Analytics. David Dietrich, EMC Education Services. April 4, 2013

Big Data Analytics. David Dietrich, EMC Education Services. April 4, 2013 Big Data Analytics Harvard-Smithsonian Center for Astrophysics Data Science Training for Librarians April 4, 2013 David Dietrich, EMC Education Services I ll go into a company and say, What data problems

More information

Mag. Vikash Kumar, Dr. Anna Fensel kumar@ftw.at, fensel@ftw.at SEMANTIC DATA ANALYTICS AS A BASIS FOR ENERGY EFFICIENCY SERVICES

Mag. Vikash Kumar, Dr. Anna Fensel kumar@ftw.at, fensel@ftw.at SEMANTIC DATA ANALYTICS AS A BASIS FOR ENERGY EFFICIENCY SERVICES Mag. Vikash Kumar, Dr. Anna Fensel kumar@ftw.at, fensel@ftw.at SEMANTIC DATA ANALYTICS AS A BASIS FOR ENERGY EFFICIENCY SERVICES Outline Big data trends changing the ways energy infrastructures operate

More information

From Stored Knowledge to Smart Knowledge

From Stored Knowledge to Smart Knowledge From Stored Knowledge to Smart Knowledge The British Library s Content Strategy 2013 2015 From Stored Knowledge to Smart Knowledge: The British Library s Content Strategy 2013 2015 Introduction The British

More information

HELSINKI UNIVERSITY OF TECHNOLOGY 26.1.2005 T-86.141 Enterprise Systems Integration, 2001. Data warehousing and Data mining: an Introduction

HELSINKI UNIVERSITY OF TECHNOLOGY 26.1.2005 T-86.141 Enterprise Systems Integration, 2001. Data warehousing and Data mining: an Introduction HELSINKI UNIVERSITY OF TECHNOLOGY 26.1.2005 T-86.141 Enterprise Systems Integration, 2001. Data warehousing and Data mining: an Introduction Federico Facca, Alessandro Gallo, federico@grafedi.it sciack@virgilio.it

More information

Text Analytics Beginner s Guide. Extracting Meaning from Unstructured Data

Text Analytics Beginner s Guide. Extracting Meaning from Unstructured Data Text Analytics Beginner s Guide Extracting Meaning from Unstructured Data Contents Text Analytics 3 Use Cases 7 Terms 9 Trends 14 Scenario 15 Resources 24 2 2013 Angoss Software Corporation. All rights

More information

Sentiment Analysis on Big Data

Sentiment Analysis on Big Data SPAN White Paper!? Sentiment Analysis on Big Data Machine Learning Approach Several sources on the web provide deep insight about people s opinions on the products and services of various companies. Social

More information

Digital libraries of the future and the role of libraries

Digital libraries of the future and the role of libraries Digital libraries of the future and the role of libraries Donatella Castelli ISTI-CNR, Pisa, Italy Abstract Purpose: To introduce the digital libraries of the future, their enabling technologies and their

More information

Introduction to Data Mining

Introduction to Data Mining Introduction to Data Mining Jay Urbain Credits: Nazli Goharian & David Grossman @ IIT Outline Introduction Data Pre-processing Data Mining Algorithms Naïve Bayes Decision Tree Neural Network Association

More information

ANALYTICS IN BIG DATA ERA

ANALYTICS IN BIG DATA ERA ANALYTICS IN BIG DATA ERA ANALYTICS TECHNOLOGY AND ARCHITECTURE TO MANAGE VELOCITY AND VARIETY, DISCOVER RELATIONSHIPS AND CLASSIFY HUGE AMOUNT OF DATA MAURIZIO SALUSTI SAS Copyr i g ht 2012, SAS Ins titut

More information

Forward Thinking for Tomorrow s Projects Requirements for Business Analytics

Forward Thinking for Tomorrow s Projects Requirements for Business Analytics Seilevel Whitepaper Forward Thinking for Tomorrow s Projects Requirements for Business Analytics By: Joy Beatty, VP of Research & Development & Karl Wiegers, Founder Process Impact We are seeing a change

More information

MEASURING GLOBAL ATTENTION: HOW THE APPINIONS PATENTED ALGORITHMS ARE REVOLUTIONIZING INFLUENCE ANALYTICS

MEASURING GLOBAL ATTENTION: HOW THE APPINIONS PATENTED ALGORITHMS ARE REVOLUTIONIZING INFLUENCE ANALYTICS WHITE PAPER MEASURING GLOBAL ATTENTION: HOW THE APPINIONS PATENTED ALGORITHMS ARE REVOLUTIONIZING INFLUENCE ANALYTICS Overview There are many associations that come to mind when people hear the word, influence.

More information

IC05 Introduction on Networks &Visualization Nov. 2009. <mathieu.bastian@gmail.com>

IC05 Introduction on Networks &Visualization Nov. 2009. <mathieu.bastian@gmail.com> IC05 Introduction on Networks &Visualization Nov. 2009 Overview 1. Networks Introduction Networks across disciplines Properties Models 2. Visualization InfoVis Data exploration

More information

Data Mining Analytics for Business Intelligence and Decision Support

Data Mining Analytics for Business Intelligence and Decision Support Data Mining Analytics for Business Intelligence and Decision Support Chid Apte, T.J. Watson Research Center, IBM Research Division Knowledge Discovery and Data Mining (KDD) techniques are used for analyzing

More information

GRAPHICAL USER INTERFACE, ACCESS, SEARCH AND REPORTING

GRAPHICAL USER INTERFACE, ACCESS, SEARCH AND REPORTING MEDIA MONITORING AND ANALYSIS GRAPHICAL USER INTERFACE, ACCESS, SEARCH AND REPORTING Searchers Reporting Delivery (Player Selection) DATA PROCESSING AND CONTENT REPOSITORY ADMINISTRATION AND MANAGEMENT

More information

Course 803401 DSS. Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization

Course 803401 DSS. Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization Oman College of Management and Technology Course 803401 DSS Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization CS/MIS Department Information Sharing

More information

Collecting and Providing Access to Large Scale Archived Web Data. Helen Hockx-Yu Head of Web Archiving, British Library

Collecting and Providing Access to Large Scale Archived Web Data. Helen Hockx-Yu Head of Web Archiving, British Library Collecting and Providing Access to Large Scale Archived Web Data Helen Hockx-Yu Head of Web Archiving, British Library Web Archives key characteristics Snapshots of web resources, taken at given point

More information

Big Data and Big Analy-cs Trends: The Promise and the Hype. Gregory Piatetsky KDnuggets

Big Data and Big Analy-cs Trends: The Promise and the Hype. Gregory Piatetsky KDnuggets Big Data and Big Analy-cs Trends: The Promise and the Hype Gregory Piatetsky KDnuggets KDnuggets 2012 1 My Data PhD in applying Machine Learning to databases Researcher at GTE Labs started the first project

More information

Big Data: Study in Structured and Unstructured Data

Big Data: Study in Structured and Unstructured Data Big Data: Study in Structured and Unstructured Data Motashim Rasool 1, Wasim Khan 2 mail2motashim@gmail.com, khanwasim051@gmail.com Abstract With the overlay of digital world, Information is available

More information

Network Architectures & Services

Network Architectures & Services Network Architectures & Services Fernando Kuipers (F.A.Kuipers@tudelft.nl) Multi-dimensional analysis Network peopleware Network software Network hardware Individual: Quality of Experience Friends: Recommendation

More information