Bridging CAQDAS with text mining: Text analyst s toolbox for Big Data: Science in the Media Project

Size: px
Start display at page:

Download "Bridging CAQDAS with text mining: Text analyst s toolbox for Big Data: Science in the Media Project"

Transcription

1 Bridging CAQDAS with text mining: Text analyst s toolbox for Big Data: Science in the Media Project Ahmet Suerdem Istanbul Bilgi University; LSE Methodology Dept. Science in the media project is funded by the British Council; TUBITAK and Indian-European Research Networking Programme In The Social Sciences (ANR- DFG-ESRC-NWO with ICSSR)

2 OUTLINE OBJECTIVE: GROUNDING TEXT ANALYTICS TOOLS TO A METHODOGICAL GROUND BY USING CAQDAS CASE: SCIENCE IN THE MEDIA MONITORING PROJECT CORPUS CONSTRUCTION SCIENCE IN THE CONTEXT: ONTOLOGIES AND CODING FRAMES DETECTING ATTITUDES: SENTIMENT ANALYSIS TRIANGULATING TEXT ANALYSIS FINDING WITH SOCIAL RESEARCH METHODS:SURVEYS; FOCUS GROUPS ETC...

3 OBJECTIVE: GROUNDING TEXT ANALYTICS TOOLS INTO A METHODOGICAL GROUND-USING CAQDAS

4 SCIENCE IN THE MEDIA MONITORING PROJECT AIM: CITIZEN RESEARCH-ENGAGING PUBLIC TO ST&I Text analytics: Monopoly of Government and Big Business Aim of SMM: providing stakeholders such as politicians, NGOs, social movements and consumer and patient associations and policy-makers as well as individual researchers with text analytics tools to monitor the public opinion about Science, Technology and Innovation (ST&I) issues as reflected in the popular media OPEN TEXT ANALYTICS Citizens should be able to use CAQDAS and text analytics tools to collect evidence for their positions

5 SCIENCE IN THE MEDIA MONITORING PROJECT LSE AND ISTANBUL BILGI UNIVERSITY KNOWLEDGE PARTNERHIP FUNDED BY THE BRITISH COUNCIL TRENDS IN THE PUBLIC OPINION ABOUT SCIENCE AND TECHNOLOGY System has several components and programs in order to crawl web, classify the contexts, store the data and analyze the text. Big data: Retrieves news as RSS feeds every two hours and puts them in a database. All the news and columns in popular newspaper Hurriyet since March 2013 Filters ST&I relevant news with the help of a dictionary Calculates some visibility indices, the proportion of ST in total news body

6 web corpora: using the web as linguistic data source 1) a web crawler; 2) a web interface for crawling management and validation; 3) conversion tools; 4) HTML cleaner tools; 5) anti-duplicate filters; 6) a PoS tagger. 7) metadata BUT also the context of communication situation, i.e. who the speaker/writer is, what is the topic, what semantic domain the topic belongs to, what the mode of communication is, etc ->BRIDGING CAQDAS W/ TEXT ANALYTICS: SOME EXERCISES

7 corpus construction A corpus is a collection of pieces of language text in electronic form, selected according to external criteria to represent, as far as possible, a language or language variety as a source of data for linguistic research. (Sinclair 2004)

8 What is the domain of Science and Technology: General vs Topical corpus Complementing the linguistic info with the contextual social info We are not only sampling text units but also opinions, attitudes behaviours, events, social representations etc... Purposive sampling: starts w/ keywords relevant to the topic; and then iterative search of other relevant keywords Usually done with ad-hoc key-word queries; GOOGLE; LEXIS/NEXIS Should be more methodological Semantic: lexical; analysis of word meanings and relations between them. Pragmatic: involve multiple USER (Audience) feedback Hermeneutical circle, use of classical CAQDAS to make the initial categories Saturation: Defining the boundaries of a knowledge domain

9 Semantic and statistical description of a topic Sub-corpora: specific functional or semantic domain, law/administration, economy,literature, fashion, etc... The gathering of linguistic data for each sub-corpus requires a targeted crawling strategy. An underlying semantic theme; a document consisting of a large number of words might be concisely modelled as deriving from a smaller number of topics. Statistical: A topic is a probability distribution over terms in a vocabulary. But also purposive: hermeneutical grounding of the terms in the social context

10 corpus theoretical paradox: solving the problem w/ text mining tools and CAQDAS Iterative: makes this a circular process: Initial keywords and maybe nothing more than the keywords We assume to select the corpus according to some representative criteria (ie keyword search) and make empirical analysis to detect the keywords Question is: how to select further keywords to be most informative about the topic domain; superordinate subordinate concepts, hyponymy; hypernymy Some text mining solutions: context determination techniques such as: Word seeding: seed a keyword reflecting the domain feature, e.g., animal automatically extract a large set of surrounding extraction patterns (context words). Can get the Hypernyms: pigs, chicken, horses etc.. LDA: automatically discovering topics that some semantic contexts (sentences, paragraphs, chapters, contain). LDA represents documents as mixtures of topics that spit out words with certain probabilities.

11 LDA: example

12 Supervised topic models, Already human-coded text segments. Use the usual CAQDAS approaches; rigorous,methodological coding and thematization of the text Then use supervised machine learning techniques such as: Supervised LDA Naive Bayes K-Means SVM Etc... Improve the topic keywords

13 Coding frame and ontological terminology engineering: Modelling concepts and the relations between them, Concept: described by means of characteristics that denote properties of individual referents belonging to the extension of that concept. Idea is similar to codebook building Indexing Available ontologies: SNOMED, DEWEY

14

15 Suggested terminological anthropology: OECD Frascati manual for ST&I classification 1. Exploration and exploitation of the Earth. 2. Infrastructure and general planning of land use. 3. Control and care of the environment. 4. Protection and improvement of human health. 5. Production, distribution and rational utilisation of energy. 6. Agricultural production and technology. 7. Industrial production and technology. 8. Social structures and relationships. 9. Exploration and exploitation of space. 10. Non-oriented research. 11. Other civil research. 12. Defence

16 Ontology learning; grounded theory; word space theory Bottom up categorization, getting the themes out of the text itself Cluster analysis, correspondence analysis, formal concept analysis, semantic network analysis Grounded theory; thematic analysis

17

18

19

20

Search and Information Retrieval

Search and Information Retrieval Search and Information Retrieval Search on the Web 1 is a daily activity for many people throughout the world Search and communication are most popular uses of the computer Applications involving search

More information

Clustering Connectionist and Statistical Language Processing

Clustering Connectionist and Statistical Language Processing Clustering Connectionist and Statistical Language Processing Frank Keller keller@coli.uni-sb.de Computerlinguistik Universität des Saarlandes Clustering p.1/21 Overview clustering vs. classification supervised

More information

Joint Research Centre

Joint Research Centre Joint Research Centre Open Source Monitoring Tools and Applications emm.newsbrief.eu Serving society Stimulating innovation Supporting legislation Open Source Monitoring - Overview EMM Introduction Custom

More information

F. Aiolli - Sistemi Informativi 2007/2008

F. Aiolli - Sistemi Informativi 2007/2008 Text Categorization Text categorization (TC - aka text classification) is the task of buiding text classifiers, i.e. sofware systems that classify documents from a domain D into a given, fixed set C =

More information

Web Archiving and Scholarly Use of Web Archives

Web Archiving and Scholarly Use of Web Archives Web Archiving and Scholarly Use of Web Archives Helen Hockx-Yu Head of Web Archiving British Library 15 April 2013 Overview 1. Introduction 2. Access and usage: UK Web Archive 3. Scholarly feedback on

More information

Delivering Smart Answers!

Delivering Smart Answers! Companion for SharePoint Topic Analyst Companion for SharePoint All Your Information Enterprise-ready Enrich SharePoint, your central place for document and workflow management, not only with an improved

More information

Survey Results: Requirements and Use Cases for Linguistic Linked Data

Survey Results: Requirements and Use Cases for Linguistic Linked Data Survey Results: Requirements and Use Cases for Linguistic Linked Data 1 Introduction This survey was conducted by the FP7 Project LIDER (http://www.lider-project.eu/) as input into the W3C Community Group

More information

Bing Liu. Web Data Mining. Exploring Hyperlinks, Contents, and Usage Data. With 177 Figures. ~ Spring~r

Bing Liu. Web Data Mining. Exploring Hyperlinks, Contents, and Usage Data. With 177 Figures. ~ Spring~r Bing Liu Web Data Mining Exploring Hyperlinks, Contents, and Usage Data With 177 Figures ~ Spring~r Table of Contents 1. Introduction.. 1 1.1. What is the World Wide Web? 1 1.2. ABrief History of the Web

More information

Computer-Based Text- and Data Analysis Technologies and Applications. Mark Cieliebak 9.6.2015

Computer-Based Text- and Data Analysis Technologies and Applications. Mark Cieliebak 9.6.2015 Computer-Based Text- and Data Analysis Technologies and Applications Mark Cieliebak 9.6.2015 Data Scientist analyze Data Library use 2 About Me Mark Cieliebak + Software Engineer & Data Scientist + PhD

More information

Scholarly Use of Web Archives

Scholarly Use of Web Archives Scholarly Use of Web Archives Helen Hockx-Yu Head of Web Archiving British Library 15 February 2013 Web Archiving initiatives worldwide http://en.wikipedia.org/wiki/file:map_of_web_archiving_initiatives_worldwide.png

More information

GRAPHICAL USER INTERFACE, ACCESS, SEARCH AND REPORTING

GRAPHICAL USER INTERFACE, ACCESS, SEARCH AND REPORTING MEDIA MONITORING AND ANALYSIS GRAPHICAL USER INTERFACE, ACCESS, SEARCH AND REPORTING Searchers Reporting Delivery (Player Selection) DATA PROCESSING AND CONTENT REPOSITORY ADMINISTRATION AND MANAGEMENT

More information

ONLINE RESUME PARSING SYSTEM USING TEXT ANALYTICS

ONLINE RESUME PARSING SYSTEM USING TEXT ANALYTICS ONLINE RESUME PARSING SYSTEM USING TEXT ANALYTICS Divyanshu Chandola 1, Aditya Garg 2, Ankit Maurya 3, Amit Kushwaha 4 1 Student, Department of Information Technology, ABES Engineering College, Uttar Pradesh,

More information

HOW TO MAKE SENSE OF BIG DATA TO BETTER DRIVE BUSINESS PROCESSES, IMPROVE DECISION-MAKING, AND SUCCESSFULLY COMPETE IN TODAY S MARKETS.

HOW TO MAKE SENSE OF BIG DATA TO BETTER DRIVE BUSINESS PROCESSES, IMPROVE DECISION-MAKING, AND SUCCESSFULLY COMPETE IN TODAY S MARKETS. HOW TO MAKE SENSE OF BIG DATA TO BETTER DRIVE BUSINESS PROCESSES, IMPROVE DECISION-MAKING, AND SUCCESSFULLY COMPETE IN TODAY S MARKETS. ALTILIA turns Big Data into Smart Data and enables businesses to

More information

Clustering Technique in Data Mining for Text Documents

Clustering Technique in Data Mining for Text Documents Clustering Technique in Data Mining for Text Documents Ms.J.Sathya Priya Assistant Professor Dept Of Information Technology. Velammal Engineering College. Chennai. Ms.S.Priyadharshini Assistant Professor

More information

Search Taxonomy. Web Search. Search Engine Optimization. Information Retrieval

Search Taxonomy. Web Search. Search Engine Optimization. Information Retrieval Information Retrieval INFO 4300 / CS 4300! Retrieval models Older models» Boolean retrieval» Vector Space model Probabilistic Models» BM25» Language models Web search» Learning to Rank Search Taxonomy!

More information

Search Result Optimization using Annotators

Search Result Optimization using Annotators Search Result Optimization using Annotators Vishal A. Kamble 1, Amit B. Chougule 2 1 Department of Computer Science and Engineering, D Y Patil College of engineering, Kolhapur, Maharashtra, India 2 Professor,

More information

Legal Informatics Final Paper Submission Creating a Legal-Focused Search Engine I. BACKGROUND II. PROBLEM AND SOLUTION

Legal Informatics Final Paper Submission Creating a Legal-Focused Search Engine I. BACKGROUND II. PROBLEM AND SOLUTION Brian Lao - bjlao Karthik Jagadeesh - kjag Legal Informatics Final Paper Submission Creating a Legal-Focused Search Engine I. BACKGROUND There is a large need for improved access to legal help. For example,

More information

Qualitative Corporate Dashboards for Corporate Monitoring Peng Jia and Miklos A. Vasarhelyi 1

Qualitative Corporate Dashboards for Corporate Monitoring Peng Jia and Miklos A. Vasarhelyi 1 Qualitative Corporate Dashboards for Corporate Monitoring Peng Jia and Miklos A. Vasarhelyi 1 Introduction Electronic Commerce 2 is accelerating dramatically changes in the business process. Electronic

More information

Search and Data Mining: Techniques. Text Mining Anya Yarygina Boris Novikov

Search and Data Mining: Techniques. Text Mining Anya Yarygina Boris Novikov Search and Data Mining: Techniques Text Mining Anya Yarygina Boris Novikov Introduction Generally used to denote any system that analyzes large quantities of natural language text and detects lexical or

More information

Research Article 2015. International Journal of Emerging Research in Management &Technology ISSN: 2278-9359 (Volume-4, Issue-4) Abstract-

Research Article 2015. International Journal of Emerging Research in Management &Technology ISSN: 2278-9359 (Volume-4, Issue-4) Abstract- International Journal of Emerging Research in Management &Technology Research Article April 2015 Enterprising Social Network Using Google Analytics- A Review Nethravathi B S, H Venugopal, M Siddappa Dept.

More information

Research Challenge on Opinion Mining and Sentiment Analysis *

Research Challenge on Opinion Mining and Sentiment Analysis * Research Challenge on Opinion Mining and Sentiment Analysis * David Osimo 1 and Francesco Mureddu 2 Draft Background The aim of this paper is to present an outline for discussion upon a new Research Challenge

More information

Text Mining and Analysis

Text Mining and Analysis Text Mining and Analysis Practical Methods, Examples, and Case Studies Using SAS Goutam Chakraborty, Murali Pagolu, Satish Garla From Text Mining and Analysis. Full book available for purchase here. Contents

More information

NOMAD Demonstration of NOMAD tools. 22 November 2013 1 st Pilot workshop, HEP, Athens

NOMAD Demonstration of NOMAD tools. 22 November 2013 1 st Pilot workshop, HEP, Athens NOMAD Demonstration of NOMAD tools 22 November 2013 1 st Pilot workshop, HEP, Athens NOMAD Consortium Project Coordinator: University of the Aegean(Greece) Project Partners: Athens Technology Center -

More information

Spatio-Temporal Patterns of Passengers Interests at London Tube Stations

Spatio-Temporal Patterns of Passengers Interests at London Tube Stations Spatio-Temporal Patterns of Passengers Interests at London Tube Stations Juntao Lai *1, Tao Cheng 1, Guy Lansley 2 1 SpaceTimeLab for Big Data Analytics, Department of Civil, Environmental &Geomatic Engineering,

More information

NOMAD: Linguistic Resources and Tools Aimed at Policy Formulation and Validation

NOMAD: Linguistic Resources and Tools Aimed at Policy Formulation and Validation NOMAD: Linguistic Resources and Tools Aimed at Policy Formulation and Validation George Kiomourtzis, George Giannakopoulos, Georgios Petasis, Pythagoras Karampiperis, Vangelis Karkaletsis {gkiom, ggianna,

More information

Provalis Research Text Analytics and the Victory Index

Provalis Research Text Analytics and the Victory Index point Provalis Research Text Analytics and the Victory Index Fern Halper, Ph.D. Fellow Daniel Kirsch Senior Analyst Provalis Research Text Analytics and the Victory Index Unstructured data is everywhere

More information

IT services for analyses of various data samples

IT services for analyses of various data samples IT services for analyses of various data samples Ján Paralič, František Babič, Martin Sarnovský, Peter Butka, Cecília Havrilová, Miroslava Muchová, Michal Puheim, Martin Mikula, Gabriel Tutoky Technical

More information

Role of Social Networking in Marketing using Data Mining

Role of Social Networking in Marketing using Data Mining Role of Social Networking in Marketing using Data Mining Mrs. Saroj Junghare Astt. Professor, Department of Computer Science and Application St. Aloysius College, Jabalpur, Madhya Pradesh, India Abstract:

More information

Introduction to Big Data Analytics p. 1 Big Data Overview p. 2 Data Structures p. 5 Analyst Perspective on Data Repositories p.

Introduction to Big Data Analytics p. 1 Big Data Overview p. 2 Data Structures p. 5 Analyst Perspective on Data Repositories p. Introduction p. xvii Introduction to Big Data Analytics p. 1 Big Data Overview p. 2 Data Structures p. 5 Analyst Perspective on Data Repositories p. 9 State of the Practice in Analytics p. 11 BI Versus

More information

Big Data uses cases and implementation pilots at the OECD

Big Data uses cases and implementation pilots at the OECD Distr. GENERAL Working Paper 28 February 2014 ENGLISH ONLY UNITED NATIONS ECONOMIC COMMISSION FOR EUROPE (ECE) CONFERENCE OF EUROPEAN STATISTICIANS ORGANISATION FOR ECONOMIC COOPERATION AND DEVELOPMENT

More information

Data Mining for Knowledge Management in Technology Enhanced Learning

Data Mining for Knowledge Management in Technology Enhanced Learning Proceedings of the 6th WSEAS International Conference on Applications of Electrical Engineering, Istanbul, Turkey, May 27-29, 2007 115 Data Mining for Knowledge Management in Technology Enhanced Learning

More information

Context Aware Predictive Analytics: Motivation, Potential, Challenges

Context Aware Predictive Analytics: Motivation, Potential, Challenges Context Aware Predictive Analytics: Motivation, Potential, Challenges Mykola Pechenizkiy Seminar 31 October 2011 University of Bournemouth, England http://www.win.tue.nl/~mpechen/projects/capa Outline

More information

PRODUCT REVIEW RANKING SUMMARIZATION

PRODUCT REVIEW RANKING SUMMARIZATION PRODUCT REVIEW RANKING SUMMARIZATION N.P.Vadivukkarasi, Research Scholar, Department of Computer Science, Kongu Arts and Science College, Erode. Dr. B. Jayanthi M.C.A., M.Phil., Ph.D., Associate Professor,

More information

Why are Organizations Interested?

Why are Organizations Interested? SAS Text Analytics Mary-Elizabeth ( M-E ) Eddlestone SAS Customer Loyalty M-E.Eddlestone@sas.com +1 (607) 256-7929 Why are Organizations Interested? Text Analytics 2009: User Perspectives on Solutions

More information

Neural Networks for Sentiment Detection in Financial Text

Neural Networks for Sentiment Detection in Financial Text Neural Networks for Sentiment Detection in Financial Text Caslav Bozic* and Detlef Seese* With a rise of algorithmic trading volume in recent years, the need for automatic analysis of financial news emerged.

More information

Collecting Polish German Parallel Corpora in the Internet

Collecting Polish German Parallel Corpora in the Internet Proceedings of the International Multiconference on ISSN 1896 7094 Computer Science and Information Technology, pp. 285 292 2007 PIPS Collecting Polish German Parallel Corpora in the Internet Monika Rosińska

More information

Web Mining. Margherita Berardi LACAM. Dipartimento di Informatica Università degli Studi di Bari berardi@di.uniba.it

Web Mining. Margherita Berardi LACAM. Dipartimento di Informatica Università degli Studi di Bari berardi@di.uniba.it Web Mining Margherita Berardi LACAM Dipartimento di Informatica Università degli Studi di Bari berardi@di.uniba.it Bari, 24 Aprile 2003 Overview Introduction Knowledge discovery from text (Web Content

More information

Research on News Video Multi-topic Extraction and Summarization

Research on News Video Multi-topic Extraction and Summarization International Journal of New Technology and Research (IJNTR) ISSN:2454-4116, Volume-2, Issue-3, March 2016 Pages 37-39 Research on News Video Multi-topic Extraction and Summarization Di Li, Hua Huo Abstract

More information

A prototype infrastructure for D Spin Services based on a flexible multilayer architecture

A prototype infrastructure for D Spin Services based on a flexible multilayer architecture A prototype infrastructure for D Spin Services based on a flexible multilayer architecture Volker Boehlke 1,, 1 NLP Group, Department of Computer Science, University of Leipzig, Johanisgasse 26, 04103

More information

Data Mining Yelp Data - Predicting rating stars from review text

Data Mining Yelp Data - Predicting rating stars from review text Data Mining Yelp Data - Predicting rating stars from review text Rakesh Chada Stony Brook University rchada@cs.stonybrook.edu Chetan Naik Stony Brook University cnaik@cs.stonybrook.edu ABSTRACT The majority

More information

Language and Computation

Language and Computation Language and Computation week 13, Thursday, April 24 Tamás Biró Yale University tamas.biro@yale.edu http://www.birot.hu/courses/2014-lc/ Tamás Biró, Yale U., Language and Computation p. 1 Practical matters

More information

An Ontology Based Text Analytics on Social Media

An Ontology Based Text Analytics on Social Media , pp.233-240 http://dx.doi.org/10.14257/ijdta.2015.8.5.20 An Ontology Based Text Analytics on Social Media Pankajdeep Kaur, Pallavi Sharma and Nikhil Vohra GNDU, Regional Campus, GNDU, Regional Campus,

More information

IT Challenges for the Library and Information Studies Sector

IT Challenges for the Library and Information Studies Sector IT Challenges for the Library and Information Studies Sector This document is intended to facilitate and stimulate discussion at the e-science Scoping Study Expert Seminar for Library and Information Studies.

More information

MIRACLE at VideoCLEF 2008: Classification of Multilingual Speech Transcripts

MIRACLE at VideoCLEF 2008: Classification of Multilingual Speech Transcripts MIRACLE at VideoCLEF 2008: Classification of Multilingual Speech Transcripts Julio Villena-Román 1,3, Sara Lana-Serrano 2,3 1 Universidad Carlos III de Madrid 2 Universidad Politécnica de Madrid 3 DAEDALUS

More information

An Open Platform for Collecting Domain Specific Web Pages and Extracting Information from Them

An Open Platform for Collecting Domain Specific Web Pages and Extracting Information from Them An Open Platform for Collecting Domain Specific Web Pages and Extracting Information from Them Vangelis Karkaletsis and Constantine D. Spyropoulos NCSR Demokritos, Institute of Informatics & Telecommunications,

More information

Social Media Monitoring Tools enhanced by Semantic Web Technologies. Presentation of the Master Thesis Fabian Gasser

Social Media Monitoring Tools enhanced by Semantic Web Technologies. Presentation of the Master Thesis Fabian Gasser Social Media Monitoring Tools enhanced by Semantic Web Technologies Presentation of the Master Thesis Fabian Gasser Contents 1. 2. 3. 4. 5. 6. 7. 8. Main Concepts Challenges Research Question Social Media

More information

Sentiment analysis on tweets in a financial domain

Sentiment analysis on tweets in a financial domain Sentiment analysis on tweets in a financial domain Jasmina Smailović 1,2, Miha Grčar 1, Martin Žnidaršič 1 1 Dept of Knowledge Technologies, Jožef Stefan Institute, Ljubljana, Slovenia 2 Jožef Stefan International

More information

Azure Machine Learning, SQL Data Mining and R

Azure Machine Learning, SQL Data Mining and R Azure Machine Learning, SQL Data Mining and R Day-by-day Agenda Prerequisites No formal prerequisites. Basic knowledge of SQL Server Data Tools, Excel and any analytical experience helps. Best of all:

More information

Intelligent Search for Answering Clinical Questions Coronado Group, Ltd. Innovation Initiatives

Intelligent Search for Answering Clinical Questions Coronado Group, Ltd. Innovation Initiatives Intelligent Search for Answering Clinical Questions Coronado Group, Ltd. Innovation Initiatives Search The Way You Think Copyright 2009 Coronado, Ltd. All rights reserved. All other product names and logos

More information

Architecture of an Ontology-Based Domain- Specific Natural Language Question Answering System

Architecture of an Ontology-Based Domain- Specific Natural Language Question Answering System Architecture of an Ontology-Based Domain- Specific Natural Language Question Answering System Athira P. M., Sreeja M. and P. C. Reghuraj Department of Computer Science and Engineering, Government Engineering

More information

Using Text and Data Mining Techniques to extract Stock Market Sentiment from Live News Streams

Using Text and Data Mining Techniques to extract Stock Market Sentiment from Live News Streams 2012 International Conference on Computer Technology and Science (ICCTS 2012) IPCSIT vol. XX (2012) (2012) IACSIT Press, Singapore Using Text and Data Mining Techniques to extract Stock Market Sentiment

More information

Using Artificial Intelligence to Manage Big Data for Litigation

Using Artificial Intelligence to Manage Big Data for Litigation FEBRUARY 3 5, 2015 / THE HILTON NEW YORK Using Artificial Intelligence to Manage Big Data for Litigation Understanding Artificial Intelligence to Make better decisions Improve the process Allay the fear

More information

DIGITAL MARKETING TRAINING

DIGITAL MARKETING TRAINING DIGITAL MARKETING TRAINING Digital Marketing Basics Keywords Research and Analysis Basics of advertising What is Digital Media? Digital Media Vs. Traditional Media Benefits of Digital marketing Latest

More information

Weblogs Content Classification Tools: performance evaluation

Weblogs Content Classification Tools: performance evaluation Weblogs Content Classification Tools: performance evaluation Jesús Tramullas a,, and Piedad Garrido a a Universidad de Zaragoza, Dept. of Library and Information Science. Pedro Cerbuna 12, 50009 Zaragoza.tramullas@unizar.es

More information

Internet of Things, data management for healthcare applications. Ontology and automatic classifications

Internet of Things, data management for healthcare applications. Ontology and automatic classifications Internet of Things, data management for healthcare applications. Ontology and automatic classifications Inge.Krogstad@nor.sas.com SAS Institute Norway Different challenges same opportunities! Data capture

More information

Data Search. Searching and Finding information in Unstructured and Structured Data Sources

Data Search. Searching and Finding information in Unstructured and Structured Data Sources 1 Data Search Searching and Finding information in Unstructured and Structured Data Sources Erik Fransen Senior Business Consultant 11.00-12.00 P.M. November, 3 IRM UK, DW/BI 2009, London Centennium BI

More information

User Guide to the Content Analysis Tool

User Guide to the Content Analysis Tool User Guide to the Content Analysis Tool User Guide To The Content Analysis Tool 1 Contents Introduction... 3 Setting Up a New Job... 3 The Dashboard... 7 Job Queue... 8 Completed Jobs List... 8 Job Details

More information

Sentiment Analysis on Big Data

Sentiment Analysis on Big Data SPAN White Paper!? Sentiment Analysis on Big Data Machine Learning Approach Several sources on the web provide deep insight about people s opinions on the products and services of various companies. Social

More information

The Data Mining Process

The Data Mining Process Sequence for Determining Necessary Data. Wrong: Catalog everything you have, and decide what data is important. Right: Work backward from the solution, define the problem explicitly, and map out the data

More information

NOS for Data Analysis (802) September 2014 V1.3

NOS for Data Analysis (802) September 2014 V1.3 NOS for Data Analysis (802) September 2014 V1.3 NOS Reference ESKITP802301 ESKITP802401 ESKITP802501 ESKITP802601 NOS Title Assist in Delivering Routine Data Analysis Studies Design and Implement Data

More information

Tibetan-Chinese Bilingual Sentences Alignment Method based on Multiple Features

Tibetan-Chinese Bilingual Sentences Alignment Method based on Multiple Features , pp.273-280 http://dx.doi.org/10.14257/ijdta.2015.8.4.27 Tibetan-Chinese Bilingual Sentences Alignment Method based on Multiple Features Lirong Qiu School of Information Engineering, MinzuUniversity of

More information

The Seven Practice Areas of Text Analytics

The Seven Practice Areas of Text Analytics Excerpt from: Practical Text Mining and Statistical Analysis for Non-Structured Text Data Applications G. Miner, D. Delen, J. Elder, A. Fast, T. Hill, and R. Nisbet, Elsevier, January 2012 Available now:

More information

Practical Semantic Web Tagging and Tag Clouds 1

Practical Semantic Web Tagging and Tag Clouds 1 BULGARIAN ACADEMY OF SCIENCES CYBERNETICS AND INFORMATION TECHNOLOGIES Volume 6, No 3 Sofia 2006 Practical Semantic Web Tagging and Tag Clouds 1 Ivo Marinchev Institute of Information Technologies, 1113

More information

WEGOV ANALYSIS TOOLS TO CONNECT POLICY MAKERS WITH CITIZENS ONLINE

WEGOV ANALYSIS TOOLS TO CONNECT POLICY MAKERS WITH CITIZENS ONLINE WEGOV ANALYSIS TOOLS TO CONNECT POLICY MAKERS WITH CITIZENS ONLINE Timo Wandhöfer, GESIS Leibniz Institute for the Social Sciences, Knowledge Technologies for the Social Sciences, Unter Sachsenhausen 6-8,

More information

Project no. 607865. Collaborative Project FP7-SEC-2013-1 www.fp7-shelp.eu. Deliverable D7.4

Project no. 607865. Collaborative Project FP7-SEC-2013-1 www.fp7-shelp.eu. Deliverable D7.4 Project no. S-HELP: Securing - Health.Emergency.Learning.Planning Development of decision support tools for improving preparedness and response of Health Services involved in emergency situations Collaborative

More information

Chapter ML:XI. XI. Cluster Analysis

Chapter ML:XI. XI. Cluster Analysis Chapter ML:XI XI. Cluster Analysis Data Mining Overview Cluster Analysis Basics Hierarchical Cluster Analysis Iterative Cluster Analysis Density-Based Cluster Analysis Cluster Evaluation Constrained Cluster

More information

Terminology Extraction from Log Files

Terminology Extraction from Log Files Terminology Extraction from Log Files Hassan Saneifar 1,2, Stéphane Bonniol 2, Anne Laurent 1, Pascal Poncelet 1, and Mathieu Roche 1 1 LIRMM - Université Montpellier 2 - CNRS 161 rue Ada, 34392 Montpellier

More information

STATE OF VERMONT. Secretary of Administration ORIGINAL POLICY ADOPTED BY STC DATE: STATUTORY REFERENCE Policy for Web Look and Feel Requirements

STATE OF VERMONT. Secretary of Administration ORIGINAL POLICY ADOPTED BY STC DATE: STATUTORY REFERENCE Policy for Web Look and Feel Requirements STATE OF VERMONT Agency of Administration STANDARD STC State Technology Collaborative ORIGINAL POLICY ADOPTED BY STC DATE: EFFECTIVE DATE ORIGINAL POLICY NUMBER ASSOCIATED DOCUMENTS Standards for Usability

More information

Big Data: Rethinking Text Visualization

Big Data: Rethinking Text Visualization Big Data: Rethinking Text Visualization Dr. Anton Heijs anton.heijs@treparel.com Treparel April 8, 2013 Abstract In this white paper we discuss text visualization approaches and how these are important

More information

Using Knowledge Extraction and Maintenance Techniques To Enhance Analytical Performance

Using Knowledge Extraction and Maintenance Techniques To Enhance Analytical Performance Using Knowledge Extraction and Maintenance Techniques To Enhance Analytical Performance David Bixler, Dan Moldovan and Abraham Fowler Language Computer Corporation 1701 N. Collins Blvd #2000 Richardson,

More information

Stock Market Prediction Using Data Mining

Stock Market Prediction Using Data Mining Stock Market Prediction Using Data Mining 1 Ruchi Desai, 2 Prof.Snehal Gandhi 1 M.E., 2 M.Tech. 1 Computer Department 1 Sarvajanik College of Engineering and Technology, Surat, Gujarat, India Abstract

More information

CAPTURING THE VALUE OF UNSTRUCTURED DATA: INTRODUCTION TO TEXT MINING

CAPTURING THE VALUE OF UNSTRUCTURED DATA: INTRODUCTION TO TEXT MINING CAPTURING THE VALUE OF UNSTRUCTURED DATA: INTRODUCTION TO TEXT MINING Mary-Elizabeth ( M-E ) Eddlestone Principal Systems Engineer, Analytics SAS Customer Loyalty, SAS Institute, Inc. Is there valuable

More information

Analyzing survey text: a brief overview

Analyzing survey text: a brief overview IBM SPSS Text Analytics for Surveys Analyzing survey text: a brief overview Learn how gives you greater insight Contents 1 Introduction 2 The role of text in survey research 2 Approaches to text mining

More information

Resolving Common Analytical Tasks in Text Databases

Resolving Common Analytical Tasks in Text Databases Resolving Common Analytical Tasks in Text Databases The work is funded by the Federal Ministry of Economic Affairs and Energy (BMWi) under grant agreement 01MD15010B. Database Systems and Text-based Information

More information

Annotea and Semantic Web Supported Collaboration

Annotea and Semantic Web Supported Collaboration Annotea and Semantic Web Supported Collaboration Marja-Riitta Koivunen, Ph.D. Annotea project Abstract Like any other technology, the Semantic Web cannot succeed if the applications using it do not serve

More information

Machine Learning using MapReduce

Machine Learning using MapReduce Machine Learning using MapReduce What is Machine Learning Machine learning is a subfield of artificial intelligence concerned with techniques that allow computers to improve their outputs based on previous

More information

Data Mining Algorithms Part 1. Dejan Sarka

Data Mining Algorithms Part 1. Dejan Sarka Data Mining Algorithms Part 1 Dejan Sarka Join the conversation on Twitter: @DevWeek #DW2015 Instructor Bio Dejan Sarka (dsarka@solidq.com) 30 years of experience SQL Server MVP, MCT, 13 books 7+ courses

More information

Service Road Map for ANDS Core Infrastructure and Applications Programs

Service Road Map for ANDS Core Infrastructure and Applications Programs Service Road Map for ANDS Core and Applications Programs Version 1.0 public exposure draft 31-March 2010 Document Target Audience This is a high level reference guide designed to communicate to ANDS external

More information

Bug Report, Feature Request, or Simply Praise? On Automatically Classifying App Reviews

Bug Report, Feature Request, or Simply Praise? On Automatically Classifying App Reviews Bug Report, Feature Request, or Simply Praise? On Automatically Classifying App Reviews Walid Maalej University of Hamburg Hamburg, Germany maalej@informatik.uni-hamburg.de Hadeer Nabil University of Hamburg

More information

1 o Semestre 2007/2008

1 o Semestre 2007/2008 Departamento de Engenharia Informática Instituto Superior Técnico 1 o Semestre 2007/2008 Outline 1 2 3 4 5 Outline 1 2 3 4 5 Exploiting Text How is text exploited? Two main directions Extraction Extraction

More information

Usage and impact of controlled vocabularies in a subject repository for indexing and retrieval

Usage and impact of controlled vocabularies in a subject repository for indexing and retrieval Usage and impact of controlled vocabularies in a subject repository for indexing and retrieval Dr. Timo Borst LIBER 2011 Barcelona 29.6.-2.7.2011 ZBW is member of the Leibniz Association Overview 1. Terminology

More information

The Framework of Network Public Opinion Monitoring and Analyzing System Based on Semantic Content Identification

The Framework of Network Public Opinion Monitoring and Analyzing System Based on Semantic Content Identification The Framework of Network Public Opinion Monitoring and Analyzing System Based on Semantic Content Identification Cheng Xian-Yi1, Zhu Ling-ling,Zhu Qian,Wang Jin The Framework of Network Public Opinion

More information

So today we shall continue our discussion on the search engines and web crawlers. (Refer Slide Time: 01:02)

So today we shall continue our discussion on the search engines and web crawlers. (Refer Slide Time: 01:02) Internet Technology Prof. Indranil Sengupta Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur Lecture No #39 Search Engines and Web Crawler :: Part 2 So today we

More information

USING SPATIAL DATA MINING TO DISCOVER THE HIDDEN RULES IN THE CRIME DATA

USING SPATIAL DATA MINING TO DISCOVER THE HIDDEN RULES IN THE CRIME DATA USING SPATIAL DATA MINING TO DISCOVER THE HIDDEN RULES IN THE CRIME DATA Karel, JANEČKA 1, Hana, HŮLOVÁ 1 1 Department of Mathematics, Faculty of Applied Sciences, University of West Bohemia Abstract Univerzitni

More information

Social Media Mining. Data Mining Essentials

Social Media Mining. Data Mining Essentials Introduction Data production rate has been increased dramatically (Big Data) and we are able store much more data than before E.g., purchase data, social media data, mobile phone data Businesses and customers

More information

ecommerce Web-Site Trust Assessment Framework Based on Web Mining Approach

ecommerce Web-Site Trust Assessment Framework Based on Web Mining Approach ecommerce Web-Site Trust Assessment Framework Based on Web Mining Approach ecommerce Web-Site Trust Assessment Framework Based on Web Mining Approach Banatus Soiraya Faculty of Technology King Mongkut's

More information

Coding science news (intrinsic and extrinsic features)

Coding science news (intrinsic and extrinsic features) Coding science news (intrinsic and extrinsic features) M I G U E L Á N G E L Q U I N T A N I L L A, C A R L O S G. F I G U E R O L A T A M A R G R O V E S 2 Science news in Spain The corpus of digital

More information

MLg. Big Data and Its Implication to Research Methodologies and Funding. Cornelia Caragea TARDIS 2014. November 7, 2014. Machine Learning Group

MLg. Big Data and Its Implication to Research Methodologies and Funding. Cornelia Caragea TARDIS 2014. November 7, 2014. Machine Learning Group Big Data and Its Implication to Research Methodologies and Funding Cornelia Caragea TARDIS 2014 November 7, 2014 UNT Computer Science and Engineering Data Everywhere Lots of data is being collected and

More information

Practical Data Science with Azure Machine Learning, SQL Data Mining, and R

Practical Data Science with Azure Machine Learning, SQL Data Mining, and R Practical Data Science with Azure Machine Learning, SQL Data Mining, and R Overview This 4-day class is the first of the two data science courses taught by Rafal Lukawiecki. Some of the topics will be

More information

Text Analytics Beginner s Guide. Extracting Meaning from Unstructured Data

Text Analytics Beginner s Guide. Extracting Meaning from Unstructured Data Text Analytics Beginner s Guide Extracting Meaning from Unstructured Data Contents Text Analytics 3 Use Cases 7 Terms 9 Trends 14 Scenario 15 Resources 24 2 2013 Angoss Software Corporation. All rights

More information

Keywords social media, internet, data, sentiment analysis, opinion mining, business

Keywords social media, internet, data, sentiment analysis, opinion mining, business Volume 5, Issue 8, August 2015 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Real time Extraction

More information

Data Mining, Predictive Analytics with Microsoft Analysis Services and Excel PowerPivot

Data Mining, Predictive Analytics with Microsoft Analysis Services and Excel PowerPivot www.etidaho.com (208) 327-0768 Data Mining, Predictive Analytics with Microsoft Analysis Services and Excel PowerPivot 3 Days About this Course This course is designed for the end users and analysts that

More information

WHAT DEVELOPERS ARE TALKING ABOUT?

WHAT DEVELOPERS ARE TALKING ABOUT? WHAT DEVELOPERS ARE TALKING ABOUT? AN ANALYSIS OF STACK OVERFLOW DATA 1. Abstract We implemented a methodology to analyze the textual content of Stack Overflow discussions. We used latent Dirichlet allocation

More information

DATA MINING TECHNOLOGY. Keywords: data mining, data warehouse, knowledge discovery, OLAP, OLAM.

DATA MINING TECHNOLOGY. Keywords: data mining, data warehouse, knowledge discovery, OLAP, OLAM. DATA MINING TECHNOLOGY Georgiana Marin 1 Abstract In terms of data processing, classical statistical models are restrictive; it requires hypotheses, the knowledge and experience of specialists, equations,

More information

SILOBREAKER ENTERPRISE SOFTWARE SUITE

SILOBREAKER ENTERPRISE SOFTWARE SUITE INSIGHT AS A SERVICE SILOBREAKER ENTERPRISE SOFTWARE SUITE Fully customizable behind your firewall! Silobreaker Enterprise Software Suite is Silobreaker s flagship product. It is ideal for organizations

More information

Interactive Dynamic Information Extraction

Interactive Dynamic Information Extraction Interactive Dynamic Information Extraction Kathrin Eichler, Holmer Hemsen, Markus Löckelt, Günter Neumann, and Norbert Reithinger Deutsches Forschungszentrum für Künstliche Intelligenz - DFKI, 66123 Saarbrücken

More information

EVILSEED: A Guided Approach to Finding Malicious Web Pages

EVILSEED: A Guided Approach to Finding Malicious Web Pages + EVILSEED: A Guided Approach to Finding Malicious Web Pages Presented by: Alaa Hassan Supervised by: Dr. Tom Chothia + Outline Introduction Introducing EVILSEED. EVILSEED Architecture. Effectiveness of

More information

Facilitating Business Process Discovery using Email Analysis

Facilitating Business Process Discovery using Email Analysis Facilitating Business Process Discovery using Email Analysis Matin Mavaddat Matin.Mavaddat@live.uwe.ac.uk Stewart Green Stewart.Green Ian Beeson Ian.Beeson Jin Sa Jin.Sa Abstract Extracting business process

More information

An Overview of Knowledge Discovery Database and Data mining Techniques

An Overview of Knowledge Discovery Database and Data mining Techniques An Overview of Knowledge Discovery Database and Data mining Techniques Priyadharsini.C 1, Dr. Antony Selvadoss Thanamani 2 M.Phil, Department of Computer Science, NGM College, Pollachi, Coimbatore, Tamilnadu,

More information

DIGITAL MARKETING. The Page Title Meta Descriptions & Meta Keywords

DIGITAL MARKETING. The Page Title Meta Descriptions & Meta Keywords DIGITAL MARKETING Digital Marketing Basics Basics of advertising What is Digital Media? Digital Media Vs. Traditional Media Benefits of Digital marketing Latest Digital marketing trends Digital media marketing

More information