Text Analysis for Big Data. Magnus Sahlgren



Similar documents
How To Make Sense Of Data With Altilia

TEXT ANALYTICS INTEGRATION



Text Mining - Scope and Applications

Introduction to Data Mining and Machine Learning Techniques. Iza Moise, Evangelos Pournaras, Dirk Helbing

Computer-Based Text- and Data Analysis Technologies and Applications. Mark Cieliebak

Data Mining for Customer Service Support. Senioritis Seminar Presentation Megan Boice Jay Carter Nick Linke KC Tobin

Sample Reporting. Analytics and Evaluation

Data, Measurements, Features

Survey Results: Requirements and Use Cases for Linguistic Linked Data

STAR WARS AND THE ART OF DATA SCIENCE

Chapter ML:XI. XI. Cluster Analysis

Reinventing Business Intelligence through Big Data

Conquering the Astronomical Data Flood through Machine

Voice. listen, understand and respond. enherent. wish, choice, or opinion. openly or formally expressed. May Merriam Webster.

The Business Accelerator. Analyse your competitors, gain insights, take actions and accelerate your sales now.

Sentiment Analysis on Big Data

Introduction to IR Systems: Supporting Boolean Text Search. Information Retrieval. IR vs. DBMS. Chapter 27, Part A

Overview, Goals, & Introductions

Going Global With Social Media Analytics

Scalable Machine Learning - or what to do with all that Big Data infrastructure

Impelsys: Your Partner for Digital Product Development & Commercialization

Big Data and Natural Language: Extracting Insight From Text

IBM Content Analytics with Enterprise Search, Version 3.0

HOW TO DO A SMART DATA PROJECT

Processing big data with natural semantics and natural language understanding using brain-like approach

Ngram Search Engine with Patterns Combining Token, POS, Chunk and NE Information

How to use Big Data in Industry 4.0 implementations. LAURI ILISON, PhD Head of Big Data and Machine Learning

Real-Time Analytics: Integrating Social Media Insights with Traditional Data

DATA SCIENCE CURRICULUM WEEK 1 ONLINE PRE-WORK INSTALLING PACKAGES COMMAND LINE CODE EDITOR PYTHON STATISTICS PROJECT O5 PROJECT O3 PROJECT O2

Data Mining Part 5. Prediction

Text Analytics The three-minute guide

Big Data for the Rest of Us Technical White Paper

Discovery of Electronically Stored Information ECBA conference Tallinn October 2012

Predicting stocks returns correlations based on unstructured data sources

Anonymizing Unstructured Data to Enable Healthcare Analytics Chris Wright, Vice President Marketing, Privacy Analytics

BIG. Big Data Analysis John Domingue (STI International and The Open University) Big Data Public Private Forum

Module Design & Enhancement. Assessment Types

What do Big Data & HAVEn mean? Robert Lejnert HP Autonomy

Big Data in Subsea Solutions

Auto-Classification for Document Archiving and Records Declaration

W H I T E P A P E R. Deriving Intelligence from Large Data Using Hadoop and Applying Analytics. Abstract

Introduction to Text Mining and Semantics. Seth Grimes -- President, Alta Plana

Big Data Text Mining and Visualization. Anton Heijs

Computational Linguistics and Learning from Big Data. Gabriel Doyle UCSD Linguistics

The Importance of Analytics

CAPTURING THE VALUE OF UNSTRUCTURED DATA: INTRODUCTION TO TEXT MINING

5 Keys to Unlocking the Big Data Analytics Puzzle. Anurag Tandon Director, Product Marketing March 26, 2014

TIETS34 Seminar: Data Mining on Biometric identification

Big Data: Rethinking Text Visualization

Clustering Technique in Data Mining for Text Documents

Search and Information Retrieval

Big Data & Security. Aljosa Pasic 12/02/2015

ANALYTICS IN BIG DATA ERA

Modeling coherence in ESOL learner texts

Business Process Services. White Paper. Predictive Analytics in HR: A Primer

Doctoral Consortium 2013 Dept. Lenguajes y Sistemas Informáticos UNED

The Big Data Paradigm Shift. Insight Through Automation

Data Mining on Social Networks. Dionysios Sotiropoulos Ph.D.

Analyzing Huge Data Sets in Forensic Investigations

Management Decision Making. Hadi Hosseini CS 330 David R. Cheriton School of Computer Science University of Waterloo July 14, 2011

Machine Learning for Data Science (CS4786) Lecture 1

Predictive Analytics Techniques: What to Use For Your Big Data. March 26, 2014 Fern Halper, PhD

ONLINE RESUME PARSING SYSTEM USING TEXT ANALYTICS

Spend Enrichment: Making better decisions starts with accurate data

Modern Data Architecture for Predictive Analytics

Pattern Insight Clone Detection

Big Data. Lyle Ungar, University of Pennsylvania

Data Science and Business Analytics Certificate Data Science and Business Intelligence Certificate

Voice of the Customer: How to Move Beyond Listening to Action Merging Text Analytics with Data Mining and Predictive Analytics

New Frontiers of Automated Content Analysis in the Social Sciences

Statistics 215b 11/20/03 D.R. Brillinger. A field in search of a definition a vague concept

Augmented Search for Web Applications. New frontier in big log data analysis and application intelligence

Sentiment Analysis. D. Skrepetos 1. University of Waterloo. NLP Presenation, 06/17/2015

Turning Big Data into a Big Opportunity

Text Mining with R. Rob Zinkov. October 19th, Rob Zinkov () Text Mining with R October 19th, / 38

VTrak G1100 Application and Performance Notes

Text Analytics Software Choosing the Right Fit

Project Management Framework for the Guidelines for the Compilation of Water Accounts and Statistics

Big Data. What is Big Data? Over the past years. Big Data. Big Data: Introduction and Applications

Online Content Optimization Using Hadoop. Jyoti Ahuja Dec

Big Data and Market Surveillance. April 28, 2014

Bridging CAQDAS with text mining: Text analyst s toolbox for Big Data: Science in the Media Project

Cutting Through The Hype: What You Need To Know About Big Data

Software Development Training Camp 1 (0-3) Prerequisite : Program development skill enhancement camp, at least 48 person-hours.

HSD. W Business Analytics (M.Sc.) IT in Business Analytics. IT Applications in Business Analytics SS2016 / 01 Introduction Thomas Zeutschler

Transcription:

Text Analysis for Big Data Magnus Sahlgren

Data Size Style (editorial vs social) Language (there are other languages than English out there!)

Data Size Style (editorial vs social) Language (there are other languages than English out there!)

Data Size Style (editorial vs social) Language (there are other languages than English out there!)

Data Size Style (editorial vs social) Language (there are other languages than English out there!)

Data Size Style (editorial vs social) Language (there are other languages than English out there!)

Data Size Style (editorial vs social) Language (there are other languages than English out there!)

Data Size Style (editorial vs social) Language (there are other languages than English out there!)

Technologies Knowledge-based (use resources like Wikipedia) Supervised machine learning (use annotated data) Unsupervised machine learning (use unstructured data)

Technologies Knowledge-based (use resources like Wikipedia) Supervised machine learning (use annotated data) Unsupervised machine learning (use unstructured data)

Technologies Knowledge-based (use resources like Wikipedia) Supervised machine learning (use annotated data) Unsupervised machine learning (use unstructured data)

Technologies Knowledge-based (use resources like Wikipedia) Supervised machine learning (use annotated data) Unsupervised machine learning (use unstructured data)

Technologies Knowledge-based (use resources like Wikipedia) Supervised machine learning (use annotated data) Unsupervised machine learning (use unstructured data)

Technologies Knowledge-based (use resources like Wikipedia) Supervised machine learning (use annotated data) Unsupervised machine learning (use unstructured data)

Technologies Knowledge-based (use resources like Wikipedia) Supervised machine learning (use annotated data) Unsupervised machine learning (use unstructured data)

Semantic memories

Semantic memories (systems that learn language by reading large amounts of text)

Semantic memories (systems that learn language by reading large amounts of text)

Insights Identify and extract items (e.g. entities and events) Find relations (e.g. synonyms and associations) Compress and refine the information (e.g. summarization and topic detection) Measure things (e.g. attitudes and opinions)

Insights Identify and extract items (e.g. entities and events) Find relations (e.g. synonyms and associations) Compress and refine the information (e.g. summarization and topic detection) Measure things (e.g. attitudes and opinions)

Insights Identify and extract items (e.g. entities and events) Find relations (e.g. synonyms and associations) Compress and refine the information (e.g. summarization and topic detection) Measure things (e.g. attitudes and opinions)

Insights Identify and extract items (e.g. entities and events) Find relations (e.g. synonyms and associations) Compress and refine the information (e.g. summarization and topic detection) Measure things (e.g. attitudes and opinions)

Insights Identify and extract items (e.g. entities and events) Find relations (e.g. synonyms and associations) Compress and refine the information (e.g. summarization and topic detection) Measure things (e.g. attitudes and opinions)

Find relations

Find relations

Find relations (lexicon.gavagai.se)

Insights Identify and extract items (e.g. entities and events) Find relations (e.g. synonyms and associations) Compress and refine the information (e.g. summarization and topic detection) Measure things (e.g. attitudes and opinions)

Insights Identify and extract items (e.g. entities and events) Find relations (e.g. synonyms and associations) Compress and refine the information (e.g. summarization and topic detection) Measure things (e.g. attitudes and opinions)

Insights Identify and extract items (e.g. entities and events) Find relations (e.g. synonyms and associations) Compress and refine the information (e.g. summarization and topic detection) Measure things (e.g. attitudes and opinions)

Compress and refine Summarization and topic detection

Compress and refine (monitor.gavagai.se) Summarization and topic detection

Compress and refine (monitor.gavagai.se) Summarization and topic detection

Compress and refine (monitor.gavagai.se) Summarization and topic detection

Insights Identify and extract items (e.g. entities and events) Find relations (e.g. synonyms and associations) Compress and refine the information (e.g. summarization and topic detection) Measure things (e.g. attitudes and opinions)

Insights Identify and extract items (e.g. entities and events) Find relations (e.g. synonyms and associations) Compress and refine the information (e.g. summarization and topic detection) Measure things (e.g. attitudes and opinions)

Insights Identify and extract items (e.g. entities and events) Find relations (e.g. synonyms and associations) Compress and refine the information (e.g. summarization and topic detection) Measure things (e.g. attitudes and opinions)

Measure Sentiment analysis

Measure Sentiment analysis Positivity vs negativity wrt the global economy in English online media

Measure Sentiment analysis Worry wrt the global economy in English online media

Measure Sentiment analysis Negativity towards China in English online media

Measure Sentiment analysis Attitude towards Sweden in Russian online media

Measure Predict

Measure Predict Rönnqvist & Sarlin (2015): Detect & Describe: deep learning of bank stress in the news