Text Mining and its Applications to Intelligence, CRM and Knowledge Management

Size: px
Start display at page:

Download "Text Mining and its Applications to Intelligence, CRM and Knowledge Management"

Transcription

1 Text Mining and its Applications to Intelligence, CRM and Knowledge Management Editor A. Zanasi TEMS Text Mining Solutions S.A. Italy WITPRESS Southampton, Boston

2 Contents Bibliographies Preface Text Mining: a new technology paradigm? xix xxvii Part 1: THEORETICAL OVERVIEW Chapter 1: Text processing and information retrieval M. Milic-Frayling 1 1 Introduction 1 2 Data gathering and extraction of text Encoding of textual information Document formats and markup languages Data collection in distributed hyperlinked environments Web crawling Distributed architecture - 'Harvest' 6 3 Text processing From grammar rules to statistical NLP Basic linguistic concepts Text processing Word segmentation Word conflation and disambiguation Part of speech tagging Parsing 16 4 Information retrieval Basic concepts Indexing Feature selection Feature weighting Retrieval models Boolean retrieval Vector space retrieval Probabilistic information retrieval Search refinement with user relevance feedback Evaluation Evaluation measures TREC - Large scale evaluation initiatives 31 5 Concluding remarks 39

3 Chapter 2: Information extraction and... Surroundings M.T.Pazienza 47 1 Introduction What is information retrieval (IR)? What is information extraction? What is full text understanding? What is mining? Is there a difference between information extraction from web documents and traditional ones? 50 2 Information extraction historical flash-back 51 3 IE systems architecture Text zoning Pre-processor Filter Pre-parser Parser Fragment combination Semantic interpretation Lexical disambiguation Co-reference resolution, discourse processing Template generation 55 4 Features of an IE system The parsing role The scenario's role 62 5 Adaptive IE systems Machine-learning for information extraction 64 6 IE systems: a few European applications NAMIC Large scale IE for automatic authoring The role of a world model as a method for event matching and co-referencing Named entity matcher Discourse processor Ontological and lexical information The NAMIC architecture CROSSMARC The CROSSMARC IE component Cross-lingual named entity recognition and classification CROSSMARC fact extraction CROSSMARC fact definition CROSSMARC ontology 70 Chapter 3: Text clustering as a mining task F. Mandreoli, R. Martoglia & P. Tiberio 75 1 Introduction, Overview on data clustering analysis 78

4 2.1 Similarity measures Clustering techniques Single-link and complete-link hierarchical methods K-means partitional methods 82 3 Problems and solutions in the text clustering field Effective extraction of meaningful features from plain texts Effective treatment of high dimensionality Interpretability of results Efficiency and scalability of the clustering process Feature selection and reduction Efficient clustering of large unstructured data sets K-Means clustering variants Relational data analysis (RDA) clustering New clustering approaches from the DataBase community Document-specific clustering approaches A short note about memory management and distribution Comprehension and navigation of clustering output Clustering web documents 99 4 Conclusions 104 Chapter 4: Text categorization F. Sebastiani Introduction The basic picture Document indexing Classifier learning Classifier evaluation Techniques Document indexing techniques Classifier learning techniques Support vector machines Boosting Applications Automatic indexing for Boolean information retrieval systems Document organization Text filtering Hierarchical categorization of web pages Word sense disambiguation Automated survey coding Automated authorship attribution and-genre classification Spam filtering Other applications Conclusion Notes 123

5 Chapter 5: Summarization and visualization D. Mladenic & M. Grobelnik Introduction Text summarization Keywords Extracting keywords from text Keyword assignment using document categorization Sentence extraction Abstract generation Text visualization Example of summarization of a document set Future directions of research and applications 140 Part 2: APPLICATIONS Chapter 6: Application integration in applied text mining D.Sullivan Introduction Business drivers and application types Customer transaction analysis Competitive intelligence Research and development support Application elements Content acquisition Internal content acquisition External content source Rights management Pre-processing Linguistic analysis Term co-occurrence Entity extraction Information extraction User analysis Content repository Security and access controls Conclusions 153 Chapter 7: ROI in text mining projects M.Ferrari Introduction The evaluation of a text mining solution The evaluation of the tangible components The evaluation of the tangible components in a text mining project: an example The evaluation of the intangible components The value chain Scoreboard 164

6 4.2 The Intangible Asset Monitor and The Skandia Navigator The balanced scorecard Conclusions 177 a) INTELLIGENCE Chapter 8: Open sources automatic analysis for corporate and government intelligence A. Zanasi Introduction New government intelligence role New challenges to the market state New intelligence cycle A help from corporate intelligence Corporate intelligence Competitive intelligence definitions CI questions Where are the answers? Open sources Definition Internet data Hosts Online databanks Proprietary sources The open sources analysis problems Other needs: forecasting and early warning systems Terrorism and other challenges to government intelligence Introduction Homeland security: DARPA and HSARPA vision Information, an arm against the asymmetric threats EELD program TIDES program Anti-terrorism tasks requiring analysis of a large quantity of text High tech terrorism Names and relationships detection Vindications analysis Info spam Identifying lobbying Monitoring a specific market sector Money laundering Insider trading Practical examples of text mining applied to the intelligence process Characteristics of high quality intelligence The modules to implement the intelligence process 196

7 6.2.1 Business discovery Solution definition Research strategy Analysis Results analysis and interpretation The reachable objectives Business cases Forecasting competitor actions Detecting competitor action in market Alliances detection Business opportunities detection Predicting biowarfare agents Supply chain management and purchasing activity Military strategy Extracting terminologies R&D activity detection ; 207 Chapter 9: A critical appraisal of text mining in an intelligence environment A.Politi Introduction Sept., intelligence and information explosion Data mining: some world relevant examples Data mining, the intelligence cycle and decision 214 Chapter 10: Marketing intelligence system to forecast telecommunications competitive landscape S.de'Rossi Introduction Italian mobile market overview TIM positioning From competitive to market intelligence Our needs The business model The intelligence needs The information source Building up the system 224 Chapter 11: Competitive intelligence for SMEs: An application to the Italian building sector G. Casoni What was the problem Edilintelligence: what is it? The text mining bricks of the solution: Theory and practice Conclusions 234

8 b)crm Chapter 12: Virtual communities: human capital and other personal characteristics extraction A. Zanasi The emergence of neo-renaissance paradigm Intellectual and human capital The real wealth Intellectual capital taxonomy Virtual communities: where text mining is applied Community structuring Participant interaction and access Content management Community leveraging Human capital in customer communities Human capital in employee community An example of human capital: Employee attitudes Vital signs monitor VSM key concepts definition Human capital in social contexts Defining anonymous terrorist authorship Digital signatures Lobby detection Monitoring of specific areas/sectors Chatlines and other open sources analysis Social network links detection Social structure Graphical representation of connections 246 Chapter 13: Customer feedbacks and opinion surveys analysis in the automotive industry L. Grivel Introduction Customer feedback analysis in Renault Objectives and problem description Analysis The technology Information extraction Skill cartridges An ontology for CRM applications Solution Feedback Opinion surveys for automotive manufacturers Objectives and problem description Analysis Technology 254

9 3.3.1 General dictionaries and rules Automotive specific dictionaries and rules Implemented solution Feedback Conclusions 257 Chapter 14: The Responsio management system M. Kockelkorn & T. Scheffer Introduction answering by semi-supervised text classification Responsio management system Case study Discussion 263 Chapter 15: TV channel provider: mining the user feedback L.K. Wives, S. Loh, J.L. Duizith&J.P. Moreira de Oliveira Introduction The case The process Conclusion 268 c) KNOWLEDGE MANAGEMENT Chapter 16: Text mining based knowledge management in banking K. Lebeth, M. Lorenz&U. Storl Introduction The document as a primary source Knowledge based search Building up a knowledge management infrastructure Integrating principles Modules Term extractor Knowledge Net Automated metatagging engine Conclusion and future work 277 Chapter 17: Text mining in life sciences J. Fluck, H. Deneke&C. Gieger Introduction Text mining - current state Methodical development Applications in life sciences Ontology development, Conclusion 283

10 Chapter 18: Information search and classification to foster innovation in SMEs The AREA Science Park experience F.Neri The AREA Science Park and its technology transfer division TEMIS online miner light, the TTD search engine for patents (TTDSE) Data selection TTDSE back-end: the knowledge extractor TTDSE front-end: the advanced search engine TTD results 289 Chapter 19: Media industry: how to improve documentalists efficiency G.Peters Introduction Text data production in media Indexing textual data Archive solutions: data bases and automatic procedures Text mining experience in Gruner + Jahr Overview The need Performance measures Extraction Personal names Organization names Utilization: lessons leamt Customization Training Documentalists Savings Conclusion 297 Chapter 20: Link analysis in crime pattern detection S.Ananyan Introduction Case overview Implementation approach Data preprocessing Structured data analysis Concept extraction Pattern analysis ' Drill-down and reporting Drill-down and reporting Automation Conclusion 313

11 Part 3: SOFTWARE Chapter 21: Text mining tools A. Zartasi Megaputer intelligence Company description Products Incorporation of domain knowledge Exporting discovered knowledge Supported languages IT requirements Customer base Partners Supported applications SAS Company description Product Incorporation of domain knowledge Supported languages IT requirements SPSS Company description Product functionality Incorporation of domain knowledge Exporting discovered knowledge Supported languages IT requirements Marketing information LexiQuest Mine a text mining application LexiQuest Categorize a categorization engine Text Mining for Clementine an add-on to the data mining suite Customer base Synthema Company description Product Incorporation of domain knowledge Exporting discovered knowledge Supported languages IT requirements Customer base TEMIS Company description Products Insight discoverer clusterer (IDC) Insight discoverer categorizer (IDK) 323

12 5.2.3 Insight discoverer extractor (IDE) and skills cartridges (SQ Online miner (OM) Xelda Incorporation of domain knowledge Supported languages IT requirements Customer base Partners 325 Others Autonomy Clearforest Convera Entrieva Fast IBM Insightful Inxight Verity 327

TEXT ANALYTICS INTEGRATION

TEXT ANALYTICS INTEGRATION TEXT ANALYTICS INTEGRATION A TELECOMMUNICATIONS BEST PRACTICES CASE STUDY VISION COMMON ANALYTICAL ENVIRONMENT Structured Unstructured Analytical Mining Text Discovery Text Categorization Text Sentiment

More information

Megaputer Intelligence

Megaputer Intelligence Megaputer Intelligence Company Profile www.megaputer.com 2012 Megaputer Intelligence Inc. Megaputer Intelligence Knowledge discovery tools for business users Easy-to-understand actionable results Data

More information

Delivering Smart Answers!

Delivering Smart Answers! Companion for SharePoint Topic Analyst Companion for SharePoint All Your Information Enterprise-ready Enrich SharePoint, your central place for document and workflow management, not only with an improved

More information

Introduction to Data Mining

Introduction to Data Mining Introduction to Data Mining Jay Urbain Credits: Nazli Goharian & David Grossman @ IIT Outline Introduction Data Pre-processing Data Mining Algorithms Naïve Bayes Decision Tree Neural Network Association

More information

Text Mining: The state of the art and the challenges

Text Mining: The state of the art and the challenges Text Mining: The state of the art and the challenges Ah-Hwee Tan Kent Ridge Digital Labs 21 Heng Mui Keng Terrace Singapore 119613 Email: ahhwee@krdl.org.sg Abstract Text mining, also known as text data

More information

Search and Information Retrieval

Search and Information Retrieval Search and Information Retrieval Search on the Web 1 is a daily activity for many people throughout the world Search and communication are most popular uses of the computer Applications involving search

More information

Bing Liu. Web Data Mining. Exploring Hyperlinks, Contents, and Usage Data. With 177 Figures. ~ Spring~r

Bing Liu. Web Data Mining. Exploring Hyperlinks, Contents, and Usage Data. With 177 Figures. ~ Spring~r Bing Liu Web Data Mining Exploring Hyperlinks, Contents, and Usage Data With 177 Figures ~ Spring~r Table of Contents 1. Introduction.. 1 1.1. What is the World Wide Web? 1 1.2. ABrief History of the Web

More information

Survey Results: Requirements and Use Cases for Linguistic Linked Data

Survey Results: Requirements and Use Cases for Linguistic Linked Data Survey Results: Requirements and Use Cases for Linguistic Linked Data 1 Introduction This survey was conducted by the FP7 Project LIDER (http://www.lider-project.eu/) as input into the W3C Community Group

More information

Introduction. A. Bellaachia Page: 1

Introduction. A. Bellaachia Page: 1 Introduction 1. Objectives... 3 2. What is Data Mining?... 4 3. Knowledge Discovery Process... 5 4. KD Process Example... 7 5. Typical Data Mining Architecture... 8 6. Database vs. Data Mining... 9 7.

More information

An Open Platform for Collecting Domain Specific Web Pages and Extracting Information from Them

An Open Platform for Collecting Domain Specific Web Pages and Extracting Information from Them An Open Platform for Collecting Domain Specific Web Pages and Extracting Information from Them Vangelis Karkaletsis and Constantine D. Spyropoulos NCSR Demokritos, Institute of Informatics & Telecommunications,

More information

IBM SPSS Modeler Premium

IBM SPSS Modeler Premium IBM SPSS Modeler Premium Improve model accuracy with structured and unstructured data, entity analytics and social network analysis Highlights Solve business problems faster with analytical techniques

More information

A Systemic Artificial Intelligence (AI) Approach to Difficult Text Analytics Tasks

A Systemic Artificial Intelligence (AI) Approach to Difficult Text Analytics Tasks A Systemic Artificial Intelligence (AI) Approach to Difficult Text Analytics Tasks Text Analytics World, Boston, 2013 Lars Hard, CTO Agenda Difficult text analytics tasks Feature extraction Bio-inspired

More information

Hexaware E-book on Predictive Analytics

Hexaware E-book on Predictive Analytics Hexaware E-book on Predictive Analytics Business Intelligence & Analytics Actionable Intelligence Enabled Published on : Feb 7, 2012 Hexaware E-book on Predictive Analytics What is Data mining? Data mining,

More information

IBM Content Analytics with Enterprise Search, Version 3.0

IBM Content Analytics with Enterprise Search, Version 3.0 IBM Content Analytics with Enterprise Search, Version 3.0 Highlights Enables greater accuracy and control over information with sophisticated natural language processing capabilities to deliver the right

More information

Text Analytics Software Choosing the Right Fit

Text Analytics Software Choosing the Right Fit Text Analytics Software Choosing the Right Fit Tom Reamy Chief Knowledge Architect KAPS Group http://www.kapsgroup.com Text Analytics World San Francisco, 2013 Agenda Introduction Text Analytics Basics

More information

Decision Support Optimization through Predictive Analytics - Leuven Statistical Day 2010

Decision Support Optimization through Predictive Analytics - Leuven Statistical Day 2010 Decision Support Optimization through Predictive Analytics - Leuven Statistical Day 2010 Ernst van Waning Senior Sales Engineer May 28, 2010 Agenda SPSS, an IBM Company SPSS Statistics User-driven product

More information

Using text mining to understand the call center customers claims

Using text mining to understand the call center customers claims Data Mining VII: Data, Text and Web Mining and their Business Applications 177 Using text mining to understand the call center customers claims G. M. Caputo, V. M. Bastos & N. F. F. Ebecken COPPE Federal

More information

Why are Organizations Interested?

Why are Organizations Interested? SAS Text Analytics Mary-Elizabeth ( M-E ) Eddlestone SAS Customer Loyalty M-E.Eddlestone@sas.com +1 (607) 256-7929 Why are Organizations Interested? Text Analytics 2009: User Perspectives on Solutions

More information

BUSINESS VALUE OF SEMANTIC TECHNOLOGY

BUSINESS VALUE OF SEMANTIC TECHNOLOGY BUSINESS VALUE OF SEMANTIC TECHNOLOGY Preliminary Findings Industry Advisory Council Emerging Technology (ET) SIG Information Sharing & Collaboration Committee July 15, 2005 Mills Davis Managing Director

More information

Microsoft FAST Search Server 2010 for SharePoint Evaluation Guide

Microsoft FAST Search Server 2010 for SharePoint Evaluation Guide Microsoft FAST Search Server 2010 for SharePoint Evaluation Guide 1 www.microsoft.com/sharepoint The information contained in this document represents the current view of Microsoft Corporation on the issues

More information

Natural Language Processing-Based COTS Software and Related Technologies Survey

Natural Language Processing-Based COTS Software and Related Technologies Survey SANDIA REPORT SAND2003-2916 Unlimited Release Printed September 2003 Natural Language Processing-Based COTS Software and Related Technologies Survey Michael G. Stickland, Shelley M. Eaton and Gregory N.

More information

C o p yr i g ht 2015, S A S I nstitute Inc. A l l r i g hts r eser v ed. INTRODUCTION TO SAS TEXT MINER

C o p yr i g ht 2015, S A S I nstitute Inc. A l l r i g hts r eser v ed. INTRODUCTION TO SAS TEXT MINER INTRODUCTION TO SAS TEXT MINER TODAY S AGENDA INTRODUCTION TO SAS TEXT MINER Define data mining Overview of SAS Enterprise Miner Describe text analytics and define text data mining Text Mining Process

More information

What do Big Data & HAVEn mean? Robert Lejnert HP Autonomy

What do Big Data & HAVEn mean? Robert Lejnert HP Autonomy What do Big Data & HAVEn mean? Robert Lejnert HP Autonomy Much higher Volumes. Processed with more Velocity. With much more Variety. Is Big Data so big? Big Data Smart Data Project HAVEn: Adaptive Intelligence

More information

dm106 TEXT MINING FOR CUSTOMER RELATIONSHIP MANAGEMENT: AN APPROACH BASED ON LATENT SEMANTIC ANALYSIS AND FUZZY CLUSTERING

dm106 TEXT MINING FOR CUSTOMER RELATIONSHIP MANAGEMENT: AN APPROACH BASED ON LATENT SEMANTIC ANALYSIS AND FUZZY CLUSTERING dm106 TEXT MINING FOR CUSTOMER RELATIONSHIP MANAGEMENT: AN APPROACH BASED ON LATENT SEMANTIC ANALYSIS AND FUZZY CLUSTERING ABSTRACT In most CRM (Customer Relationship Management) systems, information on

More information

CAPTURING THE VALUE OF UNSTRUCTURED DATA: INTRODUCTION TO TEXT MINING

CAPTURING THE VALUE OF UNSTRUCTURED DATA: INTRODUCTION TO TEXT MINING CAPTURING THE VALUE OF UNSTRUCTURED DATA: INTRODUCTION TO TEXT MINING Mary-Elizabeth ( M-E ) Eddlestone Principal Systems Engineer, Analytics SAS Customer Loyalty, SAS Institute, Inc. Is there valuable

More information

ELPUB Digital Library v2.0. Application of semantic web technologies

ELPUB Digital Library v2.0. Application of semantic web technologies ELPUB Digital Library v2.0 Application of semantic web technologies Anand BHATT a, and Bob MARTENS b a ABA-NET/Architexturez Imprints, New Delhi, India b Vienna University of Technology, Vienna, Austria

More information

Web Mining. Margherita Berardi LACAM. Dipartimento di Informatica Università degli Studi di Bari berardi@di.uniba.it

Web Mining. Margherita Berardi LACAM. Dipartimento di Informatica Università degli Studi di Bari berardi@di.uniba.it Web Mining Margherita Berardi LACAM Dipartimento di Informatica Università degli Studi di Bari berardi@di.uniba.it Bari, 24 Aprile 2003 Overview Introduction Knowledge discovery from text (Web Content

More information

[JOINT WHITE PAPER] Ontos Semantic Factory

[JOINT WHITE PAPER] Ontos Semantic Factory [] Ontos Semantic Factory JANUARY 2009 02/ 7 Executive Summary In this paper we describe Ontos Semantic Factory a platform producing semantic metadata on the basis of text (Web) content. The technology

More information

An Introduction to Data Mining

An Introduction to Data Mining An Introduction to Intel Beijing wei.heng@intel.com January 17, 2014 Outline 1 DW Overview What is Notable Application of Conference, Software and Applications Major Process in 2 Major Tasks in Detail

More information

Internet of Things, data management for healthcare applications. Ontology and automatic classifications

Internet of Things, data management for healthcare applications. Ontology and automatic classifications Internet of Things, data management for healthcare applications. Ontology and automatic classifications Inge.Krogstad@nor.sas.com SAS Institute Norway Different challenges same opportunities! Data capture

More information

TDPA: Trend Detection and Predictive Analytics

TDPA: Trend Detection and Predictive Analytics TDPA: Trend Detection and Predictive Analytics M. Sakthi ganesh 1, CH.Pradeep Reddy 2, N.Manikandan 3, DR.P.Venkata krishna 4 1. Assistant Professor, School of Information Technology & Engineering (SITE),

More information

F. Aiolli - Sistemi Informativi 2007/2008

F. Aiolli - Sistemi Informativi 2007/2008 Text Categorization Text categorization (TC - aka text classification) is the task of buiding text classifiers, i.e. sofware systems that classify documents from a domain D into a given, fixed set C =

More information

Text Mining and Analysis

Text Mining and Analysis Text Mining and Analysis Practical Methods, Examples, and Case Studies Using SAS Goutam Chakraborty, Murali Pagolu, Satish Garla From Text Mining and Analysis. Full book available for purchase here. Contents

More information

What you can accomplish with IBMContent Analytics

What you can accomplish with IBMContent Analytics What you can accomplish with IBMContent Analytics An Enterprise Content Management solution What is IBM Content Analytics? Alex On February 14-16, IBM s Watson computing system made its television debut

More information

Knowledge Management

Knowledge Management Knowledge Management Management Information Code: 164292-02 Course: Management Information Period: Autumn 2013 Professor: Sync Sangwon Lee, Ph. D D. of Information & Electronic Commerce 1 00. Contents

More information

Bridging CAQDAS with text mining: Text analyst s toolbox for Big Data: Science in the Media Project

Bridging CAQDAS with text mining: Text analyst s toolbox for Big Data: Science in the Media Project Bridging CAQDAS with text mining: Text analyst s toolbox for Big Data: Science in the Media Project Ahmet Suerdem Istanbul Bilgi University; LSE Methodology Dept. Science in the media project is funded

More information

Improve Model Accuracy with Unstructured Data

Improve Model Accuracy with Unstructured Data IBM SPSS Modeler Premium Improve Model Accuracy with Unstructured Data Highlights Easily access, prepare and integrate structured data and text, Web and survey data Support the entire data mining process

More information

Real Time Data Detecting Trend Process and Predictions using Living Analytics

Real Time Data Detecting Trend Process and Predictions using Living Analytics Real Time Data Detecting Trend Process and Predictions using Living Analytics Dr. G. Murugan Professor and Research Analyst, Velsoft Technologies, Chennai, India ABSTRACT: Real time system is a highly

More information

Chapter ML:XI. XI. Cluster Analysis

Chapter ML:XI. XI. Cluster Analysis Chapter ML:XI XI. Cluster Analysis Data Mining Overview Cluster Analysis Basics Hierarchical Cluster Analysis Iterative Cluster Analysis Density-Based Cluster Analysis Cluster Evaluation Constrained Cluster

More information

Text Analytics Evaluation Case Study - Amdocs

Text Analytics Evaluation Case Study - Amdocs Text Analytics Evaluation Case Study - Amdocs Tom Reamy Chief Knowledge Architect KAPS Group http://www.kapsgroup.com Text Analytics World October 20 New York Agenda Introduction Text Analytics Basics

More information

Module Catalogue for the Bachelor Program in Computational Linguistics at the University of Heidelberg

Module Catalogue for the Bachelor Program in Computational Linguistics at the University of Heidelberg Module Catalogue for the Bachelor Program in Computational Linguistics at the University of Heidelberg March 1, 2007 The catalogue is organized into sections of (1) obligatory modules ( Basismodule ) that

More information

Data Mining Analytics for Business Intelligence and Decision Support

Data Mining Analytics for Business Intelligence and Decision Support Data Mining Analytics for Business Intelligence and Decision Support Chid Apte, T.J. Watson Research Center, IBM Research Division Knowledge Discovery and Data Mining (KDD) techniques are used for analyzing

More information

Using LSI for Implementing Document Management Systems Turning unstructured data from a liability to an asset.

Using LSI for Implementing Document Management Systems Turning unstructured data from a liability to an asset. White Paper Using LSI for Implementing Document Management Systems Turning unstructured data from a liability to an asset. Using LSI for Implementing Document Management Systems By Mike Harrison, Director,

More information

Direct-to-Company Feedback Implementations

Direct-to-Company Feedback Implementations SEM Experience Analytics Direct-to-Company Feedback Implementations SEM Experience Analytics Listening System for Direct-to-Company Feedback Implementations SEM Experience Analytics delivers real sentiment,

More information

Provalis Research Text Analytics and the Victory Index

Provalis Research Text Analytics and the Victory Index point Provalis Research Text Analytics and the Victory Index Fern Halper, Ph.D. Fellow Daniel Kirsch Senior Analyst Provalis Research Text Analytics and the Victory Index Unstructured data is everywhere

More information

Architecture of an Ontology-Based Domain- Specific Natural Language Question Answering System

Architecture of an Ontology-Based Domain- Specific Natural Language Question Answering System Architecture of an Ontology-Based Domain- Specific Natural Language Question Answering System Athira P. M., Sreeja M. and P. C. Reghuraj Department of Computer Science and Engineering, Government Engineering

More information

BUSINESS INTELLIGENCE

BUSINESS INTELLIGENCE BUSINESS INTELLIGENCE Microsoft Dynamics NAV BUSINESS INTELLIGENCE Driving better business performance for companies with changing needs White Paper Date: January 2007 www.microsoft.com/dynamics/nav Table

More information

INTERNATIONAL MASTER IN BUSINESS ANALYTICS AND BIG DATA

INTERNATIONAL MASTER IN BUSINESS ANALYTICS AND BIG DATA POLITECNICO DI MILANO GRADUATE SCHOOL OF BUSINESS BABD INTERNATIONAL MASTER IN BUSINESS ANALYTICS AND BIG DATA Courses Description A JOINT PROGRAM WITH POLITECNICO DI MILANO SCHOOL OF MANAGEMENT PRE-COURSES

More information

SILOBREAKER ENTERPRISE SOFTWARE SUITE

SILOBREAKER ENTERPRISE SOFTWARE SUITE INSIGHT AS A SERVICE SILOBREAKER ENTERPRISE SOFTWARE SUITE Fully customizable behind your firewall! Silobreaker Enterprise Software Suite is Silobreaker s flagship product. It is ideal for organizations

More information

A Capability Model for Business Analytics: Part 2 Assessing Analytic Capabilities

A Capability Model for Business Analytics: Part 2 Assessing Analytic Capabilities A Capability Model for Business Analytics: Part 2 Assessing Analytic Capabilities The first article of this series presented the capability model for business analytics that is illustrated in Figure One.

More information

HOW TO MAKE SENSE OF BIG DATA TO BETTER DRIVE BUSINESS PROCESSES, IMPROVE DECISION-MAKING, AND SUCCESSFULLY COMPETE IN TODAY S MARKETS.

HOW TO MAKE SENSE OF BIG DATA TO BETTER DRIVE BUSINESS PROCESSES, IMPROVE DECISION-MAKING, AND SUCCESSFULLY COMPETE IN TODAY S MARKETS. HOW TO MAKE SENSE OF BIG DATA TO BETTER DRIVE BUSINESS PROCESSES, IMPROVE DECISION-MAKING, AND SUCCESSFULLY COMPETE IN TODAY S MARKETS. ALTILIA turns Big Data into Smart Data and enables businesses to

More information

3/17/2009. Knowledge Management BIKM eclassifier Integrated BIKM Tools

3/17/2009. Knowledge Management BIKM eclassifier Integrated BIKM Tools Paper by W. F. Cody J. T. Kreulen V. Krishna W. S. Spangler Presentation by Dylan Chi Discussion by Debojit Dhar THE INTEGRATION OF BUSINESS INTELLIGENCE AND KNOWLEDGE MANAGEMENT BUSINESS INTELLIGENCE

More information

Data Warehousing and Data Mining in Business Applications

Data Warehousing and Data Mining in Business Applications 133 Data Warehousing and Data Mining in Business Applications Eesha Goel CSE Deptt. GZS-PTU Campus, Bathinda. Abstract Information technology is now required in all aspect of our lives that helps in business

More information

Taking A Proactive Approach To Loyalty & Retention

Taking A Proactive Approach To Loyalty & Retention THE STATE OF Customer Analytics Taking A Proactive Approach To Loyalty & Retention By Kerry Doyle An Exclusive Research Report UBM TechWeb research conducted an online study of 339 marketing professionals

More information

Search and Data Mining: Techniques. Text Mining Anya Yarygina Boris Novikov

Search and Data Mining: Techniques. Text Mining Anya Yarygina Boris Novikov Search and Data Mining: Techniques Text Mining Anya Yarygina Boris Novikov Introduction Generally used to denote any system that analyzes large quantities of natural language text and detects lexical or

More information

Ganzheitliches Datenmanagement

Ganzheitliches Datenmanagement Ganzheitliches Datenmanagement für Hadoop Michael Kohs, Senior Sales Consultant @mikchaos The Problem with Big Data Projects in 2016 Relational, Mainframe Documents and Emails Data Modeler Data Scientist

More information

This Symposium brought to you by www.ttcus.com

This Symposium brought to you by www.ttcus.com This Symposium brought to you by www.ttcus.com Linkedin/Group: Technology Training Corporation @Techtrain Technology Training Corporation www.ttcus.com Big Data Analytics as a Service (BDAaaS) Big Data

More information

Clustering Technique in Data Mining for Text Documents

Clustering Technique in Data Mining for Text Documents Clustering Technique in Data Mining for Text Documents Ms.J.Sathya Priya Assistant Professor Dept Of Information Technology. Velammal Engineering College. Chennai. Ms.S.Priyadharshini Assistant Professor

More information

Flattening Enterprise Knowledge

Flattening Enterprise Knowledge Flattening Enterprise Knowledge Do you Control Your Content or Does Your Content Control You? 1 Executive Summary: Enterprise Content Management (ECM) is a common buzz term and every IT manager knows it

More information

Reasoning Component Architecture

Reasoning Component Architecture Architecture of a Spam Filter Application By Avi Pfeffer A spam filter consists of two components. In this article, based on my book Practical Probabilistic Programming, first describe the architecture

More information

Social Media Implementations

Social Media Implementations SEM Experience Analytics Social Media Implementations SEM Experience Analytics delivers real sentiment, meaning and trends within social media for many of the world s leading consumer brand companies.

More information

The Future of Business Analytics is Now! 2013 IBM Corporation

The Future of Business Analytics is Now! 2013 IBM Corporation The Future of Business Analytics is Now! 1 The pressures on organizations are at a point where analytics has evolved from a business initiative to a BUSINESS IMPERATIVE More organization are using analytics

More information

Text Analytics Beginner s Guide. Extracting Meaning from Unstructured Data

Text Analytics Beginner s Guide. Extracting Meaning from Unstructured Data Text Analytics Beginner s Guide Extracting Meaning from Unstructured Data Contents Text Analytics 3 Use Cases 7 Terms 9 Trends 14 Scenario 15 Resources 24 2 2013 Angoss Software Corporation. All rights

More information

PSG College of Technology, Coimbatore-641 004 Department of Computer & Information Sciences BSc (CT) G1 & G2 Sixth Semester PROJECT DETAILS.

PSG College of Technology, Coimbatore-641 004 Department of Computer & Information Sciences BSc (CT) G1 & G2 Sixth Semester PROJECT DETAILS. PSG College of Technology, Coimbatore-641 004 Department of Computer & Information Sciences BSc (CT) G1 & G2 Sixth Semester PROJECT DETAILS Project Project Title Area of Abstract No Specialization 1. Software

More information

Text Mining - Scope and Applications

Text Mining - Scope and Applications Journal of Computer Science and Applications. ISSN 2231-1270 Volume 5, Number 2 (2013), pp. 51-55 International Research Publication House http://www.irphouse.com Text Mining - Scope and Applications Miss

More information

IBM ediscovery Identification and Collection

IBM ediscovery Identification and Collection IBM ediscovery Identification and Collection Turning unstructured data into relevant data for intelligent ediscovery Highlights Analyze data in-place with detailed data explorers to gain insight into data

More information

Building a Database to Predict Customer Needs

Building a Database to Predict Customer Needs INFORMATION TECHNOLOGY TopicalNet, Inc (formerly Continuum Software, Inc.) Building a Database to Predict Customer Needs Since the early 1990s, organizations have used data warehouses and data-mining tools

More information

Animation. Intelligence. Business. Computer. Areas of Focus. Master of Science Degree Program

Animation. Intelligence. Business. Computer. Areas of Focus. Master of Science Degree Program Business Intelligence Computer Animation Master of Science Degree Program The Bachelor explosive of growth Science of Degree from the Program Internet, social networks, business networks, as well as the

More information

Using Artificial Intelligence to Manage Big Data for Litigation

Using Artificial Intelligence to Manage Big Data for Litigation FEBRUARY 3 5, 2015 / THE HILTON NEW YORK Using Artificial Intelligence to Manage Big Data for Litigation Understanding Artificial Intelligence to Make better decisions Improve the process Allay the fear

More information

Term Discrimination Based Robust Text Classification with Application to Email Spam Filtering. PhD Thesis. Khurum Nazir Junejo 2004-03-0018

Term Discrimination Based Robust Text Classification with Application to Email Spam Filtering. PhD Thesis. Khurum Nazir Junejo 2004-03-0018 Term Discrimination Based Robust Text Classification with Application to Email Spam Filtering PhD Thesis Khurum Nazir Junejo 2004-03-0018 Advisor: Dr. Asim Karim Department of Computer Science Syed Babar

More information

Natural Language to Relational Query by Using Parsing Compiler

Natural Language to Relational Query by Using Parsing Compiler Available Online at www.ijcsmc.com International Journal of Computer Science and Mobile Computing A Monthly Journal of Computer Science and Information Technology IJCSMC, Vol. 4, Issue. 3, March 2015,

More information

Crime Pattern Analysis

Crime Pattern Analysis Crime Pattern Analysis Megaputer Case Study in Text Mining Vijay Kollepara Sergei Ananyan www.megaputer.com Megaputer Intelligence 120 West Seventh Street, Suite 310 Bloomington, IN 47404 USA +1 812-330-01

More information

In this presentation, you will be introduced to data mining and the relationship with meaningful use.

In this presentation, you will be introduced to data mining and the relationship with meaningful use. In this presentation, you will be introduced to data mining and the relationship with meaningful use. Data mining refers to the art and science of intelligent data analysis. It is the application of machine

More information

An Evaluation of Unstructured Text Mining Software

An Evaluation of Unstructured Text Mining Software An Evaluation of Unstructured Text Mining Software Micah J. Crowsey, Amanda R. Ramstad, David H. Gutierrez, Gregory W. Paladino, and K. P. White, Jr., Member, IEEE Abstract Five text mining software tools

More information

Big Data Text Mining and Visualization. Anton Heijs

Big Data Text Mining and Visualization. Anton Heijs Copyright 2007 by Treparel Information Solutions BV. This report nor any part of it may be copied, circulated, quoted without prior written approval from Treparel7 Treparel Information Solutions BV Delftechpark

More information

Big Data for Investment Research Management

Big Data for Investment Research Management IDT Partners www.idtpartners.com Big Data for Investment Research Management Discover how IDT Partners helps Financial Services, Market Research, and Investment Management firms turn big data into actionable

More information

Practical Applications of DATA MINING. Sang C Suh Texas A&M University Commerce JONES & BARTLETT LEARNING

Practical Applications of DATA MINING. Sang C Suh Texas A&M University Commerce JONES & BARTLETT LEARNING Practical Applications of DATA MINING Sang C Suh Texas A&M University Commerce r 3 JONES & BARTLETT LEARNING Contents Preface xi Foreword by Murat M.Tanik xvii Foreword by John Kocur xix Chapter 1 Introduction

More information

Auto-Classification for Document Archiving and Records Declaration

Auto-Classification for Document Archiving and Records Declaration Auto-Classification for Document Archiving and Records Declaration Josemina Magdalen, Architect, IBM November 15, 2013 Agenda IBM / ECM/ Content Classification for Document Archiving and Records Management

More information

Customer Relationship Management (CRM)

Customer Relationship Management (CRM) Customer Relationship Management (CRM) Dr A. Albadvi Asst. Prof. Of IT Tarbiat Modarres University Information Technology Engineering Dept. Affiliate of Sharif University of Technology School of Management

More information

Visionet IT Modernization Empowering Change

Visionet IT Modernization Empowering Change Visionet IT Modernization A Visionet Systems White Paper September 2009 Visionet Systems Inc. 3 Cedar Brook Dr. Cranbury, NJ 08512 Tel: 609 360-0501 Table of Contents 1 Executive Summary... 4 2 Introduction...

More information

SPATIAL DATA CLASSIFICATION AND DATA MINING

SPATIAL DATA CLASSIFICATION AND DATA MINING , pp.-40-44. Available online at http://www. bioinfo. in/contents. php?id=42 SPATIAL DATA CLASSIFICATION AND DATA MINING RATHI J.B. * AND PATIL A.D. Department of Computer Science & Engineering, Jawaharlal

More information

DATA MINING TECHNIQUES AND APPLICATIONS

DATA MINING TECHNIQUES AND APPLICATIONS DATA MINING TECHNIQUES AND APPLICATIONS Mrs. Bharati M. Ramageri, Lecturer Modern Institute of Information Technology and Research, Department of Computer Application, Yamunanagar, Nigdi Pune, Maharashtra,

More information

2QWRORJ\LQWHJUDWLRQLQDPXOWLOLQJXDOHUHWDLOV\VWHP

2QWRORJ\LQWHJUDWLRQLQDPXOWLOLQJXDOHUHWDLOV\VWHP 2QWRORJ\LQWHJUDWLRQLQDPXOWLOLQJXDOHUHWDLOV\VWHP 0DULD7HUHVD3$=,(1=$L$UPDQGR67(//$72L0LFKHOH9,1',*1,L $OH[DQGURV9$/$5$.26LL9DQJHOLV.$5.$/(76,6LL (i) Department of Computer Science, Systems and Management,

More information

Big Data & Security. Aljosa Pasic 12/02/2015

Big Data & Security. Aljosa Pasic 12/02/2015 Big Data & Security Aljosa Pasic 12/02/2015 Welcome to Madrid!!! Big Data AND security: what is there on our minds? Big Data tools and technologies Big Data T&T chain and security/privacy concern mappings

More information

Predictive Analytics: Turn Information into Insights

Predictive Analytics: Turn Information into Insights Predictive Analytics: Turn Information into Insights Pallav Nuwal Business Manager; Predictive Analytics, India-South Asia pallav.nuwal@in.ibm.com +91.9820330224 Agenda IBM Predictive Analytics portfolio

More information

Data Mining Solutions for the Business Environment

Data Mining Solutions for the Business Environment Database Systems Journal vol. IV, no. 4/2013 21 Data Mining Solutions for the Business Environment Ruxandra PETRE University of Economic Studies, Bucharest, Romania ruxandra_stefania.petre@yahoo.com Over

More information

Overview. Background. Data Mining Analytics for Business Intelligence and Decision Support

Overview. Background. Data Mining Analytics for Business Intelligence and Decision Support Mining Analytics for Business Intelligence and Decision Support Chid Apte, PhD Manager, Abstraction Research Group IBM TJ Watson Research Center apte@us.ibm.com http://www.research.ibm.com/dar Overview

More information

Chapter 9 Knowledge Management

Chapter 9 Knowledge Management Turban, Aronson, and Liang Decision Support Systems and Intelligent Systems, Seventh Edition Chapter 9 Knowledge Management 9-1 Learning Objectives Define knowledge. Learn the characteristics of knowledge

More information

Introduction to Data Mining and Business Intelligence Lecture 1/DMBI/IKI83403T/MTI/UI

Introduction to Data Mining and Business Intelligence Lecture 1/DMBI/IKI83403T/MTI/UI Introduction to Data Mining and Business Intelligence Lecture 1/DMBI/IKI83403T/MTI/UI Yudho Giri Sucahyo, Ph.D, CISA (yudho@cs.ui.ac.id) Faculty of Computer Science, University of Indonesia Objectives

More information

Table of Contents. Chapter No. 1 Introduction 1. iii. xiv. xviii. xix. Page No.

Table of Contents. Chapter No. 1 Introduction 1. iii. xiv. xviii. xix. Page No. Table of Contents Title Declaration by the Candidate Certificate of Supervisor Acknowledgement Abstract List of Figures List of Tables List of Abbreviations Chapter Chapter No. 1 Introduction 1 ii iii

More information

The Text Analytics Market(s)

The Text Analytics Market(s) The Text Analytics Market(s) Competitive landscape and trends by Curt A. Monash, Ph.D. President, Monash Research Editor, Text Technologies contact@monash.com http://www.monash.com http://www.texttechnologies.com

More information

III JORNADAS DE DATA MINING

III JORNADAS DE DATA MINING III JORNADAS DE DATA MINING EN EL MARCO DE LA MAESTRÍA EN DATA MINING DE LA UNIVERSIDAD AUSTRAL PRESENTACIÓN TECNOLÓGICA IBM Alan Schcolnik, Cognos Technical Sales Team Leader, IBM Software Group. IAE

More information

Chapter 13: Knowledge Management In Nutshell. Information Technology For Management Turban, McLean, Wetherbe John Wiley & Sons, Inc.

Chapter 13: Knowledge Management In Nutshell. Information Technology For Management Turban, McLean, Wetherbe John Wiley & Sons, Inc. Chapter 13: Knowledge Management In Nutshell Information Technology For Management Turban, McLean, Wetherbe John Wiley & Sons, Inc. Objectives Define knowledge and describe the different types of knowledge.

More information

Course 803401 DSS. Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization

Course 803401 DSS. Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization Oman College of Management and Technology Course 803401 DSS Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization CS/MIS Department Information Sharing

More information

UTILIZING COMPOUND TERM PROCESSING TO ADDRESS RECORDS MANAGEMENT CHALLENGES

UTILIZING COMPOUND TERM PROCESSING TO ADDRESS RECORDS MANAGEMENT CHALLENGES UTILIZING COMPOUND TERM PROCESSING TO ADDRESS RECORDS MANAGEMENT CHALLENGES CONCEPT SEARCHING This document discusses some of the inherent challenges in implementing and maintaining a sound records management

More information

Voice. listen, understand and respond. enherent. wish, choice, or opinion. openly or formally expressed. May 2010. - Merriam Webster. www.enherent.

Voice. listen, understand and respond. enherent. wish, choice, or opinion. openly or formally expressed. May 2010. - Merriam Webster. www.enherent. Voice wish, choice, or opinion openly or formally expressed - Merriam Webster listen, understand and respond May 2010 2010 Corp. All rights reserved. www..com Overwhelming Dialog Consumers are leading

More information

The Big Data methodology in computer vision systems

The Big Data methodology in computer vision systems The Big Data methodology in computer vision systems Popov S.B. Samara State Aerospace University, Image Processing Systems Institute, Russian Academy of Sciences Abstract. I consider the advantages of

More information

Knowledgent White Paper Series. Developing an MDM Strategy WHITE PAPER. Key Components for Success

Knowledgent White Paper Series. Developing an MDM Strategy WHITE PAPER. Key Components for Success Developing an MDM Strategy Key Components for Success WHITE PAPER Table of Contents Introduction... 2 Process Considerations... 3 Architecture Considerations... 5 Conclusion... 9 About Knowledgent... 10

More information

Cleaned Data. Recommendations

Cleaned Data. Recommendations Call Center Data Analysis Megaputer Case Study in Text Mining Merete Hvalshagen www.megaputer.com Megaputer Intelligence, Inc. 120 West Seventh Street, Suite 10 Bloomington, IN 47404, USA +1 812-0-0110

More information