text data analytics insights unstructured predictive improve source Extracting Value from Unstructured Data use behavior characteristics customer

Transcription

1 models time techniques segmentation topics characteristics customer strong major processing value performance example intelligence changing better algorithms operational Data multiple Text unstructured text data extraction right use Big analytics addition Scorecard LDA NEE technology understand process structured words predictive improve source discover numerical predictors business range sales results extracted tagged change based document analysis behavior historical sources available meaningful new automated ability distributed interactions analyzing statistical decisions meaning predict entities need Semantic Extracting Value from Unstructured Data Text analytics discover new predictive customer in a major source of untapped data Number 7 October 23 Even the most customer-centric businesses today are making decisions about their customers based on only about 2% of available data. Much of the untapped 8% data that is unstructured could be yielding not only more but different and complementary. This new intelligence improves the ability to predict customer behavior, as well as understand and respond to the needs and motivations behind it. Text analytics helps to deliver this fresh intelligence by mining a major category of unstructured data one many organizations currently have on hand. Customer-related text may be abundant and available, but most organizations don t know how to efficiently extract predictive elements from it. They don t know how to reap the value of these by using them to boost the performance of predictive analytics and make better operational customer decisions. This white paper discusses how text analytics can help bring Big Data benefits to your organization s bottom line. It covers: Understanding what text analytics is and how it can help you. Done right, extracting valuable predictive from huge quantities of text takes just seconds. Boosting the performance of existing models with text-derived. Using text-derived to improve segmentation, decisions strategies and customer interactions. Exploring how text analytics reveal behavioral context, and can provide clues to what customers are thinking and feeling. Choosing the right text analytic techniques for your objectives. Make every decision count TM

2 Why Text Analytics Now As organizations strive to become more customer-centric and deliver ever-better service, they need additional sources of data-driven. Text is a major untapped potential source of these if structured in ways that make it usable by traditional analytic models. There are also opportunities to use from text independently and with from other types of unstructured data. The time to unlock this value is now. For many organizations, text currently accounts for the largest percentage of unstructured data that is on hand or easily accessible. New sources of text for instance, from blogs, Twitter feeds and other social media are proliferating. And text analytics, which provides the means to do the unlocking, has reached a level of developmental maturity suitable for widespread use. In its Hype Cycle for Big Data, 23 (July 23), Gartner positions text analytics as having high benefit and projects mainstream adoption within two to five years. What has brought text analytics to this point of readiness? Recent technological advances are solving many of the problems that posed obstacles in the past: Transforming for machine. Text in messages, call center logs, new business applications and collections agent notes is readily understood by us, but it s meaningless to traditional predictive models. A variety of machine learning methods, however, can determine what texts are about and classify them for further analysis. They can discover customer characteristics and transform them into structured numerical inputs that are comprehensible to predictive models and other types of traditional analytic algorithms. Handling complexity. Unstructured and semi-structured text (e.g., XML files, Excel spreadsheets, weblogs) is inherently complex since it may contain a wide range of content on a wide range of topics. Moreover, the potential value of text analysis is often increased by combining text of different types from multiple sources. Such a comprehensive approach may reveal complex, subtle customer behavior patterns not evident in smaller, more homogeneous document sets. But the task of collating, regularizing and organizing diverse data would be impossible without today s advanced technologies. We now have the analytic techniques and data infrastructures to essentially merge disparate varieties of text into one document for analysis. Facilitating engineering, deployment, management and regulatory compliance. While text and the process of analyzing it can be quite complex, the results need to be simple to understand and use. Today we can bring new from text analysis into predictive scorecards, for example, maintaining all the advantages they provide. The resulting scorecard has higher predictive power, but can still be engineered to meet specific business needs and regulatory requirements. It s easily deployed into rules-driven decision processes and operational workflows. It s easy to manage, including tracking performance, automating updates and measuring the impact of change. The contribution of text analysis is transparent, explainable to regulators, and documentable through automated audit trails and regular validation and compliance reporting. page 2

3 »» Scalable processing. While in many cases text analysis can be performed on a standalone PC, there are situations where the quantity of text and number of potential customer characteristics it contains are immense. Today s Big Data infrastructures can perform analysis at this scale with extreme efficiency and speed. Open source implementations of the MapReduce programming model and Hadoop distributed file system, for instance, process very large data sets in parallel across processor grids of enormous size. Storm, another open source distributed computation system, provides near-real-time distributed processing. All of these developments have come together to make now the time for getting started with text analytics. In addition, the full range of capabilities including advanced algorithms and Big Data infrastructures are becoming available through cloud-based services. Without deeppocket investment, organizations of all types and sizes can tap into these powerful technologies to better serve their customers or constituents. How to Use Text Analytics Text analytics is not a single technology. It s a convergence of many disciplines and technologies around the problem of extracting meaningful signals from text, as shown in Figure. Some of these contributors have been around for a long time, and some are newer and rapidly advancing. At the end of this paper, you ll find guidance for determining the right one for your application. Figure : Text analytics comprises many rapidly advancing technologies Inverted Index Document Matching Search Optimization Text Analytics Information Retrieval Web Mining Statistics Machine Learning Artificial Intelligence Management Science Computer Science Other Disciplines Web Analytics Web Content Mining Web Structure Analysis Co-Reference Relationship Extraction Entity Extraction Information Extraction Text Mining Classification Document Ranking Document Categorization Alert Detection Colocations Word Association Sentiment Analysis FIGURE HERE Source: Miner, Gary, et al. Practical Text Mining and Statistical Analysis for Non-structured Text Data Applications: 22. Print. Concept Extraction Tokenization Part-of-Speech Tagging Lemmatization Natural Language Processing Clustering Document Similarity Document Clustering page 3

4 Figure 2: Text-derived customer characteristics can be used in both supervised and unsupervised analytics The best place to start, however, is not with technology but with how you want to use text-extracted : Is your objective to improve an analytic model s ability to predict a particular customer behavior? Are you looking for additional into the likelihood that a customer will, say, purchase a certain product, become 6-days delinquent or attrite? Is your objective to more generally increase analytic for better segmentation, decision strategies and interactions? Are you trying, for example, to better understand why you re losing some of your best customers to attrition or how to improve your collections results with certain types of customers? As depicted in Figure 2, text-derived customer characteristics can be used for both types of objectives. On the left branch, the graphic shows that once transformed into structured numerical inputs, text-derived characteristics can be used with traditional supervised analytic techniques. These use historical data tagged with observed outcomes (e.g., fraud/no fraud, purchase within six weeks, delinquent account paid/rolled over) to train models to predict a targeted customer behavior. A text-derived characteristic could potentially be used in multiple models, where its information value depends on the degree to which it correlates with the targeted customer behavior. Improving performance of a predictive model Supervised analytics e.g., scorecards, attrition models, response models Text Mining Objective Use text-derived customer characteristics for: Increasing for segmentation & decisions Unsupervised analytics e.g., classification, segmentation, self-calibrating outlier models, Latent Dirichlet Allocation The right branch incorporates text-derived characteristics into unsupervised analytic techniques that do not require tagged historical data. These include analytic methods for improving segmentation, deepening understanding of individual customers, adapting decision strategies to customer needs and behavior context, and even triggering timely actions in response to changing customer behavior and attitudes. Let s look in more detail at how text-derived can improve results in both supervised and unsupervised analytics.»» Boost the Performance of Existing Models Using text-derived characteristics in supervised analytic techniques Organizations that already have predictive models built with structured data can boost the performance of these models by analyzing text they already have or routinely generate. Hidden in this text may be numerous customer characteristics that could have predictive value when used with characteristics from structured data. page 4

5 Target Target Extracting Value from Unstructured Data Figure 3: Transforming text into numerical representations that can be analyzed for behavior predictors Two ways of mining this potential predictive power from raw text are feature extraction and entity extraction. FICO s Semantic Scorecard technology makes use of feature extraction. As depicted in Figure 3, there are four major steps in this analytic technique: Variables Used in Traditional Scorecards Target Freeform Text Term Frequency Matrix Statistical Analysis 2 Transform text into a numerical representation in the form of a term frequency matrix. The algorithm breaks text strings into tokens meaningful pieces for analysis which in this case are terms made up of words or combinations of words. It then determines the frequency of occurrence for each term; it takes into account the descendants of word roots, synonyms, etc., and filters out frequently occurring terms that convey little meaning, such as conjunctions and relative pronouns. Filters are configurable so users can make adjustments and rapidly see how they change results. The result is a numerical representation of the text in a form that is comprehensible to algorithms and can be used as input into a variety of text analytics techniques. Enhanced Data Set Traditional Model Training & Scoring Apply statistical techniques to discover predictors with high information value. Frequently occurring terms are candidates for selection as predictive variables. Statistical analytic algorithms based on information theory identify those with high information value variables that correlate strongly with the target outcome variable. This process discovers often subtle and complex predictors hidden in the text, invisible to the human eye and mind. And it s extremely fast. Tens of thousands of records made up of hundreds of words of freeform text can be analyzed in seconds. Figure 4: Analyzing hidden signals in text lifts predictive performances PERCENTAGE BADS % 8% 6% 4% 2% Semantic Scorecard Traditional Scorecard % % 2% 4% 6% 8% % PERCENTAGE GOODS The chart shows a Semantic Scorecard outperforming a bank s traditional risk scorecard. At any percentage of goods (current and fully paid accounts), the Semantic Scorecard predicts a higher percentage of bads (charged-off and defaulted accounts). 3 4 Create an enhanced data set containing predictive characteristics extracted from both the text and the structured data. Strong predictors extracted from the text and strong predictors extracted from structured data are combined into a single tagged enhanced data set. Perform traditional model training and scoring. The enhanced data set can be analyzed by the existing structured data scorecard technology. Figure 4 shows the improved accuracy seen when the Semantic Scorecard technology was applied to a bank s existing risk scorecard for new account decisions. In other FICO research, a Semantic Scorecard incorporating predictors from notes about sales inquiries did a better job of identifying leads that result in sales than a scorecard using structured data alone. In the top 5% of scored leads, it found 3% more leads resulting in sales. Increased accuracy can lead to better prioritization and sales outcomes. page 5

6 For the same project, we also employed named entity extraction (NEE) as a complementary technique. NEE is based on natural language processing, which draws on the disciplines of computer science, artificial intelligence and linguistics. By analyzing the structure of the text, NEE determines which parts of it are likely to represent entities such as people, locations, organizations, job titles, money, percentages, dates and times. One reason NEE is compatible with scorecards is that both techniques enable strategy engineering. For every entity identified, the NEE algorithm generates a score indicating the probability that the identification is correct. Engineers can set thresholds accepting only those entities with a score above 8%, for example. In this project, we used extracted entities with a similarity-based matching algorithm to join records from different kinds of files having no direct links (e.g., a structured file containing customer information and unstructured texts about interactions with the customer). In addition, by combining extracted entities, we were able to impute an individual s authority to make a purchase decision a strong predictor for improving intelligent automated lead generation that is not directly available in the dataset. This customer characteristic and others that prove to have high information value (i.e., to correlate strongly with the targeted customer behavior) can be incorporated into the Semantic Scorecard or another type of supervised model. They can also be used in unsupervised analytics, which we ll look at next.»»understanding Customers and Their Changing Behavior Using text-derived characteristics in unsupervised analytic techniques Text-derived characteristics provide for improving population segmentation, as well as individualized customer decisions and interactions. They can also be used for models that do not require training with historical data. Consider the named entity extraction we described in the previous section. Such entities could be used to refine peer group definitions for self-calibrating outlier models that detect changes in customer behavior. These models can incorporate new customer characteristics without training on tagged historical data. In production, they determine on the fly, from ongoing customer activity, what the current normal ranges of values are for these characteristics. Out-of-range values can trigger alerts. This patented FICO technology is critical to streaming analytic problems where the algorithms must continuously update, in real time, estimates of feature distributions so that detection of outliers is always based on the current distributions. Another analytic technique effective for segmentation, as well as for detecting changing customer behavior, is Latent Dirichlet Allocation (LDA) and related methods for finding similarities in data that enable classification and grouping. LDA is an unsupervised statistical method of extracting topics, concepts and other types of meaning from unstructured data. It doesn t understand syntax or any other aspect of human language. It s looking for patterns, and it does that equally well no matter what language text is in, or even if it consists of just symbols rather than characters. LDA could be used to examine a blog with a, posts, for example, to determine what the blog is about overall. The algorithm could identify four or five predominant topics or archetypes of content. It could also be applied to a heterogeneous mix of text. In fact, any type of unstructured, semi-structured and structured data from any number of sources can be combined into a single document for LDA to find patterns across them. page 6

7 This very flexible technique is commonly used in marketing to generate archetypes for customers with similar behaviors. FICO is also using this approach to analyze a bank s call center data. The goal is to identify meaningful reasons why customers are calling, and use these to better understand and predict attrition risk. Among the useful discoveries so far: Customers mapping strongly to a frequent traveler archetype are among the least likely to attrite. Moreover, if these customers interact with a call center representative regarding a payment missed due to traveling resulting in waiver of the late fee attrition becomes even less likely. This insight could help focus the budget for late fee waivers where it will have a strong impact on improving attrition rates. In another area of application, fraud detection, FICO has developed Collaborative Profiles, a patentpending streaming version of LDA. As shown in Figure 6, by analyzing card transactions, Collaborative Profiles represent a real customer as a distribution across multiple archetypes and updates the distribution continuously, with each transaction. Figure 5: Using LDA to generate archetypes from text Call center text Bag of words collection of words from call transcripts (size indicates frequency of occurrence) Generation of call center content archetypes determining customer topics notice. I missed only one payment e I was notice. out I of missed town only and now one payment I ve h hit e I by was notice. a late out I fee of missed town that only I and don t one now feel payment I ve I d have h hit e I to by was notice. pay. a late out I I want fee of missed town that make only I and don t one now feel payment I ve my I d account have h hit e I by to was is pay. a not late out reported I want fee of town that to make I and don t now feel I ve to bureaus my I d account have h hit as delinquint. by to is pay. a not late I reported want fee I ve that to been make to I don t feel former to bureaus my more I d account have than as to delinquint. is five pay. not years I reported want and I ve to been make to ally former pay to bureaus my more bills account than on as time. delinquint. is five not Why years reported cand I ve been to bank ally know former pay to my bureaus me more bills well than on enough as time. delinquent. five Why to years cand I ve been understand bank ally know former pay that my me more I bills have well than on enough to time. travel five Why years to cand equently understand bank ally for know my pay that work my me I bills have well therefore on enough to time. travel Why to ca always equently understand meet bank for their know my that imposed work me I have well therefore dead enough to travelto lines. always If equently you understand meet want for their to my keep that imposed work my I have therefore busi dead to travel ness lines. you always If equently need you meet to want show for their to my keep me imposed work more my therefore busi dead ness lines. you always If need you meet want to show their to keep me imposed more my busi dead ness lines. you If need you want to show to keep me more my busi ness you need to show me more Karen s calls 2.%.3% 22% 38.5% 8.% Mapping of customer interactions to archetypes by % of match Figure 6: Using Collaborative Profiles to update customer archetype distributions in real time Actual customer Casey Real-time updates from transaction streams Dynamic mapping of Casey s behavior to multiple archetypes by % of match 35.2% 2.4% 5.5%.%.8% 32.8% 2.6% 49.3%.5% 4.8% 39.2%.3% 52.%.3% 7.% page 7

8 An advantage of this streaming technique is that, based on the delta between the current distribution and the updated one, Collaborative Profiles can trigger an alert that a significant change in customer behavior is underway. For example, by analyzing collectors notes over the course of several interactions regarding a delinquent bill, Collaborative Profiles could detect that the customer is becoming frustrated, angry or losing confidence in his eventual ability to repay the debt. Perhaps a new factor has entered the equation, such as a family member falling ill, which may require an adjustment in collections strategy. This type of analysis could even signal a change in intent identifying the moment when a customer who originally intended to pay consciously or unconsciously gives himself permission not to. How Deep Can We Go? Digging deeper for into not only how customers are likely to behave, but also what they re thinking and feeling is an area of text analytics generally referred to as sentiment analysis. The analytic techniques used are often based on natural language processing (NLP), but they may also be statistical or a hybrid of these. To learn more about text analytics and FICO s Semantic Scorecard technology, read the FICO ebook (registration required). Traditionally, supervised methods have been used to analyze the polarity of a document or phrase: Is it positive, negative or neutral? At the most basic level, this classification is done by simple statistical methods like key word indexing. More sophisticated NLP approaches try to understand the semantic context in which key words appear to determine degree of positive or negative sentiment. Some are able to track and compare changes in sentiment over time. Unsupervised techniques that, like LDA, can extract topics from text are also being used to identify a wider range of customer issues and attitudes. Among them are latent semantic analysis (LSA) and latent semantic indexing (LSI). These are attractive not only because they can be applied to unclassified text, but also because they can discover unknown and emerging customer sentiments. They can also be used at a macro level to identify attitudinal shifts and developing trends within a market. Some of the most promising and challenging areas of development seek to use natural language processing to understand what customers really mean when they use a set of words. For example, is the phrase That s great always positive? What does it mean when the client says Yeah, sure, sure? Often we humans express ourselves in ambiguous ways or employ sarcasm. As more and more customer interactions take place through , chat and text messaging rather than phone calls, we lose the crucial clues to meaning that come from voice tonality and emphasis. The cutting edge of sentiment analysis seeks to pick up these subtleties through other automated means. page 8

9 Choosing the Right Text Analytic Technique This overview of several text analytics applications demonstrates the wide range of methods that can be employed to extract value. Once you have the answer to the fundamental question of how you want to use text-extracted, there are other questions that can help you select the right techniques for your purpose. The simplified decision tree in Figure 7 is provided as one possible example of how to think through some of the possibilities. Figure 7: Example of sorting through text analytics possibilities to select the right approach for your purpose Text Mining Application Sentences, paragraphs, document analysis? Documents OR Individual words and phrases? Words Do you want to sort documents into groups or search for specific documents? Do you want to automatically identify specific facts or gain overall understanding? Search Sort Specific facts Understanding Information Retrieval Do you have categories already? Information Extraction No categories Have categories Is your focus on the meaning of the text or on the structure? Structure Meaning Clustering Are your documents independent or connected via hyperlinks? Independent Connected Natural Language Processing Concept Extraction Document Classification Web Mining FICO Text Analytics Solutions The latest release of FICO Model Builder, our complete analytic development environment, includes extensive capabilities for Big Data text analytics. These include FICO s proprietary Semantic Scorecard technology for combining predictors discovered in both text and structured data into a single, easy to deploy and manage scorecard. Model Builder also provides access to scalable data processing via Hadoop and to the huge library of algorithms (including for entity extraction, topic modeling and sentiment analysis) from R, the open source statistical programming language. In addition, Model Builder ships with the Tika and Lucene libraries for text extraction and indexing. FICO Decision Management Platform, accessible soon via cloud-based services, provides everything you need to quickly and efficiently bring text-extracted to bear on operational decisions. It includes Big Data infrastructure, rapid application development (RAD) tools and decision applications. FICO Model Central Solution provides the means to deploy, track, validate, monitor and manage a wide range of analytics to improve bottom-line impact and regulatory compliance. This kind of centralized control is increasingly crucial today as analytics become part of almost every customer operational decision, and the number and variety of models in use expands dramatically. Among the specific benefits for text analytics is the ability to assess change by tracking volatility in the prevalence and importance of topics, entities and other text-extracted. page 9

10 »Conclusion Make» Decisions Based on All the Data The obstacles that once forced organizations to make operational decisions based on a partial representation of customer data are now falling away. Leaders in all industries and sectors are beginning to make decisions on an increasingly wide range of Big Data including vast stores of text they already have. Text analytics are delivering abundant, fresh customer to fuel higher levels of customer service and business performance. To learn more about advances in Big Data analytics and related technologies, visit the FICO Labs Blog or read these Insights papers: When Is Big Data the Way to Customer Centricity? Is It Fraud? Or New Behavior? From Big Data to Big Marketing: Seven Essentials The Insights white paper series provides briefings on research findings and product development directions from FICO. To subscribe, go to For more information North America toll-free International web () info@fico.com FICO, Model Central and Make every decision count are trademarks or registered trademarks of Fair Isaac Corporation in the United States and in other countries. Other product and company names herein may be trademarks of their respective owners. 23 Fair Isaac Corporation. All rights reserved. 37WP /3 PDF