Using Rule-Based Natural Language Processing in Customer Experience Development

Transcription

1 Using Rule-Based Natural Language Processing in Customer Experience Development Pasi P. Tuominen, Haaga-Helia University of Applied Sciences Abstract The automated discovery of consumer opinions from different sources is of great importance for operational evaluation, marketing intelligence, service development and customer experience management. Techniques are now being established to effectively and easily mine the consumer opinions from the Web and to timely deliver them to companies. The catalyst for these changes in management ambitions is partly due to a move away from the traditional, linear marketing process to a digital model that is driven by consumer engagement and interaction (Kaplan & Haenlein, 2010). Displaying evidence from a case study, this study addresses the distinct need to monitor, assess and analyse consumer-generated content. Moreover, this research suggests that with analysis of stakeholder voices, the company is able to capture detailed and unaided requirements from stakeholders for product design and execution and to enhance management of corporate reputation in the marketplace. Keywords: Customer Experience Management, Natural Language Processing, Voice of Customer Introduction With the growing availability and popularity of opinion-rich resources such as online review sites and personal blogs, new opportunities and challenges arise as people now can, and do, actively use information technologies to seek out and understand the opinions of others. The sudden eruption of activity in the area of opinion mining and sentiment analysis, which deals with the computational treatment of opinion, sentiment, and subjectivity in text, has thus occurred at least in part as a direct response to the surge of interest in new systems that deal directly with opinions as a first-class object. Today s peer communication represents unforeseen and measurable sources of information and the automated discovery of stakeholder opinions is of great importance for evaluation of service operations and for marketing intelligence (Tsiotsou & Goldsmith, 2012). To discover threats in time, to ensure better service and to gain competitive advantage companies have to alert to even the weakest signals in their periphery. Already, companies have access to a wealth of useful information in the form of text-based feedback and discussions. However, this information comes from multiple channels and formats, which makes it hard to gain a comprehensive view on the multitude of signals. According to hospitality managers (Tuominen, 2012a, p.267), the most desired solution for capturing stakeholders voice would provide custom reports for management, construct a holistic portray of the sentiment and relevance of the discussion, follow change and trends and identify emerging topics for strategic analyses. The catalyst for these ambitions is partly due to a move away from the linear strategic process to a digital model that is driven by consumer engagement (Kaplan & Haenlein, 2010). Incorporating online discussion and customer feedback of nearly 30 restaurants and exploiting on practitioners experience, this research aims to (1) analyse the applicability of the computer aided text analysis (CATA) software to a global service provider and (2) evaluate business process implications of using the textual analysis in understanding the value of open text feedback in different levels of the organization. Literature As new technologies, new services and new customer trends emerged, companies in many industries had to watch their profits eroding and their entire business models being threatened. Day and Shoemaker (2005) use the example of two breweries to illustrate the importance of weak signals. The first company Anheuser-Busch identified the healthyliving trend early on and successfully introduced a lowcalorie beer, thus capturing 5-7% of the light beer market in U.S. Its main competitor, Coors, not having understood the potential, was slow to react to the market shift and did not follow suit until two years later. Media scholars such as Curran (2002), and Boyd- Barrett, Newbold and Van Den Bulck (2002), among others, support analysing latent as well as manifest content of texts and tend to view qualitative and quantitative content analysis as complementary parts of a continuum of methods that can be applied to capture the meaning and impact of texts. There are two main traditions in the quantitative content analysis research delineated by Weber (1983): substitution model and correlational model. In the substitution tradition, text is analysed with a priori established categories that are understood as a group of words with similar meaning

2 and/or connotations (Weber 1983, p. 140). For example, the words ice, snow, and igloo all represent the same idea of cold and, thus, can be united under one category of cold (Hogenraad, McKenzie & Peladeau, 2003). Various categories are organized into dictionaries, which are used for making necessary substitutions in the text and for obtaining category frequency counts. Over the history of content analysis, several well-known dictionaries were developed for example, the Harvard IV Psychological Dictionaries (Kelly & Stone, 1975) and the Lasswell Value Dictionary (Lasswell & Namenwirth, 1968) and lately (e.g. industry) specified ontologies (in: Chiarcos, 2012). Over the past two decades, content-analysis research has remarkably benefited from the exponentially increasing volume of electronic data, including articles in general media databases, communications in virtual communities, and textual and pictorial materials from Web sites (Wickham & Woods, 2005). Immense volumes of easily accessible textual material, speed and simplicity of the datacollection process, lack of complications associated with human subjects, and advances in development of various computer programs to support textual data analysis are factors that stimulate the use of contentanalysis research in social sciences (Macnamara, 2003; Romano et al., 2003). The large volumes of digital textual data available and the repetitiveness of the task make the computer a natural and powerful choice for content analysis. CATA software is routinely used for storage, search, and retrieval of textual data. It can also assist in theme identification and coding, a time-consuming, prone-toerror bottleneck of the content-analysis process (Romano et al., 2003) that is amplified when a large number of cases have to be processed (Macnamara, 2003; Wickham & Woods, 2005). However, the question of whether automated content analysis is too simplistic and unreliable for sophisticated interpretation of texts is still under discussion and is inherently connected to the preference for quantitative or qualitative epistemologies for the content-analysis project at hand (Newbold, Boyd-Barrett, & Van Den Bulck, 2002). The debate primarily centres on the issue of manifest versus latent content, with the concern that computerized measurement of content invariably misses such latent aspects of the text as figures of speech, irony, tone, colloquialisms, or metaphors, homonyms, and other aspects of language (Carley, 1997). Yet later research has noted that human coders also exhibit low reliability for latent content, and besides, the significance of the latent content might be overestimated in certain areas. Recently, however, the research on the consumer latent need has been a popular topic (e.g. Donnely et al., 2012). A rule-based (or ontology enhanced) semantic analysis system, on the other hand, understands language structures and the relationships between words. It has a massive set of pre-configured rules that enable automatic and accurate real-time analysis of textual data (Bolasco & Pavone, 2010). Typical rule-based software can also handle idiosyncrasies like bad grammar, spelling, and homonyms (Bolasco & Pavone, 2010). While there have been some notable examples of using CATA in hospitality and tourism-related studies and destination-image research, in particular, Mehmetoglu and Dann (2003) noted that, with few exceptions, hospitality and tourism researchers have been reluctant to rely on CATA for content analysis. In the meta-analysis of 154 studies on destination-image research published from 2000 to 2007, 53 articles reported use of qualitative approaches at some stage of research: focus groups for questionnaire development, open-ended questions in surveys, collection of textual and pictorial materials from media sources and the Internet, and so on. However, only 6 studies reported use of CATA programs for data analysis (Stepchenkova & Mills, 2010). Methodology For this study, Connexor Machinese was applied, with etuma360 interface. Machinese presents a suite of software components and solutions to normalize, categorize and extract concepts, names, facts and other text analytics from documents written for diverse audiences in many languages. To demonstrate the underlying linguistic analysis, an example feedback sentence is presented in a form of syntax tree (figure 1) and phrase tagger report (table 1). As I approached the unit I could see there was a good display of cakes and pastries all clearly labelled, the cold drinks display was also well stocked. Machinese Syntax produces functional dependencies representing relational information in sentences discovering the functional relations between words in sentences. Figure 1 Machinese syntax tree of an example customer feedback signal (source:connexor)

3 Machinese Phrase Tagger splits raw text into understandable word units and provides the possible base forms and classes for words. Machinese Phrase Tagger contains a custom lexicon mechanism, which enables researchers to add their own ontologies to the parser. These words can be, for example, domainspecific vocabularies, multi-word terms, names and places etc. Table 1 Machinese phrase tagger report of an example feedback signal. (source: Connexor) Findings In order to test the software combination, the vision and the service promise of a global hospitality operator was being analysed. The studied Coffee House chain R (further: CHCR) claims to be the provider of great coffee to busy travellers, and their desire is to be purveyors of the dedication, talent and skill of their baristas. These service promises were analysed as well as the perceptions towards the delivery of uncompromising taste. The tested hypothesis was, does devoting time and energy to providing rendezvous for people happy to escape from the hustle and bustle of life take away focus and resources that could instead be spent on providing the uncompromising taste? Furthermore, by leveraging the interactive research tool of etuma360interface (see example: fig. 3) the researcher was able to locate other signals highlighting the applicability of the software combination for customer experience management and development. Figure 3 Example of etuma360 topic / theme based interactive research tool in tag cloud mode (source:etuma360) For the analysis, and to identify relevant textual data all mystery shopper reports and twitter messages (containing #thenalaysedbrand) were collected (Figure 2). The data was collected between June 2012 and January 2013, resulting 625 comments in total. Service and smile were the biggest positive themes for CHCR, which covers e.g. staff, behavior, courtesy, eye contact etc. The personal service at CHCR is often mentioned as a plus. In general, consumers find CHCR coffees to be of great tasting. Consumers like the coffee variants at CHCR, meaning the choices among black coffee, flat white, macchiato, affocato and many others. Coffee drinkers like the yummy taste of CHCR, which is a synthesis for specific comments like freshly brewed taste, delicious, strong flavor and others. The stimulant effects of coffee are a positive for many consumers, which they express as keeps one up. For some, CHCR is a comfort drink. Figure 2 Retrieved weekly feedback signals with sentiment distribution (source:etuma360)

4 Table 2 Hot topics/themes with sentiment distribution of CHCR 06/ /2013 (negative, neutral, positive) (source:etuma360) People think CHCR coffee is costly, meaning overpriced or expensive or both. Others find it does not taste good. I dont buy Coffee House chain R coffee for the taste. I think Costa is better. I buy it so everybody knows im a hipster and i have money. Poor quality is a common negative theme, with people criticizing, among other things, the beans and the inconsistent taste. Some consumers complain about side effects, one of which is cant sleep, which was considered a positive attribute by others. Some consumers find a downside to the coffeehouse experience that CHCR likes to promote. Some consumers link the negatives of high price, poor quality and elitism in their negative comments. Not preferred taste was the most common negative comment. Some find it expensive and are overwhelmed by all the options. Emerging trends and topics When consumers name other gourmet coffee brands and CHCR in the same post, its most often to say that the other brands don t measure up to CHCR. The company says it believes in providing a rendezvous for people and consumers find that they re true to their word. So their positioning at the high end of the gourmet coffee market is right where customers perceive them. One dilemma for the company is that people wish CHCR had more coffeehouses. The locations are popular and often too crowded. Coffee House chain R wants to be perceived as a purveyor of the uncompromising taste and a segment of consumers agree. The company touts the variety of coffees they offer, and that s also a success with consumers. In fact, the findings reveal that those are the top two positive themes. Nonetheless, not good tasting is the second-biggest negative theme, which means a segment of consumers feel there s a disconnection between positioning and perception regarding the taste of CHCR coffee. Furthermore, when company sells relatively expensive, specialty, gourmet coffee, they re bound to bring out the antisnob sentiments in some consumers. CHCR has its fair share of detractors who think a CHCR coffee has become an overpriced status symbol. From a marketing point of view, the CHCR experience is a differentiator that helps attract and retain specialty coffee customers. CHCR knows this and refers to its locations as a rendezvous, that is, somewhere cosy while travelling. Apparently the flow of operations within a store is found inconvenient by some customers not enough so that they aren t coming to CHCR, but enough so that they re speaking up about it. This sound like an issue CHCR can fix by improving the layout of its stores and the way customers move through the steps of a transaction. Conclusions Content analysis is widely used in many disciplines to analyse various forms of communications, above all, those that utilize textual data. A growing number of hospitality and tourism studies employ qualitative data (interviews, open-ended questions, promotional brochures, Web based content, etc.) and, subsequently, content-analysis techniques to discern meaning from this wealth of textual material. In the past decades, many software packages, both quantitatively and qualitatively oriented, have been developed to assist researchers in content-analysis studies; nevertheless, CATA software has not been often used in hospitality and tourism research despite its obvious benefits. In the authors view, there are two main reasons for this lack of enthusiasm: first, a quantitative paradigm of content analysis is still sometimes viewed as simple word counting, which is not conducive to uncovering latent themes and issues of texts; and second, no single software package has been able to provide the full spectrum of functions that might be needed in various content-analysis projects. Thus, this article offers a transparent and effective example to facilitate content analysis of textual data typical in hospitality and tourism-related studies; it also demonstrates that the proposed approach is firmly grounded in the theory and practices of contentanalysis and the possibilities of netnographic methodology. Furthermore, this research demonstrates the possibilities how insights could be used to inform development and marketing activities including brand perceptions, brand positioning, segmentation studies, new product development and innovation, trend identification and opportunities for marketing campaign activation. The ideal strategy for companies would be to combine content-rich social media data with the customer data available in enterprise feedback channels. And, in fact, as this study demonstrates, netnographic methodology and content analysis offers transformative new opportunities to today s market researchers and marketers. However, change is not

5 easy. Many companies have their marketing and market research systems oriented around the input from regular focus groups and surveys. Finally, supporting the applicability of the process in hospitality and tourism research, the findings of this study suggest the possibility of gaining real-time actionable information for the management. The developed insight and the real-time interface was found to be able to contribute significantly to effective problem resolution, quality improvement and customer experience development. Future research As with any methodology, the appropriateness of the approach for a particular research question has to be assessed prior to using it, in order to avoid a mechanistic counting of words poorly related to a research topic. To the author s knowledge, the netnographic methodology and (hospitality) ontology enhanced natural language process has not been tested beyond the research sited in this article. While several tests were successfully conducted by the authors to verify the functionality and accuracy of Connexor Machinese and the Etuma360 interface, more testing is desirable. References Alexa, M. &. (2000). Text Analysis Software: Commonalities, Differences and Limitations: The Results of a Review. Quality & Quantity, 34, Bolasco, S. &. ((2010). ). Automatic Dictionary- and Rule-Based Systems for Extracting Information from Text. Data Analysis and Classification Studies in Classification, Data Analysis, and Knowledge Organization, Boyd-Barrett, O. v. (2002). The media book. Arnold. Carley, K. M. (1997). Extracting team mental models through textual analysis. Journal of Organizational Behavior, 18(s1), Chen, Y. &. (2007). Brand Personality of Spa Destinations (Resorts) on the Web. Proceedings of the 12th Annual Graduate Education and Graduate Students Research Conference on Hospitality and Tourism, Houston, Texas, pp Chiarcos, C. (2012). Ontologies of linguistic annotation: Survey and perspectives.. Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC). Curran, J. (2002). Media and power. Routledge. Day, G. S. (2005). Scanning the periphery. harvard business review, 83(11), 135. Donnelly, C. S. (2012). Marketing planning and digital customer loyalty data in small business. Marketing Intelligence & Planning, 30(5), Gray, J. H. (2005). Towards an integrative model of organizational culture and knowledge management. International Journal of Organisational Behaviour, 9(2), Kaplan, A. M.;& Haenlein, M. (2010). Users of the world, unite! The challenges and opportunities of Social Media. Business Horizons, 53(1), Kozinets, R. V. (2002). The field behind the screen: Using netnography for marketing research in online communities. Journal of Marketing Research, 39(February), Kozinets, R. V. (2010). Networked narratives: Understanding word-of-mouth marketing in online communities. Journal of marketing, 74(2), Macnamara, J. R. (2003). Media content analysis: Its uses, benefits and best practice methodology. Stepchenkova, S. &. (2010).. Destination image: A meta-analysis of research. Journal of Hospitality Marketing & Management, 19(6), Tsiotsou, R. H.;& Goldsmith, R. E. (2012). Strategic Marketing in Tourism Services. Bingley:GB: Emerald. Tuominen, P. (2012a). Integrated Marketing Communications and Brand Tribalism in a Postmodern Hospitaliy Reputation Management Process. Doctoral Dissertation. Hatfield: University of Hertforfdshire Press. Weber, S. (1983). A general concept of fuzzy connectives, negations and implications based on< i> t</i>-norms and< i> t</i>-conorms. Fuzzy sets and systems, 11(1), Wickham, M. &. (2005). Reflecting on the strategic use of CAQDAS to manage and report on the qualitative research process. Doctoral dissertation.