EMBEDDED SUPPORT SERVICES FOR WEB APPLICATIONS DELIVERED THROUGH THE CLOUD

Transcription

1 Dep. of Computer Science Institute for System Architecture, Computer Networks Group Master Thesis EMBEDDED SUPPORT SERVICES FOR WEB APPLICATIONS DELIVERED THROUGH THE CLOUD Xiaolong Gou Mat.-Nr.: Supervised by: Prof. Dr. rer. nat. habil. Dr. h. c. Alexander Schill Advised by: Dr.-Ing. Josef Spillner Dipl.-Wirt.-Inf. Tobias Julius Neubert Submitted on 23. September 2013

2 ii

3 TECH N ISCH.E UNIVERSITAT DRESDEN fatrlttit tnformatik, lnstitut fur Systemarchitektur, Lehrstuhl Rechlelnglze AUFGABENSTELLUNG FUR DI E MASTERARBEIT Name, Gou; Xiaol FoischunEsgebiet: Service and Cloud Forschungspt"ojekt: FlexCloud Computing as il\- Vera ntwortl i cher Fl'ochsc hu I le hrer: Thema: Prof. Dr. rer. nat. habil. Dr. h. c. Alexander Sch,ill Embedded Support Services through the Cloud Appllcatlons tor Web Applications for oellvereo delivered ZIELSTELLUNG Software delivered on demand as a service from the Cloud provides benefits to users who only need lnternet access and a service frontend such as a web browser. products of this type like SAP Cloud solutions are advantageous for customers, especially smaller enterprises, to reduce costs and implementation time and effort. Customers only pay forthe usage of the software depending on the demand. lntegrated support functionalities play an essential role for Cloud products, especially for the provisioning of interactive applications. lh"y are. essential because this scheme of hosting software transfers the cost of software maintenance and management from the service consumers to the providers. As a result' efficienly providing the service in this mass market is vital for vendor success' Embedded support in software life cycle management focuses on the above mentioned business goals. lt provides tools for users to report software problems at any point in usage time directly from the software application's Ul. lt also provides tools for tracking and resolving software issues. From a service science perspective, support services are bundled along with the domain-specific application services' SAp Business ByDesign is an on-demand enterprise software product with welldeveloped ämbedded support. The graphical user interface is currently implemented with a proprietary web browser plug-in which does not guarantee good coverage on mobile devices. To reach as many users as possible, on-deman-diervice vendors have to leverage the advantages of HTMLS, which is broadly available both on mobile devices as well as on traditional front ends. This master thesis will investigate techn.ical solutions to meet requirements of embedded support especially in web based applicationt *itf' a usei interface based on JavaScript. Reference to the existing functions in ByDesign will be taken' lt will also describe and compare implementations of functionalities based on the results of the investigation. Key aspects will be efficient and.user-friendly means for context data (like user OS, browser type, etc.) collection, the user-supp:'t interaction, as well as söftware problem solution search. On top of this practical work, the thesis will also suggest methods to measure and analyse the contribution of the implementations to an effective and efficient support process.

4 SCHWERPUN KTE I Analysis and comparison of, efibedded suppor:t services in cornmercial'saas, resu lting i n a reusa ble Su pport'a,5;a,service, defi niti on o Concept for embedded support services with dynamic web interfaöes, consistent Ul design with existing approaches, and multi-application 5urpport,based on ' loose:coupling and tight integr:ation. lmplementation as a web application conforming to HTML5 and JavaScript to ensure compatibility with web browsers on mobile platforms. Scherne for systematic measur:bment of sup$ort,effectivity'and efficiency u n te rs ch ri tt a es v eira ntw o rtl i ch e n H o c h sc h u I I e h re rs Prof. Dr. rer. nat. habil. Dr. h, c. Alexander Schill

5 CONFIRMATION I confirm that I independently prepared the thesis and that I used only the references and auxiliary means indicated in the thesis. Dresden, 23. September 2013 v

6 vi

7 ACKNOWLEDGEMENTS I would like to first thank Prof. Dr. rer. nat. habil. Dr. h. c. Alexander Schill for giving me the opportunity to write this thesis in his research group. I would like to thank a lot my advisor Dr.-Ing. Josef Spillner for his continuous support and steering my work during the past months. His openness, patience, commitment and efforts are crucial for me getting to this point. Many Thanks to Dr. Tilmann Haeberle from SAP AG who accepted me again as a student employee and provided me resources and channels to conduct my research and implement my prototype in the industry. I would like to thank my internship mentor and thesis advisor Mr. Tobias Julius Neubert from SAP AG who showed me the way to reach my thesis targets with precise planning, strict execution, agile adjustments and retrospective inspections. His help and friendliness made the past months real pleasant memory. I would like also to thank Mrs. Martina Keller, Dr. Bettina Gaa-Frost and Mrs. Alda Dollani from the cloud supportability team, Mr. Uwe Schlenker, Dr. Ralph Radermacher, Mrs. Beatrice Pasch, Mr. Juergen Heiss, Mrs. Rosamund Armstrong, Mr. Hans Gradl, Mr. Uwe Wieditz, Mrs. Nidhi Aggarwal, Mr. Lars Riecke, Mr. Robert Gaida, Mr. Markus Melchinger, Mr. Forrest Zhang, Mr. Steen Tirkov, Mr. Sascha Zankel, and Mr. Prasanth Rajan from various SAP organizations globally for accepting my interviews and giving precious feedback to my work. Special thanks are given to my family, my girlfriend Wenqian Wang and my friend Deborah Ann Howard for inspiring me to study in Germany, and most importantly simply for how you are. You are the reason that I ve never thought about giving up. vii

8 viii

9 ABSTRACT Integrated support functionalities (Embedded support) play an essential role for cloud software products in reducing the operational cost of the software service providers and improving overall user experience of the software service consumers. The current embedded support services based on ticket systems have made it possible for the end users to report software issues at any time and anywhere (UI components) in the products conveniently. The current embedded support services have drawbacks such as low first response time, poor quality of the created tickets (such as essential information to solve the issue, misleading description, etc.), low efficiency of communication between problem reporters and people solving the issues, and lack of interaction means. This thesis addresses the above mentioned problems by suggesting a new support service delivery model complementary to the existing ticket systems and implementing a software prototype based on the extensible Messaging and Presence Protocol. Unlike the many third party chatting products on the market which are functionally chat-centric, the implemented prototype provides access to different types of predefined context information thanks to its nature of being embedded in the product itself. Therefore the thesis implementation is fully context-centric and delivered to the users as part of the whole embedded support services of the cloud product. The thesis also provides results from experiments and questionnaires to demonstrate the effects and potential of the prototype. It shows that the prototype implements the embedded support services with desired features and in good quality. According to the stakeholders views, the suggested embedded support services will improve the overall process of handling software issues in SaaS organizations. ix

10 x

11 CONTENTS 1 Introduction Background Motivation Objectives Definition of Terms Description of Remaining Chapters Goal Setting Review of Literature Software Support Model and Software Support Services Software Bug Report(Ticket) Quality and Coefficients Internet Instant Messaging Internet Instant Messaging (IIM) Protocols Comparison Discussion Field Study: State of the Art Embedded Support in Enterprise SaaS Enterprise SaaS Support Structure xi

12 2.2.3 Live Support Adoption Compare and Analysis Goals and Summary Goal: Live Chat for Fast First Response Goal: Templates for Efficient Communication Goal: On-line Diagnose with Live Chat Goal: Live Chat - Solution Search Integration Goal: Chat Assistance Summary Embedded Support Services Concept and Design Concept Precap Embedded Support Services Requirement Logical Architecture Structural Architecture XMPP Client-Server Architecture JavaScript XMPP Client Structural Architecture Process Architecture Graphical User Interface Design Summary Implementation Development Platforms Programming Language XMPP Server Development Environment xii Contents

13 4.1.4 Implementation Baseline (existing feature to be adapted and reused) Packages Overview Bundling of Embedded Support Services Basic Chatting Service: Connection Management, Message and Contact Handling Connection Management Contact/Roster Management Message Extension User Interface Web-based Collaboration Service for Embedded Support Screen Sharing through XMPP DOM Elements Highlighting through XMPP Dynamic Templates Fetching Access to Holistic Context Information Service Link to holistic collection of context information HTML Page Annotation in IIM for Embedded Support Human IT-supported Support Service Business User Transfer Service Support User Assisted Ticket Creation Service Collection of Chat History as Context Information Service Automatic Chat Assistance Service Summary Evaluation Evaluation Criteria Evaluation Design Design of Tests Design of Questionnaire Contents xiii

14 5.2.3 Testing Environment Thesis Prototype Testing Set-up Results from questionnaire Respondents Job Distribution Assessment of Features Coverage Assessment of Provided Context Information Quality Estimated Influence on Tickets Estimated Influence on Customer Satisfaction Estimated Influence on SaaS Provider Cost Expectation of Support User created Ticket Results from tests XMPP Screen-sharing Accuracy XMPP Screen-sharing Performance Screen-sharing Performance compared with Citrix and Adobe products Summary Conclusion and Future Work Conclusion Answered Questions Future Work Bibliography A Acronyms 85 B Evaluation Questionnaire 87 C Screen-sharing Evaluation Results 93 xiv Contents

15 LIST OF TABLES 2.1 Important bug report description items Support Offering in non-erp software market overview Support service in Enterprise Resource Planning SaaS market overview Provided Functions of Chat as Embedded Support Services User Stories for Business User User Stories for Support User Stanza Extensions and Handlers Goals Features Mapping Hardware Environment Software Environment C.1 Screen-sharing Performance Measurement C.2 Screen-sharing Latency Comparison C.3 HTML tags with counts (shared screen) C.4 HTML tags with counts (displayed screen) xv

16 xvi List of Tables

17 LIST OF FIGURES 1.1 SaaS support ecosystem Lifecycle of a Bugzilla Bug Software bug report and its related entities in FLOSS Access in to support service in SaaS Filling in information while user creates a ticket Software Issue Handling Chain Software bug report and its related entities in Enterprise Resource Planning SaaS Ticket quality improvement New User Roles: Business User, Support User Logical Architecture Support User Logical Architecture Business User XMPP client-server Architecture JavaScript XMPP Client Structural Architecture UI Controller Component Connection and Messaging Manager Component Initial Start-up Process xvii

18 3.10 Screen-sharing and DOM Element Indication Incident reporting and content proposal Business User UI Concept Support User GUI Implementation Baseline with Added Functions Package Diagram of Thesis Implementation Object diagram depicting ConnectionManager Integrated Live Support in Embedded Support Services Implemented Business User GUI Support User GUI Context Providers Facade Screen Sharing Results Examples State Chart: find indicated UI element Screen Sharing Results Examples Captured UI Component Keys Adaptation to JavaScript Screen-shot Tool(JSST) HTML page annotation before holistic collection of context information Captured UI Component Keys Solution search with automatic search terms extraction Business User Transfer Service UI Support User Assisted Ticket Creation Service Design of Bot for Automatic Chat Assistance Automatic chat assistance Test Set-up Respondents Job Responsibilities xviii List of Figures

19 5.3 importance of chat features Estimation of Ticket Avoidance Estimated Influence on Tickets Vote for influence on customer satisfaction Influence on Workload Percentage of Tickets that Should Be Created by Support User After Chat Comparison of HTML tags and their appearances Difference of HTML tags appearances (Group 1) Screen-sharing Performance measured by milliseconds Screen-sharing Performance relative to network transfer time Screen-sharing Performance compared with Citrix GoTo Assist and Adobe Connect 77 List of Figures xix

20 xx List of Figures

21 1 INTRODUCTION This chapter starts with illustrating the context in which the thesis work addresses the target issues. After defining the context, the chapter lays out the target issues and goals of the research, continued by an overview of the structure of the whole work. 1.1 BACKGROUND When a user of a software product is hindered from getting his/her expected software functionality from the product, there might be different reasons. To categorize the various reasons, it is reasonable to first confine types of software that are addressed with this thesis. If we constrain the scope of software to interactive web-based applications delivered through the cloud, the reasons of above mentioned user s issues fall into three categories. The first category is natural reasons, such as electricity outage, natural disasters at user s side, hardware system failure, and so on. Though these issues can be tackled by fault-tolerant enhancements such as redundancy, in most cases these enhancements are hardware related, thus are transparent to software users and even software providers. The second category is low level service-related errors such as network failures, which may lead to inaccessibility to the software functions. As software (or software service) provider, these errors can be sometimes detected, however to correct them is the responsibility of service providers. The third category is software related reasons. Either the user is not familiar with the software, not well trained to find the functionality; or the software behaves abnormally; or the functionality demanded by the user is not implemented. These three problems are referred to as "software issues". Modern software providers often use technical means to try to detect these situations when they occur, to provide users support to report software issues, and to provide users solutions for these problems. 1

22 In the following sections and chapters, this report takes care of issues of interactive web-based applications delivered through the cloud, taken Enterprise Resource Planning cloud solutions as examples and implementing platforms. Unless otherwise presented, the term "software" will be used specifically to indicate interactive web-based applications delivered through the cloud; given these three reason categories above, it is reasonable to focus on tackling software issues as the software producer. Software issues happened at the user s side negatively impacts the user experience directly. If software defects are not solved by the producer, certain issues may happen repeatedly and affect more users. In a business context, there are also direct business impacts of the software issues. When this software is purchased with a guaranteed quality of service, as most enterprise applications do adhere, the user needs support from the vendor to resolve the issue. Enterprise Resource Planning solutions are nowadays deployed on cloud to reach more customers; they are good objects to study given that many of their features are intensively interactive. Enterprise Resource Planning software is designed in a way to cover users at any time at any place on any device. No matter the user is a postman who delivers parcels and collects electronic signatures, or is a salesman who always flies and is always on the way, or is an office employee who use different PC platforms, they can always access the service via the Internet. There is a fundamental difference in case of SaaS, compared with traditional on-premise solutions. Under the current popular pricing policies of SaaS products, customers do not pay for maintenance[cus10]. Of course different levels of service are usually available for customers to choose from and pay for, but it is still true that under the service agreement, the SaaS vendor is solo responsible for cost of issue resolution, thus the following formula always holds: C direct = N C (1.1) C direct stands for direct cost of total software issues, N for total number of software issues. And C for average cost of each software issue reported to the vendor side. Assume that not all software issues are resolved with user satisfaction, additional indirect costs will yield as customers decide to switch vendor or simply stop renewing contract. Complaints regarding bad support have already emerged in social networks, this bad reputation also falls into the category of indirect costs of software issues. Since the indirect costs are almost impossible to measure but necessary to consider, it is defined on coarse granularity, simply as C indirect. So far we have the equation for costs of software issues C: C = N C + C indirect (1.2). Without going too deep to the influencing factors to C and N, we know that the longer the period to resolve a software issue is, the higher C direct would be. And user satisfaction is closely related with C indirect. Maintenance costs are generally responsible for 40% to 80% of total software costs and maintenance is probably the most important life cycle phase [Gla01]. "Enhancement is responsible for roughly 60% of software maintenance cost. Error correction is roughly 17 percent."[gla01]. Conclusion can be drawn that efficient and effective support is vital for success of enterprise SaaS. 2 Chapter 1 Introduction

23 Figure 1.1: SaaS support ecosystem However, when the user creates a software issue report, it is the support engineer s job to understand the type and content of the issue, find out exactly what has happened and either fix a software bug or instruct the user right actions. Figure 1.1 is derived from a typical SaaS Enterprise Resource Planning product to show different parties involved in a software issue resolution process. In figure 1.1, internal scenario describes that software issues can be reported by the software testers and developers; in contrast to the customer scenario where tickets (software issue reports) are filed by the customers, either end users or their administrators. The figure shows different parties involved in a issue resolving process which partly depends on the experience of the support engineer but more importantly, on the information the support engineer can get about the issue. When the support engineer can reproduce the software issue with important context information like user OS, browser information, active UI component, invoked server module, then it is easier to solve the issues, the less this information is present, the more difficult it is, thus the higher C is and more likely the higher C indirect will be. There are already many tools in the market to trace software issue with simple descriptions and communication records by predefined work flow, which only starts with a description written after the issue occurred outside of the user system. Much of useful context information is lost in this way. Some other embedded support functions only works in certain browser runtime, which limit the reusability, portability, thus limit also the covered user devices. Besides the contribution of maintenance cost reduction, embedded support also is much used in today s software development and test phases. At the time a developer or a tester detects a malfunction of the software; he simply clicks a button in the tested product and files a report about exactly the software issue with exact related context information. Later on, the defect part will be taken care of by the responsible development team, with all the information collected at the scene. This whole procedure will reoccur at high frequency each development iteration, in each sprint, and in each release, until the software matures. By shortening software 1.1 Background 3

24 bug fixation time, embedded support helps also to make the product mature earlier, thus more competitive on the market. Given the status quo above, it is clear that the cost of software issues resolution must evaluated in depth. Because the target software is interactive web applications, new features based on modern Web technologies like JavaScript, the dominant front-end script language, and HTML5 must enable SaaS vendors to resolve software issues in a more timely manner, with less cost for a single issue C, and preferably in the end, with a smaller N fraction by acceleration of software maturation. 1.2 MOTIVATION According to the research on topics with embedded support concept which is done in the academia, it is found that most of the existing research focus on maintenance of software, and those researches are quite general without focus on SaaS. As a very important and widely adopted technological solution for cloud business software maintenance, embedded support is not given the deserved formal concept establishment. Also, there is no structured analysis of the purpose and challenges of embedded support in the field according to research results of the thesis. Given the huge share of maintenance cost in the total software TCO (Total Cost of Ownership), and the fact that support is an indispensable part in the maintenance workflow of modern SaaS solutions, the thesis will make efforts to establish the support concept, investigate the role and effects of support. Focusing on the embedded support paradigm, the thesis will also challenge the traditional technological methods to tackle the issue of effective support. 1.3 OBJECTIVES The major objective for this thesis is to find the critical path in software issue resolution in the lifecycle of enterprise SaaS solutions; and shorten the length of this critical path with most modern concept of embedded support. At the end of this thesis, the following questions should be studied and answered: What are the key factors influencing the monetary cost and resolution time of a software issue? Which technologies can be utilized to reduce this cost and time consumption? Are the above mentioned potential technological solutions feasible? This question will be answered by prototype building, design and implementation should be described. How can a measurement of improved user experience and reduced cost be established? What is the estimated or measured contribution of the technological solutions? 4 Chapter 1 Introduction

25 1.4 DEFINITION OF TERMS Service: The definition of "Service" is taken from OASIS SOA reference model. "A service is a mechanism to enable access to one or more capabilities, where the access is provided using a prescribed interface and is exercised consistent with constraints and policies as specified by the service description. A service is provided by an entity - the service provider for use by others, but the eventual consumers of the service may not be known to the service provider and may demonstrate uses of the service beyond the scope originally conceived by the provider". [MLM + 06] ERP: The acronym ERP stands for Enterprise Resource Planning systems. These systems integrate internal and external management of information across an entire organization embracing finance/accounting, manufacturing, sales and service, customer relationship management, etc.[wik] Cloud Computing: "Cloud computing refers to both the applications delivered as services over the Internet and the hardware and systems software in the data centers that provide those services". [AFG + 10] Everything as a Service (XaaS): According to recent study[bfb + 11], cloud computing system is now delivering everything as a service. To put it more concretely, three common XaaS examples are: IaaS, PaaS, and SaaS. IaaS stands for "Infrastructure as a Service", where infrastructure means technologies for servers, storage, networking and IT management. PaaS stands for "Platform as a service", which means the technologies to share cloud infrastructure to provide service platforms. The third example is: Software as a Service (SaaS) : According to the definition of "cloud computing"[afg + 10] the applications delivered as services have been long called "Software as a Service", or SaaS 1. In this thesis Software as a Service refers also to only the software that is delivered to the end users in the form of a service which is accessed over the Internet[AFG + 10]. Another often used synonym to SaaS is "Hosted Software" 2. Software Support and/vs. Software Maintenance: In software engineering, Software Maintenance[ISO06] refers to three kinds of software modifications, namely "adaptive maintenance", "corrective maintenance", and "emergency maintenance". All of them define modification of a software product after delivery. Purpose of modification is either correction or enhancement[iso06]. It is very important to notice that in the 2006 ISO standard, "pre-delivery activities include planning for post-delivery operations, supportability, and logistics determination". [ISO06] However, modifications pre-delivery are not included in the maintenance process. Software Support: While software maintenance focuses on change of software in one life cycle process, software support is defined closely related with the usage of the software. Because software usage happens before delivery inevitably for testing reasons, thus support covers a longer time than one life cycle process. Also, the usage of the software can yield problems which do not necessarily results in a software change, therefore educating and instructing software users are also inherently a support task. 1 SaaS is also called "On-demand Software" in many business naming paradigms. 2 The name of "Hosted Software" comes from the paradigm that software vendor hosts the software itself, customers and users do not store, install, or maintain the software and its databases. 1.4 Definition of Terms 5

26 There are two major differences between Software Support and Software Maintenance. Firstly on the time scope, maintenance only refers to modifications after the software delivery. Secondly, the subject of maintenance is developers, the object is the software itself, however, objects of the support can also be the users from users of the software, and therefore subject of support can be dedicated "support employee", whose job is to consult, educate the software users. Software Supportability: In this thesis, software supportability is defined as the ease of a software product to be supported in order to: correct a software bug; improve on performance; extend functionality; or to educate the user how to use the product. Incident: A user reported software issue. Embedded Support: The term embedded support is not to be confused with general embedded systems. "Embedded" describes the way how the support services are integrated into the software product. When the user can directly access support services from the business product, then the service is "embedded". In other words, in contrast to many existing IT support systems which standalone from the product, embedded support is part of the product itself. Context Information: "Context is any information that can be used to characterize the situation of an entity. An entity is a person, place, or object that is considered relevant to the interaction between a user and an application, including the user and applications themselves."[adb + 99] Regarding software support, important context information includes information which is extractable either from the state of the client system automatically, or from human user s input when a software issue happens. This information will be transferred to the support system and be used for incident diagnostics. 1.5 DESCRIPTION OF REMAINING CHAPTERS Chapter 2 (Goal Setting) depicts the role of software support in enterprise SaaS solutions. It shows results from the review of literature and study in the industry. This part will go into detail to find where exactly support cost happens. This chapter also introduces up-to-date research results and technologies which are leveraged to solve software support issues. The findings act as benchmarks for comparison in later chapters. In the end of the chapter, clear goals should be set up from technological aspects, which will guide the following chapters in the thesis. Chapter 3 (Concept and Design) The concept section will propose a solution to meet the objectives of this thesis and list the software requirements of the solution. The Design section describes design of solutions in more details. Different design considerations are listed and weighed. The reasons of design decision are clarified. In the end of the chapter, the selected technologies should be clear, risks and potentials of the design choice should be explicit. Chapter 4 (Implementation) The Implementation chapter answers the "How" question. This chapter describes the implementation by listing important data structure and algorithms. It also 6 Chapter 1 Introduction

27 explains how design goals are achieved in detail, some important code will appear for sake of conciseness. Chapter 5 (Evaluation) presents key effect indicators for the implementation. It describes what should be measured and how to measure. It shows the measurement results and analyses the effect of measurement. Chapter 6 (Conclusion and Future Work) summarizes the problem settings of the thesis, reviews the approaches adopted in the thesis and emphasizes the important findings, achieved results. This chapter will also provide the unanswered questions left in the field of study. 1.5 Description of Remaining Chapters 7

28 8 Chapter 1 Introduction

29 2 GOAL SETTING 2.1 REVIEW OF LITERATURE There is literature describing the complexity and cost of software support. Regarding popular software support models which define user roles and system behaviour, author of this report did not find systematic study in the academia. However, documentations of mature open source software issue tracking systems provide some insight into these models. In this section, the thesis conducts a thorough research into these research papers. The target is to discover the impact of software support and the known technological solutions to reduce the negative impacts. The existing challenges in software support are deduced from this literature review. In the discussion part, contributions of the studied literature are listed and compared; a solution proposal which uses the literature findings is set up Software Support Model and Software Support Services "Bugzilla is a Web-based general-purpose bug-tracker and testing tool originally developed and used by the Mozilla project, and licensed under the Mozilla Public License."[Wik13] According to Bugzilla s official site, it is defined as a "Defect Tracking System" which allows "individual or groups of developers to keep track of outstanding bugs in their product effectively" 1. "The life cycle, also known as work flow, of a bug is currently hardcoded into Bugzilla." 2 Figure 2.1 shows the life cycle of a Bugzilla bug. From figure 2.1, this report extracts the following four roles in Bugzilla s bug life cycle: User: bug reporter, whose reported bug needs to be confirmed or voted to be taken care of. Developer: processor of the bug report. Developer is also responsible for assigning bug reports to development

30 Development: development of software products which use BugZilla. QA (Quality Assurance 3 ): the mozilla.com QA organization is divided into teams focused on Mozilla product areas or technologies. 4 Some major differences of the Bugzilla support model compared with support models in business SaaS products described in later sections are: There is no service level agreement between each interacting roles. Bugzilla supports only software defects, general user questions are not target of answering. In Bugzilla, only approved bug reports instead of all of them are dealt with. Bugzilla does not have explicit software issue escalation. Fixes generated in bugzilla are and only are confirmed by the QA team. Gartner IT Glossary 5 defines software support services as "generally technical support or break/fix services that are delivered for specific software products. These services include revenue derived from long-term technical-support contracts or pay-as-you-go, incident-based support. Software support services typically include remote troubleshooting capabilities, installation assistance and basic usability assistance. Remote troubleshooting capabilities may be delivered via telephone and online communication media or without human assistance through automated means that reside on the customer s device or are available on the Web". Niessink [NVV00] performed a case study to investigate software maintenance as services in a large organization which provides Dutch social security system. The report divides the support services tasks into two processes. The first process is helpdesk management process; the second is the problem management process. "The implementation of these processes has been based on the Information Technology Infrastructure Library (ITIL). The ITIL process Helpdesk Management is used to guarantee the continuity of services, while the ITIL process Problem Management is used to improve the level of service in the future. So, Helpdesk Management deals with incidents, whereas Problem Management is concerned with solving the problems that cause these incidents"[nvv00]. "It is found that classification of the incidents was often impossible because free text description of the incident was too vague. In the cases were a classification was possible, 30% of the new classifications differed from the existing classification. Of the 30,000 incidents in the helpdesk database, 56% was not classified at all." At the end of the case study, the author claimed it "necessary to first implement a clear and consistent registration of the incidents that occur during service delivery, before attempting to improve the problem management process."[nvv00] Chapter 2 Goal Setting

31 Figure 2.1: Lifecycle of a Bugzilla Bug Software Bug Report(Ticket) Quality and Coefficients One research report[vm09] conducted under QualiPSo 6 project made suggestions for improving FLOSS(Free Licensed Open Source software) software quality. The recommendation 6 is "Continuous Project Improvement", the report suggested that software project improvement opportunities include bug reports and recurrent support request. This report also suggests adoption and implementation of requests/suggestions from users/ integrators and developers for both software product and process. One angle from which to know how long it takes at service vendor s side to resolve a user reported software issue is to look at how many times the ticket gets reassigned to different 6 The goal of the QualiPSo integrated project is to define and implement technologies, procedures and policies to leverage the Open Source Software (OSS) development current practices to sound and well recognized and established industrial operations.[qua09] 2.1 Review of Literature 11

32 people inside the vendor system. Bug fixing is more complex in big systems, when locating the capable and responsible personnel for a software bug itself demands efforts and skill. One research [GZNM11] describing software bug report reassignments in Microsoft 7 Windows Vista showed that there are five main causes of bug reassignments in the Windows Vista project. They are "Finding the root cause", "Determining ownership", "Poor bug report quality", "Hard to determine proper fix", and "Workload balancing" [GZNM11]. The study also shows it takes longer to close a ticket when reassignment increases. Cites from Microsoft employees say that "The most important factor in multiple reassigning in my experience is unclear bug reports" and "If a bug report cites only basic symptoms (such as crash ) and has little or no information hinting at cause (such as call stack), then triage is very difficult and a bug can end up being bounced around". The lack of context information harms software issue resolution. Hooimeijer and Weimer [HW07] produced a model for software bug report quality aimed at reducing cost of bug report triage. The study used public bug reports from the Mozilla Firefox project 8. This work also suggests features that should be emphasized when composing bug reports. Five major model features are "Self-Reported Severity" (severity of the software issue given by the issue reporter), "Readability Measures", "Daily Load", "Submitter Reputation", and "Change over time". "Priority changes" in the studied project is infrequent and considered to be negligible, therefore elided from the model features. The work used lifetime as a proxy for the cost of triaging. Major findings from this work which relates with the thesis topic are: Analysis of the models shows that "early" attachment count and comment count matter most, where comment in the study comes from "anyone", which is understood as users of the bug tracking forum. The presence of an attachment often means "expensive" bug reports. Attachment count has a negative coefficient to the cost, whereas comment count has a positive one. It is suggested by the author that this is because bugs that receive more user attention get fixed faster or important bugs have a great deal of user attention. Easier-to-read descriptions correlate with a short bug report lifetime. Patch count, escalations and de-escalations were not found to be consistently significant for various models. Bettenburg[BJS + 08]investigates quality of bug reports from the perspective of developers. Survey which involves both developers and bug reporters is conducted. 156 developer responses from APACHE; ECLIPSE, and MOZILLA projects and 310 bug reporters responses from the same projects were responding to the survey. To these two surveyed groups, two categories of questionnaires are presented, namely the "Contents of bug reports" and "Problems with bug reports". Consistency checks are performed to the received questionnaire and importance of the choice items is calculated as the percentage of the time selected as important for the developer on the base of overall used times: Importance(i) = N D1,D2(i) N D1 (i) (2.1) Chapter 2 Goal Setting

33 [BJS + 08]. 9 Important findings from the research are: "Step to reproduce" is the most important items for developers, after which follow "stack traces", "test cases", and "observed behavior". "Screenshot" is also important, but often only for GUI errors. Surprisingly, "expected behavior", "code examples", "summary", "version", "operating system", "product", and "hardware" are considered important with only lower importance. Most provided items are (in frequency-descending order) "steps to reproduce", "observed behavior", "expected behavior". Only few reporters used "stack traces", "code examples" or "test cases" to the bug reports. The most difficult to report items are (in difficulty-ascending order) "component", "stack traces" "code examples", "step to reproduce", and "test cases". Spearman correlation 10 between what developers use and what reporters provide is 0.321, a very low correlation. Spearman correlation between what developers consider important and what reporters provide is , a huge gap. Report providers know what information developers need. So ignorance is not the reason the information is not provided. An exception happens on item "Screenshot", where Spearman correlation is Incomplete information was by far the most commonly encountered problem by developers. Other common problems include errors in "step to reproduce" and "test cases"; "bug duplicates" and "incorrect version numbers, observed and expected behavior". Another challenging issue is the language fluency of the reporter. Developers are not impacted too much from bug duplicates. Bug reports containing stack traces get fixed sooner(apache/mozilla/eclipse); Bug reports that are easier to read have lower lifetimes (APACHE/MOZILLA/ECLIPSE); Including code examples into bug reports increased the chance of fixation (MOZILLA) Internet Instant Messaging A study of internet instant messaging and chat protocols[jno + 06] provides insights into technical aspects of chat protocols which are adopted by three major business internet chat products. Of the three studied products, namely AOL Instant Messenger, Microsoft Messenger, and Yahoo! Messenger, major features and functions are explored. Different underlying system architectures are depicted. They all adopted client-server architecture however with different system architecture: symmetric and asymmetric. Other technical aspects like session distribution, user authentication, data transfer are discussed and compared. At the end, SIMPLE and XMPP as IETF 11 standardized protocols are compared. 9 To be concise, for definition of question D1 and D2, please refer to the research [BJS + 08] 10 s_ rank_ correlation_ coefficient 11 Internet Engineering Task Force 2.1 Review of Literature 13

34 2.1.4 Internet Instant Messaging (IIM) Protocols Comparison The Internet Relay Chat and Extensible Messaging and Presence Protocol are two popular open standards for internet instant messaging applications. Internet Relay Chat is a text-based protocol, with the simplest client being any socket program capable of connecting to the server. [OR] Extensible Messaging and Presence Protocol (XMPP) is an application profile of the Extensible Markup Language (XML) that enables the near-real-time exchange of structured yet extensible data between any two or more network entities.[sa] It is an open technology for real-time communication, which powers a wide range of applications including instant messaging, presence, multi-party chat, voice and video calls, collaboration, lightweight middleware, content syndication, and generalized routing of XML data.[xmp13] Both standards mentioned above adopt client-server architecture. The messaging networks are both federated, meaning that chat clients exchange messages with each other via server routing Discussion This report cares about existing software support models and bug report studies in the academia because they address the issue of software issue handling based on reporting, which is also adopted as support methods by SaaS Enterprise Resource Planning vendors. And it is assumed that the cost of support is directly impacted by the quality of the software issue reports that are sent to the vendors. Figure 2.2 is derived from the above mentioned public researches conducted in open source on-premise software projects. It is an Entity-Relationship diagram which shows the main attributes composing a typical software bug report with necessary attributes. This figure also shows the importance of Ticket as the communication channel between bug reporter (end user) and bug fixer (developer). Whether or not the issue description conveys correct information about the software bug has a direct impact on the time consumption of software issue diagnostics. From the above mentioned studies, we can see that bug report (ticket) quality has a significant impact on the communication efficiency of developers and bug reporters. The gap between the user-offered and the developer-needed information is huge. There is information which is very important for the software provider to fix a software issue. However in many cases this information is either not described or not given at all. There is also evidence showing the positive correlation between a bug report s quality and the time it saves to resolve a reported software issue. Fortunately, the studies mentioned above have discovered many categories of information which should be provided to the engineer who makes efforts to try to triage and fix software defects. Combining the findings, Table 2.1 shows the information that should be correctly provided to support organization in the software issue reports. It is reasonable to deduce that if there are dedicated tools to enforce these information requirements from the software bug reporters, the quality of bug reports will be improved significantly. 14 Chapter 2 Goal Setting

35 Figure 2.2: Software bug report and its related entities in FLOSS(Free licensed open source software) The adoption of internet instant messaging by large Web application providers like Google and Facebook has provided mature technology for online communication. When properly customized for Enterprise Resource Planning SaaS support, on-line chat has potential to be adopted as a tool for embedded support. Such a tool can provide users instant access to support service with a very short first response time, thus provide better user experience. On-line chat has also many other potentials regarding fast data transfer, user friendliness and its advantage over phone. These advantages will be further discussed in the thesis. The strength of XMPP over IRC is its XML data format. This thesis can leverage its built-in XML engine and define extensions to allow features more than text messaging. The studied literature did not provide methods to prevent the users from reporting issues. Three possible explanations for this are: the objects of the studies are software bug reports, which do not include user s service requests for answering "How to" questions, so preventing users reporting an software issue is not a goal; the studied software, especially Mozilla Firefox, have narrower uses cases in comparison with highly complicated solutions like cloud Enterprise Resource Planning, so users of such software do not have a high intention to ask for help to use the software; Eclipse users studies are in most cases developers, who are already supposed to be familiar with various channels on the Internet to seek for their questions. 2.2 FIELD STUDY: STATE OF THE ART To get an overview of how support is handled in the industry, the thesis investigates major cloud Enterprise Resource Planning providers in the markets. It is found that all vendors in the field supply their customers with support. So that further field study is described focusing on the following aspects: business models for the support service; adoption of embedded support; and adoption of real-time live support. This field study should inspire on cutting-edge technologies the thesis software implementation shall leverage. Also, the solution proposed by 2.2 Field Study: State of the Art 15

36 Table 2.1: Important bug report description items Name Often missing Often incomplete often misleading hard to provide steps to No Yes Yes Yes reproduce stack Yes Yes No Yes traces test Yes Yes Yes Yes cases observed No Yes No No behaviour screenshot Yes Yes (e.g. more screenshots are No No needed, but only one is provided) expected No Yes No No behaviour component Yes Yes No Yes reporter language fluency Yes the thesis should be compared with and measured against these existing technologies. To illustrate some concepts in SaaS embedded support, this report takes SAP s cloud offerings as example, screenshots are taken in one of the SAP s cloud ERP products Embedded Support in Enterprise SaaS Nearly all solutions provide on-line case submission systems for support. These systems tracks the whole life cycle of software issue reports (tickets). Figure 2.3 shows a user interface from where users can access support services provided by the SaaS vendor. Figure 2.4 depicts a user interface of users creating a ticket in an example product(sap Business ByDesign) Enterprise SaaS Support Structure Software Issue Handling Chain It is definitely inefficient and a waste of resource to let developers spend time talking with end users on support desk without restriction since developers should focus on development tasks; general user questions should be directed to product experts or support employees. Modern SaaS Enterprise Resource Planning deploys multiple support levels in a chain as filters for general user reported issues. Figure 2.5 is derived from this filter approach and shows a typical structure of different support levels. This data flow diagram depicts the software issue report s processors in the service sequence. The Customer End User is the sender of a software issue report. The Customer Administrator is an experienced user in his organization. Traditionally, the end user cannot contact the administrator, such a one way communication paradigm is kept during the life time of a software issue report. Therefore, a chain of data flow is formed. In Figure 2.5, Direct Support is the party at the SaaS vendor who speaks the user s language and translates the software issue report into a language 16 Chapter 2 Goal Setting

37 Figure 2.3: Access in to support service in SaaS that is understood globally in the chain, such as English. Expert Support has more advanced product knowledge; it begins to handle the issue once direct support forwards the support service request to it. When expert support investigates and believes the issue is a software bug, it will send the report to Development Support, who will check the issue and fix the software bug if it exists. The reason behind this design is that most SaaS vendors decide to reduce cost by letting development support focus on only software defects, and lower support levels should filter out other kinds of software issues. The drawback of this chain can be seen from Figure 2.5, that once a software issue report is created, each processor only contacts with its neighbors. When the quality of issue report is low, it will happen that the communication loops for many rounds inside the chain before the right responsible party is found. The higher the software issue escalates, the more costly it is to require additional information from the software issue reporter because more levels of support will be involved into the communication Live Support Adoption Big software providers such as Redhat and Amazon have different support offerings for their products. By studying these support offerings, we can check if they leverage also live support. In case yes, this report tries to find out which technologies are now used to enable the live support. Table 2.2 lists the major findings from the market study in non-erp software support offerings. It reveals a tiered support model as the support services are offered in different levels and charged accordingly. 2.2 Field Study: State of the Art 17

38 Figure 2.4: Filling in information while user creates a ticket Drilling down onto Enterprise Resource Planning SaaS support services, this report reveals that most Enterprise Resource Planning SaaS vendors have to bring down the communication barrier between their support department and the users is live communication, which enables the so-called "real-time" support. This kind of real-time support in most cases indicates a direct phone call. In this sub-section, a research on this live support service offering is presented. This thesis takes examples from the market leaders in Enterprise Resource Planning SaaS products, and studies if and in case yes, what service channels are used by the vendors. Table 2.3 shows an overview of support service levels provided by the examined solutions and the existence of remote live support. The thesis takes a closer look at how these SaaS vendors provide live support: Salesforce.com As the one of the market leaders in CRM 12 field worldwide in the year of 2012, Salesforce.com 13 generated a revenue of $2.5 Billion 14. The company offers its products with 3 levels of customer support, from the second of which 24/7 phone support is available. All three levels enable user to use traditional built-in issue report functionalities. However, live chat is not available. SuccessFactors As a very strong cloud HCM 15 solution provider rated by Gartner 16, SuccessFactors 17 cloud offering provides users "Live chat for real-time answers to simple questions" Customer Relationship Management Human Capital Management Chapter 2 Goal Setting

39 Figure 2.5: Software Issue Handling Chain ORACLE CRM Ondemand 19, NET SUITE 20, Workday 21, and Sugar CRM 22 From the public research and company public websites, the thesis found that they render different support options. For example, Oracle CRM Ondemand provides phone support service to log service request during normal business hours; Net Suite provides a similar service on 24/7 base. However, there is no evidence to show these company are providing or preparing to provide remote on-line chatting Compare and Analysis The thesis found only one adoption of on-line chat into support service among all studied market players. Yet the use of the chat tool is only for answering users "simple questions". Therefore, it is reasonable to assume that there is until today no product on the market, which use chat functionality to provide on-line real time problem diagnose Field Study: State of the Art 19

40 Solution Table 2.2: Support Offering in non-erp software market overview Support levels Live support from Live support form Fastest first response Bug tracking system usage Redhat Linux 3 levels Level 2 phone 1 hour Y Y Canonical 2 levels N/A N/A N/A N/A Y Ubuntu Desktop Management Canonical 3 levels N/A N/A N/A N/A Y Ubuntu Server Management Canonical 3 levels Level 1 phone N/A N/A N/A Ubuntu Cloud Management Adobe products 8 levels Level 1 phone, 30 min Y Y online text chat, adobe connect Amazon Web 4 levels Level 3 Phone, < 15 min Y Y Services chat, live screen sharing, Online knowledge base Figure 2.6 is derived from a typical support organizational layout from a well-known Enterprise Resource Planning SaaS vendor. It is a Entity-Relationship diagram which shows the main attributes composing a typical software issue report. In comparison with Figure 2.2, we find software support related tasks are much more complex within a Enterprise Resource Planning SaaS product. 2.3 GOALS AND SUMMARY The general goal of the thesis is to use technological means to cut operational cost at SaaS vendors side. For that purpose, attempts will be made to reduce the number of software issue reports which can be resolved without entering the different support levels. Once an software issue report is created, it will be tried that support engineers at different levels can use the thesis implementation to reduce the time it costs to resolve the software issue report. For the mission mentioned above, the goals listed under focus on different aspects in software issue handling. These aspects are: to provide faster first response time; to improve user-support communication quality; to empower support engineers with more context information and to make support engineer quick access to knowledge data bases. These different aspects are technologically intersected on the point of usage of on-line chatting. 20 Chapter 2 Goal Setting

41 Table 2.3: Support service in Enterprise Resource Planning SaaS market overview Solution Number of support levels Remote live support from: Salesforce.com 4 levels Level 2 24/7, Phone Remote live support form Oracle CRM Ondemand 1 level Level 1 Business hour, phone Net Suite 3 levels Level 2 24/7, phone Workday not published not published Success Factors 3 levels Level 1 (phone only) 24/7, published 24/7, phone, chat (from level 2) Sugar CRM not not published not published published SAP Business ByDesign 3 levels Level 1 Business hour, phone Detailed requirements are not described in this chapter; they will be described in the Design and Implementation chapter afterwards Goal: Live Chat for Fast First Response From the field study, it is shown that first response time to user after a software issue takes place is one of the key performance indicators of the support service quality of the SaaS solution. The best guarantee found during the study is one hour, however this level of QoS comes with a higher price, therefore does not apply to all the customers and users. Also, this one hour first response happens usually between user and support through built-in software bug tracking system, which is slow. On-line chat can very well bridge this gap between users and support. When a user faces a software issue, he can simple ask for help by logging-in to the chat system. This request for chat will be immediately visible at the support side. Such a request can be picked-up instantly while the responsible support personnel is available. Therefore, first response time is reduced. Implementation of such a live chat tool is a goal of this thesis. To implement such an on-line chat system, it is important that the first response to user is handled by support engineer who can understands the user s language and has a general knowledge about user s product configuration. 2.3 Goals and Summary 21

42 Figure 2.6: Software bug report and its related entities in Enterprise Resource Planning SaaS Goal: Templates for Efficient Communication From the various research results presented in this thesis, and also from the interviews conducted inside SAP s support organization. It is clearly shown that the communication quality largely decides how long it takes to understand the user s software issue. In most cases, the type of issues encountered are closely related with the location when the issue happens. It is therefore important to enable the user to ask for help from support at every UI in the software product. However it is too costly to have a support engineer who chats on-line with the user who knows every product and every business process. Therefore, on-line chat should be able to provide the support engineers with communication templates which are generated from the UI where the user requests the on-line chat. Besides these UI-specific communication templates, there should be also template questions which cover highly-reused procedures related with software issue handling, such as asking the users to describe his expected system behaviour and the output he gets. 22 Chapter 2 Goal Setting

43 2.3.3 Goal: On-line Diagnose with Live Chat For support engineers who have the ability to resolve a software issue on-line during the chat. It is preferable that user context information can be supplied to assist the handling of software issue. Such context information should be pro-actively required from the customer user at the very beginning of each chat session. Once the user agrees to render the context information, this information should either be transferred to the back-end system or directly to the chatter who acts in the supporter role. In on-line chat module, either the information directly or a link to this information should be supplied Goal: Live Chat - Solution Search Integration The main reason why solution search should be done also by the support engineer rather than only the user is that often a solution file which is generated by the SaaS provider is written in professional languages, with company specific terminology. As a result, it is difficult to query solution proposals with appropriate search terms and to understand the provided results. However these terms are assumed to be well known to support engineers. Based on the idea of on-line chatting, while the support engineer is making efforts to investigate the software issue and helps the user to resolve these issues. It is important to make the support employee convenient access to the knowledge storage of software issues. This storage can be either the product data base, forums, or solution case data base Goal: Chat Assistance For Enterprise Resource Planning SaaS support via chatting, no chat bot should be directly activated to communicate with the customer user due to the low maturity of the bot s intelligence, which would upset the users. However, bot can be deployed to help support employees for analyzing user input and perform assistance tasks such as template suggestion and solution search. Interface for bot should be provided in the thesis implementation for future work. 2.4 SUMMARY Chapter two described the general business demand which drives the modern software support technologies. Based on the research from academia, major challenges in software bug reporting handling are presented, key factors influencing the cost of handling user reported software bug are analyzed. The field study section conducted a research in the SaaS Enterprise Resource Planning market, finding out what kind of support services exist. Comparing the results from the academic research and the existing support technologies, development goals are set up at the end of the chapter. Through the implementation of the described software 2.4 Summary 23

44 solution, the purpose of improving user-support communication should be achieved, In the end, the operational cost of Enterprise Resource Planning SaaS should be deducted. 24 Chapter 2 Goal Setting

45 3 EMBEDDED SUPPORT SERVICES CONCEPT AND DESIGN 3.1 CONCEPT PRECAP In the previous chapter, this report has introduced the issue escalation chain in dealing with user reported software issues in a large organization. In figure 3.1a, we can review this chain. From the field study and research in academia, we have already known that poor quality of software issue reports is the reason why lots of communication overhead exists in the support organization. These communication rounds are represented in figure 3.1a as the red circles. The bold red arch indicates the creation of a ticket, which is the start of the communication of poor quality in case the ticket is created by the customer administrator. In case the ticket is created by the end user, then the dashed bold red arch indicates this starting point. With service configuration, chat has the potential to be placed inside the customer organization so that customer end user can access the service. Therefore, a concept of "Chat" is introduced in figure 3.1b, shown as the "Chat" node in the data flow diagram. It is intended that from the services provided by "Chat", the software issue handling chain will be shortened in time. The service of chat should serve different purposes to different kinds of service consumers, which will be illustrated in section EMBEDDED SUPPORT SERVICES From figure 3.1, it can be seen that both users from SaaS provider and users from customer need services that provide "Chat". The current Embedded Support functionalities have to be re-bundled into the application service with completely new delivery channels. Such new embedded support services are consisted of many services delivered to both sides in the 25

46 (a) Software Issue Handling Chain before improvement (b) Software Issue Handling Chain with "Chat" embedded support services Figure 3.1: Ticket quality improvement interaction between support organization and the customer. The main functions the embedded support services provides to users are depicted in table 3.1. Here some of these are explained more explicitly for sake of clarity: Table 3.1: Provided Functions of Chat as Embedded Support Services id Object Function F1 SaaS vendor ticket quality enforcement tool F2 SaaS vendor support employee assistance tool F3 SaaS vendor support service request concurrent handling enabler F4 SaaS vendor support service request redirect tool F5 SaaS vendor on-line user issue diagnostic tool F6 SaaS vendor ticket number reduction tool F7 SaaS vendor user education tool F8 SaaS user fast support response channel F9 SaaS user remote assistance tool F10 SaaS user SaaS product learning channel F11 SaaS user SaaS product knowledge storage F12 SaaS user/vendor user experience improvement F13 SaaS user/vendor coordination tool in web F1: ticket quality enforcement tool Instead of allowing users create incidents freely with inaccurate, incomplete, even misleading information. The "chatter" will request the important information pro-actively during chatting. Ticket will only be created at the end of each chatting instance, with data that are manually or automatically selected by the chatting support employee. Thus, ticket quality should improve. F2: support employee assistance tool Modern SaaS comes in a way of highly complicated and integrated services. To train support employees who chat with the user to be product expert is too expensive to be realistic. Automatic chatting assistance should be provided to support chat users. This assistance service should provide templates which adapt to each 26 Chapter 3 Embedded Support Services Concept and Design

47 individual software issue context. It should also be possible that each support employee creates his own chat templates. F3: support service request concurrent handling enabler Traditional help desk approach of phone call has the restriction that one support employee can only address one user at one time. The concept of "Chatter" should empower support employees to serve multiple users simultaneously. This can be done by opening multiple chat windows. F4: support service request redirect tool There should be at least one support employee on-line who chats with the users who request the chat service. When there are multiple support employees on-line, they should have the ability to redirect a user chat request to other support employees for reasons of different expertise or load balancing. F5: on-line user issue diagnostic tool There are already existing tools for diagnostics of user reported software issues. However, the access to these tools traditionally only happens from reviewing a user ticket. When support employee or developer uses such diagnostic tool and finds it necessary to ask for clarification or complement, new circles of communication in the software issue escalation chain take place. Thus, these tools should be accessed at the "chatting time" before tickets are created. More importantly, context information regarding a software issue should be collected for diagnostics automatically at or before "chatting time". F6: ticket number reduction tool Based on the assumption that every ticket takes examining time at least on one level at SaaS vendor side in the escalation chain. It is preferred that easy-to-answer questions are directly answered during a chat instead of after entering ticket handling process. F11: SaaS product knowledge storage Within the same customer organization, administrator user deals directly with the software issues reported by the normal end user. When the root cause of a software issue is software defect, or the product went live a short time before, many software issues will be met by different end users. When one issue solution is suggested during a chat, at least administrator user should be able to store the chat instance. Later on when other users meet the same software issue, the administrator can then directly forward the chat history as solution proposal. F13: coordination tool in web Support employees at SaaS vendor s side use many technical terms in their language. However end users are in most of cases unaware of such terms. This creates a communication gap during issue description and a coordination obstacle afterwards in trouble shooting. Thus, a tool should be provided to improve the coordination capability. One idea is to let both parties access a shared HTML page and annotate it. There are two user roles established in the software requirement document of thesis, depicted by figure 3.2 The first user role is Business User, which refers to a user who requests customer support service via on-line chatting, this role replaces the "End User" and "Administrator User" roles depicted in figure 1.1. The second user role is Support User, which refers to either an employee from SaaS vendor or an experienced user from the customer organization who accepts the Business User s service requests and handles the software issue during chatting. 3.2 Embedded Support Services 27

48 Figure 3.2: New User Roles in Support Ecosystem: Business User, Support User The two roles, "Business User" and "Support User" roles, are subjects to chat service. The prototype which is to be implemented should leave it open for configuration of user roles mapping. An example mapping into the business roles is given: the Business User can either be an end user of SaaS, or a key user (business administrator) at the customer s side; while the Support User can be either a support engineer in the different levels of support organization at SaaS vendor s side, or a developer who handles a potential software defect and establishes a chat session with the software issue reporter. 3.3 REQUIREMENT Functional requirements are described in the form of user stories. Table 3.2 lists the functional requirements from the Business User s point of view; table 3.3 Support User s. Several important non-functional requirements are: NFR1: Platform Independence. Implemented solution should be compatible with the largely adopted SaaS end user devices. The execution environment should be available on both desktop computers and mobile devices. NFR2: Scalability. Implemented solution should be scalable so that increasing number of SaaS users would not dramatically harm the system s availability or performance. NFR3: Conformance of Graphical User Interface(GUI) Design. Implemented in a productive SaaS product, the solution should not break the overall GUI design principles, themes, etc... NFR4: Security. As an on-line interaction service, the solution interface should provide basic security mechanisms to prevent message flooding and script injection. 28 Chapter 3 Embedded Support Services Concept and Design

49 Table 3.2: Functional Requirements for Business User 3.4 LOGICAL ARCHITECTURE The logical architecture is shown in two separate parts from the perspective of Support User and Business User. Logical architectures which are depicted by figure 3.3 and figure 3.4 reflects the functional requirements collected in the previous sections. From figure 3.3 and figure 3.4, the major components of the thesis implementation can be derived, they are: Connection and Messaging Manager Roster Manager Context Information Provider Coordination Manager Chatting Helper Hint-providing Bot 3.4 Logical Architecture 29

50 Table 3.3: Functional Requirements for Support User Knowledge Database Searcher Conversation Templates Provider HTML Page Annotator Backend Communicator Connection and Messaging Manager is responsible for connection establishment and communication with the chat server. This logical module provides interface to higher level modules to send and receive messages of different types. Roster Manager is responsible for maintaining the roster of support user. It is the roster manager s job to establish the correlation between messages and the conversation partners. This module also maps other data such as context information to its owner in the roster. Context Information Provider encapsulates the functionalities to provide context information from a business user to his chat partner, namely the support user. Some examples of the context information are: business user identity, product configuration, the screen at which the business user experiences a software issue and starts to chat, context information that can be collected by existing incident reporting functions, etc.. Coordination Manager manages web-based on-line coordination operations. Such operations may include screen synchronization and user instruction. 30 Chapter 3 Embedded Support Services Concept and Design

51 Figure 3.3: Logical Architecture Support User Figure 3.4: Logical Architecture Business User Chatting Helper improves the quality of communication in chat by providing support user access to knowledge base according to the current chatting context. This module helps to improve the efficiency of support users by giving him templates for chatting and bots for conversation hints. It uses Hint-providing Bot, Knowledge Database Searcher, and Conversation Templates Provider. HTML Page Annotator is responsible for annotating the HTML page that will be collected as context information. Backend Communicator is responsible for persisting data, either user specific or issue specific, over each chatting session. 3.4 Logical Architecture 31

52 3.5 STRUCTURAL ARCHITECTURE This section introduces the structural architecture of the thesis implementation. It begins with an overview to the XMPP client-server architecture. Since the thesis implementation mainly focuses on the client part, this section will also take a closer look into the components designed in the JavaScript Client XMPP Client-Server Architecture Figure 3.5: XMPP client-server Architecture The XMPP network has many servers and clients, plus the servers are interconnected in a single-hop network[sast09]. Figure 3.5 is derived from the single-hop network, the blue line is the message route when XMPP Client 01 on XMPP Server 1 sends a message to XMPP Client 04 on XMPP Server 2. To implement the functionality for live support integrated into embedded support services for SaaS, the thesis assumes that XMPP servers are built as part of IaaS in the cloud infrastructure; therefore the functionalities of the thesis can be seen as an XMPP client implementation. For development and testing, a free XMPP server called Openfire 1 is deployed, which includes also built-in data bases which persist user information JavaScript XMPP Client Structural Architecture Graph 3.6 depicts the components at XMPP JavaScript client, at XMPP Server and at the backend of the supported SaaS. A layered structure which respects MVC pattern is applied. Each layer is explained in detail in this subsection Chapter 3 Embedded Support Services Concept and Design

53 Figure 3.6: Structural Architecture 3.5 Structural Architecture 33

54 View is a layer responsible for all user interface related operations. Figure 3.7 depicts this composite component. The "DOM Operator" component is responsible for CRUD operations of DOM elements in the HTML5 page. "UI Modifier" component encapsulates logical UI operations, therefore it consumes the "DOM Operator". The separated UI components provide flexibility. Figure 3.7: UI Controller Component Model layer is responsible for data operation and bottom layer chat messaging. Connection and Messaging Manager utilizes the "Strophe" 2 library which provides XMPP client-server communication for JavaScript. The Connection and Messaging Manager is also a composite component depicted by figure 3.8 where "Message Extender" provides functionality to extend message types, so that "Presence and Message Manager" is able to not only send chat content in text but also send and interpret other content types like HTML stream for screen-sharing, coordination parameters and customized messages which trigger certain chat client actions like request of sharing context information or starting incident reporting. Figure 3.8: Connection and Messaging Manager Component Backend Communicator takes care of calling the Application Server to persist and fetch user-specific data such as templates and chat history. This component uses remote communication interface to call back-end function, for example SAP s "Remote Function Call". Controller layer encapsulates the major application logic that covers the functional requirements of the implementation. As a layer sitting between View and Model, the Controller layer components separate UI specific logics and the lower data and messaging models. By such a MVC structure, both extensibility and modifiability are enhanced Chapter 3 Embedded Support Services Concept and Design

55 Context Analyzer is responsible for requesting and collecting context information. The component consumes "Connection and Messaging Manager" for transferring the requests and context data. The actions in the component are triggered by events from the UI. Context analysis operations such as screen-sharing also depend on the "UI Controller". Some of the context information is provided by existing functionalities from the backend, therefore the given component also depends on "Backend communicator" for accessing that context information. Coordination Manager is responsible for web-based real time interaction happening between chat users. This component also depends on UI Controller to render coordination page elements and the "Connection and Messaging Manager" to perform required messaging. Template Manager is responsible for template related operations. Since user specific templates need to be persisted, the given component consumes the "Backend Communicator" to manipulate stored templates. Also, this component depends on the UI to render the templates. Chat/Presence Manager is the basic component for taking user inputs from the UI and render chat responses. The component also manages presence switches and status change handling. Similar to "Template Manager", the given component have dependency on "UI Controller" and "Connection and Messaging Manager". Roster Manager is the component which operate the rosters (buddy lists) of a chat user. The given component requests rosters (buddy lists) and manages member (buddy) specific data such as test message, shared screen, and context data. Similarly, the given component has dependency on "UI Controller" and "Connection and Messaging Manager". Solution Search Controller is the component which helps the support user to find solution documents or workaround descriptions in the knowledge database. Therefore, the given component has dependency on "UI Controller" and "Backend Communicator". Bot is the component which suggests chat content to the support user. Suggestion is triggered on business user input. The given component has dependency on "UI Controller" and "Connection and Messaging Manager". 3.6 PROCESS ARCHITECTURE This section describes the chat system behaviour from the process point of view. The processes selected illustrate the components and the users behaviour including normal text chat, screen-sharing and coordination, requesting additional information, and assisted incident creation. Start up: Figure 3.9 depicts the process of chat from the moment when it is started in the HTML client to the initial screen is created. If the user is a support user, the components which are responsible for user specific templates and roster management are also created. Detailed interaction between the components is shown in the figure. 3.6 Process Architecture 35

56 Figure 3.9: Initial Start-up Process Screen sharing and real time coordination: Figure 3.10 shows an example of message sequences between the support user s and the business user s chat client. To get a shared screen, support user needs to first request sharing from business user. If the business user agrees so, HTML page will be encoded into string and transmitted via chat to the support user. Secondly, the referenced "<image>" elements in business user s HTML pages will be encoded into Base64 strings and also transmitted. When support user client receives a HTML page message or an image message, the page or the image will be displayed. The support user client then registers handlers for double click events of each DOM element in the shared page. The handler will trigger a message sending which contains the nearest parent of the DOM element clicked which has an "ID". Business user client receives this "indication" message and gets the indicated element s ID for displaying. This way, support user can show a page element for coordination easily without verbal description. Support user suggested and assisted incident creation: Figure 3.11 depicts message sequences between the support user s and the business user s chat client when the support user suggests the business user to report the software issue as incident. The request also includes that the content of the incident will be proposed by the support user for sake of description quality and accuracy. As a result, upon the agreement of business user, the chat client initiates an incident editing UI for the support user. When the support user finishes editing, this proposed incident will be transmitted and filled into the UI of incident reporting, also based on business user s consent. Starting incident reporting at business user s side is also end of chatting. 36 Chapter 3 Embedded Support Services Concept and Design

57 Figure 3.10: Screen-sharing and DOM Element Indication 3.7 GRAPHICAL USER INTERFACE DESIGN This section introduces the design of the implementation focusing on the user interface. During the user interface (UI) design, this report implementation adopts the method of user-centered design (UCD) process. The design focuses on the core needs of the two types of users specified in the software requirements. For business users, the UI should be functionally complete, which means business user can react to various action requests required by the support engineer, such as agreement over context data sharing; page annotation; basic chat input and output. The UI for business user should also be very simple. The reason is that live support services are accessed by the business user when a software issue happened and the business user is seeking support, if this services again fails to be easy-to-use(user-friendly), then the overall user experience of the SaaS product will be considerably damaged. Therefore, a simple UI with reactive extensions is designed. 3.7 Graphical User Interface Design 37

58 Figure 3.11: [Incident reporting and content proposal Figure 3.12 is conceptual design of the graphical user interface for the business user. The upper part of the chat window is the display of on-going conversation. The middle part is a control bar which is only filled with controls when the functions are available. The lower part is mainly the input field. The traffic light at the bottom of the chat window shows the presence status of the business user. A submit button is supplied for mobile devices with no physical keyboard as an alternative to the "Enter" key in the virtual software keyboard. Buttons in the output area are triggered in GUI by messages (input request) sent from the support user. The GUI in this case controls access to such functions therefore business user does not need to perform any manual search. The figure also shows extended controls for annotation of HTML page, such extension controls are added to the control bar when the business user is conducting related operations and removed once the operations finish. As a result, the GUI for business user is simplistic and functionally complete. For support users, the GUI design focuses on the tasks of support derived from the software requirements. Figure 3.13 is the conceptual design of the support user s GUI. This GUI consists of mainly 3 parts. The first part is the elements on the left, which include chat partner s (business user s) master information, the shared context information link, the roster of other support users and incoming chat service requests. The middle part of the GUI is a composition of chat and other controls, such as buttons and select lists. The third part of the support user s GUI is a representation of chat partner s screen, which is requested for sharing pro-actively by the support user. The process of UCD is iterative, new features should be integrated to the GUI with minimum efforts even when new requirements arrive. This is easier for business user s case since new 38 Chapter 3 Embedded Support Services Concept and Design

59 Figure 3.12: Business User UI Concept reaction options do not affect the general layout of UI and control bar can be cleared to instantiate new controls very easily. For support users, the UI is adjusted each time new features are implemented. Given the design that the right part of the UI for screen sharing should be viewed in full screen mode, one strategy adopted to cope with this complexity is to re-use and do changes only on the right part of the UI, keeping the left and middle part stable. Figure 3.13: Support User GUI 3.8 SUMMARY This chapter introduced the solution proposal of embedded support services delivered through Internet instant messaging. It explained the benefits and concept of this solution 3.8 Summary 39

60 proposal. Section 3.1 introduced the change of support work flow in case embedded support services of chat is introduced as a solution plan and the benefits this change can bring. Section 3.2 explained the functions of the intended services to its two kinds of service consumers. In section 3.3, software requirements were laid out. In section 3.4, logical structures which were set up to cover the functional requirements were explained. Section 3.5 breaks the system into components. Section 3.6 depicts both examples of components interaction and interaction between chat users in sequence diagrams. Section 3.7 introduces implemented UI design, implementation language and environment. 40 Chapter 3 Embedded Support Services Concept and Design

61 4 IMPLEMENTATION This chapter introduces how each component specified in the previous chapter is implemented. First, selected technologies are talked about, then code examples are given when necessary for key implementations. Diagrams are shown to describe the directory hierarchies and objects relationships. In the end, this chapter is also briefly summarized. 4.1 DEVELOPMENT PLATFORMS The prototype is going to be built in a JavaScript HTML client of a SaaS ERP product. Important development conditions are given in the following subsections Programming Language The thesis implementation is programmed in JavaScript 1, which is the client side script programming language supported by nearly all major web browsers 2. Thesis implementation uses the mandatory name spaces to load JavaScript code and follows conventions in order to be consistent with the general coding style. A JavaScript library is used for implementation: Strophe.js "Strophe.js is an XMPP library for JavaScript. Its primary purpose is to enable web-based, real-time XMPP applications that run in any browser." 3 The thesis use Strophe.js to build and extend messages of different types to communicate with the Server

62 4.1.2 XMPP Server For the thesis implementation, the selected XMPP Server is Openfire. "Openfire is a real time collaboration (RTC) server licensed under the Open Source Apache license." 4 Chat user and group data are stored in the XMPP Server. For the prototype implementation, the XMPP server is deployed on a PC in the SAP Intranet, development environment and the JavaScript client is run on another machine Development Environment The thesis implementation is built in a HTML5 client of an Enterprise Resource Planning SaaS product. Therefore naming conventions and script loading syntax are regulated by a proprietary JavaScript library. Eclipse 5 is the IDE used for the implementation. Git 6 is the distributed version control tool used. Maven 7 is used for project management. Google Chrome 8 is the browser used for development and testing Implementation Baseline (existing feature to be adapted and reused) To enable chatting with context information, it is necessary to consume the context data that is already collected by the embedded support features. Figure 4.1 gives a holistic view of all those context providers. However only some of the existing JavaScript code will be consumed, which will be explained in later sections. The thesis implementation also changes some of the existing features for new features requirements. The added functions are listed in the objects that it belongs to. This existing framework was designed for providing embedded support services in a form visible to the end user as "Reporting an incident". To get a more detailed overview of the existing infrastructure, they are now explained coarsely: DiagnosticsManager is a facade, which handles the phases and the overall context collection. This is the central entry point for embedded support, not only for context collection. ContextCollectionManager is a singleton class which handles the context collection on client side (context collection happens on both server and client sides). Here the context provider can be registered and executed Version m 42 Chapter 4 Implementation

63 ReportIncidentFloorplan is a UI factory that creates a new window that enables user to input incident facts. Context Providers is a package which contains all the "context providers". A context provider collects certain context information, either from frontend or application backend. Examples of context information are: exceptions, images on user screen, values in input fields, etc. Context Collection Phases is a package which contains abstraction of different phases during which context providers are called. The context providers are not executed all together because certain context information can only be collected at certain points in time distributed between client and server. Miscellaneous is an abstract representation of other JavaScript files that provide assisting features for context collection, they are not to be directly consumed by chat. Figure 4.1: Implementation Baseline with Added Functions Packages Overview This part will present how the different functions of the prototype are implemented. Each different aspect of functions is grossly represented by a package which contains the implementation JavaScript files. Overview of the packages is given by figure 4.2. Code in each of the packages will be explained in more detailed manner in later parts of the chapter. 4.1 Development Platforms 43

64 Figure 4.2: Package Diagram of Thesis Implementation 4.2 BUNDLING OF EMBEDDED SUPPORT SERVICES This report suggests two approaches to provide live support services to business users. The first approach is the reactive approach: currently, the embedded support services are accessed by the user from a static position in the GUI, regardless of which product component the user uses or which work center the user is currently working on. By providing an access point at the same place, user can access live support services in conformance with other embedded support services. The second approach is the proactive approach: users who are performing a certain categories of actions are suggested automatically to use live-support service, for example when the user is searching for a solution document too long without rating a helpful result, or when the user is answering a ticket which has already been sent back for detailed information. Though the motivations behind the two approaches are quite different, the implementations for both are rather similar. For conciseness, the implementation in the thesis provides only the reactive approach. The "Get help by on-line chatting" link is the service access point for business user. Through the "Log-in as support chatter" link, support user can use the service. 44 Chapter 4 Implementation

65 4.3 BASIC CHATTING SERVICE: CONNECTION MANAGEMENT, MESSAGE AND CONTACT HANDLING This section introduces implementation of the functionalities regarding basic chatting services including connection management, text messaging and contact(roster) handling Connection Management ConnectionManager.js is a singleton which is responsible for maintaining connection to the "Openfire" XMPP server from the browser, shown in listing 4.1(For conciseness, functions are only shown with their signatures whenever possible). The prototype also specifies default automatic reconnection on server timeout as long as chat is not closed by the user. 1 sap.client.embsup.xtoc.connection.connectionmanager.prototype.connect = function( 2 bisreconnect) 3 sap.client.embsup.xtoc.connection.connectionmanager.prototype.getoffline = function() Listing 4.1: Connect and disconnect functions It is also responsible for dispatching message of different types (extension) to their handlers. Listing 4.2 shows the calling in function "connect" which registers the function "on_indicator" to handle any message of type "xtocindicator", which are those that indicate an HTML element to highlight sent by the support user: 1 that.connection.addhandler(that.on_indicator, null, "message","xtocindicator"); Listing 4.2: Registering handlers for messages To have a holistic overview on the implemented message extensions and their handlers, a list of message and handler mapping is provided in table Contact/Roster Management There is no need to let a business user see his roster. But for support user, it is better to encapsulate data like chat history and user master information. The "ContactsManager" is a singleton which maintains lists including the support users and the business users that are taken over in chat by the support user. In JavaScript, there is no formal definition of a class, therefore an object diagram figure 4.3 is used to represent the relationship of the related JavaScript objects in runtime, at a moment when there are 2 contact objects created. The creation of contact objects are triggered by function "on_roster", is explained in the Message Extension subsection. Major functions of the objects in figure 4.3 are explained in detail: ConnectionManager A singleton object which requests roster and registers handlers for roster entry stanzas sent back from the server. 4.3 Basic Chatting Service: Connection Management, Message and Contact Handling 45

66 Table 4.1: Stanza Extensions and Handlers Message Type Handler Name Handler Explanation chat on_message show plain text chat presence on_presence react on presence changes xtocincidentpreparerequest on_request dispatch to function to request business user if to allow incident preparation xtocsskeyrequestmessage on_request dispatch to function to request business user if to allow send solution search keys from system messages xtocnewsusuggest on_request dispatch to function to request business user if to accept a chat transfer xtocsskeyrequestexception on_request dispatch to function to request business user if to allow send solution search keys from system exception xtocindicator on_indicator add highlight effects to indicated HTML elements xtocremotecontrol on_remotecontrol trigger click event in business user s UI when an element is clicked in the remote control mode by support user xtociframeimage on_iframeimage replace image URL with Base64 strings, show images xtocincidentpreparerequest on_request ask business user if support user can suggest incident inputs for him/her xtocincidentrequestagree on_preprqtagr start incident preparation xtocscreenrequest on_request request a screen sharing from business user xtoccontextcollectionlink Request on_request request business user to render a link to holistic context information collection xtocrequestreject on_reject show user rejection xtocnewsusuggestreject on_reject show user rejection of chat transfer xtocincidentfact on_incidentfact stop chat and start incident reporting xtocpcirequestagree on_pciagree notice support user that business user has started annotating the screen and preparing context information link xtocnewsusuggestaccept on_transferagree notice support user that business user accepts chat transfer xtocjsstcancel on_jsstcancel notice support user that business user cancelled sending context information link xtocjsstsent on_jsstsent notice support user that business user send context information link with annotated HTML page xtocreportincidentchatover on_chatover notice support user that business user stopped chatting and begins incident reporting xtocbrwoser on_browser request browser type and version from business user xtocssmsgkeys on_msgkeys display and use solution search suggested keys xtochtmlstream on_htmlstream display HTML page in an Iframe xtoctransferdetails on_transferdetails display chat history and context collection link to the next responsible support user in case chat is transferred 46 Chapter 4 Implementation

67 StanzaExtender A factory singleton which instantiates extended message types. Extensions specified are shown in the next subsection in detail. ConnectionParams A singleton which supplies connection parameters like server address, port number, initial user name(jid) and initial chat partner name(jid). ContactsManager A singleton which maintains contacts in different lists and manages rosters. Contact Each roster member is represented in the JavaScript chat client as a Contact object. It encapsulates data like name and contact information, user type and references to UI elements belonging to the contact such as sub-window for multi-chat. Message Objects which contain the content, the associated contact, and time stamp of a text chat message. Figure 4.3: Object diagram depicting ConnectionManager Message Extension 1 <message to='michael@xtoc.sap' 2 type='chat' 3 xmlns='jabber:client'> 4 <body> 5 Hallo World 6 </body> 7 </message> Listing 4.3: A normal message stanza In XMPP protocol, the data that is exchanged is in XML. There are three kinds of XML elements defined in the default XMPP namespace, each of which is called a 4.3 Basic Chatting Service: Connection Management, Message and Contact Handling 47

68 "Stanza"(<message/>, <presence/>, or <iq/> elements 9 ). "XMPP stanzas make up the core part of the protocol"[mof10]. To realize features that are beyond text chatting, extending the data format is necessary. Listing 4.3 shows a default stanza in XML which is exchanged to "michael@xtoc.sap" to the server the "type" attribute specifies that this message stanza is a simple text chat message. The extensions to XML data can be either made on attribute level or on element level. The prototype defines the following function shown in listing 4.4 in the "StanzaExtender" to extend XML on both levels: 1 sap.client.embsup.xtoc.connection.stanzaextender.prototype.extendmessagestanza = function( 2 snewtag, stextvalue, stype, omsg) Listing 4.4: Extending Message Stanza With this function, it is possible to define stanza for special purposes. For example it is possible to create a "prepare incident request" to the business user by calling 1 var omsg = sap.client.embsup.xtoc.connection.stanzaextender 2.getInstance().extendMessageStanza("Request", 3 "Y/N", "xtocincidentpreparerequest"); A message which has the type attribute value "xtocincidentpreparerequest" is then created, and it contains a child element "Request", whose value is "Y/N". Listing 4.5 shows the produced message in detail. The business user side code can determine how it handles this request flexibly. 1 <message xmlns='jabber:client' 2 to='wwq@xtoc.sap' 3 type='xtocincidentpreparerequest' 4 from='michael@xtoc.sap/495eec99'> 5 <Request> 6 Y/N 7 </Request> 8 </message> Listing 4.5: An extended message stanza 4.4 USER INTERFACE Figure 4.4 shows two links to access chat service in the help center in an Enterprise Resource Planning SaaS product. Business user can click on the link "Get help by on-line chatting" to start chatting. The implemented business user GUI is shown in figure group 4.5. Figure 4.6 shows the support user s GUI, implemented according to the design which is described subsection According to XMPP standard( the definition of XML Stanza is an XML stanza is a discrete semantic unit of structured information that is sent from one entity to another over an XML stream. An XML stanza exists at the direct child level of the root <stream/> element and is said to be well-balanced if it matches the production content of [XML]" 48 Chapter 4 Implementation

69 Figure 4.4: Integrated Live Support in Embedded Support Services 4.5 WEB-BASED COLLABORATION SERVICE FOR EMBEDDED SUPPORT Three major features are implemented for web-based collaboration between support user and business user. The support user can require HTML screen sharing, highlight a selected DOM element and refresh UI specific templates by requesting a unique identifier for current UI for current business user. The three features are explained separately in this section Screen Sharing through XMPP Instead of taking and transferring an image format picture such as PNG, the prototype transfers the target HTML page together with CSS and displays the HTML page in an "<iframe>" tag in the support user s UI. This approach lowers the programming efforts and also brings advantages, for example text on initial page can be copied in the transmitted page. This mentioned step is done with the extended message stanza of type "xtochtmlstream". The next step is to replace the entire source URLs of "<image>" tags to correctly display the images on the target page. This step is done with the extended message stanza of type "xtociframeimage". The function which provide the original HTML stream and the function which extracts referenced image into Base64 strings are existing features which locate in the "embsup/contextproviders" package, the thesis reuses many of the context providers as data sources. However the results returned by those "context providers" are not directly consumable by chat client, as they are initially designed to transfer JSON ( (JavaScript Object Notation) 10 data Web-based Collaboration Service for Embedded Support 49

70 (a) Business User GUI(without additional controls) (b) Business User GUI(with additional controls) Figure 4.5: Implemented Business User GUI to the ABAP backend. Therefore in controller objects which call the context providers, adaptation is performed and results are used to initiate specialized extended message stanzas. Figure 4.7 depicts how the context provider facade "ContextInformationProvider" consumes these existing supportability functionalities. On one hand, it creates directly context providers like "HtmlStreamProvider", "ContextImageProvider", and "MessagesProvider" because these context information can be collected lazily, which means the context information is either durably available for collection, or the context information is buffered for collection. On the other hand, "ContextInformationProvider" also calls "ContextCollectionManager" to get a list of context providers which are only created under certain situations, such as "ExceptionContextProvider". "ExceptionContextProvider" is instantiated only when unhandled exceptions occur and these are displayed to the user. As a result, "ContextInformationProvider" needs to store these transient context information for later consumption. 50 Chapter 4 Implementation

71 Figure 4.6: Support User GUI Figure 4.7: Facade for Context Collection 4.5 Web-based Collaboration Service for Embedded Support 51

72 52 Chapter 4 Implementation (a) "Session Time Out" exception window (b) Shared "Session Time Out" Exception Window (c) UI with Donut Chart (d) Shared Screen for UI with Donut Chart Figure 4.8: Screen Sharing Results Examples

73 Figure 3.10 shows the sequence diagram of this process. Figure group 4.8 shows the effects of screen sharing on randomly picked UI in the cloud product. Figure 4.8b and figure 4.8d are the support user s view on the transferred page, figure 4.8a and figure 4.8c are the original page where business user starts chat on and transferred to the support user. The DOM elements that form chat window are removed in the transferred HTML stream so that in figure 4.8b and figure 4.8d, the business user chat window is not shown. An interesting point is that the scroll bars available on the shared HTML page are also available for scrolling, since the whole page is transmitted DOM Elements Highlighting through XMPP It is necessary to let support user indicate a screen area in order to quickly and conveniently instruct actions to the business user. When HTML stream is used for re-constructing the business user page, there is loss of accuracy due to difference of browsers or screen settings. It is therefore not feasible to directly transfer the coordinates of the indicated UI elements. These differences can be noticed also in figure group 4.8, the scroll bars have different size and look different for example. A special message stanza is extended for the purpose of accurate indication of DOM elements. Figure 4.9: State Chart: find indicated UI element After the screen sharing happens, support user would have a copy of the page displayed in an "<iframe>" tag. Figure 4.9 shows the recursive approach how element is selected for the indication. On each of the DOM elements, double click event handlers are registered. In the handler, it is checked if the DOM element has an ID recursively along the event bubbling sequence. Then the ID of the clicked element or in case it does not have an ID, its nearest parent node is sent to the business user. There the ID is extracted to find its owner element. 4.5 Web-based Collaboration Service for Embedded Support 53

74 And special visual effects are added to the indicated element. In this way, regardless of the inaccuracy of screen-sharing, the DOM elements indication is always targeting at the element of which the ID matches. Results of the double clicked elements highlighting can be seen in figure group 4.10, where highlighting effects on different HTML tags are shown. The rectangles which indicate the highlighted elements are given a "blinking" effect, whose color is switching rapidly between yellow and red. (a) Highlighted "Work center view"(yellow) (b) Highlighted input field(red) (c) Highlighted Table Header(yellow) (d) Highlighted Link(red) Figure 4.10: Screen Sharing Results Examples Dynamic Templates Fetching To use communication templates effectively, the templates should be context specific. To simplify the problem, templates can be UI specific. Therefore fetching the UI specific templates in chat can be broken into two steps. First step is getting the keys of the UI; second step using the keys to query the templates from database. Since the second step involves mostly backend functions that have to be implemented in the server and there is currently no business data (templates) available. It is more practical and also technically meaningful to simulate the second step by using experimental templates in the JavaScript chat client. For the same reason, the implemented "BackendCommunicator" contains only functions with mock-up data instead of functionalities that fully consumes server side services. In the environment UI framework of the SaaS product. UI components are identified in a layered structure of different granularities. The "Work Center" is the highest level, under which 54 Chapter 4 Implementation

75 exist multiple different "Work Center Views"; under each view the "CurrentWindowID" uniquely identifies the current UI component the business user is working on. The thesis implementation extracts all three of the IDs and let support user choose which to use for fetching templates. Figure 4.11 shows the results of UI key extracting in a window that enables support user to use the captured UI component keys to search for related templates. Figure 4.11: Captured UI Component Keys 4.6 ACCESS TO HOLISTIC CONTEXT INFORMATION SERVICE Though the thesis implementation empowers chatting with many important context information that can be collected by pre-defined context providers. There is still much context information that is relevant for deeper analysis. To fully leverage the existing context collecting infrastructure in the SaaS product, it is implemented that support user can require business user to trigger a holistic context collection and the context is persisted on the server, therefore it is available for immediate and later analysis Link to holistic collection of context information Support user can view all possible context information with support tools in a structured way with this link to the holistic context information collection. Figure 4.1 shows the modified "DiagnosticsManager" has a new function which is called "createincidentsilentlywithjsst". This function implements the triggering of the above mentioned context collection HTML Page Annotation in IIM for Embedded Support To make it easier for business user to describe a software problem, especially those that are related with UI, the thesis implementation lets him annotate the HTML page. An existing HTML page annotation tool "JSSTManager" is reused for the purpose. The annotated page is later transferred into the dummy incident created, which is mentioned in the previous subsection. There is an adaptation that is done in the chat client in order to use the existing "JSSTManager". The reason is "JSSTManager" copies the entire HTML page contents into an <iframe>, and annotations are done on top of the <iframe>. So that a copy of the original page content can be modified in the <iframe> without worrying about destroying the original page. This process would also copy the current chat window, which is not desired. Therefore there are two alternatives to solve this issue: one is to temporarily remove the chat window; the other 4.6 Access to Holistic Context Information Service 55

76 Figure 4.12: Adaptation to JavaScript Screen-shot Tool (JSST) being temporarily set the "display" attribute to "none". For sake of minimal size of the resulted transferred HTML stream, the thesis implementation temporarily remove chat window till the entire page is copied and the <iframe> is rendered. Figure 4.12 shows this process as concept. Figure 4.13 shows the integrated page annotation in chat client. Figure 4.13: HTML page annotation before holistic collection of context information 4.7 HUMAN IT-SUPPORTED SUPPORT SERVICE When support user tries to resolve the business user s software issue, in many cases the support user has to refer to the internal knowledge base. For the thesis implementation s product, access to two of the search engines is provided for the support user directly in the chat 56 Chapter 4 Implementation

77 client. According to the context information support user can get from the thesis implementation, he can already search for many key words (search terms). Furthermore, it is implemented that the chat client is able to automatically extract some search terms. The thesis implementation uses two types of search terms, namely the exception information and the system message information. Figure 4.14 shows how the two types of context information can help support user quickly find existing solution documents in order to reply to business user efficiently and effectively. In figure 4.14 the business user chat client extracts the error messages and the exception information (hash keys) by calling the responsible context providers, shown in figure 4.7. The extracted information will be transferred to support user. On the support user s client, the information will be displayed in a structural way thus support user can choose what to use/combine to have the best search terms. Figure 4.14 also shows the association between the internal tickets and software issues. If the business user s software issue (software issue 3 in figure 4.14 ) has the same error messages or same exception as an already solved software issue (software issue 3 in figure 4.14 ), then it is highly possible that the solution or workaround specified in the ticket (internal ticket 3 in figure 4.14 ) which addresses this already archived issue also applies to the newly encountered software issue. Additionally, by using exception hash key or error message text, the ticket will be searched and found. Figure 4.14: Captured UI Component Keys The thesis implementation uses an <iframe> to integrate the search engine for the last step of using automatic extracted search terms, an example where business user s exception information being transferred for solution search is shown in figure group 4.15, where figure 4.15a shows the exception happened in business user s application and figure 4.15b shows the extracted hash keys and the integrated search engines. In this way, an "Human IT-supported Support Service" is implemented. 4.7 Human IT-supported Support Service 57

78 (a) Business user experiences an exception (b) Support user searches for solution documents with extracted exception hash key ) Figure 4.15: Solution search with automatic search terms extraction 58 Chapter 4 Implementation

79 4.8 BUSINESS USER TRANSFER SERVICE When a support user discovers that the business user encounters an issue which the support user cannot solve, a business user transfer service is provided so that the support user can assign the chat session to his on-line colleagues. Together with the chat transfer, the message history and the latest link to holistic context information collection is also transferred to the new support user to avoid redundant repeating conversations. Figure 4.16: Business User Transfer Service UI Figure 4.16 shows the UI where the support user just received an chat transfer from his colleague "Michael Schultze", before any conversation with the transferred business user "Business-User", he can already see the history chat conversation and access the link to holistic context information without asking the business user for that again. 4.9 SUPPORT USER ASSISTED TICKET CREATION SERVICE To improve the ticket handling process, it is important that the ticket is created with good description. When the end users (business users) cannot always deliver this, it is better to let a 4.8 Business User Transfer Service 59

80 support user who has already chatted with the business user to create a ticket for the business user. This support user assisted ticket creation service is especially important given the fact that not all software issues can be solved during a chat session, for example a newly discovered software bug. (a) Support User suggests ticket contents (b) Business User starts ticket reporting Figure 4.17: Support User Assisted Ticket Creation Service The this implementation provides such a service by rendering a UI shown in figure 4.17a to the support user where he can input subject, description, and also priority for the issue. A chat history view field is supplied for support user to review the chat session. These inputs are then sent to the business user. Upon agreement, the business user can use the suggest contents to report an incident, shown in figure 4.17b, note that the suggested contents including subject, description and priority are pre-filled into their fields. Business user is free to modify these suggested inputs COLLECTION OF CHAT HISTORY AS CONTEXT INFORMATION SERVICE In case a ticket is created after using chat services, the chat history should be persisted in the ticket because it may contain useful information. Therefore the thesis prototype implemented a service for collection of chat history as context information and defined the class "ChatHistoryCntProvider", in which chat messages are stored and upon ticket creation, sent as 60 Chapter 4 Implementation

81 part of the ticket. Persisted chat history is stored as XML data in the backend. Listing 4.6 shows an example from the loaded ticket, only two "record" elements are kept to be concise. 1 <?xml version="1.0"?> 2 <spr:contentcollection xmlns:spr=" 3 <spr:content ContextId="A97BC22D7E019842C AFC0BC4" Sanity="OK" Size="1317"> 4 <ChatRecords> 5 <Record> 6 <from>michael@xtoc.sap/725832d9</from> 7 <to>wwq@xtoc.sap</to> 8 <content>do you want to let me access your browser information?</content> 9 <time>10:46:04 AM</time> 10 </Record> 11 <Record> 12 <from>wwq@xtoc.sap</from> 13 <to>michael@xtoc.sap</to> 14 <content>hi</content> 15 <time>10:52:50 AM</time> 16 </Record> </ChatRecords> 20 </spr:content> 21 </spr:contentcollection> Listing 4.6: Persisted Chat History 4.11 AUTOMATIC CHAT ASSISTANCE SERVICE The purpose of implementing a chat bot 11 is to provide a service to the support user to quickly respond to a business user in chat. Since user experience is a major concern for the thesis implementation, bot is only used to propose response to the support user instead of directly "talking" to the business user. The designed chat bot can be seen from figure The thesis only implements the conversation interface of the chat bot and uses simple template questions to simulate a response generated by modules which processes business user inputs and query for answers. For the conversation interface part, the thesis implementation falls into the category of "Responder Bots" [GXWW11], which means that the bot s output are triggered by certain or every human user input. Figure 4.19a shows the chat bot s suggestion upon an business user (named "Thomas Mueller") input. Figure 4.19b shows that support user accepts the suggestion so that the text is automatically copied for further editing. 11 Bots are "automated programs, that is, programs that do not require a human operator. A chat bot is a program that interacts with a chat service to automate tasks for a human"[gxww11] 4.11 Automatic Chat Assistance Service 61

82 Figure 4.18: Design of Bot for Automatic Chat Assistance In this way, a service which provides automatic chat assistance to the support user is implemented, the objective of quickly replying to multiple business user is further realized SUMMARY The given chapter explained the thesis implementation from its lower level XMPP protocol stanza extensions and dependencies on existing infrastructure to examples of implementation of the context-information-rich chat. Diagrams are given both to demonstrate the working principles of the implementation and to show the results of some implementation details. Some lines of key codes are also briefly listed in order for the user to grasp the details. section 4.1 explained the development environment of the prototype implementation, declared the technologies, and depicted system dependencies. From a service point of view, section 4.2 showed how chat support service is bundled into the environment SaaS; section 4.3 showed how chatting service is implemented with messaging basics and roster handling; 62 Chapter 4 Implementation

83 (a) Bot suggests reply to the support user (b) Support user accepts the suggestion by clicking on the blue area. Suggested content is then copied to the input area. Figure 4.19: Automatic chat assistance section 4.5 showed how web based-collaboration as a service is implemented, including services like screen sharing, DOM elements highlight, and dynamic templates fetching; section 4.6 showed how a service to access holistic collection of all pre-defined context information is designed and realized through chat with a link to the context information and integration of HTML page annotation functionality; section 4.7 shows the implementation of solution search in chat which delivers the human IT-supported support service; section 4.8 shows the implementation of business user transfer service, which helps the support user to find the right colleague for the given software issue. section 4.9 shows the implementation of support user assisted ticket creation service, which would improve the ticket quality before tickets enter the handling process. section 4.10 shows the implementation of collection of chat history as context information service, which persists the chat messages once software issue is not solved and a ticket is created. section 4.11 showed the implementation of a bot interface and the design of high level bot modules, which together renders service of automatic chat assistance Summary 63

84 64 Chapter 4 Implementation

85 5 EVALUATION 5.1 EVALUATION CRITERIA In chapter 2 "Goal Setting", the general goal was defined to cut operational cost at SaaS vendor s side and to improve customer satisfaction. The following functional expectations were set for this thesis: live chat for fast first response, thus for better customer satisfaction templates for efficient communication on-line diagnose with live chat live chat - solution search integration Chat assistance For each of the goals, several services are implemented. As a recap, they are listed briefly now in table 5.1. Table 5.1: Goals Features Mapping Goals live chat for fast first response templates for efficient communication on-line diagnose with live chat live chat - solution search integration Chat assistance Main features implemented Multi-user text chat; Three types of templates(general Template; UIspecific templates; My Templates) Link to holistic context data collection support user-assisted incident creation Integration with two major support search engines Automatic search term extraction Bot interface to support user; 65

86 To evaluate if the thesis implementation fulfills the general goal, it would require deployment of the prototype in productive business systems for a long term assessment with real customer interaction. It is not feasible to fully simulate this in the manageable time scope of the thesis. Therefore the effects of the implemented thesis prototype are measured in interview questionnaire, answered by support and development employees in SAP s cloud organizations. The different types of context information which are made available for real-time consumption in the thesis prototype are based on XMPP protocol. To measure the performance of XMPP protocol transmitting the context information, especially larger data like big CSS(cascading style sheet) strings and images in Base64 formats, several features are measured on their accuracy and timing performance such as screen-sharing based on HTML. Positive and negative values for the measured attributes are given in the questionnaire in different degrees, therefore the evaluation has a clear criteria if the impacts of the prototype are contributions or not, and if yes, how much. 5.2 EVALUATION DESIGN This section will introduce why and how the evaluation is designed in its way in two parts, namely the test evaluation and the questionnaire evaluation. Questionnaire distributed can be found in appendix B, only the contents which illustrate the reason and design of the questionnaire will be discussed in the related subsection (subsection 5.2.2) Design of Tests Goals of the tests are to measure concrete performance of the thesis implementation in different dimensions of features in detail, these measurement objectives are: Screen-sharing Accuracy Screen-sharing based on HTML reconstructs the business user s page in the support user s screen. Different to screen-sharing based on screenshots, this approach can show support user each DOM element and their values without losing quality by image encoding. So texts on the page can be copied, links can be examined, IDs of the DOM elements can be retrieved. To measure the screen-sharing s accuracy, it is vital to test if all HTML tags which are relevant for display are correctly copied and reconstructed. For evaluation, these meta data are collected from both business user and support user by JavaScript code instrumentation. A comparison between the shared HTML page and the displayed shared screen is done in this chapter. Original data can be found in appendix C. XMPP Screen-sharing Performance 66 Chapter 5 Evaluation

87 Among all the context information that is structurally displayable in the chat prototype, the shared screen consumes by far the most band-width. Therefore by measuring the performance of screen-sharing, the performance aspect of the prototype can be reasonably estimated. This measurement objective is reached through several key indicators, namely the "HTML Page Collection Time", the "HTML Images String Collection Time", the "Page Display Time", and the "HTML Page Transfer Time". Screen-sharing Performance compared with Stand-alone Screen-sharing Tools The screen-sharing functionality of the thesis prototype is based on HTML transmission. It is necessary to compare the end-to-end latency of this approach with general approach which is based on screen-shot(image format) data transmission. This report selects two popular tools in the market to compare this performance, namely the Citrix GoTo Assist 1 and the Adobe Connect 2. For the Adobe and Citrix products tests, videos which are SAP Business ByDesign live demos are played in full screen in the shared screen to simulate product usage scenario in practice, a timer page is opened on top of the video so that screen shots can be made containing both the shared screen and the displayed shared screen with timers. By subtracting the two time stamps, latency can be calculated Design of Questionnaire The goal of the evaluation questionnaire is to measure how the thesis implementation realizes the functional requirements the potential of the prototype suggested embedded support services, their use cases and estimated contributions So the questionnaire is distributed to voluntary respondents from the SAP cloud organizations. The questionnaire should ask the following general categories of questions: What chat services are important for different stages in ticket handling? Does the evaluated solution implement these services to be helpful? How well does the evaluated prototype compare to other means of live support and other chat products? How will the evaluated prototype influence the workloads of support organization and SaaS provider overall? How will the evaluated prototype influence user satisfaction? Does the evaluated prototype show promising use cases for support? Evaluation Design 67

88 Respondents are required to specify their job responsibilities and the average number of weekly software problem reports (tickets) they process. The questionnaire then focuses on comparison between the respondents expectation of chat tool features that are helpful for support and the realized features of the thesis implementation. Questions are also given to see how often respondents feel the need to directly contact the software issue reporter. After which the respondents are asked to answer in which senses the implemented prototype can be better or complementary to a phone call. Respondents then face the question about what are the advantages and disadvantages of the evaluated prototype compared to existing chat solution based on HTML. In the end, respondents should tell their estimation of the evaluated prototype s influence on cost of SaaS provider and user satisfaction Testing Environment For test evaluation, the test environment is given: Hardware environment is listed in table 5.2 Table 5.2: Hardware Environment PC Model CPU RAM Lenovo Thankpad T510 Intel Core i5 (2.53GHz, 2.53GHz) 4,00GB Software environment is listed in table 5.3 Table 5.3: Software Environment Operating System Browser MS Windows 7 Enterprise, 64-bit Google Chrome (Version m) Thesis Prototype Testing Set-up Figure 5.1: Test Set-up 68 Chapter 5 Evaluation

89 Figure 5.1 depicts the test set-up used in evaluation. It is shown that chat clients are run on the same Laptop in different browser taps. XMPP server is installed on a remote server. Test clients and server are connected by the internet. 5.3 RESULTS FROM QUESTIONNAIRE After live-demo to the respondents, they are asked to fill in the questionnaire (given in appendix B). Missing results to single questions are not included in the calculation and analysis in this chapter, however the sent back questionnaires with incomplete answers are not excluded. In total eleven questionnaires were returned Respondents Job Distribution Figure 5.2 depicts the distribution of job responsibilities among the interviewed respondents. It shows that the questionnaire interview covers employees involved in various phases of software issue handling processes including support and development. Respondents also include support managers, and employees from other cross-over organisations that are stake holders to software issue handling for SAP s cloud offerings. Figure 5.2: Respondents Job Responsibilities Some of the organizations are specific to SAP, therefore not introduced in previous chapters, now they are described briefly: Support Management Respondents in this group are managers for employees in Expert support and direct support, they are also experienced employees with support professions. Support Innovations Respondents in this group define support processes which are delivered to the customers from the support organization. Quality Management Respondents in this group set quality KPIs for products, define and measure test strategies, manage test systems. 5.3 Results from questionnaire 69

90 Knowledge Management Respondents in this group write product documentations, make roll-out materials and manage terminologies used Assessment of Features Coverage Figure 5.3 shows the average importance of chat features wanted by the respondents in order to understand and solve software issue in chat separately. It shows that all the listed features have gained a rated importance over 3, where 5 is the highest importance (5 is labelled as "Very Important" in the questionnaire, whereas 1 labelled as "Not important"). Since only one respondent mentioned "Chat History Persistence" in the questionnaire and rated it only important for understanding software issue, this measured feature does not have importance for issue solving (value is zero). Figure 5.3: importance of chat features Assessment of Provided Context Information Quality To rate the quality of the prototype services implementation, respondents are asked to give their ratings of quality of each implemented feature of the prototype(s) (5, which stands for "Very Good" is highest possible score; 1 is lowest for "Very Poor") and the their best expected quality of such feature in any possible chat tool(e) (5, which stands for "Very Good" is highest possible score; 1 is lowest for "Very Poor"). Therefore, the thesis calculates the score by dividing the expected quality with the rated quality, from which the average value(score ) can be calculated using equation 5.1 out of respondents replies, given there were (n) respondents. Results are shown in figure 5.4. Those answers are higher than 1.0 were treated as 1.0 in the results, which happen when thesis implementation quality exceeds the expectation of the respondents. 70 Chapter 5 Evaluation

91 . Score = Σ( S E ) n (5.1) Figure 5.4: Estimation of Ticket Avoidance From figure 5.4, it can be seen that all of the evaluated features reach a score over 0,85 from the respondents point of view. Solution Search and bot for support user have relatively lower scores. To explain this, the reason why bot gets a lower score is probably its lack of intelligence. Some respondents gave also the feedback that the solution search can be better integrated if the searching step is automatically performed Estimated Influence on Tickets Given the assumption that on average tickets that are solved more quickly have higher quality (more complete/accurate description, more complete problem reproduction steps, etc) than those with lower quality. One question in the questionnaire is given to let respondents estimate influence on ticket resolving time if there would be enough well-trained support users and the prototype would be productized. Figure 5.5a shows this estimation from respondents after prototype demo. All answering respondents claimed that tickets will be solved faster in a previous question. The result shows that 30% of respondents believe that the prototype solution, once productized, can contribute to a reduction of ticket resolving time to 51% to 70%, 5.3 Results from questionnaire 71

92 (a) Estimation of Future Ticket Resolving Time (b) Estimation of Ticket Avoidance Figure 5.5: Estimated Influence on Tickets another 30% claimed a reduction of ticket resolving time to 31% to 50%. The other respondents were even more optimistic regarding this. Given the assumption that there would be enough well-trained support users and the prototype would be productized, figure 5.5b shows the estimation of tickets that can be avoided. It shows that half of respondents expect a 31% to 50% reduction in ticket number. 30% of all respondents expect a reduction of 11% to 30% Estimated Influence on Customer Satisfaction Figure 5.6 shows the respondents estimation of the chat tool s influence on the customer satisfaction. All respondents give positive estimations, 36% of them estimated a full customer satisfaction influence "+5". 64% of all respondents estimated satisfaction influence of "+3". Figure 5.6: Vote for influence on customer satisfaction 72 Chapter 5 Evaluation

93 5.3.6 Estimated Influence on SaaS Provider Cost Respondents are asked in the questionnaire to give their estimates of the effects of chat tool on cost, of support departments alone (shown in figure 5.7a) and of the whole SaaS provider organization (shown in figure 5.7b). (a) Estimation of New Support Workload (b) Estimation of New SaaS Provider Workload Figure 5.7: Influence on Workload About estimation of new support workload, 55% of all respondents answered that support load would be less. 18% of all respondents chose 91% to 110% of original work load; The other 27% of all respondents think that support load would increase to 111% to 150%. The respondents answers are not very unified regarding this estimation. For the workload of entire SaaS organization, 60% of all respondents estimated reduction instead of increase. 30% of all respondents claim that work load will reduce more than 50%. 10% of all respondents say workload will remain more or less the same (91% to 110% of original). The other 30% of all respondents answered that work load for whole SaaS organization will increase to 111% to 130%. Consensus opinion is not reached regarding this estimation Expectation of Support User created Ticket Respondents are asked how many tickets should be created by support user after using chat. The answers are shown in figure 5.8. Only 12% of respondents answered that they want less than 10% of all tickets to be created by a support user. Half of all respondents said this percentage should be 11% to 30%. A quarter of all respondents said this should be over 50%. In general, respondents expect that the support user inputs incorporated by the prototype can lead to better ticket quality. 5.3 Results from questionnaire 73

94 Figure 5.8: Percentage of Tickets that Should Be Created by Support User After Chat 5.4 RESULTS FROM TESTS This section introduces the results from evaluation. Original data collected in table form can be found in appendix C. In each result subsection, brief analysis of the results will also be given for further illustration XMPP Screen-sharing Accuracy Figure 5.9: Comparison of HTML tags and their appearances To evaluate the accuracy, five groups of data are collected regarding HTML tags at random UI in the tested product. Figure 5.9 answers the question what tags exist in the shared page and how many times these tags appear and these of the displayed screen. From this figure, it is shown that numbers of "<script>" and "<link>" tags are different in the shared screen. Since tags like "<meta>", "<title>" and "<script>" do not affect display, tags like "<link>" and "<style>" are merged during HTML stream collection, they are excluded from the comparison. Figure 5.10 shows the mismatches between the other tags in the first group (HTML page) of collected data. 74 Chapter 5 Evaluation

95 Figure 5.10: Difference of HTML tags appearances (Group 1) It is shown from this result in figure 5.10 that the screen-sharing is accurate regarding number of HTML elements. Tests from other four groups showed the same result of no difference regarding number of rendered HTML tags, so the diagrams for them are omitted in this report to be concise XMPP Screen-sharing Performance Figure 5.11: Screen-sharing Performance measured by milliseconds To evaluate the screen-sharing performance, this report chooses five random UI and starts screen-sharing from each of these UIs. In order to exclude network congestion spikes, for each UI the screen-sharing is repeated for five times. Therefore 25 groups of data are collected. From the results, it is shown that in most of the cases (24 out of 25), context collection of the HTML page and HTML images takes less than 100 milliseconds. In figure 5.11, all of the 25 groups of data are depicted. It shows that the transfer time on the network is the most time-consuming among the four measured activities. In the test environment HTML page transfer takes on average milliseconds. 5.4 Results from tests 75

96 Figure 5.12: Screen-sharing Performance relative to network transfer time Figure 5.12 maps the consumed time of the other three activities to the consumed time of page transfer, there it is shown that HTML image collection and HTML page collection have curve that match each other s changing tendency. Display algorithm also takes much more time than collection of the context information. In the test environment, display of the shared HTML page takes on average 358,16 milliseconds Screen-sharing Performance compared with Citrix and Adobe products Data collected for the evaluation can be found in appendix C. Figure 5.13 shows the results of screen-sharing latency of randomly selected screens. Twenty five random samples are taken for each product. This histogram shows that Adobe Connect performs the best during the test, whose 7 out of 25 latencies takes milliseconds. The average latency of Adobe and Citrix solutions are ms and ms, the thesis prototype has an average latency of ms, which are grossly three times of the Adobe latency and two times of the Adobe latency. Given that the average payload of each of the thesis shared screen is 723.2KB, which can be compressed to "ZIP" format and reaching a size of around 70KB and the results from which shows that network transmission is the dominating time consuming part, it can be reasonably assumed that with proper data compression in the future, the thesis prototype has the potential to reach better than average performance competing against these tools regarding screen-sharing speed. 76 Chapter 5 Evaluation

97 Figure 5.13: Screen-sharing Performance compared with Citrix GoTo Assist and Adobe Connect 5.5 SUMMARY This chapter gave an evaluation to the presented thesis software prototype. Section 5.1 made a recap to the thesis goals and set the criteria of the evaluation. Section 5.2 described the reasons why and how the questionnaire interview and the tests are conducted in such a way from practical point of view. Section 5.3 showed the results collected from questionnaire respondents and summarized these data. Section 5.4 delivered tests results of the prototype regarding quality and performance, comparison results with existing tools were given. The chapter showed the assessment from user point of view and revealed strengths and short comings of the thesis implementation. 5.5 Summary 77

98 78 Chapter 5 Evaluation

99 6 CONCLUSION AND FUTURE WORK 6.1 CONCLUSION The presented work aimed at improving the software issue handling process in SaaS organizations by providing additional embedded support services. Findings from researches motivated a way to reach this goal by enforcing or improving missing and low quality context information in the tickets created in the traditional approach and by resolving software issues before they enter ticket systems. Results from the field study show that using Internet instant messaging to provide software issue context information is rare in practice but has a positive effect on customer satisfaction regarding software issues. However the studied SaaS product which uses this approach lacks the focus on collection and consuming of the context information. The thesis proposed an approach to leverage Internet instant messaging to bridge the gap. On one hand, the proposed approach is a horizontal extension of current ticket system based services. On the other hand, the proposed approach helps to improve ticket quality in the preliminary ticket creation phase of the traditional approach. Built in HTML client of SAP s SaaS ERP frontend, two major types of services of the intended work were accomplished: the services that enable multi-user chat between support user and business user, and the services that are context information centric which assist the software issue handling. Chapter 1 introduced the background information of the thesis targeted problems and gave the motivation to address thesis objectives. Chapter 2 reviewed literature in researches that address the thesis targeted problems and investigated adopted industry solutions. From the research study a gap between the 79

100 existing software support offering and the necessary support service of goodquality was found. In the field study conducted in both general software markets and specifically SaaS ERP markets showed that Internet instant messaging was used as a delivery channel of support services, however with limited functionalities with a very low popularity. The idea of providing embedded support services in form of Internet instant messaging to bridge the gap and improve the whole SaaS support process was derived. Concrete goals of combining instant messaging and software issue context information were set. Chapter 3 revealed in detail the proposed software solution on conceptual level. Furthermore, software user roles in the proposed solution were created, software requirements were established. Structures of the solution were described, key interactions between structural components were depicted, service access and user interface were designed on the level of concepts. Chapter 4 showed the dependencies of thesis prototype and the development environment. The chapter then explained the implementation of the prototype separated in subsections, each of which represents an implementation of a type of embedded support service. Chapter 5 gave the design of evaluation for the thesis implementation. The solution was measured to show the quality of the implementation and also contribution as a software prototype, namely how this prototype can express the potential of the concept of delivering embedded support service in the thesis proposed way. 6.2 ANSWERED QUESTIONS What problem exist in the support organizations of software providers, especially SaaS vendors? This work investigated software issue handling process in SaaS organizations and discovered short comings of current embedded support services which are ticket centric. The current approach fails to deliver end users fast response upon software issues. Also, when tickets are handled in the SaaS vendor s typical tiered support levels, it is often found that the poor quality of the tickets lead to much communication overhead and cost in resolving process. How do researches and industry tackle these issues? Research work in the area indicated the important types of information that are relevant for software issue resolving. Quality gaps are proven by researches. In the industry, telephone support is much adopted. In contrast, instant messaging is adopted to provide fast first response, however without ability to deliver arbitrary context information, which is vital for issue resolving. How can these findings from researches and industry practice be combined and reach synergy? The thesis suggested a new model of embedded support services by combining Internet instant messaging and context data collection. A software prototype was built to prove the concept. The prototype shows that using Internet instant messaging can efficiently deliver 80 Chapter 6 Conclusion and Future Work

101 many types of context information. Evaluation based on interview questionnaire and performance analysis shows that: The prototype provided the desired embedded support services that are needed by users from support and development organizations from the studied SaaS provider. The quality of the implementation satisfies these users. Estimation given by the respondents predicts reduction in both ticket number and ticket resolving time. The questionnaire evaluation also reveals positive influence of the delivered services on customer users. Therefore, the general goal of cutting support cost in SaaS providers is achieved by this work. Additionally, by predicting a very positive influence on customer satisfaction by interviewees, and since support services are important for SaaS products contract renewal, additional economic benefits for SaaS provider are introduced. 6.3 FUTURE WORK The prototype developed shows already promising use cases. However there are many aspects of future work regarding more detailed evaluation and functional extensions. Automatic chat assistance for support user By bundling UI elements from where chat is requested and the support user s answers in a database. Some more services can be provided to support user that he/she gets those solutions which apply to the current screen to business user from other support users, once the chat is launched and the UI Ids are traced. Such services can also be provided by the model established in the thesis prototype, through the chat bot. Artificial intelligence can be implemented for this purpose to further assist the support users. Evaluation on ticket quality improvement Once the prototype is productized, it is meaningful to measure the tickets created without using chat and ones with a chat session before. The tickets should be measured on their types of addressed issue, quality (according to existing models), solution rate, and solution time. Performance Improvements for Screen-sharing Performance improvements for screen sharing can be achieved by adopting more complicated algorithms like delta transmission for HTML pages or data compression. Once the performance allows, chat should provide page refresh service which allows automatic page updates in smaller time intervals. 6.3 Future Work 81

102 82 Chapter 6 Conclusion and Future Work

103 BIBLIOGRAPHY [ADB + 99] [AFG + 10] [BFB + 11] [BJS + 08] [Cus10] [Gla01] Gregory D Abowd, Anind K Dey, Peter J Brown, Nigel Davies, Mark Smith, and Pete Steggles. Towards a better understanding of context and context-awareness. In Handheld and ubiquitous computing, pages Springer, Michael Armbrust, Armando Fox, Rean Griffith, Anthony D. Joseph, Randy Katz, Andy Konwinski, Gunho Lee, David Patterson, Ariel Rabkin, Ion Stoica, and Matei Zaharia. A view of cloud computing. Commun. ACM, 53(4):50 58, April P. Banerjee, R. Friedrich, C. Bash, P. Goldsack, B.A. Huberman, J. Manley, C. Patel, P. Ranganathan, and A. Veitch. Everything as a service: Powering the new information economy. Computer, 44(3):36 43, Nicolas Bettenburg, Sascha Just, Adrian Schröter, Cathrin Weiss, Rahul Premraj, and Thomas Zimmermann. What makes a good bug report? In Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering, SIGSOFT 08/FSE-16, pages , New York, NY, USA, ACM. Michael Cusumano. Cloud computing and saas as new computing platforms. Commun. ACM, 53(4):27 29, April R.L. Glass. Frequently forgotten fundamental facts about software engineering. Software, IEEE, 18(3): , [GXWW11] Steven Gianvecchio, Mengjun Xie, Zhenyu Wu, and Haining Wang. Humans and bots in internet chat: measurement, analysis, and automated classification. IEEE/ACM Trans. Netw., 19(5): , October [GZNM11] [HW07] Philip J. Guo, Thomas Zimmermann, Nachiappan Nagappan, and Brendan Murphy. "not my bug!" and other reasons for software bug report reassignments. In Proceedings of the ACM 2011 conference on Computer supported cooperative work, CSCW 11, pages , New York, NY, USA, ACM. Pieter Hooimeijer and Westley Weimer. Modeling bug report quality. In Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering, ASE 07, pages 34 43, New York, NY, USA, ACM. Bibliography 83

104 [ISO06] ISO/IEC 14764:2006 software engineering software life cycle processes maintenance. Technical report, [JNO + 06] R.B. Jennings, E.M. Nahum, D.P. Olshefski, D. Saha, Zon-Yin Shae, and C. Waters. A study of internet instant messaging and chat protocols. Network, IEEE, 20(4):16 21, [MLM + 06] Matthew MacKenzie, Ken Laskey, Francis McCabe, Peter Brown, and Rebekah Metz. Reference model for service oriented architecture 1.0. Technical Report wd-soa-rm-cd1, OASIS, February [Mof10] [NVV00] [OR] Jack Moffitt. Professional XMPP programming with JavaScript and jquery. Wiley. com, Frank Niessink and Hans Van Vliet. Software maintenance from a service perspective. Journal of Software Maintenance, 12(2): , J Oikarinen and D Reed. Rfc 1459: Internet relay chat protocol, may Status: EXPERIMENTAL. [Qua09] Qualipso consortium. qualipso - quality platform for open source software., [SA] [SAST09] [VM09] [Wik] P Saint-Andre. Rfc 6120: Extensible messaging and presence protocol (xmpp): Core (2011). URL ietf. org/html/rfc6120 [ ]. Peter Saint-Andre, Kevin Smith, and Remko Tronçon. XMPP: The Definitive Guide: Building Real-Time Applications with Jabber Technologies. O Reilly Media, Jose Carlos Maldonado Viviane Malheiros, Erika Höhn. Qualipso project: Quality recommendations for floss development processes Wikipedia. Enterprise resource planning. [Wik13] Wikipedia. Bugzilla, [xmp13] Xmpp, Bibliography

105 A ACRONYMS CRUD Create, Read, Update and Delete CRM Customer Relationship Management CSS Cascading Style Sheet DOM Document Object Model ERP Enterprise Resource Planning FLOSS Free Licensed Open Source software GUI Graphical User Interface HCM Human Capital Management HTML Hypertext Markup Language IDE Integrated Development Environment IIM Internet Instant Messaging ITIL Information Technology Infrastructure Library JSON JavaScript Object Notation JSST JavaScript Screenshot Tool KPI Key Performance Indicator MVC Model View Controller OS Operating System SaaS Software as a Service SOA Service Oriented Architecture QoS Quality of Service 85

106 TCO Total Cost of Ownership UCD User Centric Design UI User Interface XML Extensible Markup Language XMPP Extensible Messaging and Presence Protocol 86 Appendix A Acronyms

107 B EVALUATION QUESTIONNAIRE 87

108 88 Appendix B Evaluation Questionnaire

109 89

110 90 Appendix B Evaluation Questionnaire

111 91

112 92 Appendix B Evaluation Questionnaire

113 C SCREEN-SHARING EVALUATION RESULTS Table C.1: Screen-sharing Performance Measurement 93

114 94 Appendix C Screen-sharing Evaluation Results Table C.2: Screen-sharing Latency Comparison

115 Table C.3: HTML tags with counts (shared screen) 95

116 96 Appendix C Screen-sharing Evaluation Results Table C.4: HTML tags with counts (displayed screen)