Assessing Data Quality of ERP and CRM Systems

Transcription

1 Faculty of Technology and Society Department of Computer Science Master Thesis Project 15p, Spring 2014 Assessing Data Quality of ERP and CRM Systems By Muhammad Azeem Sarwar Supervisors: Annabella Loconsole Examiner: Helena Holmström Olsson

2 Contact information Author: Muhammad Azeem Sarwar Supervisors: Annabella Loconsole Malmö University, Department of Computer Science. Examiner: Helena Holmström Olsson Malmö University, Department of Computer Science. 2

3 Abstract Data Quality confirms the correct and meaningful representation of real world information. Researchers have proposed frameworks to measure and analyze the Data Quality. Still modern organizations find it very challenging to state the level of enterprise Data Quality maturity. This study aims at defining the Data Quality of a system also examine the Data Quality Assessment practices. A definition for Data Quality is suggested with the help of systematic literature review. Literature review also provided a list of dimensions and initiatives for Data Quality Assessment. A survey is conducted to examine these aggregated aspects of Data Quality in an organization actively using ERP and CRM systems. The survey was aimed at collecting organizational awareness of Data Quality and to study the practices followed to ensure the Data Quality in ERP and CRM systems. The survey results identified data validity, accuracy and security as the main areas of interest for Data Quality. The results also indicate that, due to audit requirements of ERP systems, ERP systems have higher demand of Data Quality as compared to CRM systems. Keywords: Data Quality, Data Quality Management, Quality Assessment, ERP, CRM. 3

4 Popular science summary Data is very centric and important ingredient of any computer or information system. The quality of data will dictate the quality of overall system and all the functions of this system. CRM (Customer Relationship Management) system is a type of software system used in organizations to make the sales process of that company easier. ERP (Enterprise Resource Planning) system is another organizational software system that helps an organization to keep track of all its assets and liabilities. This study examined the data quality of an ERP and CRM systems in an organization. We compared the views about data quality of people working with these systems to what experts of data quality say. According to our findings for good quality, the data in a system should be valid, error free and secure. Now we will use these results in future to ensure that ERP and CRM systems have valid, true and accurate data; and only people who should see or change this data are able to see or change this information. 4

5 Table of contents 1 Introduction Problem statement and motivation Research goals Research scope Results Context of study Products Customer Relationship Management system Enterprise Resource Planning system CRM Process Marketing Sales After sales support and forecasting ERP Process Accounts Revenue Finance Practices, Tools and Techniques CRM and ERP systems architecture CRM and EPR systems Integration People Organization Research method Research methods Research methods selection Research planning

6 3.4 Systematic literature review Literature searching Obtaining and assessing Review method Reading Critical evaluation Recording Data Quality Assessment Survey Defining the survey Survey planning and design Survey Execution Survey limitations Threats to validity Literature Review Literature base for this study Data Quality of CRM and ERP systems Data Quality standing Method of DQ definition selection DQ definition selection Data Quality Assessment method Practicality of Data Quality Assessment methods Survey parameters of Data Quality Assessment Data Entities selection for study Prioritization of Data Quality initiatives Literature review conclusion Survey Results Objective Survey base Results

7 5.3.1 Data Quality and Data management Data Entity ownership Data entry and validation Data Quality Audit and control Data Quality Dimension prioritization Evaluation Conclusion and future work Discussion and Conclusion Benefit to the company under study Answer the Research Questions Future work References Appendix 1: DATA QUALITY ASSESSMENT SURVEY Part 1: Data Quality Part 2: Prioritization of Data Quality initiatives

8 List of Figures FIGURE 1: SALES OPPORTUNITY STAGES FIGURE 2 : CRM AND ERP INTEGRATION FIGURE 3: LITERATURE REVIEW PROCESS [17] FIGURE 4: RESEARCH AREAS RELATED TO DATA QUALITY [3, P.17] FIGURE 5: PRIORITY OF DATA QUALITY DIMENSIONS

9 List of Tables TABLE 1 : JOB FUNCTIONS TABLE 2 RESEARCH QUESTION TO RESEARCH METHOD TABLE 3 LITERATURE TO RESEARCH QUESTIONS MAPPING TABLE 4 : DATA QUALITY DIMENSIONS TABLE 5: DATA ENTITIES IN ERP AND CRM SYSTEM TABLE 6: DATA QUALITY PRIORITIZATION TABLE 7: DATA QUALITY DEFINITION TABLE 8: DATA ENTITY OWNERSHIP TABLE 9 : DATA ENTRY AND VALIDATION TABLE 10: CRM AND ERP AUDIT REQUIREMENTS TABLE 11: PRIORITY OF DATA QUALITY DIMENSIONS

10 List of acronyms ERP CRM DQ DM MDM ER DQA DQM Enterprise Resource Planning Customer Relationship Management Data Quality Data Management Master Data Management Entity Resolution Data Quality Assessment Data Quality Management 10

11 1 Introduction Data is gradually becoming an organizational asset [1], more valuable than any other preserved or perishable asset. Organizations start to realize that to survive in a connected global economy where global boundaries do not put any restriction on the customers; they have to make informed decisions to stay in business. An up to date, connected and skillfully crafted knowledge base is the basis of all informed decisions. An organization have to change its business process in response to the changing business needs but one part of the processing stays unchanged, data. Due to the long-term business prospects and impacts of data organizations are spending a lot of resources to extract, transform and utilize data. Data has always been considered a brainchild of humankind but the technological advancements in computing have created new ideas like The internet of things [19]. In these computing models data is generated and utilized not only by humans but also by machines. The traditional communicate forms will expand to human-human, humanthing and thing-thing [19]. Radio Frequency Identification techniques (RFID), Near Field Communication (NFC) and many similar technologies are enabling seamless and effortless collection of data. This automated data collection makes the business processing smoother than ever before. At the same time more and more business processes are becoming candidate for automation. Several computer systems and automations are deployed in the companies. As the electronic existence of organization matures, it creates many independent and isolated knowledge bases; we can refer to these knowledge bases as data islands. Each data island contains valuable information related to its domain as well as other metainformation and trivial information. Once the users of a system realize that they can benefit from a piece of information hosted by a different system, they ask for integration between two systems. It does not take very long to have a web of interconnected information systems connected through different means of integrations [20]. 11

12 A data warehouse with vast amount of data and systems owning this data creates a valuable source of information for an organization, at the same time the difficulty of organization and management of this information is directly proportional to the size of data warehouse. These information systems are developed with different business needs and these systems are aimed at different domains that can create inconsistent data semantic for business entities once these systems are connected. The study and standardization of these aspects is called Data Quality. [2, p.73] 1.1 Problem statement and motivation Data Quality has been an interest of enterprise executives from the start of computer usage in industries and business [2, p.3] but recent trend of using data to make the business decisions has put immense pressure on data experts to produce data of very high integrity. The quality of data will dictate the usefulness and trust of overall information system. For instance if an online store promise a discount in product campaign, while this discount is not translated in the actual purchase due to poor synchronization between CRM and ERP systems; it will be enough to damage the customer trust forever. High Data Quality is not only considered as an extra attribute of a database anymore, rather the growing awareness of Data Quality has led to major public initiatives like the "Data Quality Act" in the USA and the "European 2003/98" directive of the European Parliament [3, p.4]. The standardization of Data Quality is a research work in progress by ISO. Data Quality standard ISO 8000 will help organization to standardize the data quality across all domains [4]. At the time of writing, this standard is still in development and no guidelines are available for its use. A CRM (Customer Relationship Management) system hosts the information of current and future customers of an organization. A CRM system stretches from Marketing, where campaigns are designed and future customer leads are generated. CRM system will then help the sales process to convert a marketing lead to an actual sales opportunity. CRM system will keep on hosting the after sales customer and support information. An ERP (Enterprise resource planning) system is more close to the actual services and production line than a CRM system. An ERP system can be categorized as management software where planning, costing and development tasks are hosted. Once a sale 12

13 opportunity is materialized in CRM system, this information is passed to ERP system. ERP system then inventories the sales against physical stock in warehouse. Enterprise Data Quality requirements and lack of an established Data Quality assessment method are the biggest motivations for this study. We will incorporate the existing findings in this field and use Data Quality assessment methods proposed by other researchers to measure the Data Quality of two connected CRM and ERP systems. The study will see how well these systems adhere to Data Quality standards. In the last part of this study we will generalize these finding for contemporary data quality issues. 1.2 Research goals The main objective of this study is to analyze the Data Quality issues associated with information systems implementation. This study will identify the gaps and problem areas with current information system implementations and analyze the factors making it harder for organizations to recognize and adapt to Data Quality best practices. The expected outcome of this study is an analysis report of understanding and assessment of Data Quality. This study will focus on answering following research questions: RQ1: How the Data Quality of CRM and ERP system is defined? RQ2: How a method for assessing Data Quality can be defined? RQ3: How well the Data Quality assessment methods work in practice? 1.3 Research scope The scope of this study is bound to the study of Data Quality standards compliance of an organization. We will execute the study on Enterprise Resource Planning (ERP) and Customer Relationship Management (CRM) systems of a public limited company selling products to support Business Intelligence. 13

14 1.4 Results This study will list the implementation constraints responsible for any quality issue. The survey results will give us an empirical view of Data Quality assessment methods feasibility and see their viability for CRM and ERP systems. 14

15 2 Context of study The purpose of this chapter is to present the organization and functionality of CRM and ERP systems and processes in our organization under study. This chapter will set the empirical perspective for this study. The contents of this chapter are writer s experience with the organization and ERP and CRM systems. This chapter will give a brief introduction of the organization where we are conducting this study. After the introduction we will explain the function of CRM and ERP system in our organization under study. In the introduction of this study we described a broad functionality of these systems. This chapter will refine the practical steps of CRM and ERP system usage in day to day business and how these systems are connected to each other. We will list different job functions responsible to operate these systems. The context description will follow the guidelines set by Petersen [23] to describe the context of industrial software engineering in research. According to this guideline a case organization is described by its Products, Processes, Practices, Tools, Techniques, People, Organization and Market. 2.1 Products We are going to use ERP and CRM system of QlikTech in this study as objects of research. In the introduction of this study we described a broad functionality of CRM and ERP systems. To set a context for this study we will explore these products in more detail Customer Relationship Management system A CRM (Customer Relationship Management) system hosts all the data, processes and business logic for current and future customers. A CRM system will use technology to automate and synchronize different departments in an organization dealing with customers. For instance marketing, sales, customer service and after-sales service departments depend heavily on the validity of information in CRM system. 15

16 To support the financial and product growth of QlikTech, we need firm, flexible and extendable CRM systems in place. QlikTech is using Salesforce as organizational CRM system due to its cloud based, on-demand and extendable service. To avoid any confusion with system names we will not use the system provider name and we will use CRM as a generic term to refer the salesforce CRM implemented in QlikTech Enterprise Resource Planning system An ERP (Enterprise Resource Planning) system is collection of services and automation to automate, monitor and track the resources of an organization. Contrary to the function of a CRM system, an ERP system is close to the organization and list all the services and products to sell as well as keeps track of internal assets of the organization. We can say that ERP is backend of CRM system, where CRM system and its functions are more visible to outer world than an EPR but all the processing of sales transitions initiated by a CRM is not possible to execute without an efficient ERP system at the backend. To support the financial and product growth of QlikTech, we need firm, flexible and extendable ERP systems in place. QlikTech was using Microsoft Dynamics ERP until July Due to the limitations in the architecture of Dynamics ERP, another cloud based ERP system Netsuite is now chosen as enterprise ERP for QlikTech. To avoid any confusion with system names we will not use the system provider name and we will use ERP as a generic term to refer the Netsuite ERP implemented in QlikTech. 2.2 CRM Process CRM system is operated by Marketing, Sales and Support departments, so CRM processes follows these working of these departments. In the following paragraphs we will break down the functionality of CRM by different departments of an organization using this system. First we will give the reasoning of use then we will brief that how a CRM is used in certain department. 16

17 2.2.1 Marketing The marketing department is responsible for bringing in the customers for business. There are usually two methods followed in the organization under study, 1) Direct marketing and 2) Alliance marketing. Direct marketing is a major marketing strategy used by businesses of all sizes, where a target audience is selected and through an organized marketing method company message is delivered to this audience. The CRM system helps us to reach target audience via marketing campaigns, Social media marketing campaigns and direct phone marketing. All these marketing transactions are recorded in a centralized CRM system so people working in different teams and locations have access to up-to date marketing status of any customer. Alliance marketing is a mean where other businesses pool together their client resources and share the potential customer information. Alliance marketers have a limited access to our CRM and ERP system where they can register this information. If a customer is introduced by an alliance partner and that customer actually buys the product then alliance partner is given a predetermined share of the revenue. This revenue distribution is tracked using ERP system. For both Alliance and Direct marketing CRM system will host marketing information starting as Marketing lead. Marketing lead is the initial entry point of a potential customer in the system. Once a marketing lead is realized to have the potential of an actual sale then it is converted to a Sales opportunity, at this point marketing department hands over the transaction to Sales department Sales Sales department is responsible for selling the product and bringing in the revenue. Once a Sales opportunity is created in CRM system then sales department takes over the process and starts a one to one coordination with the customer. A sales opportunity goes through different phases of sales. Following diagram 1 gives a transition of opportunity status during a sales cycle. These stages indicate the maturity of a single sales instance at any given time in CRM system. Figure 1 is constructed as the writer s working knowledge of CRM system in the company. 17

18 Goal Identified Goal Confirmed Goal Rejected Negotiation Finance Rejected Closed Lost Closed Won Figure 1: Sales opportunity stages Goal Identified is the first stage of opportunity registration where a customer is realized as a prospective buyer of the product. The next stage is either Goal confirmed or Goal rejected that decides if it possible to proceed with this opportunity. If the opportunity goal is confirmed then Negotiation stage is started where all the sales deal is negotiated and finalized with the customer. At the end, if the opportunity results in a successful sale then it are marked as Closed Won otherwise it is marked as Closed lost. Finance department can intervene at any time during the sales transition and reject the sale due to any legal or financial matter After sales support and forecasting After sales support phase starts once a sales deal is through and customer has purchased the product. Using the same CRM system customers can register their support requests and sales department can delegate those requests to related departments. One more important sales step called product forecasting is performed at this level and all the sales data and customer history is used to analyze and predict the future sales and 18

19 sales potential with an existing customer. Forecasting module is part of CRM system and helps Sales and marketing department to streamline the future strategy. 2.3 ERP Process ERP is managed and operated by Accounts, Revenue and Finance departments, so all ERP processes follow the operation of these departments. In the following paragraphs we will break down the functionality of ERP by different departments of an organization using this system. First we will give the reasoning of use then we will brief that how an ERP is used in certain department Accounts The accounts department manages and monitors all head of accounts in the organization. Accounts department records accounts-payable and receivable, payroll, fixed assets, inventory and all other financial elements. An ERP system is the main system of use in accounts department. ERP will hold all the accounts information for internal assets and liabilities. To facilitate the accounts department and to reduce the aspect of human error in financial figures, ERP has automated some tedious accounting tasking for example automatic revenue recognition entries and assets depreciation entries Revenue Revenue department is the bridge between CRM and ERP systems. Once a sales opportunity is negotiated in CRM and it concludes on a potential sale then a Quote is created in ERP system. A quote details more specific and detailed information about the sale. All that detailed and granular information is only available in ERP system. This point requires many validations and synchronization of data between two systems. Creating a quote might trigger some approval processes within the organization. These approvals are another way of ensuring the data validity and consistency between both CRM and ERP systems as well as sales adherence to organizational policies and international laws. Once a quote is finally agreed upon and signed by the customer that results in successful sales transaction. This transaction will also set the opportunity status 19

20 to Closed won in CRM system. In case the transaction does not end in a successful sale then opportunity is set to Closed lost status Finance Finance department is responsible to assure a smooth revenue flow in the organization. To accomplish the smooth revenue flow Finance has to monitor and audit all the sales transactions happening in the organization. Finance department makes sure that all the financial figures in the ERP system are correct, in this way they support and supplement accounts department to make a stable and auditable ERP system. The function of Finance department starts from the realization of a customer in CRM where customer background, credit history and other financial constraints are checked. If a Quote is created in the ERP system with special and out of the norm financial terms that requires approval from finance before proceeding to the sales agreement. After the sale transaction finance will invoice the customer and make sure that for a software product a valid license is delivered to the customer and customer is paying as per the agreement. 2.4 Practices, Tools and Techniques As defined earlier, QlikTech is using Salesforce platform for CRM operations and Netsuite platform for its ERP operations. This section will discuss the architecture of these systems and then we will discuss the details of integration between CRM and ERP systems CRM and ERP systems architecture Traditional CRM and ERP systems were hosting all the functions required for the system in a single piece of software. Organization interested in using these systems were required to host the CRM and ERP systems on a centralized location and all departments were accessing this information from that host. Although this approach gives more control and security over this information; but at the same time this architecture was risking the around the clock availability of a system. Our company under study is spread globally, so the hosting of a system in one location will cause undesired network delays and the difficulty of system administration around 20

21 the clock. To see this issue imagine that ERP system is hosted in Sweden while Finance department in Japan is making some big financial transactions in the morning time while people in Sweden are not yet awake. Due to some unforeseen reason the system is not accessible in Japan. This will hold the japan office for many hours until people in Sweden start working and resolve any issue in the system. This whole problem is due to the implementation nature of traditional systems. Most of the new ERP and CRM systems are aiming to use Cloud computing to deal with these issues. Cloud computing refers to the software delivered as a service as well as on demand supply of hardware resources [18]. These computing solutions are engineered to remove the need of hosting the system in one specific location and giving the autonomy of administration from anywhere in the world. Both ERP and CRM systems in our organization are using the Cloud computing architecture. It gives the leverage of remote administration and on demand resources acquisition. If any extra space of processing power is required then all of that can be ordered using a single web interface and due to the flexibility of the architecture, a system will get these extra resources within minutes CRM and EPR systems Integration If CRM and ERP system are provided from the same vendor and physically they are part of same software suite then no integration is required between two systems. In our organization under study ERP and CRM are acquired from two different vendors, so these two systems are hosted and work independent of each other. This requires an integration system to be in place to synchronize and control the information in both systems. Figure 2 shows an activity diagram of the integration steps between CRM and ERP systems. 21

22 CRM ERP Marketing Lead generation Account / Customer creation Sales Opportunity Opportunity and Accounts transfer to ERP Quote summary update on the opportunity Quote Creation License informaiton update on the opportunity Sales order and license generation Invoice the customer After Sales Support and Forcasting Figure 2 : CRM and ERP Integration The integration between ERP and CRM happens on many levels. When an account is created in CRM system and realized as a potential sales opportunity then account and sales opportunity information is transferred to ERP system. This information will be required to create a quote in ERP system. Once a quote is created in ERP then a summarized version of information is sent back to CRM system that enables sales people to quickly see and analyze quote status for an opportunity. 2.5 People This section lists all the job functions involved with the operation and implementation of ERP and CRM systems. Each job function is associated with a system that used in their day to day work. Some job functions for instance, the integration experts and finance controllers are working with more than one system in their daily work. The data in 22

23 Table 1 : Job functions, is constructed with writer s knowledge of organization under study. The information will help us understand that who is monitoring and controlling the data in the organization. For instance we can see from this table that Sales operations job function is responsible for Administration and execution of sales and CRM is their tool of work. Table 1 : Job functions Job function Description Associated systems Sales operations Administration and execution of sales. CRM CRM Administrator Administration, maintenance and general support of CRM CRM system. CRM Analyst Analyzing the sales process and making sure that CRM CRM system is capable to support the sales and marketing department. They serve as a bridge between business and technical team. CRM Developer Responsible to extend the capabilities of CRM system. CRM They usually get their requirements from CRM analyst. Integration expert Ensure timely and correct information between CRM and CRM and ERP ERP systems. Finance operations Ensure smooth Revenue insurance. CRM and ERP Account operations Responsible for all account heads and expenses in the ERP organization. ERP Administrator Administration, maintenance and general support of ERP ERP system. ERP Analyst Analyzing the sales and revenue process and making sure ERP that ERP system is capable to support the accounts and revenue department. They serve as a bridge between business and technical team. ERP Developer Responsible to extend the capabilities of ERP system. They usually get their requirements from ERP analyst. ERP 23

24 2.6 Organization We conducted this study at QlikTech AB, a software company based in Lund, Sweden. QlikTech provides a series of Business Intelligence (BI) products by the name of QlikView and Qlik Sense. QlikTech was found in Lund in 1993 as a startup with 35 people but now its staff of more than 2000 people is spread over 25 countries. QlikTech reported annual revenue of million US dollars in 2013 with an increase of 21% from These facts indicate a very high growth rate for QlikTech. The first reason for selecting QlikTech as an organization to study is the diverse and vast amount of ERP and CRM data in this organization. Customers and offices spread over many continents can give many challenges in data handling and quality. Further Qlik has been using different ERP and CRM systems to reach its current state, so system processes are mature enough to give a good representation of a working system. The second reason for selecting QlikTech as an organization to study is the writer s access to ERP and CRM system and process in this organization. A customer repository and internal assets systems are usually not disclosed to any person or authority outside of an organization. Easier access to these systems made QlikTech a good option for this study. Still we are obliged to not share any customer information or internal company information through this study. The third reason for selecting QlikTech as case organization is the understanding and familiarity of data and data related terms in the organization. QlikTech is manufacturing BI products that reply heavily on Data, so there is a good knowledge of data terms in all the departments of organization. In contrary if we pick another organization working in a different domain then business side of organization may lack the knowledge of technical terms. This factor will render our research more useful. 24

25 3 Research method In this chapter we discuss the research methodology used to evaluate candidate Data Quality Assessment methods collected through literature review. This section is divided into four sub-sections. In the first sub-section, the chosen research planning and the reason for choosing this research method is described. In the next part, the systematic literature review method and planned survey is described. In the last section, the evaluation methods which are used for the evaluation of this research are described. 3.1 Research methods Oates mentioned many strategies for research execution [17 p: 35]. Some strategies are of exploratory nature while other research strategies are of more analytical nature. This study is of analytical nature where existing findings are analyzed to evaluate the efficiency and effectiveness of these proposed methods. In the following paragraphs we will discuss different research strategies mentioned by Oates [17] and their suitability for this study. Literature review serves as the base for any research oriented study. Literature review ensures that researcher has reviewed the books, articles and other published material to see the standing of other researchers in the area. [17] This evaluation of literature gives an idea about the feasibility of the study as well as the research gaps in existing literature are identified with this review. The new study can help to fill these research gaps. For this study the use of literature review is not only limited to assess the existing research work but we have to give a definition for data quality assessment as well as visit the practical dimensions of DQA (Data Quality Assessment). With these requirements we will use literature review to give us a conceptual framework for this study and we will use systematic literature review to explore all DQA literature for DQA definition and to identify the empirical features of DQA in literature. 25

26 Survey is a systematic way to collect information of same kind from a large data source. [17] These data sources can be people, documents, events or electronic data. Usually a survey is executed on a selected sample from the whole population and this sample should be representative of the other individuals in the whole population. In our study we have to collect same type of information from a wider audience, so survey will be our data collection strategy to collect information from domain experts. Design and creation is research strategy aimed at creating new products, processes and models [17]. Due to analytical nature of our study this strategy is not suitable for our use. Experiment is a systematic study of cause and effect. A hypothesis is formed before the experiment and in a controlled environment predefined process is conducted. The output from the experiment is carefully analyzed to verify or falsify preformed hypothesis [17]. The requirement of controlling the environment and the lack of a set standard for DQA renders experiment unfit as a research strategy for this study. Case study is a research strategy to focus on a single thing and study all the characteristics and complex relationship of that single case in detail [17]. In contrast to a survey that is broader but shallow, a case study does not care about the context and try to explore all details of a single case. Although we are considering a single organization for this study we will limit ourselves to the context of DQA of ERP and CRM systems. Ethnography is a research strategy to study the way of seeing of a particular group of people. [17] DQ and DQA lies between groups of people, for example a technical person will have a different perspective of DQA as compared to the people working in finance or accounts department. This distribution was more evident from our collected results but at this phase of study we will not explore this ethnography as a research strategy and use it in the future iterations of this study. 26

27 3.2 Research methods selection In previous section we iterated different research strategies and briefly discussed the suitability of each method for our study. Runeson discussed the possibility of mixing multiple research methods to enhance the validity of a case study over opting for a single research method as vivid evidence [21]. Stake suggested the use of Triangulation to increase the precisions of an empirical study [22]. Triangulation refers to the approach of taking different perspectives and data sources into consideration, to add to the validity and precision of a qualitative study. Using Methodological triangulation we can combine different data collection methods to collect qualitative and quantitative information and combine that to answer the research questions. [22] Using Methodological triangulation, we will use systematic literature review as qualitative data collection method and survey as a quantitative data collection method. Systematic literature review will ensure that we are collecting all the data related to this study in a flexible and open manner. Survey will give us controlled and measureable data to testify against the findings of literature review. Information collected with these methods will help us to answer our research questions. 3.3 Research planning For an industrial study it is recommended to conduct a literature survey of industrial studies to determine to what degree industrial case studies covered the context identified in this study [23]. A systematic literature review will give us a literary base to conduct a survey; this survey will show us how well Data Quality Assessment is practiced against methods proposed by different researchers. For the sake of validity we are going to limit our study to published research material on Data Quality Assessment. For the selection of relevant and research work we need a basic framework to track and associate our findings from literature review to the research questions of this study. Table 2 gives a mapping of research questions to the research method answering this question. This table is further explained in next paragraphs. 27

28 Table 2 Research Question to research method Research question Systematic Literature review Survey RQ1 : DQ of ERP and CRM system X RQ 2: DQA method definition X X RQ 3: DQA in practice X X To answer first research question, How the Data Quality of CRM and ERP system is defined? with the help of detailed literature review, we have defined the attributes of a good quality CRM and ERP system. This step gave us a list of attributes and properties of a system that we can monitor in the later part of this study. We will limit the quality parameters for CRM and ERP system, as these are very commonly used systems in organization so any result inferred for these systems can be generalized for other systems. The second research question, How a method for assessing Data Quality can be defined? is targeting the Data Quality assessment in general. As we experienced from a literature review that there are many research gaps in Data Quality study. There are not enough specialized models to access the Data Quality of an ERP and CRM system. A detailed literature review gave us an insight in available Data Quality assessment methods. Now we can analyze and study the DQA dimensions in next research question. We will conduct a survey to answer third research question How well the Data Quality assessment methods work in practice? This survey will present the assessment methods identified in previous step to domain experts. For this survey the main strategy will be questionnaires, but it will also use documents, observations and personal experience as extra help in the research process. 3.4 Systematic literature review Literature review is a systematic way of finding existing work in the research area. It places the research work in context of what has already been published. There are three main aspects in a systematic literature review that makes it systematic [24] Clarity: the findings should be clear and precise by giving a good account of collected knowledge. Validity: all the literature should be collected from an authentic and valid source. 28 Auditability: it should be possible to verify the authenticity of collected literature.

29 For the selection of relevant and research work we need a basic framework to track and associate our findings from literature review to the research questions of this study. Later in this section we will discuss this framework of literature work association. Literature review process Figure 3: Literature review process [17] diagram shows different steps of literature review presented by Oates [17]. The execution of all these steps is discussed in the subsequent sub-sections. Searching Obtaining Assessing Reading Ctritical evaluation Recording Critical review 29 Figure 3: Literature review process [17] Due to the nature of the subject we will start with a broader perspective of Quality and gradually we will narrow down our finding to Data Quality Assessment method. We will find the existing work to set a standard for Data Quality. With the help of basic Data Quality definition we will map this definition to ERP and CRM systems. This first step will give us an answer to our first research question: How the Data Quality of CRM and ERP system is defined? The quality standardization established in first step will be handy when we will compare different approaches to match the data and perform comparative analysis of the data quality. This part of research will bring us closer to our second research question: How a method for assessing Data Quality can be defined? To find the answer of our third research question, we will survey the literature to find the practices of Data Quality Assessment. In the last part of literature review we will

30 review several approaches devised to access the data quality of MDM. We may need to merge different constructs of data quality assessment to form a comprehensive method. Next steps in our research, after the literature review, will prioritize only the most important constructs of selected methods Literature searching Data Quality literature is often published under Data Management headings. Various search terms around Data Quality and Data Management were used to capture maximum published literature. The selection of these keywords was an evolving process where we started with some direct keyword like Data Quality and Data Quality Assessment and in the found literature other keywords used by the researcher gave us indication of related keywords. We kept record of these research terms to avoid duplication of results. Following keyword are used to find literature for this study: Data Quality, Data Quality assessment, Data Quality measurement, Quality measurement, Data Quality management, Data Quality Monitoring, Data Quality Matrix, Data Quality evaluation, Data Quality Standardization. All keywords were used to collect research material related to all our research questions. After review (described in section 3.4.4) the collected material was distributed for its contribution to answer each research question. Each search keyword returned 0.75 to 5.34 million hits on the google scholar, with Data Quality with most number of hits (5.34 million) and Data Quality Standardization with the least number of hits (773,000). By looking at the title of the search material we analyzed that after initial 20 to 40 search results retrieved by google scholar the rest of the research results render irrelevant for this study. So, only research papers with a title relevant to this study were obtained in the next phase of literature review Obtaining and assessing To compile the list of supported literature we started our search with Google Scholar, with search terms mentioned in previous section, to find as much related research material as possible. After compiling a generic list of related literature we searched computer science specific research portals like ACM and IEEE with the same research terms. Most of the articles found through these search engines were not publically 30

31 accessible. We used Malmö university digital library to access these publications and stored the electronic copies for reading and evaluation. After the completion of this step in literature review, based on matching titles, we shortlisted 62 research papers, articles and books for review Review method The findings from previous step were categorized by their contribution to our research questions. We formulated a matrix where each research question was added as a column next to each research paper. This matrix has the dimensions of R x N+1; where R is the number of research articles we collected and N is the number of research questions we have defined in this study Reading After obtaining all literature for this study, each research paper is listed as a single line in the literature review matrix defined in review method. For reviewing an article, first its abstract was visited and if the abstract looks promising to answer any of our research question then this paper was selected for a detailed review otherwise this paper was not considered for further review. 36 research papers were shortlisted after analyzing their abstracts. Papers listed for detailed study were scanned thoroughly to answer research questions. Small description and the possible reason for selection were noted for each research paper. Reading for RQ1: After reading the papers, we identified 7 literature items containing the elements of Data Quality definition and can help us to answer our first research question about Data Quality definition. Reading for RQ2: After reading the papers there were 16 literature items containing the methods and recommendations to assess the data quality of a system. These 16 papers can help us to answer our second research question about Data Quality Assessment method. 31

32 Reading for RQ3: After reading 14 research papers were discussing the different practical dimension of DQA. This information will help us to answer the third research question on the practicality of DQA methods Critical evaluation We start evaluating all shortlist research paper after reading and marked all the corresponding research question in the matrix, this paper was helping us to answer. An X mark in front of research paper under a column for research question RQx will mean that given research paper will help us answer a certain research question RQx. In the end we only selected those research papers those have any research question marked in the matrix and rejected irrelevant research papers. Critical evaluation for RQ1: To answer the first research question, all the research papers and other literature that contains the elements of defining the Data Quality of a system were selected. There were 7 literature items containing the elements of Data Quality definition. After removing the redundant attributes and only selecting the research papers and literature with direct definition Data Quality, selected papers are limited to 4. Critical evaluation for RQ2: To answer the second research question, all the research papers and other literature that contains the practices and recommendations for Data Quality Assessment were selected. There were 16 literature items containing the methods and recommendations to assess the data quality of a system. After careful analysis of the practices and only selecting the papers with generic practices targeted at DQA, the number of research papers is limited to 6. Critical evaluation for RQ3: To answer the third research question, all the research papers that contains the different practical dimension of DQA. Security, accessibly, consistency and integrity are some of the dimension identified by researchers of DQA that we are trying to record. These DQA application dimensions were mentioned in14 research papers. 6 research papers out of these 14 papers are selected for unique dimensions Recording Recording is the step of literature review where the important informant collected after critical evaluation of literature is noted. 32

33 Recording for RQ1: For our first research question all the attributes of Data Quality definition were noted from the research papers shortlisted in critical evaluation step. These attributes are discussed in literature review chapter. Recording for RQ2: For the second research question the methods suggested for assessing the data quality of a system are noted. In the next chapter we will examine the selected DQA practices for review and evaluation. Recording for RQ3: For the third question we listed all the possible dimensions of DQA. Our survey will present these dimension for a review and give an importance index to these dimensions. The next chapter will discuss the literature review results and will serve as the last step of review process with a critical review. 3.5 Data Quality Assessment Survey Survey is a systematic way of collecting data from a diverse population. According to Oates [17] there are four stages of a survey 1) Defining survey 2) Planning and Designing surveys 3) Survey execution and 4) Survey results evaluation. We will elaborate each step from this process in coming text Defining the survey The objective of the survey is twofold. One to get an idea that how Data Quality is understood in the organization and if there is a clear ownership of data defined in the organization. Second objective was to see which areas are important for Data Quality in overall system so that future efforts for Data Quality and Data Quality Assessment will be directed on these guidelines. Complete survey is added as an appendix to this document. This survey is targeted to get user input for Data Quality and the practices to access and organize Data Quality for CRM and ERP system. This survey will help us to answer our second (RQ2) and third research question (RQ3) on the practicality of DQA methods. Further the outcome of this survey will give us idea about future research prospects in Data Quality Assessment field. 33

34 3.5.2 Survey planning and design In our case targeted domain experts are scattered globally and some communication medium was required to gather their input. From different communication methods like , online meetings, telephone, and physical survey distribution; was considered the most feasible form of communication. We will use questionnaire as the main instrument of data generation in this survey. This questionnaire will present a list of possible DQA dimensions identified in literature review to subject matter experts and it will note their feedback in a systematic way. This survey consists of two parts; first part will help to shape the Data Quality view in the organization and second part of this survey will focus on the prioritizations of Data Quality initiatives Sampling frame This survey is aimed at a certain audience, the end-users and people implementing ERP and CRM system in the organization under study. The people implementing and maintaining these systems are the ones usually responsible for all the technical aspects of Data Quality. The implementation team can give us more insights into the audit requirements, DQA standardization and organization patterns of software management. The other set of survey audience, the end users of CRM and ERP systems, are the people directly affected by any quality issue. These end users often set the functional requirements of a system. End user data entry is a vital and first step that incorporates the human intelligence to cleanse and ensure Data Quality. This makes an end-user a vital stakeholder in Data Quality Assessment study. Table 3 in previous chapter gives a list of job functions we are interested for this study. Each job function is associated with a system that used in their day to day work. Some job functions for instance, the integration experts and finance controllers are working with more than one system in their daily work. Therefore the input from these participants is evaluated for more than one system. Initially the survey was planned to be distributed to a wider audience, but due to the overlapping dates between the rollout of new ERP system, the audience was limited to just one person of a job function. As there are 10 job functions interesting for this study 34

35 were identified in the previous chapter, so we selected 10 people each with a separate job function listed in Table 1 to answer this survey. Although this limitation removed the diversity of the results and input from some important job functions like internal audit was missing, but this study gave us an initial point and we were able to see the organization wide interest in the area of Data Quality and Data Quality assessment Questionnaire design This survey is designed to answer two research questions RQ2 (How a method for assessing Data Quality can be defined?) and RQ3 (How well the Data Quality assessment methods work in practice?). Due to a wide spread audience and time zone differences, E- mail questionnaire is the primary strategy of this survey. Survey questionnaire is enclosed as appendix 1 of this study. The questionnaire is using a mixture of open and closed ended questions with the option for the survey taker to include any dimension or parameter not included in the actual answer template. Questionnaire is started with Data Quality parameters identification then it goes towards more empirical aspects of Data Quality. These empirical aspects will help us to answer RQ3. In the second part of the survey several dimensions are provided to the survey taker with an illustration of each dimension. The numerical priority provided in this step will make it possible to analyze the importance of Data Quality dimension in relation to the job function answering the questionnaire. This finding will answer the second research question of DQA method Survey Design Validation Once the initial survey was designed it was sent to three different team members in the organization for validation. One review was working in Sales operation who works with CRM system on day to day basis. Other reviewer was Director of Systems team who is responsible for overall function of ERP and CRM systems. The last reviewer was an ERP systems Developer who implements the system enhancements in ERP system. 35

36 After their review survey questions and possible answers were adjusted for better readability, precision of answers and data collection. Initial version of survey was formed as a plain Microsoft Word Document. Filling of this document was not straight forward for the survey audience. In the light of this review, the survey answers were converted to a click and select fields. This enhancement considerably reduced the survey answering time Survey Execution The survey was distributed via to all selected participant. The selection of these participants is described earlier in this chapter. distribution makes it possible to address globally spread audience and it makes it easy to trace the responses from different audience. Survey was distributed via as Microsoft Word document where drop down lists were used for easy selection. Survey was sent directly to a specified audience so it was easy to track the respondent and the job function of the respondent. Queries from survey participants were entertained via , instant messengers and voice calls. Finally all ten participants returned the filled questionnaire forms via . The results of the survey are compiled and discussed in a separate chapter Survey limitations The survey audience was limited to fewer job functions than initially planned audience. That limitation will limit the variation of survey results. Further this survey was executed on a single organization, although this organization is using the latest ERP and CRM systems and is very flexible to adapt the best practices to enhance the overall system quality but more generalization of these results will require conducting this survey across multiple organizations with different organizational structure and CRM and ERP systems used. 3.6 Threats to validity The validity of a study represents the trust worthiness of the results. Runeson and Höst suggest using four points to view the validity [21]. 36

37 Construct validity is to ensure what we have in mind for this study and what actually was investigated. As we are only referring the work published in research journals and books, it is possible to oversee some practices of DQA. The reason for skipping those practices was either the unavailability of implementation details, due to proprietary rights, or the unauthentic marketing nature of published results. So we are only limiting our research to authentically published material. For survey, it can be a risk if our survey does not focus on answering our research questions or we miss some important aspect in literature review. Internal validity refers to the validity of relation between a researched fact with a fact we are researching now. It is a threat that original fact was driven with certain condition while we miss that condition in our study. Organization wide awareness of Data Quality can be another factor that can enhance the results of this survey. During the survey people were able to realize the importance of Data Quality in the systems but they were not able to fully comprehend those benefits to their job functions. External validity is concerned with the fact that how generalized our findings can be. As we are studying a single organization at the moment for this case so it is a possible threat that our findings are specific to the implementation and usage of ERP and CRM system in our organization under study. To overcome this issue we need to extend the audience of this study in future. One possible threat is that survey audience was reduced to one person for each job function. This reduction of the survey participants resulted as the lack of diversity of collected results. Reliability aspect is concerned that what extent the data and analysis is dependent on specific researcher. If another researcher performs the same study then the research should not have extreme differences. To overcome this thread we used survey validation and peer review to reduce the preferences of a single researcher. 37

38 4 Literature Review This section explores the existing work and narrates the findings of other researchers relevant to this study in the fields of Data Quality and Data Quality Assessment. We will use systematic literature review, described as one of the research method for this study, to answer first research question (RQ1) and gave base to our survey of DQA. First section in this chapter will show the resultant matrix of literature review conducted for this study. Next we will use literature review to discuss and define DQA. After that DQA method and dimensions are formed with literature review. This section will help us to create the input parameters for our survey. In the end we will conclude with a critical analysis of literature for DQA. 4.1 Literature base for this study The literature review provided a number of articles for our search of different search keywords. After analyzing all the found literature and filtering any unrelated or redundant literature we are left with following matrix of literature review. Each line in this table presents one literature source and the rationale of selecting this source. For instance Redman [2] was selected because it provides an empirical approach for DQA and it will help us to answer RQ1, and we found Redman [2] using Data Quality as keyword of search. Table 3 Literature to research questions mapping will give us literature background to conduct this study. In the next section we will explore different options to answer RQ1 using literature. In the first column search keywords on the top are the source words to extract the corresponding research papers shown in the cells below each keyword. Second column gives the reason to shortlisting this research paper. Research question column shows the contribution of each paper to one or more research questions. For instance the research work by Redman [2] was found with keyword Data Quality and it will help us tow answer RQ1. As described in research methodology chapter 3.4.3, the original table consisted of 62 rows. Using the evaluation method the table was reduced to following state. 38

39 Search keywords and associated articles Table 3 Literature to research questions mapping Rationale of selecting this paper Research Question this paper supports Data Quality Data Quality assessment Data Quality measurement Data Quality management Data Quality Monitoring RQ 1 RQ 2 RQ 3 Redman [2] Batini [3] Loshin [5] Gives a practical aspect of data quality Data Quality: Concepts, Methodologies and Techniques The Practitioner's Guide to Data Quality Improvement X X X Otto [6] Suggests how to measuring Master Data Quality X Loshin [7] Discussion on Master Data Management X Fanclanci [8] Adds user s satisfaction to DQA measures. X Batini [11] Nousak [13] Piprani [12] Freytag [9] Completeness of integrated information sources Lists and compares techniques to assess and improve the quality of data Discusses the reasons of failing DQA A Scorecard approach to improving Data Quality X X X X X X Pipino [14] EU DQA[16] USAID [15] Describes principles to develop DQA Matrix Discusses the implementation of a Data Quality Audit Tool Handbook on Data Quality: Assessment Methods and Tools X X X X 39

40 4.2 Data Quality of CRM and ERP systems To answer our first research question, to define the Data Quality of CRM and ERP systems, we will start with the definition of Data Quality in general Data Quality standing Data quality is fairly new research area that stands on the intersection of computer sciences, business automations and statistics; as shown below in Figure 4: Research areas related to Data Quality [3, p.17]. These areas expand from more empirical domains like Management Information Systems to more theoretical domains like Knowledge representation. For all these data related fields the quality of data will dictate the outcome of any research or practice. For instance, the results from a Statistical or Data Mining analysis will never be reliable if the data quality of underlying data is not trustworthy. This illustration puts Data Quality at the center of all data related activities. [3][9] Figure 4: Research areas related to Data Quality [3, p.17] In following text we briefly examine each of the domains mentioned in the figure 4 and the relationship of Data Quality to this domain. Data Integration ensures that data across multiple data sources stays synchronized. For the case of this study, the data in ERP system is fed from the CRM system using Integration processes. If the data quality in one system is 40

41 not of high quality or if the integration process has some flaws then we can never ensure the data quality in ERP system. So Data Integration has very high influence on Data Quality and vice versa. Data mining is the study of extracting as much meaningful information to solve a problem at hand from existing data as possible. If the Data Quality of original data is not very good then results of Data Mining cannot be trustworthy. Statistics and Statistical data analysis is domain similar to data mining where scientific methods are used to extract inferences from underlying data. Poor quality of source data will result in poor statistical inferences. Knowledge representation is the study of presenting the information in a meaningful way according to given context. The quality of underlying data will directly translate to the quality of end results by knowledge representation. Management Information System (MIS) is the study of engineering system to support and maintain management and business processes. Poor Data Quality in MIS will lead to poor business decision making Method of DQ definition selection Most of the Data Quality definitions directly translate the Data Quality to a single true version of information [9]; but this definition does not consider the fact that data is subjective [5]. For instance a CRM system will collect customer information with relation to the sales department. Examples of sales customer data can be probability of securing a deal with customer, contact persons at the customer site and their connection preferences. At the same time an ERP system will need customer information to track organization resources for example customer billing address, quantity of items sold, payment method. If customer data is collected and organized with keeping a single system in mind, it will not be of high quality to all systems. The data is generated by the business processes responsible to collect relevant information, so we cannot expect a database that compensates the flaws in a business process. By the help of literature review we are able to comprehend that most organizations measure data quality in the order of 1) completeness of information 2) data life cycle 41

42 management and 3) impact on business processes [3, p.69]. Different Data Quality definitions take different perspectives; these definitions take some attributes of dimensions, models, techniques, methodologies, tools, and frameworks into consideration while deciding the overall data quality but do not give a comprehensive coverage of all these factors. We aimed for a definition that should provide all three factors discussed in the previous paragraph; of 1) completeness of information 2) data life cycle management and 3) impact on business processes DQ definition selection Data Quality is discussed by many researchers but in most of the instances the Data Quality is attributed to a single feature or activity. We were able to find three proper definitions of Data Quality in literature where DQ is described as a discipline rather than only a single activity. Batini [3], Loshin [5] and Redmin [2] defined Data Quality as a discipline. We will evaluate each definition in coming paragraphs with our selection criteria defined in previous section. Batini [3] defines Data Quality as a multidisciplinary area where quality of data in any system is directly related to the characteristics and management of that system. This definition fulfills DQ definition selection requirement 2, of data life cycle management and requirement 3, of impact on business processes, but it does not mention the completeness of information. Loshin [5] defines Data Quality as a value-adding and risk mitigation feature for business processes. According to our DQ definition selection method this definition lacks the requirements of completeness of information and data life cycle management. The definition provided by Redmin [2, p.73] fulfills all these aspects of Data Quality. This definition will be used as a base of evaluating the DQ in this study. According to this definition, to have a good data quality the data should be free of defects, possess desired functional and non-functional features and should serve the purpose it is collected and stored for. The reason for selecting this definition is the simplification and comprehensive approach it presents for Data Quality and it sums up all the DQ requirements discussed 42

43 by other researchers. The first requirement of good Data Quality, free of defects puts functional and non-functional constrains on underlying data. It implies that data can never of good quality if there are numerical, logical or functional errors in it. The second part of this definition puts the constrains of functional and non-functional features on underlying data. To have a good Data Quality, data should meet all desired relationships, associations as well as technical requirements of the system the data is collected for. The last part of this definition requires the data to serve the purpose it is collected for. It implies that decisive factor of data quality is how useful this data is for the sake it is stored for. For instance, if we are collecting customer data for CRM system then it should give a very good depiction of customer from a CRM point of view. 4.3 Data Quality Assessment method To answer our second research question on Data Quality Assessment method definition, we will first see what different perspectives of assessment are suggested by researchers. Francalanci viewed Data Quality from a user point of view, where the quality of data is defined as fitness for use [8]. This study proposed a model that ties the data quality assessment to user requirements. Loshin, required following three quality measure in an organization. [7] Assessment: the ability of a MDM system to automatically identify a candidate entity. Integration: standardization, parsing and matching/ linkage techniques to ensure a smooth data integration. Assurance: auditing compliance and a good data governance model should be in place. To analyze difference perspectives and view of Data Quality we gathered possible dimension of Data Quality identified by researchers. Table 4 : Data Quality Dimensions gives a listing of possible DQ dimensions against their sources. This table shows that some dimensions of Data Quality like Accuracy, Completeness and Timeliness of data is suggested by many researchers. While some other dimensions of 43

44 Data Quality, like Ease of manipulation, Relevance and Value-added are not commonly discussed by many researchers. Assessment Criteria Table 4 : Data Quality Dimensions Piprani [12] Nousak [13] Pipino [14] USAID [15] EU DQA[16] Accessibility X X Appropriate amount of data X Accuracy X X X X X X Comparability X Completeness X X X X X Consistency /Coherence X X X X Ease of Manipulation X Integrity X Precision X X X X Reliability X X X Relevance X Security /Confidentiality X X Temporal Relatablity (Reliability in given time) X Timeliness / Currency X X X X X X Understandability X Uniqueness X X Validity X X X Value-Added X Batini [11] 4.4 Practicality of Data Quality Assessment methods After the establishment of a DQA method, we need to analyze the practical aspects of executing this method. This step will make our way to answer third research question How well the Data Quality assessment methods work in practice? Otto and Ebner, in their study Measuring Master Data Quality [6], give a status report on current data quality measurement in practice. It is a more empirical study of data quality to see 1) if organizations are already measuring data quality 2) what do they measure and 3) how do they measure? We can use this technique to design our research around these three questions of DQA. 44

45 The activities of DQA is composed of three phases [11] 1. State reconstruction: this phase aims at collecting contextual information on organizational processes and services. This will construct a detailed present picture of the system, that picture is vital for doing any analysis or DQA activity. 2. Assessment / measurement: this phase aims at collecting information along relevant quality dimension. The collected information is measured against a reference value to diagnose the quality of system. This phase can be broken down to sub-phases a. Data Analysis: understand the data and related business rules. b. DQ requirement analysis: identify quality issues and quality targets. c. Identification of critical areas: select the most important and business critical datasets for study. d. Process modeling: identify the processes and systems updating and generating the data. e. Measurement of quality: this step identifies the quality issues for a quality dimension selected in DQ requirement analysis step. 3. Improvement: the last step identifies the problem areas and possibly suggests the measure to improve overall system quality. 4.5 Survey parameters of Data Quality Assessment In literature review we identified two important parameters of data quality in a system 1) Data Entities of a system 2) Data Quality dimensions or initiatives. In this section we will discuss and compile a list of data entities of a system and important Data Quality initiatives for our survey Data Entities selection for study The realization of data and data entities ownership within the organization is an important part of this study. A data entity is the definition of a data object in the system; this definition contains the parameters of data, relationship of an entity to other data entities and metadata about that entity. For instance data entity Customer will hold all customer 45

46 information like customer name, contact information, delivery address, shipping address, legal status of sale and list of contacts persons. Table 5: Data entities in ERP and CRM system, lists the entities included in the survey to get an idea of ownership of these entities. These entities are selected from both ERP and CRM systems, where some entities like opportunity and product exists in both systems. 46

47 Data Entity Table 5: Data entities in ERP and CRM system Description Account / Customer Asset Contract Invoice Lead Opportunity Opportunity line item Product Purchase order (Internally generated) Purchase order (Customer generated) Quotes Sales order Subsidiary / Territory Support case Users (System users) Vendor Vendor bill An entity we are selling our products to. Fixed asset management in ERP. Specifies specific level of support or service for a deal. Invoices generated and sent from ERP to our customers. A marketing lead in CRM system. A Sales Opportunity, this will result in an actual deal winning or losing. Products we are expecting to sell to our targeted customer of an opportunity. Definition of an actual product we are selling. For our internal purchase, PO serves as the official request put in ERP system. When we receive a PO from our customers who are interested in buying our products. Who manages that information? A Quote is generated on a sales opportunity for a targeted customer. The order we issue to our customers for payment. Monitoring a territory specific data restrictions and settings. Mechanism to monitor and enhance the system support. People using the systems. Entities we buy from. A bill we receive from a vendor we purchased items / services from. 47

48 4.5.2 Prioritization of Data Quality initiatives In literature review we were able to identify the list of data quality attributes. These attributes identify different technical, security and operational aspects of Data Quality. Table 6: Data Quality prioritization, describes the characteristics of Data quality in a system, identified by researchers. Table 6: Data Quality prioritization Sequence DQ Dimension Description 1 Accessibility Data is accessible to people with right permissions. 2 Appropriate amount of Each function is only provided with relevant data. data 3 Accuracy Data accurately represents the real world entity/operation. 4 Comparability It is possible to compare two data entities. 5 Completeness All the attributes of a real world entity / operation are mapped by the data. 6 Consistency Data is consistent across all the systems. /Coherence 7 Ease of Manipulation It is possible to manipulate data and the data is synchronized with other systems automatically. 8 Integrity There are automatic as well as manual rules to ensure data integrity. 9 Precision The information is precisely following the standards. 10 Reliability Users of the system have high trust in the data. 11 Relevance Relevance of collected data in the system to the actual entity. 12 Security Data is secure. /Confidentiality 13 Temporal Relatablity (Reliability in given time) Meaning of a data attribute / entity can change over time. Is it possible to relate repeatable instance to the time of its occurrence. The relation is called Temporal Relatability. 14 Timeliness Data can be retrieved in a specified amount of time. 15 Understandability It is easy to understand the data and its relationship. 16 Uniqueness There is no duplicate data in the system. 17 Validity It is possible to validate the data through relationship between entities. 18 Value-Added Data is Value-added for the business operations. 48

49 4.6 Literature review conclusion The first research question to define the Data Quality of CRM and ERP system was answered in section 4.2 with a comprehensive definition provided by Redmin [2, p.73]. According to this definition to have good data quality; data should be free of defects, possess desired features and should serve the purpose this data is collected for. There is work done for Data Quality and Data Quality Assessment but still there are many research gaps in this domain. For example there is no research targeted at Dimension selection for Data Quality Assessment. Further there are publications for CRM and ERP systems but there is no research done on Data Quality Assessment perspective. This review gives us a good ground to proceed with our study of DQA of CRM and ERP systems. We gathered all Data Quality initiatives and dimensions discussed by the researchers. This literature review gave us a view of current Data Quality and Data Quality Assessment standing. We were able to gather enough information to answer RQ1 and we gathered literature base to construct our DQA survey to answer next two research questions. 49

50 5 Survey Results In this chapter we will analyze the collected survey results. First we will describe the objective of this evaluation, the participant s involvement and finally we will conclude with a discussion and analysis of collected information. As this study was a first in the line and Data Quality initiatives for an organization so the results will decide the future direction of Data Quality Assessment and Data Quality in general. This survey will help us gather more expert knowledge in the domain to answer our research questions. 5.1 Objective The main object of this evaluation is to get a view of Data Quality maturity in the organization under study. The secondary objective is to point out the areas of improvement for Data Quality. Although there is some recognition of data quality in the organization but we need to evaluate that how close this recognition is to the Data Quality attributes identified by different researchers. 5.2 Survey base Systematic Literature Review gave us a good understanding of current research standing of Data Quality and Data Quality assessment. We were able to constitute the parameters of Data Quality and how they are effective in improving the overall system quality. In introductory chapter 2 we were able to see how ERP and CRM systems are used in real life, job functions and integration between both systems. By visiting the literature review we were able to see research gaps in the area of DQ dimension selection and DQA execution. 5.3 Results The answers from the survey participants were sent back using s that helped us tracking and associating the results to a specific job function. To analyze the results we distributed the questions into different sections. In the coming paragraphs we will discuss the collected results against the job functions who answered. 50

51 An X mark represents the selection of answer for a question and if the answer box next to a job function is empty that translates into a missing answer. In all the questions participants are given with an option to ignore the question with a Do not know option. This will help us to filter misleading information for this survey. Due to dedicated participants and access to company we were able to get a 100% response rate. Following are the figures for the response rate. Number of participants: 10 Number of respondents: 10 Response rate: 100% Data Quality and Data management The very first questions in our survey were aiming at Data quality definition and Data Quality Management plan in the organization. Answers to these questions will give an overall realization of Data Quality in our organization. Table 6: Data Quality prioritization, lists the answers of all job functions against the answers provided by them for Data Quality Definition. For the first question of Data Quality definition, the most selected answer was that data should be free of the defects and it should help an end user to perform his day to day job. As ERP and CRM system are used extensively by business users so we can conclude that Data Quality for them is problem free and Value adding characteristics of data in these systems. Technical job functions like Integration experts and Developers were also interested in the non-functional attributes of the Data Quality. Answers to second and third questions gave us the current standing of Data Quality in the organization. Five out of ten job functions were not aware of any Data Quality management plant. Some areas of CRM and ERP are automated to validate and detect data redundancy in the system. To ensure the data integrity of financial information Finance department puts a lot of manual effort and performs periodic internal audit on finance related data. 51

52 Job function Q1: Data Quality Def. Table 7: Data Quality Definition Q2: Plan for DQM Q3: DQM Plan Fitness for my daily work Fitness for decision making Fitness of data for planning Data is free of defects Data possesses desired features Sales operations X X X Do not know CRM Admin X X Do not know Data redundancy detection. Third CRM Analyst X X X Yes party service to verify address, and phone information. CRM Requirement of data validation Developer X X X Partial and coherence Integration expert X X Partial Business rules in data integration. Finance Finance data is periodically operations X X X X Yes audited. Account operations X Do not know ERP Admin X X Do not know ERP Analyst X X X X Partial ERP Developer X X Do not know 5: Do not know 3: Partial Total : Yes Accounting activities like depreciation and revenue recognition are automated to avoid Human error. Looking at the results we can say that there is a realization of Data Quality in the organization. There are many Data Quality activities performed within different departments but there is no well specified and well communicated plan for Data Quality. 52

53 5.3.2 Data Entity ownership The answer to next question will identify the ownership of different data entities in the system. Table 8: Data Entity Ownership, contains the answers provided by different job functions for the ownership of selected data entities. In section the selection of these data entities and a short description of each data entity are given. In Table 8 the resulting Job functions responsible for each entity are abbreviated as SO: Sales Operations. AO: Accounts Operations FO: Finance Operations MO: Marketing Operations PD: Product Development and Operations Admin: Administrator of CRM or ERP system IG: Internal Audit and Governance Empty box: Do not know 53

54 Job function Q4: Data Entities Ownership? Table 8: Data Entity Ownership Q4: Data entities Owners Account / customer Asset Contract Invoice Lead Opportunity Sales operations Partial SO SO FO MO SO SO PD FO SO SO IG IG CRM Admin Partial SO SO MO SO SO PD SO SO Admin IG IG CRM Analyst Partial SO SO FO MO SO SO PD FO SO SO Admin IG IG CRM Developer Partial MO SO SO PD SO SO IG IG Integration expert Yes SO SO MO SO SO PD FO SO SO Admin IG IG Finance operations Partial SO AO SO FO MO SO SO PD AO FO SO SO Admin IG IG AO AO Account operations Partial AO FO SO SO PD AO FO SO SO IG AO AO ERP Admin Partial AO SO FO SO SO PD AO SO SO Admin IG AO AO ERP Analyst Partial SO AO SO FO SO SO PD AO FO SO SO Admin IG IG AO AO ERP Developer Partial AO FO SO SO PD AO SO SO IG AO AO Opportunity line item Product Purchase order (Internal) Purchase order (External) Quotes Sales order Subsidiary / Territory Support case Users (System users) Vendor Vendor bill We can see in table 8, the ownership of a data entity in most of the cases rests with operations people. They are responsible for the data in those data entities and any Data Quality initiative will be triggered by operations. Although this is a good thing in one aspect as they are the domain experts and they know their data better than anyone else but when it comes down to technical aspects of data and data quality a business person will not necessarily have the skills to know all the technical knowledge. The answer for the question is pointing the need of involving technical stakeholders to share the ownership of Data entities. As we realized in Question three of the survey that there is no well-defined Data Quality Management plan in place, so missing the technical aspects of data quality can be a side effect of missing DQM plan. 54

55 5.3.3 Data entry and validation The question five, six and seven will give us a view of Data Entry and validation standard in the organization. If the answer for a question Yes or Partial then participants were directed to give brief explanation of their answer. As data entry and validation is very specific to the operation a system so we do not expect the users of ERP system to answer the question for CRM system and vice versa. CRM Analyst CRM Developer Table 9 : Data Entry and Validation Job Q5: Data entry standard for CRM Q6: Data entry standard for ERP Q7: How data is function validated Sales operations Yes Approval processes. Do not know When entered or during transaction CRM Admin Partial Varies from process to process. Do not know Field level validation Sales transactions with certain attributes require Finance reviews the Partial approval. Partial financial data Integration exp. Partial Partial Data is checked when entered Some validations are part of CRM system while some others are part of data integration process. Finance operations Partial Yes Account operations Do not know Yes Do not know ERP data is validated by finance operations. Integrations do not Yes validate ERP data. Data is verified manually. And special financial transactions require Finance approval before processing. Accounts and entries are approved by a supervisor. ERP Admin Do not know Do not know ERP Periodic auditing for Accounting and Financial Analyst Do not know Yes data. ERP Developer Do not know Do not know Total 5:Partial 4: Do not know 1: Yes 5: Do not know 4: Yes 1: Partial All ERP transaction are reviewed and verified. Any special transactions need finance approval. 55

56 To analyze the results of data validation we ignore the Do not know answers for question number five and six. We can see from the table above that validation on CRM side is only applied on selected areas, so that makes is partially covered. ERP contains financial and accounting data that makes it a candidate for higher data validation. The result shows, there are many manual controls put in place to monitor and validate ERP data Data Quality Audit and control Organizational audit is an official inspection of accounts, transactions and financial information required by law and often performed by an independent organization. Answers to question eight and nine will show that how important CRM and ERP systems are for organizational audit. The answer of this question was answered on a Likert scale with values from 1-5 where 1 is considered as not covered in audit and 5 is crucial that is mandatory for audit. Question nine was aimed to know if there is any internal or external standard used for Data Quality Assessment in the organization. If there is any standard used then what are the specifics of that standard. Job function Table 10: CRM and ERP Audit requirements Q8: Audit importance for CRM Q8: Audit importance for ERP Q9: Standard for DQA Sales operations Important Very important Do not know CRM Admin Very important Do not know Do not know CRM Analyst Important Very important Do not know CRM Developer Important Very important Do not know Integration exp. Important Crucial Do not know Finance operations Important Crucial Do not know Account operations Important Crucial Do not know ERP Admin Do not know Crucial Do not know ERP Analyst Very important Crucial Do not know ERP Developer Important Very important Do not know Cumulative Important Crucial 56

57 It is obvious from the results of, Table 1 : Job functions, that ERP system is mandatory for the audit purposes and all the data in ERP system should be controlled and monitored. As far as CRM is considered that is contributing in the process of organizational auditing as CRM is master data for many data entities but it is not crucial system for audit. The next question on Data Quality Assessment standardization is not known by any job function in the organization. That clearly points out the absence of a Data Quality Management and Control program in the organization for ERP and CRM systems Data Quality Dimension prioritization In the last part of this survey, a list of Data Quality dimensions is provided to the participant and they are asked to prioritize the dimensions of Data Quality important for their job function. This list will serve two purposes 1) the importance of a DQ dimension will be known for a job function 2) depending on the audit and control requirements of the system an action plan can be created to focus on highest priority DQ dimensions. In the survey execution we were able to see a difference between the prioritization of DQ dimensions by ERP and CRM job functions. In order to not miss any system view the results are divided into job function working with EPR and CRM systems. Responses from Sales operations, CRM Admin, CRM Analyst and CRM Developer are gathered under one heading, while response from Finance ops, Account ops, ERP Admin, ERP Analyst and ERP Developer are categorized under a different heading. As integration expert is working between two different systems so response from Integration expert is counted for both systems. One interesting fact that we observed that the most of the participants filled initial five to ten dimensions in their response. To harmonize the results from each response only initial seven priorities are selected. Following table gives an overall as well as CRM and ERP specific score for each DQ dimension given by participants. 57

58 Table 11: Priority of Data Quality dimensions DQ Dimension Over all CRM ERP Accessibility Appropriate amount of data Accuracy Comparability Completeness Consistency /Coherence Ease of Manipulation Integrity Precision Reliability Relevance Security /Confidentiality Temporal Relatablity Timeliness Understandability Uniqueness Validity Value-Added To visualize the DQ dimensions all measures are plotted as a bar graph in following Figure 5: Priority of Data Quality Dimensions, where DQ dimensions are present on the x-axis and y-axis represents the overall, CRM and ERP specific scoring of each dimension. 58

59 Accessibility Appropriate amount Accuracy Comparability Completeness Consistency Ease of Manipulation Integrity Precision Reliability Relevance Security Temporal Relatablity Timeliness Understandability Uniqueness Validity Value-Added Master Thesis project: Assessing Data Quality of ERP and CRM Systems Over all CRM ERP Figure 5: Priority of Data Quality Dimensions From this graph in Figure 5 we can see that overall Validity is chosen as the most important DQ dimension, followed by Accuracy and Security. For CRM system the most important DQ dimension is Value-Added, followed by Accessibility, Integrity and timeliness. For ERP system the most DQ dimension is Validity, followed by Security and Accuracy. These results show that for CRM system the most important aspect of data is the business value it adds to the day to day function of the system. While for an ERP system the most important aspect is the accuracy and the validity of information in the system. 5.4 Evaluation Survey results show that Data and all quality aspects of system are controlled and owned by the business operations. Data Quality is considered as a non-functional aspect of the system where many disconnected activities are performed by different departments but there is no single and centralized effort for Data Quality. This survey pointed the need of a harmonized and controlled Data Quality Management plan in the organization. The second part of the survey will be used as an initial feed for any Data Quality Assessment and Management program. The data and the Data Quality dimensions collected here can form an initial Data Quality Assessment matrix on which a Data Quality Management plan can be initiated. 59