Pro-How Paper: 2/00 Measuring the benefits of Business Intelligence http://www.vivaintelligence.com info@vivaintelligence.com Tel. +358-(0)9-253 212 00 Fax +358-(0)9-253 212 50
Viva Business Intelligence Oy is a Business Intelligence service company providing solutions to customers business information content and management needs. Our service offering includes: I) Business Intelligence process development, II) Intelligence content in the form of ad-hoc research and analysis as well as intelligence feed on a continuous basis, III) Business Intelligence technology solutions, as well as IV) Business Intelligence training in the form of sessions and publications. Viva Business Intelligence Oy has done extensive research in the area of Business Intelligence for our customers as well as for our internal training purposes. The series of Pro-How Papers was launched in 1998 in order to share our knowledge on Business Intelligence process development and thus promote the development of world-class Business Intelligence activities among Finnish companies. Previous Pro-How Papers in the Series: 1/98 Introduction to Business Intelligence 2/98 Developing a Business Intelligence Process 3/98 Role of Business Information in Internationally Operating Companies 1/99 Analysing Business Information Needs 2/99 Business Information Types 3/99 Business Information Collection Process 4/99 Critical Management Issues in Business Intelligence Process Development 5/99 Models for Analysing Business Information Needs 6/99 Categorising Information Sources for Business Intelligence Process 7/99 Channels for Collecting Business Information 8/99 The Intelligence Portal 1/00 Business intelligence -service as a product You can order previous Pro-How Papers free-of-charge by emailing us at info@vivaintelligence.com or via our web-page http://www.vivaintelligence.com 2
Introduction This paper examines the difficult task of measuring the benefits of business intelligence products or services. The purpose is to give reader a description of the subject. The article doesn t seek to find specific methods or models how to measure given that every organisation has different goals, objectives and requirements for measurement. The aim is to bring up earlier discussion on the subject and provide general principles, which can help organisations to establish own measurement practices for business intelligence. According to Herring (1999, 5), information users expect to get tangible benefits from using BI-products or services. They want results that support their objectives. Therefore, business intelligence products and services should focus on real and identified business objectives such as getting more customers, better products and the timely introduction of new products to the markets. From the measurement perspective this is a difficult task. Instead of measuring the direct quantitative business benefits of a decision made based on a particular piece of intelligence, other methods may be considered as well. BI-activity could be seen as an internal service with clear and defined product offering consisting of required products such as competitor or customer profiles, industry monitors and strategic outlooks. This means that like any other service, BI-service can be measured on the basis of demand and quality brought to customers, i.e. the decision-makers. Why measure BI activities? In case of insufficient top-level commitment, the strongest incentive for senior management to implement intelligence systems is to see other companies doing it and achieving cost savings (Stanat 1990, 199). One of the most effective means of demonstrating potential cost savings seems to be exploring somebody else s success first. Consequently, benchmarking business intelligence activity of other companies may become an important means of internal BI-marketing. Also Kahaner (1996, 186-187) stresses the importance of demonstrating the value-adding nature as one of the most effective approaches for assuring top management commitment. In addition top-level commitment, the development of the measurement process is critical for every business intelligence unit s existence. If the objective of the unit is growth or gaining a significant position in the organisational decision making, there has to be ways to justify the unit s existence and related benefits. Effective measurement process can be seen as a tool for this. Developing a measurement 3
process can also function as an important link between BI personnel, managers and other information customers. The defined objectives and expectations actually function as standards for evaluation of the BI personnel and the BI unit. (Simon 1998) One of the main reasons why top management may not be committed to intelligence activity is that it is difficult to prove the bottom-line benefits of such an abstract process. Kahaner (1996, 229-232) argues that the effects of intelligence are indirect as it is in the case of advertising when the decision maker does not know which part of the budget actually brought the profit. Similarly, there is usually no simple relationship between the revenues and the money spent on a particular piece of intelligence. Therefore, it may be difficult to justify the expenditures for the top management. One way of looking at the gains is to evaluate how much money has the company lost by not having effective intelligence. Nevertheless, it is difficult to realise that a lost deal or a late product launch was in fact due to inaccurate information about the competitors actions or customer preferences. In some situations, the benefits from BI activities are explicit. However, usually it is extremely difficult to trace the specific data items or pieces of intelligence that brought the benefit. Often the benefits can be identified several years after the actual decision, which makes it difficult to relate the benefit to certain decision or information (Nelke 1998, 15). Nelke has tried to measure monetary benefits obtained from information usage. According to her, it is almost impossible or impossible. However, she has identified some situations where the improved knowledge of external environment could benefit the organisation: Product launch to new markets Development of new products Selling of consultancy services Order approval New patent The issue of BI-benefits is also discussed by Gilad (1995) who emphasises that like any business initiative, setting up an intelligence process should contribute the bottom line. He further argues that studies indeed confirm a statistical link between competitive or business intelligence activities and corporate performance. As a matter of fact, the author implies that the link may be even stronger than the studies suggest. A study prepared for the Society of Intelligence Professionals indicates that companies practising high levels of intelligence are able to increase their business performance more than their less intelligent counterparts. On the basis of the following quantitative findings, it is argued that a high level of business intelligence contributes to good company performance (Jaworski&Wee 1993, ref. McGonagle&Vella 1996, 187): 4
High level of intelligence practised 37 percent higher level of product quality and 68 percent increase in business performance. High level of intelligence practised 36 percent higher level of strategic planning quality and 48 percent increase in business performance. High level of intelligence practised 50 percent higher level of market knowledge and 36 percent increase in business performance. The study indeed demonstrates that intelligence can make a substantial contribution to corporate performance. It is especially interesting to note that a high level of intelligence practised seems to contribute to higher product quality. Thus, it can be argued that research and development activity is to be viewed as an essential customer of business intelligence activity. Typically however, BI seems to be a relatively distant issue for the R&D function. Also Price Waterhouse has found comprehensive evidence on the feasibility of intelligence in a study covering 24 divisions of several aerospace and defence companies. The two-year benchmarking study revealed that among other variables, winning contracts was tremendously contributed by successful intelligence. In fact, intelligence played a crucial role in explaining why some companies won 67 percent of the contracts compared to an industry average of 18 percent. The winning companies especially focused on competitor and customer intelligence. (Kahaner 1996, 232-233) Standardisation of the BI-output, i.e. the products and related production process can be seen as a means of rendering the measurement of the whole BI process. If the organisation has a clearly defined BI product portfolio, it is possible to measure the use or demand of a specific product. The use of a particular product can function as a measure of the related intelligence need. If the use is extremely intensive, it can be argued that the product is successful and its existence justified. However, it should be mentioned that any conclusions have to be treated with caution, because the nature of each product is different and can be an explanation to it s relatively wide use. Therefore, user volume alone can not be used to justify the existence of a certain BIproduct. What to measure? According to Simon (1998) the direct monetary benefits of a business intelligence program are hard to measure. He states that the objective of a BI program is to produce and analyse information. The actual decisions based on this information are made elsewhere and the information obtained from a BI unit or department is only a part of the information used in decision making. Because of the above mentioned reason, there should be ways to measure how much the information obtained from BI activities actually affects the decision making and the decision makers. If it could be clearly shown that a certain decision was made purely based on the information 5
derived from a particular intelligence product, the measurement would be easier. Standardisation of BI-output and the production process can be a tool for this. According to Prescott & Bhardwaj (ref. Herring 1999, 15) the benefits BI/CI practitioners believe BI programs provide are: Influencing decision makers Improved early warning Identifying new opportunities Exploiting competitor vulnerabilities Sharing of ideas Better serving the company s customers According to Prescott & Bhardwaj, these benefits are directly identifiable, although there are no quantitative measures to support this. Improved market position and improved revenue/profits are not directly identifiable since they are uncertain effects. These benefits fall into the category of bottom-line measures, which usually are the most commonly requested. Simon & Blixt (ref. Herring 1999 15) have tried to measure these uncertain effects. They have described the relevant issues to be measured when considering uncertain effects or monetary benefits of business intelligence program. According to them, the relevant measures are: Quality, relevance, timeliness, and accuracy Accuracy of data in analysis Increasing number of clients and additional business from current clients Business success and performance measured by industry benchmarking. Interestingly, this set included both measures of the BI process and its products and measures of actual business success (Herring 1999, 15). Simon (1998) divides BI measures to hard and soft measures. Hard measures imply to a standardised measurement of the organisation s processes. The main interest is then e.g. in measurement of the final outcome, costs, time and quality. The soft measures, on the other hand, are more subjective in nature, which sets their reliability in question. Examples of soft measures are working habits, work atmosphere, feelings, attitudes and organisational development. Table 1. describes the soft and hard measures. 6
Table 1. The soft and hard measures Hard measures Costs -- CI contribution to the bottom line: Costs of doing the research Cost benefit of CI research (saving on organisational resources) Financial gains generated from the impact of ideas and suggestions introduced by CI practitioners Quantitative measures (output) -- measurable number of units of work produced: Clients serviced Projects completed Suggestions submitted Suggestions implemented Projects assisted Number of BI/CI staff Staff productivity Participants in the CI process (direct and indirect CI practitioner performance measures: Effective use of resources (resourceful and creative) Knowledge of CI methods Personal competencies including but not limited to computer, thinking ability, and communication ability (oral and written) Resourcefulness Quality measures: Work product measures Accuracy of information (validity and reliability) First time useable results (no rework) Time measures: Ability to produce timely information Efficiency (time for research and time of response) Soft measures Customer usability: Work habits User friendly reports Participation on teams Contributions to teams Communication skills Contact follow-ups Customer satisfaction ratings Understanding Acceptance and alliance measures: Work climate Number of requests for service Number of repeated requests for service Requests for participation in team meetings Referrals from customers Further integration of CI into projects Unit and personnel effectiveness measures: Feeling/Attitude Solicitation for services (selling of services) Attitude changes clients taking you in to confidence or consulting with you Customer loyalty rating Perception of CI contributions Relationship building (sharing of personal information) Problem solver perception Personnel development/advancement: Rewards Job effectiveness Attendance at CI orientation and training programs (participant or teaching) Promotion Pay increases Work accomplishment acknowledgements CI Practitioner performance measures: Initiative Implementation of new ideas Degree of supervision required Ability to set goals and objectives Time saved by CI interventions On-time delivery Source: Simon 1998 7
Conclusion Measurement of the BI success is an extremely difficult task due to the abstract nature of the BI-process. However, the importance of the formal measurement program is clearly recognised and there has to be ways to justify the existence of BI activity. The benefits of BI activity can be identified and listed, but quantification of these benefits seems rather difficult. This article suggests that standardisation of BI-output and the production processes may provide a solution to the measurement problem. BI-output standardisation and formulation of a product portfolio makes it possible to measure the demand for certain intelligence products in volume terms. New technology allows automatic user monitoring for the products that can be disseminated via Intranet or intelligence portals within the Intranet. Also the demand for products disseminated via other medium must be identified. In addition to volume measurement, the demand should be measured in quality terms as well. In practice this means customer satisfaction measurement. Having clearly defined BI-products with responsible owners it is easier to collect user feedback from the customers. Although BI has to bring value to the organisation, measuring BI-activity does not have to be directly linked to business performance indicators given that the relationship between the bottom-line and particular piece of intelligence is extremely difficult to be identified. Instead, treating BI as a service function with clearly defined product offering and related production processes can provide an easier approach to measuring the demand both in quantitative and qualitative terms. Moreover, by standardising the BI-output, other measurement variables, such as saving time in information handling, are significantly easier to use. 8
References Gilad, Benjamin 1995 Jaworski, Bernard & Wee Chee, Liang 1993. Competitive Intelligence: Creating Value for Organisation -Final Report on Society of Competitive Intelligence Professionals (SCIP) Sponsored Research. Society of Competitive Intelligence Profesionals, Vienna. Herring, Jan, P. 1999. Measuring the Effectiveness of Competitive Intelligence: Assessing & Communicating CI s Value to Your Organisation. Society of Competitive Intelligence Professionals, Alexandria, USA. Kahaner, Larry 1996. Competitive Intelligence. Kane Associates International Inc. New York, USA Nelke, Margareta 1998. Knowledge Management in Swedish Corporations. The Value of Information and Information Services. The Swedish Society for Technical Documentation, Stockholm. Simon, Neil J. 1998. Determing Measures of Success. Competitive Intelligence Magazine, Sept. 1998, Vol 1. No 2, p. 45-48. Stanat, Ruth 1990. The Intelligent Corporation: Creating a Shared Network for Information and Profit. AMACOM, American Management Association. New York, USA. 9