A preview of The BI Survey 12: The Results. For more information, visit:

Transcription

1 lk THE BI SURVEY The Customer Verdict The world s largest survey of business intelligence software users 11 A preview of The BI Survey 12: The Results For more information, visit: Free Preview This document is not to be shared, distributed or reproduced in any way without first purchasing licensing rights and the express prior permission of BARC.

2 2 Table of Contents Introduction... 4 Business Benefits and the BBI... 7 BBI by peer group and vendor... 9 Goal Achievement Deployment Usage in different company size classes Number of BI users BI usage by departments Product usage for application scenarios The selection process Evaluation methods Reasons to buy How products perform in the selection process Buying BI software Licenses Number of users Cost Implementing BI and Satisfaction Implementation time Project scope Support Recommendation Chosen as standard Problems in BI projects Query Performance and Data Volume Query Performance results and user satisfaction Data volumes handled per product... 73

3 3 Front ends for multidimensional databases Infor ION BI OLAP Server Microsoft SSAS Oracle Essbase IBM Cognos TM SAP BW Trending topics in BI Three-step approach to analyzing trends The frequent inaccuracy of plans User perception of IT and software trends Status and plans for trending topics in BI Visual Analysis & Data Discovery Collaboration Mobile BI Cloud BI/BIaaS... 94

4 4 Introduction This document provides an overview of the key product-related results from The BI Survey 12. The Survey provides a detailed quantitative analysis of why customers buy BI tools, what they are used for, how successful they are and why they eventually abandon them. This edition has increased the range of products reviewed, including not only products from well-known BI giants, but also specialist tools from much smaller vendors and open source vendors. The BI Survey 12 follows on from ten successful editions of The BI Survey (formerly The OLAP Survey). It is important to note that this year the numbering of The BI Survey has changed so that The Survey number now aligns to the year in which the data was collected. Based on analysis of real-world experiences of nearly 3,000 respondents, the value of The Survey is dependent on a sufficiently large, well-distributed and unbiased sample. The Survey is the largest and most thorough fact-based analysis of the BI market currently available, using eleven years of experience to analyze market trends and debunk many myths surrounding the BI industry. After data cleansing and the removal of responses from participants unable to answer specific questions on BI tools, we were left with a sample of 1872 end users, 283 consultants and 512 vendor employees. Participants from all over the world took part in The BI Survey 12. Around 20 percent of respondents had an IT job function, 15 percent were Heads of BI, and 65 percent had various lines of business job titles. The BI Survey is not based on anecdotal accounts or personal opinions, unlike much analyst research. The Survey is not intended to be a measure of market shares and unlike our research publication, The BI Verdict, it does not include product reviews or qualitative product comparisons. The BI Survey does not attempt to forecast future trends, instead providing evidence undermining the reliability of many such forecasts. In the past the vast quantity of data collected by The Survey has been presented in one large document. However, The BI Survey 12 heralds the introduction of several bitesize documents, each focusing on a specific area of The Survey.

5 5 Detailed information on the sample and Survey methodology can be found in the Sample, Products and Methodology document. For advice on selecting, managing and implementing products into your organization see the document The BI Survey 12: Best Practices. Document The BI Survey 12 - The Results The BI Survey 12 Trending Topics series The BI Survey 12 Vendor Performance Summaries The BI Survey 12 Best Practices The BI Survey 12 - Sample, Products, Methodology (this document) The BI Survey 12 - KPIs and Dashboards Description Provides an overview and analysis of the most important results of The BI Survey 12 Provides in-depth analysis of the following trending topics: SaaS/Cloud BI, Mobile BI, Big Data, Social BI, Collaboration, Self Service, Analytical Databases A series of short executive reports highlighting all the productrelated results for each vendor. This report features recommendations on how best to choose a BI product and the most important and relevant tool selection criteria. Provides details of the sample and an overview of our methodology including the entire questionnaire and details of our calculation methods. This document outlines the definitions and calculation methods of the KPIs used in The BI Survey 12. Figure 1: Overview of The BI Survey 12 The BI Survey Analyzer is an online tool allowing users to carry out their own analyses of The Survey data. Using results from this year s Survey, the tool also offers users the ability to drill down using product and demographic filters also sees the exciting introduction of a mobile version of The BI Survey Analyzer for the ipad. The mobile BI Survey Analyzer focuses on product-related results based on criteria (KPIs) such as business benefits achieved, implementation costs, functional usage, product competitiveness, innovation, performance, customer satisfaction and agility.

6 6 Figure 2: Screenshot of The BI Survey Analyzer Figure 3: Screenshot of The BI Survey Analyzer ipad App

7 7 Business Benefits and the BBI The BI Survey asks key questions regarding the benefits of BI projects. Respondents are asked to indicate the level of achievement gained from a list of eleven potential benefits. A scoring system is then used to derive a composite weighted score for each benefit, based on the level of benefit achieved. This system is called the BBI (Business Benefits Index). For further information on the calculation methods used, see the Sample, Products and Methodology document. Figure 4 shows the overall breakdown of responses to the Business Benefits questions. The six levels of achievement, along with their weightings, are shown on the horizontal axis. Results are RAG (red, amber, green) rated with green indicating higher scores and red denoting lower scores. Proven and quantified (4961) Proven, but not measured (5643) Formally claimed, but not verified (2186) Informally suspected (2741) Not achieved (2495) Got worse/more expensive (226) Do not know (5024) Weighting Better business decisions 27.9% 39.5% 12.6% 10.7% 1.8% 0.2% 7.4% Faster reporting, analysis or planning 45.2% 35.5% 7.4% 5.0% 2.0% 0.3% 4.6% Improved customer satisfaction 17.7% 29.0% 13.8% 13.8% 5.3% 0.3% 20.1% Improved data quality 29.6% 32.2% 10.6% 10.5% 6.2% 0.5% 10.3% Improved employee satisfaction 22.8% 33.9% 13.5% 13.7% 5.3% 0.7% 10.1% Increased revenues 10.3% 13.3% 11.0% 19.2% 9.0% 0.2% 36.9% More accurate reporting, analysis or planning 33.6% 38.9% 8.9% 8.7% 1.9% 0.4% 7.6% Reduced external IT costs 16.6% 14.2% 7.5% 11.8% 14.7% 3.4% 31.7% Saved business headcount 11.4% 9.5% 5.9% 13.2% 25.7% 0.9% 33.4% Saved IT headcount 11.1% 9.2% 4.9% 9.0% 29.3% 2.9% 33.6% Saved other non-it costs 8.3% 11.4% 7.2% 13.9% 16.7% 0.8% 41.7% Figure 4: Frequency of Business Benefits (n=2116)

8 8 Figure 5 shows business benefits listed in order of the most commonly achieved. Faster Business Intelligence More accurate Business Intelligence Better business decisions Improved data quality Improved employee satisfaction Improved customer satisfaction Increased revenues Reduced external IT costs Saved other non-it costs Saved business headcount Saved IT headcount Figure 5: Benefits overview (n= 2116) Figure 5 Key Findings: Overall, Faster Business Intelligence rated higher than accuracy and more than 80 percent of respondents stated that faster reporting, analysis and planning had been achieved. 72 percent reported that More accurate reporting, analysis and planning had been proven and 67 percent of respondents, up from 59 percent last year, said that Better business decisions was another benefit likely to be achieved. The ability to make better business decisions is a highly desirable benefit. However, it is a benefit that cannot be accurately scoped when developing the business case for a project. While all BI projects would hope to gain this benefit, few projects would be cost-justified against the possibility that it might one day be achieved. Improved data quality is a benefit that can be measured directly. Although often one of the most difficult factors to achieve in a business intelligence project, it ranked fourth in the list with Employee satisfaction following in fifth place, a benefit that typically arises when users have fast access to data and reports. Benefits with the lowest level of achievement were Saving headcount in IT departments, Saving headcount in business departments, Saving non-it costs and Reducing external IT costs. This is in line with our experience that companies are still looking to extend their BI capabilities, and are adding resources to implement and run them. If BI were a mature and saturated market we would see more investment in replacing existing systems but until

9 9 this pattern changes we would not expect savings in headcount and costs to move up the list of benefits. In contrast to previous results, this year the overall BBI decreased slightly from 4.89 to However, we do not think this the beginning of a long-term trend of declining benefits from BI projects. Instead we suspect that this small variation can be accounted for by the fact that The Survey sample varies from year to year. BBI by peer group and vendor The difference between business benefit achievement according to the type of product in use is reflected in the following chart. In order to compare product types we group products into specific peer groups as explained in the Sample, Products and Methodologies document. Visual Analysis & Data Discovery Small & Medium Project Vendor Dashboard Performance Management OLAP Analysis IT Giants BI Giants Large Project Vendor Enterprise Reporting Figure 6: Business benefits analyzed by peer group (n=changing bases) Visual Analysis & Data Discovery tools achieved the best BBI results reflecting their user friendliness and flexibility in allowing end-users to quickly load data, perform user analysis and take advantage of advanced visualization features. The userfriendly nature of these tools relates to business benefits achieved. Small and Medium Project vendors rank second, Dashboard vendors rank third and Performance Management products rank fourth. This is a result we have consistently observed in previous editions of The BI Survey. Specialists, or smaller vendors, tend to show better BBI results than the larger BI Giants, Large Project vendors and Enterprise Reporting vendors. There is a straight correlation between the size of the BI project (measured in data volume used or number of users served) and the size of the tool vendor. With few exceptions, smaller vendors serve smaller projects and products from larger vendors are used in larger projects.

10 10 In our opinion, there are several reasons for the good results achieved by smaller and specialist vendors: Some customers may achieve more business benefits when working with smaller vendors with whom they have a close working relationship, compared to the more arm s-length, impersonal relationship that users tend to have with large vendors. Survey results show that users usually experience better vendor support, satisfaction and recommendation with smaller vendors than with many of the large vendors. The prospect of smaller vendors being removed from a company more quickly if projects are not successful leads to better than average results for the ones that remain in use. Conversely, large vendors are able to keep customers with whom they have long-term, strategic relationships, even if individual projects or products deliver disappointing results. Products from small vendors are more likely to be selected in competitive evaluations. Users expending more effort selecting solutions show a higher level of benefit achievement. This could be because the extra diligence taken to ensure that the product fits their needs increases the probability of the project delivering business benefits. There is also the simple fact that some projects are managed better than others. But, for whatever reason, it is clear that the customers of small vendors such as Yellowfin, Dimensional Insight and Phocas are reporting more business benefits than those of larger vendors like SAP, Oracle, Microsoft and IBM. That said, several products from large vendors have above average BBI scores, most notably the multidimensional databases Oracle Essbase, IBM Cognos TM1 and Microsoft SSAS.

11 11 Figure 7 shows the business benefits scores for all products in the BI Giants peer group. Oracle Essbase IBM Cognos TM1 QlikTech Microsoft SSAS MicroStrategy Information Builders Microsoft SSRS Microsoft Excel Infor IBM Cognos BI SAP BW Oracle OBIEE SAP BO WebI Figure 7: Business benefits for the BI Giants peer group (n=949) The BI Giants peer group includes companies with annual revenues of more than $200 million and a truly international reach. These tools are usually used in enterprise scenarios. Figure 7 Key Findings: The multidimensional database Oracle Essbase ranked first in the BI Giants peer group. All five top ranking products offer in-memory database technology, while the traditional ROLAP tools from SAP, Oracle and IBM deliver the least business benefits. Three of the four multidimensional databases in this peer group (all except Infor) appear in the top five, including QlikTech (with its own in-memory database technology) and MicroStrategy, having recently expanded its caching technology. The correlation between user self service and fast query performance with business benefits has been evident in previous BI Surveys.

12 12 Arcplan 5.24 MicroStrategy Information Builders Microsoft SSRS IBM Cognos BI SAP BW Oracle OBIEE SAP BO WebI Figure 8: Business benefits for the Enterprise Reporting peer group (n=555) The Enterprise Reporting peer group includes products that can provide standard formatted reporting in a large scale enterprise situation. Arcplan, a very flexible tool for designing reporting applications, ranks highest in this peer group, followed by the specialist vendors MicroStrategy and Information Builders. Users of the large vendors SAP, Oracle and IBM report lower business benefits. Yellowfin 6.31 Dimensional Insight Arcplan BOARD QlikTech MicroStrategy Information Builders Decisyon Oracle OBIEE Figure 9: Business benefits for the Dashboard vendor peer group (n=634) The Dashboard peer group includes products that are focused on creating advanced dashboards. Users of tools from small vendors report the highest business benefit scores in the Dashboard peer group. This group is led by Yellowfin, a tool with a modern

13 13 architecture and look and feel, with most of its customers being small companies. Dimensional Insight, another specialist, comes in second, followed by two more small vendors: Arcplan and Board. BOARD TARGIT Oracle Essbase IBM Cognos TM1 Bissantz Microsoft SSAS MicroStrategy Jedox Cubeware Evidanza Microsoft Excel Pentaho Infor SAP BW Figure 10: Business benefits for the OLAP Analysis peer group (n=939) The OLAP Analysis peer group includes products that support analysis in dimensional and hierarchical data models. This peer group is crowded and highly competitive with many vendors scoring good business benefits results. The peer group is led by Board, followed by three products with very similar results: Targit, Oracle Essbase and IBM Cognos TM1. As in all previous editions of The BI Survey, SAP BW users report considerably lower business benefit achievement than any other product s users.

14 14 Dimensional Insight Phocas Tableau Cyberscience QlikTech Figure 11: Business benefits for the Visual Analysis & Data Discovery peer group (n=532) The Visual Analysis & Data Discovery peer group includes products that provide advanced visualization features. Dimensional Insight rates top in this small peer group of specialist vendors, followed by Phocas and Tableau. QlikTech, one of the most successful BI vendors over the last five years and now considered to be a BI giant, is starting to show similar results to the large vendors it used to comfortably beat in this area. BOARD Oracle Essbase IBM Cognos TM1 Jedox Cubeware Evidanza Infor Figure 12: Business benefits for the Performance Management peer group (n=420) Figure 12 Key Findings: The Performance Management peer group includes companies whose products are predominantly used in planning projects. This peer group shows above average business benefits results and is led by BOARD followed by Oracle Essbase and IBM Cognos TM1. Jedox and Cubeware hold fourth and fifth places, also scoring excellent results.

15 15 Yellowfin Dimensional Insight Phocas Tableau BOARD Cyberscience TARGIT IBM Cognos TM1 Bissantz QlikTech Microsoft SSAS Jedox Cubeware Evidanza Microsoft SSRS Pentaho Infor Decisyon Figure 13: Business benefits for the Small and Medium Project vendor peer group (n=1308) Figure 13 Key Findings: The SME Project vendor peer group includes products that are primarily used for small and medium sized projects (<500 users). The sample of Yellowfin s customers in The BI Survey 12 reveals its clear focus on small companies with less than 100 employees. Yellowfin appears to serve its customers well, scoring exceptionally high business benefits, head and shoulders above the competition. Dimensional Insight and Phocas follow in second and third places respectively.

16 16 Arcplan Oracle Essbase MicroStrategy Information Builders IBM Cognos BI SAP BW Oracle OBIEE SAP BO WebI Figure 14: Business benefits for the Large Enterprise vendor peer group (n=535) The Large Enterprise Project peer group includes products that are primarily used for large enterprise projects (>500 users). In the Large Enterprise projects peer group, a leading group of four products scored excellent business benefits: Arcplan, Oracle Essbase, MicroStrategy and Information Builders.

17 Next Steps The BI Survey Analyzer Demo The BI Survey Analyzer is an interactive online tool which enables you to perform your own custom analysis of the survey data. Click here to register for an online demo and find out how effective The BI Survey Analyzer can be in helping you to understand the strengths and weaknesses of BI products and trends in the BI market. Purchase To purchase The BI Survey 12, visit our website or contact us at: Tel: +44 (0)