Analytics A survey on analytic usage, trends, and future initiatives. Research conducted and written by:

Transcription

1 Analytics 2013 A survey on analytic usage, trends, and future initiatives Research conducted and written by: Lavastorm Analytics A global analytics software company that enables a new, agile way to analyze, optimize, and control data and business processes. March 2013

2 Table of Contents I. Executive Summary... 3 II. Methodology... 4 III. Key Takeaways... 4 IV. Description of Survey Respondents... 6 V. Survey Results... 8 VI Predictions VII. About Lavastorm

3 Executive Summary The analytics market is thriving and continues to grow at a rapid place. According to Gartner, the analytics and business intelligence market is forecasted to reach over $17 billion by This year-over-year growth will make a huge impact on how organizations conduct their day-to-day business and measure profitability. From the corner office in a large financial firm to the back office of a communications provider, analytics plays a critical role in any organization s success. Whether utilizing analytics to understand customer behavior, improve marketing effectiveness, manage risks, and improve overall business processes, more businesses are viewing analytics as mission critical. Analytics sets companies apart from their competition, it s no longer just a nice to have; it s a must-have for profitable operations. More than 600 business analysts, technologists, data analytics professionals, managers and C-level professionals across a broad variety of industries including finance, telecom, healthcare, education and computer software to name a few were polled regarding their analytic initiatives and predictions for This research details the challenges and goals that are facing analytic professionals and how they will be investing in the future. The top findings were: Approximately 60% of respondents plan to increase their analytic investments this year. The key areas of investment will be: People 53% Tools 51% Training 43% According to the respondents, their three main challenges are: Gleaning insights from data - 25% Getting access to the data 22% The ability to integrate and manipulate data 19% The most likely developments to affect the analytic community in 2013 are expected to be: An increased use of unstructured data or new data sources A shortage of analytic professionals The need for analytic tools and training 1 Forecast: Enterprise Software Markets, Worldwide; update, Gartner, December 10,

4 Methodology In order to get the most accurate insight into the market, we conducted our research within the active online analytic communities LinkedIn s Lavastorm Analytics Community Group, Data Science Central and KDNuggets with a global reach of over 75,000 analysts that offer a diverse membership of analytic professionals in varied verticals. We used an online survey tool to conduct this analytic survey. More than 600 business analysts, technologists, data analytics professionals, managers and C-level professionals were polled across a broad variety of industries including finance, telecom, healthcare and computer software regarding their analytic initiatives and predictions for Every survey participant was encouraged to answer every question. Key Takeaways: Approximately 60% of respondents are looking to increase their analytic investment this year. Decreasing investment Staying the same Not sure 10% 5% 27% 58% Increasing investment The key areas of investment: People 53% Tools 51% Training 43% Data Sources 30% None 10% The top areas of analytic investments: Predictive analytics 51% Big data 35% Dashboards 32% Reporting 31% Data exploration and discovery 30% Advanced Visualization tools 28% The biggest struggle with analytics: Gleaning insights from their data 25% Access to the data 22% Ability to manipulate and integrate their data 19% Resources 15% Requirements gathering 10% 4

5 The survey showed that the analytics industry remains extremely vibrant, with nearly 60% of respondents indicating that their company plans to increase their analytic investments in 2013 across a number of areas and tools. This could show that while people in the survey already use three self-service analytic tools on average, their current toolsets are not meeting the top three challenges of conducting complex analytics: gleaning insights from data, access to data, and the ability to integrate and manipulate data. The increased investment in 2013 is expected to be used as often for people as it is for tools. However, despite the increased investment, the survey respondents expect a shortage of analytic skills to have an impact on the industry this year. Perhaps this indicates that despite additional investment, analytic professionals expect that the demand for analytics will outpace analytic capacity for the organization. New data sources, including unstructured data, are expected to have the biggest impact on the analytics community this year. Since Big Data analytics often require organizations to not only process more data, but also to analyze additional data sources, the emphasis on new data sources is probably confirmed by the fact that 35% of respondents will put additional investment toward Big Data analytics this year. Related to self-service tools, the survey not only showed that Microsoft Excel remains the major self-service analytic tool, but that the tool landscape is quite fragmented with more than 25 in use within this community. In fact, 17 of the tools mentioned in the survey were used by fewer than 10% of the respondents. This is likely due to analyst choosing their own tools to suit their needs a trend which we see continuing as analysts seek solutions for the data integration, manipulation, and analysis challenges they still face. 5

6 Description of Survey Respondents The following charts depict the background of those professionals who completed the survey by role, department and industry. How would you classify your role? Other 3% Mgr./Director/ C-Level 20% Analysts 37% Student/Academia /Research 5% Reporting/BI 7% Technical Lead 9% Data Scientist/ Statistician/Modeler 19% 568 Survey Responses Other includes: Auditor, Risk and Consulting. What department do you reside in? Billing 1% Other 19% Research 18% Sales 4% Engineering 7% IT 16% Finance 8% Operations Marketing 577 Survey Responses 11% 16% Other includes: Analytics, Human Resources, Audit, Revenue Assurance, Credit, Prof Svcs. 6

7 Description of Survey Respondents What is your company s primary industry? Utilities 0.5% Transportation 0.9% Government 1.4% Energy 3.4% Retail 3.7% Insurance 3.9% Manufacturing 4.6% Other 16.3% Analytics 12.9% Computer Software 11.3% Finance 11.1% Data Solutions 5.1% Education Telecom 9.7% Healthcare 6.7% 8.5% 566 Survey Responses Other includes: Pharmaceutical, Consulting and Media 7

8 Survey Results Which of the following functions do you personally perform as a regular part of your job? Data analysis 81.8% Statistical modeling 54.8% Research and development 49.1% Import data from a data warehouse (or from IT) 47.0% Publish or otherwise provide data to other departments 46.0% Combine data from a warehouse with other data not in the warehouse 41.8% Filter or drill down into dashboards/ interactive reports 41.7% Quality control 21.6% Other 7.8% None 1.3% 8

9 Survey Results What is your biggest challenge in analytics today? Other Publishing results 4.6% 3.7% Requirements gathering 10.2% Gleaning insights from the data 24.8% Resources 15.3% Manipulating/ integrating data 19.4% Access to the data 21.9% 588 Survey Responses Other includes: Accuracy, Explaining the results, Getting Executive buy-in, Lack of tools, Bridging gap between analytics and the business, Technology that can handle volume of data, Developing and acceptance of analytic-driven culture, Finding the right people that can convert insights into action, People who design the data warehouses/databases not having a clear understanding of what is needed in the business. Answer Options Response Percent Response Count Gleaning insights from the data 24.8% 146 Access to the data 21.9% 129 Manipulating/integrating data 19.4% 114 Resources 15.3% 90 Requirements gathering 10.2% 60 Publishing results 3.7% 22 Other 4.6% 27 answered question 588 skipped question 15 9

10 Survey Results How would you describe your company s investment in analytics? Decreasing investment 5.2% Not sure 10.1% Staying the same 26.7% Increasing investment 58.0% Answer Options Response Percent Response Count Increasing investment 58.0% 345 Staying the same 26.7% 159 Decreasing investment 5.2% 31 Not sure 10.1% 60 answered question 588 skipped question 15 10

11 Survey Results In which areas will you be investing more resources? 53.4% 51.0% 42.9% 30.4% 9.7% 1.2% People Tools Training Data Sources None Other 588 Survey Responses Other includes: Investing in learning existing tools, budget concerns override investment, business development, not in my personal scope. Answer Options Response Percent Response Count People 53.4% 314 Tools 51.0% 300 Training 42.9% 252 Data Sources 30.4% 179 None 9.7% 57 Other 1.2% 7 Other (please specify) 8 answered question 588 skipped question 15 11

12 Survey Results What self-service analytic tool are you currently using? Excel R SAS MS Access SPSS Tableau Sql Python Business Objects StatSoft/STATISTICA SAP Matlab QlikView Microstrategy Cognos Teradata Lavastorm Toad Microsoft BI Rapid Miner TIBCO Spotfire ACL Hive Birt Logix Other 27.5% 22.8% 17.5% 16.3% 14.6% 11.5% 9.8% 8.5% 7.6% 7.0% 5.3% 4.6% 4.1% 3.4% 3.2% 2.9% 2.7% 2.5% 1.5% 1.0% 0.2% 0.2% 15.6% 35.3% 34.1% 75.6% 589 Survey Responses Other includes: Stata, Open Source, Mathmetica, Minitab, Toad, Homegrown, Knime, Datameer, Polyanalyst, Cetas, Teradata, Octave, Socrata, Java, Alteryx. 12

13 Survey Results What self-service analytic tool are you currently using? Answer Options Response Percent Response Count Excel 75.6% 445 R 35.3% 208 SAS 34.1% 201 MS Access 27.5% 162 SPSS 22.8% 134 Tableau 17.5% 103 Sql 16.3% 96 Other 15.6% 92 Python 14.6% 86 Business Objects 11.5% 68 StatSoft/STATISTICA 9.8% 58 SAP 8.5% 50 Matlab 7.6% 45 QlikView 7.0% 41 Microstrategy 5.3% 31 Cognos 4.6% 27 Teradata 4.1% 24 Lavastorm 3.4% 20 Toad 3.2% 19 Microsoft BI 2.9% 17 Rapid Miner 2.7% 16 TIBCO Spotfire 2.5% 15 ACL 1.5% 9 Hive 1.0% 6 Birt 0.2% 1 Logix 0.2% 1 Other 15.6% 92 Other (please specify) 117 answered question 589 skipped question 14 13

14 Survey Results What area is your company investing in 2013? Predictive Analytics 50.8% Big Data Dashboards Reporting Data exploration and discovery Advanced Visualization tools Performance Analytics Self-service analytic tools for business users Web Analytics Text Analytics New Databases (including columnar, non-relational) Process Analytics None Location Analytics 34.5% 32.0% 30.8% 29.3% 27.4% 26.7% 24.4% 23.5% 20.8% 16.2% 16.1% 10.0% 7.8% 591 Survey Responses Streaming Analytics 7.3% OLAP 6.1% Other includes: Machine learning, Mobile analytics, Demand forecasting and revenue optimization, ERP, Streaming data analytics, Speech Search analytics. interfaces Other 6.1% 3.9% 14

15 Survey Results What area is your company investing in 2013? Answer Options Response Percent Response Count Predictive Analytics 50.8% 300 Big Data 34.5% 204 Dashboards 32.0% 189 Reporting 30.8% 182 Data exploration and discovery 29.3% 173 Advanced Visualization tools 27.4% 162 Performance Analytics 26.7% 158 Self-service analytic tools for business users 24.4% 144 Web Analytics 23.5% 139 Text Analytics 20.8% 123 New Databases (including columnar, non-relational) 16.2% 96 Process Analytics 16.1% 95 None 10.0% 59 Location Analytics 7.8% 46 Streaming Analytics 7.3% 43 OLAP 6.1% 36 Search interfaces 6.1% 36 Other 3.9% 23 Other (please specify) 23 answered question 591 skipped question 12 15

16 2013 Predictions 26.9% Increase use of unstructured data or new data sources 25.2% Shortage of analytic professionals 17.7% Need for analytic tools and training 588 Survey Responses 15.3% Shift analytic roles from IT to the business 13.3% Analytics centralized within an organization 1.7% Other Other includes: analytics decentralized, Shake out of Big Data hype fades, increased outsourcing of analytic professionals by large corporations, drawing analytic professionals from other areas of the organization, skills for analysts becomes company wide. Answer Options Response Percent Response Count Increase use of unstructured data or new data sources 26.9% 158 Shortage of analytic professionals 25.2% 148 Need for analytic tools and training 17.7% 104 Shift analytic roles from IT to the business 15.3% 90 Analytics centralized within an organization 13.3% 78 Other 1.7% 10 answered question 588 skipped question 15 16

17 About Lavastorm Lavastorm Analytics is a global analytics software company that enables a new, agile way to analyze, optimize, and control data and business processes. The company s products provide analysts with programming powers previously limited to IT to rapidly unify disparate data, easily construct complex analytics, and effectively deliver actionable insight and results. Analytics built in Lavastorm can easily be deployed as persistent solutions and applications, providing continuous analytic monitoring across multiple systems and complex, data-intensive processes. Lavastorm s solutions have identified business improvements worth billions of dollars for some of the largest corporations in the world. For more information on Lavastorm Analytics or to download a desktop edition of the Lavastorm Analytics Engine, our data analytics software for business analysts, please visit or respectively. 17

18 Lavastorm Analytics, All rights reserved.