Irish Data Analytics Landscape Survey Analysis

Transcription

1 Irish Data Analytics Landscape Survey Analysis APRIL 2015

2

3 TABLE OF CONTENTS CONTENTS Executive Summary 1 Introduction 3 Survey Method 3 Analysis of Survey Responses 4 Summary 14 Contact Information 15

4

5 EXECUTIVE SUMMARY EXECUTIVE SUMMARY Analytics and Big Data have been receiving a lot of press attention recently - ranging from attempts to describe and define them, to assessments of how organizations can monetise their data. Within an Irish context questions such as the following are being asked: How big is the average Irish analytics team? What industries in Ireland are using analytics? Are companies in Ireland planning to expand their analytics teams in 2015? While there have been a number of European and international surveys of the analytics industry there has not been a specifically Irish survey taking the pulse of the analytics industry in Ireland. For this reason The Analytics Store launched the Irish Data Analytics Landscape Survey in December SURVEY HIGHLIGHTS The survey was conducted online in December Overall there were 94 survey responses, 75 of which completed the survey. We omit incomplete responses, so include 75 responses in our analysis. Survey Par*cipant Profile Almost 70% of participants use analytics either frequently or almost always in their decision making processes. This shows that the majority of participants in the survey are from organizations currently using analytics. The survey responses came primarily from people in large companies (56%), with 15% coming from medium sized companies and 29% from small companies. The majority of participants (over 75%) came from the banking and finance, IT, and services sectors. Organising Analy,cs Highlights Most participants are using analytics to ensure greater accuracy in decision making (68%) and/or to remove "gut instinct" from decision making (45%). Implementing successful data analytics projects is not without its challenges, and the most common challenge reported is the difficulty in hiring suitably qualified staff (45%), followed by insufficient relevant data (35%). Where to house an analytics team is a perennial question at analytics conferences. In this survey 34% of participants worked in organisations where analytics belonged to a line of business, 21% worked in organisations in which analytics belonged to IT, and 10% worked in organisations in which analytics belonged to a specific analytics team. It is promising for the Irish analytics industry that over half of the people surveyed work in companies that plan to hire new analytics staff in the coming year. The Analytics Store

6 EXECUTIVE SUMMARY Analy&cs Applica&ons Highlights Sales and marketing dominate the use of analytics, followed by finance and operations, which is consistent with international surveys. It is interesting to see that 15% of the survey participants worked in organisations at which analytics was being used in human resources. This is a growing trend that is also seen internationally. Almost all participants used reporting, and over two thirds performed statistical and exploratory analysis. Over half of the participants reported using some aspects of advanced analytics (for example predictive modelling or forecasting), which speaks to the maturity of the practice of analytics in Irish companies. Participants primarily reported using transactional data in their analytics work. The use of unstructured data (for example text, audio, images or video) is growing with 20% of respondents reporting its use. SQL and Excel are the most commonly used tools, and remain workhorses on most analytics projects. After this there were a wide spread of tools, with the open source programming language R being the next most commonly used tool. For advanced analytics tools there seems to be an even balance between GUI- based tools (for example IBM SPSS, SAS Enterprise Miner and RapidMiner), and programming languages (for examples Base SAS, R and Python). 28% of respondents had used one or more Big Data specific tools. The most commonly used Big Data tools were Hadoop, Hive, and Spark. The Analytics Store

7 INTRODUCTION INTRODUCTION How big is the average Irish analytics team? What industries in Ireland are using analytics? Are companies in Ireland planning to expand their analytics teams in 2015? While there have been a number of European and international surveys of the analytics industry 1,2,3 there has not been a specifically Irish survey taking the pulse of the analytics industry in Ireland. For this reason The Analytics Store launched the Irish Data Analytics Landscape Survey in December This document describes an analysis of the results of this survey. SURVEY METHOD The survey was conducted online in December 2014 through the SurveyMonkey platform, and promoted primarily using social media. Figure 1 shows a screenshot of the survey interface. The survey contained 16 questions, only a small number of which were mandatory. Depending on the answers that were given to certain questions participants were guided through different routes through the survey. Overall there were 94 survey responses, 75 of which completed the survey. We omit incomplete responses and so include 75 responses in our analysis. Figure 1: A screenshot of the survey interface. 1 PwC s Global Data & Analytics Survey 2014: Big Decisions and- analytics/big- decisions- survey/index.jhtml 2 NewVantage Partners 2014 Big Data Executive Survey leadership/publications- and- executive- surveys/ 3 BARC s Big Data Analytics 2014 Survey research.com/research/big- data- analytics- 2014/ The Analytics Store

8 ANALYSIS OF SURVEY RESPONSES This section analyses the responses of participants to the survey. The survey was broken down into thematic sections and responses within each section are analysed separately. CHARACTERISTICS OF SURVEY PARTICIPANTS The first four mandatory questions in the survey were designed to characterise the survey participants, and the companies in which they worked. Figure 2 and Figure 3 characterise the companies in which participants worked. Two interesting things stand out. First, the survey responses came primarily from people in large companies. Second, the majority of participants (over 75%) came from the banking and finance, IT, and services sectors. This is mostly due to the networks within which the survey was promoted, but is also indicative of the fact that analytics is primarily practiced within larger organisations and in particular industries. The other industries represented in the survey responses were hospitality and events, media, and gaming. What size is your organisation? 60% 56.0% 50% 40% 30% 20% 10% 0% 29.3% 14.7% Small Medium Large Figure 2: Question 1 - What size is your organisation? We defined small companies as having less than 20 employees, medium companies as having between 20 and 249 employees and large companies as having 250 or more employees. (Responses: 75) The Analytics Store

9 In which sector does your organisation operate? Banking & Finance 28.4% IT Services 24.3% 23.0% Other Public Sector 6.8% 6.8% Retail & Wholesale Telecommunications Manufacturing 2.7% 4.1% 4.1% 0% 5% 10% 15% 20% 25% 30% Figure 3: Question 2 - In which sector does your organisation operate? (Responses: 75) The third question in the survey asked participants to characterise their own roles at the company in which they worked. Figure 4 summarises participants responses. Half of the participants were in roles that mixed technology and business, with the remainder split between solely business roles, solely technology roles and other roles (which included finance, communications, and education). We believe that this means that the survey respondents were in a position to offer an interesting mix of views from both business- focused and technology- focused points of view. The Analytics Store

10 What best describes your role at your organisation? 60% 50% 40% 30% 20% 10% 0% 50.7% Business & Technology 20.0% 20.0% 9.3% Business Technology Other Figure 4: Question 3 - What best describes your role at your organisation? (Responses: 75) The final question in the first part of the survey asked users to characterise the use of data analytics at their companies. Figure 5 summarises participants responses. Only 7 of the survey participants (9.3%) worked in companies where data analytics was not used at all. These participants were redirected to the end of the survey and did not answer any of the other questions. The remainder of the survey participants were using analytics, at least to some extent, which validated their participation in the survey. Does your organisation use data analytics techniques to drive decision making? 50% 40% 41.3% 30% 24.0% 25.3% 20% 10% 0% 9.3% Never Rarely Frequently Almost always Figure 5: Question 4 - Does your organisation use data analytics techniques to drive decision making? (Responses: 75) The Analytics Store

11 ORGANISING ANALYTICS The next section of the survey asked participants about how analytics was used at their organisations. This began by asking what the major drivers for using analytics were. Figure 6 summarises responses to this question (participants were allowed to select multiple answers). Most participants were interested in improving decision making, either selecting the Ensuring greater accuracy in decision making option or the Removing "gut instinct" from decision making option 4. This is in line with international survey responses. What are the major drivers of the use of data analytics for decision making in your organisation? Ensuring greater accuracy in decision making 68% Exploiting our data resources 55% Removing "gut instinct" from decision making Competitive advantage 38% 42% Regulatory compliance Demonstrating our capacity for innovation 23% 20% 0% 20% 40% 60% Figure 6: Question 7 - What are the major drivers of the use of data analytics for decision making in your organisation? (Responses: 66) Implementing successful data analytics projects is not without its challenges and Question 8 in the survey addressed this. Figure 7 summarises the responses to this question. It is interesting that the most common challenge faced is the difficulty in hiring suitably qualified staff. Although there has been a growth in third level analytics courses (for example at UCD 5 and DIT 6 ) qualified, experienced analytics practitioners are still thin on the ground. It is likely that this will continue for a number of years until the current pool of graduates gain relevant experience. It is also interesting that, in spite of all we hear about the deluge of data facing us, 35% of participants saw insufficient relevant data as a challenge to their analytics projects. Relevant is the key word here. It is very often the case that although masses of data are available in organisations, there is a dearth of clean, recent, appropriate data for analytics projects. Finally, it is worth noting that only 7 participants (11%) mentioned data protection issues. We expect this to become a bigger 4 Of the 28 participants (42%) who selected the Removing "gut instinct" from decision making option 17 also selected the Ensuring greater accuracy in decision making option and 11 did not. Taking these 11 participants together with the 45 participants (68%) who selected Ensuring greater accuracy in decision making this means that 56 participants (85%) were interested in improving decisions making The Analytics Store

12 challenge in the future as awareness of data protection and privacy issues grows in the public consciousness. What challenges did you face in undertaking data analytics projects this year? DifZiculty hiring suitable staff 45% InsufZicient relevant data 35% Cost DifZiculty accessing suitable tools Lack of corporate sponsorship 23% 22% 25% Data protection issues 11% Lack of compelling business case 5% 0% 10% 20% 30% 40% Figure 7: Question 8 - What challenges did you face in undertaking data analytics projects this year? (Responses: 65) Questions 9 and 10 in the survey asked participants about the size of the analytics teams in which they worked and whether they intended to grow these teams in the coming year. Figure 8 summarises the responses to these questions. It is interesting that in the survey participant pool 26 participants (38%) work in organisations with large analytics teams (more than 10 team members). Having unpacked this responses a little, most of these respondents work either in analytic services companies or at large companies in the banking and finance sector. This is not unexpected as most banking and finance organisations have large risk teams, which do a lot of analytics work. It is promising for the Irish analytics industry that over half of the people surveyed work in companies that plan to hire new analytics staff in the coming year. A word of caution is required, however, given the fact that access to qualified staff is the number one challenge faced by practitioners. The Analytics Store

13 What is the size of the data analytics team at your organisation? Do you plan to hire additional data analytics staff in 2015? 43% 40% 38% 40% 37% 30% 30% 20% 10% 13% 19% 15% 15% 20% 10% 12% 7% 0% > 10 None 0% No > 10 (a) Figure 8: (a) Question 8 - What is the size of the data analytics team at your organisation? (Responses: 68) (b) Question 9 - Do you plan to hire additional data analytics staff in 2015? (Responses 67) Where to house an analytics team is a perennial question at analytics conferences, and Question 11 in the survey explored this. Figure 9 summarises the results. 34% of participants worked in organisations where analytics belonged to a line of business. In fact most of the other responses actually referred to a line of business so this number is really closer to 45%. This is the classic siloed approach that tends to emerge as a company begins to use analytics, and is also common internationally. Analytics also remains something associated with IT, and 21% of respondents work in companies in which this is the case. It is promising, however, that in 10% of organisations analytics belonged to a specific analytics team, which suggests that these companies are moving towards a scenario in which analytics is recognised as a function in its own right. Which department at your organisation controls data analytics? (b) Line of Business 34% IT 21% BI Team Other 15% 16% SpeciZic Analytics Team 10% Data Architecture Team 4% 0% 10% 20% 30% Figure 9: Question 11 - Which department at your organisation controls data analytics? (Responses: 68) The Analytics Store

14 ANALYTICS APPLICATIONS The next section of the survey asked participants about the ways in which they were using analytics at their companies. The first question in this section, Question 12, asked about the departments that utilised the outputs of analytics efforts. Figure 10 summarises these results. Sales and marketing dominates the use of analytics, followed by finance and operations. The prevalence of sales and marketing departments as consumers of analytics is consistent with international surveys. It is interesting to see that 9 of the survey participants (15%) worked in organisations at which analytics was being used in human resources. This is a growing trend that is also seen internationally. Which departments at your organisation utilise the outputs of data analytics projects? Sales & Marketing 75% Finance Operations 62% 62% Research & Development IT Production Other 30% 27% 23% 22% Human Resources 15% 0% 10% 20% 30% 40% 50% 60% 70% 80% Figure 10: Question 12 - Which departments at your organisation utilise the outputs of data analytics projects? (Responses: 60) It is also interesting to consider the types of analytics that people are doing and the types of data they are working with. Questions 13 and 14 dealt with this, and the results are summarised in Figure 11 and Figure 12. Almost all participants used reporting, which is not surprising, and over two thirds performed statistical and exploratory analysis. It is encouraging to note that over half of the participants reported using some aspects of advanced analytics (for example predictive modelling or forecasting). This speaks to the maturity of the practice of analytics in Irish companies. The Analytics Store

15 Which of the following data analytics techniques has your organisation used in projects this year? Reporting 92% Statistical analysis Exploratory data analysis Predictive modelling Forecasting Segmentation 68% 65% 58% 57% 55% Text analytics Association analysis 23% 23% Other 3% 0% 20% 40% 60% 80% 100% Figure 11: Question 13 - Which of the following data analytics techniques has your organisation used in projects this year? (Responses: 60) Participants primarily reported using transactional data in their analytics work. Again this is in line with international results. The use of unstructured data (for example text, audio, images or video) is growing in the international analytics community and was relatively well represented here with 12 respondents (20%) reporting its use. The Analytics Store

16 What types of data did you use in data analytics projects this year? Transactional data 85% Log data Social media data 32% 28% Unstructured data Sensor data 15% 20% Other 7% 0% 20% 40% 60% 80% 100% Figure 12: Question 14 - What types of data did you use in data analytics projects this year? (Responses: 60) Question 15 in the survey asked respondents which tools they were using for analytics projects - their responses are summarised in Figure 13. SQL and Excel were the most commonly used tools, and remain workhorses on most analytics projects. After this there were a wide spread of tools, with the open source programming language R being the next most commonly used tool. For advanced analytics tools there seems to be an even balance between GUI- based tools (for example IBM SPSS, SAS Enterprise Miner and or RapidMiner), and programming languages (for examples Base SAS, R and Python). The tools mentioned more than once under the other category were Qlikview and Teradata. The Analytics Store

17 Which of the following data analytics techniques has your organisation used in projects this year? Excel 74% SQL 58% R SAS (Base) SAS (Enterprise Miner) Microsoft SQL Server Tableau 37% 34% 32% 29% 26% Python Other IBM SPSS 15% 18% 18% Oracle Data Miner Matlab SPSS Rapidminer SAP KNIME Weka 6% 6% 6% 6% 6% 3% 3% 0% 20% 40% 60% 80% Figure 13: Question 15 - Which of the following data analytics tools or languages did your organisation use in projects this year? (Responses: 62) The final question in the survey asked participants whether they had used so- called big data tools in an analytics projects this year (a list of the following tools was included: BigML, Cassandra, Giraph, Hadoop, HBase, Hive, Mahout, Pig, and Spark). Figure 14 summarises the responses. 21 respondents (28%) had used one or more of these tools. This suggests that people are starting to work with datasets outside of the typical small transactional data that characterises early use of analytics. The most commonly used big data tools were Hadoop (15 respondents), Hive (5 respondents), and Spark (3 respondents). It will be interesting to see if the use of these types of tools increases in future iterations of this survey. The Analytics Store

18 SUMMARY Were "big data" speciwic tools or languages used by your organisation in projects this year? Yes 28% No 72% Figure 14: Question 16 - Were "big data" specific tools or languages used by your organisation in projects this year? (Respondents: 75) SUMMARY The purpose of this survey was to capture the data analytics landscape in Ireland in The number of survey respondents was not high enough to say that the results are fully representative of the entire analytics industry, but they are sufficient for interesting analysis and present a snapshot of the current state of the industry, and a baseline for comparison of future studies. The survey results paint a picture of a reasonably mature analytics industry, with many participants reporting analytics applications moving beyond descriptive analytics to the application of advanced analytics techniques (such as predictive modelling), which in some cases use interesting unstructured data sources and high- end big data toolsets. There is also some evidence for the emergence of strong, centralised analytics teams - although in most cases analytics work is siloed in a line of business. The main challenge facing analytics practitioners is a lack of experienced, qualified candidates to fill analytics roles. It is expected that this will continue to be the case for the immediate future. In- house training of existing staff is likely to be a solution to this issue for many companies. It is somewhat surprising that, in spite of the data deluge we constantly hear about, a lack of relevant data continues to be an issue for some practitioners. This is likely to continue to be the case and highlights the importance of strong data management and governance. As we repeat this survey in the coming years it will be interesting to benchmark against these results The Analytics Store

19 CONTACT INFORMATION CONTACT INFORMATION Company: Address: Tel: Website: The Analytics Store 12 Merrion Square, Dublin The Analytics Store