BIG DATA big.data
For most organisations, big data is now the reality of doing business. Technological and social innovations are resulting in huge flows of new data every day. As we enter this undeniable era of big data where more information will be captured in ever-finer detail, and from more sources than ever before, does that mean our decision-making will improve? 02
It s certainly clear that enthusiasm for the idea of big data is high, with 87% of respondents say that at least one of their organisation s departments are keen to have access to business/analytics tools and 82% of ITDMs saying it will give them a competitive advantage. Indeed, demand is almost universal across departments with just 1% who say that big data is not wanted by any in their organisations. 87% 03
Early adoption has been significant with 40% already having big data, a figure that predictably rises to nearly 50% in those larger organisations with more than 3000 employees. Perhaps as a result of their established data manipulation processes, organisations in the Financial Services and Manufacturing sectors have been quicker to perform big data analysis with 50% doing so. As have B2B companies which could be because their customer base is more concentrated and more easily defined lending itself more naturally to analysis. Within the next 3 years a further 44% intending to adopt big data and 82% agree that most organisations will be doing big data analysis in 10 years time, so big data is definitely here to stay. But why are organisations implementing big data and is it delivering the expected results? 04
Expectations are actually quite parochial when it comes to the reasons why organisations are implementing big data projects. The more nebulous aims of improving customer experience and understanding the customer better both scored higher at 44% and 42% respectively than the idea of hard financial gain increasing revenue (37%). 1 2 3 And when we look at those who actually have responsibility for big data analysis tools we find that the customer is very much king, with just over half of respondents citing improving the customer experience as their reason for adopting big data. But is the promise of big data delivering in reality? It s definitely proving more complex to implement than anticipated although there s no single clear-cut reason for it. That said, the most common issues relate to expense. 05
Of those who found big data more expensive than expected, just under a third found that their infrastructure required more changes or updates than expected, nearly the same number again incurred unexpected costs of buying new hardware in order to implement big data and a quarter had incurred unexpected costs of buying new software. Indeed cost has proved to be a significant issue with 49% saying big data has been more expensive than expected. That figure rises to 58% in the larger organisations where early adoption has been higher and a staggering 73% in B2B organisations. 31% 29% 26% 06
Perhaps this is why lack of budget is the most common reason given for why organisations are not currently using big data. 41% of those not yet using big data, and not intending to use it in the near future, say that they simply don t have the budget for it with the problem more pronounced in smaller organisations where that figure rises to 47%. Furthermore, just over a third say budget is also the most common reason preventing IT departments from providing high quality big data analysis and 65% of those using or intending to use big data agree that the requests made by other departments regarding big data are not possible due to budget. Notably those with responsibility for big data, and therefore real experience of these issues, are more likely to completely agree that departments are making impossible requests. 07
So the idea that big data is just out there waiting to be used isn t strictly true, there s a cost involved that even those used to analysing data are finding prohibitive. Even with the required budget available, finding the right people able to use big data tools and make sense of the answers generated could prove the biggest barrier to organisations benefiting from their big data. 31% 1000-3000 employees 3000+ employees 18% Of those in smaller organisations (those with 1000-3000 employees) who are finding big data more expensive than expected, there is already a clear skill shortage when it comes to implementing big data: 31% of these companies (nearly double that of the larger organisations) are finding that hiring the right people took longer than expected. For those natural data manipulators in Financial Services this was a significant issue with half struggling to find the right staff. 08
And analysing that data is also proving difficult. A lack of specialist skills is affecting the quality of data analysis with 33% reporting this is a problem for their organisation. What s more, the problems don t stop there because 45% have issues with the data they actually have managed to gather a figure that rises to 60% in B2C organisations. But it was when we looked closely at extracting real value from big data that reality most sharply departed from theory. Almost 60% are aware that some departments within their organisation have complaints about their big data tools. Even in the Financial Services sector, where data analysis is more established, a significant 44% are unhappy with big data tools and in B2C organisations this rose to 70% having departments who complain. 09
A significant number (36%) said the insights big data gave them were not reliable, rising to 53% for data users themselves who reported a variety of issues with the data. The most commonly cited complaint of those experiencing data quality issues is the changing format of data, reported by a third of respondents. But data manipulation requirements (27%), poor data quality (25%) and out of date data (24%) also feature prominently, showing that tools are often fighting a losing battle with incomplete and inconsistent data. With 31% reporting issues with the amount of time it takes to run analysis and 30% finding the tools difficult to use without training it s no wonder that 83% using or planning to use big data soon agree that big data is difficult to implement for all data sources. 10
Despite this data users can still clearly recognise the benefits of big data with a convincing 82% agreeing that big data would be beneficial if all the data could be analysed. So even given the data, they cannot extract the value in it. No wonder then that nearly 80% of those performing big data analysis agree that big data is underused.? However, whether it was the lack of skills that 1/3 report is stopping this happening, or other inherent shortcomings, only 16% are prepared to say that their big data tools are of high quality and 61% say that the information gathered from big data is difficult to action with those actually using the data tools most likely to agree. 11
Overall 59% say their requests for business analysis/tools are unfulfilled. This is borne out when we ask if departments have access to the analytics tools they d like. Somewhat inevitably IT departments are the most likely to be satisfied but even then only 37% of them have access to all the tools they would like. In all other departments only 1/3 or less have access to all the tools they would like, with 14% overall having no departments with access to all the tools they would like. 12
Big data needs to go through IT to ensure the involvement of all departments but this is causing conflict and delay so it s no major surprise that the IT department is feeling the brunt of this frustration. 44% of big data users say that departments in their organisation are unhappy that IT departments have control over big data with the issue more pronounced in B2C organisations where 55% cite this as a problem. Although IT departments in the B2B sector appear to have more control with 72% confident that no departments have rolled out a big data project on their own this is not the case in other sectors. Overall 33% say departments in their organisation have rolled out a big data project without IT support. 13
A worrying 23% believing that big data projects outside of IT are not a data security issue but security of big data is a big concern. The collection of masses of information could lead to a legal mess for companies based in the E.U. as more than 80 countries now have data privacy laws* and the European Union defines seven safe harbor privacy principles for the protection of E.U. citizens personal data. Another barrier to the success of big data could be because as yet it has failed to win the backing of the boardroom. Overall, nearly 50% agree that the Board do not regard big data as important with this more noticeable in B2C and the Retail, Distribution & Transport sector. Those actually tasked with using the data tools are the most likely to feel this lack of support. Can big data deliver without this support? * ico.org.uk/for-organisations/guide-to-data-protection/principle-8-international 14
Like most technological developments it s tempting to assume big data is the panacea but big data does not automatically mean bigger and better information. Undoubtedly, there is a real commercial advantage to be gained from big data but organisations need to ensure they have an agreed strategy and the right elements in place to make the most of these opportunities. Business leaders must take control and recognise the requirement to manage the volume of data; fully access the commercial insights; and secure the right skills. In big data, as in almost all areas of IT and business at large, technology alone will never be enough. 15
For more information on our services, to read case studies or access lots of research content visit our website: www.vansonbourne.com If you d like to know more, contact us here: Email: enquiries@vansonbourne.com Tel: 01635 550449 or follow us on: www.twitter.com/vansonbourne www.linkedin.com/company/vanson-bourne www.vansonbourne.com/news/our-blog 16