Big Data Tips the Power Balance Between IT and Business Users

Transcription

1 I D C A N A L Y S T C O N N E C T I O N Carla Arend Programme Director, European Software IDC EMEA Big Data Tips the Power Balance Between IT and Business Users Sponsored by: Computacenter November 2012 Introduction: From the Information Economy to the Intelligent Economy IDC believes that the industry is in the midst of a "once every years" shift to a new technology platform for growth and innovation we call it the third platform (the first being mainframe and second being client/server). This platform is built to support mobile devices and apps, cloud services, mobile broadband networks, social technologies and of course Big Data. This IDC paper explains the basic concepts of Big Data and looks at current adoption, benefits, challenges and emerging best practices as well as the impact that Big Data has on the power balance between IT executives and business users. At IDC, we have termed the basis of this third platform the shift from the information economy, where large amounts of information are broadly accessible through the Internet, to the intelligent economy, where the key to success is the ability to analyse and act upon new data sets to create competitive advantage. Indeed these are exciting times in the information management world, and the era of Big Data has truly arrived social media interactions, real-time sensory data feeds, geospatial information and other new data sources are presenting organisations with a range of challenges, but also significant opportunities. Perhaps a more important trend to note as we move to this intelligent economy is the way business users are increasingly setting the pace for the IT organisation. They want to consume new applications on a variety of devices, and they want to make informed decisions based on granular and real-time analysis of data. Consequently, a large share of IT budgets is controlled by business users to ensure alignment of business requirements and IT capabilities. In fact, IDC's 2012 European software survey revealed that in 66% of European organisations, the IT budget is controlled by the business units. IT organisations need to treat their business users as customers who are measuring the success of IT projects through business metrics like the quality of service and return on investment achieved. Increasingly, business users have the choice to source IT services from their internal IT departments or from cloud service providers. In order to stay relevant, IT organisations need to be closely aligned with their business users, and 99% of European IT executives have made business alignment their first priority for 2012, according to IDC's European software survey. The advent of Big Data magnifies this trend. Business users are struggling to make sense of the data and try to find the patterns that enable them to create new products and services for their

2 customers. At the same time, IT executives are challenged to provide a powerful infrastructure that can capture, manage and store this data as efficiently as possible. What is Big Data? There are many opinions formed about what Big Data is. IDC commonly describes Big Data using the following four elements: Volume. The challenge of handling ever-larger data volumes is nothing new for storage administrators. However, Big Data might actually drive companies to the limits of their current architecture faster. Variety. Big Data enables organisations to analyse data that has been generated outside of the organisation, such as social media data, as well as data that is generated from sensors, point of sales systems, RFID tags, video surveillance cameras, etc. These new data types raise new questions around information governance and add to the volume of data stored. Velocity. Data is coming into the organisation at an increasing speed, and Big Data analytics want to take advantage of it in real time. Consequently, performance is a key element of the underlying IT infrastructure. Value. Big Data analysis is done to create a unique competitive advantage for organisations, through understanding their customers' preferences better, to segment customers more granularly, and to target specific offers at precise segments. Some commercial sector examples include combining sales data with weather data and running special campaigns on rainy days, or enabling fraud prevention through the combination of credit card transaction data and location data from mobile phones. Public sector organisations are also using Big Data to prevent fraud and save tax payer money and to provide better services to citizens, particularly in the public healthcare space. Big Data use cases are emerging in every industry where added insight into customers and markets creates value and competitive advantage. Big Data is not only about specific attributes of customers' datasets; it is about using new technologies and techniques to economically manage rapid growth, increased variability and accelerating velocity in those datasets to drive business growth. Every organisation can face a Big Data challenge if its current IT infrastructure is not fit to handle the increasing requirements for availability and performance of their growing volumes of data. The Big Data Opportunity According to IDC's 2012 European software survey, 27% of European IT and business managers are currently accelerating current analytics activities by using Big Data technologies (see report Big Data: How Ready is Western Europe? IDC #LT08U). They are typically shortening the time that it takes from problem formulation to analytics result, which enables them to take decisions faster and based on more granular data. The survey also shows that 18% of European business and IT executives are already looking at creating new products and services based on the opportunities that are opened up through the deployment of Big Data technologies. Below are some of the early use cases for Big Data within the industries that IDC believes are leading the charge in this space: A telecoms organisation leveraging geospatial data to track mobile subscribers using their cell phones for file sharing. These users tend to be putting significant unnecessary extra "pressure" on the mobile network, and are also breaking the company's terms of IDC

3 service. This can be done by visualising this data to highlight the location of people who are using extremely large amounts of data, which is often an indicator of file sharing. A financial institution augmenting traditional underwriting risk analysis with census data, social media feeds, geographical hazard risk, localised job market trends and social/professional history to improve credit scoring models. A government immigration agency profiling people (including real-time facial recognition and analysis) entering a country or a region to prevent security threats at the port of arrival. However, IDC believes that the ability for any organisation to derive value from the opportunity presented by Big Data is directly correlated to its information management maturity. Figure 1 shows IDC's simple maturity model to highlight how these two are linked. Figure 1 Organisational Information Management Maturity: Maximising the Big Data Opportunity Source: IDC, 2012 This model describes an organisation's information management journey from a company that has a very "siloed" approach to data and information (stage 1), to one that can take advantage of external data to turn this into information to drive significant business impact (stage 4). It should be noted that the level of business user involvement is a critical component of each stage of this maturity model. IDC's experience has shown that maturity is largely dependent on industry, and those industries that have a B2C business model (e.g., retail banking, telecoms and retail) are the most progressive in terms of understanding the opportunity that Big Data presents. This is due to the fact that organisations in these verticals have been dealing with large and growing data volumes for some time, and their information management strategies are most advanced with significant involvement from the line of business executives. It is also clear that if most of an organisation's information management practices are at stage 1, then any attempt to manage and analyse Big Data from external sources without moving through the intermediate stages is likely 2012 IDC 3

4 to fail. Most organisations are trying find the appropriate "next step" without a forcing a major "rip and replace" of existing information management systems. It is also important to note that the Big Data opportunity is both based on the advance of technical capabilities, but more importantly on creative business users who seize the opportunities created by new technologies and deploy them to take their business to new markets, address new customer segments and develop new products and services. IT executives can evangelise about the possibilities, but the business cases need to be driven from business managers. In order to take advantage of Big Data, both IT and business managers need to work closely together and assess both technological possibilities and potential business cases and business requirements. The Challenges Associated With Big Data As Big Data is sometimes defined as the point at which your current technology architecture cannot handle the volume, variety and velocity of data anymore, it is obvious that Big Data creates IT infrastructure challenges for IT executives. From an IT executive perspective, Big Data challenges are: Data growth. How to manage, protect and archive large amounts of data efficiently. Storage technologies like scale-out NAS, scale-out file systems, auto-tiering, thin provisioning, data deduplication, snapshots, image-based backups and advances in archiving technologies have broadened the toolkit that IT executives can use to store large and growing amounts of data more efficiently. However, designing the right architecture for your business is getting ever more challenging, given the many technology options. Performance. Big Data analytics solutions are very hungry for performance, and sometimes require direct attached storage (DAS) in order to move storage and compute closer to each other and to not lose precious performance by moving data over a network. Many vendors are currently developing converged infrastructure stacks that combine compute, storage, networking and management into one building block for maximum performance. The choice of file system also has an impact on performance. Data governance and data quality. Every analysis is still as good as the data that is used as a basis for the analysis, and data quality remains an issue even with Big Data. Organisations can use a broader variety of data sources because they are no longer restricted by the strict relational database schema, but they still need to ensure the reliability and quality of the data that is used for analysis. Business value of data. A key challenge remains to understand the value of the data stored and if it justifies the cost of managing and storing it. Only business users can ultimately answer that question, but they usually do not want to spend time on classification. Engaging in the dialogue with business users from the start will help IT executives to understand which data sources need to be stored and which data sources might just pass through the organisation and have no value after they have been analysed. Skills shortage. IDC is currently observing a significant Big Data skills shortage both in the IT and the business domain. Big Data specialists are only just emerging, either through their own work experience with Big Data problems, or through evolving education as data scientists. However, many of the established infrastructure design criteria are still valid when providing infrastructure for Big Data analytics applications IDC

5 Emerging Big Data Best Practices From an IT Infrastructure Perspective IDC is observing the emergence of early best practices for Big Data implementations and as such recommends these as a starting point for end users embarking on a new information management strategy to deal with Big Data: Optimise the storage layer. Dealing with large volumes of data requires strict discipline in terms of data management in order to keep the storage footprint as small as possible. IDC recommends that organisations focus on optimising the storage layer through the use of archiving, data protection, data deduplication, thin provisioning and storage tiering technologies. Every company needs to find the appropriate mixture of these technologies to best deal with their data. Align performance requirements to business impact. Providing high-performance IT infrastructure to the business intelligence and datawarehouse applications enables business users to successfully operate data analysis and shortens time to insight and time to value. IDC believes that the use of in-memory databases, scale-out file systems, flash and other performance accelerating technologies can help with this, but only where the business case for the relevant workloads requiring these technologies will drive the necessary business benefits. Build a strong business case for Big Data. When investing in new technologies to take advantage of Big Data, it is important to build a strong business case to ensure that the cost of storing extra data sources and analysing them in a new technology platform will be outweighed by the expected benefits. This is tough for an emerging technology trend like Big Data, where most organisations are currently at the experimentation stage and the business impact is still unclear. In terms of building a business case for Big Data, IDC recommends that IT executives ensure the following: o o o Any new Big Data project should be tied to broader strategic objectives and have a true business sponsor with clear growth targets in place. Develop a "quick-win" strategy with transparent financial metrics that has buy-in from the CFO. Ensure that data governance and data quality requirements are "baked-in" ahead of project kick-off to avoid painful delays and lack of credibility with business stakeholders further down the line. Transform the IT organisation. Business users are increasingly used to having nearinstant access to their desired applications through software-as-a-service models and cloud services. By transforming IT into an agile unit, business user needs can be met faster and they will remain happy customers of their internal IT department. It is only through this level of transformation that IT can remain a valued partner to the business. Conclusion and Outlook Big Data is a transformative force and changes the power balance between IT departments and business users. Within this dynamic, the important question that needs to be answered is how the IT department can stay relevant as an IT service provider when business users want new services instantly. Big Data adoption and use cases are really driven by creative individuals on the business side of the organisation but they are also linked to the information management maturity of an organisation. IT executives need to remain influential through uncovering the emerging business requirements and communicating the new possibilities that are opening up from Big Data technologies, while at the same time providing a flexible high-performance infrastructure at an attractive cost. IT managers also need to make sure that they can measure 2012 IDC 5

6 the success of IT projects with business metrics like return on investment and that they can document the quality of service that they provide. Especially with Big Data, but also with other IT projects, it is important to understand the business case. Investment in new analytics technologies like Hadoop increases the complexity of the IT architecture, and the addition of new data sources drives up storage costs. Consequently, the benefits of Big Data analytics and the potential business value created need to be documented and presented to key business sponsors to ensure the appropriate level of buy-in. A service provider with a broad solution portfolio in both infrastructure and analytics can support you in the redesign and optimisation of your IT infrastructure. However, it is important to keep in mind that the full potential of Big Data can only be realised through close collaboration between business users and IT executives with a mutual understanding of business requirements and technological capabilities. Ask your service provider to facilitate this dialogue between these two groups of stakeholders. This will help the discussion to evolve from an "us versus them" approach between IT and business to a more strategic and constructive one. Structures, roles and skills need to be revisited not only terms of driving key business outcomes, but also where the role of the CIO, CTO and broader IT department are recognised. An approach that encompasses all of these dynamics will be critical to ensure the success of your Big Data journey. ABOUT THIS ANALYST Carla Arend is a programme director with the European Business and Enterprise Solutions (BES) team, responsible for managing the European information management research service. Arend provides industry clients with key insight into market dynamics, vendor activities and end-user trends in information management and storage software and services markets. She is also responsible for the Western European storage software data tracking product. In addition to syndicated research, Arend has worked on many custom consulting projects identifying opportunities in the European storage software and services market and has authored white papers on subjects such as Big Data, cloud storage, data protection and recovery in virtualised environments, archiving software and virtualisation software. Arend is regularly asked to speak at industry events, strategy workshops, sales kick-offs and training, partner events and end-user events. Before taking up her current role, Arend spent five years with IDC's European Software Group, where she worked on the Internet Commerce Market Model, conducted security and systems management research, and was responsible for managing IDC's European software trackers. Arend holds a BA in business administration from University of Mannheim and an MSc in business administration and management of technology from Copenhagen Business School, in addition to course work at the Asian Institute of Technology (AIT) in Bangkok. Follow Carla on Twitter here: ABOUT THIS PUBLICATION This publication was produced by IDC Go-to-Market Services. The opinion, analysis, and research results presented herein are drawn from more detailed research and analysis independently conducted and published by IDC, unless specific vendor sponsorship is noted. IDC Go-to-Market Services makes IDC content available in a wide range of formats for distribution by various companies. A license to distribute IDC content does not imply endorsement of or opinion about the licensee. COPYRIGHT AND RESTRICTIONS Any IDC information or reference to IDC that is to be used in advertising, press releases, or promotional materials requires prior written approval from IDC. For permission requests contact the GMS information line at or gms@idc.com. Translation and/or localization of this document requires an additional license from IDC. For more information on IDC visit For more information on IDC GMS visit Global Headquarters: 5 Speen Street Framingham, MA USA P F IDCAC07U IDC