Executive Summary...3. Understanding Big Data and its Implications for Businesses...4. Why Harness Big Data...4

Transcription

1

2 Contents Executive Summary...3 Understanding Big Data and its Implications for Businesses...4 Why Harness Big Data...4 The Rise of the Connected Consumer: A Game Changer...5 Real-time Business Insights: The Unique Differentiator...6 Mass Personalization: The New Mantra...6 Competitive Edge: A Constant Business Imperative...6 Making Sense of Big Data: Overcoming the Roadblocks...7 A Structured Approach to Adopting Big Data Solutions...8 Reaping the Benefits by Selecting the Right Big Data Suite...8 Befriending Big Data...9

3 Executive Summary The surge in Big Data technology adoption is redefining the way businesses operate and make decisions that are timely and insightful. As traditional Relational Database Management Systems (RDBMS) prove inadequate in processing the volume, variety, velocity and complexity of Big Data, businesses are driving tangible business outcomes using sophisticated Big Data solutions. Big Data technology helps access, manage and leverage available data in real time and is becoming more affordable than ever before. However, studies indicate that as much as 58% of Big Data projects are incomplete. This is indicative of the challenges that enterprises face in designing effective solutions that help analyze the data and drive business intelligence for enhanced decision making. This paper addresses the need for organizations to understand Big Data as essential for growth depending on the nature of the business problems they are trying to address. It further focuses on humanizing Big Data, i.e. making Big Data consumption easy for users by allowing them to combine it with existing enterprise information to derive enhanced business value. NAICS Association, The Big Data Revolution, 2014,

4 Understanding Big Data and its Implications for Businesses The definition of Big Data has now become as ubiquitous as Big Data itself. It refers to the ever growing volume, variety and velocity of data available to businesses from diverse sources across and outside the enterprise. Big Data can be broadly classified as structured and unstructured data. Structured data: Includes properly maintained records and documentations such as inventory, sales, financials, CRM, and HR records as well as census data, travel history, credit history and such macro documented records. 95% of the valuable content across an enterprise is in unstructured formats i.e. residing not in databases but in a variety of files that include , documents, presentations, and much more. Published: HP Research Labs, Annual Report 2010 Unstructured data: Comprises non-traditional data from social media, s, internal chat, photographs, blogs, sensors, podcasts, RFIDs, videos, etc., which carry rich information on consumer behavior. Un-structured data may soon account for more than 80% of the data that is generated and mined by enterprises. According to an IDC study, since 2005, unstructured data has swelled at a CAGR of 56%. At the end of 2010, 90% of all data in the digital universe was unstructured. In addition, 68% of all unstructured data in 2015 will be created by consumers. With data storage and computing servers becoming more pervasive and affordable, enterprises are now finding it easy to tap the non-structured data sources in their business intelligence analysis. However, this type of data cannot be analyzed in traditional databases and requires modern Big Data suites for effectively gathering insights. Why Harness Big Data Data centricity is no longer a choice. Organizations are spending big money to get an early bite of the Big Data benefits pie. IDC forecasts indicate that the Big Data market is set to grow to $23.76 billion in 2016 with a CAGR of 18.55%. According to reports by Bain & Company, the year 2015 will see the global financial services sector spending around $6.4 billion on Big Data, while software Booz & Co., Benefitting from Big Data Leveraging Unstructured Data Capabilities for Competitive Advantage, 2012, A study conducted by the Saïd Business School at the University of Oxford and the IBM Institute of Business Value provided the following insights: 63% respondents reported that use of information including big data and analytics is creating a competitive advantage for their organizations. Compared to companies that rely on traditional analytics, organizations that implemented big data analytics, pilot projects or deployments are 15% more likely to report benefitting from their information assets and analytics. Published in: Analytics-The real-world use of big data in financial services Booz & Co., Benefitting from Big Data Leveraging Unstructured Data Capabilities for Competitive Advantage, 2012,

5 and internet based companies are expected to bill around $2.8 billion. Even though complexities abound, brands and companies that are able to draw insights and actionable data-driven findings will be the winners. Here are some of the leading reasons for embracing an effective Big Data strategy: The Rise of the Connected Consumer: A Game Changer We live in an increasingly connected world, supported by a growing number of channels and tools that keep us hooked together. From an enterprise point of view, this collaborative environment is a game changer and has created a fresh set of challenges. With the advent of the Internet of Things, the consumer data footprint has reached an all-time high. On an average, individuals upload 15 times more data today than he did three years ago. As consumer devices interact with the business networks, organizations gain more data and information. According to IDC, the amount of digital information in the world is more than doubling every two years Enterprises that can extract enough value from the data deluge will be better positioned for sustained success. (See Figure 1) CONNECTIVITY IS EVER-INCREASING Also By 2020 there will be over 200 Billion Connected Things Global IoT Market in trillions of $ Projected Billion Processors Shipped 3 Billion Utility Meters 1.5 Billion vehicles Analytics- The real-world use of big data in financial services Bain & Company, The Who, Why and How of Big Data, 2013, IDC, The Digital Universe of Opportunities: Rich Data and the Increasing Value of the Internet of Things, 2014,

6 Real-time Business Insights: The Unique Diffe rentiator The need for testing theories and making changes to events, campaigns and strategies instantaneously with the support of near real time insights is critical for competitive advantages in tough markets. In addition, when organizations are not agile enough to harness data proactively, today s connected always on consumers freely share their opinion on social platforms, affecting the organizations star ratings. Businesses therefore need to ensure real time analysis of Big Data to accurately gauge the consumer pulse and take action in real time. We also see a growing appetite for nowcasting, which is extensively applied in economics, and consumer and market analytics. Derived from the combination of now and forecasting, it refers to real time prediction of current conditions and estimation of various metrics such as consumer confidence for real time impact. Mass Personalization: The New Mantra In the new multi-device world, users are 90% more likely to use multiple cross platform devices sequentially. Tapping consumer data across all these media is also crucial due to the increased demand for personalized content. Consumer knowledge has always been a key point of differentiation across all industries. The recently popular concept of Permission Marketing has taken this need for greater consumer knowledge to an all-time high. Big campaigns, information overload and advertising clutter are increasingly proving ineffective. Today s discerning consumers wish to see only what is relevant to them and are now empowered to filter what they don t need or want. This age of personalized information puts greater emphasis on the need for intimate knowledge about the consumer not just about their buying patterns but everything from their demography and interests to what they are currently talking about. Competitive Edge: A Constant Business Imperative Tough market dynamics and volatile economic environments compel businesses to constantly discover new ways of staying ahead of their competition. Given the ever changing industry norms and complex regulatory requirements, enterprises require an effective data driven strategy to achieve innovation and greater business value, even as they stay compliant. Incisive insights from Big Data help optimize supply chains, identify and mitigate the risk of fraud, boost organizational performance and productivity, and improve decision making. Organizations can use Big Data insights to find new customers and create new revenue streams by creating and marketing next generation products and services. With the help of insights from Big Data, they can also offer proactive maintenance to avoid the risk of early failure in products, thereby enhancing consumer experience. SAP, Nowcasting: Big Data Predicts The Present, 2012,

7 Making Sense of Big Data: Overcoming the Roadblocks In spite of the increasing interest in Big Data, several businesses continue to struggle when it comes to navigating the growing volumes of internal and external data. They are unsure of the technology they need to implement to make better sense of the data and chart a meaningful strategy for their business. Well-defined data is simple to mine and analyze. However, the growing volumes of unstructured data from disparate and complex sources such as social media, blogs and weblogs that lack specific data architecture or model pose several challenges in deriving actionable business insights. According to research by McKinsey Global Institute and McKinsey s Business Technology Office, by 2018, US alone could face a shortage of 140,000 to 190,000 people with deep analytical training (in statistics or machine learning), and another 1.5m people with the managerial and quantitative skills to be able to frame and interpret analyses effectively enough to base decisions on them. Report: Here are some of reasons why enterprises fail to realize the true benefits of Big Data: Conventional database systems and data warehouse environments fall short in acting as a complete repository of all data that is generated or acquired from a variety of sources. McKinsey Global Institute Big data: The next frontier for innovation, competition, and productivity 2011 The growing number of analytical models that are needed require a non-demand pool of distributed and parallel processing resources. Traditional tools fail to deliver accurate real time insights. There is dearth of skilled resources to mine and analyze data as well as leverage new Big Data technologies that are still raw and untested. Businesses struggle to integrate data and apply advanced analytics as they are burdened with legacy systems and incompatible standards and formats. High data preprocessing time further aggravates the complexities. McKinsey&Company, Big data: The next frontier for innovation, competition, and productivity, 2011,

8 The ground reality is that many organizations are not ready or willing to adopt an effective technology stack comprising storage, computing, analytical and visualization software applications that is imperative for harnessing large digital datasets. They are yet to implement logical steps for rolling out a successful Big Data strategy due to budgetary and technological limitations. The next section highlights ways of overcoming these barriers and leveraging the right approach to implementing relevant big data strategies. A Structured Approach to Adopting Big Data Solutions A well thought through roadmap is essential to simplify the Big Data adoption and implementation process. Businesses that are testing the waters can initially resort to small scale Big Data projects meant to explore the business benefits they offer. They can follow this up with mainstream adoption of the evolving technology by implementing a focused and systematic approach, driven by clearly defined requirements and goals. A thorough understanding of the time required for deploying Big Data solutions and their time to value is also essential. Let s examine the key steps involved in adopting Big Data technology: Define the various sources of data such as EDW, and Customer Relationship Management (CRM) and Enterprise Resource Planning (ERP) platforms Explore the relevant datasets and build strong reference architecture and governance processes Create a sound analytics modeling based on all the key business functions and their objectives Identify the infrastructural requirements including storage, server and networking infrastructure Determine the right mix of tools and technologies. Frameworks such as Hadoop, MapReduce, R, Mahout, SAS, Flume, and Lucene address specific areas. Deploy the right resources with strategic skill sets Above all, businesses can ensure maximum benefits and ROI from their Big Data projects by ensuring continuous improvements and Big Data best practices.

9 Reaping the Benefits by Selecting the Right Big Data Suite In order to ensure that Big Data is not a chaotic proliferation of yet-another thing out there, businesses need to select a Big Data technology that furthers their overall business requirements and suits their budgets. By adopting an enterprise level data analyzing tool, businesses can derive greater value out of all the data generated and significantly reduce the time to convert data into revenues. An effective Big Data solution should ideally be equipped with a Unified Data Analysis Platform(UDAP) that captures data from various sources, and facilitates efficient data storage and processing. It features predictive models, and also provides a variety of analytics options including graphical analytics, text analysis and statistical data analysis. The ideal solution should also enable the creation of a Big Data layer over an organization s existing Enterprise Data Warehouse (EDW), and help access both structured and unstructured data in the form of a data reservoir. Other desirable highlights include domain specific extensibility, scalability, data security, and the ability to accelerate data loading and movement across layers. Businesses that invest in such a solution can realize the following key benefits to outperform their competition: Ability to consolidate data sources and integrate them with existing datasets In-depth, interactive analysis Accelerated analysis lifecycle Reduced data pre-processing time Regular data analysis with faster insights for quicker responses Reduced infrastructure costs

10 Befriending Big Data Early bird organizations have already spent billions on data and analytical tools such as ERP and CRM systems and are resting on petabytes of Big Data. On the other hand, Big Data technology is only now spreading across industries and into new categories of companies that have been laggards in adopting it. Nonetheless, all enterprises with information-driven business models agree on the significant scope of Big Data and the business impact of the insights mined from them. They are therefore using their data assets to extract predictive and prescriptive insights to drive performance and growth. The current Big Data setup still requires a sophisticated architecture, keeping it beyond the reach of small-medium enterprises and startups. These establishments enjoy the advantages of closer proximity to their data sources, and therefore, possess more authentic, validated data banks. However, they lack the tools to analyze them and draw actionable insights. Businesses can choose holistic solutions from experienced service providers or customized solutions from open source products. Befriending Big Data will mean making these technologies ubiquitous, user friendly and cost effective. We are talking technology that will allow Big Data applications to be easily customized to empower businesses of all sizes with big insights. Such technology has the potential to make Big Data for all a reality. It will enable strategic decisions supported by real time insights, leading to better and economical use of resources, and elimination of wasteful trial and error runs.