Benefitting from Big Data Leveraging Unstructured Data Capabilities for Competitive Advantage

Transcription

1 Data and Technology Perspective Ramesh Nair Andy Narayanan Benefitting from Big Data Leveraging Unstructured Data Capabilities for Competitive Advantage

2 Executive Summary The emergence of digital trends has accelerated the volume, speed and variety of unstructured consumer information referred to as Big Data. Enterprises can benefit from this growth and seize market opportunities to drive value by implementing solutions that capitalize on the rampant growth of big data. The data processed can provide companies with insight about customers that can help boost productivity in areas such as marketing, operations, and risk management. But with the amount of data expanding, technology challenges, organization limitations, and privacy/trust concerns - among other obstacles businesses must approach analyzing this data with an array of new and emerging technologies. By overcoming the challenges that coincide with big technology solutions and leveraging distinct capabilities, companies across many industries can gain a tremendous competitive advantage. 1

3 Big Data Overview Unstructured consumer data, called Big Data, represents majority of growth in data volume, up 56% CAGR since 2005 Storage Cost per GB ($) 130 $20.00 By % (approximately 1,100 Exabytes) of all data in the digital universe was unstructured Digital Universe Data Storage in Exabytes 1 Exabyte = 1MM Terabytes 1,227 7,910 68% of all unstructured data in 2015 will be created by consumers $10.00 $0.50 Rapid decreases in storage costs have enabled the enterprise to store ever more data Time Source: IDC's Digital Universe Study, sponsored by EMC, June 2011 = All Digital Data = Unstructured Digital Data 2

4 Big Data Overview A first step is to understand the categories of Big Data that can be leveraged -- Structured vs. Unstructured and Internal vs. External External structured data is a logical extension of the current analytics done on internal structured data in the enterprise External Booz Big Data Framework Mobile Phone / GPS Credit History Google+ Twitter Facebook Instagram Travel History Pinterest Blogs Real Estate Records Census Data External Sensor Data This quadrant represents the largest area of opportunity for the enterprise to gather consumer insights Data Source CRM Web Profiles Online Forums Web Feeds Internal structured data is the category best understood by the enterprise, but enterprise needs to shift focus to external and unstructured data Internal Sales Records HR Records Financials Inventory Structured SharePoint Sensor Data Text Documents Unstructured Data Type Internal unstructured data is a prime learning ground for the enterprise to understand how to mine value from these data formats Source: IDC's Digital Universe Study, sponsored by EMC, June

5 Big Data Overview Notion of expanding data sets have appeared every decade or so -- but current Big Data trend is different along multiple dimensions Characteristics Defining Big Data Volume Volume of data stored in enterprise repositories have grown from megabytes and gigabytes to petabytes E.g. volume of data processed by corporations grew significantly, e.g. Google processes 20 petabytes / day By 2020, 420 Billion electronic payments are expected to be generated New York Stock Exchange creates 1 terabyte of data per day vs. Twitter feeds that generates 8 terabytes of data per day (or 80 MB per second) Variety Data variety exploded from structured and legacy data stored in enterprise repositories to unstructured, semistructured, audio, video, XML etc. Streaming data, stock quotes, social media, machine-tomachine, sensor data all drive increasing variety that needs to be processed and converted into information Velocity Speed of data movement, processing and capture in and outside enterprise went up significantly Model based business intelligence models typically takes days for processing - compared to almost real-time analytics requirements of today using incoming stream of high-velocity data E.g. ebay is addressing fraud from PayPal usage, by analyzing real-time 5 million transactions each day. Source: Information week,

6 It is different this time because not only is the data universe expanding, but the universe of data is itself expanding Big Data Overview An Expanding Data Universe An Evolving Data Universe Search volume News reference volume types of data large data petabytes Big Data Interest spike on Big Data since mid-2011 Interconnected Intelligent Access to information is democratized Mature infrastructure for person-machine and machine-machine collaboration M2M continues to grow with peer-to-peer networking Mobility and social networking trends point to a Bi-2-Bi future, i.e., billions connected to billions Data analytics has gone mainstream Existing data analysis creates new data Raw computation power has risen rapidly: today s off-the-shelf components can provide what was previously only possible using a super-computer Structured data volumes continue to rise (20% CAGR) 80% of the world s data is unstructured Unstructured data is growing at 15 times the rate of structured data Layers and layers of unstructured data gets added based on user interactions and data usage Source: Google Trends April 6, 2012 Instrumented Cost of data acquisition has plummeted Infrastructure has delivered fine-grained data in real time, analyzed with cheaper CPU cycles 5

7 Big Data Trends The rampant growth of connected technologies creates even more acceleration in Big Data trends it is a data tsunami Emerging Digital Trends Impact on Big Data A recent Booz study conducted sees three digital eras emerging by 2015 in response to the trends 1. Era of the working nomads Connected World Connected Consumer: 24/7 connection to the internet through PC and mobile devices - 80 percent of the world population will use mobile phones by 2020 Increase in telecommuting will be enabled by connectivity Work previously done face to face will become more digital with more data stored in the enterprise 2. Era of the On Life Ubiquitous and seamless connectivity will be readily available further enabling social networking Standards will be in place for worldwide seamless mobile communication to complete the connected world 3. Era of the Smart Cloud Information Osmosis Social Networking Consumption of digital information by consumers will increase - by 2020 there will be 4.7 billion internet users globally Nonlinear information consumption will become the norm as consumers pick and choose the information they want from the vast pool of available data Increase in the number of internet users will drive even more growth in social networks - on average a user will have a contacts in their online social network Social Storefront will create more unstructured enterprise data as consumers migrate increasing amounts of purchasing activity to online Smart cloud technology will emerge to respond to the needs of the enterprise and the consumer Increase in the sensor economy will require remote data storage capabilities to hold the data collected by the devices Sensor Economy Enhanced technological capabilities for sensor devices will usher in the Era of the Sensor Economy Environment and location aware devices used by consumers will deliver increasingly large amounts of digital data to the enterprise 1) Hal Varian, Computer Mediated Transactions, 2010 Ely Lecture at the American Economics Association 2,3) IDC's Digital Universe Study, sponsored by EMC, June

8 Big Data Opportunities Enterprises can benefit from Big Data in several areas including Customer Insights, Marketing, Operations and Risk Management Big Data Benefit Areas Examples Targeted Marketing using Social Trends Accurate Business Insights Customer Segmentation Sales & Marketing Lead Capture Real-time Decisioning Customer Churn Detection Customer Analytics Marketing Analytics Customer Driven Marketing: Targeting promotions and personalizing offers based on individual purchasing behavior, churn prevention Product Recommendation: Collaborative filtering, multi-channel activity based recommendations Marketing Mix Modeling: Optimizing marketing mix and promotions by using econometric modeling to assess sales lift of different marketing tools and identifying most effective Pricing Optimization: Using data to assess demand sensitivity to pricing to optimize pricing through various points of product life cycle Fraud Detection Risk Quantification Market Sentiment Trending Business Change Understanding Planning & Forecasting Better Cost Analysis Customer Behavior Analysis Leveraging Click-stream Manufacturing Yield Improvements Other 4 6 % Response Web/Mobile/ Social Analytics Operational Effectiveness Fraud and Risk Analytics Customer Activity Analysis: Storing customer preferences to customize display, tracking usage to measure web metrics Social Media Monitoring: Analyze consumer sentiments towards company and products on social media platforms Operational data analytics leveraging large manufacturing data to improve process and product quality Improved planning and forecasting leveraging large historic process, resource and product data Large customer, transaction and market data analysis for customer and product risk quantification Real-time fraud detection leveraging data from POS, transactional and analytical systems Source: TDWI Research (Q4, 2011), analysis 7

9 Big Data Challenges However, significant challenges exist in implementing Big Data solutions and using it to drive value in the enterprise Big Data Challenges Solution Maturity Limited number of large implementation of Big Data solutions exist in the enterprise Most of the enterprise implementations are in pilot stages Organization Limitations Talent Lack of truly skilled professionals on the types of data, and its appropriate use Culture Organizations have not yet fully realized the implication of Big Data on business modeling and insights, and IT architecture and execution Privacy/Trust Concerns As more data is available, acceptable use of personal data becomes of greater concern to customers Amount of data is increasing faster than organizations can properly secure the data Technology Velocity, volume, and variety of data show no signs of slowing which will require new technology solutions 8

10 Secondly, the solution needs to be architected -- to benefit from Big Data, an array of new and emerging technologies is needed Customer Interface Sales & Marketing Risk & Fraud Internal Operations & Processes Capabilities Seamless, crosschannel portal Better platforms for custom products and services Better access to customer insights A holistic view of the customer High end analytics to proactively predicting fraud Enhanced data models for advanced risk management Enhanced corporate systems to support HR and other internal processes Platform for managing the product and service portfolio Financial data management (e.g., A/R and A/P reconciliation) Mobility High End Analytics Data Management Next Gen Data Processing Big Data Capability Enablers Use of mobile devices for real-time analytics Anytime, anywhere connectivity Full population analytics Real-time consumer insights Analytics embedded into business processes (risk management, marketing, sales) Integration of relational and nonrelational structures Big Data sourced from a very large number of providers and partners Fundamental changes in enterprise data architecture and governance Specialized database management systems (columnar) integrated into enterprise architecture In-memory processing coupled with event and trigger based processing 5 6 Cloud Computing Service Oriented Architecture Technology Architecture Supporting Enablers On-demand, standardized infrastructure, platform, processes and services External on-demand services (such as analytical computing) Service-based technology architecture Re-usable, plug-and-play services 9

11 Technology Implications These technologies and their use on Big Data will have a major impact on the IT programs and portfolios 1 2 Enablers Architecture Technology Data & Applications Mobility High End Analytics Separation of user interface from back end as enterprise applications are accessed by multiple customer devices Infrastructure strategy to enable full population analytics Data architecture for real time contextaware analytical insights Migration to web-centric tools that reduce dependency on specific devices and platforms Database and infrastructure to support inmemory processing To provide a full range of functions on mobile channels, build infrastructure to support apps and use cloud-sourced services A data strategy to support high end analytics and insights based on recurring customer events 3 4 Data Management Next Gen Data Processing Enterprise data warehouse and analytical data structure enhancement to accommodate unstructured data from multiple external providers Data, query and analysis architecture to support new database constructs Platforms (Apache, LexisNexis), tools (Hadoop, MapReduce) and delivery strategies for big data management Integrated data architecture to manage relational data with new data structures (columnar or inverted DBMS) Access to critical data sources (such as social media) needed for customer and process insights Refinement of data integration processes and tools to integrate unstructured data with relational data Deployment of data warehouses and downstream applications to bring together traditional RDBMS with new columnar or inverted data 5 Cloud Computing Cloud strategy (private, public, hybrid) to source infrastructure, platform and applications to manage data intensive analytics, with minimal impact to internal systems Cloud services that support mobile payments and complex analytics Cloud-based platforms to enhance app. development capabilities Cloud sourcing structured/unstructured data from multiple external sources Data sensitivity classifications to design clouds with appropriate security 6 Service Oriented Architecture Refinement of the SOA implementation approach to provide outputs of Big Data processing as services Refinement of the enterprise technology strategy to ensure support for service-based interoperability Common services to be provided to front, middle and back office applications by aligning SOA strategy with enterprise business model 10

12 Technology Initiatives Existing data and technology initiatives should be augmented with Big Data technology enablers Enterprise Initiative CRM Platform Customer Portal Some Illustrative Scenarios Explore hosting of CRM platform in the cloud Extend CRM platform by integrating social media technology to enhance customer view Integrate operational analytics with CRM modules to aid sales interaction Enhance use of interfaces for internal customers to allow for seamless integration with external interfaces Incorporate all customer data into one portal so the consumer experience can be further customized Customer Analytics and Insights Use new analytical tools to process a high volume of data to deliver customer insights Create custom or personalized products and services based on consumer insights and additional data collected about consumers from multiple channels Operations Management Customized product and service offerings will require operational adjustments Use analytics to monitor operations processes and identify inefficiencies Enhance operations by delivering more real-time data to consumers Enterprise Data Management Risk Management Explore alternative technologies for data integration across channels (i.e. Hadoop, MapReduce) Leverage technology to integrate structured enterprise data with unstructured data from social media and other sources New analytical tools and processes can enable enhanced risk management models and proactive fraud and loss prediction Information security initiatives will also be necessary to protect data especially in light of trends such as Bring Your Own Device 11

13 Progressive Snapshot is a perfect example of commercial usage of Big Data technology Progressive Snapshot Case Study 1 Industry Initiatives Overview Progressive Insurance, as part of its Pay As You Drive program, offers drivers the chance to lower their premiums based on real-time analysis of their driving habits Drivers plug a device, the Snapshot, that collects large volume of data (e.g., time of the day, miles driven, number of hard brakes) over a period of time. Based on analysis of the data, progressive offers a discount to individual s insurance premium Technology Snapshot combines Big Data analysis with mobile computing, and cellular communications technologies. The device is plugged into the car s on-board diagnostic port ; as the customer drives, a large volume of data is collected in a mobile, next generation database and analysed The device leverages AT&T network to share the information wirelessly with Progressive -- this is not fitted with GPS for privacy reasons Future Plan Next generation database technology is becoming cheaper, making the Machine-to-Machine devices like Snap-shot more affordable -- consumerization of this technology is expected to happen within next 3-5 years In insurance industry, the competition for Big Data technology has already started Allstate has started providing similar solution as Snapshot to their customer base for premium discounts Source: research 12

14 Other leading companies across industries are beginning to leverage Big Data capabilities for competitive advantage Industry Initiatives Case Study 2 Case Study 3 Case Study 4 WellPoint and IBM to use Watson to improve patient care Intuit and 10gen enable real-time insights for customer websites Officemax uses ParAccel for effective marketing and sourcing Wellpoint to develop and launch Watsonbased solutions to help improve patient care through the delivery of up-to-date, evidence-based health care Watson can process about 200 million pages of content in less than three seconds as it leverages analytical tool to analyze huge volumes of data to aid decision-making IBM will develop the base Watson healthcare analytics based on Watson s capabilities, for WellPoint's needs this will be first commercial application of the Watson technology Intuit Websites, uses a next generation database (MongoDB) from and real-time analytics tool from 10gen to derive interesting and actionable patterns from their 500,000+ customer website traffic This next generation database enables high performance unstructured data analysis and gleaning customer insights at a lower cost Deployment of MongoDB is faster than relational databases and easily integrates structured data with unstructured data OfficeMax moved from its maxed-out Teradata Data Warehouse to ParAccel, a columnar database for complex analytical queries ParAccel was implemented in a private cloud to provide horizontal scalability The solution is connected with Teradata, eliminating the need to additional data transformation and manipulation -- saving considerable additional effort Source: research 13

15 Contact Information Chicago Andy Narayanan Principal Florham Park, NJ Ramesh Nair Partner Team Members: Arindam Chatterjee (Sr. Associate), Balu Nair (Sr. Associate), Alanna Hynes (Associate), Lucas Streit (Associate), and Jae Chang (Associate) 14

16 The most recent list of our offices and affiliates, with addresses and telephone numbers, can be found on our website, booz.com Worldwide Offices Asia Beijing Delhi Hong Kong Mumbai Seoul Shanghai Taipei Tokyo Australia, New Zealand & Southeast Asia Auckland Bangkok Brisbane Canberra Jakarta Kuala Lumpur Melbourne Sydney Europe Amsterdam Berlin Copenhagen Dublin Düsseldorf Frankfurt Helsinki Istanbul London Madrid Milan Moscow Munich Paris Rome Stockholm Stuttgart Vienna Warsaw Zurich Middle East Abu Dhabi Beirut Cairo Doha Dubai Riyadh North America Atlanta Boston Chicago Cleveland Dallas DC Detroit Florham Park Houston Los Angeles Mexico City New York City Parsippany San Francisco South America Buenos Aires Rio de Janeiro Santiago São Paulo is a leading global management consulting firm focused on serving and shaping the senior agenda of the world s leading institutions. Our founder, Edwin Booz, launched the profession when he established the first management consulting firm in Chicago in Today, we operate globally with more than 3,000 people in 60 offices around the world. We believe passionately that essential advantage lies within and that a few differentiating capabilities drive any organization s identity and success. We work with our clients to discover and build those strengths and capture the market opportunities where they can earn the right to win. We are a firm of practical strategists known for our functional expertise, industry foresight, and sleeves rolled up approach to working with our clients. To learn more about or to access its thought leadership, visit booz.com. Our award-winning management magazine, strategy+business, is available at strategy-business.com Inc. 15