Big Data The next big thing

Similar documents
Big Data. Fast Forward. Putting data to productive use

Are You Ready for Big Data?

Hadoop Beyond Hype: Complex Adaptive Systems Conference Nov 16, Viswa Sharma Solutions Architect Tata Consultancy Services

Are You Ready for Big Data?

BIG DATA ANALYTICS REFERENCE ARCHITECTURES AND CASE STUDIES

What happens when Big Data and Master Data come together?

Big Analytics: A Next Generation Roadmap

Impact of Big Data in Oil & Gas Industry. Pranaya Sangvai Reliance Industries Limited 04 Feb 15, DEJ, Mumbai, India.

Next presentation starting soon Business Analytics using Big Data to gain competitive advantage

Big Data Executive Survey

VIEWPOINT. High Performance Analytics. Industry Context and Trends

Create and Drive Big Data Success Don t Get Left Behind

Danny Wang, Ph.D. Vice President of Business Strategy and Risk Management Republic Bank

Understanding Your Customer Journey by Extending Adobe Analytics with Big Data

5 Keys to Unlocking the Big Data Analytics Puzzle. Anurag Tandon Director, Product Marketing March 26, 2014

How Big Is Big Data Adoption? Survey Results. Survey Results Big Data Company Strategy... 6

BIG DATA STRATEGY. Rama Kattunga Chair at American institute of Big Data Professionals. Building Big Data Strategy For Your Organization

Risk & Procurement Analytics: A Promising Future. July 2013

How To Understand The Benefits Of Big Data

The 4 Pillars of Technosoft s Big Data Practice

Deploying Big Data to the Cloud: Roadmap for Success

Apache Hadoop Patterns of Use

Chapter 1. Contrasting traditional and visual analytics approaches

The Canadian Realities of Big Data and Business Analytics. Utsav Arora February 12, 2014

2015 Analyst and Advisor Summit. Advanced Data Analytics Dr. Rod Fontecilla Vice President, Application Services, Chief Data Scientist

Strategic Decisions Supported by SAP Big Data Solutions. Angélica Bedoya / Strategic Solutions GTM Mar /2014

How To Handle Big Data With A Data Scientist

Big Data Are You Ready? Jorge Plascencia Solution Architect Manager

Industry Impact of Big Data in the Cloud: An IBM Perspective

The Business Value of Predictive Analytics

How To Understand The Business Case For Big Data

Big Data Challenges and Success Factors. Deloitte Analytics Your data, inside out

Machina Research. Where is the value in IoT? IoT data and analytics may have an answer. Emil Berthelsen, Principal Analyst April 28, 2016

Hadoop Market - Global Industry Analysis, Size, Share, Growth, Trends, and Forecast,

Analyzing Big Data: The Path to Competitive Advantage

Advanced In-Database Analytics

Majed Al-Ghandour, PhD, PE, CPM Division of Planning and Programming NCDOT 2016 NCAMPO Conference- Greensboro, NC May 12, 2016

The Future of Data Management

Big Data and Analytics in Government

Datenverwaltung im Wandel - Building an Enterprise Data Hub with

Big Impacts from Big Data UNION SQUARE ADVISORS LLC

How To Use Big Data Effectively

EVERYTHING THAT MATTERS IN ADVANCED ANALYTICS

Big Data: What You Should Know. Mark Child Research Manager - Software IDC CEMA

Big Data Use Cases Update

Forecast of Big Data Trends. Assoc. Prof. Dr. Thanachart Numnonda Executive Director IMC Institute 3 September 2014

Business Analytics In a Big Data World Ted Malone Solutions Architect Data Platform and Cloud Microsoft Federal

End to End Solution to Accelerate Data Warehouse Optimization. Franco Flore Alliance Sales Director - APJ

Big Data Analytics: 14 November 2013

Sunnie Chung. Cleveland State University

Hadoop Market - Global Industry Analysis, Size, Share, Growth, Trends, And Forecast,

The Enterprise Data Hub and The Modern Information Architecture

Data Refinery with Big Data Aspects

AGENDA. What is BIG DATA? What is Hadoop? Why Microsoft? The Microsoft BIG DATA story. Our BIG DATA Roadmap. Hadoop PDW

Big Data Analytics for Space Exploration, Entrepreneurship and Policy Opportunities. Tiffani Crawford, PhD

This Symposium brought to you by

Bruhati Technologies. About us. ISO 9001:2008 certified. Technology fit for Business

The Missing Data Scientists.

Integrating a Big Data Platform into Government:

Doing Multidisciplinary Research in Data Science

Demystifying Big Data Government Agencies & The Big Data Phenomenon

BIG DATA FUNDAMENTALS

BIG DATA: FIVE TACTICS TO MODERNIZE YOUR DATA WAREHOUSE

The Next Wave of Data Management. Is Big Data The New Normal?

Transforming the Telecoms Business using Big Data and Analytics

Chapter 6. Foundations of Business Intelligence: Databases and Information Management

Advanced Analytics. The Way Forward for Businesses. Dr. Sujatha R Upadhyaya

Mind Commerce. Commerce Publishing v3122/ Publisher Sample

Big Data Analytics. Copyright 2011 EMC Corporation. All rights reserved.

High-Performance Analytics

Data Virtualization A Potential Antidote for Big Data Growing Pains

Introducing Oracle Exalytics In-Memory Machine

Banking On A Customer-Centric Approach To Data

Big Data & QlikView. Democratizing Big Data Analytics. David Freriks Principal Solution Architect

The 3 questions to ask yourself about BIG DATA

CONNECTING DATA WITH BUSINESS

WHITE PAPER ON. Operational Analytics. HTC Global Services Inc. Do not copy or distribute.

Big Data Analytics Best Practices

Surfing the Data Tsunami: A New Paradigm for Big Data Processing and Analytics

BUY BIG DATA IN RETAIL

Keywords Big Data; OODBMS; RDBMS; hadoop; EDM; learning analytics, data abundance.

Customized Report- Big Data

The Future of Data Management with Hadoop and the Enterprise Data Hub

BIG DATA I N B A N K I N G

Mike Maxey. Senior Director Product Marketing Greenplum A Division of EMC. Copyright 2011 EMC Corporation. All rights reserved.

Big Data Er Big Data bare en døgnflue? Lasse Bache-Mathiesen CTO BIM Norway

Harnessing the power of advanced analytics with IBM Netezza

Navigating Big Data business analytics

Aligning Your Strategic Initiatives with a Realistic Big Data Analytics Roadmap

Exploiting Data at Rest and Data in Motion with a Big Data Platform

Transcription:

Big Data The next big thing

India to capitalize on the growing Big Data market opportunity 1 Global Big Data market opportunity is estimated to grow at 45% annually to reach ~USD 25 billion by 2015, from the current size of about ~USD 8.0 billion Indian Big Data industry expected to grow from ~USD 200 million in 2012 to ~USD 1.0 billion in 2015 at a CAGR of over 83% 2 Big Data industry characterized by technological advancements, niche startups, and large number of M&A s Big Data industry gaining traction in India, with IT services/analytics providers offering business centric solutions 3 Limited talent availability and low awareness about the benefits of Big Data may inhibit market growth India has early mover advantage vis-àvis other geographies in creating a strong base of Big Data workforce 4 Big Data ecosystem continues to evolve over 2012-2013 through sustained investments to strengthen existing technologies & building strong talent pool Indian service providers mirroring their global counterparts with key strategies primarily focusing on talent development, portfolio enhancement, and go-to-market partnerships 2

Big Data is defined by Volume, Variety and Velocity What is Big Data? Big Data relates to rapidly growing, Structured and Unstructured datasets with sizes beyond the ability of conventional database tools to store, manage, and analyze them. In addition to its size and complexity, it refers to its ability to help in Evidence-Based Decision-making, having a high impact on business operations Speed, Accuracy and Complexity of Intelligence Small Data Sets Advanced Small Data Sets Traditional Size of Data Big Data Big Data Gigabytes Terabytes Petabytes Zetabytes Big Data Traditional 3Vs Velocity Volume Variety 3 1 Large quantity of data which may be enterprisespecific or general and public or private 2 Diverse set of data being created, such as social networking feeds, video and audio files, email, sensor data and other raw data Speed of data inflow as well as rate at which this fast-moving data needs to be stored 3

Volume The data being generated globally is undergoing exponential growth Variety Velocity Growth of global data, 2009-2020 Zetabytes CAGR (2009-2020) 41.0% 0.8 1.9 7.9 35.0 2009 2011 2015 2020 Implication on an organization Need for large storage capacity Need for quick retrieval of data Enable informed decision-making effectively, leveraging large data sets. For e.g.,: Turn 12 TB of Tweets created each day into improved product sentiment analysis Convert 350 billion annual meter readings to better predict power consumption Key drivers for deployment of larger datasets Total storage costs, 2005-2015 USD / gigabyte 18.9 1.6 0.7 2005 2011 2015 *CARC Cumulative Aggregate Rate of Change Note: ZB stands for Zetabytes; * Cumulative Annual Rate of Change Source: IDC; CRISIL GR&A analysis Creation of data from multiple sources and touch-points Distributed storage techniques and cloud computing enabling organizations to store large amount of data at lower costs Emergence of innovative open source software and architecture such as Hadoop Distributed File System and MapReduce Roll-out of 100G Ethernet cables for fast information retrieval 4

Volume Today 80% of data existing in any enterprise is unstructured data Variety Velocity Introduction Variety of sources from where data is being generated has also undergone a shift The types of data being created has changed from structured to semi-structured to unstructured data Structured Data Resides in formal data stores RDBMS and Data Warehouse; grouped in the form of rows or columns Accounts for ~10% of the total data existing currently RDBMS (e.g., ERP and CRM Data Warehousing Microsoft Project Plan File Implication for organization Need to manage broad range of data types Process analytic queries across numerous data types Semi-Structured Data A form of structured data that does not conform with the formal structure of data models Accounts for ~10% of the total data existing currently Solutions required Need to extract meaningful analysis from this data has led to several technologies to gain traction Examples include NoSQL databases to store unstructured data as well as innovative processing methods like Hadoop and massive parallel processing (MPP) Unstructured Data Video Web logs & clickstreams Audio Comprises data formats which cannot be stored in row/ column format like audio files, video, clickstream data, Accounts for ~80% of the total data existing currently Text message Sensor data/ M2M Blogs Weather patterns Location coordinates Email Social media Geospatial data Source: Industry reporting; CRISIL GR&A analysis 5

Big Data Analytics helps derive insights from about 1.0 million updates/minute from Facebook & Twitter alone Volume Variety Velocity 7,000+ photos on flickr 700,000+ Facebook updates 1,500+ blog posts 600+ videos on YouTube Data velocity/ per minute 200 million+ emails sent 2 million+ Google search queries 400,000+ minutes of Skype calling Big Data velocity enabling real time use of data Big Data is also characterized by velocity or speed i.e. frequency of data generation or the frequency of data delivery New age communication channels such as mobile phones, emails, social networking has increased the rate of information flows Examples: Location based marketing based on user location sensed by mobile towers Satellite images can help monitor and analyze troop movements, a flood plane, cloud patterns, or forest fires USD 300,000+ are spent on online shopping 400,000+ tweets on Twitter 3500+ ticks per minute in securities trading Video analysis systems could monitor a sensitive or valuable facility, watching for possible intruders and alert authorities in real time Source: Industry reporting; CRISIL GR&A analysis 6

Level of Complexity Big Data is application of advanced techniques on Big Datasets; answers questions previously considered beyond reach Evolution of Prescriptive Predictive Descriptive Standard reports Adhoc reports Big Data is where advanced analytic techniques are applied on Big Data sets The term came into play late 2011 early 2012 Query drill down Alerts Statistical analysis Big Data Complex event processing Extreme SQL Forecasting Predictive modeling Optimization Stochastic optimizatio n Natural Language Processing Multivariate statistical analysis Online analytical processing (OLAP) Data mining Basic What happened? When did it happen? What was the its impact? Time series analysis Behavioral Social network Semantic Visualization Analytic database functions Advanced Constraint based BI Why did it happen? When will it happen again? What caused it to happen? What can be done to avoid it? Late 1990s 2000 onwards Analytics as a separate value chain function Time In-database 7

Technology and Services players are investing to build platforms, service models and Big Data skills Key Players Hardware/Software Providers IT and IT Enabled Service Providers Most of the technology and software companies in the Big Data space are focusing on Storage technology, parallel computing and visualization techniques Most of the Indian IT players have set up internal COEs, dedicated teams to execute pilot projects and proof of concepts around Big Data Pure play Analytics companies are investing in technology/programming skills to be able to help clients in their Big Data projects Source: Industry reporting; CRISIL GR&A analysis 8

Financial Services, Retail and Telecom are likely to be the early adopters of the Big Data owing to enormous digital data flow Financial services Retail Indian service providers like Infosys, Fractal are enabling Big Data in the area of fraud detection, CRM and customer loyalty program, trading pattern analysis, risk calculation on large portfolio of loans Key Adopters: JPMorgan Chase, Merrill Lynch, HSBC, American Express, Goldman Sachs, Barclays, Bank of America, Citigroup, and Wells Fargo Both brick and mortar as well as online retailers are increasing their adoption of Big Data for real time analysis of purchase behavior and buying patterns, enhanced customer segmentation and customer loyalty Key Adopters: Walmart & Sears Telecom Telecom players are increasingly focusing on Big Data to limit churn rates, build loyalty and provide multichannel and multi-service applications by proactively analyzing the subscriber and network data Key Adopters: Airtel ( India ) Manufacturing Indian service providers are enabling manufacturing companies through Big Data in the areas of accurate demand forecasting, optimization of operations, inventory management, open innovation and better analysis of post sales feedback in real time Public Sector Key benefits of big data in public sector include: Intelligence to counter national threats, Forecast economic events, Traffic management, Environment monitoring, energy/ waste management, etc. Healthcare Healthcare players use Big Data Next-generation sequencing and mapping for genomics, analysis of correlation between treatments & outcomes and real time data from medical devices for better patient care Source: Industry reporting; CRISIL GR&A analysis 9

Large IT players leveraging M&As to add Big Data capabilities to their service portfolios 3 Area Acquirer Target Company Date Deal value Rationale Oct. '11 USD 1.1 billion Develop a comprehensive platform to analyze Big Data Data Management Oct. '11 USD 10.2 billion Leverage Autonomy s pattern-matching technology that recognizes unstructured data, understands it and processes it Mar. '11 USD 263 million Strengthen position in data warehousing market through expertise in SQL and MapReduce-based analysis Jun. '12 N.A. Extend Smarter Commerce suite with qualitative software May. '12 N.A. Leverage data navigation technologies for Big Data by automating discovery of through innovative index and search capabilities Advanced May. '12 N.A. Addition of sales performance May. '12 N.A. Enhance Big Data marketing Apr. '12 N.A. Acquisition of spend and procurement Mar. '12 N.A. Accelerate development of Big Data analytic applications Mar. '11 N.A. Enhance real time business for Big Data Key highlights M&As in the Big Data space have tripled in the first half of 2012 Acquisition targets are mainly innovative Big Data start-ups N.A. is not available. Source: Industry reporting; CRISIL GR&A analysis M&As with bigger deal value are happening in data management 10

Global Big Data market to reach ~USD 25 billion by 2015,with a 45% share of IT & IT-enabled services Global Big Data Market Size, 2011 2015E USD billion Global Big Data market by category, 2015F Percent 100%= ~USD 25.0 billion 24.0-26.0 29%-31% 44%-46% IT & IT enabled Services Hardware 5.3-5.6 8.0-8.5 Software 24%-26% 2011E 2012E 2015F Rising need to focus on leveraging multiple data sources, to maintain competitive advantage Shortage of analytical and managerial talent Drivers Continued investments from service providers in Big Data technologies and tools, particularly in Big Data Companies to define Big Data implementation strategy for taking more informed decisions Big Data deployments to become mainstream in verticals like utilities, transportation, energy, etc. Restraints Regulations about data privacy and security Roadblocks in managing change during transition to data driven organizations Restrained adoption in new geographies such as the Middle East Source: Industry reporting; CRISIL GR&A analysis 11

India s opportunity is expected to be ~8.9% of Global Big Data IT and IT enabled services market by 2015E India Big Data outsourcing opportunity, 2011 2015E USD billion 1.1-1.2 India Big Data outsourcing opportunity, by category, 2015F, Percent 100%= ~USD 1.1 billion 24%-27% Pure-play Analytics firms ~0.1 ~0.2 73%-76% Integrated IT/ BPO players 2011E 2012E 2015F Key growth drivers include: 1 Aggressive efforts for Big Data specific talent development 2 3 Increasing domain expertise and breadth of services offered by Indian service providers Effective strategies of Indian service providers through synergies of international partnerships 4 Rising domestic demand for Big Data implementations across industry verticals Source: Industry reporting; CRISIL GR&A analysis 12

Emergence of niche startups and technological developments fostering growth in the Big Data industry 1 2 3 4 Market Trends and Developments Converging technology trends in data storage, processing, and are driving adoption Emergence of niche Big Data startups driving technological innovation Large IT players leveraging M&A s to add Big Data capabilities to their service portfolios Talent shortage is one of the biggest challenges of the Big Data space Impact on Industry 5 Lack of awareness about benefits of Big Data may limit enterprise adoption 6 Regulations driving the adoption in various industry verticals Positive impact Neutral/ Limited impact Negative impact 13

4 Potential shortfall of 1.5 million Data-Savvy Managers and ~150,000 Data Scientists in the US in 2018 Demand-supply gap for data scientists* in US, 2018 440K-490K Role in Ecosystem Requisite educational qualifications Other expertise 300K 140K 190K 50%-60% gap relative to supply Data Scientists Big Data Business intelligence Visualization Advanced degree like M.S. or Ph.D., in mathematics, statistics, economics, computer science or any decision sciences Expertise in data skills to extract data, use of modeling & simulations Multi-disciplinary knowledge of business to find insights 2018E Supply 2018E Demand Demand-supply gap for data-savvy managers* in US, 2018 4.0 million Data-savvy Managers Project management across the Big Data ecosystem - Consulting services - Implementation - Infrastructure management - Analytics Advanced business degree such as MBA, M.S. or managerial diplomas Knowledge of statistics and/or machine learning to frame key questions and analyze answers Conceptual knowledge of business to interpret and challenge the insights Ability to make decisions using Big Data insights 2.5 million 2018E Supply 1.5 million 60% gap relative to supply 2018E Demand Technical Engineers Technical support in hardware & software across the Big Data ecosystem for: - Data architecture - Data administration - Developer environment - Applications Having a degree in computer science, information technology, systems engineering. or related disciplines *Analysts with deep analytical training; **Managers to analyze Big Data and make decisions based on their findings; Source: McKinsey Global Institute; CRISIL GR&A analysis Possessing data management knowledge IT skills to develop, implement, and maintain hardware and software 14

4 India has an early mover advantage vis-à-vis other geographies in creating a strong base of Big Data workforce India s Big Data talent requirement*, 2011-2015 Annual potential talent pool available for Big Data in India ~15,000 20,000 IT Professionals Graduates in Math/science 700,000 2,800,000 ~1,000-1,500 ~4,000 6,000 Engineers Graduates in Economics Post graduates in Math/science MBAs PhDs 500,000 350,000 300,000 250,000 14,000 While total talent availability is high, ready to hire talent for Big Data would be just ~3%-5% of the total 2011E 2013F 2015F State of Big Data talent across key geographies Graduates in statistics 5,000 United States Talent availability Ready to hire talent Service provider efforts to build such talent Overall relative ranking High High High I India High Medium High II Poland Medium High Medium III China High Low Low IV India is expected to be in the forefront of Big Data and related IT services, fueling demand for data scientists and IT engineers estimated at about 15,000 20,000 by 2015 India follows closely behind the US in terms of Big Data talent availability and service provider's initiatives to build such talent India is ahead of most outsourcing destinations like China, Poland and Philippines, in terms of talent availability. - IT companies like EMC, Oracle, IBM, Infosys, etc., are leveraging their academic alliance programs, with universities in India and overseas to introduce courses in various areas of Big Data Philippines Low Low Low V *Includes data scientists and technical engineers Source: Industry reporting; CRISIL GR&A analysis High Medium Low 15

www.crisil.com/gra