IDC FutureScape: Worldwide Big Data and Analytics 2016 Predictions APEJ Implications

Transcription

1 IDC FutureScape: Worldwide Big Data and Analytics 2016 Predictions APEJ Implications Qiao Li, Sr. Market Analyst, Big Data & Analytics Chris Zhang, Sr. Market Analyst, Software Chwee Kan Chua, AVP, Big Data & Analytics, Cognitive Computing December 2015 An IDC Excerpt Please refer to for the worldwide report # & #AP

2 In this Study This IDC study provides IDC's top 10 Asia/Pacific (excluding Japan), or APEJ, predictions for the 2016 IT buyer when it comes to understanding and investing in Big Data and analytics (BDA) technologies and services. Adoption of BDA will scale across APEJ in 2016 and beyond. With the digitalization of everything, the ability to take smarter, quicker, and more automated decisions and actions will increasingly become a competitive necessity. We expect adoption to scale mainly from the most ambitious organizations to the broader market players. Leading organizations will scale their BDA use to be cross-functional, combining analytics with other applications and embedding intelligence in every process in the next few years. This document offers IDC analysts' collective understanding of major industry transitions and advice to IT buyers to consider in their strategic planning in relation to Big Data and analytics investment and operations. We advise decision makers to approach each prediction in three steps: Assess its relevance: Should I pay heed to this prediction? Does this prediction apply to me? Can I reasonably enough ignore it? What do I risk if I ignore it? Strategy is, after all, as much about what you decide to do as what you decide not to do. Assess its urgency: Does it apply to me now or in the future? If it applies in the future, when do I have to get started to deliver enabling capabilities as needed? Assess its resource requirements: What resources do I need and at what costs? What would I have to forego or postpone to achieve the capability? What do I have to speed up to achieve it? What priority does this prediction have relative to other projects consuming resources? In the following slide, Figure 1 presents IDC's Big Data and analytics top 10 predictions in terms of their likely impact across the enterprise and the time it will take for the predictions to reach mainstream. By mainstream, IDC means the broad middle of the bell curve of adoption (i.e., the 40 60% of enterprises that are neither the first movers and early adopters nor the last to act). Each bubble's size provides a rough indicator of the complexity and/or cost an enterprise will incur in acting on the prediction. IDC Visit us at IDC.com and follow us on 2

3 A single department or a business unit ORGANIZATIONAL IMPACT Multiple departments or business units Companywide Worldwide Big Data and Analytics Predictions: APEJ Implications FIGURE Cloud BD&A. Through 2020, spending on cloud-based BDA technology will grow 3x faster than spending for on-premises solutions in APEJ; open source technology will represent the core of this new architecture. Cognitive computing. By 2020, 40% of all business analytics software will incorporate prescriptive analytics built on cognitive computing functionality. Labor shortage. Shortage of skilled staff will persist and extend from data scientists to architects and experts in data management; Big Data related professional services will have a 29% CAGR in APEJ by In-memory computing. By 2020, 75% of databases (relational and nonrelational) will be based on memory-optimized technology. Distributed micro analytics. By 2020, distributed micro analytics and data manipulation will be part of 80% of Big Data and analytics deployments. Self-service. Through 2020, spending on self-service visual discovery and data preparation market will grow 2.5x faster than traditional ITcontrolled tools for similar functionality. Data monetization. By 2020, data monetization efforts will result in enterprises pursuing digital transformation initiatives, increasing the marketplace's consumption of their own data by 100-fold or more. Analyzable data. By 2020, the high-value data part of the Digital Universe that is worth analyzing to achieve actionable intelligence will double TIME (MONTHS) TO MAINSTREAM Worldwide APEJ 9 10 Actionable information. By 2020, 40% of information delivered to decision makers will be considered by them as always actionable, doubling the rate from the current (2015) level. BDA value. By 2020, organizations able to analyze all relevant data and deliver actionable information will achieve an extra US$65 billion in productivity benefits over their less analytically-oriented peers. 3

4 Prediction #1: Cloud-Based BDA Technology Through 2020, Spending on Cloud-Based BDA Technology Will Grow 3x Faster Than Spending for On-Premises Solutions in APEJ; Open Source Technology Will Represent the Core of This New Architecture. IT impact Complexity in managing on- and off-premises data from disparate sources and types in a hybrid environment will persist. Integration, orchestration, provisioning, and data management requirements will increase during the transition to cloud. IT needs to work closely with business to identify and prioritize workloads to move to the cloud, if not all entirely. New skill sets are required for open source and cloud technology. BU impact Data and privacy regulations differ across countries, preventing full usage of cloud-based BDA. Security and privacy policies need to be well understood and weighed against potential benefits and risks, alongside with regulations and accountability. The enterprise may be exposed to greater security and continuity risks, as well as vendor lock-ins, particularly with the use of public cloud services. Opex model will enable organizations (and new segments such as SMBs, which might not have the monetary resources previously) to embark on BDA initiatives. 4

5 Prediction #1: Cloud-Based BDA Technology Through 2020, Spending on Cloud-Based BDA Technology Will Grow 3x Faster Than Spending for On-Premises Solutions in APEJ; Open Source Technology Will Represent the Core of This New Architecture. Essential guidance for tech buyers Hire the right skill sets (e.g., data architects, data management experts) to manage data and systems in a hybrid environment. Engage with cloud service providers to assess their commitment to open source and the value-add they provide with their commercial solutions. Develop a detailed profile of security requirements across business processes, LOBs, and geographies as well as establish critical continuity requirements involving recovery point objectives (RPOs) and recovery time objectives (RTOs). 5

6 Prediction #2: Cognitive Computing Functionality By 2020, 40% of All Business Analytics Software Will Incorporate Prescriptive Analytics Built On Cognitive Computing Functionality IT impact The use of cognitive computing systems will uncover new insights but also shed light on data in ways that may expose new data privacy and access issues think of the Snowden effects. Involvement in the training of such prescriptive analytics will require the involvement of both content experts, and business and IT users, who will all need to collaborate more closely. BU impact Early adoptions will realize significant competitive advantage from the use of cognitive-based platforms over more risk-averse organizations. Initial distrust of the prescriptive solution will have to be overcome with user education/training, the introduction of new information governance, and transparency policies and procedures. Demand for self-service and cognitive-based services will increase, from both external forces (e.g., customer service) and internal stakeholders (e.g., employees' training programs). Cognitive computing will also challenge some white-collar professional workers and their existing roles in the organization. 6

7 Prediction #2: Cognitive Computing Functionality By 2020, 40% of All Business Analytics Software Will Incorporate Prescriptive Analytics Built on Cognitive Computing Functionality Essential guidance for tech buyers IT leaders need to evaluate existing BDA investments and vendors product road maps to ensure relevancy (protect vs. rip and replace). Ensure that IT has a seat at the table when projects to train and curate content needed to train cognitive computing systems are launched. Employees need to realize and evolve with the aid of automated data science that frees them up from certain tasks and allows them to engage in more high-value activities. Humans are the ultimate decision makers. 7

8 Prediction #3: Shortage of Skilled Staff Shortage of Skilled Staff Will Persist and Extend from Data Scientists to Architects and Experts in Data Management; Big Data Related Professional Services Will Have a 29% CAGR in APEJ by IT impact Unfulfilled end-user expectations will further sour the already tense relationship between business and IT at many organizations. Having the right data architecture and data and preparation skills will go a long way in ensuring the fulfillment of end-user BDA expectations. Most technical workforce are still equipped with legacy programming and traditional database skill sets, while business users struggle with know-how to use the analysis of Big Data to make effective decisions. BU impact Business process changes will affect IT's role and impact on the organizations. There will be a need for ever closer collaboration with line-of-business colleagues. The right combination of business and analytical skills among existing users to make business decisions based on analytics is lacking. 8

9 Prediction #3: Shortage of Skilled Staff Shortage of Skilled Staff Will Persist and Extend from Data Scientists to Architects and Experts in Data Management; Big Data Related Professional Services Will Have a 29% CAGR in APEJ by Essential guidance for tech buyers Identify skills gap among the existing workforce and provide upgrading/re-training opportunities to handle BDA technology. Talent quest and retention bright brains flock to Facebook, Google to work on leading edge and more exciting projects. Need to offer more than paycheck. Provide proper incentives to motivate and empower employees to adopt and integrate BDA as part of their culture. Consider external service providers. Business processes will often have to be changed as the organization takes on the digital journey. Business consulting services will be critical in these business process transformation projects. 9

10 Prediction #4: Memory-Optimized Technology By 2020, 75% of Databases (Relational and Nonrelational) Will Be Based on Memory-Optimized Technology IT impact Adoption of real-time streaming analytics will require IT to incorporate results of this type of analytics into operational applications such as ERP. The new data architecture will require significant enhancements to existing applications or development of new applications that can take advantage of the new memory-based database platform. Memory-optimized technology does not demand a great deal of training to understand or consulting to implement. In fact, it makes a DBMS simpler to set up and manage. BU impact SMBs or some applications that do not require low latency access may not be able to justify investment costs. BU will need to identify bottlenecks and assess suitability of memory optimized databases not all workloads/applications are created equal. Organizations will have to promote a data-driven culture within to take full advantage of the real-time insights made possible by memory-optimized technology. New consulting services may very often be required. 10

11 Prediction #4: Memory-Optimized Technology By 2020, 75% of Databases (Relational and Nonrelational) Will Be Based on Memory-Optimized Technology Essential guidance for tech buyers Do not get too caught up in benchmarks and performance claims on a comparative basis, but pick a vendor and technology that are trustworthy and compatible with your organization's data management strategy. The new data architecture will require significant enhancements to existing applications or development of new applications that can take advantage of the new memory-based database platform. IT will need to ensure that data and technology are available to support the real-time information access and analysis capability. 11

12 Prediction #5: Distributed Micro Analytics By 2020, Distributed Micro Analytics and Data Manipulation Will Be Part of 80% of Big Data and Analytics Deployments IT impact Distributed micro analytics and data manipulation services will change how analytic applications are developed. IT will need to adjust their practices accordingly to take advantage of the most appropriate services to provide their organization with tailored capabilities. The management of these components will initially be complicated due to the involvement of services from multiple vendors. IT will have to guide the organization's decision about which applications can tolerate such complexity. BU impact If done right, agility will be a key benefit to adopt distributed services in Big Data and analytics deployment. Organizations can expect more agile development and flexible changes to drive experimentation and innovation. Business users, especially business analysts, need to be actively involved in the procurement and deployment processes to ensure the analytics applications align to business requirements. 12

13 Prediction #5: Distributed Micro Analytics By 2020, Distributed Micro Analytics and Data Manipulation Will Be Part of 80% of Big Data and Analytics Deployments Essential guidance for tech buyers Be prepared to change the development team's structure and practices to ready it for a more agile and faster development cycle that consists of an assembly of fit-for-purpose components available as both cloud and onpremises services. Understand the inventory of micro analytics and data manipulation services available in the market, the relationship among vendors providing such services, and the underlying core component (some open source). Proceed with an incremental approach in deploying distributed micro analytics, and be aware that design and management can be complex. 13

14 Prediction #6: Self-Service Through 2020, Spending on Self-Service Visual Discovery and Data Preparation Market Will Grow 2.5x Faster Than Traditional IT-Controlled Tools for Similar Functionality IT impact Responding to the demand for self-service BDA technology will necessitate a reassessment of current centralized IT practices. IT will need to recognize the full range of different BDA needs and ensure that the full technology stack or services are available to address the self-service needs of the user group. Self-service will allow business users to rely less on IT, but IT still needs to collaborate closely with LOB to ensure that data governance and security processes are in place. BU impact Business users and analysts will find it easier to manipulate and present data on a daily basis. Training on data manipulation, visualization, and interpretation will still be necessary to ensure a wider adoption of self-service analytics. Organizations that encourage use of data, data discovery, and best practices sharing will build a data-driven culture across BUs. 14

15 Prediction #6: Self-Service Through 2020, Spending on Self-Service Visual Discovery and Data Preparation Market Will Grow 2.5x Faster Than Traditional IT-Controlled Tools for Similar Functionality Essential guidance for tech buyers Because of the unpredictable nature of self-service data preparation and analysis, IT will have to incorporate more analytics into the underlying BDA platform to ensure workload requirements are met with minimal ongoing maintenance costs. Organizations should focus on data governance, security, training, and vendor management and not attempt to prescribe all the needed end-user technologies. Sharing and collaboration among IT, business analysts, and business users will be key to translate data into value in business processes. 15

16 Prediction #7: Data Monetization By 2020, Data Monetization Efforts Will Result in Enterprises Pursuing Digital Transformation Initiatives, Increasing the Marketplace's Consumption of Their Own Data by 100-fold or More IT impact Packaging and pricing methods for licensing data to third parties that are germane to the type of data and industry being served will need to be developed. Securing, masking, and cleansing data will be required to prepare it for sale. IT needs to drive infrastructure modernization to allow easy integration, access, and consumption of data such that BUs can experiment, share, and innovate new monetization approaches. BU impact LOB and the senior leadership team will be at the center of driving the data monetization efforts, because they have the domain knowledge of the data and of how they can be packaged to serve their current and adjacent market. New data monetization models are increasingly being created via API programs, which will only be capitalized on if the BU collaborates closely with IT, or has its own internal technology capability. Analytics talent will be crucial to building commercial value-added contents. 16

17 Prediction #7: Data Monetization By 2020, Data Monetization Efforts Will Result in Enterprises Pursuing Digital Transformation Initiatives, Increasing the Marketplace's Consumption of Their Own Data by 100-fold or More Essential guidance for tech buyers Organizations shall consider not only legal requirement but also consumer expectation of privacy. To ensure compliance, IT will need to participate in data governance and privacy committees. IT will need to support systems focused on making data and value-added content available to external parties. CXOs investing in data monetization will need to drive innovative monetization models. They also need to put acquisition and retention of data analytics talent as a top priority. 17

18 Prediction #8: Analyzable Data By 2020, the High-value Data Part of the Digital Universe That Is Worth Analyzing to Achieve Actionable Intelligence Will Double IT impact IT groups will have to become knowledgeable in data management and analytics technology that addresses many forms of unstructured content. The use of rich media analytics will have a significant impact on BDA infrastructure (storage and compute) investment. Capacity planning will account for capturing, storing, and retaining a greater volume of high-value data. BU impact Integration and analysis of different high-value data sources will provide additional insights and perspectives for decision making. BUs will see the need to evolve business processes to fully leverage actionable intelligence such as preventive maintenance based on sensor data analysis, proactive surveillance based on video analytics, and omnichannel customer services based on customer interactions. 18

19 Prediction #8: Analyzable Data By 2020, the High-value Data Part of the Digital Universe That Is Worth Analyzing to Achieve Actionable Intelligence Will Double Essential guidance for tech buyers Image, video, and audio analytics technology and vendors providing these solutions need to be considered as part of the broader BDA technology portfolio. Data architecture will need to consider different data sets, including rich media data, as critical assets. The skills to build relevant data architecture and analyze all high-value data will need to be developed. 19

20 Prediction #9: Actionable Information By 2020, 40% of Information Delivered to Decision Makers Will Be Considered by Them as Always Actionable, Doubling the Rate from the Current (2015) Level IT impact Current data delivery methods and practices that rely mostly on reports and dashboards fail to deliver actionable information to most internal users. This negatively impacts IT's value proposition and relationship with business users. The definition of actionable incorporates factors about data such as timely, trusted, high-quality, complete, and granular. The IT role has to expand to encapsulate processes and technology to ensure a high level for all of these factors. BU impact The BU can expect more prescriptive insights delivered to it. But business users need to work with business analysts to ensure these information align with their business priorities. Business processes and culture need to evolve to leverage these actionable insights in appropriate business processes. BU needs to provide KPIs and feedback to continuously improve the information delivered. 20

21 Prediction #9: Actionable Information By 2020, 40% of Information Delivered to Decision Makers Will Be Considered by Them as Always Actionable, Doubling the Rate from the Current (2015) Level Essential guidance for tech buyers It's critical that BDA projects can be tied to business value and vision to ensure information derived from BDA projects is considered actionable. Delivery of actionable information should be viewed across the enterprise not just to analysts. Today, frontline employees are least supported with actionable information a shortcoming that should be addressed by integrating analytics with transactional or operational solutions. Business analysts, users, and IT shall collaborate to continuously improve information delivered, deriving more actionable insights consistently as business evolves. 21

22 Prediction #10: BDA Value By 2020, Organizations Able to Analyze All Relevant Data and Deliver Actionable Information Will Achieve an Extra US$65 Billion in Productivity Benefits Over Their Less Analytically-Oriented Peers IT impact The performance gap between more analytically oriented and data-driven organizations and their less data-savvy peers will widen. In the commercial sphere, the true competitive advantage will start manifesting itself in financial metrics. The success of organizations that achieve outsized benefits will place additional requirements on IT to make BDA solutions more pervasively available and participate in new forms of agile experimentations that continuously evaluate new solutions, including the performance, scalability, and functionality. BU impact Early adopters that transform their processes to be more analytics-driven will enjoy greater performance benefits compared with their peers. BU will set clearer KPIs to measure quantifiable benefits, both for accounting success and identifying gaps. Companies will find it even harder to retain good analytics talent as more organizations realize benefits from analytics adoption. 22

23 Prediction #10: BDA Value By 2020, Organizations Able to Analyze All Relevant Data and Deliver Actionable Information Will Achieve an Extra US$65 Billion in Productivity Benefits Over Their Less Analytically-Oriented Peers Essential guidance for tech buyers Set measurable and realistic KPIs in terms of productivity, revenue, or costs at the start of any BDA initiative this will measure successes and guide future investment. IT should play a more active role in measuring the impact of better data and analytics. Business analysts and IT should play a more active role in acting as a center of knowledge sharing to ensure BDA best practices are effectively disseminated across the enterprise. 23

24 Predictions at a Glance: APEJ Implications 1. Cloud BD&A. Through 2020, spending on cloud-based BDA technology will grow 3x faster than spending for on-premises solutions in APEJ; open source technology will represent the core of this new architecture. 2. Cognitive computing. By 2020, 40% of all business analytics software will incorporate prescriptive analytics built on cognitive computing functionality. 3. Labor shortage. Shortage of skilled staff will persist and extend from data scientists to architects and experts in data management; Big Data related professional services will have a 29% CAGR in Asia/Pacific by In-memory computing. By 2020, 75% of databases (relational and nonrelational) will be based on memory-optimized technology. 5. Distributed micro analytics. By 2020, distributed micro analytics and data manipulation will be part of 80% of Big Data and analytics deployments. 6. Self-service. Through 2020, spending on self-service visual discovery and data preparation market will grow 2.5x faster than traditional IT-controlled tools for similar functionality. 7. Data monetization. By 2020, data monetization efforts will result in enterprises pursuing digital transformation initiatives, increasing the marketplace's consumption of their own data by 100-fold or more. 8. Analyzable data. By 2020, the high-value data part of the Digital Universe that is worth analyzing to achieve actionable intelligence will double. 9. Actionable information. By 2020, 40% of information delivered to decision makers will be considered by them as always actionable, doubling the rate from the current (2015) level. 10. BDA value. By 2020, organizations able to analyze all relevant data and deliver actionable information will achieve an extra US$65 billion in productivity benefits over their less analyticallyoriented peers. 24