Self-Service Big Data Analytics for Line of Business

Transcription

1 I D C A N A L Y S T C O N N E C T I O N Dan Vesset Program Vice President, Business Analytics and Big Data Self-Service Big Data Analytics for Line of Business March 2015 Big data, in all its forms, is being analyzed across industries, company sizes, and geographies. Studies show a causal relationship between data-driven decision making and better business outcomes be they profitability and competitive advantage or fulfillment of a chosen mission. Ultimately, big data collection and storage is not an end in itself. It's about analyzing the data to support or automate decisions that drive positive outcomes. Deploying self-service analytics can yield faster time to insight by making more comprehensive and more granular data available to the business analyst and other line-of-business (LOB) practitioners. Self-service solutions also free IT to address other requirements needed to capitalize on big data assets. The following questions were posed by Datameer to Dan Vesset, program vice president of IDC's Business Analytics and Big Data research, on behalf of Datameer's customers. Q. It seems like we have reached a point in time when there is less talk about what is big data and more about what can and should be done with data and analytics. How can big data be used as a competitive advantage? A. It is important not to view big data technology projects as an end in themselves. They should always align with specific organizational goals. Competitive advantage can be derived from the ability to better anticipate the future, react faster to external (or internal) events, or more optimally manage today's resources. For instance, large industrial companies that run power plants or sell and service heavy machinery are doing just that by focusing on predictive asset maintenance. They use data from sensors and operational systems of these big assets (the Internet of Things data) as well as financial systems data to project the likelihood of adverse performance or of an adverse event that might cause a shutdown. Some companies have found success in being able to prevent such drastic measures by finding a precursor to a big failure in the data and then fixing it before something costly or catastrophic happens. Other organizations have found competitive advantage in changing how they interact with their customers thanks to faster access to more comprehensive data. Instead of just looking at call records or transaction histories, they're combining that internal customer data with data from external sources, such as a customer interaction with the company on mobile apps, Facebook or Twitter, Web sites, digital ads, and campaigns; they're applying content analysis to that customer's interaction with the company together with third-party data IDC 1889

2 such as cookies or credit ratings. By combining structured, semistructured, and unstructured data, leading retailers, financial services firms, and telecommunications companies are extracting insights to better service the customer. Of course, the goal is not always competitive advantage, especially in the public sector, healthcare, or education. There it may be a different organizational goal: better graduation rates or positive treatment outcomes. Again, the benefits come from faster time to insight and the more comprehensive and more granular data available to analysts and decision makers. Q. Over the past two or more years, most of the big data market attention has been on data scientists. That being the case, how well do you think the needs of business analysts are addressed in today's market? A. Not very well. In our research, only 30% of business analysts say their business analytics technology meets their own analytics and decision support requirements. There is no question that data scientists those with a mix of deep analytics, computer science skills, and business expertise can have a strong impact on an organization's ability to analyze data. However, for every data scientist, there are dozens of business analysts who require access to information. So the almost exclusive focus on the data scientist is not the optimal approach to the deployment of big data solutions. Today, the mantra of business analysts is self-serve. Everyone wants data at their fingertips in a way that makes it easy to analyze. However, there's tension between the line of business, the business analyst, and IT. Centralized IT may think that it controls the data and all the processes related to managing and analyzing it. In response, business analysts turn to their own data and technology. They try to bypass IT or create "shadow IT." While that approach might work short term for a particular analyst or their immediate group, longer term, it exposes the organization to the greater risks of inconsistent metrics, poor data quality and, potentially, data security issues. So it's incumbent on all organizations to ensure that IT recognizes the value of the business analyst and that it doesn't try to do all things data related. IT certainly has an important role in ensuring the infrastructure exists to make the appropriate enterprise data sources available and to leverage big data to facilitate faster data access to the line of business. But the analysis and the tools to view that analysis should be left to the business analyst. There's a need to ensure that this division of labor exists to guarantee that the core competencies of each group are maximized. Q. One of the biggest drivers of the big data and analytics market is the demand for selfservice. The prevailing view of self-service is mostly about ad hoc data visualization, yet IDC studies have shown that in fact 80% or more of a typical business analytics project can be consumed by data integration and preparation. How are companies addressing this barrier to true self-service and faster time to decisions? A. First, there can't be true end-user self-service in data analysis without considering the analytic process from data acquisition to facilitating action. Most solutions focus on the middle step of self-service data manipulation: slicing and dicing and visualizing data. Very few also support self-service data acquisition and even fewer support the ability to automate taking actions based on the analysis, such as pushing a data set that meets certain criteria into a marketing automation application. There is a need to extend self-service to incorporate these steps of the data analysis pipeline. One of the emerging best practices that overcome these challenges involves enabling business analysts with technology that makes them less dependent on the historically high IDC 2

3 friction relationship with IT. Typically, line-of-business requirements are passed to the IT group, which conducts a formal requirements-gathering process, then develops new technology and/or identifies relevant data sources, followed by delivery of the originally requested information to users. This process can take weeks or months. Leading organizations are changing the relationship between central IT, analysts, and LOB users. These organizations are embracing rather than fighting the concept of "shadow It." Instead of controlling all things data, the IT group in these organizations focuses on on-premise technology provisioning, data governance, security, and vendor management tasks. IT also often acts as a facilitator of knowledge sharing across the organization. This attitude and approach allows analytics and LOB groups to focus on what they do best with the tools that fit them best. Increasingly, these are tools that enable on-demand integration of structured, semistructured, and unstructured data and the analysis of this data using interactive and visual interfaces. In many cases, Hadoop-based technologies are the go-to analytics software to support the scenario described previously. This represents another emerging best practice in the big data and analytics arena architectures that embrace fit-for-purpose technologies and augment existing relational data warehouse platforms. The latter continues to serve an important need for highly structured reporting in support of performance management, but use cases such as exploration and discovery are best served by nonrelational or nonschematic technologies. Q. What do business executives expect to discover from the use of big data in the areas of marketing or operations? A. Some executives have very high expectations and hire data scientists to uncover the "hidden nuggets" within large data sets in the belief that unique information (unknown to their company's experienced staff) is hidden there that could drive competitive advantage. The reality in most cases is that the data and analytics helps support and enhance the existing processes by allowing decision makers to ask new questions, to iterate through potential scenarios faster, or to assess risk-adjusted options based on comprehensive data sets. These insights drive operational and tactical decisions that can result in significant returns. A couple of recent examples include a communications company identifying $150 million in savings from network capacity optimization and an energy firm gaining an extra $120 million from additional oil production. Consider marketing campaign optimization in retail. The process itself has existed since there have been retailers, but what it takes to succeed in today's market is substantially different. With a multitude of customer interaction touch points, the volume and variety of data needed to integrate and analyze has expanded drastically. And that is what executives expect: That analyzing the data can change how campaign optimization or fraud detection or customer segmentation or logistics optimization is done. There's now an ability to apply technology to automate certain processes or just make them more efficient through the use of appropriate tools. There's less manual work for systems administrators, for database administrators, for data preparation professionals, and even for the analysts themselves. Another example is fraud detection. It is a process associated with financial services, but it is relevant across other industries such as retail and government. Retailers are combining and analyzing internal data and external data to uncover nonobvious relationships among employees to identify insider theft. Federal agencies are combining vast amounts of internal operational and reference data from a range of entitlement systems with geospatial data and external transactional data to identify individuals and organized crime groups that are defrauding the government. Any time there's an opportunity to prevent fraud or to uncover it 2015 IDC 3

4 faster, that's a direct benefit to the bottom line and something that is on the mind of every executive. Q. What are the top industries using big data, and can you elaborate on the top use cases for each example? A. Big data is applicable across industries. It's prevalent in financial services, manufacturing, and professional services, such as advertising, information services, software development, engineering, consulting, auditing, and law firms. There are also successful cases in telecommunications, retail, education, utilities, oil and gas, and other economic sectors. The top use cases can be broadly categorized as analytics about people, things, and money. In the first category are customer analytics in industries such as telecommunications, retail, financial services, and media and entertainment. But this category also includes analytics about alternate definitions of customers such as patients in healthcare, students in education, or citizens in the public sector. In all cases, the focus is on a deeper understanding of human behavior by combining transactional, behavioral, attitudinal, interactional, and descriptive data from a variety of internal and external sources. In the second category are operational analytics in asset-intensive industries such as utilities, oil and gas, manufacturing, and logistics. These include applications such as predictive maintenance, supply chain optimization, improved demand and supply planning, production line quality monitoring, and a growing number of cases related to the analysis of the Internet of Things data from sensors on physical products and infrastructure. In the third category is financial, fraud, and risk analytics in industries such as financial services and government, but this category is applicable across industries depending on the definition of "risk." In most cases, application of big data analytics in any given industry and organization applies across all three of these categories. For example, when a telecommunications company acts as a service provider, it is looking for better intelligence on customer behavior to help prevent customer churn and to upsell additional services. At the same time, big data is applied to internal decision making in network capacity optimization to ensure that the service the company provides meets customer expectations while minimizing operational costs. The same company may also be applying big data analytics to revenue assurance when tracking the complex web of cross-industry payment processes. In financial services, we see risk-based or real-time risk assessment applied to portfolio analysis, to lending practices, and to customer retention and sales processes. Big data technology today allows for much faster assessment of risk decisions. In some cases, risk decisions are done instantly and influence how lending or investment decisions get made. In healthcare, big data is being applied to medical research to accelerate the R&D process and to minimize hospital re-admittance rates. In utilities, use of big data in the form of video and images is enabling a new generation of vegetation management applications. It is difficult to precisely rank the top use cases because, as the hype surrounding big data has subsided, the breadth of applications has mushroomed to include a full range of processes across industries, company sizes, and geographic regions IDC 4

5 ABOUT THIS ANALYST Dan Vesset is program vice president of IDC's Business Analytics research. Mr. Vesset's research and consulting is currently focused on the business analytics, business intelligence, and data warehousing software markets. Mr. Vesset is also the co-lead of IDC's Big Data research. He has authored numerous research publications, is a frequent speaker at business analytics conferences and seminars worldwide, contributes to IDC's Business Analytics and Big Data blog, and tweets ABOUT THIS PUBLICATION This publication was produced by IDC Custom Solutions. The opinion, analysis, and research results presented herein are drawn from more detailed research and analysis independently conducted and published by IDC, unless specific vendor sponsorship is noted. IDC Custom Solutions makes IDC content available in a wide range of formats for distribution by various companies. A license to distribute IDC content does not imply endorsement of or opinion about the licensee. COPYRIGHT AND RESTRICTIONS Any IDC information or reference to IDC that is to be used in advertising, press releases, or promotional materials requires prior written approval from IDC. For permission requests, contact the IDC Custom Solutions information line at or gms@idc.com. Translation and/or localization of this document require an additional license from IDC. For more information on IDC, visit For more information on IDC Custom Solutions, visit Global Headquarters: 5 Speen Street Framingham, MA USA P F IDC 5