I D C R E T A I L I N S I G H T S O P I N I O N

Transcription

1 Global Headquarters: 5 Speen Street Framingham, MA USA P F Big Data and Analytics in Retail: U n l o cking Hidden Opportunities W H I T E P A P E R Sponsored by: HP Greg Girard September 2012 I D C R E T A I L I N S I G H T S O P I N I O N The new information economy increasingly digital and intelligent defines the terms of reference for retail today, and within this milieu, mastering Big Data has become a cornerstone for success. Mastering the four dimensions of Big Data social, channel/customer, market, and supplier/product for actionable and monetized insight requires newfound technical, organizational, process, and decision management frameworks to handle three challenging Vs of Big Data and analytics volume, velocity, and variety in order to deliver the other V value. Value from Big Data and analytics can come from three sources gaining insight to improve processes and resource allocation, personalizing and localizing offers, and creating community as the nexus for branding and customer engagement. Extracting value economically from Big Data requires a new generation of technologies and architectures shared information ecosystems designed to enable the high-velocity capture, discovery, analysis, and application of insights. Several analytical underpinnings of value creation are clearly evident, including context-aware and pattern-based analytics, quantitative natural language processing and social network analysis, and the extension of retail data models beyond conventional enterprise entities and ontologies. Applying these insights to control and optimize operations, tactics, and strategies may require new business processes, decision management frameworks, and roles and organizational constructs. Opportunities to apply Big Data analytical insight reach deep into digital or social marketing or customer insight the organizations where these efforts often originate but just as important and likely more important, these opportunities extend into brand management and marketing, merchandising, product management, localization, pricing, fulfillment, and all channels of commerce. What do the dimensions of "big," "fast," and "varied" mean with respect to the opportunities to create value from Big Data and analytics in your business? Accurately evaluating competitive and market scenarios that impact your Big Data and analytics business case calculus requires the strategic and tactical points of view of line-of-business leadership. Don't September 2012, IDC Retail Insights #GRI237012

2 let Big Data and analytics become the bright shiny new object of your attention. Forgetting that value from Big Data and analytics doesn't materialize just because you have invested in these capabilities will lead to too many "dry wells" filled with misspent time, talent, and treasure. S I T U A T I O N O V E R V I E W T h e N e w I n f o r m a t i o n E n v i r o n m e n t o f R e t a i l As the digital world continues to emerge, information has become the singular environment within which retailers earn customer loyalty, bring successful new products to market, collaborate through supply chains with business partners, enable associates, reduce risk, ensure compliance, and above all, burnish their brand. At least that's the potential if only retailers master Big Data. As more Big Data and potential insight amass within and beyond the enterprise, traditional approaches to acquiring and analyzing relevant data and applying insight gained from interrogating the data no longer suffice. Barriers to capitalizing on this potential include the technical capabilities of conventional information technologies as well as the processes, decision management frameworks, and organizational constructs by which organizations govern data, develop information from the data, and apply insight to operational, tactical, and strategic decisions. By some estimates, up to 90% of the world's information lies hidden in forms and formats beyond the competencies of the information management technologies commonly used in retail. Retailers own some of this hidden information within their four walls or a cloud some of it unstructured (e.g., in product descriptions, customer s, contact center notes, promotional agreements) and some of it structured (Excel spreadsheets being the nearly ubiquitous example). More and more information hidden in Big Data, however, lies beyond a retailer's information management assets residing in social data, customer data, market data, and supplier data. Big Data in the Social Context of Retail The impact of Big Data its promise amid peril is easiest to see in how it plays out in the inherently social context of retail. A growing and diverse number of consumers now interconnect and influence one another with their use of social networking and media YouTube, Facebook, Twitter, LinkedIn, Google+, blogs, wikis, and ratings and reviews. The speed, scale, and viral aspects of social media and networking influences, whether they are facts, opinions, or misinformation, dwarf old modalities of "word of mouth" and "power of the pen" pressures on shaping consumers' attitudes and opinions about brands as well as consumers' buying patterns and propensities. Page 2 #GRI IDC Retail Insights

3 Conventional ways by which retailers seek to understand consumer sentiments and the sources of influence on those sentiments fall short of the mark when their customers engage one another through social media and networks. Traditional information management approaches will no longer suffice to address these emerging demands and the resulting complexity. The disparity between the speed and strength of consumer sentiments spreading across social networks and the limits of what retailers have to track these sentiments puts retailers at risk of losing control of their brand, the loyalty of their best customers, and market share, revenue, and margins. The imperative retailers face is deciding how they should ready themselves to understand, react, and ideally engage their consumers in this brave new world. The key lies in mastering new competencies in Big Data and analytics to successfully and automatically integrate their insight into the various retail business processes. L a t e n t V a l u e : Th e F i r s t V i n B i g D a t a Social data is one of four dimensions of Big Data in retail. Customer data sources, market data sources, and supply data sources round out the IDC Retail Insights typology of Big Data in retail (see Figure 1). Insights hidden in the Big Data of each dimension can improve decisions inherent in that dimension; for example, insight from loyalty and shopper profile data can improve customer segmentation and promotional targeting, and in another example, real-time stock (distribution center [DC], in-transit, store) data can improve next-day or same-day replenishment. Increasingly, however, new insight can be drawn by mashing up data across domains. For example, contextual analysis of contact center call logs, tweets, ratings and reviews, lot tracking of shipment records, product design information, and loyalty data taken together could spot product defects from customer feedback, identify root causes from product design information, identify stores holding any defective merchandise, and find loyal customers who purchased the item. In more general terms, Big Data and analytics hold the potential to dramatically increase three imperatives of omnichannel retail: Insight applied in commerce (online and in-store); marketing (brand management, advertising, and promotions); fulfillment (from private label management to customer orders); and merchandising (from curating core assortments to localizing prices and promotions) Personalization applied in private promotions and offers, dynamic configuration of online assortments, and digital content presentation based on knowing the customer at any touch point and anticipating his or her needs 2012 IDC Retail Insights #GRI Page 3

4 Supply Data Sources Customer Data Sources Community applied in physical or digital social venues to harness the influence of social networks and media to create personal engagement with the brand F I G U R E 1 R e t a i l B i g D a t a T y p o l o g y Purchase orders Shipments Returns RFID and sensors DC stock Store stock Receipts DC issues Trade promotions Product information Design specifications Market intelligence Compliance Twitter Facebook Games Shutterfly Pinterest Google+ YouTube LinkedIn Blogs Wikis Foursquare Shopkick Yelp Citysearch Social Data Sources Enterprise Systems of Record Billions of Interactions Millions of Transactions Market Data Sources Trade Syndicated Demographic Geospatial Events Venues Business Economic Weather Market baskets Shopping carts Loyalty and profiles Offer/response Shopping list Click stream SMS Associates' black books Search terms and patterns Ratings and reviews Downloads Geospatial QR scans App usage Web chat Contact center Sensor Survey and focus group Source: IDC Retail Insights, 2012 V o l u m e, V e l o c i t y, V a r i e t y : T he O t h e r V s o f B i g D a t a O p p o r t u n i t y Realizing the opportunities from analyzing Big Data in individual domains and mashing up Big Data across domains, in other words, harvesting its latent value, is a function of an organization's ability to Page 4 #GRI IDC Retail Insights

5 master the inherent volume, variety, and velocity of the types of Big Data shown in Figure 1: Data volumes growing from terabytes to petabytes, exabytes, or more. IDC Retail Insights estimates a base of 9.6PB held in social media today with 10.2PB being added in the year ahead. We also estimate conservatively that 19.8PB of created content drives 14,000PB to 18,000PB of content consumed. Data varieties extending in formats from structured to semistructured and unstructured and in sources from core transactional enterprise systems one's own or those of trading partners through which retailers operate and control their businesses; to collaborative and communications systems (e.g., , enterprise wikis, and contact center logs); and to external informational systems some structured (e.g., demographic data) but the vast volume unstructured (e.g., tweets or posts to a company's Facebook page). Data velocities and their business process and decision management frameworks changing from those amenably serviced by batch processes to those stubbornly dependent on the streaming of data acquisition, analysis, and application of insight. F U T U R E O U T L O O K Market and financial imperatives present the strategic context of an urgent and critical need to master the "supply" and "apply" sides of Big Data and analytics in retail. Merchandising and marketing present an excellent example of this need in the context of the ongoing shift of power to the consumer. The forefront of insights in these two domains has long since marched past the private knowledge domains of the merchant prince who controlled product risk and is now marching past the enterprise knowledge domain of spreadsheet merchants as they control market risk to the Big Data knowledge domain of the social merchant controlling customer risk. Today's retailer is dependent on mastering all three knowledge domains. S u p p l y S i d e a n d A p p l y S i d e o f B i g D a t a and A n a l y t i c s Mastery of Big Data's four Vs is a function of two capabilities supply and apply. Both dimensions require agility in the solutions that supply Big Data and the processes that apply its insights. Full realization of the value hidden in Big Data in the strategic context of real-time omnichannel retail wherein the consumer holds the balance of power requires new consumption models from real-time contextual decision management frameworks to mobile presentation of visually rendered analytics in an enterprise context and mobile personalized content in the context of instrumented, informed, and interconnected customers IDC Retail Insights #GRI Page 5

6 Supply Side of Extracting Value from the Volume, Variety, and Velocity of Big Data Extracting value economically from Big Data requires a new generation of technologies and architectures designed to enable the high-velocity capture, discovery, analysis, and application of insights. These IT assets encompass the hardware and the software that integrate, organize, manage, analyze, and present data that is characterized by the four Vs of Big Data opportunity. This set of capabilities is the "supply side" of Big Data competence. Apply Side of Extracting Value from the Volume, Variety, and Velocity of Big Data A second set of capabilities of equal importance addresses the "apply side" of Big Data competence the business processes, decision management frameworks, and roles and organizational constructs by which an organization consumes Big Data insight to control and optimize its operations, tactics, and strategies to achieve its business objectives. Shared Information Ecosystems The value of Big Data and analytics cannot be fully realized unless its insights are applied in the retail enterprise ecosystem context of other business intelligence systems, in particular, in their service of marketing and positioning, merchandising and product development, supply and fulfillment, and omnichannel customer engagement through store and ecommerce channels. The contextualization of Big Data insight in the ecosystem of enterprise systems of processes and systems of record includes: Aligning analytics with business concerns (e.g., product quality, brand position, customer relationships, product development, marketing campaigns) Managing the life cycles of Big Data defining, harvesting, organizing, analyzing, storing, reporting, and disposing of the data at the clock speed of the business processes in which it is consumed Reporting and applying insights in process decision management frameworks (e.g., roles; decision rights; operational, tactical, and strategic time frames and spans of control; and plan, do, analyze cycle times) Page 6 #GRI IDC Retail Insights

7 R e t a i l B i g D a t a and A n a l y t i c s M a t u r i t y Figure 2 presents the five dimensions of Big Data and analytics in the context of retail: Intent context of strategic, tactical, and operational decision management Process relevancy of Big Data and analytics to roles, responsibilities, and outcomes People alignment of organization and cultural norms with the use of Big Data and analytics Data governance and life-cycle management of voluminous, varied, and high-velocity data Technology priority focus on hardware, software, and services for economical supply of relevant Big Data and analytics F I G U R E 2 R e t a i l B i g D a t a a n d A n a l y t i c s M a t u r i t y M o d e l Clear view of business and market trends relating to the power of Big Data analytics for insight, personalization, and community formation; the competitive risks and rewards related to the use of Big Data analytics; and commitment to infusing the omnichannel enterprise with the capabilities of Big Data analytics Intent Organizational alignment to collaborate with Big Data analytics, define roles and training for skills to apply the insights of Big Data analytics, and align compensation to create a Big Data analytics retail culture People Big Data Maturity Process Big Data insight made relevant and applied in processes that span departments and functions with clarity of roles, responsibilities, decision management frameworks, and dependencies for continuous plan, do, analyze cycles Data Technology Governance and MDM of structured and unstructured Big Data for consistent enterprise and external data frameworks for analytical support of business processes IT priority focus on technologies and architectures designed to extract value economically from very large volumes of a wide variety of data by enabling high-velocity capture, discovery, analysis, and application of insights to decision making Source: IDC Retail Insights, IDC Retail Insights #GRI Page 7

8 V a l u e C r e a t i o n O p p o r t u n i t i e s f r o m B i g D a t a and A n a l y t i c s i n R e t a i l Ultimately, in the business context of retail, value creation from Big Data and analytics should result in the monetization of hidden insight revealed and applied. Commercial paths to monetization can lead to product and service innovation and with it differentiation, premium pricing, and customer loyalty; process control compliance, product quality, security, and fraud prevention; and optimization of processes and decisions cost control, margin growth, and efficiencies. Analytical Underpinnings of Value Creation The range of new analytical underpinnings for monetizing Big Data insight includes the following: Context-aware analytics: Integrating Big Data insight with enterprise transactional data and business intelligence (e.g., predictive analytics, forecasting, and root cause analysis) such that the situational awareness of Big Data analytics complements and supports human judgment Broad-scope pattern-based analytics: Proactively seeking episodic, recurrent, and systemic patterns or systems of factors within process and organizational domains ("silos") and supersystems of factors spanning silos from conventional and unconventional sources that can positively or negatively impact outcomes Quantitative natural language processing analytics: Discovery and disambiguation of meaning primarily in digital communications (e.g., s, tweets, blogs, and Facebook posts) and content repositories (e.g., product descriptions in product data management system or customer ratings and reviews) Extension of retail data models beyond conventional enterprise entities and ontologies (e.g., foundational notions of products, customers, regions, stores, promotions, channels) to include friends, followers, sentiments, and interests as extensions of customer attributes; "likes," "wants," and reviews as extensions of product attributes; and forums and topics at the nexus of people, places, and products Opportunities for Applying Big Data and Analytics in the Line of Business Opportunities to monetize Big Data and analytics span core retail processes including brand management and marketing; merchandising; fulfillment; and commerce. Examples and use cases of Big Data and analytics for each of these processes are shown in Table 1. Page 8 #GRI IDC Retail Insights

9 T A B L E 1 O p p o r t u n i t i e s t o M o n e t i z e B i g D a t a a n d A n a l y t i c s i n R e t a i l Process Process Responsibilities Sample Applications and Use Cases Brand management and marketing Merchandising Fulfillment Commerce Brand differentiation Support and execution of brand management differentiation strategy via development and execution of media campaigns Customer analytics, insights, and segmentation Creation of promotional campaigns to shape customer demand in collaboration with marketing Creation of promotional campaigns to shape customer demand in collaboration with merchandising Selection, curation, localization, allocation, distribution, and pricing of merchandise in all commerce channels Creation of promotional campaigns to shape customer demand in collaboration with marketing Sourcing, developing, designing, and delivering of merchandise into commerce channels in collaboration with merchandising Fulfilling customer orders from own network, suppliers, and marketplace Presenting and selling merchandise and services across all channels Operating all channels of trade, including stores, catalog, call centers, and digital mobile, ecommerce, social, and thirdparty channels Identification and characterization of threats to brand values (e.g., boycotts or campaigns reacting to unmet corporate social responsibility commitments) Intelligence about strengths and weaknesses of competitors by customer segments, merchandise categories, product quality, pricing, locations, and other categories of strategic importance Intelligence about customer sentiment trends by customer segments, merchandise categories, product quality, pricing, locations, and other categories of strategic importance Identification of networks, nodes, and personalities of viral influence Assessment of the impact of marketing campaigns and events on consumer awareness, sentiments, behaviors, and intentions Intelligence about new product development, design, and introduction aspects that delight customers, meet their expectations, or are of low value Intelligence about local demand for products and services not carried in local assortment Early warning of product defects and shortfalls of performance and attributes against customer expectations Intelligence about new product development, design, and introduction aspects that delight customers, meet their expectations, or are of low value Store- and market-specific customer concerns and delights regarding store operations, associates' engagement practices and product knowledge, customer service, crowds, wait times, and so forth Real-time in-store, near-store customer tweets for customer service, product information, and product location Customer concerns and delights about performance and characteristics of ecommerce, mobile, social, and other digital channels Source: IDC Retail Insights, IDC Retail Insights #GRI Page 9

10 C H A L L E N G E S What do the dimensions of "big," "fast," and "varied" mean with respect to the opportunities to create value from Big Data and analytics in your business? The answers will define the terms of reference for your Big Data and analytics value creation journey. Gaining clarity and consensus on these definitions will require you to address technical, cultural, and organizational challenges. Left unattended, these challenges will delay and distort identification and definition of gaps in the five dimensions of your Big Data and analytics maturity profile intent, process, people, data, and technology. Ignoring the dictum that "better can be the enemy of good enough" can derail your Big Data business case calculus. Accurately evaluating competitive and market scenarios that impact your Big Data and analytics business case calculus requires the strategic and tactical points of view of line-of-business leadership. Against that input, the ability to define investment tripwires investment priorities, dependencies, and time lines for scenarios most likely to come sooner and create more risk and using them to sequence and schedule investments under likely scenarios requires the strategic and tactical points of view of information technology leadership. Becoming mesmerized by the allure of Big Data and analytics as a bright shiny new object of your attention presents another challenge to be managed. Opportunities to create value from Big Data and analytics projects don't materialize just because you have invested in these projects. Forgetting that will lead to too many "dry wells" filled with lost time, talent, and treasure. O V E R V I E W O F H P B I G D A T A A N D A N A L Y T I C S O F F E R I N G S HP offers a wide range of information management and analytics (IM&A) capabilities drawn from its enterprise services, software products, and cloud and security platforms to address Big Data challenges and opportunities in retail. HP IM&A services include information strategy and organization, information management and architecture, business analytics and information delivery, and social intelligence. HP provides these consulting services around its own Big Data software assets, Autonomy and Vertica, and third-party software assets, in particular SAP HANA and Microsoft SharePoint/BI platform. HP supports this with a variety of software and solution delivery models including onpremise installation, hosted, software as a service (SaaS), cloud computing, and multitenant SaaS (cloud deployment). Page 10 #GRI IDC Retail Insights

11 HP focuses its capabilities to enable retailers to proactively manage information-related business risk, enhance customer experiences, and optimize business performance to create competitive advantage. HP's Autonomy software asset helps retailers develop connected intelligence from structured and unstructured data for actionable decisions that improve business performance. HP's Vertica analytics database delivers scalable performance on Big Data queries enabling real-time decision making to be embedded in retailer processes in order to optimize business performance. Vertica and Autonomy deliver a powerful combination for real-time analytics and decision making using structured and unstructured data across the enterprise. S t r e n g t h s a n d C h a l l e n g e s HP offers a product and services portfolio that is consistent with what IDC Retail Insights expects from a market-leading technology provider to the retail industry. In addition to the software and services capabilities noted previously, HP provides the following infrastructure component to enable Big Data application deployments: Cloud computing. HP can offer clients a variety of enterprisegrade cloud computing solutions, including public, hybrid, and private. The ability to offer private cloud to retailers is considered a strength for a number of reasons, not the least of which is a tighter security model. Security. HP offers clients a security strategy through the HP Security Framework, designed to offer end-to-end information security plans and execution road maps. Because of the sensitive nature of customer data and the requirements and penalties imposed by the regulators, security is top of mind for industry IT executives. Mobility. HP's approach to enabling enterprise mobility is suited for organizations that wish to reach their constituents across multiple networks and devices by delivering applications, content, and services in a scalable, secure, and reliable way. This approach leverages HP's global applications services capabilities to provide the architecture, systems engineering, development, and support services. Combined, they help an organization simplify its applications and extend them where necessary as well as build mobile business-to-business, business-to-consumer, and businessto-employee applications. This approach also leverages HP's service-oriented architecture based integration architecture and is enabled by development and security frameworks that help create componentized building blocks from monolithic legacy applications to develop and deploy mobile applications IDC Retail Insights #GRI Page 11

12 HP faces several unique market challenges as well as many of the same market challenges as other enterprise vendors servicing the Big Data marketplace: Demonstrating leadership in the transition from information management to Big Data. HP's legacy information management solutions, including TRIM, Data Protect, and others, are not broadly implemented across the retail marketplace. HP must more effectively demonstrate leading capabilities at the component level with high-profile Autonomy and Vertica wins as well as highlight its ability to deliver end-to-end Big Data software, services, and hardware with lighthouse and marquee clients. Offering competitive, differentiated solutions portfolios. The retail market, as a traditional early adopter of emerging technologies, senses the value of the Big Data trend and is budgeting accordingly. HP's competitors are also focused on expanding their solution portfolios on the Big Data trend in terms of breadth and depth of product capabilities, IT infrastructure, and professional services. HP must leverage its industry position and demonstrate the value of a Big Data relationship with HP to gain Big Data market share. Channel network management. HP's channel strength is also a weakness. Successful execution of HP's Big Data strategy to broaden its portfolio will require that HP reinforce its position in the enterprise space without alienating the channel that has been so beneficial to the organization. E S S E N T I A L G U I D A N C E Adopt an enterprisewide approach to Big Data analytics. The design, development, and deployment of Big Data analytical capabilities should be seen from the outset as an enterprisewide undertaking even as nascent initiatives of the sort incubate within marketing, in particular digital marketing, where Big Data analytics are applied to social Big Data, or in ecommerce and personalization where Big Data analytics are applied to customer Big Data. The design objectives of an enterprisewide approach to Big Data analytics should include: Program governance led by a leadership team drawn from across four core omnichannel processes: marketing, commerce, merchandising, and fulfillment Alignment of insights from the four quadrants of Big Data sources social, customer, market, and supply with the four dimensions of retail intelligence customer intelligence, offer intelligence, channel intelligence, and merchandise intelligence Page 12 #GRI IDC Retail Insights

13 Delivering insights within the decision management context of the roles they inform that is, task aligned and just in time and the style in which and the speed at which the decisions are made Look for monetization opportunities across the board. Big Data analytics can inform decisions beyond marketing in merchandising, commerce, and fulfillment. Actionable insights include marketing campaigns; product design and quality; new product development and introduction; assortment localization; pricing, promotions, and personalization; crisis management, selling techniques, and store associate management; store operations and design; and competitive intelligence and differentiation. Maintain a readiness for rapid evolution of Big Data analytics. Big Data analytics is a new area; new capabilities for and opportunities to apply Big Data analytics are emerging quickly. Managing rapid change on the supply and apply sides of Big Data analytics requires agility in a host of technical competencies (e.g., data management and governance, commodity hardware configurations); organizational dimensions of skills, roles, and decision rights; consumption models and use cases; and more. Retailers will increasingly need to find sources of innovation and should look to Big Data information partners. Retailers should screen for partners that provide best-in-class solutions consisting of open source technologies/capabilities and proprietary technologies/capabilities, as needed, that are best suited for their unique requirements. C o p y r i g h t N o t i c e Copyright 2012 IDC Retail Insights. Reproduction without written permission is completely forbidden. External Publication of IDC Retail Insights Information and Data: Any IDC Retail Insights information that is to be used in advertising, press releases, or promotional materials requires prior written approval from the appropriate IDC Retail Insights Vice President. A draft of the proposed document should accompany any such request. IDC Retail Insights reserves the right to deny approval of external usage for any reason. This document was reprinted by HP with permission from IDC Retail Insights IDC Retail Insights #GRI Page 13