SOLUTION ENGINEERING. The way to drive business value from Big Data

Transcription

1 SOLUTION ENGINEERING The way to drive business value from Big Data There is no single technology solution to address the challenges of Big Data. Simply installing Hadoop or predictive analytics or an appliance is not enough. The ability to successfully drive business value from Big Data using advanced analytics also requires a new process: solution engineering. EMC PERSPECTIVE

2 IDEA IN BRIEF: SOLUTION ENGINEERING PROCESS 1. Understand how your organization makes money 2. Identify your organization s key business initiatives 3. Brainstorm Big Data business impact 4. Decompose the business initiative into use cases 5. Prove out the use case 6. Design and implement a Big Data solution SOLUTION ENGINEERING DEFINED Solution engineering is a process for identifying and decomposing an organization s key business initiatives into its business enabling capabilities and supporting technology components to support decision-making and data monetization efforts. This process is analogous to one you would use to build a complex design using LEGO bricks. The most successful LEGO projects are those that have an end in mind a thoroughly defined, well-scoped solution. Do you want to build a pirate ship, a castle, or a space ship? With LEGO bricks, you can build all three, plus many, many more. However, each solution requires a different set of bricks in different configurations and with a different set of instructions. Much like building with LEGO bricks, a successful Big Data business initiative requires that you identify up front what solution your organization is trying to build, and then assemble the right data and technology capabilities, in the right order, to deliver it successfully. THE SOLUTION ENGINEERING PROCESS This document describes a solution engineering process for architecting and developing data- and analytics-enabled business solutions. This process requires an up-front effort to grasp how your organization makes money, understand the organization s strategic nouns and the role they play in powering your organization s value creation process, and to comprehend the details (e.g., business objectives, key performance indicators, key line of business stakeholders) of your organization s key business initiatives on which to focus your solution engineering efforts. STEP 1: UNDERSTAND HOW THE ORGANIZATION MAKES MONEY Imagine that you are the general manager: how can your organization make more money? What can you do to increase revenues, decrease costs, reduce risks, or increase compliance? Organizations can tune many dials in order to make more money. Increasing revenue, for example, can include initiatives such as increasing the number of premium or gold card customers, increasing store or site traffic, reducing customer churn, increasing revenue per shopping occurrence, increasing private label sales as percentage of market basket, increasing cross-sell/cross-sell effectiveness, and optimizing promotional effectiveness (see chart below). Figure 1. Potential areas for making more money

3 Next, identify and understand your organization s strategic nouns and the role they play in driving the money-making capabilities of the organization. For example, if you re in the airline industry, hubs are a very important noun of your business, and any way that you can increase the number of flights per hub for example, by decreasing airplane turn-times or improving terminal/ramp efficiencies means more flights per day, which equals more money. Also, invest some time actually using your organization s products or services as well as and observing or using competitive products so you can directly experience each company s value propositions to their customers and partners and see how their money-making efforts work. For example, if your organization is in the Business-to-Consumer (B2C) market, you can leverage customer engagement data to uncover insights that can help to optimize the customer engagement process in order to create more profitable customers. The challenge is understanding how to leverage these Big Data analytics to move customers up the profitability curve while servicing unprofitable customers in a more cost-effective manner. STEP 2: IDENTIFY YOUR ORGANIZATION S KEY BUSINESS INITIATIVES The next step is to do some primary research to understand your organization s key business initiatives. This includes reading the annual report, listening to quarterly financial analyst calls, and searching for recent executive management speeches and presentations. If possible, interview senior business management to understand their top business opportunities as well as their perceptions of the key challenges that might prevent the organization from successfully executing against these opportunities. For each identified business initiative, capture key information such as roles and responsibilities of business stakeholders, key performance indicators, and the metrics by which success of the business initiative will be measured, timeframe or roadmap for delivery, critical success factors, desired outcomes, and key tasks. STEP 3: BRAINSTORM BIG DATA BUSINESS IMPACT The next step in the solution engineering process is to brainstorm how Big Data and advanced analytics can impact the targeted business initiative. Big Data and advanced analytics can power an organization s key business initiatives in five ways: 1. Provide access to more detailed structured data at the lowest level of transaction granularity, which enables higher-fidelity, more detailed decisions. For example, detailed structured data like customer loyalty transactions facilitates decision-making and data monetization opportunities at the individual customer, seasonal/holiday, and local levels. 2. Provide access to new unstructured data sources both internal sources like web logs, consumer comments, and s, as well as external sources like social media and mobile that enable more robust, more complete decisions. These new, diverse data sources provide new variables, metrics, and dimensions that can be integrated into analytic models to yield actionable, material business insights and recommendations. 3. Provide access to machine- or sensor-generated data (smart grids, connected cars, smart appliances), which enables more timely operational 3

4 decisions including the ability to support predictive maintenance. Additionally, smartphone-generated location data enables location-based analysis that can drive real-time consumer offers and engagement recommendations. 4. Provide high-velocity/low-latency data access (reduced time delay between the data event and the analysis of that data) to enable more-frequent, more-timely decisions and data monetization. This could include the creation of on-demand customer segments (based upon the results of some major event like the Super Bowl) as well as real-time location-based insights from real-time smartphone location feeds. 5. Finally, deploy predictive analytics to uncover causality buried in the data. Predictive analytics can enable a different mindset with your business stakeholders, encouraging them to use new verbs like optimize, predict, recommend, score, and forecast as they explore new decision-making and data monetization opportunities. Specific examples of probing business questions are provided later in this paper. STEP 4: DECOMPOSE THE BUSINESS INITIATIVE INTO USE CASES The next step is to conduct a series of interviews and ideation workshops in order to brainstorm, define, and prioritize the use cases that support the targeted business initiative. For each use case, capture the following information: Targeted personas and stakeholders, including their role, responsibilities, and expectations Business questions that the stakeholders are trying to answer or could be trying to answer if they had access to more detailed, more diverse data sources Business decisions that the stakeholders are trying to make, and the supporting decision processes including timing, decision flow/process, and downstream stakeholders Key performance indicators and key metrics against which business success will be measured Data requirements, including sources, availability, access methods, update frequency, granularity, dimensionality, and hierarchies User-experience requirements, coupled closely with the user s decision-making process Analytic algorithms and modeling requirements (predict, forecast, optimize, recommend) 4

5 STEP 5: PROVE OUT THE USE CASE Now is the time to bring data and technology together to prove out the solution. At this point, you should have a solid understanding of the desired data and the necessary technology capabilities to build out a Proof of Value where the desired solution is flushed out using the full depth of available data and full breadth of technology capabilities. This process should include: Gathering required data, both internal as well as external data sources, and exploring the use of third-party data to help broaden the quality and depth of the analytics Defining data transformation and data enrichment processes Defining analytic modeling requirements Defining user-experience requirements STEP 6: DESIGN AND IMPLEMENT THE BIG DATA SOLUTION. Based upon the results of the Proof of Value, define the detailed data and technology architecture and develop a roadmap for integrating the analytic models and insights into the operational and management systems. The implementation plan and roadmap will need to address the following: Internal and external data sources and data access requirements Data management capabilities, including master data management, data quality, and data governance Data modeling capabilities such as data schema and key-value pairs Business intelligence capabilities such as performance monitoring, reporting, alerts, dashboards, and KPIs Advanced analytic capabilities such as statistics, predictive modeling, and data mining, including real-time analysis capabilities User-experience requirements such as operational systems, management systems, and dashboards SOLUTION ENGINEERING TOMORROW S BUSINESS SOLUTIONS Solution engineering will become increasingly important as 1) the amount and variety of data continues to evolve, 2) technology data management and analytics capabilities continue to expand (fueled by both venture capitalists and the explosive growth of the open source movement) and 3) mobile devices and smaller form-factor mobile apps redefine the user experience. As the data and technology sands shift under our feet, it will become even more important to focus on delivering business solutions that have a high return on investment and a short payback period. So, how do you apply solution engineering to some of your highest potential business opportunities? Let s walk through some examples. 5

6 CUSTOMER BEHAVIORAL ANALYSIS The Big Data opportunity in Customer Behavioral Analysis is to combine your detailed customer transactions with social media and mobile data to uncover new customer and product usage insights that can optimize your customer engagement lifecycle processes. These insights can ultimately lead to personalized marketing, especially when coupled with the real-time insights that can be obtained from mobile apps. To gain new insights about a customer s behavior, you first need to integrate your detailed customer engagement transactions such as sales history, returns, payment history, call center notes, consumer comments, conversations, and web clicks. Then, you can utilize advanced analytics on these transactions to identify and score your most valuable customers and customer segments, create behavioral categories, and use these customer models and scores to refine your target customer profiles and customer segmentation strategies. For example, you can search, monitor, and capture relevant product and company blog information from product and company advocates on sites such as WordPress, Blogger, Tumblr, and blogspot. You can also capture and aggregate all social media data feeds from sites such as Facebook, Twitter, LinkedIn, and Pinterest. Through the use of text analytics and/or Hadoop/MapReduce, you can mine the social media and blog data to uncover new insights about your customer s interests, passions, affiliations, and associations that can be used to refine your target customer profiles and customer segment models. And by leveraging mobile app capabilities, you can uncover real-time insights about your customers location, purchase behaviors, and propensities to drive real-time, location-based promotions and offers. PREDICTIVE MAINTENANCE Perhaps the most significant business opportunity for business-to-business (B2B) companies in the Big Data space is to provide predictive maintenance services to their business (and possibly consumer) markets. Big Data analytics can leverage sensor-generated data from appliances, equipment, and machinery to analyze, score, and predict the maintenance requirements of machinery in real time. Any industry that operates machinery automobiles, airplanes, trains, farming, construction, appliances, turbines, servers, business equipment can benefit from predictive maintenance that is enabled by sensor-generated data coupled with realtime analytics. In order to gain new predictive maintenance insights, you must capture raw, unstructured appliance, equipment, and machinery sensor-generated logs in real time, as is (no data pre-processing required, no pre-defined data schemas), using Hadoop and HDFS. By applying advanced analytics against your historical performance data, you can build predictive models of what constitutes normal appliance, equipment, and machinery performance at the individual unit and component levels. You can then leverage advanced data enrichment techniques such as creating frequency, recency, and sequencing to identify combinations of events or event thresholds that may be indicative of maintenance needs. Furthermore, by integrating external, dynamic data sources (e.g., weather, traffic, = economic), you can identify new variables that can enhance the predictive models for example, quantify the impact humidity might have on the performance of your wind turbines or rail cars. 6

7 Leveraging a real-time analytics environment allows you to compare streaming sensor data to your performance models and control charts in real time, and flag and score any potential performance problems. For example, whenever a problem situation is uncovered, you can send out automated alerts to concerned parties (technicians, consumers) including recommended maintenance information such as location, estimated replacement parts. Capturing wear data from the replaced parts enables you to continuously refine the predictive maintenance models, which allows you to aggregate and analyze the wear data in order to create, package, and sell performance insights back to appliance, machinery, product, and component manufacturers. MARKETING EFFECTIVENESS Every company spends money on marketing, and increasing portions of that spend is going to highly measurable digital media channels. Quantifying the effectiveness of marketing spend across both online and offline channels (such as TV, print, and radio) is a difficult challenge. Organizations that can more accurately quantify and attribute credit to the marketing channels and marketing treatments that are driving business and sales performance are better positioned to optimize their marketing spend. In order to better measure marketing effectiveness, you need to: Aggregate all marketing spend, at the lowest level of detail, across all online and offline marketing channels including digital, mobile, TV, print, and radio. Aggregate all offline sales activities (calls, bids, sales) with online conversion events, and associate these activities back to the different marketing activities and spend. For digital and mobile data feeds, capture the market basket of log and web click transactions and conversions for each user (cookie-level detail). Create advanced composite metrics associated with marketing treatment frequency, recency, and sequencing in order to quantify the effectiveness of the digital marketing treatments (attribution analysis). Augment campaign data with external data such as weather, economic, and local events to improve campaign modeling and predictive effectiveness. Benchmark current campaign performance against previous and similar ( like ) campaigns to identify and quantify previous campaign business drivers. Leverage prospect data captured via third-party direct marketing campaigns to build out a prospect database against which you will run your direct marketing customer acquisition campaigns. Develop a testing or experimentation strategy in order to continuously test the effectiveness of different marketing treatments, messaging, and channels. Create a social media strategy that facilitates the capture and analysis of social media data about customers interests, passions, affiliations, and associations that can improve customer profiling, segmentation, and targeting effectiveness. Capture ongoing, real-time social media feeds to analyze and monitor customer and campaign sentiment. 7

8 The insights from the marketing performance analytics enable you to create actionable and insightful digital marketing dashboards that drive both pre-campaign media mix allocation recommendations as well as in-flight campaign performance recommendations. FRAUD REDUCTION Big Data provides new and innovative technologies to identify potentially fraudulent activities in real time. New data sources such as social media and detailed web and mobile activities, and new Big Data innovations like real-time analytics, are enabling organizations to move beyond the traditional static fraud models to create dynamic, self-learning fraud models to combat ever-evolving fraud activities. For example, deploying a real-time data platform can enable you to capture and manage a high volume of real-time data feeds (purchases, authorizations, etc.) from multiple internal and external data sources. You can also use in-database analytics to accelerate the development and refinement of fraud prediction models leveraging historical transactions integrated with real-time activities and transactions. By utilizing MPP-based analytics, you have the ability to analyze real-time transactions and flag unusual transactions, behaviors, and tendencies across thousands of dimensions and dimensional combinations, comparing those scores to historical norms and models to flag potential fraud situations. Additionally, integrating social media data allows you to identify potential networks or associations of fraud perpetrators. You can also employ advanced data enrichment techniques such as frequency, recency, and sequencing of activities and transactions to create more advanced profiles of potentially fraudulent activities, behaviors, and propensities. Businesses that have a higher than normal propensity for potentially fraudulent behaviors can integrate mobile data with location-based analytics to dynamically identify and monitor activities and transactions at physical locations and web sites. Integrating real-time fraud detection models into operational systems can quickly challenge specific transactions and grouping of transactions, with the goal of identifying fraudulent transactions while the transaction is still in process. NETWORK OPTIMIZATION Whether you operate a network of devices (servers, ATMs, switching stations, wind turbines) or outlets (stores, sites, branches), there are invaluable sources of customer and product data that can be leveraged to ensure that you have the right nodes in the right locations at the right time to provide an exhilarating customer experience. Over- and under-capacity is always a key challenge to networks, and those capacity requirements and needs can change rapidly based upon customer and product behaviors and tendencies, and current market events (e.g., storms, news, local events). In order to optimize your network operations, aggregate your network node data at the lowest level of detail across all your different network components and elements. Integrating social and mobile consumer data enables you to identify and quantify changes in customer, network, and market preferences and behavioral tendencies that may impact network loads. 8

9 You could augment these data assets with external data sources such as weather, local events, holidays, and local economic data to provide new predictive metrics that can improve the predictive capabilities of capacity planning and resource scheduling models. Furthermore, by using advanced analytics to project network capacity requirements you can calculate key network support variables such as personnel, inventory, replacement parts, and maintenance scheduling. Lastly, by leveraging real-time analytics, you can shift and reallocate network capacity (resource scheduling, ramping up or ramping down cloud resources) to map to daily/hourly and location usage patterns. SUMMARY Being a solution engineer requires a strong understanding not only of the business problems that your organization is trying to address, but also of the capabilities of new Big Data and advanced analytics innovations. Applying the six-step solution engineering process described in this paper ensures that you deploy the right technology capabilities at the right time to solve the right business problem. CONTACT US To learn more about how EMC products, services, and solutions can help solve your business and IT challenges, contact your local representative or authorized reseller or visit us at EMC 2, EMC, and the EMC logo are registered trademarks or trademarks of EMC Corporation in the United States and other countries. All other trademarks used herein are the property of their respective owners. Copyright 2013 EMC Corporation. All rights reserved. Published in the USA. 4/13 EMC Perspective H EMC believes the information in this document is accurate as of its publication date. The information is subject to change without notice.