Predicting win/loss probabilities with pipeline data Model uses SAS, HPE Vertica, QlikView for sales opportunity predictive analytics

Transcription

1 Case Study Objective Provide a method to identify which quarterly sales opportunities are most likely to close successfully or not. Predicting win/loss probabilities with pipeline data Model uses SAS, HPE Vertica, QlikView for sales opportunity predictive analytics Approach Analyze historical deal data with known Win/Loss outcomes to create a model that indicates the Win/Loss probability of an opportunity. Then leverage the HPE Vertica Data Analytics engine, a key technology supporting HPE Haven, to create a proof of concept for the solution. IT Matters Solution uses SAS analytics software, a SAS-generated scoring algorithm, a high-performance HPE Vertica database, and QlikView on-demand business reports. Data is pulled from the HPE SalesForce. com CRM system; sampled 800,000 opportunities, from a cross section of accounts. Predictive analytics model uses a variety of techniques that learn from data to make predictions about the future. Business Matters Improved business decisions with data-driven analytics. For example, the analytics might suggest that altering the product portfolio mix or adding new upsell/cross-sell options could increase the Win rate. More effectively deployed sales representatives to targeted opportunities and customers. Improved executive alignment by planning and managing the sales pipeline more effectively. SIM IT and the Advanced Analytics teams found relationships between 1) opportunity age, 2) opportunity velocity (speed opportunity moves through sales stages), and 3) customer buying patterns (frequency, product, value). In 2013, Hewlett Packard Enterprise determined that it needed a new way to identify which quarterly sales opportunities were most likely to close successfully or not. The goal: Help sales managers as they allocate limited sales resources to accounts in the current sales pipeline. By June the following year, HPE s IT Software Information Management (SIM) team had successfully developed a model to determine probability for a sales opportunity. Imagine the advantages for your sales organization if you could know, in advance, what deals are likely to close before the end of your earnings period. You could also know what deals are not going to close at least not in the near term. Naturally, you d be able determine what priorities to assign accounts over the coming weeks, when to simply nurture a customer s growing interests without expending too many precious resources, vs. when to apply laser-focus attention to those accounts that will keep your earnings in the black. This capability may soon become a new business practice for companies seeking improved sales forecasting, thanks to HPE technologies supporting the growing field of predictive analytics. It s something HPE s own sales teams are exploring to become more effective, and we think it should capture the interest of almost any mid to large size business.

2 Page 2 1 Tender Also known as Request for Pricing, combined with request for information in one document, often used in public sector purchases. A single tender document is sent out to several suppliers, which are returned completed. Usually all opened on the same day. Background Here is a situation many companies find themselves in, and which Hewlett Packard Enterprises recently determined to improve. Across our company, we have a range of highly experienced sales managers that use their sales expertise, industry knowledge, customer awareness, and market intelligence to evaluate and predict the outcomes of key opportunities they are working on, says Dominic Townsend, an expert in Business Intelligence systems for HPE. These can be classified as soft skills, i.e., highly valuable skills that can, at best, be applied subjectively by each individual manager. While there is general agreement on the standards used to judge any given sales opportunity, the interpretation of these standards will naturally vary. And it is seldom that each sales manager has the bandwidth to critically evaluate every opportunity the sales team works on. With tens of thousands of sales team members throughout the world, HPE serves more than a billion customers in 170 countries; in 2013, the Company determined that it needed a new way to identify opportunities most likely to close successfully when additional sales resources were being considered. Recognizing an opportunity to apply Big Data analytics techniques to this fairly common business problem, HPE s IT Software Information Management (SIM) team was tasked in the fall of 2013 to develop a model that predicts the probability of an opportunity successfully closing (a win) or not closing (a loss). The following story describes how Big Data analytics can be used to solve such a problem. The business case for predictive analytics But before describing the details behind this application of Big Data techniques to the sales pipeline, let s consider the expected business benefits. Not that is possible to put an exact dollar value on the value of a better predictive method, or the cost of not having one, says Townsend. But higher win rates could translate into millions of dollars in increased sales. The question is: How can greater visibility into near-term opportunities help a sales manager improve win rates? Townsend offers a hypothetical case: An inside sales person for a Seller discovers an opportunity for Customer 1, and is eager to promote it. A field specialist is then engaged, along with additional field sales resources who provide supporting documentation, industry metrics, etc. The customer then moves to a tender situation, 1 where the Seller s additional company resources get involved in the writing and fulfilment of the tender response. Test equipment is then supplied with additional resources to install and maintain. The Seller s opportunity owner is now committed, past the point of no return. Even if the sales team have doubts, the opportunity from this point will be constantly forecast, with the Seller putting more resources into it. This will cost tens of thousands of dollars over the lifecycle of a large, long-term opportunity. Consider the untold story. Customer 1 has never purchased this type of equipment before. They typically use a reseller who is actively involved in promoting multi-vendor solutions. They also have a history of getting quotes and not purchasing. The last time they purchased any significant equipment from the Seller was over a year ago. The question arises: Is this opportunity worth all the resources that the Seller has put into it? A similar size opportunity comes up for Customer 2. Consider the differences in their story: Customer 2 has recently changed its reseller partner to one that actively promotes the Seller, and Customer 2 has also made an investment in other equipment from the same Seller. They rarely get quotes, but when they do, they have a strong history of buying. To top it off, they have a great relationship with the Seller s field sales manager as well. Obviously this is a much more promising and positive start to a sales cycle. However, as this opportunity is unfolding, the Seller s resources are being tied up with Customer 1; consequently, Customer 2 is not getting the service they require, and eventually the opportunity is lost. Customer 2 buys elsewhere. In the meantime,

3 Page 3 Customer 1 also buys elsewhere, as they have a history of doing. The Seller has alienated an existing customer, and failed to win any business, creating a lose-lose result with more harm than good done to both relationships. While Townsend admits that this is an extreme example, it can happen, on a different scale, in any company. What you need is a fast, intelligent way to look at hundreds of opportunities in a short space of time. That s what predictive analytics is all about. It lets you interrogate a wide range of data points to understand the probable outcome, and decide where to allocate resources. Instead of losing two opportunities, as in that example, not to mention a significant amount of time and money in presales activities, you can concentrate your efforts on winning one good opportunity and avoiding losses elsewhere. In a large, global company, the benefit of predictive analytics is simply a matter of scale. Single opportunities often run to millions of dollars, and small changes in the win rate can have a huge financial impact. On the scale of global business, there are millions of opportunities open at any given time, totaling billions of potential dollars, Townsend adds. If predictive analytics are applied to even a small portion of these opportunities, the improvement in accurate forecasting, resource allocation, customer retention and satisfaction will contribute significantly to win rate. Proof of concept: Learning from historical data HPE s SIM IT and Advanced Analytics teams worked together to develop the model as a proof of concept (PoC) using SAS, HPE Vertica, and real-time QlikView reports. The PoC focused on the Enterprise Group in the U.S., and was completed in June For this project, the HPE team analyzed historical deal data with known Win/Loss outcomes to create a model that indicates the Win/Loss probability of an opportunity. The resulting solution uses SAS analytics software, a SAS-generated scoring algorithm, a high-performance HPE Vertica database, and QlikView on-demand business reports. The PoC project began with the HPE Enterprise Group in the U.S., whose Enterprise Data Warehouse (EDW) hosts data on sales opportunities worth billions of dollars. This data is pulled from the HPE SalesForce.com customer relationship management (CRM) system. To address the business challenge posed in this PoC, SIM IT and the Advanced Analytics teams used predictive analytics to detect undiscovered relationships in the data that could help signal the probability of a win. For example, if a common pattern for sales opportunities with a good possibility of being won is known, sales teams could devote more time to those opportunities. Alternatively, if an opportunity has a lower possibility of being won, the sales representative might take proactive steps to significantly increase the possibility of winning it, or to reduce the number of assigned resources that could be better directed toward more likely wins. Basic methodology The SIM IT and the Advanced Analytics teams set out to create a predictive analytics model using a variety of techniques that learn from data to make predictions about the future. As with most predictive models, they developed a scoring system, with each score corresponding to the likelihood that a particular behavior or outcome will occur. The process requires data mining (i.e., analyzing data to find hidden patterns or relationships), sophisticated statistical analysis, and machine learning algorithms. The goal is to identify hard-to-see trends and relationships, which, in this case, could be used to predict a future win or loss in the sales pipeline. The team s first step was to generate a data set that contains actual historical data set of sales deals known to have been won or lost. This allowed the formulation of the prediction problem as a Supervised Classification problem. Under Supervised Classification, the model classifies a target as a category yes or no (or in this case, Won or Lost) and uses data with known target values to supervise the model s learning process.

4 Page 4 Figure 1. PoC solution data flow SAS Enterprise Miner (EM) and Enterprise Guide (EG), in conjunction with a high-speed Vertica database, were used to explore and analyze the data, choose the most accurate prediction model, create the scoring algorithm, and push the results to Vertica. Finally, the teams developed QlikView reports designed for sales groups that could be generated in real-time. The PoC took data from 800,000 opportunities, from a cross section of accounts. Because the sample size was so large, we were able to get some very accurate results, says Townsend. There were no specific deal sizes that stood out. The model thrives on a wide spectrum of data as it learns. If we had used a specific set of accounts or deal size, we would have essentially limited the learning aspect of the model, which would lead to less accurate results. Data flow and key visualizations One of the team s primary goals for the proof of concept was to understand the correlation of different predictor variables and how the opportunity could be better understood as a result. These variables are obtained from several data sources, as shown in Figure 1. The data flow of the PoC is shown in Figure 1. The data sources shown in Figure 1 include transactional data on leads, opportunities, etc. Master data is the reference data that can be correlated with the transactional data to enrich the information. For example, you can understand the customer attributes better by joining it to the customer dimension, explains Indranil Das, IT architect for HPE s sales and marketing portfolio team. We can understand the customer s RFM scores 2 better. Or we can better understand the impact of a given product on the opportunity when it is correlated with its performance and market response. This can help us understand whether the customer will buy the product at the given price or not. Or, we can see the potential to cross-sell Product A with Product B, to upsell and attach opportunities to maximize revenue. If we find a given customer is price sensitive and typically does not buy attached services, then we are able to make informed choices regarding this opportunity. Once the team determined the basic data flow, they used QlikView tools to create a real-time visual interface, which needed to present a range of information in the same visual space. It was important that our sales managers and operations teams were not baffled by too much information at one time, says Townsend. But it was equally vital that they be able to see the pipeline by specific products, departments, dates, opportunity stages, revenue, and performance to goal, by sales team or sales person. This enables them to do a wide range of analysis on one dashboard, combined with the forward looking analytics, which would be analyzed by the same parameters. The QlikView front-end was designed to not overwhelm, but rather inform users as they begin approaching their half yearly targets about the differences between the sales rep s gut feel forecast and the HPE Vertica analytics forecast, says Das. These two forecasts can be easily compared. We have a preliminary QlikView dashboard to provide some suggestive measures that Sales Reps and Managers can take to improve the chances of winning these opportunities.

5 Page 5 Figure 2. Sales Manager Dashboard: This screen shows how the pipeline is split by product and values in the different forecast categories, at far right center (Upside, Committed, Pipeline, etc.). 2 RFM is a method for determining customer value based on recency, frequency, and monetary value. The QlikView configuration provides a tabbed interface, as shown in the following examples. As shown in Figure 2, values are translated into monthly figures, by the same pipeline categories. This is shown in the center chart. At the bottom of the screen we see win loss ratios, along with pipeline shape and values. We think sales managers in any company would be overjoyed to have all this information in one screen, but it is only a small example of what we have achieved here, says Townsend. In Figure 2, note that the analytics from the PoC are accessed via the Analytics tab, the fifth tab at the top of the screen. By clicking on that tab, the user can drill down into the analytics as shown in the following figures. Figure 3 shows the top five opportunities predicted to win, and Figure 4 shows the top five predicted not to win. The graphs compare monthly sales goals, sales representative forecasts, and the forecasts of the SAS-Vertica PoC solution. Pop-up windows provide more details. From the top part of the screen shown in Figure 3, the user can see where these opportunities sit in the CRM according to the forecast category, whether they are simply part of the sales pipeline, or the rep has committed them (meaning they are sure to be won), or upside, meaning there is a strong likelihood the opportunity will be won, but based more on the sales rep s opinion than any specific commitment from the customer. In the bottom section of the screen the entire pipeline (for that Sales Manager) is displayed. The black horizontal line represents the target, or goal. The purple line shows what the sales rep is committing (is certain will happen) to the pipeline, while the highlighted orange line is the Vertica forecast. This is where the predictive analytics come in. This Vertica forecast is based on the analysis of the trends and relationships described earlier and is set against a bar chart backdrop of all opportunities, broken down by the Forecast Categories. With the type of information shown in Figure 3 and 4, sales managers can better focus their resources and efforts on the opportunities identified as the most likely wins. For example, the sales manager could choose to focus resources on opportunities with a 60-70% win probability (see Figure 3, top part of screen, marked Forecast accuracy ) Lower-scoring opportunities are less likely to benefit from the additional resources, and higher-scoring opportunities are likely to close successfully without intervention. Results & preliminary findings The PoC was successfully completed in June Townsend describes what success means: The criteria for success was clearly marked by the accuracy of the predictions. Taking opportunity data and scoring it was not enough. We had to follow the outcome of the opportunities, and compare this with the results. Fortunately HPE s Vertica allowed us

6 Page 6 Figure 3. Top five opportunities predicted to win. A sales manager can choose to look at any amount of opportunities (in this case the top five). to use historical trending data to make this analysis. By using a snapshot of data from a previous month, scoring those opportunities, and comparing them to the live result, we were able to have that Eureka moment. Across the PoC, the SIM IT and the Advanced Analytics teams found relationships between 1) opportunity age, 2) opportunity velocity (how quickly the opportunity moves through the sales stages), and 3 customer buying patterns (frequency, product, value). Here are a few findings: The longer an opportunity remains open, the less likely it is to close as a win. However, the outcome depends on product mix; opportunities involving complex products or multiples of products take longer to close, so the relationship between opportunity age and product has to be understood. As the opportunity moves through the pipeline, it moves through several sales stages. These are: 01 Understand Customer, 02 Validate Opportunity, 03 Qualify Opportunity, 04A Develop Solution, 04B Propose Solution 05 Negotiate and Close, and 06 Won and Deploy. The opportunity may be lost or cancelled at any sales stage. Sometimes an opportunity may skip a sales stage; there is no hard and fast rule. Age/ velocity for a given sales stage gives an indication of how fast the opportunity is moving through the pipeline. Opportunities that linger in any one sales stage (say, Stage 03-Qualify the opportunity) have a greater tendency to close lost, although this has to be correlated against the sales team and product mix. Customer buying patterns are key indicators. The team found that individual accounts have unique buying patterns and frequencies. When these indicators are combined with product and age information, they make very reliable cues as to if and when an opportunity will be won. Business benefits of a Solution I was very pleased with the SAS/Vertica/ QlikView proof of concept. It is an effective solution in its own right, and it lays the foundation for the follow-on project that leverages HPE Vertica capabilities. This valuable work will let us move forward and leverage that capability into our BI portfolio, not only providing great business intelligence, but also showing how we can use our data to improve HPE s value to its customers, partners, investors, and staff, said Townsend. With a successful completion of the proof-ofconcept in June 2014, there is now business interest in an implementation project, to be rolled out initially to HPE s U.S. Enterprise sales organizations during the second quarter of FY2015. The solution could

7 Page 7 Figure 4. Top five opportunities predicted not to win. significantly reduce sales costs and improve customer retention and acquisition. As Townsend explains, A company can be more agile around opportunities if it has this degree of guidance as to the outcome. Although it will never be 100% accurate, we can make decision based on this which will reduce sales costs. For example, if we know a customer will always buy from us, we can offer great service without throwing a lot of resource at the opportunity. If we know historically a customer requires a lot of support through the buying process, we can align those resources for a better customer experience. In addition, sales managers gain visibility into the sales pipeline to identify opportunities with the potential to be successfully closed by proactively adding sales representatives, scheduling executive visits, or sweetening the deal. Sales managers can also better monitor a sales representative s progress toward goals. For example, most sales reps will have goals or targets, says Townsend. We have found consistently that a sales rep will usually be overly optimistic when it comes to forecasting within a CRM system, while in reality, the opposite may be true. The predictive analytics engine can reduce that overly optimistic human element and allow sales managers to make different comparisons to the target. A manager may say, This is what you forecast in our CRM, this is what Vertica tells us about the close date. Let s talk about the gaps between these data points and your target. In summary, predictive analytics when applied to a sales pipeline allows the sales manager to: Increase the win rate by converting more opportunities to closed sales. More effectively forecast and measure progress toward sales targets. Spot early signs of issues in the sales pipeline and proactively address them by, for example, getting a sales rep to focus on a particular product line, timescale, or any other attribute, says Townsend. Ideally, a manager can take action before the shortfall in pipeline turns into a missed target. Typically exposed can be a reps over reliance on one big opportunity, or an over optimistic timescale on a customer s purchase. Sometimes incorrect resources are engaged on an opportunity, or simply the wrong product is being positioned. Improve business decisions with data-driven analytics. For example, the analytics might suggest that altering the product portfolio mix or adding new upsell/cross-sell options could increase the Win rate. More effectively deploy sales representatives to targeted opportunities and customers. Improve executive alignment by planning and managing the sales pipeline more effectively.

8 Customer at a glance Software HPE Haven Vertica engine SAS Real-time QlikView reporting Instead of losing two opportunities, not to mention a significant amount of time and money in presales activities, you can concentrate your efforts on winning one good opportunity and avoiding losses elsewhere. Dominic Townsend, Business Intelligence Systems Expert, Hewlett Packard Enterprise Sign up for updates Rate this document Going forward To date, the POC has been presented to External and Internal customers, to our executives at SVP level, and too many of HPE s internal BI teams. Opportunities for further research include: Applying a Predicted close date accuracy component to the prediction model. With this component we could expand the predictive analysis to examine the sales rep imputed ship date. We use the analysis from HPE Vertica to examine the accuracy of this ship date, and suggest a more accurate one based on the patterns and trends for that Account, Product, Sales team, and Partner. Determining model accuracy for new customers. New customers are a challenge since we do not have any buying pattern for them, and there are fewer data points on which to base analysis. For example there are no buying patterns and fewer engagements for the model to learn from. We would need to further develop the learning tree to take this on effectively. Market research on the customer and similar equivalent size customers can be used as a starting point. We also need to consider different aspects such as customer statement of work, strength of our products, competitors involved in the bidding process, potential discounts, post sales services, and the overall experience of our sales force in working with the customers. Applying market-basket analysis to model. The predictive engine learns based on the data fed into it. With the current PoC solution, that data is only based on what the teams had internally, but if there are trends across the IT marketplace, these could be built into the design. Identifying guidance for sales organizations to help win opportunities. This guidance could be based simply on the variables examined. For example, instead of offering the typical one-phrase recommendation (e.g., review product mix, review sales team, review partner strategy ) a best practice might involve guidance tailored to the products, customers, timescales, etc. For example: This customer typically buys on a six month timescale, the predicted close date for this opportunity is not within this cycle, check timescales with your customer or Opportunities for these products usually take more than 2 weeks to close, your customer has not purchased them before, offer customer more presales support. The goal would be to better equip the sales person with information about the product, account, timescales, and buying patterns to provide better service to the customer, which in turn will inspire greater customer confidence in making the purchasing decision. Copyright 2014, 2015 Hewlett Packard Enterprise Development LP. The information contained herein is subject to change without notice. The only warranties for HPE products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HPE shall not be liable for technical or editorial errors or omissions contained herein. 4AA5-5980ENW, November 2015, Rev. 1