Buy versus Build Considerations for Clients Purchasing CLO Dashboard

Buy versus Build Considerations for Clients Purchasing CLO Dashboard Prepared by Zeroed-In Technologies for use by clients evaluating CLO Dashboard against their internal development of similar executive performance dashboard software for their learning organizations. 2005 Zeroed-In Technologies, LLC Page 1 of 13

Table of Contents Buy versus Build... 1 Executive Summary... 3 What is the competitive advantage gained from the purchase and use of the application?... 6 Is internal development of this technology part of your company s core business focus?... 8 What is the cost for initial development, implementation and adding enhancements versus the set amount for a vendor supplied solution?... 9 How does the required expertise for internal development match up against the vendor s leadership position?... 11 How long will it take?... 12 Can you afford to take on the risk of possible failure with internal development?... 12 Will you be able to continue to fund internal support and upgrades after implementation?... 12 How valuable will it be to have industry specific benchmarking data to compare your internal measurements to?... 12 Are your competitors going to stay with internal solutions or best of class off the shelf solutions?... 13 CLO Dashboard current considerations for future enhancements... 13 2005 Zeroed-In Technologies, LLC Page 2 of 13

Executive Summary Buy versus Build Decision for Acquiring CLO Dashboard Capabilities In many cases, when executives begin their decision process for acquiring an enhancement or new software application they must first answer the question of whether to buy or build the features and capabilities they have determined are critical. The factors impacting this decision usually involve corporate IT policy on software development prioritization, actual comparisons of costs to create the functions needed with internal resources, the time fairly estimated for internal/external delivery of the defined functionality and widely varying estimating rules for judging between the criteria. An added dimension of the internal resource allotment prioritization is caused by unavoidable conflicts for internal resources that can complicate the buy versus build comparison further. As an aid to our Zeroed-In Technologies prospects and customers, we are providing here a possible format that may be helpful in successfully evaluating the cost/benefit comparison between the purchase of CLO Dashboard and the alternative of committing to the attempted development of equivalent dashboard systems by internal IT resources. From our experience, we feel that learning executives faced with the need to gain use of an action enabled, top down, desktop view for driving successful alignment of the learning investments they manage with the corporate strategic priorities of the enterprise their learning initiatives serve should answer the following questions before making the buy versus build decision: 1. What is the competitive advantage gained from the purchase and use of the application? This question is critical to the buy versus build decision because it sets the basis for value of both alternatives. Whether the dashboard is built internally or purchased must be judged on how well, how soon and how reliably it will deliver the benefits that can contribute to bottom line results for the business drivers of corporate or institutional strategy. If use of the dashboard does provide a competitive advantage, then the value of that advantage will be an important consideration in accurately assigning appropriate weighting factors to the differences uncovered during the contrasting proposals for internal or external software sources. In the direct case of CLO Dashboard, for example, the value to a chief learning officer for gaining immediate use of the dashboard to measure and statistically correlate field operational impact resulting from key learning initiatives should be included in direct contrast to the lost opportunity cost resulting from the delay in waiting for internal IT resources to complete their needs analysis, project planning, code development, application debugging and test/production process while responding to the numerous other development requests they receive and respond to. Assigning monetary estimates to the impact of the competitive advantage gained will vary according to the industry and company specifics but it is almost universally an important consideration. As one corporate executive stated, the most effective enterprise strategy is to have your competitors not succeed in enacting their own strategy. Since control of a competitor s strategy alignment and enactment is not within your power, the value to be assigned to the successful use of dashboard software to aid in keeping your organization s learning investments aligned with corporate strategy is based on the competitive advantage you may lose through delay. 2005 Zeroed-In Technologies, LLC Page 3 of 13

2. Is internal development of this technology part of your company s core business focus? It is generally accepted that resources engaged in creating solutions not proprietary to the company and its core business competitive advantage should be minimized to allow for those internal resources to be used for the secret sauce that creates real value in competitive differentiation. A payroll system might be a good example. In almost all cases, having an internally created payroll system will not gain you market share. The ability for off-the-shelf systems or outsourcing vendors to do this function more efficiently make it easy to rule out developing a payroll software system internally. The same holds true for your learning and performance management dashboard. 3. What is the cost for initial development, implementation and adding enhancements versus the set amount for a vendor supplied solution? Setting the cost estimate for the vendor solution is fairly straightforward. In the case of CLO Dashboard the installation and implementation for up to 10 key measures can be accomplished in one week. The cost of creating and successfully implementing an internal version of the dashboard will vary by company. However, the history of the software industry has shown that vendor applications have a significant advantage over internal development due to the ability to spread the cost of development, maintenance and feature enhancements across the greater funding base of multiple client sites. 4. How does the required expertise for internal development match up against the vendor s leadership position? This difference in alternatives requires a qualitative view of the previous history of the architects of the proposed solution. In the case of CLO Dashboard, our founder and President, Chris Moore, has been an award winning technology innovator and industry spokesman since the inception of our learning industry s move to web based management systems. This factor analysis must put a weighting on the risk factor of any single company having and retaining the same level of experience and insight into the current and future perspectives of executives responsible for global learning organizations. Can your internal team build leading-edge technology? Will the internal programmers doing this work in your organization today still be involved in maintaining and enhancing the internal system to keep pace with the innovations of this application area in the future? 5. How long will it take? If building technology is not your company's highest priority, it will be difficult to compete with a company that builds technology as its core competency. A technology development company will acquire the best developers and allocate more money in order to develop new technologies faster then its competition. It is highly unlikely that you could build technology faster than these companies can. Understanding the performance measurement space is critical in developing an efficient, yet robust, system. 6. Can you afford to take on the risk of possible failure with internal development? The vendor supplied system is already performing to its warranted functionality. With any internal IT project there is always the possibility that the proposed project plan will 2005 Zeroed-In Technologies, LLC Page 4 of 13

not be successful in producing the defined capabilities or remaining active through to the desired completion of the specified capabilities promised for delivery. 7. Will you be able to continue to fund internal support and upgrades after implementation? While software vendors invest their expanding resources gained from additional clients to enhancing the dashboard capabilities, will your budget allow you to keep pace with internal development project costs? 8. How valuable will it be to have industry specific benchmarking data to compare your internal measurements to? Zeroed-In Technologies is also creating a database for use as a benchmarking tool for our customers. As participating clients submit data anonymously for key learning indicator performance in their companies, we will be adding dashboard features to allow our clients to compare their own performance profile to the results from other clients according to the key learning indicator and key measure results we will be compiling across customers and industries. 9. Are your competitors going to stay with internal solutions or best of class off the shelf solutions? Your buy versus build decision does not exist in a closed system within your company or institution. If your competitors are staying with internally developed systems or delaying any decision at all, one of your considerations should be centered on moving to a more effective, vendor supplied solution before they gain similar advantages. This is a framework for conceptually putting the key elements into your chart for contrasting alternatives between buy versus build. Assigning specific numbers and detailing and quantifying each option must be worked through in your individual circumstances. We will put forth an example here of what a summary buy versus build proposal might look like in supporting a decision to purchase CLO Dashboard. 2005 Zeroed-In Technologies, LLC Page 5 of 13

What is the competitive advantage gained from the purchase and use of the application? There has been wide acceptance of the importance of hiring, retaining and training of all company employees as the key component of attaining or acquiring a competitive advantage in today s commercial marketplace. The basis for a company s advantage in their industry often lies within the speed with which learning interventions can be designed, developed and effectively distributed to a modern, global workforce. To understand the value of investments in learning projects and programs that have been funded and initiated, it is critical to view the effectiveness of the learning activity in creating a result in direct business impact. Achieving a top down view of this correlation between dollars spent on learning to close a workforce skill gap for competitive advantage and the actual business impact realized in operational measurements, has been difficult if not impossible to gather in summary fashion suitable for executive use. One indication of how important this capability is to executives attempting to manage the alignment of their learning investments with corporate strategic goals can be found in the results of a recent web survey conducted by TrainingOutsourcing. In the results shown below, it is clear that executives value the capacity to see reliable Kirkpatrick Level 4 results as their highest priority for having effective control in providing learning and knowledge management support to their corporate success. Poll submitted with permission from TrainingOutsourcing As it relates to the buy versus build decision, this perspective contributes two vital criteria for the assessment. 2005 Zeroed-In Technologies, LLC Page 6 of 13

First of all, if an internally developed solution were able to provide the same, comprehensive business performance management capabilities found within CLO Dashboard ( and that will be analyzed for feasibility in later sections ), what will be the cost of missed opportunities before the internal solution is delivered? The CLO Dashboard client can capitalize on having better and more effective measurements in place typically within a week. For learning executives selecting the build scenario, that contrasting interval needed for an internal IT project to conceptualize, design, create, test and implement an internal version of a learning executive dashboard will have a cost of delayed competitive advantage. Assigning a dollar value to this would be based on how important it is be operating at the highest level of effectiveness over the period of two to six months needed to replicate the capabilities internally to match what CLO Dashboard has available to clients today. In the second case, will your competitors be moving their level of alignment effectiveness more quickly? Based on the urgency of the priorities outlined above, it is reasonable to assume that the pressure to have more effective measurement tools and the resulting management capabilities in place will drive clients to gain use of this software application as soon as it is feasible. To quantify this aspect of the buy versus build decision, it might be useful to estimate how much more effective you think your workforce could be if the correct learning interventions were put into effect two to six months earlier. For example, in a sales organization, a starting point may be to look at a 1% to 5% increase in sales revenue for a new product launch over the two to six month period. Certainly, the learning intervention for the new product launch would be able to proceed whether CLO Dashboard ( or the internally built dashboard ) were in place or not. However, the ability to monitor and manage the statistically validated direct impact on business outcomes would be subject to your current IT constraints for getting this kind of accuracy from other learning system reporting formats or Excel spreadsheet composite formats. The difference in cost between buy versus build must be weighted against this lost opportunity to manage all aspects of learning alignment before your IT organization can supply their internally built version of a comparable dashboard. And, this same cost assignment should be applied to every new enhancement that will be forthcoming for CLO Dashboard clients while your internal organization adapts to changes in the industry. 2005 Zeroed-In Technologies, LLC Page 7 of 13

Is internal development of this technology part of your company s core business focus? This is a straightforward question for your organization. However, assigning a dollar amount to it as part of the decision criteria requires that your IT organization provides an estimate of what cost would be associated with the delay in completing projects for applications that are directly related to proprietary systems that are part of the core business differentiation for your company. The practical ramification of this cost comparison is more typically associated with the delays experienced in getting the IT project for your learning organization actually completed, tested and delivered. To put a dollar amount on this factor it is appropriate to review historically the number of cases were learning software application projects have either been denied funding, delayed at some point or delivered absent of key functionality that was part of the original design because of changing IT priorities to support software development that is more directly related to proprietary areas of your company s core business. 2005 Zeroed-In Technologies, LLC Page 8 of 13

What is the cost for initial development, implementation and adding enhancements versus the set amount for a vendor supplied solution? For this initial estimate it might be helpful to supply a generic dialogue that would reflect the typical dialogue between IT and internal development clients at their company. Once the needs analysis is done, we can probably have some level of a beta version in 5 to 6 months. If you have staff available to QC that version in a test environment we can make corrections and then begin working on a plan to move into a production environment. As long as you complete the QC in a timely fashion it should be ready to use in production 6 to 10 months from today. As part of this scenario, where delivery dates and project time lines are often missed, the cost of internal resources may initially and optimistically be estimated at 16 to 24 man months at the internal IT support rate to replicate functionality comparable to what is available in CLO Dashboard. Based on industry experience, it is more likely that this dialogue and resulting deadlines and costs will continue to be stretched and prolonged with every project checkpoint where learning application end users are asked to participate in screen capture reviews or functionality testing points. The cost of this interfacing with internal project teams will be the IT resources required plus the time your staff will be involved in testing and corrections to the prototype of the application at every stage. The best source of what the total cost will be is subject to your own experience in getting software applications developed internally. An estimate in terms of direct IT resources for developing a similar application would be to apply at least 3 man months for development at your internal IT rate. Add to this the cost of your staff at somewhere between 16 and 24 man months over the period from design to use in the production environment. For ensuing feature enhancement you will be required to estimate at least one man month per feature and the corresponding use of your staff members time to accomplish integration of a new feature. The cost of CLO Dashboard is easily determined by reviewing the attached pricing sheet and the chart of pricing and installation charges set for various levels of utilization. We have an inexpensive entry level for use of CLO Dashboard set currently at $900 per month under an annual subscription license with a one time $5,000 Installation and Implementation Fee ( for up to 10 key measures ). There are a number of approaches to setting the actual cost and benefits into equations that can reflect the comparison for a buy vs. build decision. The savings in the vendor supplied software comes from reuse across a wide range of customers. In a 2004 article 2005 Zeroed-In Technologies, LLC Page 9 of 13

(http://www.computerworld.com/developmenttopics/development/story/0,10801,89602,0 0.html ) appearing in Computerworld ( 02/2004) and describing the cost avoidance realized in the reuse of software, the following dollar amounts and savings estimates were provided. The bottom line is always king, especially in today's resource-constrained environment. Warm and fuzzy notions of "reuse is good, therefore let's do reuse" can only go so far towards getting the necessary management buy-in (along with the staffing and tools required to successfully execute a reuse initiative). To succeed, you must be able to present a compelling business case for reuse using realistic and easy-to-understand metrics. We have found that combining pragmatic and honest assumptions with a simple, easy-tofollow business model provides all the necessary evidence to sell reuse to management. We start by asking: "How much effort could you save by reusing something rather than writing it yourself?" Typical answers span a wide range (usually from 50% to 100% savings) and depend on factors related to your environment and existing applications. With this in mind (and based on a lot of hard data), we estimate that a reasonable savings due to reuse is approximately 80% of the cost needed to develop the same software development asset (SDA) for one-time use. At an industry-average cost to develop new software of around $100 per line, this means that every 1,000 lines of reused code yields a Development Cost Avoidance of $80,000! As described previously in this document, approximately 20 man months of effort has been invested in the design, development and creation of CLO Dashboard to this point in the software life cycle ( Release 1.1 ) with further enhancement for functionality included in the long term product road map. A reasonable cost comparison might be to apply the range supplied by Computerword ( $100 per line of software ) to the number of lines of code for applications currently produced by internal developers over a 20 man month period. Another basic comparison guideline could use a conversion factor allowing for an average man month cost. For example, assuming a fully loaded annual cost for a software developer at $75,000 per year, the cost per month converts to $6250. If the skill sets of the internal application developers matched Zeroed-In Technologies staff and Chris Moore s experience, the cost estimate for internal development could be estimated in the following short hand calculation: $6250 x 20 man months of development = $125,000 for a similar dashboard capability delivered by internal resources. These comparison amounts would not include the development of new features/functionality in the dashboard, the ongoing maintenance costs or the total costs of operational personnel involved during interface check points on an internal software development project. It is offered here as the possible guideline for a rough comparison to the initial license, implementation and annual subscription rates included in the attached CLO Dashboard pricing sheet. 2005 Zeroed-In Technologies, LLC Page 10 of 13

How does the required expertise for internal development match up against the vendor s leadership position? A summary of the credentials attained by Chris Moore throughout his award winning career of technology innovations in the learning industry is provided below along with a listing of recently published articles in the area of Workforce Performance and Learning Management Metrics. Please feel free to contact us for additional background if it is needed to provide a basis for comparison to the internal resources suggested for assignment to your dashboard project. As President and Founder, Chris is the driving force behind Zeroed-In's strategy. He is highly regarded as a visionary in learning technologies having pioneered the concept of enterprise learning management and other key innovations throughout the late 1980's and 90's. Prior to founding Zeroed-In, Chris served as Chief Technology Officer for THINQ Learning Solutions, Inc. where he was the visionary and driver behind the TrainingServer suite of products. During his 15-year tenure at THINQ, Chris transferred his experiences and vision into a framework called The Learning Management Maturity Model (LM3 ). As a first of its kind, THINQ's LM3 combines organizational process maturity and various technologies to help organizations define and execute on their learning and performance strategy. His efforts have won him respect as a speaker and author by numerous trade associations and journals including ASTD, CLO Magazine, HR Executive, Training Director's Forum, and The Conference Board. Chris holds a BS in Business Information Systems from the University of Phoenix. Links to recent articles and white papers authored by Chris Moore are provided below and can also be accessed from the Zeroed-In Technologies web site. Measuring Success: Capturing the Right Metrics, Workforce Performance Solutions Magazine August 2005. Get Zeroed-In on Learning Measurement - Summer 2005, Zeroed-In Technologies Newsletter Summer 2005. Measuring Effectiveness With Learning Analytics, CLO Magazine May 2005. Aetna: Measuring Up With an Effective Turnaround, CLO Magazine May 2005. Seven (7) key learning indicators your CEO really needs to know this white paper is available from the Zeroed-In Technologies web site. 2005 Zeroed-In Technologies, LLC Page 11 of 13

How long will it take? The amount of time to complete a full cycle of development for a dashboard that is similar in function to CLO Dashboard was briefly reviewed in the section on internal cost estimation. For this aspect of the buy versus build comparison it will be appropriate to apply the experience gap between the extensive industry expertise possessed by Zeroed- In Technologies key technical resources and your internal IT project members. It may be an approximate estimate on the time required to add 3 to 4 months to the costing period unless the internal resources are somewhat comparable in terms of experience in the learning industry. Can you afford to take on the risk of possible failure with internal development? The answer to this question can be provided through a review of your company s past record for supplying internally developed applications that are not part of the core business competency for your industry. Will you be able to continue to fund internal support and upgrades after implementation? You will need to use your learning organization s history of successfully or unsuccessfully seeking funding for enhancements to internally developed software that are not part of the core business competency for your industry. Typically, there are a number of instances where funding for enhancements for this renewed IT support is lacking for enhancements and maintenance. How valuable will it be to have industry specific benchmarking data to compare your internal measurements to? The value to you of having a reference point for comparing the performance of your learning organization on key measures against peer groups delivering enterprise training programs and the industry your company competes in cannot be gauged precisely until the service is provided. A simple basis for the value might be in asking the binary question of what is the benefit to this information versus not having it all. With an internally developed dashboard the key measures and performance statistics related to business outcome will certainly not be provided in a format that allows for the anonymous comparison as is planned for use by Zeroed-In Dashboard clients in our benchmarking service. 2005 Zeroed-In Technologies, LLC Page 12 of 13

Are your competitors going to stay with internal solutions or best of class off the shelf solutions? Using a large body of historical examples as a guide the answer to this question is clear. As competition for more efficient software continues to develop for use by training organizations supporting a global workforce, off the shelf applications will become more effective and comprehensive while internally developed applications will fall behind due to the smaller basis for funding innovations and maintenance. The market will adapt to this trend by selecting the most cost effective vendor supplied software solutions in most cases where the application focus is not proprietary to the core business of the client. It is reasonable to assume that competitors in your industry will move to vendor supplied, enterprise learning dashboard software as the market matures. CLO Dashboard current considerations for future enhancements The product roadmap for CLO Dashboard continues the enhancement of industry leading innovations in business performance management for learning executives. For consideration to preview these roadmap plans, please contact the Zeroed-In Technologies sales contact for your account. # # # This document provided compliments of Zeroed-In Technologies. Zeroed-In Technologies markets CLO Dashboard, a strategic reporting and decision making system designed specifically for chief learning officers and training executives. Zeroed-In provides software and services that help visualize, measure, and execute effective learning and performance strategies. By combining sound strategy with proven process and advanced technologies, Zeroed-In helps customers optimize their business by achieving the most effective, high-performing workforces. Find out more at: www.getzeroedin.com Zeroed-In Technologies 332 E Maple Rd, Suite 100 Linthicum, MD 21090 Phone: (410) 859-0478 Fax: (443) 517-4000 Email: info@getzeroedin.com 2005 Zeroed-In Technologies, LLC Page 13 of 13