SEER for Software - Going Beyond Out of the Box David DeWitt Director of Software and IT Consulting SEER for Software is considered by a large percentage of the estimation community to be the Gold Standard for software cost estimation. In fact, the Gartner Group says: Estimation is a key aspect of capability maturity SEER-SEM and capability go hand in hand. Organizations will experience at least a 50 percent reduction in their estimates variance vs. actual costs within two years. A high-level SEER for Software estimate can be developed in a matter of minutes. Simply define a project by platform, application, development methodology, governing standards, and a size value; and SEER will generate an Industry Based Estimate. But what if greater precision is necessary than is provided by the standard SEER for Software configuration? That s when calibration and customization come into play. The focus of this paper will be to discuss two areas where the SEER for Software parametric model can be customized to incorporate an organizations unique day to day operations as well as their observed performance using Custom Knowledge Bases and Software Sizing Proxies. Knowledge Bases Knowledge Base Overview Galorath Definition: A Knowledge Base is a set of pre-defined settings for a subset of the model s technology parameters based on some key project characteristics. SEER for Software s core model is configured to a particular circumstance ( out-of-the-box ) by a set of knowledge bases, and it s these knowledge bases that are calibrated based on new industry information and trends. Hence the term Industry Based Estimate. These knowledge bases correspond to specific people/process/technology related parameter values. In fact, each knowledge base is defined specifically to the underlying subset of likely parameters, some visible to users, and others normally hidden. For example a unique knowledge base may be used when developing a Customer Relationship Management application, or if developing to the Salesforce.com platform, or even to capture the efficiencies of an experienced Agile development team. All of these people/process/technology characteristics are captured and reside in one of over two-hundred unique knowledge bases delivered with the application. So how does Galorath develop the knowledge bases? These days software estimation vendors are competing to have the largest repositories of completed software projects, and the customer is 222 N. Sepulveda Blvd., Ste. 1700 El Segundo, CA 90245 o. +1 310.414.3222 f. +1 310.414.3220 www.galorath.com Suite 10, Basepoint Business Centre Andover, Hampshire SP10 3FG United Kingdom o. +44 (0) 207 788 9042 f +44 (0) 207 138 2642
encouraging this competition, which is fundamentally good. However, there is more to insuring the accuracy of an estimation model than just having a lot of data points sitting on the proverbial shelf. Galorath constantly collects data from many sources, both public and private. Data is scrubbed, normalized and processed to be useful using methods that Galorath routinely and openly discusses In addition to data, numerous other sources play an important role in maintaining the accuracy of SEER for Software estimates such as development trends All of this data is then represented in the various knowledge bases, language factors and size metrics offered by SEER-SEM The overall evolution of SEER for Software is best called innovation-driven. While data analysis is a very important part of how we maintain SEER for Software, we also continually enhance the model s ability to estimate real world projects. Based on these results, new and updated knowledge bases, language factors and size metrics are added to SEER-SEM as required.. These enhancements have often been industry firsts: flexible project staffing, off-the-shelf (COTS) integration modeling, translation of estimates into detailed project plans having intricate interdependencies, cloud computing solutions, to name only a few. All these innovations, alongside data-driven updates, serve an important role in insuring the model s precision. The bottom line is most customers are highly successful using the default knowledge bases to generate an estimate. However, those times when using SEER for Software Out-Of-The-Box does not fit the bill such as unique operating platform or application characterization, or are working to a hybrid development life cycle or governance standard - custom knowledge bases can be generated to allow the model to estimate an organization s unique project characteristics. Anatomy of a Knowledge Base SEER for Software deploys over a hundred unique parameters for any single estimation element. It is the knowledge bases that specify the pre-defined settings. For example, the Personnel Capabilities and Experience parameters are used to characterize a development team and will accept a range of Least, Likely, and Most values. Figure 1 displays the state of these parameters prior to selecting a knowledge base. As can be seen, the settings indicate that the values associated with all of the parameters are essentially not weighted. Figure 1- Default parameter set with no Knowledge base Figure 2- Knowledge base selected
Figure 2 represents the parameter settings after a knowledge base has been selected. In this case the Platform knowledge base was set to Web Based Development. Using this knowledge base SEER for Software adjusts the individual parameters if required based upon the data collection and investigation described previously. In essence the Industry sets the values. This process of knowledge base driven parameter distribution applies to ALL the parameters within all of the categories as shown in Figure 3. However, each knowledge base (with the exception of Platform) has a specific subset of parameters that are modified. So let s talk about the different kinds of knowledge base categories. Figure 3 - Parameter categories The knowledge bases are broken into the following characteristic types (categories): Platform What does this run on? (e.g., Web Development, Mainframe, Salesforce, ERP) Application What does it do? (e.g., Data Mining, Workflow, Decision Support) Acquisition Method How does it come to life? (e.g., New code, Integrated, Minor modification) Development Method How will it be built? (e.g., Waterfall, Prototype, Agile) Development Standard What regulatory governance? (e.g., IEEE, Mil-Std, ISO, Commercial) Class Explicit set of parameters to change (wild cards) The Platform knowledge base potentially modifies EVERY parameter hence laying a foundation for the remaining knowledge bases to come along afterwards and modify a smaller set of specific parameters that suit a unique knowledge base such as a team using prototype techniques instead of basic waterfall. (The mapping of categories to parameters can be found in the SEER for Software Detailed Reference). In summary selecting various groups of knowledge bases will set the parametric values for you; there is no need to set each parameter individually.
Custom Knowledge Bases The good news is Knowledge Bases can be customized to fit your organizational needs. Customization takes on two flavors modifying an existing knowledge base or creating a completely new knowledge base. A knowledge base should be modified if the default values established by Galorath for an Industry setting do not reflect what is observed or required by an organization. This is often discovered after generating several estimates and the team finds that they are consistently changing a parameter away from the default value. In this instance, the user can modify the knowledge base so that in the future the proper settings are already embedded in the knowledge base. To do this, one only needs to simply open the knowledge base and change the parameter then save the knowledge base for future use. It s that simple! Following best practices the user is encouraged to change the name of the knowledge base internally (how it appears in the menu list) and the filename to reflect the modification, ie: Change the Workflow.app to XYZ-Workflow.app. This process is described below and in SEER for Software Detailed Reference. Modifying an Existing Knowledge Base To create a knowledge base from an existing one: 1. Open an existing knowledge base file. 2. Enter a new Description for the knowledge base. 3. Modify the parameters to suit your requirements. When you have set all the parameter values correctly, proceed to the next step. 4. From the File menu, select Save As. The Save a Knowledge Base dialog box appears with a list of category selections. Choose a category for the new knowledge base. 5. Click OK to exit the Save a Knowledge Base dialog box. 6. In the Save As dialog box, select the knowledge base file type and enter a name for the new knowledge base. Generating a new knowledge base takes a bit more research and preliminary investigation prior to deploying the model. Typically, a user will use an existing knowledge base and modify as many of the parameters as is required to describe the new thing being characterized. However, should the user wish to start from scratch, it s a simple step to create a new knowledge base with all the parameters set to no value. The user then begins the process of specifying what values any or all the parameters should take to fulfill the characteristics of the item being described, (ie: a new hardware device platform, or internal governance standards).
Creating a New Knowledge Base from Scratch 1. From the File menu, select New. The Create/Modify WBS Element dialog box appears. Do not make knowledge base selections. 2. Enter a Description for the knowledge base. Click OK to exit the Create/Modify WBS Element dialog box. 3. From the Estimate menu, select Clear Kbase to No Knowledge. This erases all parameter values, setting them to appear as asterisks (***) or zeroes. If you do not use Clear Kbase to No Knowledge, parameters such as Analyst Capabilities will retain their default nominal values (Nom-Nom-Nom) and will overwrite existing parameter values when loaded. 4. Enter the parameter values for your knowledge base, as you would enter them for a project. When you have set all the parameter values correctly, proceed to the next step. 5. From the File menu, select Save As. Follow the steps for saving a knowledge base, as described in steps 4, 5, and 6 in Modifying an Existing Knowledge Base, above. Whichever method you use, the next time you create a new project or WBS element, your new knowledge base should appear in the category to which you assigned it. Class Knowledge Bases While it is simple to work with custom knowledge bases, there is a special category of knowledge bases known as Class knowledge bases. These are essentially knowledge bases that contain a set of wild cards - if you will - to modify any desired parameter(s). For example let s assume you have two vendors on a project and one vendor is much more experienced or less expensive than the other. A class knowledge base can be created let s call it the Preferred Vendor class. This custom knowledge base is created with parameters set to reflect the qualities of a higher performing team or lower dollar cost values. Once the estimation element is created the user can simply apply Preferred Vendor class knowledge base to the estimate and only those unique parameters will be modified. Again a very quick and elegant way to manage a cost estimate.
Software Sizing Proxies Proxies an Overview In the software world, size is a measure of functionality and this functionality is the main driver of the development effort, cost and schedule via parametric models. Traditional expressions of size include measures such as lines of code, number of features, or functions, function points and their derivatives, to highlight a few. Consequently, functionality must be expressed in size. But what if you don t know the size of your software? That is where SEER Proxies come into play. Proxies are customized size metrics based on anything which can be counted, and which roughly corresponds to your project's development effort and schedule. SEER-SEM converts proxy input to standard units, using established conversion factors. For example, if you know that an application's size corresponds to the number of screens, you can create a proxy named Screens, which will convert the screen count into function points. What You Need To Know Proxies can be used for anything that has an associated effort and convert it into an equivalent lines of code or function points, (ie: Developing Workflows, Writing Queries, constructing Facts and Dimension table, etc.). To build a proxy the user only needs to know two things: 1. how much effort it takes to do whatever it is that is being performed 2. what the domain is in which it is being performed (Knowledge bases) The SEER for Software tool contains a proxy conversion mechanism that accepts these two inputs and converts them to a sizing value. Figure 4 - Converting "effort" into size
However, there is one catch, when building the knowledge base, the user needs to create a relationship between effort and size. This is called a sizing factor and it is this value that is fed into the proxy building tool. Calculating a sizing factor is very easy to do using the SEER for Software model as the model understands and calculates productivity. For example: If the team knows it can generate ten average workflows in a month, using a two person team, the model can easily calculate how much size can be accomplished for that two person team in one month. That calculated size then becomes the sizing scale per workflow. This is described in detail below. Building a Proxy The first step in building a proxy is to identify the project characteristics. This will be fed into the model using knowledge bases. For this example we will use Workflows for the proxy. Step 1 - Establish the Program Knowledge Bases Platform Application type Development Method Development Standards These settings are the same settings that will later be used when using the proxy for the estimate. The goal is to allow the model to establish the likely productivity associated with these knowledge bases. Figure 5 - Setting Knowledge Bases for the Proxy Step 2 Identify the approximate duration for a project during which this work will take place. This may sound like an odd request but the SEER for Software model can account for entropy that is larger projects take longer and are less productive than smaller projects. If the team was simply building one workflow for the project then the productivity would be quite high. But what if the team has to build hundreds of workflows over the duration of the project productivity will decrease with the increased volume of work to perform that s entropy! Using the SEER for Software Calibration/Design to Size mode, the user simply enters in the Effort Months associated for a typical project that will be building workflows. In the example in Figure 6, the proxy will be built with an assumption that a typical project takes 12 effort months (2 people for six months or 3 for four or ). Figure 6 - Design to Size mode reflecting likely effort months
Once this value is entered, the model will automatically calculate how many lines of code or function points are possible during that time period. For our example shown in Figure 7, the model indicated that a little over 3,000 lines of code can be developed (Requirements, Design, Code, Test, Acceptance Test and Integrate) in 12 effort months. (Note: If we want to isolate just the development effort then set the two Yes switches in Figure 6 to No ). Figure 7 Lines of code possible for 12 effort months Step 3 Determine individual programmer productivity. We now enter the lines of code value into the model to generate an estimate for the work and to generate a productivity value. From the Quick Estimate report we can see that the model assumes a productivity of 290 lines of code per person month while building workflows. Previously we stated that our two person team can generate ten average workflows per month. 580 LOC per month / ten workflows = 58 lines of code per workflow. Voila! You now have a sizing factor. Figure 8 - Quick Estimate report identifies productivity Step 4 Generate the Proxy. Now that we know the sizing factor for one average workflow we can generate the proxy using the Proxy Sizing tool inside SEER for Software. I went ahead and made some assumptions about how long the team would take to build a simple and complex proxy as well. To build the proxy: 1. Click New 2. Give it a Name 3. Enter a file name 4. Create the Input Names (Scale) 5. Apply the Factors 6. Click Save Figure 9- Using the Proxy Builder tool
Step 5 Generate an estimate using the proxy. The proxy is now ready to be used in an estimate. The estimate is generated the same as always, however, now instead of entering a lines of code count or a function point count you can simply enter how many workflows need to be created! The model will do the conversion. Figure 10 shows how the values are entered. For this example the team has indicated they will build between 10 to 15 simple workflows, between 18 to 25 Average workflows and 2 complex workflows. This is an important feature that should not be overlooked the ability to introduce uncertainty into the estimate using a range. Every good estimate should include risk and uncertainty! Figure 10 - Entering values into the proxy As was just illustrated generating a proxy to capture size volume is quite easy. It is also easily adaptable and scalable for teams. Typically a project will be composed of various teams working on different technologies. Proxies make it easy to keep the team focused upon the work at hand and not inventing a lines-of-code count for the cost estimate. The size reflects that performance observed on previous projects. Summary SEER for Software has proven itself to be a reliable cost estimation model regardless of technology or business sector. Most organizations use the product right out-of-the-box and make small fine tuning parameter adjustments or calibrations to the model. However, should a time come when the model does not exactly fit the particular requirements for a given project constraint it is easy to extend the model to accommodate nearly every possible contingency. Custom Knowledge Bases and Proxy Sizing are just two of the many ways of Going Beyond Out of the Box