Improve Your Software Development Lifecycle Process - Practical Tips and Guidelines

Transcription

1 Improve Your Software Development Lifecycle Process - Practical Tips and Guidelines Co-Authors: Daniele Chenal and Paul Schwartz Audience: Project Managers, Product Managers, Business Analysts, IT Managers, QA Managers and other Business Stakeholders involved with software development processes. Purpose: Improving software development lifecycle processes have been shown to significantly improve time-to-market (or time-to-release), lower costs and improve application quality. To assist organizations in implementing a more structured development lifecycle process and overcome some of the typical challenges associated with team collaboration and change management, this paper provides an introduction to the fundamentals of SDLC (Software Development Life Cycle) frameworks (Waterfall, Iterative and Agile) and includes: The common phases, activities and typical deliverables of an SDLC methodology Basic guidelines, tips and templates by phase Tools to consider that can aid in project management, collaboration, communication and change management Table of Contents: Introduction.2 SDLC Methodologies... 4 Activities and Deliverables By Phase.6 Practical Tips and Guidelines By Phase 9 Tools to Consider.16 Templates.17 About The Authors 25 Copyright 2010-QAvantage Page 1

2 Introduction Companies continue to look for ways to lower development costs while improving software quality, and speeding up their development cycles. Some of the statistics below explain some of the drivers behind this. The average cost overrun for large companies is 178% The average cost overrun for medium companies is 182% The average cost overrun for small companies is 214% Some of the major issues sited as causes for these overruns include; lack of user involvement, clear requirements, and proper planning. Source: Standish Group For a software company developing commercial software, and IT organization delivering business applications to your users or Systems Integrators delivering solutions to your customers improving your overall development lifecycles will yield benefits to you and your respective customers. While the authors of this paper are strong proponents of implementing a single, collaborative SDLC (Software Development Lifecycle) platform, the information defined in this paper should prove beneficial regardless if this type of tool is utilized. This paper offers tips, templates and other guidelines to improve key areas of collaboration, communication and information management all of which can have a significant impact on improving your processes and lowering costs. Plan for Change A key lesson learned over many years is that regardless of the methodology you choose, you need to plan for the changes that will occur. From the inception of the project and through-out the lifecycle some fundamental things to consider include: Requirement changes and the impact of those changes across teams o Including co-located resources, technical writers, QA and others affected by changes Change in resources o People change positions, leave companies, get sick and go on vacation Change of scope o Due to changing business/market conditions With this in mind, you need to have in place at least basic tools and process to manage not only the lifecycle but more importantly the impact of inevitable change. Unmanaged change causes confusion, loss of knowledge, miscommunication, rework, and inefficiencies. This of course translates into loss of time, money and often missed intended business objectives. As most individuals experienced in software development would agree, the more agile your organization seeks to be and the more you strive to perform activities in parallel, the greater your need for the following: A clear communication plan A sound technique for requirements management and their relationship to other deliverables A single source of all current project information (Project Knowledge Base) Page 2

3 Figure 1: Project Knowledge Base Figure 1 illustrates the fundamental types of information gathered in most organizations. We believe that while development methodologies will vary, this represents a very typical arrangement of the kinds of data that needs to be managed and the basic information flow. This paper will use this perspective, avoiding an orientation to a specific methodology, each of which has different advantages depending on the type of project being executed. Our aim is to provide recommendations for improvement irrespective of the methodology applied. Before we examine detailed areas and provide examples for improving processes, let s look at the types of SDLC methodology frameworks in use today, further illustrating the common characteristics they share and their differences. Page 3

4 SDLC Methodologies Waterfall Framework Defined: To follow the waterfall model, one proceeds from one phase to the next in a purely sequential manner. Arguments for: time spent early on making sure that requirements and design are absolutely correct is very useful in economic terms (it will save you much time and effort later), it places emphasis on documentation (such as requirements documents and design documents) as well as source code. Arguments against: The waterfall model however is argued by many to be a bad idea in practice, mainly because of their belief that it is impossible to get one phase of a software product's lifecycle "perfected" before moving on to the next phases and learning from them clients may not be aware of exactly what requirements they want before they see a working prototype and can comment upon it; they may change their requirements constantly, and program designers and implementers may have little control over this. The content above is licensed under the GNU Free Documentation License. It uses material from the Wikipedia article Waterfall Model Figure 2 Iterative Framework Defined: The basic idea behind iterative enhancement is to develop a software system incrementally, allowing the developer to take advantage of what was being learned during the development of earlier, incremental, deliverable versions of the system. Learning comes from both the development and use of the system, where possible. Using iterations, a project will have one overall phase plan, but multiple iteration plans. Figure 3 Page 4

5 The content above is licensed under the GNU Free Documentation License. It uses material from the Wikipedia article Iterative and incremental development Agile Framework Defined: Most agile methods share iterative development's emphasis on building releasable software in short time periods. Agile methods differ from iterative methods in that their time period is measured in weeks rather than months and work is performed in a highly collaborative manner, Agile methods attempt to minimize risk by developing software in short timeboxes (or sprints), called iterations, which typically last one to four weeks. Each iteration is like a miniature software project and includes all the tasks necessary to release the mini-increment - planning, requirements analysis, design, coding, testing and documentation. Combined with the preference for face-to-face communication, agile methods produce very little written documentation relative to other methods. This has resulted in criticism of agile methods as being undisciplined. Figure 4 The content above is licensed under the GNU Free Documentation License. It uses material from the Wikipedia article Agile Software Development With some methodologies some activities are performed in parallel while in others activities are performed in sequence. While each methodology framework follows a different approach in terms of the time intervals per phase, pre-requisites to move to the next phase and the degree of documentation produced, the phases generally consist of similar activities and produce similar deliverables. Page 5

6 Activities and Deliverables By Phase The table below illustrates the types of activities and the deliverables found in each phase of a product development lifecycle. Again, dependant upon your methodology the degree of formal documentation you produce will likely vary based on the methodology employed. Project Life- Cycle Activities Owner Other Participants Project Conceptualization (Kick-Off) Project Concept Business Requirements Phase Gather Requirements Requirement Walkthrough, Prioritization and Approval Business Business Analysis and Design Phase Analyze Requirements and Design Project Estimation and Scheduling Detailed Design Specifications/ Prototype Project Estimation and Scheduling IT (Dev) PM IT(Dev) PM IT (Dev) and QA IT (Dev) and QA QA and Business QA, IT and Business QA, IT and Business QA, IT and Business Development and Unit Testing Phase Typical Deliverables Define Key Business Objectives Identify Project Sponsorship and Project Manager Determine additional Stakeholders Determine Project Type, Scope and Phases Define Communication Plan Kick-Off Project Gather and Document High Level Business Requirements Requirements Review and Prioritization Stakeholder Approval Development IT (Dev) Develop Software Units Define Impacted Systems, Dependencies, and Requirement Related Issues Define complexity of each requirement Document Greater Requirement Details as needed Re-prioritize Requirements as needed Estimate Level of Effort/Costs/Scheduling Perform Initial Risk Analysis Reprioritize requirements as needed Define and Obtain consensus on change of scope Develop detailed technical specifications/prototypes Walkthrough and review Confirm all specifications are reviewed and approved Adjust project phases Adjust Estimate Level of Effort/Costs/Scheduling Perform Second Risk Analysis Reprioritize requirements as needed Define and Obtain consensus on change of scope Test Planning IT (Dev) QA Define Test Scenarios Define Test Quality Gates and Criteria Unit Test IT (Dev) QA Execute Unit Test Plans Page 6

7 Project Life- Cycle Activities Execution Project Scheduling Adjustments Owner PM Integration Testing Phase Define Logical Units Execute Integration Test IT (Dev) IT (Dev) Project Scheduling PM Adjustments System Testing Phase Functional Test Planning QA/ Business Other Participants QA, IT(Dev) QA QA QA, IT(Dev) IT (Dev) Typical Deliverables Correct Defects Retest (Regression Test) until quality criteria is met Evaluate Quality Gate criteria for go/no go to Integration Testing Risk Assessment Reprioritize Requirements as needed Define and Obtain consensus on change of scope Define Logical Units (LU) Prioritize LUs Develop Integration (LU) Test Scripts Develop Quality Criteria and Measurements prior to moving to System Testing Execute LU Test Scripts Certify All LUs have executed Document Pass/Fails Results (defects) Correct Defects Retest (Regression Test) until quality criteria is met Evaluate Quality Gate criteria for go/no go to Integration Testing Risk Assessment Reprioritize Requirements as needed Define and Obtain consensus on Out of Scope Items Define Functional and System Test Cases Define Quality Gates and Measurements to Proceed to User Acceptance Testing Functional Test Execution QA IT(Dev) Assign Tests Document Pass/Fail Results (defects) Correct Defects Retest (Regression Test) until quality criteria is met Evaluate Quality Gate criteria for go/no go to User Acceptance Testing User Acceptance Testing Phase Execute User Acceptance Test Business QA/IT(Dev) Define Quality criteria for passing UAT BU executes Testing Document Pass/Fail Results (Defects) Correct Issues based on quality criteria Retest until quality criteria is met Define and Obtain consensus on Out of Scope Items Document Phase Documentation Review and Approval PM Technical Writers/QA/IT (Dev) Produce Documentation Drafts (On-Line Help, User Guides, Administrative Guides, API Documentation, Training Materials QA Reviews, Editing Document Publishing Managing On-going Change Follow-On Releases Define Requirements Business QA/IT(Dev) Define the Project Type (Minor Enhancement, Major Enhancement, New Capabilities Large Scale, New Page 7

8 Project Life- Cycle Activities Owner Other Participants Typical Deliverables Capabilities Small Scale) Execute the Phases Appropriate for the Project Type Page 8

9 Practical Tips and Guidelines By Phase Project Conceptualization Phase Define Project Types Not all projects will go through a complete lifecycle or require all deliverables. Consider defining a set of Project Types to assist in the management of each project. Consider classifying your projects based on various project traits. Based on those project types and your methodology, define the phases and deliverables that each project type will follow. As you scope each new project, you can then manage it accordingly. Project Type Project Traits Phases New Application Rough Order of Magnitude Over 6 Mos Entirely New Functionality and/or Complex Integration Requirements Enhancements to Existing Application/ Major Release Rough Order of Magnitude 3 6 Mos New Functionality + Enhancements Moderate if any Integration Requirements Full Lifecycle (All Phases), Multiple Iterations Full Lifecycle (All Phases) Single Iteration Enhancements to Existing Application/ Minor Release Minor Enhancements Rough Order of Magnitude 1-3 Mos Enhancements to existing functions No Integration Requirements Rough Order of Magnitude < 1 month Minor Enhancements to existing functions No Integration Requirements Detailed Requirements, Development and Testing Phases Only Limited Requirements, Development and Testing Phases Only Define The Communication Plan Successful projects are delivered by teams that have a clear understanding of the project, their roles and responsibilities and the process governing the work deliverables and approvals of a given project. Most likely, many of your projects also overlap business channels, groups and/or resources. Because of this, it is extremely important to initiate any project with a clear and comprehensive communication plan. A well defined plan should include details such the key business objectives of the project, all team members and their roles, approvals and the quality gates that will be used to manage the phases along with the methods of communication to be used through-out the project. Not only will this help in maintaining clear lines of communication, it can ensure early detection of conflicts and aid in the rapid resolution of issues throughout the lifecycle of the project. A COMMUNICATION PLAN TEMPLATE CAN BE FOUND IN THE TEMPLATES SECTION Standardized Templates and Knowledge Management Consider including in your communication plan, the location where all project source information be maintained (a shared drive, a Portal, an Intranet site, an SDLC tool), templates to use for specific deliverables, and basic project guidelines. Enforce discipline to keep information current. Page 9

10 A PROJECT GUIDELINES TEMPLATE CAN BE FOUND IN THE TEMPLATES SECTION Requirements Definition Phase Gathering and Managing Requirements In this phase your Business Analysts will be gathering and documenting the requirements. The level of detail defined will vary by the type of application, the complexity of the application and other factors. For more complex requirements, considering defining use case examples, the types of users of the application and process flow charts. You may choose to document your requirements in a single or multiple Word documents, an Excel Spreadsheet, a database or a requirements prioritization tool. At a minimum your requirements should include: Number of the Requirement Name of the Requirement Status Category of the Requirement Description of the Requirement Along with the intended purpose and benefit Why to implement the Requirement (Commitments, Competitive Advantage, User Satisfaction, etc.) Priority You may also want to consider documenting high level requirements only first, prioritizing them and then defining them in greater detail in order of priority. If the project facilitates this, your technical staff can then start working on more detailed designs and be assured the highest priorities are being delivered first. A REQUIREMENTS TEMPLATE CAN BE FOUND IN THE TEMPLATES SECTION Keep in mind as you gather and document requirements to provide enough information about the requirement to aid in the developers understanding of the intended business function of the requirement. You also need to provide your QA team measurable inputs and outputs to develop their test cases. Additional information to be considered in the long term tracking of the requirement is: Complexity Dependencies Systems Impacted Risks Constraints At the onset of definition and prioritization phase all the items defined above will likely not yet be known. However, during the analysis and design phase, tracking this information will help you resolve conflicts, plan staged deliveries (releases), estimate level of effort and timelines and make the necessary trade-off decisions that often come with resource and time constraints. Page 10

11 Lastly, as you determine which tool(s) you will use to track requirements, keep in mind the volume of requirements and the likelihood of change. Will the tool you choose facilitated your needs? Consider how easy or difficult it will be to: prioritize and reprioritize your requirements; defer them to future release/phases yet not loose track of them; define and map dependencies between them or other projects; obtain and track approvals; and communicate changes across teams. At a minimum, if you are using Word documents to define the details of your requirements, create a spreadsheet to track the requirements as well. Using a numbering method and link related information. Example: Req. Req. Name Category Status Priority Dependencies Specifications Complexity No. R-001 Report A Reporting Draft HIGH R-002, R-003 S-001 MEDIUM Requirements Prioritization Here again, there are many techniques for prioritizing requirements and this will also vary based on factors such as the number of stakeholders involved, the volume of requirements, your capacity for development, the maturity of the software and so on. At a minimum, you need to: determine the levels of priority (priority 1, 2, 3 or High, Medium, Low, etc.); the factors to be used to make a priority determination; and the method for determining priority. Keeping it simple will make it easier to make trade-offs down the road, help you streamline the decision making process and prevent unnecessary work. Clearly define what each level of priority means, for example: Requirement Priority Priority Defined High Medium Low A critical requirement; product is not acceptable unless this requirement is satisfied Required eventually but could wait Would be nice to have someday if resources permit There are more analytical methods to ranking requirement priorities based on requirement attributes such as cost, value, risk, etc.you can build a model in Excel or purchase one of numerous tools. First you define the attributes, then using a scale of 1 10 for example, give each attribute a score and then total the score for a given requirement. Requirement Attribute Attribute Attribute Score If there are many stakeholders involved, consider voting as a method of prioritization. Survey tools can help you tally the results from multiple stakeholder. = Page 11

12 Analysis and Design Phase Requirement Complexity A major challenge of any software project is correctly estimating the level of effort across all teams (to define, code, test, document, etc.). The complexity of requirements will obviously directly effect time/cost estimates and release timelines. Consider developing a requirements complexity matrix as defined below rather than trying to calculate the actual level of effort for each requirement. With experience and history, the model will become more fine tuned over time. Using a estimation model will speed the process of making both the initial estimates and on-going adjustments (due to scope changes) faster. It will also allow you to move requirements from one release to another and back again while at the same time providing you the ability to perform what-if type of analysis with a clear understanding of the impact those requirements will have. First define a complexity ranking (HIGH, MEDIUM, LOW). Assign a time estimate to each phase per requirement. Ie: 8 hrs, 1 man day, etc. Requirement Traits Complexity Ranking Phase Time Estimates (hrs) Represents completely new functionality Impacts Multiple Applications/Modules Impacts Multiple Presentations (Form, Report, Data Entry) HIGH Define Requirements: 8 Design: 16 Development: 32 Unit Testing: 16 Integration Testing:16 User Acceptance Testing:16 Documentation: 4 Represents major enhancement to existing functionality Impacts one or more modules Impacts one or more Presentations Requires several variables to implement MEDIUM Requirements: 4 Design: 8 Development: 16 Unit Testing: 16 Integration Testing:16 User Acceptance Testing:16 Documentation: 2 Represents minor enhancement to existing functionality Impacts one module or application Impacts one Presentations LOW Requirements: 2 Design: 4 Development: 8 Unit Testing: 8 Integration Testing:8 User Acceptance Testing:8 Documentation:.5 By multiplying the number of requirements by the time estimates in each phase and then dividing by the number of resources you can calculate an initial rough estimate of time/cost and release timeframe. Page 12

13 Complexity Number of requirements Phase Estimate High 10 Define Requirements: 8 Design: 16 Development: 32 Unit Testing: 16 Integration Testing:16 User Acceptance Testing:16 Documentation: 4 Number of Resources Estimate Hrs Total Hrs for HIGH 400 Adjust estimates based on skill level and experience. Technical Specifications Management If the methodology you have selected includes documenting detailed requirements or technical specifications consider again how to best track and manage your specifications. A TECHNICAL SPECIFICATIONS TEMPLATE CAN BE FOUND IN THE APPENDIX At a minimum your specifications should include: Number of the Specification Name of the Specification Category of the Specification Details of the Specification - include what requirement(s) the specification is addressing Risk Analysis Considering gathering, documenting and tracking the following items as part of your project to be used as a basis for periodic risk assessment. Issues should also be tracked and managed with assignments and resolution timeframes. Page 13

14 Constraints Resources Budget Other Dependencies Other Issues How formal a risk method you use will clearly depend on the complexity of the project, the significance of the project and it s related impact to the business. Testing Phases When developing test plans, map the requirements directly to the test cases. Be sure Use Cases with expected inputs and outputs are clear. During testing phases as your Quality Assurance Team, Developers and Business Users test against these pre-defined test plans, be sure there is clear guidance on how to classify and create a defect (bug/ticket). Defect validation, reproduction and tracing can be extremely difficult and time consuming, especially if the information provided is vague. Without clear guidance and well define defects this will lengthen cycles and timelines to release. Defining Defect Tracking Classification Guidelines Defect Category High - Fatal/Show Stopper Medium - Critical/Major Low - Minor/Cosmetic Defect Category Defined (Traits) Causes system crash or prevention of system/feature use, unrecoverable error, significant incorrect behavior Causes recoverable error, Typo s, Field or Button Placements, color, image or style guide mistake, Defining Defect Creation Guidelines Be sure all testers have a clear understanding of how a defect should be entered. Provide clear guidance on what information is needed to validate, reproduce and trace the defect. Consider requiring: - mapping the test and the defect to the requirement(s). In this was\y it can be traced back to the intended business functionality - that testers add screen shot attachments - outline of click path that lead up to the error - name screens, forms, reports that can be referenced in the ticket Page 14

15 Be sure your teams are clear are what level of quality each testing phase is targeted to achieve. Often a certain number of defects are acceptable at different points of testing. These thresholds should be established and communicated. Defining Quality Criteria Quality Target Level Level 1 Level 2 Level 3 Quality Criteria Acceptable Threshold Zero Defects Remain X Number of Cosmetic/Minor Defects Remain X Number of Critical Defects Remain Page 15

16 Tools To Consider Project Risk Assessment Tool A fairly extensive yet simpl to use and customizable Excel based spreadsheet tool (RapidRiskCheckList) is offered free and can be downloaded at: Requirements Prioritization Tools Low cost product management tools, including feature prioritization and other MS Office templates are offered by StraightForward Tools and can be downloaded at Comprehensive enterprise requirement planning software applications include: Among others Feature Plan, and Accept Survey Tools can be useful in requirement prioritization facilitating stakeholder voting and result reporting. Zoomerang and KeySurvey are good low cost tools for this purpose. Complete SDLC Tools These tools generally offer a complete set of capabilities to manage the entire lifecycle including project management, requirements management, test case management and defect management. Rather than having each team with tools of their own, these tools can significantly improve team collaboration, change management, etc. An affordable, yet very comprehensive collaborative SDLC software application is available from QAVantage at Higher cost enterprise solutions are offered by HP and IBM among others. Defect Management Tools There are numerous defect tracking tools, both freeware and commercial products. This list is just a few of those tools. Bugzilla (freeware) Buggit (freeware) ClearQuest (Part of the Rational Suite from IBM) Defect Tracker (From Pragmatic) Straightforwardtools (Excel Tool) QuickBugs (Excel Software) DevTrack (TechExcel, Inc) TestTrack Pro (From Seapine Software) Testing Tools There are too numerous a collection of testing tools, both freeware and commercial products. A good website for this is Page 16

17 Templates Project Communication Plan Template Project Information Project Name: Project Type: Project Manager: Project Sponsor: Key Business Objectives: Project Scope: 4. Team Members and Stakeholders Organization Role First Name / Last Name Address Contact Number(s) Recurring Meetings Communication Items Participants Frequency Owner Quality Gates and Approvals Quality Gate Approver(s) Timing Owner Page 17

18 General Project Guidelines Template PROJECT TYPE DEFINITION Project Type Type Defined (Traits) Phases By Type REQUIREMENT PRIORITY DEFINITION Requirement Priority Requirement Priority Defined (Traits) REQUIREMENT CATEGORY DEFINITION Requirement Categories REQUIREMENT COMPLEXITY DEFINITION Requirement Complexity Complexity Defined (Traits) Phase Time Estimates Page 18

19 DEFECT CATEGORIZATION Defect Category Defect Category Defined (Traits) TESTING QUALITY GATES Quality Gate Gate Criteria Defined (Traits) Page 19

20 Requirement Document Template Requirement No./Feature Product/Application Target Release R-000/Feature A Product/Application Name/Code Name X.X Author Category Priority High Medium Low Complexity High Medium Low Status Draft In Review Approved Deferred Review and Final Sign-off Stakeholder/Name: Date: Stakeholder/Name: Date: Document Revision History Date Name Notes 00/00/00 Enter Name Comments 1. Source and Problem Statements 1.1. Is this product/application to address a customer(s) need, shortcoming of existing application(s), etc Sub Item under Sub Item under Define the shortcomings of existing applications/lack of application features Sub Item under Sub Item under Requirements Overview 2.1. General description of the product/feature(s) Page 20

21 Sub Item under Sub Item under Purpose of feature(s)/intended user(s) Sub Item under Sub Item under High Level Diagram/Interaction Flow Role A Role B System A A B D C Capability/Area/Function Description A Performer Description/Use Case Input Output B Performer Description/Use Case Page 21

22 Input Output 4. Systems Impacted 4.1. A 4.2. B 4.3. C 5. Dependencies 5.1. A 5.2. B 5.3. C 6. Risks 6.1. A 6.2. B 6.3. C 7. Constraints 7.1. A 7.2. B 7.3. C 8. Documentation Requirements 8.1. A 8.2. B 8.3. C Page 22

23 Detail Design/Technical Specification Document Template Specification No. S-000 Product/Application Product/Application Name/Code Name Target Release Version X.X Mapped to Requirement(s) R-001, R-002 Author Status Draft In Review Approved Deferred User Interface Requirements New Screens New Design New Icons. Review and Final Sign-off Stakeholder/Name: Date: Stakeholder/Name: Date: Document Revision History Date Name Notes 00/00/00 Enter Name Comments 1. Detailed Requirements/Specifications a. Feature/Function Detailed Description Page 23

24 b. Feature/Function Description c. Feature/Function Description d. Feature/Function Description Page 24

25 2. UI Requirements/Terminology Standards a. Terminology Standards b. UI Requirements i. Sub Item under 2.2 ii. Sub Item under 2.2 c. Icons i. Sub Item under 2.2 ii. Sub Item under 2.2 d. Style Guidelines to Follow 3. Expected Volumes/Performance a. Users i. Number Frequent Users ii. Number Occasional Users iii. Response Times b. Transactions i. Volume ii. Processing Times c. Documents/Data 4. Platform Requirements a. Operating System(s) b. Desktop Client OS c. Application Server d. Database 5. International/Localization Requirements a. User Interface i. Date, Time, XXX b. Data Entry/Storage c. Documentation d. On-Line Help Page 25

26 e. Search 6. Data Requirements 7. Integration Requirements Page 26

27 About the Authors Daniele Chenal is the CEO of The Chenal Group and author of StraightForward Tools. With 20 years of experience in the Software Industry and as a former Vice President of Product Management and Marketing, Daniele developed StraightForward Tools in 2006 to bring simple yet effective MS Office template tools to Product Managers and Marketers helping them improve their productivity. Daniele also provides Product Strategy and Product Management consulting services to mid-tier software companies. Daniele can be reach at: Dchenal@thechenalgroup.com. Paul Schwatrz is the CEO of QAVantage. For over 15 years Paul has built and led quality assurance organizations for multiple Fortune 500 companies. QAVantage was formed to bring to market RTIME, a collaborative SDLC software platform developed in 1997 and refined through the course of complex project engagements. Paul can be reached at Paul.Schwartz@QAvantage.com. QAVantage, RTIME, Straightforward Tools, Feature Plan, Zoomerang, KeySurvey, Wikkipedia, MS Excel and other products referenced herein are trademarks of their respective owners. Page 27