A GUIDE FOR ACQUIRING SOFTWARE DEVELOPMENT SERVICES

Transcription

1 A GUIDE FOR ACQUIRING SOFTWARE DEVELOPMENT SERVICES ACQ. SFTWR DEV. SERV--A GUIDE FOR ACQUIRING SOFTWARE DEVELOPMENT SERVICES A GUIDE FOR ACQUIRING SOFTWARE DEVELOPMENT SERVICES EXECUTIVE SUMMARY The General Services Administration (GSA), Information Resources Management Service (IRMS), develops Governmentwide policies and guidance for automated data processing, records, and telecommunications. IRMS' objective is to ensure that Federal agencies acquire, manage, and use Federal information processing (FIP) resources that meet their requirements in an economical and efficient manner. This Guide provides Government program, information resources management, and contracting officials with an introduction to the acquisition of software development services. The Guide, written for readers unfamiliar with the Federal acquisition process for software development services, is not a complete reference work or a regulation and, therefore, directs the reader to other sources for detailed information and guidance. FOREWORD The General Services Administration (GSA) is issuing this Guide to assist Federal agencies in acquiring software development services. It should be used in conjunction with existing regulations. This is one of a series of acquisition guides under development by the Policy Analysis Division of the Information Resources Management Service (IRMS). The Federal Systems Integration and Management Center (FEDSIM) is providing technical assistance in developing the series. The GSA Project Manager was Gloria Sochon, with assistance from Robert Karr. GSA gratefully acknowledges Rex Bolton, U.S. Army, who assisted in the review of the Guide as a member of the Interagency Acquisition Guide Advisory Group, and the following GSA staff: Doug Arai, Diane Herdt, Judy Steele, and Jeff Tucker. To obtain copies of this publication or to be added to the GSA IRM Reference Center mailing list, please use the order request forms provided as the last pages of this document. We welcome your comments regarding this Guide and the Acquisition Guide Series. Please use the "Reader's Comments" sheet at the back of the Guide or contact the Policy Analysis Division at (202) with your suggestions or questions. G. Martin Wagner Acting Commissioner Information Resources Management Service U.S. General Services Administration This Guide was written by American Management Systems, Inc. Arlington, Virginia TABLE OF CONTENTS Page 1. INTRODUCTION THE ACQUISITION GUIDE SERIES a. Background b. Audiences c. Objectives SCOPE OF THE SOFTWARE DEVELOPMENT SERVICES GUIDE a. The Complete Acquisition Life-Cycle b. Limitations ORGANIZATION OF THE SOFTWARE DEVELOPMENT SERVICES GUIDE a. General Information b. Life-Cycle Focus GENERAL INFORMATION DEFINITION OF KEY ACQUISITION TERMS a. Federal Information Processing Resources b. Acquisition c. Requirements Analysis d. Analysis of Alternatives e. Acquisition Strategy WHAT ARE SOFTWARE DEVELOPMENT SERVICES? a. Software (1) Already-existing Software (2) Custom-developed Software b. Scope of Software Development Services Acquisitions (1) Specific Services That Could be Included in a Software Development Services Acquisition (2)

2 Software Development Services Acquisitions vs. Systems Integration Services Acquisitions c. Relation of Software Development Services to Other FIP Resources d. Relation of This Guide to Others in the Acquisition Guide Series ROLES AND RESPONSIBILITIES a. End User b. Program Manager c. Information Resources Management/Technical Representative d. Contracting Officer e. Contracting Officer's Technical Representative KEYS TO SUCCESSFUL ACQUISITIONS a. Define Requirements Functionally b. Obtain Full and Open Competition c. Guard Procurement-Sensitive Information d. Identify, Agree on, and Document Assumptions and Constraints e. Involve All Three Staff Disciplines f. Consider Agency-Specific Requirements or Restrictions g. Begin the Acquisition Process Early PROTESTS SPECIAL CHARACTERISTICS OF SOFTWARE DEVELOPMENT SERVICES HOW SOFTWARE IS DEVELOPED a. Typical Life-Cycle Stages (1) Functional and Data Requirements (2) Design (3) Build (4) Test (5) Implement (6) Operate and Maintain b. Tools Used in the Software Development Process (1) Programming Languages (2) Automated Software Development (3) Prototyping SOFTWARE LIFE-CYCLE MANAGEMENT METHODOLOGIES a. Methodology Defined b. What a Methodology Should Include c. Benefits and Issues of Following a Life-Cycle Methodology d. Should the Agency Specify a Methodology? QUALITY a. Two Measures of Quality b. The Importance of Quality in Software Development Services Acquisitions c. Predicting Quality Before Award d. Ensuring Quality After Award MEASURING PROGRESS INVOLVEMENT OF MULTIPLE CONTRACTORS DURING SOFTWARE DEVELOPMENT SIGNIFICANT CONCERNS IN DEVELOPING SOFTWARE a. Development of Open Systems b. Designing-in Computer Security Throughout the Life-Cycle c. Software License Rights d. Data Administration Considerations e. Emerging Trends in Software Development (1) Changes in the Economics of Software Development (2) Rightsizing (3) Incremental Software Development (4) Interactive Design Techniques (5) Object-Oriented Programming (6) Reengineering (7) Reverse Engineering OVERVIEW OF THE ACQUISITION LIFE-CYCLE PLANNING PHASE a. Identifying Automation Needs b. Planning c. Budgeting ACQUISITION PHASE a. Requirements Analysis b. Analysis of Alternatives c. Developing the Source Selection Plan (SSP) d. Developing the Solicitation e. Issuing the Solicitation f. Receiving Bids or Proposals g. Source Selection (1) Evaluating Bids or Proposals (2) Selecting the Contractor (3) Awarding the Contract IMPLEMENTATION PHASE ACTIVITIES AT THE END OF THE ACQUISITION LIFE- CYCLE REQUIREMENTS ANALYSIS INTRODUCTION KEY FACTORS FOR SUCCESSFUL REQUIREMENTS ANALYSES a. Identify, Agree on, and Document Assumptions and Constraints b. Devote an Appropriate Level of Effort to the Requirements Analysis c. Reassess Requirements Periodically d. Distinguish Between Mandatory Requirements and Optional Capabilities ROLES AND RESPONSIBILITIES a. Program Personnel b. IRM/Technical Personnel c. Contracting Personnel PRELIMINARY STEPS a. Determining Overall Information Needs (1) Determining the Agency's Business Profile (2) Determining What the Current Systems Can Support (3) Defining Information Technology

3 Objectives b. Planning and Budgeting for Software Projects (1) Identifying Software Projects (2) Preparing Plans (3) Budgeting (4) Deciding Whether to Continue TYPES OF REQUIREMENTS TO IDENTIFY a. Software Functional and Performance Requirements (1) Capabilities (2) User Interface (3) Workload (4) Speed (5) Growth (6) Reliability (7) Availability (8) Accessibility by Individuals with Disabilities (9) Control (10) Security and Privacy (11) Training (12) Conversion (13) Documentation (14) User and Operator Support (15) Standards b. Technical Environment Requirements (1) Hardware Environment (2) System Software Environment (3) Telecommunications Environment (4) Integration with Other Systems DOCUMENTING THE REQUIREMENTS ANALYSIS ANALYSIS OF ALTERNATIVES OBJECTIVE a. Analysis of Technical Options b. Analysis of Acquisition Options CHOOSING BETWEEN ALREADY-AVAILABLE AND CUSTOM- DEVELOPED SOFTWARE a. Mandatory-for-Use Programs b. Mandatory-for-Consideration Programs c. Reengineering Existing Agency Software d. Using Another Agency's Software e. Commercial Software f. Custom-Developed Software KEY FACTORS FOR SUCCESSFUL ANALYSES OF ALTERNATIVES a. Base the Analysis on the Agency's Requirements b. Complete the Analysis Before Deciding on the Solution c. Identify, Agree on, and Document Assumptions and Constraints d. Use a Limited Set of Key Criteria to Distinguish among Options e. Do Not Eliminate Options by Dictating a Specific Approach f. Recognize that Requirements and Alternatives Evolve Over Time g. Test the Sensitivity of the Analysis to Changes in Assumptions and Constraints ROLES AND RESPONSIBILITIES a. IRM/Technical Personnel b. Program Personnel c. Contracting Personnel SURVEYING THE MARKET a. Sources of Information (1) Trade Publications (2) Vendor Marketing Literature (3) Other Publications (4) Trade Shows and Exhibitions (5) User Groups and Other Professional Associations (6) Other Government Users (7) Request For Information (RFI) b. What Information to Gather (1) Technical Feasibility (2) Service Availability (3) Degree of Competition Available (4) Pricing Information (5) Industry Support and Contractual Practices ANALYSIS OF TECHNICAL OPTIONS a. Performing Software Development Services On-site or Off-site b. Limiting the Techniques and Tools Used c. Limiting Unilateral Personnel Replacement d. Deciding the Range of Software Development Services e. Choosing Among Technical Options (1) Identifying the Options That Meet All Mandatory Requirements (2) Identifying the Key Discriminators (3) Comparisons f. Documenting the Technical Choice ANALYSIS OF ACQUISITION OPTIONS a. Noncontractual Options (1) Do Nothing (2) Reassign Software Development Services Resources within the Agency (3) Reschedule Software Development Services (4) Share Software Development Services with Another Agency b. Contractual Options c. Choosing Among the Acquisition Options d. Document the Analysis of Acquisition Options DOCUMENTING THE ACQUISITION STRATEGY COMPETITION REQUIREMENTS CONTRACTING UNDER FULL AND OPEN COMPETITION DEGREES OF COMPETITION a. Full and Open Competition b. Full and Open Competition After Exclusion of Sources c. Other Than Full and Open Competition (1) Circumstances Permitting Other Than Full and Open Competition (2) Justification for

4 Other Than Full and Open Competition COMPETITION ADVOCATES REVIEW, APPROVAL, AND AUTHORIZATION PROCESS THRESHOLD LEVELS AGENCY-LEVEL REVIEW AND APPROVAL GSA REVIEW AND AUTHORIZATION GENERAL CONTRACTING CONSIDERATIONS SOUND ACQUISITION PLANNING AND EXECUTION a. Match the Level of Effort to the Size and Scope of the Acquisition b. Maximize Competition c. Develop Clear Specifications and Work Statements d. Avoid Providing Information to Potential Offerors Unless Authorized e. Document and Follow the Source Selection Process f. Involve Each Discipline throughout the Acquisition Process g. Actively Seek Parallel Reviews and Approvals h. Document the Bases for Decisions i. Assign a Team with a Level of Expertise Equal to the Size and Scope of the Acquisition APPLICABLE REGULATIONS AND GUIDANCE ROLES AND RESPONSIBILITIES a. Program Personnel b. IRM/Technical Personnel c. Contracting Personnel IDENTIFYING REQUIREMENTS FOR THE SOFTWARE DEVELOPMENT CONTRACTOR a. Specific Development Services b. Government and Contractor Responsibilities c. Estimated Labor Hours by Personnel Category d. Minimum Personnel and Corporate Qualifications e. Deliverable Items and Delivery Schedule f. Interim Deliverables and Reviews g. Performance Measures (1) Quantifiable Performance Measures (2) Qualitative Software-Specific Measures (3) Compliance with Documentation Standards.9-33 (4) Compliance with Statement of Work (SOW).9-34 h. Review, Approval, and Acceptance Procedures i. Financial Measures (1) Incentives (2) Liquidated Damages (3) Deduction Schedules j. Status Reports and Progress Briefings k. Standards l. Development Methodology m. Technical Development Environment n. Software Maintenance o. Configuration Management p. Site Security q. Personnel Security r. Records Management s. Budget t. Contract Life SELECTING A CONTRACT TYPE a. Contract Types (1) Fixed-Price Contracts (2) Cost-Reimbursement Contracts (3) T M Contracts (4) Incentive Contracts b. Issuing Agencywide Indefinite Delivery Contracts CONTRACTING BY NEGOTIATION ATTRIBUTES OF NEGOTIATED CONTRACTS a. Differences Between Negotiation and Other Contracting Methods b. Contract Attributes THE SOURCE SELECTION PLAN (SSP) a. Source Selection Team Organization b. Determining the Evaluation Factors (1) Price or Cost (2) Technical or Quality c. Source Selection Approaches STEPS IN CONTRACTING BY NEGOTIATION a. Prepare and Issue the Solicitation (1) Write the Solicitation (2) Develop an Independent Government Cost Estimate (IGCE) (3) Obtain Industry Comments on the Draft Solicitation (Optional) (4) Hold a Presolicitation Conference (Optional) (5) Develop Source Selection Materials (6) Publicize the Solicitation in the CBD (7) Issue the Solicitation (8) Hold a Preproposal Conference (Optional) (9) Answer Questions and Amend the Solicitation b. Evaluate Proposals (1) Train the SST (2) Receive Proposals (3) Determine Whether Proposals Comply with Solicitation Instructions (4) Evaluate Proposals Against Minimum Technical Requirements (5) Request Clarification (6) Rate Technical Proposals (7) Conduct Initial Total Cost Evaluation (8) Establish the Competitive Range c. Selecting a Contractor (1) Conduct Discussions and Negotiations (2) Request BAFOs (3) Evaluate BAFOs (4) Recommend the Apparent Winner (5) Conduct Responsibility and Legal Reviews (6) Select the Prospective Contractor...

5 10-45 (7) Notify Unsuccessful Offerors (8) Debrief Unsuccessful Offerors OTHER ACQUISITION METHODS ACQUISITION BY SMALL PURCHASE a. Purpose of Small Purchase Procedures b. Using Small Businesses c. Competition (1) Purchases of $2,500 or less (2) Purchases between $2,500 and $25, ACQUISITION FROM ESTABLISHED SOURCES OF SUPPLY a. Benefits of Using Established Sources of Supply b. Available Sources (1) Agencywide Indefinite Delivery Contracts (2) GSA Contracts ACQUISITION BY SEALED BIDDING CONTRACT ADMINISTRATION ROLES AND RESPONSIBILITIES a. Contracting Personnel b. IRM/Technical Personnel c. Program Personnel PREPARING FOR THE CONTRACTOR a. Obtaining and Preparing Space, GFE, and GFI (1) Space (2) Government Furnished Equipment (3) Government Furnished Information b. Preparing Agency Personnel to Work with the Contractor c. Reviewing the Contractor's Project Plan d. Developing a Tracking and Reporting System POST-AWARD CONFERENCE CONTRACT MONITORING a. Communicating with the Contractor b. Enforcing Key Personnel Clauses c. Obtaining Performance (1) Technical Performance (2) Cost Performance (3) Schedule Performance d. Determining Awards and Incentives e. Exercising Contract Options f. Contract Modifications (1) Administrative Changes (2) Requirements Changes g. Warranties h. Quality Assurance (QA) Deduction Schedules i. Claims j. Contract Terminations (1) Reasons for Termination (2) Procedures for Termination k. Contract Closeout l. Contract Reviews and Audits m. Importance of the COTR in Software Contract Monitoring SOFTWARE DEVELOPMENT FUNCTIONAL AND DATA REQUIREMENTS ANALYSIS a. Determine Detailed Functional Requirements b. Determine Detailed Data Requirements DESIGN a. Preliminary Design b. Detailed Design c. Design Review BUILD TESTING a. Ensure Thorough Testing b. Levels of Testing (1) Unit (2) Integration (3) System READINESS REVIEW SOFTWARE TESTING AND ACCEPTANCE SOFTWARE QUALITY ASSURANCE (QA) a. Development Contractor b. Agency c. Support Contractor DETERMINING TEST OBJECTIVES DECIDING WHAT TO TEST TYPES OF TESTING a. Design Testing b. Software Testing CHOOSING AMONG TEST METHODS a. Benchmark Testing b. Black Box Testing c. White Box Testing d. Testing with Fabricated Data e. Testing with Real Data DETERMINING RESOURCES FOR TESTING ACCEPTING THE SOFTWARE AFTER SUCCESSFUL TESTING SOFTWARE IMPLEMENTATION IMPLEMENTATION a. Implementation Steps (1) Pre-installation (2) Installation (3) Post-installation b. Implementation Considerations (1) Cutover Approach (2) Site Preparation (3) Data Conversion (4) Interface Programs (5) Procedural Changes (6) Documentation (7) Preparing the Users (8) User Training (9) Back-up Procedures (10) Security SOFTWARE OPERATION AND MAINTENANCE SOFTWARE OPERATION AND SUPPORT a. Performance Monitoring

6 b. Continued Configuration Management SOFTWARE MAINTENANCE SOFTWARE MODIFICATION/ENHANCEMENT APPENDIX A - GLOSSARY AND ACRONYMS A-1 APPENDIX B - LEGISLATIVE AND REGULATORY ENVIRONMENT..... B-1 APPENDIX C - BIBLIOGRAPHY C-1 APPENDIX D - GSA ASSISTANCE PROGRAMS D-1 APPENDIX E - CONTACTS FOR MORE INFORMATION E-1 APPENDIX F - OFFICE OF FEDERAL PROCUREMENT POLICY (OFPP) POLICY LETTER F-1 LIST OF EXHIBITS Page 3-1 Differences Between Source and Object Code The Acquisition Life-Cycle Overview of the Analysis of Alternatives Process Key Factors for Successful Analyses of Alternatives The Analysis Process Examples of Information Sources for the Software Development Services Market Survey Examples of Costs and Benefits by Type Typical Agency-Level Reviews Acquisition Steps Uniform Contract Format Division of Contract Administration Tasks Typical Contract Administration Responsibilities Sample Agenda Items Sample Activities of Software Implementation Advantages and Disadvantages of Cutover Approaches A GUIDE FOR ACQUIRING SOFTWARE DEVELOPMENT SERVICES CHAPTER 1: INTRODUCTION 1.1 THE ACQUISITION GUIDE SERIES The General Services Administration (GSA) initiated the Acquisition Guide series to help promote effective and efficient acquisition of Federal information processing (FIP) resources. The Policy Analysis Division (KMP) of the Office of Information Resources Management Policy (KM) develops and issues the guides. a. Background The acquisition guides resulted from a survey of Federal agencies by GSA to determine the types of guidance that agencies would find helpful when acquiring FIP resources. In addition to guidance on approaches and techniques, the guides also address laws, regulations, directives, and policies that affect Federal acquisitions. b. Audiences GSA developed the guides for Government professionals who are new to, unfamiliar with, or need to refresh their knowledge of the process for acquiring FIP resources. While other groups may also find them useful, the guides are directed to the three staffs most involved in the acquisition of FIP resources: o Program personnel (users supported directly by the FIP resources being acquired) -- For functional expertise o Information Resource Management (IRM)/Technical personnel -- For technical expertise on FIP resources o Contracting personnel -- For contracting expertise In discussing each phase of the acquisition life-cycle, the guides outline the roles and responsibilities for each of the three staff disciplines. c. Objectives The guides are intended to provide basic advisory and factual information to acquisition team members who are relatively new to or want to reinforce their knowledge of the process. They are not intended to be encyclopedias of acquisition information. Where applicable, they refer to the most commonly

7 applied laws, regulations, and guidance and then direct the reader to other sources for more complete information. 1.2 SCOPE OF THE SOFTWARE DEVELOPMENT SERVICES GUIDE This Guide, entitled A Guide for Acquiring Software Development Services, includes all phases of the acquisition life-cycle, from the first consideration of software as part of the solution to an agency's information need to contract closeout. It addresses acquisitions that focus on software development services. A Guide for Acquiring Systems Integration Services, another guide in this series, covers acquisitions that include other products and services (e.g., hardware, maintenance services) as well as software development services. While this Guide does not describe every activity at every step of the acquisition process, it does provide an overview of these steps and the key success factors for completing them and the overall acquisition successfully. a. The Complete Acquisition Life-Cycle This Guide covers four phases of the acquisition life-cycle process: o Planning -- This phase consists of the early steps of an acquisition, including identifying the agency's information needs, identifying the needs that can be met through software, and budgeting resources for it. o Acquisition -- This phase consists of activities necessary for acquiring software development services, including developing the solicitation. These activities include the requirements analysis, the analysis of alternatives, and other documentation to support the acquisition strategy. The analysis of alternatives examines the choice between software already available (e.g., commercial) and custom-developed. It also examines both technical options (i.e., how the software development services will be performed) and acquisition options (i.e., what source will provide the services, including both contractual and noncontractual). Finally, this phase usually includes the solicitation of proposals or bids, their evaluation, and award of a contract. o Implementation -- This phase consists of tasks from contract award to the point at which the contractor has completed the software development services required under the contract. o End of Life-Cycle -- This phase consists of all the tasks for closing out the contract and making final payment to the contractor. Chapter 4 provides additional information about the acquisition life-cycle, including typical tasks for each phase. b. Limitations This Guide is not a complete source for information about software development services acquisitions. Agencies' requirements are so varied that one guide could not adequately address every situation. In addition, the restrictions that apply to Federal agencies are subject to change. Rather than duplicate the sources that articulate these restrictions, including the Federal Information Resources Management Regulation (FIRMR), the Federal Acquisition Regulation (FAR), the Federal Property Management Regulation (FPMR), Federal Information Processing Standards Publications (FIPS PUBS), Office of Management and Budget (OMB) Circulars and Bulletins, and agency-specific directives, this Guide directs the reader to the appropriate sources for current information. 1.3 ORGANIZATION OF THE SOFTWARE DEVELOPMENT SERVICES GUIDE a. General Information In Chapter 2, the Guide provides the overall context for software development services acquisitions, including the scope of software development services, key terms, roles of each staff discipline, and the key factors for successful acquisitions. Chapter 3 describes special characteristics of software development services.

8 b. Life-Cycle Focus In addition to describing special characteristics of software development services, Chapter 3 provides an overview of the software development life-cycle, as distinguished from the acquisition life-cycle. Chapter 4 provides an overview of the acquisition life-cycle. Chapters 5 through 12 proceed sequentially through the activities to acquire a software development services contractor. Chapter 5 describes the requirements analysis, which defines the agency's software needs in functional and technical terms. Chapter 6 covers the analysis of alternatives, which examines both technical (what to acquire) and acquisition (how to acquire) options. Chapters 7, 8, and 9 provide general guidance for activities important to acquiring software development services by contract. Chapter 9 also describes how to define the requirements for the development process that the contractor must follow. Chapter 10 covers the steps of contracting by negotiation. Chapter 11 addresses other acquisition methods. Finally, Chapter 12 describes contract administration and implementation activities. Chapters 13 through 16 describe the process of developing, implementing, and maintaining the new software. For convenient reference, Appendix A provides a glossary of commonly used terms and acronyms. Appendix B describes the regulatory environment for acquisitions. Appendix C is a bibliography of other publications related to software development services. Appendix D describes GSA assistance programs that may be useful in software development services acquisitions, and Appendix E provides a list of contacts for more information. Appendix F is a policy letter about quality in contracting for services from the Office of Federal Procurement Policy (OFPP). A GUIDE FOR ACQUIRING SOFTWARE DEVELOPMENT SERVICES CHAPTER 2: GENERAL INFORMATION This chapter provides the following important general information about software development services acquisitions: o Definition of key terms used in the Guide o Scope of software development services acquisitions o Roles and responsibilities of Government personnel who participate in the acquisition process o Key actions that will help make software development services acquisitions successful o Protests and ways to avoid them 2.1 DEFINITION OF KEY ACQUISITION TERMS The Federal Acquisition Regulation (FAR) and the Federal Information Resources Management Regulation (FIRMR) define most terms used in an acquisition of Federal information processing (FIP) resources. Definitions of the following key terms are included to provide maximum use of this Guide. a. Federal Information Processing Resources The FIRMR defines FIP resources as automatic data processing equipment (ADPE) and any equipment or interconnected system or subsystems of equipment that is used in the automatic acquisition, storage, manipulation, management, movement, control, display, switching, interchange, transmission, or reception of data or information-- (1) by a Federal agency, or (2) under a contract with a Federal agency which-- (a) requires the use of such equipment, or (b) requires the performance of a service or the furnishing of a product which is performed or produced making significant use of such equipment. Such term includes computers; ancillary equipment; software, firmware, and similar procedures; services, including support services; and related resources as defined by regulations issued by the Administrator of General Services. b. Acquisition According to the FAR, acquisition is: [T]he acquiring by contract with appropriated funds of supplies or services (including construction) by and for the use of the Federal Government through purchase or lease, whether the supplies or services are already in existence or must be created, developed, demonstrated, and evaluated. Acquisition begins at the point when agency needs are established and includes the description of requirements to satisfy agency needs, solicitation and selection of sources, award of contracts, contract financing, contract performance, contract administration, and those

9 technical and management functions directly related to the process of fulfilling agency needs by contract. c. Requirements Analysis A requirements analysis establishes the agency's need for software development support services. It includes the early stages of determining information needs based on the agency's mission and activities, identifying how software can help achieve long-term automation objectives and short-term automation needs, and considering budgeting needs for the software. The initial requirements analysis also forms the basis for later development of detailed objectives and requirements for the software. d. Analysis of Alternatives While the requirements analysis establishes the agency's need for software, the analysis of alternatives establishes how they will be met. This includes choosing between already-existing software and custom-developed software. This analysis helps the agency choose the most advantageous alternative (consisting of technical and acquisition options) for its need. The analysis also examines both noncontractual (e.g., sharing with another agency) and contractual choices. e. Acquisition Strategy The end result of the analysis of alternatives is an acquisition strategy that defines, in detail, the mechanisms and methodology to be used to acquire software development services. It includes decisions about the degree of competition, contract type (including pricing structure), and delivery schedule. 2.2 WHAT ARE SOFTWARE DEVELOPMENT SERVICES? a. Software According to the FIRMR, FIP software means any software, including firmware, specifically designed to make use of and extend the capabilities of FIP equipment. Software was invented in the early days of computers, when equipment was designed to solve specific numerical problems (e.g., artillery ballistics). The equipment could not easily be adapted to solve other problems; therefore, engineers conceived separate categories of "hardware" and "software," so computers could support a wider range of problems. They built general capabilities (e.g., data storage, basic calculation) into the physical hardware and left the solution of specific problems to software, which told the hardware in what order to apply the general capabilities. Thus, software is the set of instructions that tells the hardware what to do. A series of instructions that performs a particular task is called a "program." Software stored in non-volatile memory (i.e., memory that is not erased when the computer is turned off) is called "firmware." Software falls into two broad categories: o System Software -- This directs the operation of the hardware on which the software runs. System software also supports the development and operation of applications software (see below). The basic classes of systems software include-- oo Operating environment -- Operating system, security, job scheduling, transaction processing, and performance monitoring software oo Applications development -- Programming languages, database management systems, text editing, computer-aided software engineering (CASE), debugging tools, and test data generators oo Utility -- Code conversion, copying, disk management, tape management, backup, archiving, recovery, printing, chargeback and billing, and configuration management o Application Software -- This provides the capabilities to meet the software users' computing needs and support their job activities. The basic classes of applications software include-- oo Office Tools -- Word processing, spreadsheets, electronic mail, graphics, and desktop publishing oo Functional -- Software that supports agency missions and activities, including financial management, project management, inventory control, etc. This Guide distinguishes between software that already exists and software that is custom developed to meet the agency's information needs. The Guide covers acquisition of services to develop functional application software to solve a particular information need

10 that cannot be met with existing software. (1) Already-existing Software Software may already exist that can meet the agency's needs. Software packages and tools sold commercially are referred to as commercial software. The FIRMR defines commercial software as software that is available through lease or purchase in the commercial market from a concern representing itself to have ownership of marketing rights in the software. This includes software furnished as part of the FIP system, but separately priced. An operating system would be an example of software that falls into this category. The other category of existing software covers software packages that, while not commercial, were developed by an agency to solve a similar need. Other agencies may be able to utilize this existing software for their own purposes with little or no customization. The acquisition of already-existing software, commercial or otherwise, is outside the scope of this acquisition guide. A Guide for Acquiring Commercial Software, another guide in this series, describes the acquisition of commercial software in detail. (2) Custom-developed Software In certain circumstances, acquiring existing software might not meet the needs of or be most advantageous to an agency. In this case, the agency would develop new application software. The decision to use existing software or develop new software should be based on two factors: (1) the ability of existing software to meet the agency's requirements and (2) the risk tradeoff between customizing existing software and developing new software. A. Ability to Meet Agency Requirements To determine whether existing software can meet its requirements, the agency must first define the functional and performance capabilities the software must have (e.g., the ability to consolidate X million records of size Y during Z hours). The agency must then determine if software packages available in the marketplace or from other agencies can meet its requirements. The software must not only provide all the capabilities that the agency defined, but must also operate in the agency's FIP resources environment (i.e., with the existing or planned hardware and systems software). If commercial software or another agency's software meets the agency's requirements at a reasonable price, it should acquire the existing software rather than develop new software. In some cases, the agency may find it advantageous to modernize its FIP environment (hardware and software) to take advantage of existing software. Even if no specific software package meets all the agency's requirements, it may be possible to customize one so that it meets the requirements. If customization is required, the agency should determine if the degree of modification is within acceptable limits. B. Risks of Customizing Existing Software Before deciding to customize existing software, the agency should assess the risks involved. When a significant portion of an existing software package's capabilities is altered, many of its benefits may be lost or reduced. Such benefits include: o Ease of training, maintenance, and support o Using proven software with an established track record o Compatibility and integration with other existing software o Ability to use standard documentation and training materials o The software developer's warranty against defects Another risk of customization is that the new code may affect the software's performance or introduce errors into the existing code. In addition, later versions of the software may be incompatible with the customized code or require a significant rewrite of it. The agency should weigh these risks and their costs against the risks and costs of customizing existing software to determine which would be safer and cheaper. b. Scope of Software Development Services Acquisitions (1) Specific Services That Could be Included in a Software Development Services Acquisition To make full use of new software, the agency may also require related services, such as training, conversion, software maintenance, and user support. Many software developers offer these additional services. If the agency can obtain them from the same source as the software development services, they could be covered by a single software development services acquisition. (2) Software Development Services Acquisitions vs. Systems Integration Services Acquisitions Systems integration, a widely-used term in both Government and industry, refers to the acquisition of large, complex information systems with integrated support. As discussed in A Guide for Acquiring Systems Integration Services, another guide in this series, a systems integration services (SIS) acquisition has the following key elements: o It is a complete information solution o It includes several types of FIP resources o A single vendor is responsible for the project Because SIS acquisitions are intended to solve information problems, they frequently include custom-developed software as a component.

11 However, they also include other components (e.g., hardware). SIS acquisitions are outside the scope of this guide. However, this guide supplements A Guide for Acquiring Systems Integration Services in the detailed guidance it provides about the software development component of an SIS acquisition. c. Relation of Software Development Services to Other FIP Resources Software development services are one type of FIP support service. FIP support services are defined in FIRMR Part as "any commercial nonpersonal services, including FIP maintenance, used in support of FIP equipment, software, or services." FIRMR Bulletin A-1 lists several examples of FIP support services, including custom software development. d. Relation of This Guide to Others in the Acquisition Guide Series In addition to the commercial software and SIS guides mentioned above, three other guides in this series provide agencies with useful information about acquiring software development services: o A Guide to Acquiring FIP Support Services covers FIP support services as a group. o A Guide to Requirements Analysis and Analysis of Alternatives describes policies and procedures for carrying out these two steps of the FIP resources acquisition process. o A Guide to Performance and Capability Validation (P CV), describes P CVs and their use in acquiring FIP resources, including software. 2.3 ROLES AND RESPONSIBILITIES A successful software development services acquisition requires involvement of program, Information Resource Management (IRM)/technical, and contracting staff. Some responsibilities are specified by regulation. Others, however, may vary from acquisition to acquisition and from agency to agency. For these, the nature and degree of involvement varies with the complexity of the acquisition and the stage of the acquisition life-cycle. A senior official in the agency should designate a team with responsibility for each large, complex software development services acquisition. The team should consist of representatives from each staff discipline (program, IRM/technical, contracting). The team is responsible for ensuring that the acquisition-- o Successfully meets the agency's needs, o Satisfies all legal and regulatory requirements, and o Remains on schedule and within budget. Teamwork is an important factor in successful software development services acquisitions. Although team members have specific assignments, success requires both cooperation and consultation among the three staff disciplines. By conducting their assignments in parallel (e.g., reviewing solicitations) rather than sequentially, they can significantly shorten the acquisition lead time. Most large, complex software development services acquisitions involve six staff roles: End User, Program Manager (PM), IRM/Technical Representative, Contracting Officer (CO), Contracting Officer's Representative (COR), and Contracting Officer's Technical Representative (COTR). Representatives from the three staff disciplines take these roles. a. End User The end users will interact with and use the software on a day-to- day basis. The ultimate success of the acquisition will be judged by how well the software supports the end users' needs, including both system capabilities and ease of use. End users are typically program (i.e., line) staff, although they could be IRM/technical staff for particular applications (e.g., specialized data manipulation utilities). End users should be involved throughout the life-cycle of each major acquisition of software development services, as well as during the development of the actual software. Their input is particularly valuable during the initial requirements analysis phase. They should remain involved, by receiving status updates and serving as advisors, whenever a long time lag develops between the requirements analysis and the contract award.

12 b. Program Manager The agency should designate a PM for all software development services acquisitions. The PM represents the end users throughout the acquisition and is responsible for ensuring that the software meets the agency's long- and short-term needs. Initially, the PM may participate in strategic and tactical planning that leads to the acquisition. The PM will typically complete the analyses and documents (e.g., analysis of alternatives) required by the FIRMR. The PM may also help prepare some of the supporting acquisition documents (e.g., program-related portions of the solicitation). c. Information Resources Management/Technical Representative The IRM/technical organization provides technical expertise to the PM and the CO throughout the acquisition process. One of their key functions is translating end users' requirements for automated support of particular business functions into specific types of FIP resources, such as software development services. As acquisition requirements dictate, the IRM/technical staff may be called upon to assist in: o Defining requirements for software functions, the technical environment in which the software will run, and the process by which it will be developed o Conducting market research o Writing statements of work, specifications, evaluation factors and criteria, and technical material for the solicitation o Performing the technical evaluation of proposals d. Contracting Officer FAR Part 1 places authority and responsibility with agency heads to contract for authorized supplies and services. Agency heads, in turn, delegate to COs the authority to enter into, administer, and terminate contracts. Agency heads (or their designees) issue warrants to COs stating the limits of their authority. However, only GSA can provide agencies with the authority to contract for FIP resources. A delegation of procurement authority (DPA) to the agency's Designated Senior Official (DSO) provides this authority. GSA provides DPAs to DSOs in three ways: o A regulatory DPA with thresholds up to those set forth in the FIRMR o A specific agency DPA with thresholds higher or lower than those provided by the FIRMR o A specific acquisition DPA in response to an Agency Procurement Request (APR) Although the agency's DSO may redelegate GSA's exclusive authority for FIP resources to qualified officials, only a CO may enter into and sign a contract on behalf of the Government; therefore, GSA's procurement authority for FIP resources must be redelegated to the CO. e. Contracting Officer's Technical Representative A CO usually administers several contracts at the same time and generally designates a COTR to handle specific contract administration functions. The COTR usually has technical expertise and may come from either the program or IRM/technical organization. Typically, the COTR serves as a technical liaison between the Government and the contractor. Duties assigned may include monitoring the contractor's technical, schedule, and cost performance against the terms of the contract, approving invoices, and accepting deliverables. The contract must identify the COTR, who is limited to the activities specifically authorized in the contract and identified in writing by the CO. The COTR does not have authorization to change (add, delete, or modify) contract terms, conditions, or requirements, or to take any action that might give this appearance. The CO alone has the authority to make changes. Any changes must be confirmed in writing. A very large, complex contract for software development services may require multiple COTRs. In this case, the CO may appoint a COR as an interface between the COTRs and the CO. The COR is frequently a contract specialist, but may also come from the program or IRM/technical organization. Some agencies use the terms COR and COTR interchangeably. 2.4 KEYS TO SUCCESSFUL ACQUISITIONS Successful acquisitions have several basic characteristics. By following the advice below, agencies can avoid the majority of situations that hamper the acquisition of software development services.

13 a. Define Requirements Functionally Avoid deciding in advance on a specific solution to the agency's software development needs. With the tremendous growth in Government computing over the last decade, program personnel have become sophisticated and knowledgeable about information products and services and about the detailed aspects of systems. Every office develops its own computer "experts" who have their own favorite user interfaces, database management systems (DBMS), programming languages, CASE tools, and development methodologies. Sometimes users will borrow brand names adopted by particular vendors and inappropriately apply them to whole classes of products or services. These factors can lead agencies to think of their software development needs in terms of a particular interface, DBMS, CASE tool, or methodology. Avoid this pitfall, especially for large or complex acquisitions, for the following reasons: o Using brand names, even unintentionally, precludes full and open competition, which is mandated by statute. o An in-house "expert's" particular favorite may not be well suited for less sophisticated users in the organization. o In some cases, real or perceived bias towards a particular vendor's products or services has resulted in contract protests, adverse publicity, and Congressional investigations. Instead of naming a specific interface, DBMS, or other piece of system software, focus on defining the agency's underlying functional needs þ the capabilities that the new software should provide. Chapter 5, Requirements Analysis, describes this functional approach to defining software requirements in detail. Similarly, avoid naming a specific CASE tool or proprietary development methodology, but instead define the agency's needs for quality and efficiency in the development process. Chapter 9, General Contracting Considerations, describes the functional approach to the software development process. Occasionally, it may be appropriate to specify a product, methodology, or tool by name; e.g., when the agency has already established an organizational standard by careful study and competition. However, the agency must justify such action using the procedures discussed in Chapter 7, Competition Requirements. b. Obtain Full and Open Competition As mandated by statute, COs must promote and provide for full and open competition when soliciting offers and awarding contracts. A competitive market serves the best interests of the agencies, the taxpayers, and the vendor community. Agencies' interests are best served because full and open competition provides them with the widest possible offerings to meet their information needs. The taxpayers' interests are best served because full and open competition encourages vendors to offer their best prices. Vendors' interests are best served because their access to Government business is maximized. Although the contracting staff is usually sensitive to the need for full and open competition, program and IRM/technical staffs sometimes overlook its benefits. Therefore, the acquisition team should periodically take time for a deliberate, objective review of the requirements, alternatives, solicitations, source selection plan, and other materials to ensure that they do not unnecessarily limit competition. Although full and open competition is generally in the agency's best interests, occasionally circumstances may arise in which it becomes necessary to restrict competition to a degree. This is discussed in more detail in Chapter 7, Competition Requirements. c. Guard Procurement-Sensitive Information One essential element for maintaining full and open competition is ensuring that all potential offerors have access to the same information about the acquisition. At the same time, offerors do not need access to all information about the acquisition. The agency must withhold some information from offerors, such as the Government's independent cost estimate and the source selection materials, in order not to influence their proposals. No offeror should be given information about the number or identity of other offerors or the nature of their technical offerings or cost. An offeror with access to information that others lack has an unfair advantage. This offeror could use the information to change its proposal and win the contract, even though another offeror's proposal would have provided more value to the Government. Agency personnel need to exercise special caution if contractor personnel are working on site at the agency. Most often the disclosure of information happens accidentally: an overheard remark, a report lying open on a desk, a slip during a phone conversation. However, laws

14 prohibit the release of procurement-sensitive information (e.g., the Procurement Integrity Act, Trade Secrets Act). The release of such information has sometimes led to protests, scandal, adverse publicity, Congressional investigations, and criminal and civil penalties. d. Identify, Agree on, and Document Assumptions and Constraints Identify explicitly assumptions used in the analyses supporting the acquisition. Assumptions can be both quantitative (e.g., the contract life will be five years) and qualitative (e.g., the agency will need to lease extra office space before awarding a contract for on-site software development services). Program, IRM/technical, and contracting staff should agree that the assumptions reflect the agency's best estimate. Document assumptions so they can be discussed, analyzed, and modified if necessary. Also, test the sensitivity of the analyses and conclusions to changes in the assumptions. Finally, document the constraints that affect the software development services acquisition. These include any policy, staffing, organizational, or budgetary factors that limit the agency's discretion. For example, the agency's need to monitor the software development process closely may be constrained by lack of space for on-site contractor personnel. Documenting the constraints and their impacts on the acquisition is essential. e. Involve All Three Staff Disciplines Chapter 1, Introduction, points out that this Guide was written for three audiences: program, IRM/technical, and contracting personnel. All three staff disciplines bring knowledge and skills to the acquisition process. The program staff understands the requirements to be met. The IRM/technical staff helps translate these requirements into the software development services to be acquired. They do this because they are familiar with the types of software development services available in the marketplace. The contracting staff provides liaison with the vendor community and is generally more aware of any legislative and regulatory constraints affecting the acquisition. The CO manages the contractual aspects of the acquisition. Although one group may have the lead responsibility for a particular step in the acquisition life-cycle, it should seek the advice and counsel of the other two during that step. In addition, each group should be kept informed of the status of the acquisition throughout its life-cycle. Sharing information will help keep the acquisition running smoothly and identify any potential problems or constraints early, when they are easier to handle. Parallel involvement and review can also shorten the acquisition lead time. f. Consider Agency-Specific Requirements or Restrictions This Guide is written for Governmentwide use. It describes legislative and regulatory constraints and acquisition procedures that apply to every agency. In addition, an agency may have its own specific documentation requirements, review and approval process, or contracting mechanisms (e.g., an agencywide requirements-type contract already in place). Review and understand the agency's internal procedures and situation before acting on the advice and guidance provided in this Guide. g. Begin the Acquisition Process Early The acquisition process for any type of FIP resources may be a lengthy one because of the legislative and regulatory framework in which it operates. To protect the interests of the taxpayers, the Government, and industry, agencies must fully analyze and document their requirements. They must weigh and document the benefits and costs of alternatives. They must follow review and approval procedures -- both internal and external to the agency. They must solicit bids or proposals, give offerors time to respond, and evaluate the bids or proposals carefully. If an offeror files a protest, it generally must be resolved before the contract is awarded or the software development services can be provided. Each of these steps takes time. Even in an acquisition of software development services for a small system, months may elapse between the initial determination of the need and the acquisition of the software development services. For large negotiated acquisitions of software development services to support mission-critical systems, the elapsed time could be 18 to 36 months. An agency also must have funds available to acquire the software development services. The normal budget lead time is two

15 years. Take these lead times into account in planning the acquisition. The length of time an acquisition requires often surprises program and IRM/technical staff. An agency can minimize frustration by beginning the process early, before the need for the software development services or the resulting new system becomes acute, and anticipating the documentation needs for future acquisition activities during the early stages of the process. 2.5 PROTESTS Protests challenge contract awards or other actions of agencies on specific acquisitions. The number of protests has increased dramatically since the passage of the Competition in Contracting Act (CICA). Protests occur when an interested party challenges the agency's solicitation document, objects to a contract award or proposed award, or perceives a procedural violation during the acquisition process. Many protests assert that an interested party has been or will be wrongfully excluded from consideration for award. Protests are usually filed with the agency, the General Accounting Office (GAO), or the General Services Board of Contract Appeals (GSBCA). While the procedures of and remedies available from each of these organizations differ, a successful protest could result in one or more of the following: o Suspending, revoking, or revising the agency's procurement authority o Modifying or terminating the contract o Withholding exercise of options under the contract o Asking for new Best and Final Offers (BAFOs) or reopening negotiations o Amending the solicitation or issuing a new one o Awarding protest costs, proposal preparation costs, and attorney's fees o Awarding the contract to the protestor These are serious consequences and may prevent the agency from acquiring or using the software development services until the protest is resolved. Agencies can avoid many protests by anticipating the potential impact of decisions made while developing and executing the acquisition. To conduct successful acquisitions, the agency should: o Eliminate unnecessarily restrictive specifications o State the agency's requirements clearly and exactly o Develop fair and meaningful evaluation factors and apply them during source selection o Provide all potential offerors with the same information o Develop and follow an explicit source selection plan o Document decisions made throughout the acquisition life- cycle, particularly those restricting competition o Follow all pertinent Government and agency laws, regulations, and procedures o Review past GSBCA and GAO protest rulings before completing the FIRMR studies and source selection plan A GUIDE FOR ACQUIRING SOFTWARE DEVELOPMENT SERVICES CHAPTER 3: SPECIAL CHARACTERISTICS OF SOFTWARE DEVELOPMENT SERVICES Acquisitions of software development services, commercial software, and Federal information processing (FIP) support services are similar in a number of ways. Two other guides in this series, A Guide for Acquiring Commercial Software and A Guide for Acquiring FIP Support Services, discuss the latter commodities. However, each commodity also has unique characteristics that must be identified and addressed to conduct a successful acquisition. This chapter discusses some unique characteristics of software development services acquisitions that program, Information Resources Management (IRM)/technical, and contracting personnel should know about. However, these discussions are overviews, not comprehensive treatises. A full, detailed discussion of all aspects of software development is beyond the scope of this Guide. For this reason, IRM/technical and/or program staff experts in software development should always be involved in an acquisition of software development services. This chapter assumes that the Government will acquire the needed software development services through contracting. If an agency uses another acquisition method (e.g., a GSA assistance program), the Government developer would typically assume contractor responsibilities. 3.1 HOW SOFTWARE IS DEVELOPED Software development is still largely a labor-intensive process. Although automated tools and thousands of books and training courses are available to assist design, programming, and testing, the process requires craftsmanship by individuals or small teams of developers. Standardization in

16 software design has not evolved to the degree that it has in hardware design, and software cannot be manufactured on an assembly line. This section provides an overview of the typical steps, techniques, and tools used to develop software. The combination will vary among development projects, depending on an agency's internal development standards and the size and complexity of the software system. a. Typical Life-Cycle Stages Most software goes through several identifiable stages from initial requirements to operational system. Sometimes these stages are explicitly defined and followed rigorously, especially for large systems that need the efforts of many people. For smaller systems, some of the stages may be extremely short, very informal, or not conducted at all. Some or all of the stages may be conducted by agency personnel, some or all by a contractor. The names of the stages, their duration, the allocation of development activities among them, and the organization responsible for carrying them out vary for different lifecycle management methodologies and for different projects. The explanation presented in this Guide describes only one of many possibilities, but it represents a common division of stages. Chapters provide details on each stage. (1) Functional and Data Requirements In the functional and data requirements stage, the agency and/or contractor defines in detail the system requirements, including both the automated software and related manual activities/procedures. However, this is not the first time the agency has defined requirements. It is an iterative process. Before this stage, during overall FIP resources planning, the agency developed a high- level view of its information needs. After the agency identified specific software initiatives, it developed more detailed requirements for the capabilities the software must have, the workload it must support, etc. However, during these initial requirements analysis activities, described in Chapter 5, the agency knows only that it needs software. It has not yet analyzed whether commercial software can meet its needs or whether it requires customdeveloped software. At the beginning of the development life-cycle, the agency or the contractor defines the requirements for the software's capabilities and features at much greater levels of detail. Agency program and IRM/technical staff must be deeply involved in identifying detailed requirements to ensure the software meets the agency's and users' needs. The two primary classes of requirements are: o Functional -- These describe aspects such as: oo The flows of data to, from, and within the software oo The sources or recipients of the data oo How the software manipulates or transforms the data oo The workload volumes for the data flows oo Performance requirements (e.g., response times) for specific functional processes oo The data contained in user interface/data entry screens and output reports, potentially including their physical layout The functional requirements can be described by narrative, graphics (e.g., data flow or process flow diagrams, structured flowcharts or matrices), or a combination. o Data -- These requirements describe the data the system will maintain. Data models may be developed and the data normalized to eliminate unnecessary elements. Often, the developer creates a data dictionary, defining each element in terms of name, attributes (e.g., alphanumeric, logical), length, format, permissible values, ownership (i.e., who can create the data element, who can modify it), and so forth. The functional and data requirements may be called by other names, especially if graphical representations (e.g., data flow diagrams) divide the software into logical modules. Different methodologies call this step the system concept, conceptual design, or logical design. Before moving to the next stage, the agency ensures that the technical environment (i.e., hardware, systems software, telecommunications) has been specified for both the development and the operation of the software. This should occur during the requirements analysis described in Chapters 5 and 9, well before the development contract is awarded and the development process begins. (2) Design After the agency or the contractor establishes the functional and data requirements, the contractor knows "what" the software must do. In the design stage, the contractor defines "how" the software will do it. Frequently, more than one solution is available, so the contractor must consider several alternative physical methods for implementing the functions and managing the data, then choose the one that will most effectively meet the contract requirements. At this point in the development life-cycle, the software moves from logical to physical structures (programs and data files). The contractor defines programs to carry out specific system functions and divides the data into data stores, files, or data bases. In presenting the design graphically, the contractor can use different representations (e.g., data

17 or process flow diagrams, flowcharts). Several methodologies refer to this first portion of the design stage as the general design. During the next portion, detailed design, the contractor writes specifications for the programs and data files, providing the level of detail programmers need for coding. The specifications can be represented in several different ways, depending on the life-cycle methodology used [e.g., hierarchical input-process-output (HIPO) charts, process action diagrams, Structured English, pseudo code, Warnier-Orr diagrams]. Computer-Aided Software Engineering (CASE) tools can automate much of both the general and detailed design activities, including graphic representations, narrative, and pseudo coding. The agency's primary responsibilities typically lie in the review of written design documents, which may also include graphical representations, and plans (e.g., Test Plan). The design methodology should include periodic review by users to ensure that the emerging design continues to meet agency needs. To assist this review, the agency may want to construct a manual or automated Requirements Traceability Matrix (RTM) that maps functional requirements to physical "configuration items," such as programs or databases. Then, whenever a physical software component is reviewed, audited, or tested during the development life-cycle, the agency can use the RTM to identify the related functional requirements. (3) Build During the build stage, programmers write or generate the software code (i.e., instructions the computer can understand). These instructions control the various hardware, systems software, and telecommunications components. The programmers usually develop the software components in sections from smallest to largest: first program units, then modules (groups of units) supporting major processes, then interfaces among modules and with external systems. The programs are debugged (i.e., errors are detected and removed) and documented. Finally, the entire software system is made ready for testing. The agency's primary activities during this stage include conducting regular audits, walkthroughs, and reviews of build stage products (e.g., code walkthroughs), as well as monitoring status and test reports submitted by the contractor. Different methodologies may refer to this stage as construction, coding, or programming. (4) Test Testing takes place in two major stages. First, the development contractor conducts its own tests of the software. Some life-cycle methodologies consider this testing part of the build stage. The contractor conducts unit tests, integration tests, and system tests, correcting errors as they surface. The contractor may also conduct regression testing, ensuring that correcting a particular part of the software does not cause problems in other parts. The contractor should conduct testing in accordance with a Test Plan that the agency reviewed and approved. The agency monitors the contractor's testing process and results, ensuring that the contractor conducts thorough testing consistent with the approved Test Plan and corrects any errors. In the second stage of testing, the agency takes a much more active role, conducting its own system tests to determine whether the software meets the contract requirements and can be accepted for production use. The agency may duplicate some of the contractor's tests using its own test data. The agency may also validate performance with benchmarking tests. (5) Implement The implementation stage includes preinstallation activities (e.g., site preparation, user training), installation of the software in the agency's hardware/systems software environment, and post-installation activities such as converting data from the old software systems to the new. If the software is installed at multiple sites, the implementation activities may take place in repeated waves. Depending on the tasks defined in the contract, the primary responsibility for software implementation may lie with the contractor, the agency, or a combination of the two. (6) Operate and Maintain When the software has been successfully implemented at all sites, and users and operators trained to use it, the system moves into the operation and maintenance stage. The software is now stable, with few changes except to correct lingering errors or to improve performance (i.e., tuning). The software may also require modification if the underlying hardware, systems software, or telecommunications environment changes. As new requirements develop, the software may be enhanced to add new functions, features, or capabilities. Software operation typically falls under the agency's responsibility. Maintenance may become the responsibility of the development contractor, if required by the contract. Alternatively, agency personnel may assume maintenance responsibility. The decision will be based on many factors, such as the availability of trained personnel within the agency, other sources of maintenance services (e.g., established sources of supply or a separate contract), for this system or a group of systems.

18 b. Tools Used in the Software Development Process (1) Programming Languages Programming languages are the primary tools of software development. Using these languages, a programmer writes code telling the hardware and systems software what to do. Dozens of programming languages exist, some proprietary for a specific product or platform, others in wider use. A. Terminology Agency personnel involved in acquisitions of software development services need to understand some of the terms used to describe programming languages: o Source code -- Instructions that the programmer writes. They must be translated into a form understandable by the machine (i.e., the hardware and operating system) before they can be executed. o Object code -- Instructions translated into machine language. For the hardware to execute the instructions, a further translation into binary code is required. o Compilers, assemblers, and interpreters -- Programs that translate the source code into object code. Compilers typically convert large sections of code at a time, saving it for later execution. Interpreters translate program commands one at a time during execution of the program. Compilers may generate assembly language, an intermediate level, first, then translate it into machine language. Figure 3-1 illustrates the difference between source code, assembly language, machine language, and machine binary code. For some particularly technical software functions, a few programmers may need to write assembly language directly. However, in most cases, programmers can rely on the "higher level" source code languages. Differences Between Source and Object Code B. Third and Fourth Generation Languages Since invention of the computer a half-century ago, programming languages have gone through several generations. Currently, programmers use third and fourth generation languages. The mostly widely used third generation languages (also known as procedural languages) include BASIC, COBOL, C, FORTRAN, PL/1, and PASCAL, which have fairly rigid structure and syntax. They are most useful for large, batch-oriented systems or systems with many users [where efficiency and central processing unit (CPU) utilization are important], or where many files are interacting or complex algorithms are used. Fourth generation languages (4GLs) provide much more flexibility, potentially reducing dramatically the time needed for coding. However, no 4GL has yet achieved the wide use of COBOL or other popular third generation languages. Many are proprietary, linked to a particular database management system (DBMS) environment. In fact, one of the primary advantages they offer over third generation languages is their powerful data query capabilities. Some 4GLs also include prototyping capabilities, permitting faster design and development of the user interface portions of the software. (2) Automated Software Development Until the last decade, software development was a largely manual process, with only a few automated tools (e.g., text editors, debuggers) to support system designers and programmers. However, recently the software industry has produced a large number of automated tools supporting every stage of the development process. While these tools offer the opportunity for greater productivity and faster development and implementation of systems, many organizations find that their greatest value is in improving the quality of the software developed and, in the more rigorous documentation of the software, permitting easier maintenance and reuse. A. Computer-Aided Software Engineering CASE tools cover a wide range of software designer and programmer productivity aids. In its broadest sense, CASE is commercial software that aids in the analysis, design, development, testing, and maintenance of computer programs and their documentation. The CASE tool captures plans, models, and designs and stores the information in a central database, repository, or encyclopedia. CASE tools are divided into four major categories. o Upper CASE -- These tools support the front end development processes, such as planning, analysis, requirements, and design. o Lower CASE -- These tools support the back end development processes, such as building the physical databases and coding the application programs that access them. o Integrated CASE (I-CASE) -- These tools, typically provided by a single vendor, include both the Upper and Lower CASE components, providing a complete development environment. o Component CASE (C-CASE) -- These tools consist of Upper and Lower CASE components from multiple vendors integrated to provide a complete development environment. Unlike I-CASE tools, which are often proprietary, C-CASE offers an open system development capability and the ability to match the capabilities of a particular CASE component to the agency's requirements. Upper CASE tools are currently the most mature and provide the greatest functionality. Many products support specific Lower CASE functions, but fewer tools support all back end development processes in a consistent, well-integrated manner. I-CASE and C- CASE tools offer

19 promise and are evolving quickly, but no product yet offers complete automation of the development process. CASE tools work best teamed with a software life-cycle management methodology. Some CASE tools are designed to work best with specific methodologies. CASE tools can include the following components: o Project management support o Table and matrix manipulation tools to support strategic and tactical planning o Graphics tools for design elements such as data flow diagrams, models, and process action diagrams o Prototyping tools for creating user interface screens and reports and for modeling or simulating system processes o Requirements traceability tools to track agency and user requirements to particular software modules or programs o A data dictionary o A central repository for all design information o Design analyzers (e.g., for finding outputs without related inputs, for normalizing data) o Automatic code generators o Testing tools Although CASE tools offer productivity savings and the opportunity to implement systems more quickly, many organizations derive their biggest benefits from improving the quality of the developed systems and ensuring the completeness of their documentation. The growth of I-CASE and C-CASE tools also offers the promise of easy portability of software among platforms. Upper CASE components need to capture the requirements and design information only once. Then, to move from one platform to another, the agency would regenerate the application using the Lower CASE components for the new target platform. B. Code Generation Code generators automatically create program code, data definition language, and utility control language directly from detailed design information or program specifications. While usually part of an I-CASE or C-CASE system, independent code generators also exist. The code generator toolkit can produce both source code and, using compilers and assemblers, object code. The toolkit usually also includes editing, debugging, and unit testing tools. C. Code Analysis Another class of tools analyze the code after its development. The tools can identify both over-used portions of code that cause bottlenecks and portions not used at all. Code profilers monitor the code while it is running, determining which portions execute most often, take the longest to run, or both. Programmers can use the results to restructure the code so that it runs more efficiently. Profilers are most useful for analyzing code developed by large numbers of people working on large software systems whose efforts may not have been tightly coordinated or controlled. Profilers are also useful for advanced software using object-oriented techniques or graphical user interfaces (GUIs). The code for such systems may become too complex for a single person's full understanding. Another class of code analyzers review the code in static mode (i.e., while not running), looking for "go to" statements, use of subroutines, etc., then represent the results graphically. In general, well organized, well-written code executes faster and is easier to maintain than "spaghetti" code (i.e., poorly organized and inefficient code). Static code analyzers can also review code for adherence to development standards, such as naming conventions and formats. Finally, code analyzers can produce "metrics" that monitor and manage the development process. Chapter 9 describes metrics in greater detail. D. Configuration Management Configuration management is the process of controlling changes to parts of the software and related items (e.g., databases, documentation) so that they stay consistent and do not get placed in production before they are ready. It is a complex undertaking, especially for large and complex software systems. A number of automated tools are available for keeping an inventory of the various portions of the software, allowing change control managers to define libraries, conditions for check-in and check-out, and so forth. The tools can also help assess the magnitude and impacts of a proposed change. E. Testing When software systems were simpler and smaller, programmers could hand test most or all of their code. However, today's complex systems, which incorporate elements needing more lines of code (e.g., GUIs), require the type of complex testing that only automated tools can reasonably accomplish. A number of these tools are available to support the testing process: o Project management -- This software can schedule tests, manage testing resources, and monitor progress against the test plan. o Data capture and replay -- Useful for "black box" testing to focus on inputs and outputs rather than the internal algorithms of a program. These tools capture keystrokes for test scripts and the programs' outputs to establish baselines. The tools can then replay the data to run the tests, comparing actual versus expected results and reporting on data mismatches, error counts, and response time. o Profilers -- Using code profiling tools, testers can determine which portions of the software code tests have exercised. o Interactive testing/debugging -- These tools are specifically designed for the interim tests used as a part of good programming practice. Programmers can insert breakpoints, display data names as they are accessed, and record execution paths of programs and subroutines. o Test data

20 generators -- Particularly useful for large software systems and for performance testing, these tools automatically generate large numbers of "dummy" records with the required layouts and characteristics. o Test case generators -- These tools, which are relatively new, identify and design test cases based on design specifications. o Traceability tools -- These tools map the functional and data requirements to specific software components (e.g., programs) for tracking compliance with requirements during testing. An RTM is an example of such a tool. F. Documentation A key factor in the successful support and maintenance of software is thorough documentation. Not only must manuals be developed for users and operators, but the software itself must be documented in terms of purpose, design, source code, logic, etc. Good programming practices dictate that some of the code be self- documenting through insertion of comment statements and explanations. A number of automated tools may provide support for system documentation. Improvements in word processors and text editors make it easier for programmers to write, edit, debug, modify, and add documentation statements to their code. Also, CASE tools can capture much of the requirements, general, and detailed design information in electronic form as it is created. (3) Prototyping A key factor in the successful development and implementation of software is that it meets the needs of its users. However, program personnel, managers, and sometimes even IRM/technical personnel have a difficult time envisioning how software will look and operate by reviewing design documentation. Thus, prototyping (i.e., creating a skeletal, but operational, model of the software or one or more of its elements) is gaining a wide following in systems development. By showing the prototype to users and managers and then varying the elements based on their comments, developers can accelerate the requirements and general design tasks. The growth of prototyping has been facilitated by the transition from traditional batch systems to more interactive ones and by the wide availability of CASE tools, many of which have prototyping tools built in. The prototype can include one or many software components. The developer could prototype just data entry screens, screen dialogues, GUI elements, internal processes, or data content. Alternatively, these and other elements could be integrated into a model of the overall software system. Finally, the developers could build a "proof of concept" prototype showing that a particular combination of custom-developed and commercial off-the- shelf (COTS) tools can work together. Two different approaches apply to prototyping: "throw-away" and "evolve." In both approaches, developers mock-up the software. Users and others then exercise the mock-up as though it were the operational system. They feed back to the developers any needed changes. The developers change and users re-exercise the mock-up. Through an iterative process, the developer modifies the prototype until it meets the users' requirements. In the throw-away approach, the prototype software is not implemented. Instead, it serves merely as the specification to build software with more traditional methods and tools. The prototype software is thrown away after the system becomes operational. One advantage of throw-away prototyping is that the prototype and the production version need not be developed on the same hardware and systems software platform. The developers can use special prototyping software (sometimes included as part of a CASE tool) to develop the prototype quickly and easily. In the evolutionary approach, the prototype is gradually enhanced and modified (e.g., adding internal and batch processing) to become the production system. One concept currently emerging, Rapid Application Development (RAD), uses the evolutionary approach. RAD combines prototyping techniques and extensive user interaction to shorten the development period dramatically. The advantage of the evolutionary approach is that the system usually gets into users' hands more quickly than by following a standard development methodology. For that reason, it is an increasingly popular part of software development projects. However, prototyping also has the following potential disadvantages: o Because the prototype design focuses on user interface elements and is thus incomplete, it may result in a highly inefficient system. o Design documentation may not be as thorough, making later maintenance more difficult. o Program staff could originally advocate a throwaway prototype, then want to keep it as an evolutionary system. This not only raises the potential issues listed above, it may also subvert the contract scope (e.g., a requirements analysis contractor might actually perform software development by calling it prototyping). While none of these is a "fatal flaw," an agency should focus on mitigating such problems if it plans to use prototyping. The agency should ensure that the selected prototyping method results in comprehensive and detailed documentation. Proper use of CASE, C-CASE, and I-CASE tools can assist in producing efficient, well- documented systems that are easy to maintain.