16 CRITICAL SOFTWARE PRACTICES FOR PERFORMANCE-BASED MANAGEMENT

Transcription

1 16 CRITICAL SOFTWARE PRACTICES FOR PERFORMANCE-BASED MANAGEMENT PURPOSE PROJECT INTEGRITY 1. ADOPT CONTINUOUS PROGRAM RISK MANAGEMENT ESTIMATE COST AND SCHEDULE EMPIRICALLY USE METRICS TO MANAGE TRACK EARNED VALUE TRACK DEFECTS AGAINST QUALITY TARGETS TREAT PEOPLE AS THE MOST IMPORTANT RESOURCE CONSTRUCTION INTEGRITY 7. ADOPT LIFE CYCLE CONFIGURATION MANAGEMENT MANAGE AND TRACE REQUIREMENTS USE SYSTEM-BASED SOFTWARE DESIGN ENSURE DATA AND DATABASE INTEROPERABILITY DEFINE AND CONTROL INTERFACES DESIGN TWICE, CODE ONCE ASSESS REUSE RISKS AND COSTS PRODUCT STABILITY AND INTEGRITY 14. INSPECT REQUIREMENTS AND DESIGN MANAGE TESTING AS A CONTINUOUS PROCESS COMPILE AND SMOKE TEST FREQUENTLY f 34 16cspv5-2.dc

2 Purpse This paper utlines the 16 Critical Sftware Practices that serve as the basis fr implementing effective perfrmance-based management f sftware-intensive prjects. They are intended t be used by prgrams desiring t implement effective highleverage practices t imprve their bttm-line measures time t fielding, quality, cst, predictability, and custmer satisfactin and are fr CIOs, PMs, spnsring agencies, sftware prject managers, and thers invlved in sftware engineering. The 16 Critical Sftware Practices fr Perfrmance-based Management and Templates cntain the 16 practices (9 best and 7 sustaining) that are the key t aviding significant prblems fr sftware develpment prjects. These practices have been gathered frm the crucible f real-wrld, large-scale, sftware develpment and maintenance prjects. Tgether they cnstitute a set f high-leverage disciplines that are fcused n imprving a prject s bttm line. This dcument is intended t define the essential ingredients f each best and sustaining practice. These practices are the starting pint fr structuring and deplying an effective prcess fr managing large-scale sftware develpment and maintenance. They may be tailred t the particular culture, envirnment, and prgram phases f a prgram. Of curse, these practices cannt help death march prgrams that are expected t deliver under impssible schedule deadlines with inadequate funding and withut the required staffing with essential skills. The 16 practices are mre effective and efficient when there is an rganizatinal infrastructure that facilitates their implementatin. This infrastructure termed the rganizatinal verlay, addresses imprtant aspects such as rganizatinal cmmitment and training. The rganizatinal verlay will be addressed separately. 2 f 34 16cspv5-2.dc

3 PROJECT INTEGRITY 1. Adpt Cntinuus Prgram Risk Management Practice Essentials: 1. Risk management is a cntinuus prcess beginning with the definitin f the cncept and ending with system retirement. 2. Risk management is a prgram respnsibility impacting n and supprted by all rganizatinal elements. 3. All prgrams need t assign a risk fficer as a fcal pint fr risk management and maintain a reserve t enable and fund risk mitigatin. 4. Risk need t be identified and managed acrss the life f the prgram. 5. All risks identified shuld be analyzed, priritized by impact and likelihd f ccurrence and tracked thrugh an autmated risk management tl. 6. High-pririty risks need t be reprted t management n a frequent and regular basis. 7. The culture f all stakehlders must supprt the risk prcess (reprting, incentives, award fee plan, fee distributin, resurces). 8. Fr medium and high risks, a risk cntrl prfile shuld be develped. 9. The risk identificatin prcess shuld be a well-planned, well-dcumented, and tracked prcess. Metrics shuld be used t determine if the risk prcess is identifying risks and if these risks are being used in decisin-making. Implementatin Guidelines: 1. Risk management shuld cmmence prir t cntract award and shall be a factr in the award prcess. 2. The DEVELOPER needs t establish and implement a prject Risk Management Plan that, at a minimum, defines hw pints 3 thrugh 8 will be implemented. The plan and infrastructure (tls, rganizatinal assignments, and management prcedures) will be agreed t by the ACQUIRER and the DEVELOPER and need t be placed under cnfiguratin management (CM). 3. DEVELOPER and ACQUIRER senir management shuld establish reprting mechanisms and emplyee incentives in which all members f the prject staff are encuraged t identify risks and ptential prblems and are rewarded when risks and ptential prblems are identified early. The ACQUIRER needs t address risk management explicitly in its cntract award fee plan, and the DEVELOPER needs t prvide fr the direct distributin t all emplyees in furtherance f establishing and maintaining a risk culture. 4. Risk identificatin shuld be accmplished in facilitated meetings attended by prject persnnel mst familiar with the area fr which risks are being identified. A persn familiar with prblems frm similar prjects in this area in the past shuld participate in these meetings when pssible. Risk identificatin shuld include risks thrughut the life cycle in at least the areas f cst, schedule, technical, staffing, external dependencies, supprtability, and maintainability and shuld include rganizatinal and prgrammatic plitical risks. Risk identificatin need t be updated at least mnthly. Identified risks shuld be characterized in terms f their likelihd f ccurrence and the impact f their ccurrence. Risk mitigatin activities need t be included in the prject s task activity netwrk. 5. Bth the DEVELOPER and the ACQUIRER shuld designate and assign senir members f the technical staff as risk fficers t reprt directly t their respective prgram managers and shuld charter this rle with independent identificatin and management f risks acrss the prgram and grant the authrity needed t carry ut this respnsibility. 6. Each medium-impact and high-impact risk shuld be described by a cmplete Risk Cntrl Prfile. 3 f 34 16cspv5-2.dc

4 7. Peridically updated estimates f the cst and schedule at cmpletin shuld include prbable csts and schedule impact due t risk items that have nt yet been reslved. 8. The DEVELOPER and ACQUIRER risk fficers need t update the risk data and database n the schedule defined in the Risk Management Plan. All risks intended fr mitigatin and any thers that are n the critical path and their status against the mitigatin strategy shuld be summarized. Newly identified risks shuld g thrugh the same prcesses as the riginally identified risks. 9. Risk management shuld cmmence prir t cntract award and be cnsidered fr inclusin as a factr in the award prcess. 10. Learn frm the mistakes f thers tday s prblem is tmrrw s risk. One cmmn Risk area is subcntractr management. Anther cmmn risk area is inadequate planning. A significant risk in prjects is miscmmunicatin. Anther Risk area is lack f fcus. 11. Schedule risk-reslutin tasks with sufficient slack t implement a wrkarund. 12. The supplier/develper shuld prpse, establish, and implement a prject Risk Management Plan that, at a minimum, defines hw the next six pints will be implemented. The plan and infrastructure (tls, rganizatinal assignments, and management prcedures) shuld be agreed t between the acquirer and the supplier/develper and placed under cnfiguratin management (CM). a. Supplier/develper and acquirer senir management shuld establish reprting mechanisms and emplyee incentives in which all members f the prject staff are encuraged t identify risks and ptential prblems and are rewarded when risks and ptential prblems are identified early. The acquirer shuld address risk management explicitly in its cntract award fee plan when risk is a majr r significant cnsideratin, and the supplier/develper shuld prvide fr the direct distributin f all nn-trivial risk data t all emplyees in furtherance f establishing and maintaining a risk management culture. b. Risk identificatin shuld be updated at least mnthly. c. Risk mitigatin activities shuld be included in the prject's task activity netwrk. d. Bth the supplier/develper and the acquirer shuld designate a senir member f the technical staff as a risk fficer t reprt directly t his/her respective prgram manager and charter this rle with independent identificatin and management f risks acrss the prgram and grant the authrity needed t carry ut this respnsibility. e. Each medium-impact and high-impact risk will be described by a cmplete Risk Cntrl Prfile, which shuld include fur basic items: definitin f risk index, which is impact likelihd; transitin scenari, which includes initiating key pivtal events and likely risk ccurrence; ptential mitigatin strategies; and key actin triggers tied t metrics that initiate varius mitigatin activities. f. The supplier/develper and acquirer risk fficers will update the risk data and the integrated supplier/develper database n the schedule defined in the Risk Management Plan. All risks intended fr mitigatin and any thers that are n the critical path and their status against the mitigatin strategy will be summarized and reprted, and an agreement will be reached n planned actin and mitigatin. Newly identified risks shuld g thrugh the same prcesses as the riginally identified risks. g. One f the mst significant risks a prject faces is miscmmunicatin. Effective planning is a necessary step fr minimizing this risk. At the highest level this starts with a clearly defined, crdinated and apprved statement f wrk (SOW). The SOW sets the prject framewrk by identifying the scpe f wrk, gals and bjectives (implementatin, quality, maintenance), identifying users and peratinal envirnment, standards that must be fllwed, prject cst and schedule cnstraints, dependencies n utside agencies, and resurces. The SOW becmes a key surce dcument in establishing high level prgram risks. 13. Planning is a critical step in the prject prcess because it utlines and rganizes the activities needed fr success. One cmmn risk is inadequate planning. T minimize this risk, there shuld be a dcumented prcedure fr develping a plan. 4 f 34 16cspv5-2.dc

5 Items t be addressed by the prcedure include the parties invlved in planning, the types f plans needed, the frm and frmat, and the plan cntents. Guidance fr plan cntents typically includes the fact that the plan shuld be based n and cnfrm t custmer standards and needs, prject standards and needs, the SOW, and prject requirements. Planning is cnducted thrughut the life f the prject frm acquisitin/slicitatin thrugh system remval. Planning shuld include all direct and supprt activities needed (e,g QA, CM,). 14. Management must remain actively invlved in the prject. Peridically the prject shuld be reviewed with senir management (f all participants). Items t be cvered include technical/cst/schedule perfrmance, staffing, Issues, Risks. These reviews shuld be frmal with minutes and actin items. The PM shuld hld similar. mre frequent reviews, based n prgram milestnes r events. All participants shuld participate. Prgress shuld be tracked against the SOW and requirements. Dependencies shuld be fcused n. Issues shuld be addressed. Risks shuld be addressed. This review als shuld be frmal with minutes taken and actin items tracked. Management reviews are cnducted thrughut the life f the prject, frm acquisitin thrugh remval. 15. When a sub is invlved, they shuld fllw the same versight, and management principles as the prime, thugh the scpe may be scaled. 16. One cmmn Risk area is subcntractr management. T minimize risks in the area the fllwing shuld be accmplished: A subcntract SOW is prepared/reviewed/agreed upn revised when needed, and that a plan t select the cntractr is prepared cncurrently with the subcntract SOW. Further, the sftware subcntractr shuld be selected based upn an evaluatin f the subcntractr's ability t d the wrk. Selectin shuld fllw a dcumented prcedure cvers the evaluatin f prpsals submitted, prir perfrmance n similar wrk, lcatin f the sub relative t the prime, sftware engineering and management capabilities, staff available, prir experience, and available resurces (facilities, hardware, sftware, training). 17. One cmmn Risk area is subcntractr management. T minimize risks in the area the fllwing shuld be accmplished: The agreement between the prime and the sub shuld be the basis fr managing the cntract (SOW, Terms and Cnditins, Requirements fr the prducts t be prduced, Dependencies between the sub and the prime, prducts t be delivered, cnditins under which revisins t prducts will be submitted, acceptance prcedures and criteria, and prcedures t be used fr mnitring the sub). The sub shuld prduce an SDP fr prime review and apprval. The plan shuld cver the apprpriate items frm the prime's SDP. 18. Planning is a critical step in the prject prcess because it utlines and rganizes the activities needed fr success. One cmmn risk is inadequate planning. T minimize this risk, the planning prcess shuld crdinate dependent activities. It shuld als address supprt f the sftware team in ther supprt activities and delineate budgets/resurces. The plan shuld be reviewed and apprved by all prgram participants at a senir level; The SDP which results shuld cver all activities needed t achieve prject success. Items t be included are: Prject purpse, scpe, gals, and bjectives, Prject life cycle, Prject prcedures, methds and standards, Sftware prducts t be develped (including interim), Size estimates f prducts, Estimates f prject cst and effrt, Estimates fr use f critical assets, Milestnes and schedules, Identificatin and assessment f sftware risks and the risk prcess t be fllwed, and plans fr necessary tls, facilities and training. 19. Planning is a critical step in the prject prcess because it utlines and rganizes the activities needed fr success. One cmmn risk is inadequate planning. T minimize this risk, Planning fr supprt activities (e.g. SQA) shuld be as rigrus as ther activities. Fr SQA, there shuld be a plan that cvers the respnsibilities and authrity f SQA, identifies the SQA resurces, Identify the funding fr SQA, Identify SQA's rle in establishing an SDP, standards and prcedures, identifies evaluatins t be accmplished, identifies audits t be perfrmed, identify the prcess and standards t be fllwed in reviews/audits, Identify the prcedures t be used fr recrding and tracking prblems, Identify the recrds SWA must keep, and identify the frequency/methd f prviding feedback t sftware engineering grups. 20. A significant risk in prjects is miscmmunicatin. A dcumented plan (SDP, etc) is an excellent cmmunicatin tl. The plan prvides baseline schedule, cntractual cmmitments, assignment f respnsibilities, crdinates activities between grups, shuld be available t all team members, reflects inter-grup cmmitments and dependencies, is kept current, and is reviewed by all team members and apprved by the PM. 21. The prject shuld have a mechanism fr identifying, negtiating, and tracking dependencies between rganizatins. Each dependency shuld be identified, each shuld be negtiated, need dates and delivery frmats shuld the reslved, agreements shuld be dcumented and apprved by all parties. When grups cannt reslve an issue, there shuld be a dcumented prcess fr reslving (e.g. elevate t managers) 5 f 34 16cspv5-2.dc

6 22. Prject planning includes develping and dcumenting, in the SDP, prject sftware prcesses tailred frm the rganizatin's standard. The prcess accunts fr all life cycle phases, and cntractual waivers. Prcess plans are reviewed by Quality and management and apprved by senir management. Changes t the sftware prcess are managed and cntrlled with reviews and management apprvals. 2. Estimate Cst and Schedule Empirically Practice Essentials: 1. Initial sftware estimates and schedules shuld be lked n as high risk due t the lack f definitive infrmatin available at the time they are defined. 2. The estimates and schedules shuld be refined as mre infrmatin becmes available. 3. At every majr prgram review, csts-t-cmplete and rescheduling shuld be presented t identify deviatins frm the riginal cst and schedule baselines and t anticipate the likelihd f cst and schedule risks ccurring. 4. All estimates shuld be validated using a cst mdel, a sanity check shuld be cnducted cmparing prjected resurce requirements, and schedule cmmitments shuld be made. 5. Every task within a wrk breakdwn structure (WBS) level needs t have an assciated cst estimate and schedule. These tasks shuld be tracked using earned value. 6. All csts estimates and schedules need t be apprved prir t the start f any wrk. 7. A sftware size estimate shuld be an input t the cst estimate. The size estimate shuld include an estimate f the equivalent new develpment lines f surce cde fr planned legacy reuse, cmmercial-ff-the-shelf (COTS) and gvernment-ff-the-shelf (GOTS) sftware. 8. When COTS applicatin sftware is invlved, an analysis f the fit f the COTS t the current applicatin shuld be made prir t cst estimatin. 9. The schedule shuld nt be unreasnably cmpressed. If experience shws that similar prjects have taken a given length f time, the new prject shuld nt be expected t cmplete in less than 85 percent f that time. 10. Schedules and cst estimates are influenced by a number f factrs, nt all f which are under a prgram manager's cntrl. D nt expect an estimate t be extremely accurate. A gd estimatin technique will be within 20 percent fr cst and 5 percent fr schedule. Implementatin Guidelines: 1. Estimate the cst, effrt, and schedule fr a prject fr planning purpses and as a yardstick fr measuring perfrmance (tracking). Sftware size and cst need t be estimated prir t beginning wrk n any incremental release. 2. Sftware cst estimatin shuld be a recnciliatin between a tp-dwn estimate (based n an empirical mdel; e.g., parametric, cst) and a bttm-up engineering estimate. 3. Sftware cst estimatin shuld als be subjected t a sanity check by cmparing it with industry nrms and specifically with the DEVELOPER s past perfrmance in areas such as prductivity and percentage f ttal cst in varius functins and prject phases. 4. All f the sftware csts need t be assciated with the apprpriate lwer-level sftware tasks in the prject activity netwrk. Allcate the estimated ttal prject labr effrt amng all the tasks in the activity netwrk 5. Prductivity estimate shuld be a functin f sftware size and type f sftware: Real-time, Engineering systems, Business systems. 6. Prductivity estimate shuld be based n prductivity metrics frm past prjects f the perfrming rganizatin using the planned prcesses and technlgy. 7. Prductivity estimate shuld be reduced fr new peple hired int the rganizatin. 6 f 34 16cspv5-2.dc

7 8. Estimate shuld be adjusted fr experience base (fr example: new applicatin -- n valid cmparisn available, n experts with experience in applicatin). 9. Estimate the cst f cde reuse by first identifying the specific cde mdules t be reused and then estimating the percentage f the reuse cde in the applicatin that is new r mdified. 10. Explicitly identify all surce cde mdules t be reused and estimate the amunt f cde mdificatin and new cde t write t integrate each identified mdule. 11. Estimating cst and schedule shuld be cnsistent with rganizatinal plicy and prcedures. Likewise the evaluatin f cst and schedule estimates fr sftware acquisitin shuld be cnsistent with rganizatinal plicy and prcedures. 12. Subject cst estimate t sanity-check rules f thumb: percentage f ttal cst fr individual phases and functins, prductivity cmputed frm size and labr effrt estimates, and cst estimate fr the effrt t mdify and integrate reuse cde, industry nrms. 13. Need system perspective (hardware interfaces, sensitivity t peratinal requirements) 14. There shuld be a dcumented prcedure fr estimating csts and schedules t ensure cnsistency. This prcedure shuld be adapted ver time based n the experience frm the prject. The prcedure shuld call fr estimatin f detailed wrk packages, based n histrical data, dcumenting assumptins and review/apprval f estimates. 15. There shuld be a dcumented prcedure fr estimating use f critical assets. The prcedure shuld call fr assets t be identified, the estimates t be based n the sftware prducts t be used, traffic and the peratins cncept. The estimates shuld be reviewed and agreed t by all parties at the system level. 16. Measurements shuld be made fr all tasks t determine cst and schedule status. This includes supprt tasks such as SQA and SCM. 3. Use Metrics t Manage Practice Essentials: 1. All prgrams shuld have in place a metrics prgram t mnitr issues and determine the likelihd f risks ccurring. 2. Metrics shuld be defined as part f definitin f prcess, identificatin f risks r issues, r determinatin f prject success factrs. 3. All metrics definitin need t include descriptin, quantitative bunds, and expected areas f applicatin 4. All prgrams need t assign an rganizatinal respnsibility fr identificatin, cllectin, analysis, and reprting f metrics thrughut the prgram s life 5. Metrics infrmatin shuld be used as ne f the primary inputs fr prgram decisins. The metrics prgram needs t be cntinuus. 6. Early warning metrics frm all rganizatins invlved in the prgram shuld be reprted t the buyer's PM in mnthly status reprt. 7. Early warning metrics reprted t prject managers shuld be based n data current t at least ne week befre the date the reprt is submitted. 8. At a minimum, metrics shuld measure and track schedules, budgets, prductivity, defects, quality, and risk. In additin t this basic set, metrics shuld be added, as necessary, t mnitr specific prject cncerns. 7 f 34 16cspv5-2.dc

8 Implementatin Guidelines: 1. Every prject shuld have a prject plan with a detail activity netwrk that defines the prcess the team will fllw, rganizes and crdinates the wrk, and estimates and allcates cst and schedule amng tasks. The plan shuld be brad enugh t include each sub-prcess/phase. The prject plan needs t include adequate measurement in each f these five categries. a. Early indicatins f prblems, b. Quality f the prducts, c. Effectiveness f the prcesses, d. Cnfrmance t the prcess, and e. Prvisin f a basis fr future estimatin f cst, quality, and schedule. 2. Metrics shuld be sufficiently brad based. Data that prvides insight shuld be cllected fr each prcess, sub-prcess and phase. 3. T use these metrics effectively, threshlds need t be established fr these metrics. These threshlds shuld be estimated initially using suggested industry nrms fr varius prject classes. Lcal threshlds will evlve ver time, based upn experience (see 1.e abve). Vilatin f a threshld value shuld trigger further analysis and decisin making. 4. Cntinuus data n schedule, risks, libraries, effrt expenditures, and ther measures f prgress shuld be available t all prject persnnel alng with the latest revisin f prject plans. 5. Prcess-effectiveness metrics shuld include defect remval efficiency, percentage f develpment cst due t rewrk, defect leakage thrugh inspectins, initial size estimate and size f delivered sftware, initial estimates f changes t reuse/cots/gots cde and actual changes made 6. Cllect and reprt at least the early warning metrics frm the SPMN Cntrl Panel. 7. Quality metrics shuld be established at prgram inceptin and include cde cmplexity and delivered defect density. 8. Cllect and reprt the number f defects fund in each frmal inspectin and leakage f defects thrugh inspectins. 9. Early-warning metrics reprted t prject managers shuld be based n data current t at least ne week befre the date the reprt is submitted. 10. Metrics data shuld be easily available t all persns n the develpment/sustainment team. 11. The metrics selected t be a prject s main indicatrs f health r dysfunctin shuld prvide prgram wide visibility int the prject s status and an accurate cmparisn f prgress against the plan. 12. At a minimum, metrics shuld measure and track schedules, budgets, prductivity, defects, quality, and risk 13. In additin t this basic set, metrics shuld be added, as necessary, t mnitr specific prject cncerns. 14. Cntinuus data n schedule, risks, libraries, effrt expenditures, and ther measures f prgress will be available t all prject persnnel (supplier/develper and acquirer) alng with the latest revisin f prject plans. 15. Data t supprt future cst estimatin shuld include: Actual cst Actual prductivity Actual size Percentage f ttal cde that is reuse cde Percentage f initial reuse cde SLOC that is mdified r added cde t allw reuse in the new applicatin Type f sftware (IS, engineering, real-time, hard real-time) Technical and management methds and prcesses fllwed 8 f 34 16cspv5-2.dc

9 Average mnthly requirements and staff vlatility 4. Track Earned Value Practice Essentials: 1. Earned value prject management requires a wrk breakdwn structure, wrk packages, activity netwrks at every WBS level, accurate estimates, and implementatin f a cnsistent and planned prcess. 2. Earned value requires each task t have bth entry and exit criteria and a step t validate that these criteria have been met prir t the award f the credit. 3. Earned value credit is binary with zer percent being given befre task cmpletin and 100 percent when cmpletin is validated. 4. Earned value metrics need t be cllected n a frequent and regular basis cnsistent with the reprting cycle required with the WBS level. (At the lwest level f the wrk package, the earned value reprting shuld never be less frequent than 2 weeks). 5. Earned value, and the assciated budgets schedules, and WBS elements need t be re-planned whenever material changes t the prgram structure are required (e.g., requirements, grwth, budget changes, schedule issues, rganizatinal change). 6. Earned value is an essential indicatr and shuld be used as an essential metric by the risk management prcess. 7. When defects are fixed r new requirements implemented n the prduct f a task fr which earned value credit has been given, this wrk shuld be in a new rewrk task. Implementatin Guidelines: 1. Prgress twards prducing the prducts shuld be measured within the designated cst and schedule allcatins. 2. THE DEVELOPER shuld develp and maintain a hierarchical task activity netwrk based n allcated requirements that includes the tasks fr all effrt that will be charged t the prgram. All level f effrt (LOE) tasks need t have measurable milestnes. All tasks that are nt LOE shuld explicitly identify the prducts prduced by the task and have explicit and measurable exit criteria based n these prducts. 3. N task shuld have a budget r planned calendar time duratin that is greater than the cst and schedule uncertainty that is acceptable fr the prgram. The gal fr task duratin is n lnger than tw calendar weeks f effrt. 4. Each task that cnsumes resurces needs t have a cst budget allcated t it and the crrespnding staff and ther resurces that will cnsume this budget. Staff resurces shuld be defined by persn hurs r days fr each labr categry wrking n the task. 5. Fr each identified significant risk item, a specific risk mitigatin/reslutin task shuld be defined and inserted int the activity netwrk. 6. The cst reprting system fr the ttal prject needs t segregate the sftware effrt int sftware tasks s that the sftware effrt can be tracked separately frm the nn-sftware tasks. 7. Milestnes fr all external dependencies shuld be included in the activity netwrk. 8. Earned value metrics need t be cllected fr each schedule level and be made available t all members f the DEVELOPER and gvernment prject teams mnthly. These metrics are: a cmparisn f Budgeted Cst f Wrk Scheduled (BCWS), Budgeted Cst f Wrk Perfrmed (BCWP), and Actual Cst f Wrk Perfrmed (ACWP). A 9 f 34 16cspv5-2.dc

10 cmparisn f BCWP and ACWP, a Cst Perfrmance Index, a Schedule Perfrmance Index, and a T-Cmplete Cst Perfrmance Index. 9. The lwest-level schedules shuld be statused weekly. 10. The high-level schedules shuld be statused at least mnthly. 11. Earned value reprts shuld be based n data that is n mre than tw weeks ld. 12. Dcument, track, and manage critical path and schedule dependencies accrding t a defined prcess. 13. Structure the hierarchical activity netwrk t rll up charges int cst accunts. 14. Try t structure the hierarchical activity netwrk s that task predecessr/successr relatinships are nly between child tasks f the same parent task. Have always been able t d this Makes activity netwrk less cmplex Prvides structure fr repeatable prcesses Write a descriptin fr each task in the activity netwrk that includes: Identificatin f the task input Identificatin f the task prduct(s) Identificatin f the methds t be fllwed t cnvert task input t utput prducts Unambiguus definitin f task exit criteria 15. Enter the initial activity netwrk and resurces cnsumed during the prject int an autmated prject management tl. A spreadsheet is nt an adequate prject management tl. 16. All tasks/activities shuld be tracked and metrics gathered t determine task effectiveness. This requires bth data cllectin and analysis 17. Task entry and exit criteria shuld be defined fr all tasks 18. Resurces shuld be allcated t each task (labr hurs by labr categry, resurces ther than labr) 19. Enter the initial activity netwrk and resurces cnsumed during the prject int an autmated prject management tl (a spreadsheet is nt an adequate prject management tl) 20. Manage yur resurces actively 21. Develp accurate resurce prjectins 22. Critical path and schedule dependencies are tracked, dcumented and managed accrding t a defined prcess. 23. Prgress Twards Prducing Prducts Measured: Cst v. Schedule 24. Develp a hierarchical Task Activity net 25. Earned Value Metrics fr each schedule level & made available mnthly f: BCWS, BCWP and ACWP CPI, SPI (Schedule CPI), T-Cmplete Perfrmance Index (TCPI), which is BAC - BCWP/EAC - ACWP SPI (Schedule perfrmance index) TCCP 26. Use EV Reprting data the is less than tw weeks ld 27. Segregate Sftware Tasks t have greater visibility 10 f 34 16cspv5-2.dc

11 5. Track Defects against Quality Targets Practice Essentials: 1. All prgrams need t have pre-negtiated quality targets, which is an abslute requirement t be met prir t acceptance by the custmer. 2. Prgrams shuld implement practices t find defects early in the prcess and as clse in time t creatin f the defect as pssible and shuld manage this defect rate against the quality target. 3. Metrics need t be cllected as a result f the practices used t mnitr defects, which will indicate the number f defects, defect leakage, and defect remval efficiency. 4. Quality targets need t be redefined and renegtiated as essential prgram cnditins change r custmer requirements are mdified. 5. Cmpliance with quality targets shuld be reprted t custmers n a frequent and regular basis, alng with an identificatin f the risk assciated with meeting these targets at delivery. 6. Meeting quality targets shuld be a subject at every majr prgram review. 7. Prduct quality gals fr delivered prducts shuld be unambiguusly defined at prject inceptin. Understandability Mdularity Defect density May have different gals fr different types f applicatins May have different gals fr different parts f the same applicatin 8. Surce cde cyclmatic cmplexity < 15 fr cde units shuld be ne understandability gal. 9. Task prducts shuld be inspected against task exit criteria and defects fixed r frmally deferred befre earned value credit is given fr the task. 10. Frm prject inceptin Defects shuld include defects in requirements. Classify defects by level f severity. Unambiguus definitins f severity levels at prject inceptin Rules abut when fixes must be made fr each f the severity levels 11. Custmer, prject managers, and supplier/develper executives achieve a culture that rewards, nt penalizes, finding defects when made. 12. Defect metrics shuld be cllected and peridically reprted, including: Number f defects fund in each inspectin and each test Fr each defect When each defect was created When each defect was fund The number f inspectins in which the defect was present but nt fund The number f defects clsed and currently pen by categry f defect: requirements, architecture, detailed design, cde, and dcumentatin. 11 f 34 16cspv5-2.dc

12 13. CM shuld manage defects. All verified defect reprts turned ver t CM CM maintains status f defect clsure CM perfrms versin cntrl f prducts in the develpmental and peratinal baselines 14. Baseline f custmer-accepted deliverables is made. 15. Develpmental baseline fr stability f prducts shared amng the develpment team is made. 16. Treat tls as a subset f CM (addressed in CMM) 17. Autmated tls slve prject prblems mre efficiently than manual techniques. Implementatin Guidelines: 1. The ACQUIRER and the DEVELOPER need t establish quality targets fr subsystem sftware depending n its requirements fr high integrity. A missin-critical/safety-critical system may have different quality targets fr each subsystem cmpnent. System Quality Assurance needs t mnitr quality targets and reprt defects as per the Quality Plan. 2. Quality targets can be under change cntrl and established at the design, cding, integratin, test, and peratinal levels. 3. Quality targets shuld address the number f defects by pririty and by their fix rate. 4. Actual quality r defects detected and remved shuld be tracked against the quality targets. 5. Peridic estimates f the cst and schedule at cmpletin shuld be based n the actual versus targeted quality. 6. Use an empirical cst mdel, such as SLIM, peridically during develpment t prject latent defects and estimate rewrk remaining t prject cmpletin based n defects fund t date. 7. Defect remval efficiency fr develpment is cmputed as the rati-number f defects fund frm prject inceptin t delivery: number f defects fund frm prject inceptin thrugh the first year 6. Treat Peple-as the Mst Imprtant Resurce Practice Essentials: 1. A primary prgram fcus shuld be staffing psitins with qualified persnnel and retaining this staff thrugh the life f the prject. 2. The prgram shuld nt implement practices (e.g., excessive unpaid vertime) that will frce vluntary staff turnver. 3. The staff shuld be rewarded fr perfrmance against expectatins and prgram requirements. 4. Prfessinal grwth pprtunities such as training shuld be made available t the staff. 5. All staff members need t be prvided facilities, tls, and wrk areas adequate t allw efficient and prductive perfrmance f their respnsibilities. 6. The effectiveness and mrale f the staff shuld be a factr in rewarding management. 7. Cmpensate key sftware staff as well as cmpetitrs fr the same level f skill. 8. Persnal Satisfactin 9. Prvide Adequate Resurces 12 f 34 16cspv5-2.dc

13 Implementatin Guidelines: 1. DEVELOPER senir management needs t wrk t ensure that all prjects maintain a high degree f persnnel satisfactin and team chesin and shuld identify and implement practices designed t achieve high levels f staff retentin as measured by industry standards. The DEVELOPER shuld emply fcus grups and surveys t assess emplyee perceptins and suggestins fr change. 2. DEVELOPER senir management shuld prvide the prject with adequate staff, supprted by facilities and tls t develp the sftware system efficiently. Emplyee fcus grups and surveys shuld be used t assess this adequacy. 3. The training f DEVELOPER and ACQUIRER persnnel shuld include training accrding t a prject training plan in all the prcesses, develpment and management tls, and methds specified in the sftware develpment plan. 4. The DEVELOPER and the ACQUIRER shuld determine the existing skills f all systems, sftware, and management persnnel and prvide training, accrding t the needs f each rle, in the prcesses, develpment and management tls, and methds specified in the Sftware Develpment Plan (SDP) 5. Training is an essential element in recruiting and retentin f qualified emplyees. It is als essential t maintaining a high level f prductivity. Imprtant training cnsideratins are: The training f supplier/develper and acquirer persnnel will be accrding t a prject-training plan in all the prcesses, standards, develpment and management tls, and methds specified in the Sftware Develpment Plan (SDP). Sft skills such as teamwrk shuld nt be verlked. The supplier/develper and the acquirer will determine the existing skills f all systems, sftware, and management persnnel and will prvide training t the supplier/develper staff and acquirer staff invlved in mnitring the prject activities, accrding t the needs f each rle, in the prcesses, develpment and management tls, and methds specified in the SDP. Qualified individuals r rganizatins will prvide training. Training prgrams will be cnsistent with rganizatinal and persnal bjectives and plans by determining training needs, identifying methds and surces f training, and prviding waivers when knwledge and skills exist. Training recrds will be kept cncerning curse cmpletin vs. individual training plans and prject requirements. In an effrt t prvide quality training, students, instructrs, and management will review all curses. The training prgram will be independently reviewed n a peridic basis t ensure cnsistency, relevance. When different rganizatins must wrk tgether, tls, training and prcedures shuld be cmpatible. 6. Exceptinally gd perfrmers shuld receive much larger raises, bnuses, and better prmtins than pr perfrmers. 7. Senir management shuld cntinuusly survey industry t identify persnnel practices that are successful fr hiring and retaining peple with sftware skills fr which market demand is much greater than supply. Fr example: Telecmmuting fr mthers f yung children Summer jbs fr cllege sftware majrs 8. Cmpanies shuld pay fr training in sftware skills t career-transitin nn-sftware emplyees int sftware jbs. Underemplyed engineers shuld be ne surce. 9. Training per training plan in categries specified in SDP: Sft skills such as teamwrk shuld nt be verlked. Team members, especially leaders, shuld receive rientatin training in the specific methds, prcesses, and standards used by ther grups they must cperate with and wrk with in rder t better understand hw t develp prcesses and prcedures that are cmpatible and supprtive. The training plan shuld utline al activities needed t realize training, including, resurces, prcedures fr identifying needs, apprvals, and alternative training vehicles. The prject training plan shuld feed an rganizatinal plan, which manages and cntrls the training prgram as a whle t meet rganizatinal gals as efficiently as pssible, The rganizatinal plan shuld prvide mechanism fr prject input, review, crdinatin and plan changes. 10. Determine existing skill sets & prvide needed training 11. Make prfessinal grwth pprtunities such as training available t the staff. 13 f 34 16cspv5-2.dc

14 12. All staff members shuld be prvided facilities, tls, equipment, and wrk areas adequate t allw efficient and prductive perfrmance f their respnsibilities. 13. Cmpanies shuld pay fr training in sftware skills t career-transitin nn-sftware emplyees int sftware jbs (underemplyed engineers shuld be ne surce) 14. Prvide staff with the tls t be efficient and prductive (sftware, equipment, facilities, wrk areas) 15. The training f supplier/develper and acquirer persnnel will be accrding t a prject training plan in all the prcesses, develpment and management tls, and methds specified in the Sftware Develpment Plan (SDP). 16. The supplier/develper and the acquirer will determine the existing skills f all systems, sftware, and management persnnel and will prvide training t the supplier/develper staff and acquirer staff invlved in mnitring the prject activities, accrding t the needs f each rle, in the prcesses, develpment and management tls, and methds specified in the SDP. 17. Dmain expertise and sftware engineering experience are the valuable assets. In staffing a team, maximize these characteristics. Organize s that team members with expertise and experience can mst readily identity prblems and elevate them fr crrectin. 18. It is critical that Management put smene in charge. A sftware prject manager with the necessary backgrund and skill shuld be identified and assigned respnsibility fr building an effective team, develping a sftware develpment plan and prcess that is cnsistent with the systems develpment prcess and successfully delivering the prject. Management shuld prvide the PM with the assets needed t succeed as well as clear, bjective measures f success. The PM shuld als have the authrity necessary t cmmit the rganizatin. 19. It is critical that staff members have visibility int their accmplishments and the impact it has n the prject. The planning prcess shuld result in a detailed prgram wrk breakdwn structure that identifies clear, measurable bjectives and respnsibility fr each task. Each task shuld be traceable t the verall prject bjectives. Task managers/teams shuld be held accuntable fr their assigned, agreed t tasks. 20. Training will be prvided by qualified individuals r rganizatins. 21. Training prgrams will be cnsistent with rganizatinal and persnal bjectives and plans by determining training needs, identifying methds and surces f training, and prviding waivers when knwledge and skills exist. 22. Training recrds will be kept cncerning curse cmpletin vs. individual training plans and prject requirements. 23. In an effrt t prvide quality training, all curses will be reviewed by students, instructrs, and management. The training prgram will be independently reviewed n a peridic basis t ensure cnsistency, relevance. 24. When different rganizatins must wrk tgether, tls, training and prcedures shuld be cmpatible. 25. Each rganizatin shuld have a training plan that addresses lng and shrt range training needs based n the rganizatins needs and bjectives. The plan shuld address what training is needed and when, hw training will be btained (internal/external), funding and resurces, standards fr instructinal materials, schedules fr in-huse develped materials, training schedules and prcedures fr selecting peple t be trained, registering, maintaining recrds, evaluatin and feedback. Fr in-huse materials, there shuld be a curse descriptin, a dcumented review prcess and a prcess fr cntrl/managing materials. 26. Cmpanies shuld pay fr training in sftware skills t career-transitin nn-sftware emplyees int sftware jbs (underemplyed engineers shuld be ne surce) 27. Make prfessinal grwth pprtunities such as training available t the staff.. 14 f 34 16cspv5-2.dc

15 CONSTRUCTION INTEGRITY 7. Adpt Life Cycle Cnfiguratin Management Practice Essentials: 1. All prgrams, irrespective f size, need t manage infrmatin thrugh a preplanned cnfiguratin management (CM) prcess. 2. CM has tw aspects: frmal CM, which manages custmer-apprved baseline infrmatin, and develpment CM, which manages shared infrmatin nt yet apprved by the custmer. 3. Bth frmal and develpment CM shuld uniquely identify managed infrmatin, cntrl changes t this infrmatin thrugh a structure f bards, prvide status f all infrmatin either under cntrl r released frm CM, and cnduct nging reviews and audits t ensure that the infrmatin under cntrl is the same as that submitted. 4. The apprval fr a change t cntrlled infrmatin must be made by the highest-level rganizatin which last apprved the infrmatin prir t placing it under CM. 5. CM shuld be implemented in a centralized library supprted by an autmated tl. 6. CM needs t be a cntinuus prcess implemented at the beginning f a prgram and cntinuing until prduct retirement. 7. A CM manager shuld be specifically assigned. 8. The CM prcess tgether with autmated tls used by CM will maintain versin and semantic cnsistency between cnfiguratin items (CIs). 9. CM tasks shuld be integrated with quality assurance (QA) tasks that verify that prducts input t CM baselines have met their acceptance criteria. 10. CM shuld mnitr and cntrl the delivery and release-t-peratin prcess. Implementatin Guidelines: 1. CM plans need t be develped by the ACQUIRER and the DEVELOPER t facilitate management cntrl f infrmatin they wn. The CM prcedures f the ACQUIRER serve as the requirements fr the CM plan that describes and dcuments hw the DEVELOPER will implement a single CM prcess. This plan shuld cntrl frmal baselines and will include engineering infrmatin, reprts, analysis infrmatin, test infrmatin, user infrmatin, and any ther infrmatin apprved fr use r shared within the prgram. The CM prcess shuld include DEVELOPER-cntrlled and -develped baselines as well as ACQUIRER-cntrlled baselines. It shuld als include release prcedures fr all classes f prducts under cntrl, means fr identificatin, change cntrl prcedures, status f prducts, and reviews and audits f infrmatin under CM cntrl. The CM plan needs t be cnsistent with ther plans and prcedures used by the prject. 2. The tw types f baselines managed by CM are develpmental and frmal. Develpmental baselines include all sftware, artifacts, dcumentatin, tls, and ther prducts nt yet apprved fr delivery t the ACQUIRER but essential fr successful prductin. Frmal baselines are infrmatin/prducts (sftware, artifacts, r dcumentatin) delivered and accepted by the ACQUIRER. Develpmental baselines are wned by the DEVELOPER while frmal baselines are wned by the ACQUIRER. 3. All infrmatin placed under CM as a result f meeting task exit criteria need t be uniquely identified by CM and placed under CM cntrl. This includes sftware, artifacts, dcuments, cmmercial ff-the-shelf (COTS), gvernment ff-the-shelf (GOTS), perating systems, middleware, database management systems, database infrmatin, and any ther infrmatin necessary t build, release, verify, and/r validate the prduct. 4. The CM prcess shuld be rganizatinally centered in a prject library. This library will be the repsitry (current and histrical) f all cntrlled prducts. The ACQUIRER and the DEVELOPER will implement an rganizatinally specific library. The library(s) will be partitined accrding t the level f cntrl f the infrmatin. 15 f 34 16cspv5-2.dc

16 5. All infrmatin managed by CM is subject t change cntrl. Change cntrl cnsists f: a. Identificatin b. Reprting c. Analysis d. Implementatin 6. The change cntrl prcess needs t be implemented thrugh an apprpriate change mechanism tied t wh wns the infrmatin: a. Change cntrl bards, which manage frmal baseline prducts. b. Interface bards, which manage jintly wned infrmatin c. Engineering review bards, which manage DEVELOPER-cntrlled infrmatin. 7. Any infrmatin released frm the CM library shuld be described by a Versin Descriptin Dcument (Sftware Versin Descriptin under 498). The versin descriptin shuld cnsist f any inventry f all cmpnents by versin identifier, an identificatin f pen prblems, clsed prblems, differences between versins, ntes and assumptins, and build instructins. Additinally, each library partitin shuld be described by a current versin descriptin that cntains the same infrmatin. 8. Tls are an imprtant cmpnent in the develpment and cntrl prcesses f sftware develpment. Great care shuld be exercised in the selectin, use, maintenance, and cnfiguratin cntrl f autmated tls. Autmated tls have the ptential t slve prject prblem mre efficiently than manual techniques 9. The CM prcess will include supplier/develper-cntrlled develped baselines as well as acquirer-cntrlled baselines. It will als include release prcedures fr all classes f prducts under cntrl, means fr identificatin, change cntrl prcedures, status f prducts, and reviews and audits f infrmatin under CM cntrl. The CM plan will be cnsistent with ther plans and prcedures used by the prject. 10. There are tw different transitins that must be managed by CM: Engineering Transitin transitins r release f infrmal, nn-base lined infrmatin between activities r rganizatins; and Baseline Transitin transitins f frmal dcumentatin r prducts characterized by a frmal apprval r acceptance and prduct transfer. CM identificatin segregates infrmatin t ensure that the prper level f apprval is applied t each piece f infrmatin (inprcess infrmatin nt yet shared r apprved, cnfiguratin management infrmatin, cntrlled prgram infrmatin, test infrmatin, pre-released infrmatin internally apprved fr release, custmer-apprved r base lined infrmatin 11. The written CM plan shuld include: Identificatin f the cnfiguratin items in delivered-prduct and develpmental baselines Details f the change-cntrl prcess fr each f the delivered-prduct and develpmental baselines Identificatin f external interfaces and ICWGs Details f cnfiguratin status reprting Details f versin cntrl prcesses Details f the cnfiguratin audit prcesses Details f the use f autmated tls by CM 12. The develpmental baseline will include: All task utput that is input t ther tasks Output files and databases f autmated tls used fr analysis, design, requirements traceability, and dcument prductin Test input data, test drivers, and successful test results 16 f 34 16cspv5-2.dc

17 Deficiency (prblem, truble) reprts Autmated tls used fr develpment and COTS prducts, such as database management systems (DBMSs), that are part f the delivered sftware Operating systems and middleware 13. CM shuld cllect and reprt the CM churn metric as an indicatr f ptentially excessive rewrk. 14. Develp CM Plan 15. Develpment Baseline & Frmal Baseline 16. All inf under CM meet exit criteria, uniquely identified and cntrlled 17. Single Organizatin Prject library which supprts the prject. The library shuld supprt multiple cntrl levels, prvide fr strage and retrieval f cnfiguratin items, prvide fr sharing and transfer f CI's between grups and cntrl levels, help in the use f prduct standards, prvide fr strage and recvery f archive versins f CI's, help ensure the crrect creatin f crrect prducts frm the library, prvide fr strage and retrieval f SCM recrds, supprt prductin f SCM reprts, and prvide fr the maintenance f library structure and cntents (e.g. backup/recvery f files). 18. Develp and mnitr executin f a frmal Change Cntrl prcess 19. Identificatin. Prducts are identified fr CM cntrl fllwing established/dcumented criteria. Once identified fr CM, they are assigned unique identifiers, the characteristics f each CI are specified, the baseline t which the CI belngs is specified, as well as the pint when the item will cme under CM cntrl and the persn respnsible fr the item is identified. 20. Reprting. Standard reprts include SCCB minutes, change request summary, truble reprt summary, baseline change summary, CI revisin histry, baseline status, audit results. As tls imprve, much f this infrmatin becmes mre readily available nline. If prcesses can be adapted t make this infrmatin mre readily available via n-line access, this capability shuld be explited. Other reprts/infrmatin releases may be needed t accmplish CM's bjective f keeping all shared infrmatin cnsistent. 21. Change Mechanism. Changes shuld be made t baselines accrding t an rganized, planned, and dcumented prcedure. Reviews/tests shuld be prefrmed t ensure that changes have nt caused unexpected effects n frmal baselines, nly CI's apprved by the SCCB are entered int the frmal baseline, and CI's are checked in and ut in a manner that maintains the crrectness and integrity f the library (e.g. verifying that revisins are authrized, creating a change lg, maintaining a cpy f the change, archiving the riginal versin) 22. Change Cntrl Bards (frmal baseline) 23. Three types f autmated tls: narrw spectrum single task (structured design tl), brad spectrum acrss tasks (traceability tl), integrating technlgy integrates tls (infrmatin management tls) 24. The CM prcess tgether with autmated tls used by CM will maintain versin and semantic cnsistency between CI s. 25. The CM prcess will include supplier/develper-cntrlled develped baselines as well as acquirer-cntrlled baselines. The develpmental baseline will include: all task utput that is input t ther tasks; utput files and databases f autmated tls used fr analysis, design, requirements traceability, and dcument prductin; test input data, test drivers, and successful test results; deficiency (prblem, truble) reprts; autmated tls used fr develpment and COTS prducts such as database management systems (DBMS s), that are part f the delivered sftware; and perating systems and middleware. 26. Test infrmatin is all test dcumentatin and versins f sftware released fr test, which includes: test scenaris, data, and results, test cnfiguratins, test cases, sftware cnfiguratins under test and cnfiguratin dcumentatin, patches and unqualified surce mdificatins, special test data, recrds, and audit trails. 27. The wrking area fr CM includes: wrkspace fr staging dcumentatin and sftware releases; tls and files used by the prject r needed t build r cntrl prject infrmatin and/r deliverables; prject plans, prcedures, and reprts; schedules, budgets, and CM recrds; and audit trails, recrds, and lessns-learned infrmatin essential fr certificatin, apprval, r acceptance 17 f 34 16cspv5-2.dc

18 28. The tw types f baselines managed by CM are develpmental and frmal. Develpmental baselines include all sftware, artifacts, dcumentatin, tls, and ther prducts nt yet apprved fr delivery t the acquirer but essential fr successful prductin. Frmal baselines are infrmatin/prducts (sftware, artifacts, r dcumentatin) delivered and accepted by the acquirer. The supplier/develper wns develpmental baselines while the acquirer wns frmal baselines. 29. All infrmatin managed by CM will be subject t change cntrl. Change cntrl cnsists f: identificatin, reprting, analysis, and implementatin. 30. All infrmatin that is used by mre than ne persn shuld be under CM cntrl. 31. A CM manager shuld be specifically assigned 32. Any infrmatin released frm the CM library will be described by a Versin Descriptin Dcument (VDD) (Sftware Versin Descriptin under MIL-STD-498 and EIA/IEEE J-STD-016). The versin descriptin will cnsist f any inventry f all cmpnents by versin identifier, an identificatin f pen prblems, clsed prblems, differences between versins, ntes and assumptins, and build instructins. Additinally each library partitin will be described by a current versin descriptin that cntains the same infrmatin. 8. Manage and Trace Requirements Practice Essentials: 1. Befre any design is initiated, requirements fr that segment f the sftware need t be agreed t. 2. Requirements tracing shuld be a cntinuus prcess prviding the means t trace frm the user requirement t the lwest level sftware cmpnent. 3. Tracing shall exist nt nly t user requirements but als between prducts and the test cases used t verify their successful implementatin. 4. All prducts that are used as part f the trace need t be under cnfiguratin cntrl. 5. Requirements tracing shuld use a tl and be kept current as prducts are apprved and placed under CM. 6. Requirements tracing shuld address system, hardware, and sftware and the prcess shuld be defined in the system engineering management plan and the sftware develpment plan. 7. System requirements shuld include scenaris (sequences f events) that culd ccur in specify system input that will cause the system t perate under highest stress. include business rules. include specificatin f all external systems t which there will be an electrnic interface. 8. Design will nt begin fr any design layer until all requirements have been allcated and traced t that layer, including perfrmance, reliability, safety, external interface, and security requirements. 9. If develpment will be dne with incremental releases, a release build plan shuld be develped at prject inceptin where all existing system requirements are traced int releases. 10. If the incremental release mdel is t result in evlutinary definitin f system requirements, whenever new system requirements are defined, they shuld be traced int future releases defined by the release build plan. 18 f 34 16cspv5-2.dc

19 Implementatin Guidelines: 1. The prgram needs t define and implement a requirements management plan that addresses system, hardware, and sftware requirements. This plan shuld be linked t the SDP. 2. All requirements need t be dcumented, reviewed, and entered int a requirements management tl and put under CM. This requirements infrmatin shuld be kept current. 3. The CM plan shuld describe the prcess fr keeping requirements data internally cnsistent and cnsistent with ther prject data. 4. Requirements traceability needs t be maintained thrugh specificatin, design, cde, and testing. 5. Requirements shuld be visible t all prject participants. 6. An verall requirement n a successful prject is the definitin and maintenance f gd prcesses. Prcess requirements definitin starts at the system requirements Management shuld demnstrate their cmmitment t sftware engineering practices by requiring via written plicy an SEPG that crdinates develpment prcesses and imprvement activities. An SEPG shuld be established and staffed by experienced prfessinals, with time t devte. All sftware and systems engineering disciplines shuld be represented. The SEPG shuld develp and maintain the standard develpment prcess (r crdinate this activity) and all prcess assets. The rganizatin's standard prcess is maintained accrding t a dcumented prcess. The prcess is dcumented accrding t rganizatinal standards. Guidelines/criteria fr tailring the standard prcess are defined & maintained. A sftware prcess database is established and maintained. A prcess library is maintained. 7. Individual requirements shuld be linked t ne r mre critical requirements categries such as safety, security, and perfrmance. 8. Requirements vlatility metrics shuld be cmputed frm the requirements traceability database. 9. All requirements dcumented are reviewed & put in RM tl under CM 10. CM plan describe maintaining Requirements (cnsistent) 11. Traceability thrugh specificatin, design, cde & testing 12. Develp Requirements Management Plan (linked t SDP) 13. Requirements visible t all 14. Requirements must trace dwn frm system requirements thrugh all derived requirements and layers f design t the lwest-level sftware and hardware cnfiguratin items. 15. Requirements tracing shuld be dne using an autmated requirements traceability tl. 16. Trace system requirements dwn thrugh all derived requirements and layers f design t the lwest level and t individual test cases 17. All requirements will be dcumented, reviewed, and entered int a requirements management tl and put under CM. This requirements infrmatin will be kept current. 18. The prgram will define and implement a requirements management plan that addresses system, hardware, and sftware requirements. This plan will be linked t the SDP. 19. Senir management shuld establish and cmmunicate a frmal plicy that establishes their cmmitment t gd sftware engineering practices and prject success. Items t be cvered include: sftware requirements will be derived frm and 19 f 34 16cspv5-2.dc

20 always retain a systems perspective, that all requirements and their allcatins will be dcumented, frmally reviewed and apprved befre they are used as the basis fr future wrk, and that changes t requirements will cnsistently incrprated int all prject plans, prducts and activities. Further the plicy shuld indicate that prject activities will be designed t efficiently and effectively implement the system and sftware requirements and that frmal plans will be used t dcument prgram activities. When bjectives are nt met immediate actin will be taken. Plans will be crdinated and apprved by participants. Plicy shuld emphasize training - making sure training necessary t perfrm SDP tasks is prvided. 20. Requirements will be reviewed t make sure they are "gd" (i.e. testable, clear, prperly stated, cnsistent, and testable) 21. Requirements shuld be the basis fr all prgram plans and activities; SQA shuld review/audit the prducts and activities fr all activities t ensure prcess cmpliance and prduct quality - prblems identified are assigned t the respnsible party fr crrectin and cnsistent incrpratin with ther prject activities - the prcess that allwed the failure is reviewed and crrected. Metrics n prblems are used t track prject wide trends. Bth trends and specific prblems shuld be reprted t the PM. Crrective actins shuld be tacked t clsure. 22. Senir management shuld establish and cmmunicate a frmal plicy that establishes their cmmitment t gd sftware engineering practices and prject success. The plicy shuld als call fr crdinatin and agreement between team members fr prject activities - when negtiatin cannt reach agreement, then issues shuld be elevated fr immediate reslutin. W.R.T CM, the plicy shuld specify that respnsibility fr CM is explicitly assigned, that SCM shall be implemented thrughut the prject life cycle and integrated with CM at the system level, that CM is t be used fr all prducts, shared infrmatin and tls, that prjects will have a repsitry fr CM units and recrds and that SCM baselines and activities shuld be audited peridically. W.R.T. SQA, the plicy shuld specify that SQA will be used fr all prjects, prvide a reprting avenue fr SQA t senir management, and that senir management will regularly review SQA activities. 23. Senir management shuld establish and cmmunicate a frmal plicy that establishes their cmmitment t gd sftware engineering practices and prject success. W.R.T. Subcntract mgmt, plicy shuld state that dcumented standards and prcedures will be used t select subcntractrs and manage subcntracts, that cntracts are the basis fr managing the sub, and cntractual changes are made by mutual agreement f all parties. SQA shuld pay specific attentin t the fllwing activities: Planning (schedule estimating/planning, activities fr reviewing and making prject cmmitments, activities/standards fr preparing plans (e.g. SDP, CM, QA, Risk), and cntent f plans). SQA shuld share their results with the custmer's SQA rganizatin. Recall that SQA activities are activities as well and shuld be independently reviewed. 24. Management shuld demnstrate their cmmitment t sftware engineering practices by requiring via written plicy an SEPG that crdinates develpment prcesses and imprvement activities. This wuld include maintaining an rganizatinal fcus, peridic assessments f develpment prcesses, establishing a cnsistent prcess that can be tailred fr each prject, and prliferatin f experiences, tls, and lessns between prjects. Mgmt shuld reflect their cmmitment via their actins, fcus, pririties, prmtins, cmpensatin plans and cmmitment f resurces. Mgmt shuld ensure that imprvement bjectives supprt strategic bjectives and guide imprvement activities. Mgmt plicy shuld als call fr a standard sftware prcess, tailring t each prject, maintenance f prcess assets (tls, guides), and rganized cllectin f prject results t use in imprving the standard prcess 25. Mgmt shuld demnstrate their cmmitment t sftware engineering practices by requiring via written plicy an SEPG that crdinates develpment prcesses and imprvement activities. This wuld include maintaining an rganizatinal fcus, peridic assessments f develpment prcesses, establishing a cnsistent prcess that can be tailred fr each prject, and prliferatin f experiences, tls, and lessns between prjects. Mgmt shuld reflect their cmmitment via their actins, fcus, pririties, prmtins, cmpensatin plans and cmmitment f resurces. Mgmt shuld ensure that imprvement bjectives supprt strategic bjectives and guide imprvement activities. Mgmt plicy shuld als call fr a standard sftware prcess, tailring t each prject, maintenance f prcess assets (tls, guides), and rganized cllectin f prject results t use in imprving the standard prcess 26. An SEPG shuld be established and staffed by experienced prfessinals, with time t devte. All sftware and systems engineering disciplines shuld be represented. The SEPG shuld develp an imprvement plan that uses infrmatin frm assessments, defines imprvement activities/schedules, defines respnsibilities, identifies resurces, underges peer review/update and is agreed t by the rganizatin's managers. Prcess imprvement activities shuld be managed as a prject, with crrespnding discipline. The bjective fr each activity shuld be clearly stated (experiment, adptin, etc). The SEPG shuld develp/imprve the rganizatin's standard develpment prcess and each prject's tailred prcesses. 20 f 34 16cspv5-2.dc

21 Inf n develpment prcesses & imprvement activities shuld be kept in a database easily accessible by all team members. Tls/prcesses that prve effective shuld be transferred t the rganizatin in an rganized manner IAW the plan. SEPG activities shuld be advertised & regularly seek feedback 27. The SEPG shuld develp and maintain the standard develpment prcess (r crdinate this activity) and all prcess assets. 28. The rganizatin's standard prcess is maintained accrding t a dcumented prcess which ensures that: the prcess satisfies all rganizatinal plicies, standards, etc., satisfies the plicies, standards, etc. nrmally impsed by custmers, state f the art tls and methds are used when apprpriate, internal prcess interfaces are described, interfaces with systems engineering/cntract mgmt/dcumentatin supprt are identified, a prcess exists fr prpsing/reviewing/agreeing t/implementing prcess changes exists, the prcess underges peer review, and is placed under CM 29. The prcess is dcumented accrding t rganizatinal standards and is decmpsed int understandable segments, each prcess is well defined, and prcess inter-relatinships are defined. Apprved sftware life cycle mdels are identified, The mdels must be cmpatible with the rganizatin's systems apprach and standard sftware prcesses, changes t the mdels are crdinated/apprved/dcumented in an rganized manner 30. Guidelines/criteria fr tailring the standard prcess are defined & maintained. The guidelines/criteria cver selecting and tailring the life cycle, the prcess, and standards fr dcumenting the prject's prcess. Tailring criteria/guideline are placed under CM cntrl and fllw that prcess fr review and update. The SEPG apprves the changes. 31. A sftware prcess database is established and maintained. The db cllects and makes available inf n sftware wrk prducts. Data entered is reviewed t ensure integrity. The DB is cntrlled/managed. User access is cntrlled t ensure cmpleteness, accuracy and integrity. The data is available t all wh need it, hwever. 32. A prcess library is maintained. Candidate dcuments are reviewed prir t inclusin. Dcuments are catalgued fr easy access. The library is kept current (revisins psted). Cntents are available t all rganizatinal members/team members. Use f dcuments is mnitred t determine if dcument is still relevant. The library cntents are managed/cntrlled. 9. Use System-Based Sftware Design Practice Essentials: 1. All methds used t define system architecture and sftware design shuld be dcumented in the system engineering management plan and sftware develpment plan and be frequently and regularly evaluated thrugh audits cnducted by an independent prgram rganizatin. 2. Sftware engineering needs t participate in the definitin f system architectures and shuld prvide an acceptance gate befre sftware requirements are defined. 3. The allcatin f system architecture t hardware, sftware, r peratinal prcedures needs t be the result f a predefined engineering prcess and be tracked thrugh traceability and frequent quality evaluatins. 4. All agreed t system architectures, sftware requirements, and sftware design decisins shuld be placed under CM cntrl when they are apprved fr prgram implementatin. 5. All architecture and design cmpnents need t be apprved thrugh an inspectin prir t release t CM. This inspectin shuld evaluate the prcess used t develp the prduct, the frm and structure f the prduct, the technical integrity, and the adequacy t supprt future applicatins f the prduct t prgram needs. 6. All system architecture decisins shuld be based n a predefined engineering prcess and trade studies cnducted t evaluate alternatives. 7. System and sftware architectures shuld be develped frm the same partitining perspective t minimize the cmplexity f requirements allcatin/traceability: Infrmatin, Objects, Functins, States. 21 f 34 16cspv5-2.dc

22 8. Sftware engineers shuld participate in system architecture design and design f the client/server netwrk n which the sftware will execute. 9. The system architecture design shuld supprt security, reliability, perfrmance, safety, and interperability requirements. 10. The system architecture design shuld supprt reuse f legacy sftware and integratin f COTS/GOTS sftware in a way that minimizes mdificatins and additins t the reuse sftware. 11. System and sftware architecture design shuld give views f static, dynamic, and physical structures. Static views shw cmpnents that can exist. They cntain n tempral infrmatin. Static views als include threads f cllabratin. Dynamic views shw tempral, cncurrency, and synchrnizatin behavir. They include interactins in time sequences and sequences f states. The physical view describes the mapping f the sftware and databases nt the hardware and reflects its distributed aspect. 12. The system architecture shuld be mdeled and simulated befre the start f sftware architecture design t verify supprt fr security, reliability, perfrmance, and safety requirements. Implementatin Guidelines: 1. The DEVELOPER shuld ensure that the system and sftware architectures are develped and maintained cnsistent with standards, methdlgies, and external interfaces specified in the system and sftware develpment plans. 2. Sftware engineers need t be an integral part f the team perfrming systems engineering tasks that influence sftware. 3. Systems engineering requirements trade studies shuld include effrts t mitigate sftware risks. 4. System architecture specificatins need t be maintained under CM. 5. The system and sftware architecture and architecture methds need t be cnsistent with each ther. 6. System requirements, including derived requirements, need t be dcumented and allcated t hardware cmpnents and sftware cmpnents. 7. The requirements fr each sftware cmpnent in the system architecture and derived requirements need t be allcated amng all cmpnents and interfaces f the sftware cmpnent in the system architecture. 8. Sftware engineers will be an integral part f the team perfrming systems engineering tasks that influence sftware. Sftware engineering persnnel shuld be integral team members in all preliminary prject phases. This includes cncept explratin, feasibility studies and prpsal preparatin. This is especially imprtant during the negtiatins phases when prject cmmitments are being made 9. When an incremental develpment life cycle mdel is fllwed, the system architecture shuld be develped as part f the first incremental release t minimize the risk f architecture rewrk causing majr ripple-effect rewrk. 10. The structured methds fr system and sftware architecture design shuld be selected first, and then autmated tls selected that implement the selected methds. 11. CSCI Architectures is cnsistent, specified in System & Sftware Develpment Plans 12. SW Engineers are part f Systems Engineering 13. Use System Engineering trade studies t mitigate risks 14. System and Sftware architecture cnsistent 15. System Requirements & Derived Requirements are allcated t HW & SW 16. Sftware engineers shuld participate in system architecture design and design f the client/server netwrk n which the sftware will execute. 22 f 34 16cspv5-2.dc

23 17. Autmated CASE tls shuld be used fr system and sftware architecture design. CASE tls autmatically find errrs in cmpleteness and cnsistency. A drawing tl is nt a design CASE tl. 18. Planning fr sftware activities shuld begin at the same time as system planning activities. The CM prcess shuld be explited t keep these parallel effrts cnsistent. This crdinated planning activity shuld cntinue thrughut the prject life. As part f the crdinated planning a sftware lifecycle that supprts the system bjective and schedule will be agreed t and dcumented. Planning shuld include direct and supprt activities (e.g. SQA). All plans shuld be reviewed and agreed t by all participants and impacted parties. 19. Autmated CASE tls shuld be used fr system and sftware architecture design. CASE tls autmatically find errrs in cmpleteness and cnsistency. A drawing tl is nt a design CASE tl. 10. Ensure Data and Database Interperability Practice Essentials: 1. All data and database implementatin decisins shuld cnsider interperability issues and, as interperability factrs change, these decisins shuld be revisited. 2. Prgram standards shuld exist fr database implementatin and fr the data elements that are included. These standards shuld include prcess standards fr defining the database and entering infrmatin int it and prduct standards that define the structure, elements, and ther essential database factrs. 3. All data and databases shuld be structured in accrdance with prgram requirements, such as the DII COE, t prvide interperability with ther systems. 4. All databases shared with the prgram need t be under CM cntrl and managed thrugh the prgram change prcess. 5. Databases and data shuld be integrated acrss the prgram with data redundancy kept t a minimum. 6. When using multiple COTS packages, cmpatibility f the data/referential integrity mechanisms need t be cnsidered t ensure cnsistency between databases. 7. Dwnlad and read dcuments frm DD data administratin hme page. Nt t check ff cmpliance with DD plicy, but t get guidance n hw t lwer the cst f data interperability with yur externals 8. Infrmatin systems shuld be designed with very lse cupling between hardware, persistent data, and applicatin sftware. 9. N change t an applicatin sftware prgram accessing a database shuld frce a change t any ther applicatin sftware prgram accessing the same database. 10. Data elements shuld be identified using several prcesses. Analysis f business r missin prcesses using a structured methd Definitin f persistent data transactins and business rules Develpment f user interface prttype with participatin f peratinal users Analysis f persistent data needs f all applicatin sftware t interface the database Reverse engineering f legacy applicatin sftware may be required A cmplete inventry f all external interfaces 11. Relatinships between data items shuld be defined based n queries t be made n the database. 23 f 34 16cspv5-2.dc

24 12. Data element names, definitins, minimum accuracy, data type, units f measure, and range f values shuld be defined t minimize the amunt f translatin required t share data with external systems. 13. Business rules and high-vlume transactins n the database shuld be specified befre database physical design begins. 14. Data security requirements, including security level f aggregated data, shuld be specified befre database design begins. Implementatin Guidelines: 1. The DEVELOPER needs t ensure that data files and databases are develped with standards and methdlgies. 2. The DEVELOPER needs t ensure that data entities and data elements are cnsistent with the DD data mdel. 3. All data and databases shuld be structured in cmpliance with DII COE t prvide interperability with ther systems. 4. Data integrity and referential integrity shuld be maintained autmatically by COTS DBMSs r ther COTS sftware packages. The DEVELOPER shuld avid develping its package, if at all pssible. Befre selecting multiple COTS sftware packages, the DEVELOPER shuld study the cmpatibility f the data/referential integrity mechanisms f these COTS packages and btain assurance frm the COTS vendrs first. 5. Unnecessary data redundancy shuld be reduced t minimum. 6. Data and databases shuld be integrated as much as pssible. Except data fr temprary use r fr analysis/reprt purpses, each data item shuld be updated nly nce, and the changes shuld prpagate autmatically everywhere. 7. Quick respnse t high-vlume transactins shuld be a majr cnsideratin in the physical database design f n-line transactin prcessing (OLTP) databases that supprt peratins. 8. Decisin-supprt databases shuld be designed t supprt ad hc queries and shuld be separate frm OLTP databases. 9. Database design shuld be cmpleted in the first, r a very early, release in an incremental-release develpment. 10. Capabilities f the COTS DBMS used t implement the database shuld be selected with a detailed understanding f which capabilities are prprietary. 11. The ease f linking database queries t the database vendr's prduct t widgets and fields in screens develped with a GUI-builder tl shuld be a majr factr in the selectin f a GUI-builder tl. 12. If databases managed by COTS DBMS frm different vendrs are t be integrated with a vendr gateway prduct, carefully determine the prprietary features f the gateway and hw it limits database access, particularly t DBMS, frm vendrs ther than the gateway vendr. 13. Design f database physical distributin, replicatin, and levels f aggregatin n a netwrk shuld be a trade-ff between prviding current data with quick respnse t users and reducing the bandwidth needs frm the netwrk. 14. When legacy data will be used in a new database structure, plan a task with adequate time and budget t cnvert legacy data t the new frmat. 15. This task shuld include analyzing the legacy data fr errrs and fixing them 16. All data and databases shuld be structured in cmpliance with Defense Infrmatin Infrastructure Cmmn Operating Envirnment (DII COE) t prvide interperability with ther systems. 17. The ease f linking database queries t the database vendrs prduct t widgets and fields in screens develped with a GUI-builder tl shuld be a majr factr in the selectin f a GUI-builder tl. 11. Define and Cntrl Interfaces Practice Essentials: 1. Befre cmpletin f system-level requirements, a cmplete inventry f all external interfaces needs t be cmpleted. 24 f 34 16cspv5-2.dc

25 2. All external interfaces need t be described as t surce, frmat, structure, cntent, and methd f supprt and this definitin, r interface prfile, needs t be placed under CM cntrl. 3. Any changes t this interface prfile shuld require cncurrence by the interface wners prir t being made. 4. Internal sftware interfaces shuld be defined as part f the design prcess and managed thrugh CM. 5. Interfaces shuld be inspected as part f the sftware inspectin prcess. 6. Each sftware r system interface needs t be tested individually and a test f interface supprt shuld be cnducted in a stressed and anmalus test envirnment. 7. The requirements fr each external electrnic interface shuld include the items listed in paragraph 3 f the MIL-STD-498 data item descriptin DI-IPSC The design specificatin fr each external electrnic interface shuld include the items listed in paragraph 3 f the MIL-STD- 498 data item descriptin DI-IPSC Future user participatin in the develpment f a user interface prttype shuld be a primary structured methd fr users t define system requirements. 10. Interface Change Cncurrence (by wners) 11. Milestnes in Prject Activity Netwrk (see belw fr detail) 12. Cntrl Subsystem Interface at Prgram Level 13. Applicatin external interfaces, interfaces with middleware and perating systems, and interfaces f COTS prducts integrated int the applicatin shuld cmply with applicable public, pen API standards and data interperability standards unless there is a cmpelling reasn t d therwise. Cmply with JTA interface standards 14. D nt assume that if tw interfacing applicatins cmply with the JTA and DII COE TAFIM interface standards that they will interperate. 15. A standardized API shuld be used fr accessing security mechanisms. 16. APIs are defined primarily in supprt f applicatin prtability. Implementatin Guidelines: 1. All internal and external interfaces need t be dcumented and maintained under CM cntrl. 2. Changes t interfaces require cncurrence by the interface wners prir t being made. 3. Milestnes related t external interfaces shuld be tracked in the prject activity netwrk. [Keep these milestnes ff yur critical path.] 4. Subsystem interfaces shuld be cntrlled at the prgram level. 5. Develp user interface prttype used in system requirements definitin with a GUI-builder tl in a way that the prttype, with minimal mdificatin, will be the user interface sftware f the delivered applicatin. 6. The user interface prttype used fr system requirements definitin shuld include screens and navigatin between screens. 7. Milestnes related t external interfaces--such as cmpletin f specificatins and cmpletin f system interface-shuld be included in the prject activity netwrk. 8. Cmpliance with pen, public standards fr APIs shuld be a majr factr in selecting COTS perating system and middleware prducts and COTS applicatin and service/utility prducts. 9. If yu plan significant reuse f legacy sftware, design the sftware architecture t minimize changes t legacy mdule interfaces. 10. With a GUI-builder tl, develp the user interface prttype specified in the system requirements definitin in a way that the prttype, with minimal mdificatin, will be the user interface sftware f the delivered applicatin. This user interface prttype shuld include screens and navigatin between screens. 25 f 34 16cspv5-2.dc

26 11. Gain a gd understanding f middleware, client/server netwrk, and Internet perating system API ptins and pitfalls. Electrnic interfaces cnnect mdules in the architecture Applicatin Prgramming Interface (API) is defined as the interface between the applicatin sftware and the applicatin platfrm External Envirnment Interface cntains the external entities with which the applicatin platfrm exchanges infrmatin. The External Envirnment Interface (EEI) is the interface between the applicatin platfrm and the external envirnment acrss which infrmatin is exchanged. EEI may be divided int these grups: Human/Cmputer Interactin Services EEI, Infrmatin Services EEI, and Cmmunicatins Services EEI. 12. Design Twice, Cde Once Practice Essentials: 1. All design prcesses shuld fllw methds dcumented in the sftware develpment plan. 2. All designs need t be subject t verificatin f characteristics, which are included as part f the design standards fr the prduct prduced. 3. Cmpleted Architectures in First Release 4. Verify System & Sftware Architecture per SDP 5. All designs shuld be evaluated thrugh a structured inspectin prir t release t CM. This inspectin shuld cnsider reuse, perfrmance, interperability, security, safety, reliability, and limitatins. 6. Traceability needs t be maintained thrugh the design and verified as part f the inspectin prcess. 7. Critical cmpnents shuld be evaluated thrugh a specific white-bx test level step. 8. Design can be incrementally specified when an incremental release r evlutin life cycle mdel is used prvided the CM prcess is adequate t supprt cntrl f incremental designs and the inspectin prcess is adapted t this requirement. 9. The sftware detailed design methds shuld be cnsistent with the design methds used fr architecture design. 10. End-user functinality shuld be visible in the sftware design descriptin. 11. The executin prcess shuld be visible in the sftware design, including perfrmance, availability, cncurrency, faulttlerance, threads f cntrl, and prcesses (grups f tasks that frm an executable unit). 12. Physical sftware cmpnents and their interfaces shuld be visible in a design descriptin. 13. The mapping f physical sftware cmpnents nt individual hardware cmpnents shuld be visible. 14. States and state transitin shuld be visible in the design descriptin if the system has mre than ne state. 15. Each f the abve views f the sftware shuld be dcumented in a graphical ntatin. 16. Operatinal scenaris shuld be defined that shw hw the different views f design wrk tgether. Implementatin Guidelines: 1. When reuse f existing sftware is planned, the system and sftware architectures shuld be designed t facilitate this reuse. 26 f 34 16cspv5-2.dc

27 2. When an incremental release life cycle mdel is planned, the system and sftware architectures need t be cmpleted in the first release r, at mst, extended in releases after the first withut changes t the architecture f previus releases. 3. The system and sftware architectures will be verified using methds specified in the SDP. This verificatin will be cnducted during a structured inspectin f the sftware architecture and will include crrbratin that the architecture will supprt all reuse, perfrmance, interperability, security, safety, and reliability requirements. The architecture will be under CM. 4. The structured design methd shuld be selected first, then select an autmated tl that enfrces the selected methd. 5. Each f the design views defined in the universal markup language (UML) ntatin guide shuld be cnsidered: Static structure diagrams Use CASE diagrams Sequence diagrams Cllabratin diagrams State chart diagrams Activity diagrams Implementatin diagrams 6. Design shuld nt be represented in a prgram design language (pseudcde) because pseudcde affects the partitining and cmplexity management reasns fr design. 7. Design shuld be dne t minimize cmplexity and maximize understandability. 8. Design diagrams shuld be develped with a CASE tl that enfrces ntatin and semantic rules, nt with a drawing tl. 9. The apprach t design shuld never be t write cde befre design and then reverse engineer the design frm this cde. 10. A prgramming language shuld be selected that was develped t implement the design methd. 13. Assess Reuse Risks and Csts Practice Essentials; 1. The use f reuse cmpnents, COTS, GOTS, r any ther nn-develpmental items (NDI) shuld be treated as a risk and managed thrugh risk management. 2. Applicatin f reuse cmpnents, COTS, GOTS, r any ther NDI will be made nly after successful cmpletin f a NDI acceptance inspectin. This inspectin needs t cnsider the prcess used t develp it, hw it was dcument, number f users, user experience, and cmpliance with essential prgram cnsideratins such as safety r security. 3. Befre a decisin is made t reuse a prduct r t acquire COTS, GOTS, r NDI, a cmplete cst trade-ff shuld be made cnsidering the full life cycle csts, update requirements, maintenance csts, warranty and licensing csts, and any ther cnsideratins which impact use f the prduct thrughut its life cycle. [Avid need fr cstly develpment f "wrappers" t translate reuse sftware external interfaces] 4. All reuse prducts, COTS, GOTS, r NDI decisins shuld be based n architectural and design definitins and be traceable back t an apprved user requirement. 5. All reuse cmpnents, COTS, and COTS need t be tested individually first against prgram requirements and in an integrated sftware and system cnfiguratin prir t release fr testing accrding t the prgram test plan. 6. Reuse, COTS, GOTS, and NDI decisins will be cntinuusly revisited as prgram cnditins change 7. Establish quantified selectin criteria and acceptability threshlds 27 f 34 16cspv5-2.dc

28 Fit f functinality t current applicatin Cmpliance f external and system sftware interfaces with external engineering interface, API, and datainterperability standards Interfaces with yur perating systems and middleware Reuse architecture t which it was designed Quality Perfrmance under stress f current applicatin Range f values fr input variables Security Quality f dcumentatin 8. It is almst certain that there will be prprietary features even when the COTS are marketed as "pen." If yur applicatin will perate n a client/server netwrk, analyze the ease f distributing the COTS prduct and its utput data n a netwrk 9. Develp cde fr future reuse t integrate int an architecture framewrk that is well defined and is supprted by many sftware cmpanies There are tw COTS architectural framewrks that dminate in cmmercial markets: CORBA/JavaBeans ActiveX/DCOM Cnsider mre stringent develpment prcess fr cde develped fr future reuse than fr cde develped nly fr the current applicatin Estimate the additinal cst f develpment fr future reuse fr input t the decisin t develp fr reuse 10. Reverse engineer legacy sftware when develping new system t replace legacy sftware Requirements Algrithms Functins Business rules Dcumentatin likely nt t be cnsistent with cde Much internal t sftware likely invisible t users 11. Reverse engineer with an autmated tl used by a persn wh can read surce cde and wh understands the dmain 12. Establish Reuse Plan 13. Reuse plan in SDP, Evaluate Reuse vs. System Requirements 14. Per Plan, System Engineering Prcess identifies sftware t reuse 15. Identify Reuse in Test Plan 16. Cst Estimate f Integratin and licenses ver Life Cycle Implementatin Guidelines: 1. The DEVELOPER will establish a reuse plan fr the integratin f COTS, GOTS, and in-huse sftware. This plan needs t include discussin and allcatin f whm and by what prcess reused sftware cde is tested, verified, mdified, and maintained. 28 f 34 16cspv5-2.dc

29 2. The reuse plan shuld be in the SDP and dcument an apprach fr evaluating and enfrcing reused functinality against system requirements. 3. The reuse plan shuld suggest a system engineering prcess that identifies sftware requirements by taking existing, reusable sftware cmpnents int accunt. 4. The test plan shuld identify the testing f the integrated reused cde. 5. When integrating COTS, GOTS, and in-huse sftware, ensure accurate cst estimatin f integrating the reused cde int the system. The cst f integrating unmdified reused cde is apprximately ne-third the cst f develping cde withut reuse. 6. The DEVELOPER and the ACQUIRER need t be able t plan fr the estimated csts f btaining the necessary develpment and run-time licenses ver the system s life cycle and the maintenance/supprt critical t the prduct, including surce cde availability. 7. Define yur reuse sftware selectin criteria in writing. 8. Talk with users f the COTS r GOTS prduct befre selectin. 9. Ask the nes the vendr referenced. 10. Find yur wn references independent f vendr-prvided references. 11. Talk t users with similar percentage fit f the COTS prduct t their applicatin and similar peratinal envirnment. 12. Try and get hard data n the prductivity achieved using tls prvided by the COTS vendrs t mdify their COTS prducts. 13. It will cst rganizatins that nrmally develp wrld-class quality sftware abut 30 percent mre t develp cde fr future reuse. 14. It can cst the average shp mre than by a factr f 4 t develp cde fr future reuse. 15. Reverse engineer legacy sftware when develping a new system t replace legacy sftware (requirements, algrithms, functins, business rules, dcumentatin likely t nt be cnsistent with cde, much internal t sftware likely invisible t users, reverse engineer with an autmated tl used by a persn wh can read surce cde and wh understands the dmain) PRODUCT STABILITY AND INTEGRITY 14. Inspect Requirements and Design Practice Essentials: 1. All prducts that are placed under CM and are used as a basis fr subsequent develpment need t be subjected t successful cmpletin f a frmal inspectin prir t its release t CM. 2. Gal f 80% defect detectin 3. N CM until satisfactin f Peer Review 4. Metrics n defects 5. Exit criteria fr nn-loe earned value as gates fr increasing CM 6. Structured architecture inspectin 7. The inspectin needs t fllw a rigrus prcess defined in the sftware develpment plan and shuld be based n agreed-t entry and exit criteria fr that specific prduct. 29 f 34 16cspv5-2.dc

30 8. At the inspectin, specific metrics shuld be cllected and tracked which will describe defects, defect remval efficiency, and efficiency f the inspectin prcess. 9. All prducts t be placed under CM shuld be inspected as clse t their prductin as feasible. 10. Inspectins shuld be cnducted beginning with cncept definitin and ending with cmpletin f the engineering prcess. 11. The prgram needs t fund inspectins and track rewrk savings 12. System and sftware architectures will, in additin t frmal inspectins, be mdeled and simulated t verify that the architecture supprts perfrmance, reliability, security, and safety requirements. 13. Frmal inspectins and architecture mdeling/simulatin will be cnducted in accrdance with detailed instructins cntained in the SDP. 14. Start nw and get the team trained, then g and d it. 15. There is n reasn t wait fr the right time t emply inspectins in a prduct. 16. Begin immediately. Even if the prject is halfway thrugh the testing stage, the team can begin inspecting fixes t defects discvered during test. 17. The payff is immediate. 18. N sner will inspectins be started than defects will begin t be remved. 19. These wuld nly have been fund later at a higher cst. 20. Mdeling and simulatin f architecture shuld be dne with autmatic tls, verify the architecture will supprt perfrmance requirements, and use input that represents stress lads that might ccur during peratin. Implementatin Guidelines: 1. The DEVELOPER will implement a frmal, structured inspectin/peer review prcess that begins with the first system requirements prducts and cntinue thrugh architecture, design, cde, integratin, testing, and dcumentatin prducts and plans. The plan needs t be dcumented and cntrlled as per the SDP. 2. The prject shuld set a gal f finding at least 80 percent f the defects in every prduct underging a structured peer review r ther frmal inspectin. 3. Prducts shuld nt be accepted int a CM baseline until they have satisfactrily cmpleted a structured peer review. 4. The DEVELOPER needs t cllect and reprt metrics cncerning the number f defects fund in each structured peer review, the time between creating and finding each defect, where and when the defect was identified, and the efficiency f defect remval. 5. Successful cmpletin f inspectins shuld act as the task exit criteria fr nn-level-f-effrt earned value metrics (and ther metrics used t capture effectiveness f the frmal inspectin prcess) and as gates t place items under increasing levels f CM cntrl. 6. The DEVELOPER shuld use a structured architecture inspectin technique t verify crrectness and related system perfrmance characteristics. 7. Implement six well-defined inspectin steps: Planning Defining and verifying inspectin entry criteria Arranging the availability f the right participants Overview Educatin f participants in what is t be inspected Assignment f rles t participants 30 f 34 16cspv5-2.dc

31 Preparatin--participants learn the material Inspectin--find defects but d nt lk fr fixes Rewrk--prduct authr fixes defects Fllw up--verificatin fixes made and n new defects 8. Participatin in inspectin shuld be limited t 2-hur perids with n mre than tw 2-hur inspectin perids per day. 9. Inspectin training shuld be cmprehensive Three 3-hur sessins Vide tape f actual inspectin Students bserve actual inspectin 10. Architecture mdeling and simulatin shuld be dne using COTS mdeling and simulatin prducts. 11. Inspectins shuld be cnducted n small prducts that can be cmpletely understd by an individual. 12. Mdeling and simulatin f architecture shuld: Be dne with autmatic tls Verify the architecture will supprt perfrmance requirements Use input that represents stress lads that might ccur during peratin. 15. Manage Testing as a Cntinuus Prcess Practice Essentials: 1. All testing shuld fllw a preplanned prcess, which is agreed t and funded. 2. Every prduct that is placed under CM shuld be tested by a crrespnding testing activity. 3. All tests shuld cnsider nt nly a nminal system cnditin but als address anmalus and recvery aspects f the system. 4. Prir t delivery, the system needs t be tested in a stressed envirnment, nminally in excess f 150 percent f its rated capacities. 5. All test prducts (test cases, data, tls, cnfiguratin, and criteria) shuld be released thrugh CM and be dcumented in a sftware versin descriptin dcument. 6. Every test shuld be described in traceable prcedures and have pass-fail criteria included. 7. Cncurrent with test planning: analyze cde t identify ptential lckups, nn-develpers cnduct unstructured free-play test 8. Sftware develpment prjects will deliver test prducts t minimize the cst f regressin test during sustainment. 9. Deliverable test prducts will underg frmal inspectins. 10. GPO will plan the test prcess & dcument this plan with test cases and detailed descriptins. Include use cases re peratinal missin scenaris 11. Cnsistent with RFP, award fee incentives 31 f 34 16cspv5-2.dc

32 Implementatin Guidelines: 1. The testing prcess must be cnsistent with the RFP and the cntract. The award fee shuld incentivize implementatin f the testing practices described belw. 2. The ACQUIRER and DEVELOPER need t plan their prtin f the test prcess and dcument this plan with test cases and detailed test descriptins. These test cases shuld use cases based n prjected peratinal missin scenaris. 3. The testing prcess shuld als include stress/lad testing fr stability purpse (i.e., at 95% CPU use, system stability is still guaranteed.) 4. The test plan shuld include a justifiable testing stppage criteria. This gives testers a gal. If yur testing satisfies these criteria, then the prduct is ready fr release. 5. The test prcess shuld thrughly test the interfaces between any in-huse and COTS functinality. These tests shuld include timing between COTS functinality and the bespken functinality. The test plans need t pay serius attentin t hw t demnstrate that, if the COTS sftware fails, hw t test that the rest f the sftware can recver adequately. This invlves sme very serius stress testing using fault injectin testing. 6. Sftware testing shuld include a traceable white-bx and ther test prcess verifying implemented sftware against CMcntrlled design dcumentatin and the requirements traceability matrix. 7. A level f the white-bx test cverage shuld be specified that is apprpriate fr the sftware being tested. 8. The white-bx and ther testing shuld use autmated tls t instrument the sftware t measure test cverage. 9. All builds fr white-bx testing need t be dne with surce cde btained frm the CM library. 10. Frequent builds require test autmatin, since mre frequent cmpiles will frce quick turnarund n all tests, especially during regressin testing. Hwever, this requires a high degree f test autmatin. 11. A black-bx test f integratin builds needs t include functinal, interface, errr recvery, stress, and ut-f-bunds input testing. 12. Reused cmpnents and bjects require high-level testing cnsistent with the peratinal/target envirnment. 13. Sftware testing includes a separate black-bx test level t validate implemented sftware. All black-bx sftware tests shuld trace t cntrlled requirements and be executed using sftware built frm cntrlled CM libraries. 14. In additin t static requirements, a black-bx test f the fully integrated system will be against scenaris sequences f events designed t mdel field peratin. 15. Perfrmance testing fr systems (e.g., perfrming 10,000 tests/secnd still yields respnse times under 2 secnds) shuld be tested as an integral part f the black-bx test prcess. 16. Test t design dcumentatin & requirements in CM 17. Use Autmated tls 18. CM surce cde fr builds fr white bx 19. Black bx test f integratin builds 20. Black bx tests t mdel field peratin as well as static requirements 21. Independent audits & reprting t GPO 22. Each test t include pass/fail criteria 23. Ensure Test Planning is accmplished 24. Autmated tls shuld be used t instrument cde t measure path cverage fr white-bx tests. 25. Autmated tls shuld be used t autmatically recrd key strkes and muse clicks that execute a test and then playback this recrding t execute regressin tests and cmpare test results with crrect results. All builds fr white-bx testing will be dne with surce cde btained frm the CM library. 32 f 34 16cspv5-2.dc

33 26. An independent QA team shuld peridically audit selected test cases, test traceability, test executin, and test reprts prviding the results f this audit t the ACQUIRER. (The results f this r similar audits may be used as a factr in the calculatin f Award Fee.) 27. The implementatin f test levels t a specific prgram must be adapted t the specific prgram strategy used as per DODI and i.e., Grand Design, Incremental Develpment (Preplanned Prduct Imprvement), r Evlutinary Strategies. 28. Deliverable test-related prducts shuld include: Test cases Requirements traceability t individual test cases Test descriptins Test drivers Test input data Results frm successful tests Test tls 29. An integratin-test build plan shuld be develped that identifies the sequence f integratin builds and testing t accmplish ne r mre f the fllwing: Minimize the amunt f test driver sftware that has t be develped Early test f high-risk cde Early test f safety- r security-critical cde Minimize duratin f integratin test Autmated tls shuld be used t instrument cde t measure path cverage fr white-bx test. 16. Cmpile and Smke Test Frequently Practice Essentials: 1. All tests shuld use systems that are built n a frequent and regular basis (nminally n less than twice a week). 2. All new releases shuld be regressin tested by CM prir t release t the test rganizatin. 3. Smke testing shuld qualify new capability r cmpnents nly after successful regressin test cmpletin. 4. All smke tests shuld be based n a pre-apprved and traceable prcedure and run by an independent rganizatin (nt the engineers wh prduced it). 5. All defects identified shuld be dcumented and be subject t the prgram change cntrl prcess. 6. Smke test results shuld be visible and prvided t all prject persnnel. 7. Integratin build will be dne by CM and will include preparatin f a build VDD that identifies the sftware unit versins in the build and the pen and fixed defects against the build. 8. Integratin test will execute CM-cntrlled test descriptins. 9. CM will mnitr the status f fixing defects and will track the sftware versins in which each defect is first fixed. 33 f 34 16cspv5-2.dc

34 Implementatin Guidelines: 1. Frm the earliest pprtunity t assess the prgress f develped cde, the DEVELOPER needs t use a prcess f frequent (ne- t tw-week intervals) sftware cmpile-builds as a means fr finding sftware integratin prblems early. 2. It is required that a regressin facility that incrprates a full functinal test suite be applied with the build strategy. 3. Results f testing f each sftware build shuld be made available t all prject persnnel 4. The prject must mitigate the risk f lsing the benefit f frequent builds. 5. The prject must have a firm, enfrced standard fr what cnstitutes an acceptable prcess and prduct. 6. The prject enfrced standard needs t set a quality level that's strict enugh t keep critical defects ut but lenient enugh t disregard trivial defects. 7. Inspectins shuld mnitr, n a mre frequent schedule than the build cycle, prduct quality. 8. CM, nt engineering, shuld wn prducts apprved fr the next build. 9. Frm cntrlled libraries, CM: cmpiles all files, libraries, and ther cmpnents successfully; links all files, libraries, and ther cmpnents successfully; ensures thrugh inspectin and test recrds that cmpnents d nt cntain any shwstpper bugs that prevent the prgram frm being executed r that make it hazardus t perate; ensures that the build passes the smke test. 10. It desn t matter hw many r few cmpnents are replaced. 11. The smke test shuld exercise the entire system frm end-t-end. It need nt be exhaustive. 12. Fcus f smke test is expsing prblems and defects, nt prving system r sftware wrks. 13. The smke test shuld be thrugh enugh that, if the build passes, yu can assume that it is stable enugh t supprt subsequent test levels. 14. Smke tests require adequate time t run and sufficient time t analyze. 15. Prcess requires substantial autmatin (simulatin, emulatin, instrumentatin, analysis). 16. Require develpers t execute all critical paths thrugh the cde 17. Cncurrent with test planning: analyze cde t identify ptential lckups, nn-develpers cnduct unstructured free-play test 18. Binary patches will never be made t cde during integratin test. 19. When any change is made t surce cde frm the develpmental baseline during frequent cmpile and test, the versin number f the surce cde will be changed and the mdified cde turned in t CM. 20. CM will ensure that every sftware unit versin includes all the defect fixes made t all earlier versins. 21. Autmatin shuld be used in the integratin build prcess s that mst f the time between successive integratin tests is dedicated t integratin tests. 22. Integratin test shuld migrate t the same hardware/system sftware envirnment that will be used in the field. 34 f 34 16cspv5-2.dc