The Software Audit Guide John W. Helgeson ASQ Quality Press Milwaukee, Wisconsin
Contents Preface Acknowledgments xv Xlx Part I Audit Fundamentals 1 Chapter 1 Introduction 3 1.1 Definitions 7 1.2 Process and Quality 7 1.3 Quality 8 1.4 Auditor's Notes 8 1.5 Types of Audits 9 1.6 Standards 10 1.7 Capability Maturity Model Integration 11 1.8 Hierarchy of Standards and Requirements 12 1.9 Quality Record 14 1.10 Timeline and Name Game 14 1.11 General Audit Process 15 1.11.1 Purpose and Scope 16 1.11.2 Contact and Document Requests 16 1.11.3 Review the Documents 16 1.11.4 Announcement 17 1.11.5 Writing the Checklist 17 1.11.6 Opening Meeting 17 1.11.7 The Audit 17 1.11.8 Closing Meeting 17 1.11.9 Writing the Report 17 1.11.10 Present the Report to the Client 18 1.11.11 Review the Corrective Actions 18 1.11.12 Auditing Corrective Actions 18 1.12 How to Use the Questions 18 1.13 Examples 19 vii
viii Contents Chapter 2 Software Audit Life Cycle 21 2.1 Software Development Life Cycles 22 2.1.1 The Waterfall Model 23 2.1.2 The Spiral Model 25 2.1.3 The Incremental Model 26 2.1.4 Prototyping 27 2.1.5 The Object-Oriented Programming System (OOPS) 28 2.1.6 Blending of Models (or Hybrid) 29 2.1.7 Cleanroom 29 2.1.8 Rapid Application Development 29 2.1.9 Agile 29 2.1.10 Extreme Programming 30 2.2 How Audits Fit into the Life Cycle 30 2.2.1 Proof of Concept (or Demo) 32 2.2.2 Idea/Functional Specification 34 2.2.3 Analysis/Planning 37 2.2.4 Software Design 41 2.2.5 Coding (and Unit Testing) 46 2.2.6 Test (System Integration) 49 2.2.7 Maintenance (Production and Deployment) 51 Part II Audit Activities 57 Chapter 3 The Audit's Purpose and Scope 59 3.1 Purpose 61 3.2 Scope 63 3.3 Management's Part in an Audit 64 3.4 Preparing for the Audit 64 Chapter 4 Opening Meeting 67 4.1 The Meeting 67 4.1.1 Introductions 68 4.1.2 Present the Purpose and Scope of the Audit 68 4.1.3 Review Previous Audits 68 4.1.4 Explain the Quality Audit 68 4.1.5 Explain How the Audit Will Work 69 4.1.6 Daily Meetings 69 4.1.7 Questions and Answers 70 4.2 An Escort 70 4.3 Work Space 70 4.4 A Brief Tour 70 4.5 Meeting the People 71 4.6 Informal Overview Presentation 72 4.7 Explain the Questions 73 Chapters Conducting the Audit 75 5.1 Roles of the Auditors 76 5.1.1 Lead Auditor 76 5.1.2 Auditor 76
Contents ix 5.2 The Audit 77 5.3 WorkSheet/Checklist 78 5.4 Findings Form 79 Chapter 6 Meetings and Final Report 83 6.1 Caucus Meeting 83 6.2 Closing (Exiting) Meeting 84 6.3 Final Report 85 6.3.1 Executive Summary 86 6.3.2 Identify Documents Reviewed 87 6.3.3 Preparation Dates 87 6.3.4 Specify the Standards 87 6.3.5 Software Risk Analysis 87 6.3.6 Description of Areas 87 6.4 Writing the Report 88 6.5 Detail Finding Pages 89 6.6 Presenting the Report 90 6.7 Corrective Action Requests 90 Chapter 7 Audit Follow-up and Closure 91 7.1 Immediate Actions 92 7.2 Temporary Actions 93 7.3 Permanent Actions 94 7.4 Conclusion 95 Part III Audit Constants 97 Chapter 8 "Project" Management 99 8.1 Upper and Middle Management 99 8.2 Accounting 100 8.3 Project Manager '02 Chapter 9 Schedule 105 9.1 Types of Project Management Systems 107 9.2 Simple Charts 108 9.3 Interdependences 109 9.4 Partially Completed 110 9.5 Baselines 113 9.6 Critical Path 114 9.7 More Schedule Points 117 Chapter 10 Project Status Reports 119 Chapter 11 Configuration Management 121 11.1 Document Control 123 11.2 Baselining 124 11.3 Software Configuration Control 125 11.4 Incident Tracking 128 11.5 Incident Report and Analysis 131 11.6 Software Meeting or Board 133 11.7 Backups and Recovery 135
x Contents 11.8 Disk Array File Server 137 11.9 Formal Release 138 11.10 Field Configuration Control 140 Chapter 12 Software Quality and Continuous Improvement 143 12.1 Software Quality 143 12.2 Continuous Improvement 145 Chapter 13 Customer Satisfaction 149 Part IV Audit Processes 153 Chapter 14 Reviewing Software Documents 155 14.1 The Governing Standards 155 14.2 Quality Manual 157 14.3 Contractual Documentation 159 14.4 Organizational Chart 160 14.5 Schedule 162 14.6 General Documents Discussion 164 14.7 Software Development Plan 166 14.8 Functional Specification 167 14.9 Software Requirement Specification 170 14.10 Software Design Specification 173 14.11 Software Configuration Management Plan 177 14.12 Performance Specification 178 14.13 Processes, Procedures, and Work Instructions 179 14.14 Test Plan 180 14.15 Software Quality Plan 181 14.16 Database Layouts/Schemas 181 Chapter IS Peer Reviews 183 15.1 Peer Review Basics 185 15.2 Peer Review of the Sales Order 187 15.3 Peer Review of the Software Development Plan 188 15.4 Peer Review of the Requirements or Functional Specification 189 15.5 Peer Review of the Design Plan 190 15.6 Peer Review of the Code 191 15.7 Peer Review of the Test Plan 192 Chapter 16 Hidden Software 195 16.1 Direct Software 195 16.1.1 S pecialized Drivers 197 16.1.2 Interface Routines for Databases 197 16.1.3 Package Software 199 16.1.4 Protocols for Specialized Hardware 201 16.1.5 Operating Systems 201 16.2 Indirect Software 203 16.2.1 Homegrown Debugging Software 203 16.2.2 Simulation Software/Test Equipment 204 16.2.3 Automated Software Testing Scripts 205
Contents xi 16.2.4 Installation Scripts 205 16.2.5 Software to Populate a Database 206 16.3 Summary 206 Chapter 17 Firmware 207 17.1 Planning 208 17.2 Design 209 17.3 Testing 210 17.4 Configuration Control 211 17.5 Production 211 Chapter 18 Software Coding Standards 215 18.1 Programs and the Documentation 217 18.2 Program Headers 217 18.3 Body of the Program 219 18.4 Naming Conventions 220 18.5 Structures 221 18.6 Auto-Generated Code 222 Chapter 19 Metrics 223 19.1 Aging and Priority Metrics 225 19.2 Performance Metrics 226 19.3 Complexity Metrics 226 Chapter 20 Independent Verification and Validation 229 20.1 Verification 230 20.2 Validation 234 Chapter 21 Testing 237 21.1 Unit Testing 240 21.2 Functional Testing 241 21.3 Integration Testing 243 21.4 Acceptance Testing 246 21.5 Regression Testing 248 21.6 Automated Testing 249 21.7 Performance Testing 251 Final Thoughts 255 Index 257