Change Management in Families of Safety-Critical Embedded Systems

Transcription

1 Change Management in Families of Safety-Critical Embedded Systems Zoë Rachael Stephenson This thesis is submitted in partial fulfilment of the requirements for the degree of Doctor of Philosophy. University of York York YO10 5DD UK Department of Computer Science March 2002

2 Abstract This thesis addresses the problem of understanding change and reducing the work needed to estimate and respond to change in families of safety-critical embedded systems. Explicit family feature modelling techniques are developed that record the context within which a feature is valid for each family member. These features are combined with a description of their allowed variation among different members, to provide a complete family feature model. These techniques are used to create a family feature model for a number of industrial projects. Comparisons are made between the ability of the family model and the project processes to accurately estimate change impact. Results show that the family model provides more accurate change impact estimation than the existing project processes. It also provides an understanding of the role of domain knowledge in impact estimation, a method by which different types of specification may be traced to one another throughout the development process, and a process by which individual feature descriptions are transformed into single family descriptions. 2

3 Contents Abstract 2 Acknowledgement 12 Declaration 13 1 Introduction Embedded Systems Operating Constraints Safety-related Software Families Change Characteristics Hypothesis Literature Review Design Process Development Lifecycle Models Automation Traceability Rationale Description Families and Domains Change Impact Software Change Impact Non-software Change Summary Feature Representation 59 3

4 CONTENTS CONTENTS 3.1 Analysis Model Key Concepts Objectives graphs IBIS REMAP OSCS Redux DRCS Summary Representation Assessment Goals Decisions Results Context and Assumptions Summary Representation Synthesis Goal Decision Solution Property Assertions Decision Traces Notation Trace Sheets Analysis and Argumentation Summary Family Representation Introduction Review Design Rationale Approaches Product-line Approaches

5 CONTENTS CONTENTS 4.3 Model Development Feature Association Choice and Choice Constraints Feature Dependencies Trace Sheets Summary Family Feature Model Analysis Scoping Sources Analysis Results Modelling Industrial Project Data Summary Feature Ordering and Resolution Introduction Resolution Representation Motivation Single Model Rationale Process Rationale Specific Resolution Guidelines Correspondence Elaborate Constraint Option Lists Parameters Complex Dependencies Summary Feature Ordering

6 CONTENTS CONTENTS 6.7 Variation and Volatility Hypothesis Method Results Evaluation Conclusions Impact Analysis Accuracy Introduction Family Model Impact Analysis Software Model Impact Analysis Project Data and Measurement Experimental Assessment Hypothesis Method Results Evaluation Conclusions Conclusions and Further Work Review Findings Decision Tracing Findings Family Model Findings Resolution Findings Impact Assessment Findings Further Work A Family Feature Models 195 Definitions 203 Index 212 6

7 List of Figures 1.1 Generic Design Process Context Embedded System Customers Software and Hardware Control Preliminary Concept Summary Program Production [8] Waterfall Model [82] V Model [87, p36] Spiral Model [10, p25] Organisational Context of Draco [69, p565] Intent Structure [93] Example Design Development: Intent Structure IBIS Example Design Development: IBIS The REMAP Scheme [79] Example Design Development: REMAP The OSC Shell Scheme [2] Example Design Development: OSC Shell The Redux Scheme [76] Example Design Development: Redux The DRCS Scheme [50] Example Design Development: DRCS Intent Specification Structure[54, p4] Extended Automatic Programming Paradigm [5, p1263] Example Extended FODA Feature Model [20]

8 LIST OF FIGURES LIST OF FIGURES 3.1 Concept Summary Objectives Graph UML Model IBIS UML Model REMAP UML Model OSCS UML Model Redux UML Model DRCS UML Model IBIS Noise Reduction Example [56, p83] Revised Concept Summary Decision Representation Approach Decision Trace Symbols Decision and Feature Model Decision Trace Relationships Decision Trace References Decision Trace Reference Expressions Trace Sheet Layout Decision Element Specialisation Context Introduction and Extraction Analysis and Argumentation Separate Selection Model Choice Elements: Objectives Graphs Choice Elements: REMAP Choice Elements: OSC Shell Choice Elements: DRCS Choice Elements: Redux Example Abstract Decision Tree [71, p3] Feature Association Feature Association Notation Choice Representation Parameter Representation Choice Notation

9 LIST OF FIGURES LIST OF FIGURES 4.13 Combinatorial Example Extended Choice Constraint Notation Feature Dependency Simple Feature Dependency Notation Complex Feature Dependency Notation Abbreviated Feature Dependency Notation Dependency Example Trace Sheets with Choices Choice Model Summary Choice Notation Summary Example Project Data Example Resolved Model: Thrust Reverser Locks Feature Resolution Representation Feature Resolution and Metamodel Correspondence UML Correspondence UML Correspondence with Composition UML Correspondence with Multiple Composition UML Correspondence with Transitive Associations UML Elaboration with Composites UML Elaboration for Feature Interaction UML Elaboration for Feature Interaction Between Models UML Constraint Incorporation UML Constraint with Modification UML Option UML Option with Common Association UML Choice List UML Choice List through Generalisation UML Selection UML Selection through Generalisation UML Multiplicity Parameter UML Multiplicity Constant

10 LIST OF FIGURES LIST OF FIGURES 6.21 UML Multiplicity Type UML Feature Dependency UML List Feature Dependency Feature Resolution Order Example Features Resolved: (F1,F2,F3,F4) Features Resolved: (F1,F4,F3,F2) Example Dependency Graph Example Trace Sheet Family Selections Correlation of Project Data Volatility with Feature Volatility Correlation of Project Data Volatility with Feature Volatility for Reduced Family Scope Impact Analysis by Features Feature Resolution Impact Analysis by Software Model Algorithmic Expression Example Statechart Example Actual Impact Set Sizes Project Discrepancy and Feature Iterations

11 List of Tables 3.1 Decision Representations Development Progress Representations Common Types of Choice Description Choice Operator Abbreviations Industrial Project Accuracy Data Feature Model Accuracy Data Comparison of Actual Impact Set Sizes

12 Acknowledgement The author would like to acknowledge the support of the Engineering and Physical Sciences Research Council Systems Engineering for Business Process Change managed research programme. Work presented in this thesis was carried out under grant GR/L4872 within that programme. The author would also like to thank Rolls-Royce plc. for additional support and access to industrial project data and domain experts. 12

13 Declaration This thesis makes use of a case study model. Parts of the specification of this model were validated in collaboration with Simon Burton, and all of the detailed implementation work was carried out by Darren Buttle. All other work presented in the thesis was carried out by the author. Earlier results from this strand of work have been presented in a number of workshops, covering issues concerning the safe context of reuse [88], the relationship between change and product families [84], the relationship between decisions and features [85] and the integration of the context analysis techniques into a complete process [1]. Contributions have also been made to a book [17] and a journal paper [15]. 13

14 14 Declaration

15 Chapter 1 Introduction Embedded systems are commonplace nowadays, found in systems such as mobile telephones, televisual equipment, washing machines, dryers, cars and aircraft engines. The development of such systems is a complex engineering process, involving many different disciplines, but it is usually characterised as a design process it takes requirements from customers, designs a product for delivery, assembles it from components and custom parts, and supplies it to the customer. This customer/supplier context is shown in Figure 1.1. The customers are usually everyday consumers, but in some situations, such as the development of car or aircraft engine systems, the customer is an organisation that is also developing a system. In modern embedded systems, there is always a software element. Furthermore, that software is typically used to control a larger system of which it is only one part. The presence of software in the embedded system means that there is a software engineering process within that complex engineering process. Embedded systems research has traditionally not been a popular aspect of software engineering, but recent trends have been addressing this area, particularly product-line software engineering [26]. A product line is a group of systems that share enough similarities for it to be beneficial to build a production environment specifically for that group of systems and embedded systems typically have this type of similarity. This thesis addresses issues surrounding the development of embedded software for families of safety-critical systems. Not only do the issues of embedded software development and safety provide challenges for the process, but that process must reflect the needs of family development as well. This introduces a number of factors, discussed in the following sections. 1.1 Embedded Systems An embedded system is a system that exists within the context of a larger engineered product or environment, called the embedding system. The embedded system typically controls the operation of its surrounding system, although in some situations, such as the mobile telephone, the embedding system is primarily there to make the device portable. When the embedded 15

16 1.1. EMBEDDED SYSTEMS Introduction KEY Business Unit Organisation Role Role Customer System Customer System requirements System System Supplier System Supplier Component Customer Component requirements Components Design Process Component Supplier Component Supplier Figure 1.1: Generic Design Process Context system controls the embedding system, it is called an embedded control system. In customer and supplier terms, the developer of the embedded control system supplies that system to the embedding system developer, as depicted in Figure 1.2. That system is then supplied to yet another customer, whose needs have to be reflected to some extent in the embedded control system requirements. The embedding system developer is responsible for translating those needs into the specification of a control system, and for integrating that control system with the embedding system. This type of control system is implemented as a combination of control hardware and control software, which increases the flexibility of the system at the cost of additional complexity. This complexity is illustrated in Figure 1.3. Here, the software is encapsulated within an execution unit, which is just one of a number of systems from a range of suppliers that make up the control system. It can be seen, then, that complexity arises not only from the software/hardware split, but from a range of sources: The software executes on a processor with processor execution support from other hardware such as watchdog timers and specialised input/output connections. Some control functions may be implemented using hardware devices. These may supply inputs and/or outputs for the software, or they may function completely independently. They may be implemented using a fixed technology such as custom-made integrated circuits, or use a more flexible technology such as field-programmable gate arrays (FP- GAs) 1. 1 FPGAs are a type of integrated circuit that can be reprogrammed after manufacture. 16

17 Introduction 1.1. EMBEDDED SYSTEMS KEY Business Unit Organisation Role Role Customer System Customer System requirements System Indirect customer Embedding System System Supplier Control System Customer Direct customer Control system requirements Control system Supplier Embedded Control system Control System Supplier Figure 1.2: Embedded System Customers KEY Business Unit Role Role Control system Control System Supplier Control Unit Customer Software and execution requirements Software system Software execution Hardware Customer Control Unit Supplier Software Customer Hardware device Control Unit Supplier Execution hardware requirements Hardware Software requirements Software Software Hardware Supplier Software Software Supplier Figure 1.3: Software and Hardware Control 17

18 1.1. EMBEDDED SYSTEMS Introduction There may be any number of maintenance, fault diagnosis and alternative function provisions (such as pre-flight checks in an aerospace application) that require additional functionality alongside and within the software. Sensors and actuators are connected, perhaps through intermediate media, and through input and output connections, to the software. Control devices implement the control functions needed in the physical world, manipulating the embedding system. Some are under the control of actuators connected to the embedded control system, but others are controlled independently, such as an emergency power-off switch for safety reasons. The control system may employ some physical redundancy or distribution to improve its fault-tolerance and performance. The system being controlled may be a part of a larger collection of systems, all of which must take into account details of the others operation. An assembly system must have details of the sources of the parts that it assembles. An engine system must be developed according to the vehicle it powers. A mobile telephone must interface with a telecommunications network. The system may be used in different ways at different times. Most embedded control systems will have an engineering mode in which additional information is presented and maintenance functions can be accessed. Some may also have to correlate their mode with that of the other systems with which they co-operate. Each of these sources provides additional complexity in the form of information that must be taken into account during the design of the software. Some of this information takes the form of interface definitions, some is mixed in with requirements definitions, and some is domain knowledge that might never be recorded explicitly. This information provides the context for the design process. The term context is fairly general, and will be qualified where possible; some examples of such qualification are: Context Information that is taken into account when judging a particular design. System Context All information that is taken into account when designing a whole system. This could also be viewed as domain knowledge. Feature Context All of the the information that pertains to a single feature. Sufficient Context Enough explicit contextual information to be able to judge a design effectively. This terminology is necessarily subjective; the unifying characteristic of design is that it is performed and understood by individuals, and each individual has a different view of the design. 18

19 Introduction 1.2. OPERATING CONSTRAINTS It would be tempting to assume that all of the context can be made explicit, or even that all known tacit context can be named and referred to. The definition of sufficient context above replaces this assumption with a weaker, more useful assumption that enough of the context can be recorded to allow any designer in the same profession to understand the way in which the system has been designed. Experiments involving system context have shown that at most ten percent of that context will be represented explicitly in the resulting code [69, p569]. In the development of an embedded system, there is a danger that important dependencies between embedding system elements will not have an explicit representation in the implementation. With such a poor description of the traceability between features of the embedding system and the required properties of the embedded system, the accuracy of change impact analysis will be severely reduced. 1.2 Operating Constraints The systems studied in this thesis are engine control systems produced by Rolls-Royce plc. These systems are developed under a number of significant constraints, and these constraints apply to some degree to the work discussed here. In the development process for a safety-critical system, human responsibility for the behaviour of the system is crucial. There must have been human involvement in every aspect of the resulting product, to some degree. Purely automatic generative techniques would need some kind of external assessment; they would only be of benefit if they were significantly easier for a human engineer to assess than if the engineer were more directly involved in the design process. Instead, the work in this thesis will focus on improving the ability to judge design effort that is being reused. One of the important features of a safety-critical product is the argument that is put forward to support the claim that it is safe. This is typically based on evidence that shows that certain processes have been carried out and that certain properties of the system hold. The methods proposed in this thesis should complement this process, providing a way of identifying design decisions and properties, and linking them into the argumentation process. Each product that is produced matches with one of a number of customers, and a subset of a small number of suppliers. There is no guarantee that any one of these other organisations will be amenable to family-related descriptions of the control product. Even if a customer is amenable to a family-related description, it is likely that they will impose their own model for the description of families, regardless of any such model in use within the supplier organisation. Hence, issues that relate to the arrangement of the products as a family should be kept separate from issues about the decisions that affect a single product. This is especially important in situations where parts of the system are sometimes under the control of another organisation. Safety concerns are dealt with in several steps throughout the development, each step assessing a design description and propagating derived requirements to the next step. This causes 19

20 1.3. SAFETY-RELATED SOFTWARE Introduction an interweaving of requirements and design information throughout the development process. Consequently, there is no single point at which the process stops dealing with problem descriptions and starts dealing with solution specifications. Even though there is a clear problem space and a clear solution space, the process does not flow neatly from one to another. Any such separation would be extremely contrived given the nature of the process, and so the techniques developed in the thesis will ignore the use of the separation between the two spaces as way of managing complexity. There are a number of different languages and notations in use for the description of the different members of the control system family, for different stages of development, and for different parts of the control system itself. There is no single representative notation from this set that would be simple to introduce for the convenience of a broad readership. Instead, UML notations will be used to describe products, even though UML itself is not yet in widespread use in the safety-critical systems field. 1.3 Safety-related Software Embedded control software will often control a system that, in some situations, could directly lead to injury or loss of life. Examples include nuclear reactors and aircraft engines. To provide assurance that such systems are safe to operate, monitoring authorities and legislation are put in place so that the relevant authority must certify the system as safe before it may be put into operation. Certification involves the presentation of an argument claiming that the system is safe. The argument appeals to the requirements laid down in the relevant safety standard, and supports this claim with evidence about the system s development. Together, these form a safety case. While evidence about the software is necessary, it alone cannot show that the software is safe, nor that the system is safe. Evidence about the software is combined with evidence about the system to show that the system is safe, and that the software plays an appropriate role within the system. In many cases, tools will have been used to generate some or all of the implementation. Such tools must be demonstrated to have the same level of integrity as the implementation itself. This places an additional burden on the software developer in choosing and using development tools. Software safety evidence will typically involve software provisions to mitigate against faults and hazards in the system, and analysis to show that the software itself operates correctly in its context. The need to be able to perform this analysis imposes constraints on the way the software is developed. This ensures that the appropriate analysis can be carried out. Specification conformance is one aspect of this analysis. This shows that an implementation meets its specification. Testing is used to gain confidence that this is the case, but testing cannot generally provide evidence of complete conformance. If necessary, this can be shown through 20

21 Introduction 1.4. FAMILIES proof. Proof, and other kinds of analysis, impose a number of very specific and important constraints on the implementation: All memory allocation is statically bounded. Any use of dynamic memory allocation must be shown to have a static upper bound, to ensure that the software will never try to use more memory than it has available. In particular, the stack size during execution must be statically bounded. This means that calling structures must be carefully examined, and the use of recursion as an implementation technique is severely limited. All loops are statically bounded. A control system must operate in real time, and therefore deadlines from the embedding system, the requirements and the devices that support operation and execution of the software must all be met. Schedulability analysis is used to show that deadlines are being met, and that requires worst-case execution times for each task an unbounded loop prevents the calculation of this value. All code is reachable. This helps to eliminate the possibility of untestable code in the implementation. Execution paths are statically determined. This is necessary for the effective application of proof of specification conformance, and it prevents the general use of reflection and dynamic dispatch as ways of promoting flexibility in the implementation. Increasingly, these constraints are being met by modern compiler technology such as analysis of garbage collection for real-time systems [58, 75] and the use of optimising compilers to reduce dynamic dispatch to static method calls wherever possible. However, to use such technology it must be effective in worst-case situations, and it must be demonstrated to be safe to at least the same level as the implementation that it creates. Neither of these conditions hold in general at present, and without them compiler technology cannot be use as the basis of a family-oriented approach in this domain. 1.4 Families Families of systems are groups of systems that share common characteristics, typically the types of operation that they perform, the types of information that they manipulate, or the types of role that they play. The similarities between family members are exploited to increase the potential for reuse and automatic generation within the family. As with any reuse-based technique, there is an initial set-up cost that the organisation is expected to recover through cost savings on subsequent projects. The technology used to support family-based development is generally based around automated tools. This is a significant issue for safety-related software development, but also for embedded software development as well: 21

22 1.5. CHANGE CHARACTERISTICS Introduction There will already be a number of tools in use for the development of the control software and the rest of the control system. If family automation software tools are unable to work alongside those tools, there will be significant resistance to their use within the organisation. As explained previously, since the tools generate the implementation, or assist in generation, they must be demonstrably correct and safe in doing so. This also imposes an additional cost in obtaining and maintaining development tools. The generated implementation must obey all of the constraints set out previously regarding static determination and reachability. Current family generator technology does not necessarily meet these constraints well. Additional work would be required, firstly to check whether constraints are being met, and then to perform any changes to the technology to ensure that they are being met. This extra work would obviously increase the cost of using family member generation tools. For safety-critical software, the important issue is how to organise the information needed to correctly build and assess the software. The focus of the current generations of family management systems is product reuse, run-time dynamism and automated generation, none of which provide better facilities for the assessment of the software product. Hence, none of these technologies is considered to be an adequate solution, although they may eventually support the solution that is developed. When talking about family development, there are two levels that must be distinguished. Firstly, there are decisions that pertain to the development of each individual family member. Then, there are choices that determine the particular member of the family to be built. In this thesis, the terms choice and decision will be used consistently in this manner; choices select among different family members, while decisions are about the development progress of an individual family member. This is especially important when discussing rationale the rationale for decisions for a particular family member will typically be based on trade-off analyses and other engineering judgements, whereas the rationale for a choice among family members is dependent on the customer domain. 1.5 Change Characteristics There are a number of stakeholders that influence the development of embedded control software, and changes may arise from any of them. The stakeholders and the kinds of change they generate for this type of development are typically as follows: Customer The customer will make changes in the specification of the system in reviews during development, validation during deployment, and upgrades during operation. If the customer is integrating the embedding system with other systems, there will also 22

23 Introduction 1.6. HYPOTHESIS be changes arising from misunderstandings or errors in that interface specification too. These changes may arise and be dealt with at any level in the chain of customers and suppliers that leads to the software development. The nature of the domain of interest to this thesis is such that these changes are typically small, frequent and widely distributed across the specification. Certifier A certification authority can cause change either by altering its safety requirements, or by rejecting a safety case based on a failure to provide a compelling argument with respect to some of those requirements. System Integration between the control system and the rest of the embedding system can cause change, at any point during development. Most commonly, changes will occur when a full integration is attempted, and these tend to cause a change in the control software rather than a change to the embedding system the software is generally seen as being more amenable to change. Direct Supplier The most direct supplier for the control software is in fact the supplier of the hardware execution platform. Changes in the specification or availability of the processor or its support hardware, or even a change to the speed of the processor (e.g. to accommodate heat management requirements) may affect the software. Indirect Supplier Indirect suppliers are those that supply other parts. These include the actuators, sensors and control devices of the control system, plus the suppliers of other devices and subsystems for the embedding system. These details are all reflected in the software in some way, and changes in their behaviour or even in the way in which they are composed will affect the software. To be effective in managing change, it must be possible to address and assess changes that arise from any of these sources. However, unless the software is organised with respect to the structure of each of these sources, those changes will be spread out across different parts of the software. In practice, this kind of organisation has been found difficult to achieve, with architectural style [74] and aspect orientation [49] being used to address these multiple dimensions where necessary. 1.6 Hypothesis Software development in this field is beset by a number of closely-related problems: Dependencies among entities in the system context, especially in other control system components, are not necessarily reflected in the interfaces within the software. This reduces the accuracy of change impact analysis based on design and implementation results alone. 23

24 1.6. HYPOTHESIS Introduction family domain feature domain Choice select Feature Figure 1.4: Preliminary Concept Summary The constraints of safety-related software development prohibit the use of most techniques currently employed to give flexibility to a software implementation. Constant, small changes from a variety of sources mean that many changes are made in potentially disparate parts of the software implementation. Thus there is a need for techniques that address these problems. They mainly concern the frequency of change and the way in which its impact is determined. Hence, this thesis will address the following hypothesis: In families of safety-critical embedded systems, change impact can be predicted more accurately by modelling the relevant context for each feature early than by only analysing context when features change. To test this hypothesis, a model is required for the representation of features and their relevant context. This is addressed in Chapter 3 with a decision-based development model. The features must combine to form family members; the description of allowed combinations is explored in Chapter 4. The general relationship between the description of families and that of features is shown in Figure 1.4, in terms of related domains and the elements that they contain. Chapter 6 studies the way in which features combine. Finally, in Chapter 7, the accuracy of the feature model is assessed by comparing its impact analysis properties with data from projects in which impact analysis is not based on an explicit model of system context, but instead performed by domain experts directly. 24

25 Chapter 2 Literature Review The introduction in the previous chapter identified problems in representing system context information, performing impact analysis, and dealing with small changes from a variety of sources. Aspects of these issues are addressed in existing literature; this chapter reviews the relevant literature to put those issues in context and identify the approaches taken to address them. Where those approaches could conflict with the requirements of a safety-critical system design process, those conflicts are also described. Section 2.1 reviews design processes and the approaches taken to record system context information such as design rationale. In Section 2.2, domain-based techniques are reviewed, and the dependency models used in family-based techniques are assessed. Section 2.3 examines change impact analysis techniques. The major findings are summarised in Section Design Process Development Lifecycle Models Early work on the nature of software design described it in terms of stages of development, from a less well-developed description to a more complete one. This view, initially coming from similar development processes used in industry, gave rise to the first attempts at defining a software development process model. Benington describes a program production process used for the development of a real-time defence system during the mid-1950s [8]; the diagram in Figure 2.1 is adapted from Figure 4 of that paper. In this process, the operational plan and operational specifications represent the bulk of the requirements. The machine specifications step of the process is indicative of the expensive, non-standardised computing hardware of the time. Further levels of refinement are given in terms of programs and code, followed by a number of testing and integration stages based on the specifications and plans developed earlier. Interestingly, the following observation is made about testing: 25

26 2.1. DESIGN PROCESS Literature Review OPERATIONAL PLAN MACHINE SPECIFICATIONS OPERATIONAL SPECIFICATIONS PROGRAM SPECIFICATIONS CODING SPECIFICATIONS CODING DESIGN TESTING PARAMETER TESTING (SPECIFICATIONS) ASSEMBLY TESTING (SPECIFICATIONS) SHAKEDOWN SYSTEM EVALUATION Figure 2.1: Program Production [8] It is certain that the program will never be subjected to all possible input conditions during its lifetime. For this reason, one must accept the fact that testing will be sampling only. [8] This forms a basis for later assertions about the adequacy of testing as a method for eliminating errors. However, the process as described does not show how to manage the results of tests and feed that information back into that process. A later work on software development introduces a complete waterfall model [82]. The paper starts with a simple model based on observations from other work (including the paper introduced above) and elaborates it in accordance with a number of recommendations: 1. Complete program design before analysis and coding begins. This equates to the modern practice of responding to requirements with an architectural design before deciding on more detailed implementation matters (although architectural models emphasise structure and decomposition [32].) 2. Documentation must be current and complete. Even in this early model, documentation of the activities at various stages is considered extremely important; the author claims that if a review of the documentation finds it to be inadequate, all other activities must be suspended to make way for documentation improvement an even more radical notion today than it might have been at the time. There has always been a need for documentation, especially with the move to record more design rationale [67]. 26

27 Literature Review 2.1. DESIGN PROCESS 3. Do the job twice if possible. The idea of prototyping and iterative product refinement had been in place in the manufacturing industry well before this work, but this is an early attempt to document that process for software development. 4. Testing must be planned, controlled and monitored. In this version of the process, tests are planned according to the specification of the program design (the architectural design introduced above.) Previous models did not explicitly describe a test planning process, now regarded as an important aspect of testing. 5. Involve the Customer. The constant interest in requirements elicitation techniques [22] indicates that the problem of incomplete, inconsistent customer requirements that are prone to change still pervades software development. This recommendation allows for customer reviews throughout the process to ensure that the development is still relevant to the customer. The final version of the model is shown in Figure 2.2, adapted from Figure 10 of the original source material. In a paper aimed at debugging techniques, Mills [66] prescribes a top-down process based on the idea of describing a program in terms of standard control structures and code elements, and calls to subprograms. These subprograms are then described in the same manner, until no further subprogram calls are left. This is a kind of refinement process, and indeed it was heralded as a way to manage program complexity, and also as a vehicle for structured inductive program proof if the program control structures and code elements meet the requirements, assuming that the subprograms meet their requirements, then the program as a whole meets its requirements. This is one of the more rigorous interpretations of the top-down development paradigm, interestingly presented in terms of the single linguistic framework of the final implementation. A different take on the software development process is found in a paper on system decomposition [70]. Here, information-hiding criteria are compared to the temporal modularisation produced by classical refinement. Information hiding was found to produce designs in which modules could be easily reused to form a family of similar programs. This work emphasises the need to consider families of programs; it even goes as far as to make a recommendation about the way in which design decisions ought to be tackled: We propose... one begins with a list of difficult design decisions or design decisions which are likely to change. [70] The intent here is to ensure that those decisions are encapsulated in modules with more stable interfaces before making use of those modules, and hence isolating changes. The notion of a software engineering process, and a comprehensive description of that process, appeared in 1976 [9]. However, reasonably mature process models describing development and feedback were in use throughout the 1970s Feedback was a part of the waterfall model 27

28 28 Figure 2.2: Waterfall Model [82] SYSTEM REQUIREMENTS GENERATION DOCUMENT NO. 1 SOFTWARE REQUIREMENTS SYSTEM REQUIREMENTS DOCUMENT NO. 2 PRELIMINARY DESIGN (SPEC) SOFTWARE REQUIREMENTS DOCUMENT NO. 3 INTERFACE DESIGN (SPEC) PRELIMINARY PROGRAM DESIGN PSR PRELIMINARY SOFTWARE REVIEW DOCUMENT NO. 4 FINAL DESIGN (SPEC) ANALYSIS DOCUMENT NO. 4 FINAL DESIGN (SPEC) PRELIMINARY DESIGN ANALYSIS PROGRAM DESIGN CSR CRITICAL SOFTWARE REVIEW PROGRAM DESIGN CODING DOCUMENT NO. 5 TEST PLAN (SPEC) TESTING USAGE CODING TESTING FSAR FINAL SOFTWARE ACCEPTANCE REVIEW OPERATIONS DOCUMENT NO. 6 OPERATING INSTRUCTIONS 2.1. DESIGN PROCESS Literature Review

29 Literature Review 2.1. DESIGN PROCESS as described by Royce [82], and became an explicit organisational feature in the description of the V model [87]. This model takes the stages of development from a waterfall-style process model, and arranges them in a V shape: This format is intended to depict the complementary processes of problem decomposition on the left and system recomposition on the right. [87, p38] The diagram in Figure 2.3 shows the original arrangement of activities in the V model. A key aspect of this process model is that the recomposition activities include testing and validation, taking into account information from the corresponding decomposition stage. If these processes fail, feedback is given to that decomposition stage so that the problem may be addressed. For some situations, however, there may be no way of addressing the problem at that stage, and the software engineer must revert to an even earlier design to try a different approach. This design backtracking phenomenon is identified in a paper that attempts to formalise the design step in terms of a restatement between linguistic systems [53]. Here it is shown that situations can exist in which no further progress can be made in refining a design to a new form because of the forms that the design has taken along the design path. A similar situation can occur in program families; certain decisions may be made early in the design that preclude some subsequent decisions [71]. A radical shift in thinking from linear and iterative process models occurred with the introduction of the spiral development model [10], shown in Figure 2.4. This model organises development into cycles of assessment, analysis, prototyping, implementation, testing and planning, with commitments being made in moving from one cycle to another. The emphasis in this model, which is usually specialised for use with a particular development, is on iterative design cycles with consensus between stakeholders as risks are mitigated and commitments are made. Unlike the more sophisticated versions of the waterfall model, there is no explicit test planning early on, although that could be incorporated into the testing phase of early cycles to plan for the testing of later cycles commitments. Following on from these efforts, a number of standardisations of the software lifecycle process have been proposed [91, 44]. The most recent software lifecycle process standard is ISO12207 [45]. These efforts serve to standardise the terminology of process models and to ensure that all of the necessary lifecycle components have been accounted for. They must still be instantiated for use in a particular situation, however. Models of the development lifecycle represent the development of a single product, from concept to realisation. There is no support in the standard lifecycle models, nor in the standards documents, for the development of families of systems. Change is addressed, but in terms of validation and rework rather than specifically taking into account the differences between family members. 29

30 30 Figure 2.3: V Model [87, p36] (HAC) SOFTWARE DEVELOPMENT LIFE CYCLE REQUIRED OPERATIONAL CAPABILITY (ROC) RESEARCH PHASE CONCEPTUAL PHASE DESIGN & DEVELOPMENT PHASE ENGINEERING STUDIES EXPERIMENTAL DEVELOPMENT CONCEPT EVALUATION SYSTEM REQUIREMENTS REVIEW (SRR) SRR SYSTEM DECOMPOSITION & ALLOCATION SYSTEM DESIGN REVIEW (SDR) SOFTWARE REQUIREMENTS SOFTWARE SUBSYSTEM DESIGN REVIEW PRELIMINARY SOFTWARE ANALYSIS * DESIGN PRELIMINARY DESIGN REVIEW (PDR) DETAILS SOFTWARE ANALYSIS & DESIGN CRITICAL DESIGN REVIEW (COR) CERTIFICATION VALIDATION VALIDATION VALIDATION VERIFICATION VERIFICATION VERIFICATION VERIFICATION CODING & DEBUG START CPCI TEST SOFTWARE CPCI TESTING SOFTWARE SUBSYSTEM INTEGRATION & TESTING TEST & EVALUATION PHASE SOFTWARE SYSTEM INTEGRATION & TESTING SOFTWARE/ HARDWARE INTEGRATION & TESTING SYSTEM PERFORMANCE TESTING ACCEPTANCE TEST & EVALUATION FUNCTIONAL CONFIGURATION AUDIT (FCA) END OF OT&E OPERATIONAL TESTING & EVALUATION (OT&E) CUSTOMER DELIVERY & PHYSICAL CONFIGURATION AUDIT (PCA) OPERATIONS & MAINTENANCE (KDTM) PHASE OPERATION & MAINTENANCE (O & M) 2.1. DESIGN PROCESS Literature Review