Stages in Teaching Formal Methods A. J. Cowling Structure of Presentation Introduction to Issues Motivation for this work. Analysis of the Role of Formal Methods Define their scope; Review their treatment in SE2004; Describe a theoretical model for the development of SE skills; Analyse the stages defined by this model. Conclusions What students need to learn about formal methods. Department of Computer Science University of Sheffield Limitations and Future Work The details of how formal methods should be taught. Motivation (1) Issues in Teaching Formal Methods It is difficult to define which methods are formal: all computing is based on computation and information, these are essentially formal concepts. Formal methods are valuable in some situations; but Successful systems are often built without them. Many formal methods are difficult to teach: they are based on mathematical concepts that are unfamiliar, and therefore the notation they use is also unfamiliar; Also, many of them operate separately from other SE methods: hence it is not clear to students how the formal methods fit in. Motivation (2) Negative Approach to These Problems Try to ignore them! This is dangerous, because: engineering must apply scientific principles, formal methods embody these scientific principles, (more so than many other aspects of SE), ignoring formal methods means ignoring these principles, which suggests that SE is not engineering. Positive Approach to These Problems Try to analyse the problems and overcome them: identify what students should be taught about formal methods, and when they should be taught them. SE2004 attempted this to some extent: more experience is now available, and a theoretical model for stages in developing SE skills. The Scope of Formal Methods (1) Possible Definition 1 Any method that is based on some formal mathematical model; But, information and computation are mathematical concepts: all methods are based on these concepts, so all methods would be classed as formal, even if abstractions hide the mathematical models, as in UML diagrams; So this definition is too general. The Scope of Formal Methods (2) Possible Definition 2 Any method that involves direct manipulation of a mathematical model; This eliminates purely diagrammatic methods; But it includes several other common methods: database normalisation (manipulates the relational model of the database structure), checking test sets for coverage (coverage criteria are defined from formal models of system structures); And these methods are not usually classed as formal; So this definition is still too general. 1
The Scope of Formal Methods (3) Possible Definition 3 Focus on the purposes of the models, as well as their types; Any method that involves: explicit creation of a mathematical model of a system, in order to check multiple properties of the system; Excludes single-purpose models: eg database normalisation or test set coverage. Implies that a formal method should involve: building some formal specification of a system (once its requirements have been analysed), using this specification during system development (ie design and construction); The Scope of Formal Methods (4) Implications of These definitions The differences reflect a distinction between CS & SE: definitions 1 and 2 focus on CS, definition 3 focuses on SE. Formal Methods in Computer Science Science creates models of concepts; It uses them to analyse their properties. Formal Methods in Software Engineering Engineering builds models of systems It uses them to make design decisions. The Scope of Formal Methods (5) Contrasting Roles Mathematicians prove, engineers calculate (Hoare): but, for engineers these calculations may involve proofs that required system properties will hold; Formal methods in SE must therefore support: calculating properties of systems, based on formal models of them; They therefore require both validation and verification: validation of the system models, verification of the calculations; The adopted definition captures these properties. FM in Curriculum Models (1) Formal Methods in CS2001 Defines fundamental mathematics knowledge; Defines fundamental formal concepts: information and computation; Defines some formal models for CS: finite-state automata, invariants, assertions and contracts. FM in Curriculum Models (2) Formal Methods Knowledge in SE2004 It imports fundamental material from CS2001: Mathematical and Engineering Fundamentals knowledge area, Computing Essentials knowledge area; It includes some applications of this material: state-based construction techniques, in CMP.ct.7, design by contract, in CMP.ct.5; It requires a general introduction to specification languages in MAA.rsd.3; It relates uses of formal methods to processes: formal validation of requirements, in MAA.vv.5, formal design analysis, in DES.ste.4 (optional), derivation of metrics from models, in VAV.fnd.4, applications to testing, in VAV.tst; But, it only allocates limited time to any of these. FM in Curriculum Models (3) Formal Methods in SE2004 Courses Fundamental mathematics covered in imported courses: Discrete Structures (CS105, CS106); Not tied to the introductory SE courses (SE101, SE102) but brief coverage in the alternative course (SE201); One core package introduces formal methods early: formal construction techniques in Software Construction (SE211), other applications in Software Quality Assurance (SE321) and Software Requirements Analysis (SE322); The other core package covers them later: some applications in Software Testing (SE221), these come before specification languages, which are in Formal Methods in Software Engineering (SE313). 2
Developing Knowledge & Skills (1) A Theoretical Framework Developed through reflection on the curriculum at Sheffield and particularly on its key structural feature; Curriculum based on a sequence of major projects: one per academic year, each focuses on developing a complete software system; These projects form the spine of the curriculum: other courses are fitted in round them, and provide the necessary knowledge and skills. Developing Knowledge & Skills (2) Framework Structure Based on two characteristics of the system developed: What issues need to be considered in the development: initially, just functional correctness and basic usability, later on, quality requirements as well; Scale and size of the systems: simple programs, performing single functions, set of functions, using persistent storage, larger scale systems. Developing Knowledge & Skills (3) Four Stages of the Framework Stage 0: basic programming; Stage 1: Software Development: covering a full life cycle, just concerned with functional correctness and basic usability, essential to all computing programmes, not just SE ones; Stage 2: Software Engineering: covers meeting quality requirements as well, scale of systems constrained by the limits of a degree programme; Stage 3: professional scale software engineering: the eventual goal, but beyond the scope of this paper. Formal Methods in Programming A Clash of Definitions Programming focuses on the construction stage; Formal methods should be applicable at multiple stages; Hence, their application to programming is very limited. The Role of Formal Methods in Programming Formal methods should not be ignored completely; Single programs still need to be specified; Fundamental concepts such as invariants can be applied; Even though SE2004 does not require this. FM in Software Development (1) Focus of Software Development Functional correctness and basic usability; In principle, formal methods should be applicable to these; But, their formal models must be validated: against the requirements for the system, which is also a key to ensuring usability, but neither can contribute much to the other; In practice, therefore, the role of formal methods can only be in helping to ensure functional correctness. FM in Software Development (2) Achieving Functional Correctness This involves three issues: How can requirements for systems be specified? How can these specifications then be used in development? to ensure the systems conform to their requirements; How can the specifications then be validated? the process will be independent of the kinds of models. Specifying Requirements Involves two stages: requirements elicitation, and requirements modelling, where the set of models forms the specification; Properties of the set of models: must define the required behaviour precisely, so that conformance can be determined unambiguously. 3
Modelling Requirements (1) Basic UML Diagrams Class diagram: identifies required external structure of persistent data; Use case diagrams: identify required functions; Possibly activity diagrams: identify constraints on the order of function invocation. Limitations of These Models They do not define the required behaviour of functions; In particular, they do not define how functions: need to use the persistent data, or must change the persistent data. Modelling Requirements (2) Additional Models Needed to define the behaviour precisely; Can identify two groups: those that stay within the scope of UML, and those that use other models. Further UML Models State machine diagrams can describe: steps in carrying out a function, or how functions change the state of objects; Collaboration diagrams can describe: sequences of interactions between users and the system, in the form of system sequence diagrams; OCL annotations can define aspects of behaviour. Modelling Requirements (3) Other Models Structured text, as story cards or structured use cases; Formal specifications in some suitable language. Evaluation of the Additional Models System sequence diagrams, or similar state machines: illustrate the interactions, but do not define their effects; State machines for classes: usually produced during the design stage, hence not usually part of the requirements specification, but should be validated against a specification; Structured text models: inherently imprecise; Hence, none of these achieve the goals; Formal methods are needed. Modelling Requirements (4) Ensuring Functional Correctness Design stage models should be verified against the specification: state machine models of functions or classes, detailed sequence diagram models, methods for doing these may be advanced material; Testing during the construction stage: test cases must be consistent with the specification, test sets should cover the specification, measuring coverage may be advanced material; Informal specifications may not be adequate; Formal specifications should be: precise enough to make these checks unambiguous, complete enough to allow all of them. Formal Methods and SE2004 (1) The Need for Formal Methods Required to describe dynamic behaviour precisely: in particular, to define how functions affect persistent data; Required for designing and testing systems: to verify designs, to ensure correctness of implementations; Required pedagogically to encourage good practice: avoid students believing that formal methods are unnecessary. The Structure of SE2004 Introductory course sequences lead to core courses; Introductory course sequences cover software development: not a precise equivalence, but is implied by the descriptions of SE200 and SE201; Introductory courses do not require any formal methods; So SE2004 does not meet the need for formal methods. Formal Methods and SE2004 (2) The Software Engineering Stage Corresponds to the SE2004 core course sequences; These are where formal methods are covered: although some key topics are not core knowledge. Consistency with the Framework Programming stage: Some formal methods can be used, but SE2004 does not require them; Software Development stage: formal methods are needed, but SE2004 does not require them; Software Engineering stage: SE2004 is more-or-less consistent. 4
Conclusions The Need to Study Formal Methods Even simple introductory processes must include them: to define basic functional behaviour, and in particular how functions affect persistent data, to give a basis for ensuring functional correctness; They must be introduced early in the curriculum: some elements are needed in the introductory courses, which is earlier than SE2004 recommends. The End Thanks for your attention!! Any Questions? Further Questions Which elements of formal methods must come early? Requirements for the methods have been defined, but: how do different methods support these requirements? how should formal and diagrammatic methods be balanced? These need further analysis. 5