Runtime Verification of Computer Programs and its Application in Programming Education

Transcription

1 Runtime Verification of Computer Programs its Application in Programming Education Magdalina V. Todorova, Petar R. Armyanov Abstract The paper presents a technique for runtime program verification its application in computer programming education. This approach is experimentally introduced in computer science curricula at Faculty of Mathematics Informatics, Sofia University "St. Kliment Ohridski". Runtime verification combines formal verification methods with program execution. The proposed formal verification method is based on axiomatic semantics, enriched with techniques such as design by contract class invariant. Specification of pre- postconditions, the invariants, the loop termination functions are implemented by assertions. The presented approach is illustrated by examples for runtime verification for both traditional procedural programs, object-oriented programs in C++ programming language. Index Terms computer science education, formal verification, runtime, software design, software engineering A I. INTRODUCTION mongst the most important requirements for qualities of a programming system are: correctness, security, stability, efficiency in usage of time memory, adaptability, portability. For a system to be correct, it needs to correspond to its initial purpose. The latter is often ensured by tests visual analysis. The testing of a programming system consists of analysis of its processing a given set of input data. Practice shows that each of the different stages of a testing provides less than 50% of the found errors [1]. In a study [2], which is widely cited as classical, Myers asked a group of experienced programmers to test a program, which contained 15 known errors. The average number of found errors was 5, the best programmer found only 9 errors. Thus, testing might help to find some errors, but throughout as it may be, it never guaranties the absence of errors. In various programming systems, the evidence of even minor errors is strongly prohibited. For example, an error occurring in systems such as: space apparatuses, systems for managing nuclear powerstations, systems for managing secret data bases, systems for electronic trade, others, may lead to economic losses or loss of human lives. Systems like these, where the cost of any error is very high, motivate using a principally different method from testing on the programs. Such a method is the formal verification. Formal verification of a program in respect to an input/output specification means proving that the program is partially correct according the input/output specification, that it completes its execution in respect to the input specification. It is also said that the program has total correctness according to the input/output specification. T he program is partly correct in respect to input/output specification when the program execution starts with input data, satisfies given preconditions terminates with the output specification correct. T here are many ways of performing verification. For educational purposes in the field of programming, a method combining elements of formal verification program execution has been applied on the courses Introduction in Programming, Object-Oriented Programming Data Structures for students of Informatics in Sofia University. The method is based on associating certain statements with the program (or with parts of it), which must be valid through execution. This approach is often preferred to pure formal methods of verification, which are expensive difficult to implement. II. DESCRIPTION OF THE APPROACH The approach integrates axiomatic semantic with the techniques of design by contract invariant of class. It focuses on the change of variables values during the program execution, so it can be considered as a dynamic approach. A non-formal description of this approach suitable for educational purposes is given in [3]. The inclusion of preconditions postconditions, as well as invariants (for cycles classes) loop termination functions is done through assertions. The syntax of the latter is in accordance with the programming language C++, which the students' courses are based on. A. Axiomatic semantics There are two ways of defining axiomatic semantic defined by Robert Floyd C.A.R. Hoare. For this research we use Hoare s [4] definitions transforming predicates defined by D. Gries [5]. The axiomatic semantics of a program is most often connected with the proof that the program is total correct (or only correct) when a plain static analysis of its text is performed. It is based on a condition, called a predicate, describing the program state (programming fragment or operator) in a moment of its execution. An axiomatic specification is connected with a program S: {PRE S {POST, called also Hoare's triple. Here PRE POST are predicates describing respectively the precondition GSTF

2 GSTF Journal of Mathematics, Statistics Operations Research (JMSOR) TABLE Vol.1 I No.1 AXIOMATIC SEMANTICS postcondition connected with S. A precondition is a predicate, which describes the conditions to be fulfilled at the moment before the execution of the program (the programming fragment or the operator). The postcondition is a predicate describing the calculation process in the end of the program execution. The pair <PRE, POST> is called input/output specification (or simply specification) of S. The semantics of a Hoare's triple gives the total correctness of the program (programming fragment) S with respect to the particular specification. In general, for a given program S a postcondition POST, there are numerous preconditions PRE, for which it is true that {PRE S {POST. There is one precondition, however, that is the most general or weakest assertion which satisfies {PRE S {POST. This is called the weakest precondition of the program (program fragment or statement) S the postcondition POST. It is written Wp(S, POST), where Wp is a function, called also a predicate transformer. Using these definitions, we have the property: {PRE S {POST if only if Wp(S, POST) where => means implication. The following subset of C++ statements is used for the purposes of runtime verification training: empty statement, assignment statement, compound statement (block), conditional statement loop statement. The rules for finding the transforming predicates of the statements above are given in [5]. The corollary rules defining the axiomatic semantics are described in Table I. Example: Prove the correctness of the Hoare's triple {n > 0 int S = 0; int i = 1; while (i <= n) { S = S + i; i = i + 1; {S == n*(n+1)/2 Note: The brackets of while are bold in order to differ from the brackets of the Hoare's triple. This program fragment calculates the sum n, for a positive integer n. We chose the following loop invariant I loop termination function t for the while loop statement: I: S == (i-1)*i/2 && 1 <= i && i <= n+1 t: n+1-i We will use the statement if {P S1 {Q {Q S2 {R then {P S1; S2 {R will prove: {n > 0 POST Premise Conclusion Wp(empty_statement, POST) domain(e) POST(x e) Wp(x = e, POST) Wp(S1,Wp(S2;...;Sn, POST)) domain(b) AND PRE B => Wp(S1, POST) AND PRE B => Wp(S2, POST) I AND I B => POST AND I B => Wp(Stat, I) AND I B t > 0 AND I B => Wp(t_old = t; Stat, t < t_old) Wp({S1; S2;...; Sn, POST) Wp(if (B) S1; else S2, POST) Wp(while (B) Stat, POST), where I is the loop invariant, t the loop termination function int S = 0; int i = 1; {S == (i-1)*i/2 && 1 <= i && i <= n+1 {S == (i-1)*i/2 && 1 <= i && i <= n+1 while (i <= n) { S = S + i; i = i + 1; {S == n*(n+1)/2 The correctness of the first triple follows from trueness of the implication: n>0 => Wp(S = 0; i =1, S == (i-1)*i/2 && 1 <= i && i <= n+1) which is equivalent to (n>0) => (0 = 0 && 1 <= 1 && 1 <= n+1) In order to prove the correctness of the second triple, we have to prove the axioms for the while statement (Table I): S == (i-1)*i/2 && 1 <= i && i <= n+1&&!(i <= n) => S == n*(n+1)/2 S == (i-1)*i/2 && 1 <= i && i <= n+1&& (i <= n) => Wp(S=S+i; i=i+1, S==(i-1)*i/2&& 1<=i && i<=n+1) S == (i-1)*i/2 && 1 <= i && i <= n+1&& (i <= n) => n+1-i > 0 last S == (i-1)*i/2 && 1 <= i && i <= n+1&& (i <= n) => Wp(t_old = n+1-i; S = S + i; i = i+1, n+1-i < t_old) GSTF

3 In above implications the C++ operators &&! are used instead of logical operators The first implication is true, because from i <= n+1 &&!(i<=n) follows that i == n+1. As we have Wp(S=S+i; i=i+1, S==(i-1)*i/2 && 1<=i && i<=n+1) = S + i == i*(i+1)/2 && 1 <= i+1 && i+1 <= n+1 S == (i-1)*i/2, we prove the correctness of the second implication by using simple transformations. The third implication is true because from i <= n we have that i < n+1, so n+1-i > 0. And the last implication follows from Wp(t_old = n+1-i; S = S + i; i = i+1, n+1-i < t_old) = n+1-i-1 < n+1-i Thus we prove the total correctness of the program fragment. B. Design by contract Designing by contract is a technique for identifying interface errors in programs. B. Meyer presents it through numerous publications, connected with the design of objectoriented programs [6]. The idea in the case of two functions is as follows: the calling function is a client which requests for some service, the called function is a provider, who has to deal with the service under certain conditions. While using the service, the client must comply with given rules, the provider takes a particular amount of responsibility. Therefore, a contract is formed, which defines the obligations the rights, Table II. In its simplest outlook, the contract consists of a system of preconditions postconditions. The idea of designing by contract in the case of object-oriented programming is analogical. The objects methods of a class are most often used by the methods of other classes. The recent classes are called clients. Rules (contracts) have to be defined between Client Provider Obligations To fulfill the precondition To fulfill the postcondition TABLE II CONTRACTS Rights To have the requested result To receive input data, which comply with the precondition each class its clients. According to these rules, the client has to provide correct input data the class-provider has to find give back the requested result [6]. The client has to guarantee that certain preconditions will be true before the execution of the methods. In turn, the class-provider guarantees that certain postconditions will be fulfilled after the methods execution. The realization of designing by contract in C++ can be achieved by macro assert, by using exceptions or by combining both techniques. Although combining rules exceptions is the preferred technique [7], we use assertions for training purposes. Assertions are one of the most useful techniques for finding errors in a code. In addition, they provide information for the error location. The macro assert(<predicate>) in the language C++, performs the following actions: if <predicate> is true, the execution of the program continues. Else, the execution of the program ends with an error message, which includes its exact location in the code (file name line number), as well as the text of the predicate, that was not true. In addition, assertions are often eliminated from the program release builds, so not to affect the total performance of the application. C. Invariant of a class Each object in a class possesses multiple properties, which connect it with the class. These properties are defined by a predicate, called invariant of the class. The invariant must be true after the execution of each constructor, before after the execution of each method in the class. Let PRE i POST i are the precondition postcondition of the i-th constructor C i or the i-th method M i of a class; the invariant of the class, which these methods belong to, is Class_invariant. The class is correct for the respective predicates if: - for each constructor C i it is true that {PRE i C i {POST i Class_invariant - for each method M i it is true that {PRE i Class_invariant M i {POST i Class_invariant For example, a class, representing a fraction storing integers for a numerator denominator, can have a class invariant that enforces the denominator to be different from 0. We can use additional class invariant that requires the numerator denominator of the fraction to be mutually prime integers. III. METHODOLOGY OF VERIFICATION DURING EXECUTION The runtime verification consists of the following steps: - realizing the program (imperative or object-oriented); - building a formal specification for the program; - connecting the program with the formal specification, using an assert statement, which often uses preprocessor directives for the predicates; - executing the program according to the formal specification. The formal specification of the program is given in a language which is a subset of the first-order predicate calculus according to the syntax of the programming language used. Macro assert is chosen in order to add rules. This choice is made because assertions are among the most useful automatic GSTF

4 techniques for error identification, as well as for giving information about the error. A. Runtime verification of non-recursive functions In the approach we use, certain predicates are attached to every function we want to verify. They represent the preconditions postconditions of the functions, as well as a loop invariant loop termination function for each loop in the function body. Verification in respect to that specification means: to prove correctness of these attached predicates for the current values of the function arguments on each iteration of the loop execution. Example: For a function Sum that calculates the sum of the integer numbers between 1 n (defined below) the predicates: PRE: n>0 POST: S == n*(n+1)/2 we have to execute the function in addition to executing the assertions that hold the correctness of the predicates. #define PRE (n>0) #define POST (S == n*(n+1)/2) #define I (S==(i-1)*i/2 && 1<=i \ && i<=n+1) #define B (i <= n) int Sum(int n) { assert(pre); int S = 0; int i = 1; assert (I); while(i <= n) { int t_old = n+1-i; S = S + i; i = i + 1; int t = n+1-i; assert(i); assert(!b!i (t>0 && t<t_old)); assert(post); return S; int main() { for(int i = 1; i <= 10; i++) cout << Sum(i) << endl; return 0; The assertion of invariant I at certain positions in the code corresponds to the requirement that predicate I holds before after the execution of each iteration of the while loop statement. The statement assert(!b!i (t > 0 && t < t_old)); which corresponds to the implication B I => (t > 0) Wp(t_old = t; S = S+i; i = i+1, t < t_old), (where t: n+1-i) asserts that while the loop statement is executed, the value of the loop termination function t decreases. This value is integer positive, so the body of the loop will be executed finite number of times. In this case, the assertion of I is not necessary, but it is left for completeness. The statement assert(post) at the end of the fragment is used to prove the trueness of the predicate I B => POST The main function can be seen as a client of the service, provided by the Sum function. This service is provided according to a contract where both parties have rights obligations, shown in the Table III. Client main Provider Sum TABLE III EXAMPLE FOR CONTRACTS Obligations To satisfy the precondition: n > 0 To satisfy the postcondition: S == n*(n+1)/2 Rights To receive the requested result sum of the number 1 to n To receive an input that satisfies condition n > 0 n is integer. In the example this is a number from 1 to 10. More about the methodology other examples of a runtime verification of functions are given in [3]. B. Runtime verification of classes The invariant is defined for a class for each of its methods precondition postcondition. For each class constructor only the postcondition must hold at the end of its body. A loop invariant a loop termination function are connected to each loop statement in the body of a class method. In case some methods in the class refer to other methods in the same class, contracts are designed. The predicates are defined by the C++ language syntax are embedded in the code by macro assert. The following class, realized with the C++ language, defines an abstract data type queue. The queue is represented as a non-cyclic by means of a sequence of no more than MaxQue integer numbers a front, a front+1,, a rear-1, where front rear are pointers to the first element after the last element of the queue. In case front == rear, the queue is empty, if rear == MaxQue, the queue is full. A good choice of an invariant of the class is 0 <= front && front <= MaxQue && 0 <= rear && rear <= MaxQue && front <= rear It defines the existence of the queue. Each method of the class GSTF

5 is connected to a precondition a postcondition. Invariants loop terminating functions for cycles are not defined, because the methods of the class do not contain loop statements. In order to verify a class, we need to build the specification the class invariant into each method of the class. The class invariant must be true after each execution of the constructor, before after each execution of the other methods of the class. The following code is a sample implementation of the class queue with a check for its total correctness. The rule front <= rear in some of the preconditions is omitted, because the invariant of the class contains it. const int MaxQue = 5; #define class_inv (0 <= front && \ front <= MaxQue&& \ 0 <= rear && \ rear <= MaxQue&& \ front <= rear) class queue { private: int a[maxque]; int front, rear; public: queue() { front = rear = 0; assert(front == rear && front == 0 && bool empty() const { assert( bool res = (front==rear); assert(res==(front==rear) && return res; void push(int x) { assert(!full() && int rear_old = rear; a[rear] = x; rear = rear+1; assert(!empty() && a[rear_old] == x && rear == rear_old+1 && int pop() { assert(!empty() && int x = a[front]; int front_old = front; front++; assert(x == a[front_old] && front == front_old+1 && return x; bool full() const ; { assert( bool res = rear == MaxQue; assert(res==(rear==maxque) && return res; int length() const { assert( int res = rear - front; assert(res == rear-front && return res; The most difficult part in the realization of a runtime verification is defining the invariants of the loop statements. For training purposes, the methodology suggested in [5] is applied. Furthermore, suitable invariants of the classes in object-oriented programs are difficult to be defined. They have to be clear not to impose too many restrictions on the development efficiency of the class realization. Defining them is performed on the basis of non-formal analysis of the methods of classes. IV. TRAINING APPROACH The runtime verification education in the Faculty of Mathematics Informatics starts by the course Introduction in Programming continues through Object-Oriented Programming Data Structures. Students have to take these courses during first three terms of their study of Informatics. Each of the courses has workload of 75 lecture hours has a corresponding computer practicum course with workload of 30 hours in a computer lab. In 8 lecture hours of Introduction in Programming, students study the basics of axiomatic semantic assertion technique. They have to know the basics of mathematical logic. As a result of successful learning, students can verify simple programs written in C++ programming language. The design by contract technique is studied in 4 lecture hours of the Object-Oriented Programming course. 5 hours of practical sessions are dedicated to application of this approach to verification of classes. Verification of classes implementing data structures is studied in Data Structures course, in 5 lecture hours. As a result of the course, students acquire skills to apply runtime verification for programs that use certain abstract data structure implementations. Because of the complexity the discussed thematic, the approach project-based learning [8, 9] is used. The main reason for this decision is that the project-based method is among the most appreciated by the trainees. Its focus is on stimulating student interest in a particular task, requiring a stable base knowledge in the field of the project. By the means of working on the project, practical knowledge abilities are gained demonstrated. The development of projects on runtime verification has the following stages: - formulating the project (assignment of the topic, GSTF

6 objectives tasks of the project); - forming the groups of students to work on it, introducing the task of the project dividing the sub tasks within the groups; - instructing the trainees, as well as defining the criteria of project assessment; - working on the project; - developing project presentations; - presenting the projects; - assessment. The chosen training method gives the educational process an opportunity to apply procedures, which model the stages of thinking when solving a problem. For example: formulating the problem, suggesting hypothesis, verification of the hypothesis, formulating conclusions, practical application, summarization. Through this approach, training is following a scientific research model. During the training sessions, we discovered that students gladly seek non-trivial, non-conventional original ways for solving their project tasks, especially when searching for adequate invariants. While developing their projects, they often use scientific results, connected to the topic, but not studied during the training program. The most useful interesting part of the training is the discussion held while presenting the projects. We complement examination with computer testing [10, 11, 12], which is among the most promising, effective liked by the students methods of assessment. V. CONCLUSION As a result of the training, the students learn the basics of formal program verification, the techniques: predicate transformer, design by contract assertions. They achieve practical skills for runtime verification of imperative object-oriented programs. The subject viewed in this paper is important as it is an inseparable part of the process of developing real software products, especially when the price of each error is high. Studying these verification techniques in the beginning of the educational process is a step forward towards producing highly qualified software engineering experts. The suggested approach can be successfully integrated with the use of proper e-learning environments like the popular platform Moodle extended with tools for automation of the submission checking the results of programming assignments [13], digital libraries with learning materials supporting the basic courses in Programming Data Structures [14, 15], etc. REFERENCES [1] J. Capers, Estimating Software Costs, New York, NY McGraw Hill, [2] G. J. Myers, The Art of Software Testing, New York, NY John Wiley & Sons, [3] M. Todorova, Runtime program verification, in Journal of Mathematics Informatics, Published by the Bulgarian Ministry of Education, Sofia, 2009, No. 1, pp [4] C. A. R. Hoare, Proof of Correctness of Data Representations, in Acta Informatica, 1972, Vol. 1, N 4, pp [5] D. Gries, The Science of Programming, Springer-Verlag, Berlin New York, [6] B. Meyer, Applying design by contract, in Computer 25(10), 1992, pp [7] J. Torjo, C++: Implementing Design by Contract to reduce bugs, Available: 26.html, July [8] P. Azalov D. Richards, Project Based Teaching of Intermediate Programming, International Symposium IGIP/IEEE/ASEE 2004, September 27 30, Fribourg, Switzerl, pp [9] М. Todorova, H. Hristov, E. Stefanova, N. Nikolova, How to build up contemporary software professionals (Project Based Learning in Data Structure Programming), Second International Conference on Software, Services Semantic Technologies S3T, September 11 12, 2010, Varna, Bulgaria, pp [10] I. Peneva Kr. Yordzhev, Internet Testing latest trend in research psychometric, in Trakia Journal of Sciences, Vol. 7, Suppl. 2, pp [11] I. Peneva, K. Gaidarov, Kr. Yordzhev, Computer administering of psychological tests, in Proceedings of the Third International Scientific Conference FMNS2009, 3 7 June 2009, Blagoevgrad, Bulgaria, pp [12] Kr. Yordzhev, I. Peneva, B. Kirilieva-Shivarova, A Relational Model of Personality Psychological Tests, in Proceedings of the Third International Scientific Conference FMNS2009, 3 7 June 2009, Blagoevgrad, Bulgaria, pp [13] A. Semerdzhiev, T. Trifonov, M. Nisheva, Automated Tools for Assisting the Educational Process in Informatics, in International Conference on Application of Information Communication Technology in Economy Education ICAICTEE 2011 (Sofia, Bulgaria, December 2 3, 2011), ISBN , pp (in Bulgarian). [14] P. Pavlov M. Nisheva-Pavlova, Some IT Aspects of Building Digital Libraries with Learning Materials, in Review of the National Center for Digitization, Belgrade, ISSN , Vol. 13, 2008, pp [15] M. Nisheva-Pavlova P. Pavlov, Design Principles of a Digital Library with Learning Materials, in Proceedings of the Third International Conference on Information Systems Grid Technologies (Sofia, May 28 29, 2009), Sofia, 2009, pp Magdalina Todorova is associated professor in Faculty of Mathematics Informatics in the Sofia University, Bulgaria; ( todorova_magda@ hotmail.com). ACKNOWLEDGMENT The paper is supported by Grant Software modeling theoretical, applied methodological aspects from Sofia University Research Fund, Petar Armyanov is assistant professor also a PhD student in Faculty of Mathematics Informatics in the Sofia University, Bulgaria; ( parmyanov@fmi.uni-sofia.bg) GSTF