Continuity Analysis of Programs

Transcription

1 Continity Analysis of Programs Swarat Chadhri Pennsylvania State University Smit Glwani Microsoft Research Roberto Lblinerman Pennsylvania State University Abstract We present an analysis to atomatically determine if a program represents a continos fnction, or eqivalently, if infinitesimal changes to its inpts can only case infinitesimal changes to its otpts. The analysis can be sed to verify the robstness of programs whose inpts can have small amonts of error and ncertainty e.g., embedded controllers processing slightly nreliable sensor data, or handheld devices sing slightly stale satellite data. Continity is a fndamental notion in mathematics. However, it is difficlt to apply continity proofs from real analysis to fnctions that are coded as imperative programs, especially when they se diverse data types and featres sch as assignments, branches, and loops. We associate data types with metric spaces as opposed to jst sets of vales, and continity of typed programs is phrased in terms of these spaces. Or analysis redces qestions abot continity to verification conditions that do not refer to infinitesimal changes and can be discharged sing off-the-shelf SMT solvers. Challenges arise in proving continity of programs with branches and loops, as a small pertrbation in the vale of a variable often leads to divergent control-flow that can lead to large changes in vales of variables. Or proof rles identify appropriate synchronization points between exections and their pertrbed conterparts, and establish that vales of certain variables converge back to the original reslts in spite of temporary divergence. We prove or analysis sond with respect to the traditional ɛ-δ definition of continity. We demonstrate the precision of or analysis by applying it to a range of classic algorithms, inclding algorithms for array sorting, shortest paths in graphs, minimm spanning trees, and combinatorial optimization. A prototype implementation based on the Z3 SMT-solver is also presented. Categories and Sbject Descriptors F.3.2 [Logics and Meanings of Programs]: Semantics of programming langages Program analysis.; F.3.1 [Logics and Meanings of Programs]: Specifying and Verifying and Reasoning abot Programs Mechanical verification; G.1.0 [Nmerical Analysis]: General Error analysis, Stability General Terms Theory, Verification Keywords Continity, Program Analysis, Uncertainty, Robstness, Pertrbations, Proof Rles. Permission to make digital or hard copies of all or part of this work for personal or classroom se is granted withot fee provided that copies are not made or distribted for profit or commercial advantage and that copies bear this notice and the fll citation on the first page. To copy otherwise, to repblish, to post on servers or to redistribte to lists, reqires prior specific permission and/or a fee. POPL 10, Janary 17 23, 2010, Madrid, Spain. Copyright c 2010 ACM /10/01... $10.00 DIJK(G : graph, src : node) 1 for each node v in G 2 do d[v] := ; prev[v] := UNDEF ; 3 d[src] := 0; WL := set of all nodes in G; 4 while WL 5 do choose node w WL sch that d[w] is minimal; 6 remove w from WL; 7 for each neighbor v of w 8 do z := d[w] + G[w, v]; 9 if z < d[v] 10 then d[v] := z; prev[v] := w 1. Introdction Figre 1. Dijkstra s shortest-path algorithm Uncertainty in comptation has long been a qestion of interest in compting [8]. An important reason for the ncertain behavior of programs is erroneos data [21]: the traffic data that a GPS device ses to plan a path may be slightly stale at the time of comptation [15], and the sensor data that an aircraft controller processes may be slightly wrong [11]. In a world where comptation is increasingly intertwined with sensor-derived perceptions of the physical world [12], sch ncertain inpts are biqitos, and the assrance that programs respond robstly to them often vital. Does the otpt of a GPS device change only slightly in response to flctations in its inpts? If so, can we prove this fact atomatically? Robstness of programs to small amonts of error and ncertainty in their inpts can be defined via the mathematical notion of continity. Recall that a fnction f(x) : R R is continos at the point x = c if, for all infinitesimal deviations of x from c, the vale of f(x) deviates at most infinitesimally from f(c). This provides a concrete definition of robstness: if a program implements a fnction that is continos at c, then its otpt is not affected by small flctations of its inpt variable arond the vale c. To see this definition of continity of programs and its application in specifying robstness, consider an algorithm rotinely sed by path-planning GPS devices: Dijkstra s shortest-path algorithm. A program Dijk implementing the algorithm is shown in Figre 1 here, G is a graph with real edge-weights, src is the sorce node, and G[, v] is the weight of the edge (, v). It is a property of Dijk that the set of paths that it comptes can change completely in response to small pertrbations to G (by this, let s mean that the weight of some edge of G changes slightly). However, what if or robstness reqirement asserts that it is the weight of the shortest path that mst be robst to small changes to G? In other words, assming the array d of shortest-path distances is the program s otpt, is the program continos? We note that it is d changes at most infinitesimally if G changes infinitesimally. Qestions of continity and robstness appear rotinely in the literatre on dynamical and hybrid systems [16, 17]. However, these approaches apply to systems defined by differential eqations, hybrid atomata [1], or graph models [18]. In the program

2 verification literatre, robstness has previosly been considered in the restricted settings of fnctional synchronos programs [2], finite-state systems [10], and floating-point rondoff errors [6, 7, 13, 14]. Also, for prely nmerical programs, robstness can be analyzed by abstract interpretation sing existing domains [4, 5]. In contrast, this paper assmes a framing of robstness in terms of continity, and presents a general proof framework for continity that applies to programs sch as Dijk that se data-strctres sch as graphs and arrays, as well as featres like imperative assignments, branches, and loops. The search for sch a proof framework, however, is fraght with challenges. Even a program whose inpts and otpts are over continos domains may se temporary variables of discrete types, and maniplate data sing imperative assignments, branches, and loops. It can have mltiple inpts and otpts, and an otpt can be continos in one inpt bt not in another. Indeed, prior work [9] has arged for a notion of continity for software, bt failed to offer new program analysis techniqes, conclding that it is not possible in practice to mechanically test for continity in the presence of loops. Recall the seemingly simple continity property of the program Dijk: if d is its otpt, then it is continos. However, it is highly challenging to establish this property from the text of Dijk. One way to prove it wold be to first prove that Dijk comptes shortest paths, and then to establish that the costs of these paths are continos in the weights of the edges of G. Sch a proof, however, wold be highly specialized and impossible to atomate. What we want, therefore, is a proof methodology that reasons abot continity withot aiming to prove fll fnctional correctness, is applicable to a wide range of algorithms, and can be atomated. Here we present sch a method. We highlight below some of the challenges that arise and or proof rles for addressing them. Presence of Control-flow. One challenge in proving continity of programs is control-flow: a small pertrbation can case control to flow along a different branch leading to a syntactically divergent behavior. For example, consider the branch in Lines 9-10 in Dijk, which allow semantically different behaviors of either setting d[v] to z or leaving d[v] nchanged. We present a rle for proving continity of sch if-then-else code-fragments. The key idea is to show that the two (otherwise semantically different) branches become semantically eqivalent in sitations (known as discontinities) where the conditional can flip its vale. Using this rle (ITE- 1), we can show that the l-vale d[v] is continos after the codefragment if d[v] < z then d[v] := z. This is becase the conditional d[v] < z can flip vales on small pertrbations only when d[v] was already close to z; however, nder sch a condition the expressions d[v] and z evalate to approximately the same vale. Non-indctiveness of continity. The next challenge comes in extending the continity proofs to loops. A natral approach is to set p an indctive framework for establishing continity dring each loop iteration (rle SIMPLE-LOOP). However, it trns ot that continity is not an indctive property for several loops (nlike invariants), meaning that the program variables that are continos at the end of the loop are not necessarily continos in each loop iteration. For example, while the array d is a continos fnction of G on termination of Dijk, it is not continos across each loop iteration. This is becase the array d is pdated in each loop iteration based on the choice of w from the workset W sch that d[w] is minimal. Now small flctations in the inpt weights can case small flctations in the elements of d, casing it to choose a very different node w and potentially alter d completely. Key to solving this challenge is the observation that if we grop some loop iterations together, then continity becomes an indctive property of the gropings. These gropings are referred to as epochs, and they have the property that the constitent iterations can be exected in any order withot violating the semantics of the program. The LOOP proof-rle discharges this obligation by establishing commtativity of the loop body. Retrning to Dijkstra s algorithm, this groping is based on the set of elements w that have similar weight d[w]. The property of this groping is that P (w 1); P (w 2) is semantically eqivalent to P (w 2); P (w 1) where w 1 and w 2 are two elements sch that d[w 1] = d[w 2], where P (w) represents the code-fragment in Lines Pertrbations in Nmber of Loop Iterations. Another challenge in continity proofs for loops is that the nmber of loop iterations may differ as a reslt of small pertrbations to the inpts. We note that whenever sch a behavior happens in continos loops, then the effect of the extra iterations either in the original or the pertrbed exection is almost eqal to that of a skip-statement. This property called synchronized termination condition is asserted in or rles for loops. (Dijkstra s algorithm does not exemplify this challenge thogh, as the loop body is exected once for each graph node regardless of small changes to the edge weights.) 1.1 Contribtions and Organization of the Paper This paper makes the following contribtions. We formalize the notion of continity of programs by associating data-types with metric spaces and operators with continity specifications (Sec. 2). We present strctral rles to prove the continity of programs in presence of control-flow (Sec. 4) and loops (Sec. 5), after establishing a formalism to reason abot continity of expressions in Sec. 3. These proof rles reqire establishing standard properties of code-fragments, in particlar, establishing eqivalence or commtativity, which can be discharged sing off-the-shelf SMT solvers or assertion checkers. We prove or proof rles sond with respect to the standard ɛ-δ definition of continity. This is qite challenging becase the proof rles do not refer to ɛ or δ. We demonstrate the precision of or proof rles by showing that or framework can be sed to prove continity of several continos classical algorithms. Sec. 6 discsses or implementation of a prototype of or framework that discharges proof rles sing the SMT-solver Z3. Or crrent implementation reqires the ser to provide some annotations to identify the reqisite components of the LOOP proof rle, thogh there are heristics that can be sed to atomate this step. 2. Problem formlation In this section, we fix a notion of continity for imperative programs and formlate the problem of continity analysis. First, we define a langage whose semantics allows for a notion of distances between states in fact, states are now elements of a metric space. Second, we define continity for programs as standard mathematical continity applied to their semantic fnctions. As programs have mltiple inpts and observable otpts, we allow for statements sch as Program P is continos in inpt x bt not in inpt y, meaning that a small change to x mst case small changes to the observable program otpts, bt that a small change to y may change the observable otpts arbitrarily. Programs and expressions. We begin by fixing a simple imperative langage (henceforth called IMP). The langage has a single non-standard featre: a distance metric for each data type. Types here represent metric spaces rather than sets of vales, and the semantics of expressions and programs are given by fnctions between metric spaces. This lets s define continity of programs sing standard mathematical machinery.

3 Let a distance be either a non-negative real, or a special vale satisfying x < for all x R 0. We define metric spaces over these distances in the sal way. Also, we assme: A set of data types. Each type τ in this set is associated with distance metric dist τ, and represents a metric space Val τ whose distance measre is dist τ. The space Val τ is known as the space of vales of τ. For or convenience, we assme that each Val τ contains a special vale representing ndefined. In particlar, we allow the types bool and real of booleans and reals. The type real is associated with the standard Eclidean metric, defined as dist real(x, y) = x y if x, y, and dist real(x, y) = otherwise. The metric on bool is the discrete metric, defined as dist bool(x, y) = 0 if x = y, and otherwise. A niverse Var of typed variables. A set O of (primitive) operators. Each operator op comes with a niqe signatre op : τ(p 1 : τ 1,..., p n : τ n), where for all i, p i / Var. Intitively, p i is a formal parameter of type τ i, and τ is the type of the otpt vale. For example, the real type comes with operators for addition, mltiplication, division, etc. The syntax of expressions e is now given by e ::= x op(e 1,..., e n), where x Var and op O. Here op(e 1,..., e n) is an application of the operator op on the operands e 1,..., e n. The set of variables appearing in the text of e is denoted by Var(e). For easier reading, we often write or expressions in infix. Expressions are typed by a natral set of typing rles as or analysis is orthogonal to this type system, we assme all or expressions to be well-typed. As for programs P, they have the syntax: P ::= skip x := e if b then P 1 else P 2 while b do P 1 P 1; P 2 where e is an expression and b a boolean expression. We denote the set of variables appearing in the text of P by Var(P ). For convenience, we sometimes annotate statements within a program with labels l. The interpretation is standard: l represents the control point immediate preceding the statement it labels. As for semantics, let s first define a state: Definition 1 (State). A state is a map σ assigning a vale in Val τ to each x Var of type τ. The set of all states is denoted by Σ. The semantics of an expression e of type τ is now defined as a fnction [[e]] of the form Σ Val τ. As expressions are bilt sing operators, we presppose a semantic map [[op]] for each operator op O. Let op have the signatre op : τ(p 1 : τ 1,..., p n : τ n), and let Σ op be the set of maps assigning sitably typed vales to the p i s. Then [[op]] is a map of type Σ op Val τ. The semantic fnction [[e]] for an expression e is now defined as: { [[op]]([e1]](σ),..., [[e [[e]](σ) = n]](σ)) if e = op(e 1,..., e n) σ(x) if e = x Var. As for programs, we se a standard fnctional (denotational) semantics [23] for them. For simplicity, let s only consider programs that terminate on all inpts. The semantic fnction for a program P is then a map [[P ]] of type Σ Σ sch that for all states σ in, [[P ]](σ in) is the state at which P terminates after starting exection from the state σ in. The indctive definition of [[P ]], being standard, is omitted. Note that both [[e]] and [[P ]] are fnctions between metric spaces. Continity. Now that we have defined the semantics of programs as maps between metric spaces, we can se the standard ɛ-δ definition [20] to define their continity. As programs sally have mltiple inpts and otpts, we consider a notion of continity that is parameterized by a set of inpt variables In and a set of observable variables Obs. For a set of variables V, let s call two states V -close if they differ at most slightly in the vales of the variables in V, and V - eqivalent if they agree on vales of all variables in V. Let a state σ be a small pertrbation of a state σ if the vale of each variable in In is approximately eqal, and the vale of each variable not in In exactly the same, in σ and σ. We define: Definition 2 (Pertrbations, V -closeness, V -eqivalence). For ɛ R +, a state σ, and a set In Var of inpt variables, a state σ is an ɛ-pertrbation of a state σ (written as Pert ɛ,in(σ, σ )) if for all variables x In of type τ, we have dist τ (σ(x), σ (x)) < ɛ, and for all variables y / In of type τ, we have σ(y) = σ (y). For V Var and ɛ R, the states σ and σ are ɛ-close in V (written as σ ɛ,v σ ) if for all x V of type τ, we have dist τ (σ(x), σ (x)) < ɛ. The states are V -eqivalent (written as σ V σ ) if for all x V, we have σ(x) = σ (x). Continity of programs and expressions can now be defined by applying the traditional ɛ-δ definition: Definition 3 (Continity of expressions and programs). Let In Var be a set of inpt variables. An expression e of type τ is continos at a state σ in In if for all ɛ R +, there exists a δ R + sch that for all σ satisfying Pert δ,in (σ, σ ), we have dist τ ([[e]](σ), [[e]](σ )) < ɛ. A program P is continos at a state σ in a set In of inpt variables and a set Obs Var of observable variables if for all ɛ R +, there exists a δ R + sch that for all σ satisfying Pert δ,in (σ, σ ), we have [[P ]](σ) ɛ,obs [[P ]](σ ). Intitively, if e is continos in In, then small changes to variables in In can change its vale at most slightly, and if P is continos in In and Obs, then small changes to variables in In can only case small changes to variables in Obs (variables otside Obs can be affected arbitrarily). While the ɛ-δ definition can be directly sed in continity proofs [20], sch proofs are highly semantic. More syntactic proofs wold reason indctively, sing axioms, inference rles, and invariants. The appeal of a framework for sch proofs wold be twofold. First, instead of closed-form mathematical expressions, it wold target programs that may often not correspond to cleanly defined or easily identifiable mathematical fnctions. Second, it wold allow mechanization, even atomation. Therefore, we formlate the problem of continity analysis: Problem (Continity analysis). Develop a set of syntactic proof rles that can sondly and completely determine if a program P is continos in a set In of inpt variables and a set Obs of observable variables, at each state σ satisfying a property c. In the next few sections, we present or soltion to this problem. Or rles are sond. While we do not claim completeness, we offer an empirical sbstitte: nearly-atomatic continity proofs for 11 classic algorithms, most of them picked from a standard ndergradate textbook [3]. 3. Continity jdgments and specifications In this section, we define the basic bilding blocks of or reasoning framework. These inclde the jdgments it otpts, as well as the ser-provided specifications that parameterize it. 3.1 Continity jdgments Sppose or goal is to jdge the continity of an expression e or a program P in a set In of inpt variables and, in the latter case, a set Obs of observable variables. Instead of obtaining jdgments

4 that hold a specific state σ, we jdge continity at a set of states symbolically represented by a formla b. Therefore, we define: Definition 4 (Continity jdgment). A continity jdgment for an expression e is a term b Cont(e, In), where b is a formla with free variables in Var, and In Var. A jdgment for a program P is a term b Cont(P, In, Obs), where b is a formla over Var, and In, Obs Var. The jdgment b Cont(e, In) is read as: e is continos in In at each state σ satisfying the property b. The jdgment b Cont(P, In, Obs) says that the program P is continos in the set In of inpt variables and the set Obs of observable variables at all states satisfying b. The jdgments are sond if these statements are tre according to the definition of continity in Definition 3. Note that for a jdgment b Cont(P, In, Obs) (similarly, b Cont(e, In)) to be sond, it sffices for In to be an nderapproximation of the set of inpt variables, and Obs to be an overapproximation of the set of observable variables, in which P (similarly, e) is continos. Example 1. The expression (x+y), where + denotes real addition, is always continos in {x, y}. On the other hand, the expression x y, which evalates to for y = 0, is not always continos. Two sond jdgments involving it are tre Cont( x, {x}) and y (y 0) Cont( x, {x, y}), which say that: (1) the reslt of y division is always continos in the dividend, and (2) continos in all non-zero divisors. Now consider the type (τ real) of real-valed arrays: partial fnctions from the index type τ to the type real. For any sch array A, we define Dom(A) to denote the domain of A i.e., the set of all x sch that A[x] is defined. Let s consider the following spremm metric: dist τ real(a, B) = max i Valτ {dist real(a[i], B[i])} if Dom(A) = Dom(B) otherwise. Intitively, the distance between A and B is the maximm distance between elements in the same position in the two arrays. Now consider the array-pdate operator Upd, commonly sed to model writes to arrays. The operator has the parameters A (of type τ real), i (an integer), and p (a real), and retrns the array A sch that A [i] = p, and A [j] = A[j] for all j i. To exclde erroneos writes, let Upd evalate to if p = or if A contains an ndefined vale. In that case, the following jdgment is sond: (p ) ( i, A[i] ) Cont(Upd(A, i, p), {A, i, p}). Observe that Upd(A, i, p) is jdged to be continos in i. The reason is that as i is drawn from a discrete metric space (the int type), the only way to change it infinitesimally is to not change i at all. Continity in i follows trivially. Example 2. Consider the program P = x := x + 1; y := z/x. A sond continity jdgment for P is (x + 1 0) Cont(P, {x, y, z}, {y}). Now consider the following program P : if (x 0) then r := y else r := z. Denote by c the formla (x = 0) (y = z). Then the continity jdgment c Cont(P, {x, y, z}, {r}) is sond. To see why, note that for fixed x and z, an infinitesimal change in y either cases no change to the final vale of r (this happens if x < 0), or changes it infinitesimally. A similar argment holds for infinitesimal changes to z. As for x, the gard (x 0) is continos in x (i.e., is not affected by small changes to x) at all x 0. As a reslt, nder the precondition x 0, infinitesimal changes to x, y, z changes the final vale of r at most infinitesimally. At x = 0, of corse, the gard is discontinos i.e., can change vale on an infinitesimal change to x. In this case, an infinitesimal change to x may case the otpt r to change from the vale of y to that of z. However, by or precondition, we have y = z whenever x = 0. Ths means that even if the gard evalates differently, the observable otpt r is affected at most infinitesimally by infinitesimal changes to x, y, z. In other words, nder the assmed conditions, the discontinos behavior of the gard does not affect the continity of P in r. Now consider the jdgment tre Cont(P, {y, z}, {r}). As we only assert continity in inpts y and z, it is clearly sond. 3.2 Continity specifications As the operators in or programming langage can be arbitrary, we need to know their continity properties to do continity proofs. This information is provided by the programmer throgh a set of continity specifications. We define: Definition 5 (Continity specification). A continity specification for an operator op, with the signatre op : τ(p 1 : τ 1,..., p n : τ n), is a term c S, where c is a boolean expression over p 1,..., p n and S {p 1,..., p n}. An operator is allowed to have mltiple specifications. Sppose the operator op has a specification c S. The interpretation is that the semantic map [[op]] is continos in S at each state over {p 1,..., p n} that satisfies c. The specification is sond if this is indeed the case. Intitively, application of the operator preserves the continity properties of argments corresponding to parameters p i S, and can potentially introdce discontinities in the remaining argments. Example 3. Let the real addition operator have the signatre + : real(x : real, y : real). A sond specification for it is tre {x, y}. Now consider the real division operator /, with similar signatre. Two sond specifications for it are tre {x} and (y 0) {x, y}. Continity specifications as above have a natral relationship with modlar analysis. While operators in programming langages are sally low-level primitives, nothing in or definition prevents an operator from being a procedral abstraction of a program. As the reasoning framework assmes its continity properties, it defines a level of abstraction at which continity is jdged. In the crrent paper, we assme that this procedral abstraction is defined by the programmer. In ftre work, we will consider an interprocedral continity analysis, where operator specifications are generalized into continity smmaries mined from a program. 4. Analysis of expressions and loop-free programs In this section, we begin the presentation of or analysis. The main contribtions presented here are: (1) a continity analysis of expressions throgh strctral indction; and (2) an analysis of branching programs based on identification of the discontinities of a boolean expression, and qeries abot program eqivalence discharged throgh an SMT-solver. 4.1 Analysis of expressions The main idea behind continity analysis of expressions is simple: an expression e is continos in In if it is obtained by recrsively applying continos operators on variables in In. If e has a sbexpression e that is either discontinos or an argment to an operator that does not preserve its continity, then we shold jdge e to be discontinos in all variables in e. The inference rles for the analysis are presented in Figre 2. Here, for expressions c, e 1,..., e n, the notation c[x 1 e 1,..., x n e n] denotes the expression obtained by sbstitting each variable x i in c by e i. The rle BASE states that a variable is always continos in itself. WEAKEN observes that a continity

5 (Base) (Weaken) (Join) (Frame) (Op) x Var tre Cont(x, {x}) b Cont(e, In) b b In In b Cont(e, In ) b Cont(e, In 1 ) b Cont(e, In 2 ) b Cont(e, In 1 In 2 ) b Cont(e, In) z / Var(e) b Cont(e, In {z}) op has parameters p 1,..., p n and a specification c S p i S. b Cont(e i, In) p i / S. In Var(e i ) = c = c[p 1 e 1,..., p n e n] c b Cont(op(e 1,..., e n), In) Figre 2. Continity analysis of expressions. jdgment can be sondly weakened by restricting the set of inpt variables in which continity is asserted, or the set of states at which continity is jdged. FRAME observes that an expression is always continos in variables to which it does not refer. As for JOIN, it ses the mathematical fact that if a fnction is continos in two sets of inpt parameters, then it is continos in their nion. The rle OP derives continity jdgments for expressions e = op(e 1,..., e n), where op is an operator. Intitively, if e i is continos in a set of variables In, and if op is continos in its i-th parameter, then e is also continos in each x In. The sitation is complicated by the fact that a variable can appear in mltiple e i s, and that if x In appears in any e j sch that op is not continos in p j or e j is not continos in x, then e is potentially discontinos in x. Ths, we mst ensre that In Var(e j) = for all sch e j; also, each e i sch that p i S mst be continos in all the variables in In. The rle OP has these conditions as premises. Example 4. Consider the expression e = x, where x and y x+y are real-valed variables, and the jdgment (x > 0) (y > 0) Cont(e, {y}). To prove it in or system sing specifications of real addition and division as before, we first se the rles BASE and FRAME, OP, and the specification of + to prove that tre Cont((x + y), {x, y}). Now we derive that tre Cont(x, {x, y}), then se OP and the specification of / to show that whenever (x+y) 0, e is continos in {x, y}. Finally, we se WEAKEN to get the desired jdgment. Using indction and ɛ-δ reasoning, we can show that: Theorem 1. If all operator specifications are sond, the proof system in Figre 2 only derives sond continity jdgments. 4.2 Analysis of loop-free programs The analysis of loop-free programs brings ot some sbtleties e.g., to prove the continity of programs with branches, we mst discharge a program eqivalence qery throgh an SMT-solver. Example 5. Recall the program in Example 2: P = x := x + 1; y := z/x. As arged earlier, the jdgment (x + 1 0) Cont(P, {x, y, z}, {y}) is sond. One proof for it is as follows: 1. Show that (x + 1) is always continos in x. From this, derive the jdgment tre Cont(x := x + 1, {x, y, z}, {x}). 2. Establish that (x 0) Cont(y := z/x, {x}, {y}). 3. Propagate backward the condition for continity of P 2, obtaining the precondition (x + 1) 0 for P. 4. Compose the jdgments in (1) and (2), sing the fact that x is the only observable otpt in the former and the only inpt in the latter. This gives s the desired jdgment. (Skip) (Join) (Weaken) (Frame) (Assign-1) (Assign-2) (Seqence) (Ite-1) (Ite-2) tre Cont(skip,, ) b Cont(P, In 1, Obs) b Cont(P, In 2, Obs) b Cont(P, In 1 In 2, Obs) b Cont(P, In, Obs) b b Obs Obs In In b Cont(P, In, Obs ) b Cont(P, In, Obs) z / Var(P ) b Cont(P, In {z}, Obs {z}) b Cont(e, In) b Cont(x := e, In {x}, Var(e) {x}) b Cont(x := e, Var(e) {x}, Var(e) \ {x}) b 1 Cont(P 1, In 1, Obs 1 ) In 2 Obs 1 b 2 Cont(P 2, In 2, Obs 2 ) {b 1 }P 1 {b 2 } b 1 Cont(P 1 ; P 2, In 1, Obs 2 ) c Cont(P 1, In, Obs) c Cont(P 2, In, Obs) c Cont(b, Var(b)) (c c ) (P 1 Obs P 2 ) c Cont(if b then P 1 else P 2, In, Obs) c Cont(P 1, In, Obs) c Cont(P 2, In, Obs) c Cont(b, In ) c c Cont(if b then P 1 else P 2, In In, Obs) Figre 3. Continity analysis of loop-free programs. Now consider the program P from Example 2: if (x 0) then r := y else r := z. As we arged earlier, the jdgment c Cont(P, {x, y, z}, {r}), where c eqals (x = 0) (y = z). A proof can have the following components: 1. Identify an overapproximation Σ of the set of states (in this case captred by the formla (x = 0)) at which the loop gard is discontinos i.e., can flip on a small pertrbation. 2. Assming c holds initially, the two branches of P, when exected independently from a state in Σ, terminate at states agreeing on the vale of r. 3. Each branch is continos at all states in c, in the set {x, y, z} of inpt variables and the set {r} of observable variables. Here, condition (2) asserts that even if a state execting along a branch were exected along the other branch, the observable reslt wold be the same. Together with condition (3), it implies that even if an exection and its pertrbed conterpart follow different branches, they reach approximately eqivalent states at the joinpoint. Ths, it asserts a form of synchronization following a period of divergence between the original and pertrbed exections. Now consider the sond jdgment tre Cont(P, {y, z}, {r}). This time, we establish that: (1) each branch of P is nconditionally continos in y and z, and (2) the gard (x 0) is nconditionally continos in these variables as well. Let min be the program if (x y) then x else y compting the minimm of two real-valed variables. Using a similar style of proof as for P, we can establish the sond jdgment tre Cont(min, {x, y}, {x, y}) i.e., the fact that min is nconditionally continos. A similar argment holds for max. Let s now try to systematize the ideas in the above examples into a set of inference rles. We need some more machinery: Discontinities of a boolean expression. Or rle for branchstatements reqires s to identify the set of states where a boolean expression b is discontinos. For sondness, it sffices to work

6 with an overapproximation of this set. This can be obtained by inferring a jdgment of the form c Cont(b, Var) abot b by the sondness of or analysis for expressions, c is overapproximates of the set of states where b is discontinos in any variable. As we have not written continity specifications for boolean operators so far, let s do so now. To jdge the continity of boolean expressions, we plg these into the system in Figre 2. Example 6. For simplicity, let s only consider boolean expressions over the types real and bool. We allow the standard comparison operators =,, and > with signatres sch as : bool(x : real, y : real); we also have the sal real arithmetic operators. For boolean arithmetic, we se the standard operators,, and. Specifications for operators involving booleans are as follows: We specify each comparison operator in {=, >, <,, } (let it have formal parameters p and q) as (p q) {p, q}. It is easy to see that this specification is sond. The operators and (with formal parameters p and q) have the specification tre {p, q}; the operator (with parameter q) has the specification tre {q}. The reason (which also showed p in Example 1) is that these operators have discrete inpts. This implies that the only way to infinitesimally change an inpt x to any of them is to not change x at all. Unconditional continity in the parameters follows trivially. For example, consider the boolean expression (x 2 4) (y < 10) (let s call it b). The reader can verify that sing the above specifications, we can derive the jdgment (x 2 4) (y 10) Cont(b, {x, y}, {x, y}). It follows that ((x 2 4) (y 10)) overapproximates the set of states at which b is discontinos. Hoare triples and program eqivalence. To propagate conditions for continity throgh a program, we generate Hoare triples as verification conditions. These are compted sing an off-the-shelf invariant generator. Additionally, we assme a solver for eqivalence of programs 1. Let V Var and c be a logical formla; now consider programs P 1 and P 2. We say that P 1 and P 2 are V -eqivalent nder c, and write c (P 1 V P 2), if for each state σ that satisfies c, we have [[P 1]](σ) V [[P 2]](σ). Or rle for branches generates qeries abot program eqivalence as verification conditions. The rles. Or rles for continity analysis of loop-free programs are presented in Figre 3. Here, the rle JOIN is the analog of the rle JOIN for expressions. The rle WEAKEN lets s weaken a jdgment by restricting either the precondition nder which continity holds, or restricting the sets of inpt and observable variables. The rle FRAME says that a program is always continos in variables not in its text. The rle SKIP (taken together with the rle FRAME) says that skip-statements are always continos in every variable. The rle ASSIGN-1 says that if the right-hand side of an assignment statement is continos in In, then the statement is continos in In even if the lvale x is observable. ASSIGN-2 says that if x is not observable, then the statement is nconditionally continos. The rle SEQUENCE addresses seqential composition, systematizing the insight in Example 2. Sppose P 1 is exected from a state σ satisfying b 1 and ends at σ. We have {b 1}P 1{b 2}; therefore, P 2 is continos in In 2 at σ. Now sppose we modify σ by infinitesimally changing some variables in In 1; the altered otpt state σ of P 1 approximately agrees with σ on the vales of variables in In 2 (as In 2 Obs 1). De to the premise abot P 2, this can only change the otpt of P 2 infinitesimally (assming Obs 2 is the set of observable variables). The rle ITE-1 generalizes the first continity jdgment for the program P made in Example 2. Here, c is an overapproximation 1 We note that program eqivalence is a well-stdied problem, an important application being translation validation [19] of optimizing compilers. FLOYD-WARSHALL(G : graph) 1 for k := 1 to n 2 do for i, j := 1 to n 3 if G[i, j] > G[i, k] + G[k, j] 4 then G[i, j] := G[i, k] + G[k, j]; prev[i, j] := prev[k, j] Figre 4. Floyd-Warshall algorithm for all-pairs shortest paths of the set of states at which the gard b is discontinos. As P 1 and P 2 are eqivalent whenever (c c ), it does not matter if the gard flips as a reslt of pertrbations the if-statement is continos in each variable in which both branches are continos. As for the rle ITE-2, it generalizes the second jdgment abot P in Example 2. The (conditional) eqivalence of P 1 and P 2 is not a premise here; therefore, the if-statement is garanteed to be continos only in the variables in which b is continos. The inferred precondition for continity is also restricted. Using indction and ɛ-δ reasoning, we can show that: Theorem 2. The inference rles in Figre 3 are sond. Example 7. Consider the program P = P 1; P 2, where P 1 is x := y/z and P 2 is the program if (x 0) then r := y else r := z. Let V = {x, y, z, r}, and let c be ((x = 0) (y = z)). To establish the jdgment (y 0) (z 0) Cont(P, V, V ), we first prove the jdgment (y 0) (z 0) Cont(P 1, V, V ) this reqires a continity proof for the expression y/z, and se of the rles ASSIGN, FRAME and WEAKEN. Next we show that c Cont(P 2, V, V ), sing, among others, the rle ITE-1. Finally, we apply the rle for seqential composition. 5. Continity analysis of programs with loops In this section, we present or continity analysis of programs with loops. The main conceptal contribtions presented here are: (1) An indctive rle for proving the continity of loops, based on a generalization of or rle for loop-free programs. (2) A second rle for loops, based of indction where the basic step is a seqence of loop iterations (known as an epoch) rather than a single iteration. The latter rle is needed becase many important applications cannot be proved continos by an ordinary indctive argment. A sondness proof for it is also sketched. 5.1 Analysis of loops by indction We start with a motivating example: Example 8 (Floyd-Warshall algorithm). Let s consider the Floyd- Warshall all-pairs shortest-path algorithm with path recovery (Figre 4). Let s call this program FW ; on termination, G[i, j] contains the weight of the shortest path from i to j, and prev[i, j] contains a node sch that for some shortest path from i to j, the node right before j is prev[i, j]. We view a graph G as a fnction from a set of edges each edge being a pair (, v) of natral nmbers to a set of real-valed edge-weights. Ths, G is a real-valed array as in Example 1. Let the metric on real-valed arrays be as in Example 1; the metric on the discrete array prev and the variables i, j is the discrete metric (previosly sed on the bool type). Then: (1) if a graph G has a node or edge that G does not, then the distance between G and G is, and (2) otherwise, the distance between them is max (,v) Dom(G) { G[, v] G [, v] }. In other words, a small change to G is defined as a small change to edge-weights keeping the node and edge strctre intact. As the final vale of G[i, j] gives the weight of the shortest path between i and j, the continity claim tre Cont(FW, {G}, {G}) is sond (however, the claim tre Cont(FW, {G}, {prev}) is not a previosly valid shortest path may become invalidated de

7 (Simple-loop) I(c) Cont(R, X, X) c Term(P, X) c Sep(P, X) c Cont(P, X, X) Figre 5. Rle SIMPLE-LOOP. (Here, P = while b do (l : R).) to pertrbations.) We can establish this property by indction. Let R be the body of the inner loop (Line 3). Using an analysis as in the previos section, we have tre Cont(R, {G}, {G}). Now let R i represent i repetitions of R i.e., the first i iterations of the loop taken together. Indctively assme that R i is continos in G i.e., a small change to the initial vale of G leads to a small pertrbation of G at the end of the i-th iteration. By the continity of R in G, R i+1 is also continos in G. Finally, observe that the expressions garding the two loops here are continos in G in other words, small changes to G do not affect them. Therefore, an exection and its pertrbed conterpart have the same nmber of loop iterations. This establishes that the entire loop represents a continos fnction. Let s now systematize the ideas in this example into an indctive proof rle. Let P be a program of the form while b do (l : R). Or goal here is to indctively derive jdgments of the form c Cont(P, X, X), where X is an indctively obtained set of variables. However, we need some more machinery. Trace semantics and invariants. As or reasoning for loops reqires indctive invariants, it reqires a trace semantics of programs. Let s define the trace of P from a state σ in as the niqe seqence π = σ 0σ 1... σ n+1 sch that σ 0 = σ in, and for each 0 i n, {σ i}r{σ i+1} and σ i satisfies b. Let c be an initial condition (given as a logical formla). We define a state σ to be reachable from c if it appears in the trace from a state satisfying c (note that in all reachable states, control is implicitly at the label l). The loop invariant I(c) of Q is an overapproximation of the set of states reachable from c. Or proofs assme a sond procedre to generate loop invariants. The analysis. Let s now proceed to or continity analysis. As before, or strategy is to discharge verification conditions that do not refer to continity or infinitesimal changes. In particlar, the following conditions are sed: Separability. A set of variables X is separable in a program P if the vale of any z X at the terminal state of P only depends on the vales of variables in X at the initial state. Formally, we say that X is separable in P nder an initial condition c, and write c Sep(P, X), if for all states σ, σ reachable from c sch that σ X σ, we have [[P ]](σ) X [[P ]](σ ). Under this condition, we have an alternative definition of continity that is eqivalent to the one that we have been sing: P is continos at a state σ in a set X of inpt variables and the same set X of observable variables if for all ɛ R +, there exists a δ R + sch that for all σ satisfying σ δ,x σ, we have [[P ]](σ) ɛ,x [[P ]](σ ). This definition is sefl as we se the relation indctively. Synchronized termination. P flfills the synchronized termination condition with respect to an initial condition c and a set of variables X (written as c Term(P, X)) if Var(b) X, and one of the following holds: (1) The loop condition b satisfies tre Cont(b, X); (2) Let the formla c represent an overapproximation of the set of states reachable from c where b is discontinos in X. Then we have c R X skip. Intitively, this condition handles scenarios where an exection from a pertrbed state violates the loop condition earlier or later than it wold in the original exection. Under synchronized termination, the exection that contines does not veer too far from the state where it was when the other exection ended. Of these, separability can be established sing simple slicing. In the absence of nested loops, synchronized termination can be checked sing an SMT-solver. If the loop body contains a nested loop, it can be checked sing an SMT-solver in conjnction with an invariant generator for the inner loop [19]. Or rle SIMPLE-LOOP for indctively proving the continity of P is now as in Fig. 5. Let s now see its se in an example: Example 9. We revisit the program FW in Figre 4. (We assme it to be rewritten as a while-program in the obvios way.) Let X = {G, i, j}. First, we observe that tre Sep(FW, X). As arged before, letting R be the loop body, we have tre Cont(R, X, X). Finally, the loop gard b only involves discrete variables therefore, by the argment in Example 1, it is always continos in X, which means that tre Term(FW, X). The sond jdgment tre Cont(FW, X, X) follows. Using the WEAKEN rle from before, we can now obtain the jdgment tre Cont(FW, {G}, {G}). Of corse, this example does not illstrate the sbtleties of the synchronized termination condition. For a more serios se of this condition, see Examples 11 and 13. As for sondness, we have: Theorem 3. The inference rle SIMPLE-LOOP is sond. Proof sketch. Consider an arbitrary trace π = σ 0σ 1... σ m+1 of P starting from a state σ 0 that satisfies c, and any ɛ R +. Let s gess a seqence δ 0, δ 1,..., δ m+1 sch that: (1) δ m+1 < ɛ, and (2) for all states s, s reachable from c, if s and s are δ i- close (in X), then if t and t are the states satisfying {s}r{t} and {s }R{t }, then s and t are δ i+1-close (in X). Sch a seqence exists as the loop body R is continos. Now select δ = δ 0, and consider a state σ 0 sch that σ 0 δ,x σ 0. Recall that we assme that all programs in or setting are terminating. Therefore, any trace from σ 0 is of the form π = σ 0... σ n+1 (withot loss of generality, assme that n m). By the continity of the loop body, we have σ 1 δ1,x σ 1. As the synchronized termination condition holds, one possibility is that b is continos at both σ 1 and σ 1, In this case, either both traces contine exection into the next epoch, or none do. The other possibility is that one of the traces violates b early de to pertrbations, bt in this case the rest of the other trace is eqivalent to a skip-statement. Generalizing indctively, we conclde that σ m+1 ɛ,x σ n+1. As π, π are arbitrary traces, the program P is continos. 5.2 Continity analysis by indction over epochs Unsrprisingly, there are many continos applications whose continity cannot be proved by the rle SIMPLE-LOOP. Pleasantly, many applications in this category are amenable to proof by richer form of indction that we have identified, and will now present. We start with two examples: Example 10 (Dijkstra s algorithm). Consider or code for Dijkstra s algorithm (Figre 1; code partially reprodced in Fig. 6) once again. The one otpt variable is d the array that, on termination, contains the weights of all shortest paths from src. The metric on G is as in Example 8. Note that Line 5 in this code selects a node w sch that d[w] is minimal, so that to implement it, we reqire a mechanism to break ties on d[w]. In practice, sch tie-breaking is implemented sing an arbitrary linear order on the nodes. Sch implementations, however, are ad hoc and can easily break indctive reasoning for continity. To be on the safe side, we conservatively abstract the program by replacing the selection in Line 5 by nondeterministic choice. It is

8 4 while WL 5 do choose node w WL sch that d[w] is minimal 6 remove w from WL; 7 for each neighbor v of w... Figre 6. Dijkstra s shortest-path algorithm FRAC-KNAP(v : int real, c : int real, Bdget : v = c = n 1 for i := 0 to (n 1) 2 do sed[i] := 0 ; 3 cr c := Bdget; 4 while cr c > 0 5 do choose item m sch that t = (v[m]/c[m]) is maximal and sed[m] = 0; 6 sed[m] := 1; cr c := cr c c[m] 7 tot v := tot v +v[m]; 8 if cr c < 0 9 then tot v := tot v v[m]; 10 tot v := tot v +(1 + cr c /c[m]) v[m] Figre 7. Greedy Fractional Knapsack easy to see that a proof of continity for this abstraction implies a proof of continity of the algorithm with a correct, deterministic implementation of tie-breaking. Let s call this abstraction Dijk. Usally, sch abstractions correspond to the psedocode of the algorithm nder consideration, and are easily bilt from code. While the abstraction Dijk may seem to be nondeterministic, in reality it is not every initial state here leads to a niqe terminal state. Also, as d contains weights of shortest paths on termination, the jdgment tre Cont(Dijk, {G}, {d}) is sond. Proving it, however, is challenging. Assme the indctive hypothesis that d only changes slightly de to a small change to G. Now sppose that before the pertrbation, nodes w 1 and w 2 were tied in the vale of d[ ] and we chose w 1, and that after the pertrbation, we choose w 2. Clearly, this can completely change the vale of d at the end of Line 10. Ths, the continity of d is not an indctive property. However, consider a maximal set of sccessive iterations in an exection processing elements tied on d[ ]. Let s view this collection of iterations sbseqently called an epoch as a map from an inpt state σ 0 to a final vale of d. It so happens that this map is robst to permtations i.e., if σ 0 is fixed, then however we reorder the iterations in the collection, so is the vale of the array d at the state σ 1. Second, small pertrbations to σ 0 can lead to arbitrary reorderings of the iterations however, they only lead to small pertrbations to the vale of d in σ 1 (on the other hand, the vale of prev may change completely). This is the insight we se in or proof rle. Example 11 (Greedy Fractional Knapsack). Consider the Knapsack problem from combinatorial optimization. We are given a set of items {1,..., n}, each item i being associated with a cost c[i] and a vale v[i] (we assme that c and v are given as arrays of non-negative reals). We are also given a non-negative, real-valed bdget. The goal is to identify a sbset sed {1,..., n} sch that the constraint j sed c[i] Bdget is satisfied, and the vale of tot v = j sed v[i] is maximized. Let the observable variable be tot v; as small pertrbations can trn previosly feasible soltions infeasible, a program Knap solving this problem correctly is discontinos in the inpts c and Bdget. At the same time, it is continos in the inpt v. Or analysis can establish the continity of Knap in v (see Section 6). For now we focs on the fractional variant of the problem, which has a greedy, optimal, polynomial soltion and is more interesting from the continity perspective. Here the algorithm can pick fractions of items, so that elements of sed can be any real nmber 0 r 1. The goal is to maximize n i=i sed[i] v[i] P ::= all syntactic forms in IMP the form Q below l: while b do θ := vale U sch that Γ[] is minimized; R(θ, Γ, U) Figre 8. The langage LIMP (P represents programs). while ensring that n i=i sed[i] c[i] Bdget. This algorithm is continos in all its inpts, as we can adjst the qantities in sed infinitesimally to satisfy the feasibility condition even when the inpts change infinitesimally. To see why proving this is hard, consider a program FracKnap coding the algorithm (Fig. 7). Here, cr c tracks the part of the bdget yet to be spent; the algorithm greedily adds elements to sed, compensating with a fractional choice when cr c becomes negative. Line 5 involves choosing an item m sch that (v[m]/c[m]) is maximal, and once again, we abstract this choice by nondeterminism. It is now easy to see that continity of tot v is not indctive; one can also see that the observations made at the end of Example 10 apply. However, one difference is that the the condition of the main loop (Line 4) here can be affected by slight changes to cr c. Therefore, proving this program reqires a more sophisticated se of the synchronized termination than what we saw before A langage of nondeterministic abstractions Let s now develop a rle that can handle the isses raised by these examples. To express or conservative abstractions, we extend the langage IMP with a syntactic form for loops with restricted nondeterministic choice. We call this extended langage LIMP. Its syntax is as in Figre 8. Here: U is a set the iteration space for the loop in the syntactic form Q. Its elements are called choices. θ is a special variable, called the crrent choice variable. Every iteration starts by picking an element of U and storing it in θ. Γ is a real-valed array with Dom(Γ) = U. If is a choice, then Γ[] is its weight. The weight acts as a selection criterion for choices iterations always select minimal-weight choices. Mltiple choices can have the same weight, leading to nondeterministic exection. R(θ, Γ, U) (henceforth jst R) is an IMP program that does not write to θ. It can read θ and read or pdate the iteration space U and the weight array Γ. We call a program of form Q an abstract loop henceforth, Q denotes an arbitrary, fixed abstract loop. For simplicity, we only consider the analysis of abstract loops an extension to all LIMP programs is easy. Also, we restrict orselves to programs that terminate on all inpts. The main loops in or codes in Figres 1 and 7 are abstract loops. For example, the workset WL, the node w, and the array d in Figre 1 respectively correspond to the iteration space U, the choice variable θ, and the map Γ of choice weights. While the weight array is not an explicit variable in Figre 7, it can be added as an axiliary variable. by a simple instrmentation rotine. The fnctional semantics of Q is defined in a standard way. De to nondeterminism, Q may have mltiple exections from a state; conseqently, [[Q]] comprises mappings of the type σ Σ, where σ is a state, and Σ the set of states at which Q may terminate on starting from σ. We skip the detailed indctive definition. Continity. Continity is defined in the style of Def. 3: Q is continos at a state σ 0 in a set In of inpt variables and a set Obs of observable variables if for all ɛ R +, there is a δ R + sch that for all σ 0 satisfying Pert δ,in (σ 0, σ 0), all σ 1 [[Q]](σ 0), and all σ 1 [[Q]](σ 0), we have σ 0 ɛ,obs σ 1. Note that if Q is

9 continos, then for states σ 0, σ 1, and σ 2 sch that {σ 1, σ 2} [[Q]](σ 0), we mst have have σ 1 Obs σ 2. Ths, thogh Q ses a choice constrct, its behavior is not really nondeterministic. Trace semantics and invariants. De to nondeterminism, the trace semantics for abstract loops is richer than that for IMP. Let s denote the body of the top-level loop of Q by B. For U, let the parameterized iteration B be the program (θ := ; R) that represents an iteration of the loop with θ set to. For states σ, σ, we say that there is a -labeled transition from σ to σ, and write σ σ, if (1) at the state σ, Γ[] is a minimal weight in the array Γ; and (2) the Hoare triple {σ}b {σ } holds. Intitively, σ and σ are states at the loop header (label l) in sccessive iterations. Condition (1) asserts that is a permissible choice for Q at σ. Condition (2) says that assming is the chosen vale for θ in a loop iteration, σ is the state at its end. Note that or transition system is deterministic i.e., for fixed σ and, there is at most one σ sch that σ σ. Let, be nonempty seqences over U, and let U. We say that there is a -labeled transition from σ to σ if one of the following conditions holds: = and σ σ, =., and there exists a state σ sch that: (1) σ σ, (2) σ satisfies the loop condition b, and (3) σ σ. A trace of Q from a state σ in is now defined as a seqence π = σ n σ1... σn σn+1, where σ 0 = σ in, and for each 0 i n, σ i i σi+1 and σ i satisfies b. Here, the transition σ i i σi+1 represents a seqence of loop iterations leading Q from σ i to σ i+1. Note that σ n+1 may not satisfy b if it does not, then it is the terminal state of Q. If each i is of the form i U, then π is said to be a U-trace. Clearly, Q can have mltiple traces from a given state. At the same time, if = m and there is a transition σ 0 σ m+1, then Q has a niqe U-trace of the form σ 0 0 σ1... m σ m+1. We denote this trace by Expose(σ 0 σ m+1). For an initial condition c, a state σ is reachable from c if it appears in some trace from a state satisfying c. A transition σ σ is reachable from c if σ is reachable from c. The loop invariant I(c) of Q is an overapproximation of the set of states reachable from c The analysis Now we present or continity analysis. As in Section 5, or goal is to obtain a continity jdgment c Cont(Q, X, X), where X is an indctively obtained set of variables. As hinted at in Example 10, we perform indction over clsters of sccessive loop iterations parameterized by choices of eqal weight. We call these clsters epochs. Pleasantly, while the notion of epochs is crcial for or sondness argment, it is invisible to the ser of the rle, who discharges verification conditions jst as before. Verification conditions and rle definition. We start by defining or rle and its verification conditions. Once again, we discharge the conditions of synchronized termination and separability (these are defined as before). In addition, we discharge the conditions of Γ-monotonicity and commtativity. The former property asserts that the weight of a choice does not increase dring exections of Q. Formally, the program Q is Γ- monotonic nder the initial condition c if for all states σ, σ I(c) sch that there is a transition from σ to σ, we have σ(γ[v]) σ (Γ[v]) for all v U. The second condition says that parameterized iterations can be commted. Let s define: σ 0. σ 1 v.. σ 2.. Obs Obs σ 0(Γ[]) = σ 0(Γ[v]) = σ 0 v. σ 1. σ 2 σ 1(Γ[v]) = σ 1(Γ[]) Figre 9. Commtativity Definition 6 (Commtativity). The parameterized iterations B and B v commte nder the initial condition c and the set Obs of observable variables if for all states σ 0, σ 0, σ 1, σ 2 sch that: (1) σ 0 Obs σ 0; (2) σ 0, σ 1, and σ 0 satisfy the loop invariant I(c); (3) {σ 0}B {σ 1} and {σ 1}B v{σ 2}; and (4) σ 0(Γ()) = σ 0(Γ(v)) = σ 1(Γ(v)), there are states σ 1 and σ 2 sch that {σ 0}B v{σ 1}, {σ 1}B {σ 2}, and σ 1 satisfies I(c) σ 1(Γ()) = σ 0(Γ(v)) σ 2 Obs σ 2. The program Q is commtative nder c and the set Obs of variables (written as c Comm(Q, Obs)) if for all, v, B and B v commte nder c. A commtation diagram captring the relationship between σ 0, σ 1, etc. in the above definition is given in Fig. 9. Note that given σ 0,, and v, the states σ 1 and σ 2 are niqe. Also note that the property defined here is stronger than commtativity in the sal sense, as it asserts properties of weights of choices. Or proof rle for abstract loops is now presented in Fig. 10. Intitively, the rle performs indction over seqences of epochs. As we mentioned in Example 10, small pertrbations will reorder loop iterations within an epoch; however, a sbtle implication of or premises is that sch reorderings do not affect continity at the end of each epoch. Before presenting a sondness argment and defining epochs formally, let s apply the rle to or two examples. Example 12 (Dijkstra s algorithm). Let s now revisit or implementation Dijk of Dijkstra s algorithm, and derive the continity jdgment tre Cont(Dijk, X, X), where X = {G, d, WL} (this can be sbseqently weakened to jdgments like tre Cont(Dijk, {G}, {d})). Here, the array d corresponds to Γ in the syntax of LIMP, and lines 6-10 correspond to the program R. First, we observe that Dijk is d-monotonic (and that the reasoning establishing this is simple). Also, X-separability is obvios. As in case of the Floyd-Warshall algorithm, synchronized termination holds as the loop condition, only involving a discrete variable, is nconditionally continos in the set of inpt variables X. Finally, we observe that lines 6-10 are also commtative by or definition. By the rle LOOP, the desired continity jdgment follows. Example 13 (Fractional Knapsack). Now we consider the program FracKnap (Fig. 7), recast as a LIMP program sing an axiliary array Γ sch that at the beginning of each loop iteration, we have Γ[i] = c[i]/v[i]. Let s verify the jdgment tre Cont(FracKnap, X, X), where X = {Γ, Items, cr c, tot v, c, v}. Once again, separability of X is obvios, and Γ-monotonicity and commtativity can be verified with some effort. The synchronized termination condition, however, is more interesting that in the proof of Dijk, as the loop condition (cr c > 0) is not always continos in X. To see that the condition holds, let c be the formla (cr c = 0) captring the set of states where the loop condition is discontinos. Under this condition, Lines 6 10, taken together, are eqivalent to a skip-statement. Therefore, we have tre Term(FracKnap, X). By the rle LOOP, we have tre Cont(FracKnap, X, X). Sondness. Now we sketch an argment for the sondness of the rle LOOP. Let s start by defining epochs formally:

10 (Loop) U, Γ X I(c) Comm(Q, X) Q is Γ-monotonic nder c c Sep(Q, X) I(c) Cont(R, X, X) c Term(Q, X) c Cont(Q, X, X) Figre 10. Proof rle LOOP for programs with loops (Q is an abstract loop, and S Obs Q) Definition 7 (Epochs). Consider a transition η = σ 0 σ m+1, with Expose(σ 0 σ m+1) = σ 0 0 σ1... m σm+1. The transition η is an epoch if: 1. For all 0 j < m, we have σ j(γ[ j]) = σ j+1(γ[ j+1]). m+1 2. Q has no transition σ m+1 σ m+2 sch that σ m(γ[ m]) = σ m+1(γ[ m+1]). The epoch is said to have weight σ 0(Γ( 0)). Intitively, an epoch is a maximal seqence of iterations that agree on choice-weights. For or proofs, we also need a notion of δ-epochs, which are jst like epochs, except they allow a margin of error δ between the weights of the choices made in sccessive iterations. Formally, for δ R +, a transition η as in Definition 7 is a δ-epoch of Q if for some W R, we have: 1. For all 0 j < m, σ j(γ( j)) W < δ. 2. There is no transition σ m+1 m+1 σ m+2 in Q sch that σ m+1(γ( m+1)) W < δ. Note that every U-trace π = σ 0 0 σ1... σm+1 corresponds to a niqe trace Epochize(π) = σ 0 0 σ n 1... σ n+1 sch that σ 0 = σ 0, σ m+1 = σ n+1, and for each i, σ i i σ i+1 is an epoch. This trace represents the breakdown of π into epochs. For δ R +, the trace Epochize δ (π), representing the breakdown of π into δ-epochs, is similarly defined. Now we define a notion of continity for epochs. Definition 8 (Continity of epochs). An epoch η = σ 0 σ 1 of Q is continos with respect to a set In of inpt variables and a set Obs of observable variables if for all ɛ R +, there exists a δ R + sch that for all states σ 0 satisfying Pert δ,in (σ 0, σ 0), every δ-epoch σ 0 σ 1 satisfies the property σ 1 ɛ,obs σ 1. The crx of or sondness argment is that nder the premises of the rle LOOP, every epoch of Q is continos. This is established by the following theorem: Theorem 4. Sppose the following conditions hold for a set of variables X Var(Q) and an initial condition c: 1. Q is Γ-monotonic nder c 4. U, Γ X 2. I(c) Comm(Q, X) 5. c Sep(Q, X) 3. I(c) Cont(R, X, X) 6. c Term(Q, X) Then every epoch of Q reachable from c is continos in inpt variables X and observable variables X. The proof involves a lemma proving the determinism of epochs: Lemma 1. Sppose the premises of Theorem 4 hold. Then if η = σ 0 σ 1 is an epoch reachable from c, then for all epochs η = σ 0 σ 1 sch that σ 0 X σ 0, we have: (1) is a permtation of ; and (2) σ 1 X σ 1. Proof sketch. Let W be the weight of η, and define a variable U W whose vale at a state is the set of choices in U with weight W. As U X, U W has the same vale in X-eqivalent states; as epochs are maximal, η terminates only when b is violated or U W is empty. Withot loss of generality, assme that and are seqences of distinct choices. Sppose is the first choice in that does not appear in ; let = 1 2. Now we have the following possibilities: (a) the exection of 1 added the choice to U W m 0.. σ 0 σ 1 σ 2... Epochize(π). δ0,x. δ1,x. δ2,x σ 0. σ 1. σ Epochize δ (π ) Figre 11. Indction over epochs by setting Γ[] = W ; (b) some iteration B v in, where v, removed the choice from U W by setting Γ[] > W or removing from U; (c) at some point dring the exection of before cold be selected, the loop condition b was violated. Each of these scenarios are rled ot by or assmed conditions. We only show how to handle case (b). As Q is Γ-monotonic, we have the property that if Γ[] > W at some point in, then Γ[] > W at all prior points in i.e., never had the weight W in η. As for being removed from U in before ever being selected, this violates commtativity. As for postcondition (2), it follows from commtativity if is of length two or more. If = for some choice, then the postcondition follows from the separability of X. (As an aside, the above implies that nder the premises of the rle LOOP, epochs are observationally deterministic: epochs starting from X-eqivalent states always end in X-eqivalent states.) Now we establish a lemma connecting each δ-epoch to an epoch to which it is close. (The proof is qite involved de to lack of space, we only give the intitions here.) Consider, for sfficiently small δ, an arbitrary δ-epoch η sch that Expose(η) = σ 1 0 σ 1... n σn+1 and a state σ 0 sch that σ 0 δ,x σ 0. As η is a δ- epoch, it is possible to pertrb every state appearing in Expose(η) by an amont less than δ to get a U-trace π sch that: (1) π starts with σ 0; and (2) if σ i is the i-th state in π, then for all i, we have σ i(γ[ i]) = σ i+1(γ[ i+1]). We can now show that, if the premises of Theorem 4 hold, then this trace can be exected by Q and, in fact, is of the form Expose(η ) for some epoch η of Q. Ths we have: Lemma 2. Assme that the premises of Theorem 4 hold. Then for all ɛ R +, there exists a δ R + sch that for all δ-epochs η = σ 0 σ 1 and all states σ 0 sch that σ 0 δ,x σ 0, there is an epoch η = σ 0 σ 1 sch that: (1) =, and (2) σ 1 ɛ,x σ 1. Proof sketch for Theorem 4. Now we can establish Theorem 4. Let η = σ 0 σ 1 be any epoch, and let ɛ R +. Select a δ small enogh for Lemma 2 to hold. Consider any δ-epoch η = σ 0 σ 1 sch that σ 0 δ,x σ 0. By Lemma 2, there is an epoch η = σ 0 σ 1 sch that σ 1 ɛ,x σ 1. As η and η are epochs from the same state, by Lemma 1, we have σ 1 X σ 1. Bt this means that σ 1 ɛ,x σ 1. This establishes the continity of η. Sondness for rle LOOP now follows in a straightforward way. The argment is similar to that for Theorem 3 however, this time we se epochs, rather than individal loop iterations, as the basic steps of indction. While continity may be broken inside an epoch, it is, by Theorem 4, reinstated at its end. Intitively, any two traces of Q starting from arbitrarily close states synchronize reaching observationally close states at the ends of epochs (the sitation is sketched in Figre 11). We have: Theorem 5. The proof rle LOOP is sond. Proof sketch. Consider an arbitrary U-trace π of Q starting from a state σ 0 that satisfies c, the trace Epochize(π) = σ 0 m 0... σ m+1, and any ɛ R +. Select a seqence of δ i s, with δ = δ 0, jst as in Theorem 3, and consider a state σ 0 sch that σ 0 δ,x σ 0.

11 Example Time Simple-loop or No Loop or Term. Expressions (U, Γ(), θ) BbbleSort Simple-loop No InsertionSort (Oter) Simple-loop Yes InsertionSort (Inner) Simple-loop Yes Update(A,j+1,z) SelectionSort (Oter) Simple-loop No A[s] SelectionSort (Inner) Simple-loop No MergeSort ({(, ) 1 n, 1 m}, No Min(A[], B[ ]), (i, j)) Dijkstra (Q, dist[], m) No Bellman-Ford Simple-loop No Floyd-Warshall Simple-loop No Krskal (Q = Edges(G), W (, ), (i, j)) No ({, G F, Prim F }, No W (, ), (v, v )) Frac. Knapsack 0.38 ({1.. n}, v[]/c[], m) Yes Int. Knapsack 3.22 No Loop No A[1..i], Array2Set (A[i.. n 1]) Table 1. Benchmark Examples Let π be any U-trace from σ 0, and let Epochize γ (π ) = σ n σ n+1 (withot loss of generality, assme that n m). By the continity of epochs we have σ 1 δ1,x σ 1. Generalizing indctively, and sing the synchronized termination condition as before, we conclde that σ m+1 ɛ,x σ n+1. As π, π are arbitrary traces, Q is continos. 6. Experiments We chose several classic continos algorithms (mostly from a standard ndergradate text on algorithms [3]) to empirically evalate the precision and completeness of or proof rles. Or rles were able to prove the continity of 11/13 examples that we tried. An important step before the application of or proof rles LOOP is the transformation of loops into abstract loops as described in Section 5.2, which reqires identifying the iteration space U, the crrent choice variable θ, and the weight fnction Γ. Of corse, if the rle SIMPLE-LOOP is applicable, then these steps are not needed. Table 1 describes the rles, and parameters U, θ, and Γ, needed in each of or applications. In some cases, we also needed to introdce some axiliary variables since or framework tracks continity of program fragments with respect to a set of observation variables. (An alternative wold have been to define or framework to track continity of expressions, and in fact, this is another interesting aspect of continity proofs for programs. Sch an extension to or framework is not difficlt, bt we avoided this to keep the presentation of the framework simpler.) The colmn Expressions contains the expressions represented by axiliary variables in the varios examples. Transformation of loops into abstract loops and introdction of axiliary variables were performed manally. However, there are heristics that can be sed to atomate this step. Sorting Algorithms. Consider a sorting algorithm that takes in an array A in and retrns a sorted array A ot. Sch an algorithm is continos in A in a small change to A in can only reslt in a small change to A ot[i], for all i. The observation reqires a bit of thoght as the position of a given element in the otpt array A ot may change completely on small pertrbations. Indeed, consider an algorithm that retrns a representation of the sorted array in the form of an array of indices In into the original inpt array (i.e., i < j A in[in[i]] < A in[in[j]]), rather than a sort of the original array. Sch an algorithm is discontinos in A in. Or proof rles can establish continity of the three standard iterative sorting algorithms: BbbleSort, InsertionSort, Selection- Sort. Or proof rles can also establish continity for MergeSort, bt are nable to establish the continity of Qicksort. This is becase the continity proof for MergeSort is indctive with respect to the recrsive calls (i.e., continity holds at every recrsive call to MergeSort), bt this is not the case with QickSort. This is not nexpected since we have not addressed the interprocedral variant of continity analysis in this paper. The proof of continity for each of the sorting algorithms trns ot to be qite different from each other (sggesting the fndamentally different ways in which these algorithms operate). We point ot some of the interesting aspects of the continity proofs for each of these examples. Bbblesort is the simplest of all where the continity proof is indctive for both its loops, and the interesting part is to establish the continity of the loop-body. This involves proving that the Swap operation that swaps two elements of an array is continos, which reqires an application of proof rle ITE-1. The proof of continity of InsertionSort is also indctive for both its loops. However, establishing continity of the inner loop has two interesting aspects. It reqires an application of the synchronized termination condition, and reqires establishing continity of (the axiliary variable representing) the expression pdate(a, j + 1, z) (note that the loop is actally discontinos in A, bt to indctively prove the continity of the oter loop, we in fact need to prove continity of the inner loop with respect to pdate(a, j + 1, z)). The proof of continity of SelectionSort is also indctive for both its loops. The interesting part is to note that the oter loop is not continos in A. It is actally continos in the expressions A[1..i] and the set Array2Set(A[i,.., n 1]), which sffices to establish the continity of A[1..n] when the loop terminates since i = n otside the loop. Similarly, the inner loop is continos in A[s] as opposed to s, bt this sffices to prove the desired continity property of the oter loop. For MergeSort, the challenging part is to establish the continity of the Merge procedre, which is not indctive, and reqires sing the proof rle LOOP in its generality. Shortest Path Algorithms. The path retrned by any shortest path algorithm is ssceptible to small pertrbations in the edge weights of the inpt graph. However, the vale of the shortest path retrned by the shortest path algorithms is actally continos in the inpt graph. Or proof rles can establish this property for each of the three shortest path algorithms that we considered. Among these, Dijkstra s algorithm is the most interesting one, reqiring se of the proof rle LOOP in its generality. The continity proof of Bellman- Ford and Floyd Warshall is relatively easy since it is indctive. Minimm Spanning Tree Algorithms. The spanning tree retrned by minimm spanning tree algorithms can vary widely pon small pertrbations in the edge weights of the inpt graph. However, the weight of the minimm spanning tree retrned by the minimm spanning tree algorithms is actally continos in the edge weights of the inpt graph. Or proof rles can establish this property for Krskal and Prim algorithms, bt fail for Borvka s algorithm. The continity proofs for both Krskal and Prim algorithms are not indctive and reqire an application of the proof rle LOOP in its generality. However, or proof rles are not precise enogh to establish the continity of Borvka s algorithm. Knapsack Algorithms. The integer-knapsack algorithm takes as inpt a weight array c and a vale array v containing the weight and vale respectively of varios objects, and a knapsack capacity Bdget and retrns the set of items with maximm combined vale tot v sch that their combined weight is less than the knapsack capacity. The vale of tot v is discontinos in c and Bdget since small pertrbations may make an object now no longer fit in the knapsack (or the other way rond). However, it is interesting to

12 KRUSKAL(G : graph) 1 for each node v in G do C[v] := {v}; 2 Q := set of all edges in G; cost := 0; T := ; 3 while Q 4 do choose edge (v, w) Q sch that G(v, w) is minimal; 5 remove (v, w) from Q; 6 if C[v] C[w] 7 then add edge (v, w) to T ; 8 cost := cost +G(v, w); 9 C[v] := C[w] := C[v] C[w]; PRIM(G : graph) 1 for each node v in G 2 do d[v] := ; parent[v] := UNDEF; 3 s := arbitrary node in G; d[s] := 0; 4 cost := 0; F := {s}; 5 while F < G 6 do choose node v / F with a minimal-cost edge (v, v ) into F 7 F := F {v}; cost := cost + G(v, v ); 8 for each neighbor w of v 9 do if d[w] > d[v] + G(v, w); 10 then d[w] := d[v] + G(v, w); 11 parent[w] := v; MERGESORT(A : realarr) 1 if A 1 2 then retrn A; 3 m := A /2 ; 4 A 1 := A[0.. m]; A 2 := A[m A 1]; 5 B 1 := MERGESORT(A 1); 6 B 2 := MERGESORT(A 2); 7 retrn MERGE(B 2, B 2); MERGE(A 1 : realarr, A 2 : realarr) 1 i := 0; j := 0; k := 0; 2 while k < A 1 + A 2 3 do if (i A 1 ) or (A 1[i] > A 2[j]) 4 then reslt[k] := A 2[j]; 5 j := j + 1; k := k + 1; 6 else reslt[k] := A 1[i]; 7 i := i + 1; k := k + 1; 8 retrn reslt; BELLMAN-FORD(G : graph, src : node) 1 for each node v in G 2 do d[v] := ; parent[v] := UNDEF; 3 d[src] := 0; 4 for each node in G 5 do for each edge (v, w) of G 6 do if d[v] + G(v, w) < d[w] 7 then d[w] := d[v] + G(v, w); 8 parent[w] := v; KNAPSACK(v : realarr, c : realarr, j : int, W : real, ) 1 if j = 0 then retrn 0 2 else if W = 0 then retrn 0 3 else if (c[j] > W ) then retrn KNAPSACK(v, c, j 1, W ) 4 else z 1 := KNAPSACK(v, c, j 1, W ); 5 z 2 := KNAPSACK(v, c, j 1, W c[j]); 6 retrn max{z 1, z 2} INSERTION-SORT(A : realarr) 1 for i := 1 to ( A 1) 2 do z := A[i]; j := i 1; 3 while j 0 and A[j] > z 4 do A[j + 1] := A[j]; j := j 1; 5 A[j + 1] := z; SELECTION-SORT(A : realarr) 1 for i := 1 to ( A 1) 2 do s := i; 3 for j := i + 1 to ( A 1) 4 do if (A[j] < A[s]) s := j; 5 swap(a[i], A[s]); BUBBLE-SORT(A : realarr) 1 for i := 1 to ( A 1); 2 do for j := 1 to ( A 1); 3 do if (A[i] > A[i + 1]) 4 then swap(a[i], A[i + 1]); Figre 12. Psedocode for experiments note that tot v is actally continos in v. Or proof-rles are able to establish this property indctively across the different recrsive calls of Knapsack after proving that continity of the recrsionfree part, which reqires an application of proof rle ITE-1. As for fractional knapsack, it was proved as in Example 13. Implementation and Experimental Setp. The method has been implemented in C# relying on the Z3 SMT solver to discharge proof obligations and the Phoenix Compiler Framework to process the inpt program. The analysis is implemented as a fixpoint comptation to find the soltion of dataflow eqations derived from the proof rles, where some proof obligations are sbmitted to the SMT-solver in the process. The SMT-solver is sed by only some of the proof rles e.g., the rle ITE-1 reqires the proof of eqivalence of branches at the discontinities of the condition variable, and the se of a continity specification for, say, division reqires s to determine if the divisor is non-zero. Commtativity and early termination proofs, reqired to prove continity of loops, are also sbmitted to the SMT-solver. As mentioned earlier, we manally rewrote some of the programs to fit the abstraction langage LIMP. Also, a few examples involved a nested loop inside the abstract loop. In these, we hand-wrote an invariant for the inner loop (however, this step can be atomated sing more powerfl invariant generators). The performance reslts reported in table 1 were obtained on a Core2 Do 2.53 Ghz with 4GB of RAM. 7. Conclsion and ftre work We have presented a program analysis to atomatically determine if a program implements a continos fnction. The practical motivation is the verification of robstness properties of programs whose inpts can have small amonts of error and ncertainty. This work is the first in a planned series of papers on the analysis of robstness and stability of programs, and its applications in the verification of software rnning on cyber-physical systems [12]. In particlar, we plan to explore the following qestions: Qantitative analysis. Rather than knowing whether a program is continos, what if we want bonds on changes to otpts on small changes to inpts? We plan to answer this qestion by developing a qantitative continity analysis. Sch an analysis will be closely related to the problem of differentiating or finitedifferencing programs. Safe handling of discontinities. Many practical programs are discontinos bt still safe. For example, a controller that is otherwise continos might switch itself off discontinosly when it changes mode. How do we reason abot robstness in sch settings? A possible soltion is to allow a specification for how the program shold behave at discontinities. A robstness proof now establishes that at every inpt state, the program is either continos, or follows the specification. Conterexample generation. Can we generate inpts that, when changed slightly, case large changes in the program s behavior? Modlar analysis. What abot interprocedral continity analysis, hinted at in Sec. 3? Applications otside robstness. Does or proof methodology have applications in a contexts otside of robstness? In particlar, or proof rle LOOP establishes the observational determinism of nondeterministic abstractions expressible in the langage LIMP. Can it be sed in determinism proofs for concrrent programs [22]? Stability. Can we extend the techniqes here to do program analysis with respect to control-theoretic stability properties e.g., asymptotic and Lyapnov stability [16, 17]? References [1] Rajeev Alr, Costas Corcobetis, Thomas A. Henzinger, and Pei- Hsin Ho. Hybrid atomata: An algorithmic approach to the specification and verification of hybrid systems. In Hybrid Systems, [2] Yamine Aït Amer, Gérard Bel, Frédéric Boniol, S. Pairalt, and Virginie Wiels. Robstness analysis of avionics embedded systems. In LCTES, pages , [3] Thomas H. Cormen, Charles E. Leiserson, and Ronald L. Rivest. Introdction to algorithms. MIT Press and McGraw-Hill, [4] Patrick Cosot. Proving the absence of rn-time errors in safetycritical avionics code. In EMSOFT, pages 7 9, 2007.

13 [5] Patrick Cosot, Radhia Cosot, Jérôme Feret, Larent Maborgne, Antoine Miné, David Monniax, and Xavier Rival. The ASTREÉ analyzer. In ESOP, pages 21 30, [6] Eric Gobalt. Static analyses of the precision of floating-point operations. In SAS, pages , [7] Eric Gobalt, Matthie Martel, and Sylvie Ptot. Asserting the precision of floating-point comptations: A simple abstract interpreter. In ESOP, [8] Joseph Halpern. Reasoning abot ncertainty. The MIT Press, [9] Dick Hamlet. Continity in sofware systems. In ISSTA, pages , [10] Mats Per Erik Heimdahl, Ynja Choi, and Michael W. Whalen. Deviation analysis: A new se of model checking. Atom. Softw. Eng., 12(3): , [11] Myron Kayton and Walter R. Fried. Avionics navigation systems. Wiley-IEEE, [12] Edward A. Lee. Cyber physical systems: Design challenges. In ISORC, pages , [13] Matthie Martel. Propagation of rondoff errors in finite precision comptations: A semantics approach. In ESOP, pages , [14] Antoine Miné. Relational abstract domains for the detection of floating-point rn-time errors. In ESOP, pages 3 17, [15] Bradford Parkinson and James Spiker. The global positioning system: Theory and applications (Volme II). AIAA, [16] Stefan Pettersson and Bengt Lennartson. Stability and robstness for hybrid systems. In Decision and Control, volme 2, pages , Dec [17] Andreas Podelski and Silke Wagner. Model checking of hybrid systems: From reachability towards stability. In HSCC, pages , [18] Mardavij Roozbehani, Alexandre Megretski, Emilio Frazzoli, and Eric Feron. Distribted lyapnov fnctions in analysis of graph models of software. In HSCC, pages , [19] Ofer Strichman. Regression verification: Proving the eqivalence of similar programs. In CAV, [20] Wilson Stherland. Introdction to metric and topological spaces. Oxford University Press, [21] John Taylor. An introdction to error analysis: the stdy of ncertainties in physical measrements. University Science Books, [22] Tachio Terachi and Alex Aiken. A capability calcls for concrrency and determinism. In CONCUR, pages , [23] Glynn Winskel. The formal semantics of programming langages. The MIT Press, 1993.