Test Input Generation with Java PathFinder

Tes Inpu Generaion wih Java PahFinder Willem Visser RIACS/USRA NASA Ames Research Cener Moffe Field, CA 94035, USA wvisser@email.arc.nasa.gov Corina S. Păsăreanu Kesrel Technology NASA Ames Research Cener Moffe Field, CA 94035, USA pcorina@email.arc.nasa.gov Sarfraz Khurshid UT ARISE Universiy of Texas a Ausin Ausin, Texas 78712, USA khurshid@ece.uexas.edu ABSTRACT We show how model checking and symbolic execuion can be used o generae es inpus o achieve srucural coverage of code ha manipulaes complex daa srucures. We focus on obaining branch-coverage during uni esing of some of he core mehods of he red-black ree implemenaion in he Java TreeMap library, using he Java PahFinder model checker. Three differen es generaion echniques will be inroduced and compared, namely, sraigh model checking of he code, model checking used in a black-box fashion o generae all inpus up o a fixed size, and lasly, model checking used during whie-box es inpu generaion. The main conribuion of his work is o show how efficien whie-box es inpu generaion can be done for code manipulaing complex daa, aking ino accoun complex mehod precondiions. Caegories and Subjec Descripors: D.2.4 [Sofware Engineering]: Tesing and Debugging Symbolic Execuion General Terms: Algorihms, Verificaion Keywords: Tesing Objec-oriened Programs, Model Checking, Symbolic Execuion, Coverage, Red-Black Trees 1. INTRODUCTION Sofware esing, he mos commonly used echnique for validaing he qualiy of sofware, is a labor inensive process, and ypically accouns for abou half he oal cos of sofware developmen and mainenance [9]. Auomaing esing would no only reduce he cos of producing sofware bu also increase he reliabiliy of modern sofware. A recen repor by he Naional Insiue of Sandards and Technology esimaes ha sofware failures currenly cos he US economy abou $60 billion every year, and ha improvemens in sofware esing infrasrucure migh save one-hird of his cos [1]. Auomaed es case generaion has been well sudied in he lieraure (see Secion 6), bu mos of his work has focused on he generaion of es inpus conaining simple Permission o make digial or hard copies of all or par of his work for personal or classroom use is graned wihou fee provided ha copies are no made or disribued for profi or commercial advanage and ha copies bear his noice and he full ciaion on he firs page. To copy oherwise, o republish, o pos on servers or o redisribue o liss, requires prior specific permission and/or a fee. ISSTA 04, July 11 14, 2004, Boson, Massachuses, USA. Copyrigh 2004 ACM 1-58113-820-2/04/0007 $5.00. (unsrucured) daa. In his paper we ll address he problem of doing es inpu generaion for code ha manipulaes complex daa srucures. The main research challenge in his area is how o do efficien es inpu generaion ha will obain high code coverage we will show how symbolic execuion over complex daa can address his problem. Model checking [13] has been hugely popular for he las wo decades. More recenly he applicaion of model checking o he analysis of sofware programs has also come o he fore [7,15,27,43]. Model checking programs however is hard due o he complexiy of he code and i ofen canno compleely analyze he program s sae space since i runs ou of memory. For his reason some of he mos popular program model checkers rely on (predicae) absracions [7, 27] o reduce he size of he sae space, bu hese echniques are no well suied for handling code ha manipulaes complex daa hey inroduce oo many predicaes, making he absracion process inefficien. We will show ha alhough a program model checker (wihou relying on absracion) canno always achieve good code coverage when dealing wih programs manipulaing complex daa, augmening i wih symbolic execuion (which can be seen as a form of absracion), can resul in he generaion of ess ha will achieve high code coverage. There has been an acive research communiy invesigaing he generaion of es inpus wih he use of model checking [3,4,21,26,28] he focus is on specificaion-based es inpu generaion (i.e. black-box esing) where coverage of he specificaion is he goal. Model checking lends iself o es inpu generaion, since one simply specifies as a se of (emporal) properies ha a specific coverage canno be achieved and he model checker will find counerexamples, if hey exis, ha hen can easily be ransformed ino es inpus o achieve he saed coverage goal. Symbolic execuion has long been advocaed as a means for doing efficien es inpu generaion [31], bu mos of he ensuing research has focused on generaing ess for simple daa ypes (inegers for he mos par). In previous work [30] we developed a verificaion framework based on symbolic execuion and model checking ha handles dynamically allocaed srucures (e.g. liss and rees), simple (primiive) daa (e.g. inegers and srings) and concurrency. The framework uses mehod precondiions o iniialize fields only wih valid values and mehod poscondiions as es oracles o es a mehod s correcness. We show here how we used and exended he symbolic execuion framework from [30] o perform auomaed es inpu generaion for uni esing of Java programs. To gen- 97

erae inpus ha mee a given esing crierion for a paricular mehod under es, we model check he mehod. The esing crierion is encoded as a se of properies he model checker should check for. Counerexamples o he properies represen pahs ha saisfy he coverage crierion. Symbolic execuion (which is performed during model checking) compues a represenaion (a se of consrains) of all he inpus ha execue hose pahs. The acual esing requires solving he inpu consrains in order o insaniae es inpus hacanhenbeexecued. The framework uses lazy iniializaion, i.e. i iniializes he componens of he mehod inpus on an as needed basis, wihou requiring an a priori bound on inpu sizes. While [30] describes in deail how symbolic execuion and lazy iniializaion are used during model checking, we highligh here he use of complex precondiions during lazy iniializaion, o iniialize he inpus only wih valid values. In paricular, we highligh a key feaure of our framework: he use of precondiions ha are conservaive, i.e. hey may be evaluaed on parially iniialized srucures and reurn false only if he iniialized fields of he inpu srucure violae a consrain in he precondiion. This imporan feaure was menioned briefly in [30]; we elaborae on i here. We show how lazy iniializaion in combinaion wih he use of conservaive precondiions during iniializaion o eliminae incorrec srucures resuls in a powerful and efficien way of performing symbolic execuion of code ha manipulaes complex srucured daa. We also show here how we solve he inpu consrains in order o ge he es inpus ha are necessary for he acual esing. To illusrae he flexibiliy of our framework, we conras he above whie-box echnique wih a black-box echnique where we use he mehod precondiions o sysemaically generae all he (non-isomorphic) es inpus up o a given size; his is done by symbolically execuing he code for he precondiion. This laer approach has similariies o he echniques employed in he Kora ool [10] which also execues he code for he precondiion, bu i does no use symbolic execuion (for he primiive daa) and lazy iniializaion. We evaluae our approaches by generaing ess for he red-black ree [16] implemenaion in he Java TreeMap library. The conribuions of our work are: A powerful and flexible es inpu generaion framework for uni esing. The framework uses an efficien approach o he symbolic execuion of code manipulaing complex daa srucures, ha akes ino accoun precondiions o sop he analysis of infeasible pahs as soon as possible. The framework can be used uniformly boh for whie-box and black-box esing. We show how our framework can be used for generaing ess for code manipulaing complex daa srucures - specifically red-black rees. We illusrae he flexibiliy of model checking as a ool for es inpu creaion by comparing sraigh model checking of he code under es, a black-box approach and a whie-box approach. 2. BACKGROUND We describe here he Java PahFinder (JPF) model checker [43] ha has been exended wih a symbolic execuion cain x, y; 1:if (x > y) { 2: x = x + y; 3: y = x - y; 4: x = x - y; 5: if (x > y) 6: asser(false); x: Y, y: X PC: X>Y & Y>X FALSE! 1 x: X, y: Y PC: X>Y 2 x: X+Y, y: Y PC: X>Y 3 x: X+Y, y: X PC: X>Y 4 x: Y, y: X PC: X>Y 5 5 x: X, y: Y PC: rue 1 x: X, y: Y PC: X<=Y x: Y, y: X PC: X>Y & Y<=X Figure 1: Code ha swaps wo inegers and he corresponding symbolic execuion ree, where ransiions are labelled wih program conrol poins pabiliy. We show in Secion 4 how we use his exension of JPF for whie-box and black-box es inpu generaion. 2.1 Java PahFinder JPF is an explici-sae model checker for Java programs ha is buil on op of a cusom-made Java Virual Machine (JVM). JPF can handle all of he language feaures of Java and i also reas nondeerminisic choice expressed in annoaions of he program being analyzed annoaions are added o he programs hrough mehod calls o a special class Verify. The following mehods from he Verify class will be used in his paper: randombool() reurns a boolean value nondeerminisically. random(n) reurns values [0,n] nondeerminisically. ignoreif(cond) forces he model checker o backrack when cond evaluaes o rue. JPF has previously been used o find errors in a number of complex sysems including he real-ime operaing sysem DEOS from Honeywell [39] and a prooype Mars Rover developed a NASA Ames (called K9) [11]. More recenly i was also used as a means for generaing inpu plans ha he curren K9 rover akes as inpu [5] he plans were generaed in a black-box fashion similar o he echnique described in secion 4.2. 2.2 Symbolic Execuion in Java PahFinder In his secion we give some background on symbolic execuion and we presen he symbolic execuion framework used for reasoning abou Java programs. 2.2.1 Background: Symbolic Execuion The main idea behind symbolic execuion [31] is o use symbolic values, insead of acual daa, as inpu values and o represen he values of program variables as symbolic expressions. As a resul, he oupus compued by a program are expressed as a funcion of he symbolic inpus. The sae of a symbolically execued program includes he (symbolic) values of program variables, a pah condiion (PC) 98

and a program couner. The pah condiion is a (quanifier free) boolean formula over he symbolic inpus; i accumulaes consrains which he inpus mus saisfy in order for an execuion o follow he paricular associaed pah. The program couner defines he saemen o be execued. A symbolic execuion ree characerizes he execuion pahs followed during he symbolic execuion of a program. The nodes represen program saes and he arcs represen ransiions beween saes. Consider he code fragmen in Figure 1, which swaps he values of ineger variables x and y, whenx is greaer han y. Figure 1 also shows he corresponding symbolic execuion ree. Iniially, PC is rue and x and y have symbolic values X and Y, respecively. A each branch poin, PC is updaed wih assumpions abou he inpus, in order o choose beween alernaive pahs. For example, afer he execuion of he firs saemen, boh hen and else alernaives of he if saemen are possible, and PC is updaed accordingly. If he pah condiion becomes false, i.e. here is no se of inpus ha saisfy i, his means ha he symbolic sae is no reachable, and symbolic execuion does no coninue for ha pah. For example, saemen (6) is unreachable. 2.2.2 Generalized Symbolic Execuion In [30] we describe an algorihm for generalizing radiional symbolic execuion o suppor advanced consrucs of modern programming languages, such as Java and C++. The algorihm handles dynamically allocaed srucures, primiive daa and concurrency. We have since exended he work in [30] by adding suppor for symbolic execuion of arrays. The algorihm sars execuion of a mehod on inpus wih uniniialized fields and uses lazy iniializaion o assign values o hese fields, i.e. i iniializes fields when hey are firs accessed during he mehod s symbolic execuion. This allows symbolic execuion of mehods wihou requiring an a priori bound on he number of inpu objecs. When he execuion accesses an uniniialized reference field, he algorihm nondeerminisically iniializes he field o null, o a reference o a new objec wih uniniialized fields, or o a reference of an objec creaed during a prior field iniializaion; his sysemaically reas aliasing. When he execuion accesses an uniniialized primiive (or sring) field, he algorihm firs iniializes he field o a new symbolic value of he appropriae ype and hen he execuion proceeds. When a branching condiion on primiive fields is evaluaed, he algorihm nondeerminisically adds he condiion or is negaion o he corresponding pah condiion and checks he pah condiion s saisfiabiliy using a decision procedure. If he pah condiion becomes infeasible, he curren execuion erminaes (i.e. he algorihm backracks). 2.2.3 Framework Our symbolic execuion framework is buil on op of he JPF model checker. To enable JPF o perform symbolic execuion (and lazy iniializaion), he original program is insrumened by doing a source o source ranslaion ha adds nondeerminism and suppor for manipulaing formulas ha represen pah condiions 1. 1 The ineresed reader is referred o [30] for a deailed descripion of he code insrumenaion class Node { in elem; Node ; /* precondiion: acyclic() */ void foo() { 1: if (elem >.elem) 2: =.; Figure 2: Simple example o illusrae generalized symbolic execuion The model checker checks he insrumened program using is usual sae space exploraion echniques essenially, he model checker explores he symbolic execuion ree of he program. A sae includes a heap configuraion, a pah condiion on primiive fields, and hread scheduling. Whenever a pah condiion is updaed, i is checked for saisfiabiliy using an appropriae decision procedure, such as he Omega library [40] for linear ineger consrains. If he pah condiion is unsaisfiable, he model checker backracks. The framework can be used for es inpu generaion and for finding counerexamples o safey properies. For es inpu generaion, he model checker generaes pahs ha are winesses o a esing crierion encoded as a se of properies. For every repored pah, he model checker also repors he inpu heap configuraion (encoding consrains on reference fields), he pah condiion for he primiive inpu fields, and hread scheduling, which can be used o reproduce he error. Noe ha performing (forward) symbolic execuion on programs wih loops can explore infinie execuion rees. This is why, for sysemaic sae space exploraion, he framework uses deph firs search wih ieraive deepening or breadh firs search. Alhough we concenrae in his paper on he analysis of sequenial code, i is worh menioning ha our symbolic execuion framework handles concurrency, as i uses he model checker o sysemaically analyze hread inerleavings. Using a model checker as a search engine for our framework allows us o also exploi oher buil-in capabiliies of he model checker, such as backracking, differen search capabiliies (e.g. heurisic search), and echniques ha comba sae-explosion (e.g. parial order and symmery reducions). We should also noe ha, alhough we consider here branch coverage as a meric for esing, our framework can handle oher esing crieria ha can be encoded as properies he model checker should check for (e.g. daa-flow based coverage). 2.2.4 Illusraion We illusrae he generalized symbolic execuion on a simple example (see Figure 2). Class Node implemens singlylinked liss; he fields elem and represen, respecively, he node s ineger value and a reference o he node. Figure 3 gives (par of) he corresponding code ha was insrumened for symbolic execuion: concree ypes were replaced wih symbolic ypes (library classes ha we provide) and concree operaions wih mehod calls ha implemen equivalen operaions on symbolic ypes. Class Expression suppors manipulaion of symbolic inegers. 99

class Node { Expression elem; Node ; boolean is_iniialized = false; boolean _elem_is_iniialized = false; saic Vecor v = new Vecor(); saic {v.add(null); Node _new_node() { in i = Verify.random(v.size()); if(i<v.size()) reurn (Node)v.elemenA(i); Node n = new Node(); v.add(n); reurn n; Node _ge_() { if(! is_iniialized) { is_iniialized=rue; = Node._new_Node(); Verify.ignoreIf(!precondiion());//e.g. acyclic reurn ; void foo() { 1: if(_ge_elem()._gt(._ge_elem())) 2: _se_(._ge_()); E0 E1 E0 Figure 3: Insrumened code E0 E0 E1? 2 2 E1 E0 E1? Inpu srucure: E1 1 X E0 E1 E0 E1 E0 PC: E0>E1 X E1 PC: rue Figure 4: Symbolic execuion ree (excerps) 1 PC: E0<=E1 Field reads and updaes are replaced by ge and se mehods (ge mehods implemen he lazy iniializaion). For each field in he original class declaraion, boolean fields (i.e. is iniialized and elem is iniialized) are added. These fields are se o rue by ge (se) mehods. Vecor v sores he inpu objecs ha are creaed as a resul of lazy iniializaion. The helper mehod new Node, which is called by ge node, uses he elemens in vecor v o sysemaically iniialize inpu reference fields, according o differen aliasing possibiliies. Figure 4 illusraes he pahs ha are generaed during he symbolic execuion of he code of mehod foo, for a given inpu srucure. Each node of he execuion ree denoes a symbolic sae. Branching in he ree corresponds o a nondeerminisic choice ha is inroduced o build a pah condiion or o handle aliasing. Edges labelled wih numbers refer o program saemens and hose wihou numbers o lazy iniializaion seps. The value? for an elem field indicaes ha he field is no iniialized and he cloud indicaes ha he field is no iniialized; null nodes are no represened. Srucures represen consrains on reference fields, e.g. he inpu srucure in Figure 4 represens all (cyclic or acyclic) liss wih a leas wo nodes such ha poins o he second node. As we will explain laer in Secion 4.3, mehod precondiions can be used during lazy iniializaion: if he inpu srucure violaes he precondiion, he model checker backracks (i.e. call o mehod Verify.ignoreIf in Figure 3). For example, if we consider he precondiion ha he inpu lis should be acyclic, he algorihm does no explore he ransiions marked wih an X in Figure 4. 3. CASE STUDY: RED-BLACK TREES We have used our inpu generaion echniques for esing a variey of programs, including mehods of classes in he java.uil package. Mos of hese programs manipulae complex daa srucures. In paricular, we illusrae our echniques on he Java implemenaion of red-black rees given in java.uil.treemap from he sandard Java libraries (version 1.3). Red-black rees [16] are binary search rees wih one exra bi of informaion per node: is color, which can be eiher red or black. By resricing he way nodes are colored on a pah from he roo o a leaf, red-black rees ensure ha he ree is balanced, i.e. guaranee ha basic dynamic se operaions on a red-black ree ake O(log n) ime in he wors case. A binary search ree is a red-black ree if: 1. Every node is eiher red or black. 2. The roo is black. 3. If a node is red, hen boh is children are black. 4. Every simple pah from he roo node o a descendan leaf conains he same number of black nodes. All four of hese red-black properies are expressible in Java. We use hese four properies ogeher wih he basic properies of binary search rees o define a repok mehod, i.e. a Java predicae ha checks he represenaion invarian (or class invarian) of he corresponding daa srucure. In his case, repok checks if he inpu is a red-black ree. Figure 5 gives par of he java.uil.treemap declaraion and Figure 6 gives a fragmen of code from repok represening he hird red-black ree propery: red nodes have only black children. The generaion of red-black rees is paricularly ineresing for our framework, due o heir complex srucure (i.e. hey have primiive fields, back poiners, ec.). In he secion, we presen differen echniques ha we used o generae es inpus for he implemenaion of red-black rees in java.uil.treemap. We will illusrae es inpu generaion for several helper mehods for he pu and remove mehods in class java.uil.treemap, whichare responsible for adding and deleing he node corresponding 100

public class TreeMap { Enry roo; saic final boolean RED = false; saic final boolean BLACK = rue; saic class Enry implemens Map.Enry { Objec key; Objec value; Enry lef; Enry righ; Enry paren; boolean color; /* precondiion: repok(roo); */ public Objec remove(objec key) { public Objec add(objec key) { Figure 5: Declaraion of java.uil.treemap boolean repok(enry e) { // RedHasOnlyBlackChildren worklis = new LinkedLis(); worklis.add(e); while (!worklis.isempy()) { Enry curren=(enry)worklis.removefirs(); Enry cl = curren.lef; Enry cr = curren.righ; if(curren.color == RED) { if(cl!= null && cl.color == RED) { reurn false; if(cr!= null && cr.color == RED) { reurn false; if (cl!= null) worklis.add(cl); if (cr!= null) worklis.add(cr); reurn rue; Figure 6: Mehod repok (excerps). o a given key from he ree. We should noe ha deleion is he mos complex operaion among he sandard operaions on red-black rees and involves roaions. Togeher wih he auxiliary mehods, addiion ogeher wih deleion in java.uil.treemap are abou 300 lines of Java code. The (implici) precondiion for boh pu and remove mehods requires he inpu o saisfy is class invarian (i.e. repok): he inpu mus be a red-black ree. 4. TEST INPUT GENERATION In his secion we will illusrae hree applicaions of model checking o he es inpu generaion of sofware manipulaing a complex daa srucure. We will focus on he implemenaion of he pu and remove mehods for red-black rees in he Java TreeMap library. As a esing crierion we use source code level branch-coverage since we wan o compare he black-box o he whie-box approaches o es inpu generaion. As menioned in Secion 2.2.3 our framework can also handle oher kinds of esing crieria. We aim o generae a se of non-isomorphic ess, which mees he desired coverage crieria. Isomorphism among ess is defined as isomorphism among graphs where he heap of a Java program is viewed as an edge-labeled graph: node ideniies are permuable, while primiive values are no [10]. Noe ha he Java semanics do no allow objec allocaion o dicae he exac objec ideniies, which implies ha iniializing a es inpu (a he concree represenaion level by seing field values or a he absrac level using a sequence of mehod invocaions) more han once (say for regression esing) does no generae idenical srucures bu i generaes isomorphic srucures. Firs we show how a model checker can be used o do he esing, by execuing sequences of mehod calls in he daa srucure s inerface. Secondly, we show how we can use our symbolic execuion framework o build all (non-isomorphic) inpu rees up o a given small size ha are o be used for he (black-box) esing of he mehod. This is done by symbolically execuing he Java code of he mehod s precondiion (in his case he code of repok). Lasly, we show how our framework can be used for whiebox es inpu generaion and how conservaive precondiions are used during lazy iniializaion, o sop he analysis of infeasible pahs. We also show how he inpu consrains compued by symbolic execuion are solved o provide he inpus for he acual esing. 4.1 Model Checking as Tesing When doing model checking here is a clear disincion beween he sysem being analyzed and he environmen of he sysem, i.e. he inpus ha he sysem akes. Whenever he environmen is under-approximaed (less behaviors are considered han are presen in he acual environmen) during model checking hen model checking becomes a form of esing. Noe ha more ofen han no his is he case during he model checking of sofware, since he environmen is usually very large. Considering his connecion, one can herefore use a model checker o generae inpus and analyze he code on hose inpus. To illusrae his idea we show how one can es he Java TreeMap library by analyzing all sequences of pu and remove calls on a se wih maximally N elemens using he JPF model checker (Figure 7). Noe ha, for his example, we are more ineresed in he coverage of he code, raher han correcness and hence we only use he model checker s defaul properies (uncaugh excepions being he mos imporan here) as an oracle in general our approach allows more general oracles, including mehod poscondiions expressed as Java predicaes. 4.2 Inpu Generaion For Black-box Tesing Our framework can be used o auomaically generae Java daa srucures from a descripion of mehod precon- 101

public saic in N = 5; public saic TreeMap = new TreeMap(); public saic Ineger[] elems; saic {elems = new Ineger[N]; for (in i = 0;i< N; i++) elems[i] = new Ineger(i); public saic void main(sring[] args) { while (rue) { if (Verify.randomBool()).pu(elems[Verify.random(N-1)],null); else.remove(elems[verify.random(n-1)]); Figure 7: Model checking as esing diions. Noe ha for sequenial code, generalized symbolic execuion explores only pahs on non-isomorphic inpus. Therefore, we can generae non-isomorphic inpu srucures ha saisfy a precondiion, by applying generalized symbolic execuion o he code of he precondiion. Once we have an inpu srucure, we use off-he-shelf consrain solvers for solving he consrains in he pah condiion, hus obaining he es inpu. Tes inpus can hen be used for black-box esing of he mehod under es. The drawback of his approach is ha here is no relaionship beween he inpus and he code coverage. On he oher hand, if one would be ineresed in covering he inpu specificaion raher han he code under es, his black-box mehod achieves full coverage of he inpu srucures up o a given bound. In order o es he pu and remove mehods we auomaically generaed all (non-isomorphic) inpu rees up o a given small size from he Java descripion of he mehod s precondiion (i.e. he srucural invarian), hus eliminaing he need o consruc he inpus using a sequence of mehod calls. Our framework symbolically execues repok and i generaes he inpu srucures whenever repok reurns rue. We pu a limi on he number of generaed objecs: whenever he size of he vecor ha sores he objecs creaed during lazy iniializaion (see Secion 2) reaches ha limi, he model checker backracks. As a resul, all he inpu srucures saisfying repok wih size up o he specified limi are creaed. Our approach can be conrased wih a brue force approach, where one will firs generae all possible rees up o a given size according o he class definiion, and hen would apply repok o selec only valid red-black rees. Our approach scales beer since we generae rees on demand (wih lazy iniializaion) and we backrack as soon as a redblack ree propery is violaed, hus pruning large porions of he search space. I is imporan o noe ha he acual srucure of repok is crucial o he efficiency of our mehod. If repok would firs evaluae he ree and only a he end deermine wheher he ree is valid, our approach would be equivalen o generaing all rees before pruning. 4.3 Inpu Generaion for Whie-box Tesing Our symbolic execuion framework can be used for inpu generaion during whie box esing. To generae inpus ha mee a given esing crierion, our framework is used o symbolically execue he mehod under es and o model check i agains properies ha encode he esing crierion. Counerexamples o he properies represen pahs ha saisfy he crierion. For every pah, our framework also repors an inpu srucure and a pah condiion on he primiive inpu values, which ogeher define a se of consrains ha he inpus should saisfy in order o execue ha pah 2. A paricular characerisic of our framework is ha i uses mehod precondiions during wo phases of he inpu generaion o eliminae infeasible srucures: a conservaive precondiion, ha can deal wih parially iniialized srucures, is used during lazy iniializaion (see Secion 4.3.1) when a counerexample is found he srucural consrain for he pah is used as inpu o a concree precondiion (he same one used in Secion 4.2) o solve he consrains wih only valid inpus (see Secion 4.3.3) 4.3.1 Conservaive Precondiions We use precondiions in iniializing fields (see Figure 3). In paricular, a field is no iniialized o a value ha violaes he precondiion. Noice ha we evaluae a precondiion on a srucure ha sill may have some uniniialized fields, herefore we require he precondiion o be conservaive, i.e. reurn false only if he iniialized fields of he srucure violae a consrain in he precondiion. Consider he analysis of he remove mehod in he redblack ree implemenaions. The mehod has as precondiion he class invarian of he red-black ree daa srucure (i.e. he repok Java predicae). The conservaive version of he precondiion ha we used during lazy iniializaion is illusraed in Figure 8. Boolean flags lef is iniialized and righ is iniialized were added by our code insrumenaion o keep rack of uniniialized inpu fields (see Secion 2.2.4). The code of conservaive repok is idenical o ha of repok, wih he excepion ha he consrains encoded in repok are only evaluaed on iniialized fields. As an example, assume ha here are hree inpu rees as illusraed in Figure 9 ha are creaed during he analysis of he remove mehod (a a lazy iniializaion sep). Round filled nodes represen enries colored black and empy nodes represen enries colored red; null nodes are no represened. As before, a cloud denoes an uniniialized field inuiively represening a se of nodes, since i can be lazily iniialized o differen nodes. For simpliciy of presenaion, we omi o represen he key and value fields. Figure 9 also shows he resuls of evaluaing he conservaive repok on he ree srucures. The firs ree violaes he repok, nomaer wha he concree value of he cloud is, since red nodes canno have red children. In his case he model checker will backrack and i will no consider his srucure any furher. The second ree is a concree srucure ha saisfies repok. The hird ree represens concree rees ha may or may no saisfy repok; conservaive repok reurns rue (or raher Don Know), and he analysis coninues. We should noe ha he lazy iniializaion of inpu fields in our framework is relaed o maerializaion of summary nodes in shape analysis [33], while he conservaive precondiions can be formulaed in he conex of absrac inerpreaion. We would like o explore hese connecions furher. 2 Our framework also repors he hread scheduling informaion, in he case of muli-hreaded code 102

boolean conservaive_repok(enry e) { // RedHasOnlyBlackChildren worklis = new LinkedLis(); worklis.add(e); while (!worklis.isempy()) { Enry curren=(enry)worklis.removefirs(); Enry cl = curren.lef; Enry cr = curren.righ; if (curren.color == RED) { if (curren._lef_is_iniialized && cl!= null && cl.color == RED) { reurn false; if (curren._righ_is_iniialized && cr!= null && cr.color == RED) { reurn false; if (curren._lef_is_iniialized && cl!= null) worklis.add(cl); if (curren._righ_is_iniialized && cr!= null) worklis.add(cr); reurn rue; Figure 8: Predicae conservaive repok FALSE TRUE TRUE (Don Know) Figure 9: Evaluaion of conservaive repok on 3 srucures 4.3.2 Handling Desrucive Updaes The lazy iniializaion algorihm builds he inpu srucures on an as needed basis, when hey are firs accessed during symbolic execuion. If he code under analysis performs desrucive updaes, he srucure of he inpus can be los. To creae es inpus we herefore need o reconsruc hese inpu srucures. As an example, consider he srucures in he leaves of he symbolic execuion ree depiced in Figure 4, which are he resul of a desrucive updae; hese srucures no longer conain he informaion ha in he inpu srucure, here is a link beween he firs wo nodes. In order o recover he original inpu srucures, we keep mappings beween objecs wih uniniialized fields and objecs Symbolic inpu srucure saisfying conservaive_repok() Soluion ha saisfies repok() No a soluion! (Consrain 4 violaed.) Figure 10: Concreizaion of symbolic srucures ha are creaed when hose fields are iniialized; hese mappings are used o reconsruc inpu srucures. Noe ha whenever he precondiion needs o be evaluaed during lazy iniializaion, we firs reconsruc he inpu srucures and hen we evaluae he precondiion on hese reconsruced inpu srucures. 4.3.3 Solving Consrains The resul of symbolically execuing a paricular program pah is a heap srucure, ha encodes consrains on reference fields, and a pah condiion, ha encodes consrains on primiive daa. These consrains define he inpus ha will allow he execuion of he pah. In order o obain he acual es inpus, we have o solve hese consrains, i.e. we have o build acual Java daa srucures ha can be used during esing. In our framework, we solve hese consrains separaely for srucures and primiive daa. We firs conver symbolic srucures o concree srucures ha have no uniniialized fields and ha also saisfy he mehod s precondiion. If here are no mehod precondiions, his amouns o simply seing all he uniniialized fields o null. Oherwise, he symbolic inpu srucure (which saisfies he conservaive precondiion) is used as inpu o he code for he concree precondiion which is symbolically execued o obain he concree srucure, in a way similar o he es inpu generaion mehod described in Secion 4.2. The consrains in he pah condiion are hen solved using an off-he-shelf consrain solver. We should noe ha no all he concreizaions of a symbolic srucure ha saisfies he conservaive precondiion, are valid soluions. An example is given in Figure 10. 5. EXPERIMENTS As menioned before we measured branch-coverage for he mehods of he TreeMap class and for simpliciy we will only consider here 3 helper mehods: deleeenry (del), fixaferdeleeion (fixd) and fixaferinserion (fixi). The opimal 3 branch-coverage ha can be achieved for hese mehods are 86% (19 ou of 22) for deleeenry, 100% (20 ou of 20) for fixaferdeleion and 88% (14 ou of 16) for fixaferinserion. Noe ha 100% saemen coverage can be obained for all 3 mehods - he uncovered branches are all for missing else branches where he if opion is always aken. 3 The res of he branches can be shown o be infeasible. 103

Resources % Coverage N Time Mem Saes del fixd fixi 1 1.7 2.2 28 18 0 0 2 1.9 2.2 104 68 5 6 3 2.4 2.3 730 68 5 75 4 7.8 3 10109 86 90 88 5 127 8.8 194525 86 90 88 6 Ou of Memory Table 1: Model Checking as Tesing All resuls were obained using he JPF model checker (version 3.1.1) on a 2.2 Ghz dual processor Penium wih 1 Gb of memory running Windows 2000 wih Java 1.4.2. 5.1 Model Checking as Tesing The resuls in Table 1 show he coverage achieved in he hree mehods when model checking he code in Figure 7 for differen values of N (he number of enries o be added o he TreeMap). The coverage numbers were gahered using JPF s faciliy o calculae branch-coverage on he byecode level during execuion a simple ransformaion was done o obain branch-coverage on he source code level. Noe ha his echnique is also used for he branch-coverage calculaions in he res of he secion. The able also conains he ime aken (in seconds), he memory consumed (in Mb) and he number of saes generaed during model checking. The resuls indicae ha his approach does no scale well. Unil size 4 he resuls give he appearance of acually being very good (fas, low memory and reasonable coverage), bu he exponenial explosion becomes apparen a size 5 and size 6 canno be handled in is enirey. For fixaferdeleion, 100% branch-coverage is no achieved; fixaferdeleion is called from deleeenry hence i is no oo surprising ha here is a large jump in is coverage fromsize3o4. An advanage of he model checking approach is ha i obains good pah (behavioral) coverage for small inpu domains as well as for sysems where esing is fundamenally hard, such as concurren sysems. In he domain considered here, namely (sequenial) programs manipulaing complex daa, his echnique of esing could be an appropriae firs pass a finding errors, bu o obain good srucural coverage one needs a more sophisicaed approach. 5.2 Black-box Table 2 shows he resuls for black-box srucural coverage up o a fixed size (N). The inpu rees were generaed by doing a symbolic execuion of he repok mehod using JPF. For each of he inpu rees all possible node deleions and one new inserion were hen execued (Tess) and he code coverage measured. We also repor he oal number of rees creaed (Srucs) up o a specific size N (in parenhesis only he number of rees of size N is given) as well as how many srucures where considered (Candidaes) by he lazy iniializaion of he code wihin repok. Since he memory consumpion was minimal (less han 10Mb for he cases shown) we only repor on he ime aken o generae he rees he ime for running he ess were negligible. Noe ha since he srucures are generaed up o a given size, he resuls include all he smaller srucures as well. Saisics % Coverage N Srucs Candidaes Time Tess del fixd fixi 1 1(1) 5 2.4 2 18 0 6 2 3(2) 24 2.9 8 68 5 6 3 5(2) 103 4.7 16 72 50 88 4 9(4) 432 12 36 86 90 88 5 17(8) 1830 44 84 86 100 88 6 33(16) 7942 212 196 86 100 88 Table 2: Black-box Srucural Tess Saisics % Coverage Time Mem Tess del fixd fixi 72 5 11 (53) 86 100 88 Table 3: Whie-box Tess Afer 1 minue all he rees required o achieve opimal coverage of he code were generaed size 5. Noe ha he rees of size N creaed by analyzing all sequences of pu and remove operaions (Secion 5.1) are a subse of he rees of size N allowed by repok we believe his is due o he class invarian (repok) being more permissive han he propery mainained by sequences of pu/remove operaions. Indeed (say for performance concerns) mehods may mainain properies ha are sronger han he saed class invarian, hereby disallowing cerain srucures ha are oherwise valid from being generaed during execuions of mehod sequences. 5.3 Whie-box The resuls from doing a whie-box analysis of he mehods of TreeMap o obain a se of es inpus o achieve opimal branch-coverage of he hree mehods is given in Table 3. We repor on he ime aken in seconds, memory usage in Mb, he number of ess run (wih he number of ess generaed before removing duplicaes in parenhesis) and he coverage obained. Alhough we only menion he inpu rees we are considering, each es inpu consiss of an inpu ree as well as he node o pu/remove. Noe ha here we don parameerize he resuls wih he size of he rees (as in Table 2), since he goal is o cover all branches and ha is achieved wih differen size rees. We do however limi he size of he rees ha we are looking for o size 5 and smaller. As o be expeced he coverage obained is opimal. There are many duplicaes amongs he inpu rees generaed o cover all he 53 branches in he code only 11 unique inpu rees are required. The 11 rees are made up of all rees of size 1, 2, 3 and 4, bu only woreesofsize5(ouofhe8possiblerees). 5.4 Discussion The lazy iniializaion of repok ha we use for black-box es inpu generaion can be compared o he approach aken by Kora [10]. Kora generaes inpus from consrains given as Java predicaes and i uses backracking (as we also do) bu i moniors execuions of repok on fully iniialized inpus wihin an a priori given inpu size. For he example used here he number of candidae srucures he wo echniques consider is very similar. However, if one also inroduces ineger values in he srucures, our approach considers a lo fewer srucures, since i uses symbolic execuion ogeher 104

wih ineger consrain solving whereas Kora has o enumerae he ineger fields. The fac ha in he whie-box approach only 11 es inpus are required versus he 84 o obain he same (opimal) coverage in he black-box approach illusraes he power of using a goal-direced whie-box approach over a blind blackbox approach o es inpu generaion for obaining high coverage. We believe ha for more complicaed srucural invarians he difference in es inpu size for he black and whie box approach would be even more pronounced. A drawback of our curren approach is ha we canno deermine wheher code is unreachable when he code conains cycles as is he case in he red-black ree examples considered here. We are considering echniques such as auomaic invarian generaion [38] and he use of shape-predicaes and absracion o address his problem. An inheren drawback of symbolic execuion is he srengh of he decision procedures used o check for infeasible pah condiions. Currenly we can only deal wih linear ineger consrains, bu we hope o add more powerful decision procedures in he near fuure. We should noe ha we have experimened wih differen approaches o represening daa srucures, e.g. hey could be compleely symbolic or represened as parially iniialized Java srucures, as in he conex of he work presened here. We used his laer approach because i faciliaes he evaluaion of precondiions wrien as Java predicaes. 6. RELATED WORK 6.1 Specificaion-based Tesing The idea of using consrains o represen inpus daes back a leas hree decades [14, 29, 31, 41]; he idea has been implemened in various ools including EFFIGY [31], TEGTGEN [32], and INKA [24]. Bu mos of he prior work has been o solve consrains on primiive daa, such as inegers and booleans, and no o solve consrains on complex srucures, which requires very differen consrain solving echniques. Some recen frameworks, mos noably TesEra [35] and Kora [10,34], do suppor non-isomorphic generaion of complex srucures, such as red-black rees. TesEra generaes inpus from consrains given in Alloy, a firs-order declaraive language based on relaions. TesEra uses off-he-shelf SAT solvers o solve consrains. We have already discussed abou Kora in Secion 5.4. The Kora algorihm has recenly been included in he AsmL Tes Generaor [20] o enable generaion of srucures. TesEra and Kora focus on solving srucural consrains. They do no direcly solve consrains on primiive daa and insead, sysemaically ry all primiive values wihin given bounds, which may be inefficien. Furher, TesEra and Kora have been used for black-box esing and no in a whie-box seing. An early paper by Goodenough and Gerhar [23] emphasizes he imporance of specificaion-based esing. Various projecs auomae es case generaion from specificaions, such as Z specificaions [18], UML saechars [37], ADL specificaions [12], or AsmL specificaions [25]. These specificaions ypically do no involve srucurally complex inpus and hey do no address objec-oriened programs. Doong and Frankl [19] use algebraic specificaions o generae ess (including oracles) for objec oriened programs. Their ASTOOT ool generaes sequences of inerface evens and checks wheher he resuling objecs are observaionally equivalen (as specified by he algebraic specificaion). Alhough here we were only ineresed in generaing ess covering he inpu srucures (black-box) and code (whiebox), using an algebraic specificaion o creae addiional ess and check he funcional requiremens of he code is a sraigh-forward exension. Garganini and Heimeyer [21] use a model checker o generae ess ha violae known properies of a specificaion given in he SCR noaion. Ammann and Black [3, 4] combine model checking and muaion analysis o generae es cases from a specificaion. Rayadurgam e al. use a srucural coverage based approach o generae es cases from specificaions given in RSML e by using a model checker [26]. Lee e al. [28] define a framework for using emporal logic o specify daa-flow es coverage. 6.2 Saic Analysis The Three-Valued-Logic Analyzer (TVLA) [33, 42] is he firs saic analysis sysem ha can verify preservaion of he lis srucure in programs ha perform lis reversals via desrucive updaes o he inpu lis. TVLA has been used o analyze small programs ha manipulae doubly linked liss and circular liss, as well as some soring programs. Recenly, he TVLA sysem was exended o also deal wih precondiions on shape-graphs [44]. The poiner asserion logic engine (PALE) [36] can verify a large class of daa srucures ha can be represened by a spanning ree backbone, wih possibly addiional poiners. These daa srucures include doubly linked liss, rees wih paren poiners, and hreaded rees. While saic analysis of program properies is a promising approach for ensuring program correcness in he long run, he curren saic analysis echniques can only verify limied program properies. For example, none of he above echniques can verify correcness of implemenaions of balanced rees, such as red-black rees. Tesing, on he oher hand, is very general and can verify any decidable program propery for realisically large implemenaions, bu for bounded inpus. 6.3 Sofware Model Checking There has been a lo of recen ineres in applying model checking o sofware [7,8,15,17,22,27,43]. Mos of his work has focused on checking even sequences, specified in emporal logic or as API usage rules in he form of finie sae machines. These approaches offer srong guaranees: if a program is successfully checked, here is no inpu/execuion ha would lead o an error. However, hey ypically did no consider linked daa srucures or considered hem only o reduce he sae space o be explored and no o check he daa srucures hemselves. Our work shows how o enable an off-he-shelf model checker o check for properies of complex srucures, aking ino accoun complex precondiions. Our algorihms can be implemened in a sraighforward fashion o enable oher model checkers ha suppor dynamic srucures o check srucural properies oo. Recenly wo popular sofware model checkers BLAST and SLAM, boh based on predicae absracion, were used o do whie-box es inpu generaion [2, 6]. In boh cases he goal is o generae ess ha will cover a specific predicae or a combinaion of predicaes. These echniques do 105

no focus on generaing complex es inpus and hey can no handle complex precondiions. 7. CONCLUSION The main conribuion of his work was o show how complex precondiions can be used o allow efficien symbolic execuion of code manipulaing complex daa o generae es inpus obaining high code coverage. In paricular we illusraed how o use a conservaive precondiion ha evaluaes symbolic srucures (i.e. srucures ha are no fully iniialized) o eliminae srucures ha canno lead o valid concree inpus ha will achieve he saed coverage. This conservaive precondiion can be seen as an absrac version of he concree precondiion, since i will disallow invalid srucures, bu migh accep srucures ha can be insaniaed o concree srucures ha will fail he concree precondiion. Alhough we creaed he conservaive precondiion by hand, we would like o invesigae how o generae i direcly from he concree precondiion by using echniques from absrac inerpreaion. In our experimens he precondiion was he class invarian, bu he approach can handle any precondiion expressed as a Java predicae. We also showed wo oher approaches o using model checking and symbolic execuion for esing. Firsly, he mos radiional approach from a model checking perspecive, where one simply applies model checking o he sysem under es. This approach can obain high levels of behavioral coverage, bu only for small configuraions of daa srucures. Secondly, we showed ha by symbolically execuing he code for he precondiion one can efficienly obain ess suiable for black-box esing. This second approach shows he flexibiliy of our lazy iniializaion approach o symbolic execuion, and i resembles he algorihm employed by he Kora ool [10] ha has been highly successful in generaing es inpus. We believe ha a flexible approach o esing complex sofware is very imporan. To his end we hink he echniques covered here can be seen as a coninuum in he following fashion. If he code o be analyzed doesn have a full specificaion, one can use he black-box approach ha only considers he srucure of he inpus o generae ess. Noe ha he srucure of he inpus mus be known and in our case given as a Java predicae. If a specificaion does exis (for example an algebraic specificaion as used in [19]) hen a specificaion cenered approach o es inpu generaion can be used o augmen he above. Noe, his requires he specificaion o be given in a noaion accepable o a model checker, again Java in our case. A his poin he black-box approach has been exhaused and one needs o consider he code (whie-box) o generae addiional ess. Alhough we only considered simple coverage crieria here o drive he symbolic execuion based es inpu generaion, one can specify any crieria ha can be expressed as properies o he model checker, for example, predicae coverage [2,6]. Lasly, esing is no sufficien in all cases, for example esing a concurren program is nooriously incomplee, and a more powerful echnique such as program model checking can hen be used. Acknowledgmens We would like o hank Darko Marinov for insighful discussions and for sharing wih us his Java implemenaion of he class invarian for red-black rees. We would also like o hank he reviewers for heir deailed commens ha allowed us o grealy improve he paper. 8. REFERENCES [1] The economic impacs of inadequae infrasrucure for sofware esing. Naional Insiue of Sandards and Technology, Planning repor 02-3, May 2002. [2] R. J. Adam J. Chlipala, Thomas A. Henzinger and R. Majumdar. Generaing ess from counerexamples. In Proceedings of he 26h Inernaional Conference on Sofware Engineering (ICSE), Edinburgh, Scoland, May 2004. [3] P.AmmannandP.Black.Aspecificaion-based coverage meric o evaluae es ses. In Proceedings of he 4h IEEE Inernaional Symposium on High Assurance Sysems and Engineering, 1999. [4] P. Ammann, P. E. Black, and W. Majurski. Using model checking o generae ess from specificaions. In Proceedings of he 2nd IEEE Inernaional Conference on Formal Engineering Mehods, 1998. [5] C. Arho, D. Drusinsky, A. Goldberg, K. Havelund, M. Lowry, C. Pasareanu, G. Rosu, and W. Visser. Experimens wih es case generaion and runime analysis. In Proceedings of he 10h Inernaional Workshop on Absrac Sae Machines, Taormina, Ialy, March 2003. [6] T. Ball. Absracion-guided es generaion: A case sudy, 2003. Microsof Research Technical Repor MSR-TR-2003-86. [7] T. Ball and S. K. Rajamani. Auomaically validaing emporal safey properies of inerfaces. In Proc. 8h Inernaional SPIN Workshop on Model Checking of Sofware, pages 103 122, 2001. [8] T. Ball and S. K. Rajamani. The SLAM projec: Debugging sysem sofware via saic analysis. In Proc. 29h Annual ACM Symposium on he Principles of Programming Languages (POPL), pages 1 3, 2002. [9] B. Beizer. Sofware Tesing Techniques. Inernaional Thomson Compuer Press, 1990. [10] C. Boyapai, S. Khurshid, and D. Marinov. Kora: Auomaed esing based on Java predicaes. In Proc. Inernaional Symposium on Sofware Tesing and Analysis (ISSTA), pages 123 133, July 2002. [11] G. Bra, D. Giannakopoulou, A. Goldberg, K. Havelund, M. Lowry, C. Pasareanu, A. Vene, W. Visser, and R. Washingon. Experimenal evaluaion of verificaion and validaion ools on marian rover sofware. In Proceedings of he SEI/CM Sofware Model Checking Workshop, Pisburgh, March 2003. To Appear in Formal Mehods in Sysem Design Journal. [12] J. Chang and D. J. Richardson. Srucural specificaion-based esing: Auomaed suppor and experimenal evaluaion. In Proc. 7h ACM SIGSOFT Symposium on he Foundaions of Sofware Engineering (FSE), pages 285 302, Sep. 1999. [13] E. M. Clarke, O. Grumberg, and D. A. Peled. Model Checking. The MIT Press, Cambridge, MA, 1999. [14] L. A. Clarke. A sysem o generae es daa and symbolically execue programs. IEEE Transacions on Sofware Engineering, Sep. 1976. 106

[15] J. Corbe, M. Dwyer, J. Hacliff, C. Pasareanu, Robby, S. Laubach, and H. Zheng. Bandera: Exracing finie-sae models from Java source code. In Proc. 22nd Inernaional Conference on Sofware Engineering (ICSE), June 2000. [16] T. H. Cormen, C. E. Leiserson, and R. L. Rives. Inroducion o Algorihms. The MIT Press, Cambridge, MA, 1990. [17] C. Demarini, R. Iosif, and R. Siso. A deadlock deecion ool for concurren Java programs. Sofware - Pracice and Experience, July 1999. [18] M. R. Dona. Auomaing formal specificaion based esing. In Proc. Conference on Theory and Pracice of Sofware Developmen, volume 1214, pages 833 847, Lille, France, 1997. [19] R.-K. Doong and P. G. Frankl. The asoo approach o esing objec-oriened programs. ACM Transacions on Sofware Engineering and Mehodology (TOSEM), 3(2):101 130, 1994. [20] Foundaions of Sofware Engineering, Microsof Research. The AsmL es generaor ool. hp://research.microsof.com/fse/asml/doc/ AsmLTeser.hml. [21] A. Garganini and C. Heimeyer. Using model checking o generae ess from requiremens specificaions. In Proceedings of he 7h European engineering conference held joinly wih he 7h ACM SIGSOFT Inernaional Symposium on Foundaions of Sofware Engineering (FSE). Springer-Verlag, 1999. [22] P. Godefroid. Model checking for programming languages using VeriSof. In Proceedings of he 24h Annual ACM Symposium on he Principles of Programming Languages (POPL), pages 174 186, Paris, France, Jan. 1997. [23] J. Goodenough and S. Gerhar. Toward a heory of es daa selecion. IEEE Transacions on Sofware Engineering, June 1975. [24] A. Golieb, B. Boella, and M. Rueher. Auomaic es daa generaion using consrain solving echniques. In Proc. Inernaional Symposium on Sofware Tesing and Analysis (ISSTA), Clearwaer Beach, FL, 1998. [25] W. Grieskamp, Y. Gurevich, W. Schule, and M. Veanes. Generaing finie sae machines from absrac sae machines. In Proc. Inernaional Symposium on Sofware Tesing and Analysis (ISSTA), pages 112 122, July 2002. [26] M.P.E.Heimdahl,S.Rayadurgam,W.Visser, D. George, and J. Gao. Auo-generaing es sequences using model checkers: A case sudy. In Proc. 3rd Inernaional Workshop on Formal Approaches o Tesing of Sofware (FATES), Monreal, Canada, Oc. 2003. [27] T. A. Henzinger, R. Jhala, R. Majumdar, and G. Sure. Sofware verificaion wih blas. In Proceedings of he Tenh Inernaional SPIN Workshop on Model Checking of Sofware, volume 2648 of LNCS, 2003. [28] H. S. Hong, I. Lee, O. Sokolsky, and H. Ural. A emporal logic based heory of es coverage and generaion. In Proc. 8h Inernaional Conference on Tools and Algorihms for Consrucion and Analysis of Sysems (TACAS), Grenoble, France, April 2002. [29] J. C. Huang. An approach o program esing. ACM Compuing Surveys, 7(3), 1975. [30] S. Khurshid, C. Pasareanu, and W. Visser. Generalized symbolic execuion for model checking and esing. In Proc. 9h Inernaional Conference on Tools and Algorihms for Consrucion and Analysis of Sysems (TACAS), Warsaw, Poland, April 2003. [31] J. C. King. Symbolic execuion and program esing. Commun. ACM, 19(7):385 394, 1976. [32] B. Korel. Auomaed es daa generaion for programs wih procedures. San Diego, CA, 1996. [33] T. Lev-Ami and M. Sagiv. TVLA: A sysem for implemening saic analyses. In Proc. Saic Analysis Symposium, Sana Barbara, CA, June 2000. [34] D. Marinov. Tesing Using a Solver for Imperaive Consrains. PhD hesis, Compuer Science and Arificial Inelligence Laboraory, Massachuses Insiue of Technology, 2004. (o appear). [35] D. Marinov and S. Khurshid. TesEra: A novel framework for auomaed esing of Java programs. In Proc. 16h IEEE Inernaional Conference on Auomaed Sofware Engineering (ASE), San Diego, CA, Nov. 2001. [36] A. Moeller and M. I. Schwarzbach. The poiner asserion logic engine. In Proc. SIGPLAN Conference on Programming Languages Design and Implemenaion, Snowbird, UT, June 2001. [37] J. Offu and A. Abdurazik. Generaing ess from UML specificaions. In Proc. Second Inernaional Conference on he Unified Modeling Language, Oc. 1999. [38] C. Pasareanu and W. Visser. Verificaion of java programs using symbolic execuion and invarian generaion. In Proceedings of he 11h Inernaional SPIN Workshop on Model Checking of Sofware, volume 2989 of LNCS. Springer-Verlag, 2004. [39] J. Penix, W. Visser, E. Engsrom, A. Larson, and N. Weininger. Verificaion of Time Pariioning in he DEOS Scheduler Kernel. In Proceedings of he 22nd Inernaional Conference on Sofware Engineering (ICSE), Limeric, Ireland., June 2000. ACM Press. [40] W. Pugh. The Omega es: A fas and pracical ineger programming algorihm for dependence analysis. Communicaions of he ACM, 31(8), Aug. 1992. [41] C. V. Ramamoorhy, S.-B. F. Ho, and W. T. Chen. On he auomaed generaion of program es daa. IEEE Transacions on Sofware Engineering, 2(4), 1976. [42] M. Sagiv, T. Reps, and R. Wilhelm. Solving shape-analysis problems in languages wih desrucive updaing. ACM Trans. Prog. Lang. Sys., Jan. 1998. [43] W. Visser, K. Havelund, G. Bra, S.-J. Park, and F. Lerda. Model Checking Programs. Auomaed Sofware Engineering Journal, 10(2), April 2003. [44] G. Yorsh, T. W. Reps, and S. Sagiv. Symbolically compuing mos-precise absrac operaions for shape analysis. In Proceedings of he 10h Inernaional Conference on Tools and Algorihms for Consrucion and Analysis of Sysems (TACAS), Barcelona, Spain, April 2004. 107