Evaluatio of Differet Fitess Fuctios for the Evolutioary Testig of a Autoomous Parkig System Joachim Wegeer 1, Oliver Bühler 2 1 DaimlerChrysler AG, Research ad Techology, Alt-Moabit 96 a, D-1559 Berli, Germay Joachim.Wegeer@DaimlerChrysler.com 2 STZ Softwaretechik, Im Gaugemaier 2, D-7373 Esslige-Zell, Germay Oliver.Buehler@stz-softwaretechik.de Abstract. The method of evolutioary fuctioal testig allows for the automatio of testig by trasformig the test case desig ito a optimizatio problem. For this aim it is ecessary to defie a suitable fitess fuctio. I this paper two differet fitess fuctios are compared for the testig of a autoomous parkig system. The autoomous parkig system is executed with the geerated test scearios, the fitess for each test sceario is calculated o basis of a evaluatio of the quality of the parkig maeuver calculated by the autoomous parkig system. A umerical aalysis shows, that the proposed area criterio supports a faster covergece of the optimizatio compared to the proposed distace criterio ad that the proposed area criterio describes a efficiet method to fid fuctioal errors i the system i a automated way. 1 Itroductio Electroic cotrol uits (ECUs) i cars take over more ad more complex tasks. New applicatios such as autoomous parkig systems, itelliget cruise cotrol systems trackig the distace to vehicles drivig ahead or emergecy breakig systems rely o the sesor-based calculatio of distaces to other objects. For such applicatios errors i the ECU's software ca result i high costs. Therefore, the aim is to fid as may errors as possible by testig the systems before they are released. I practice, dyamic testig is the aalytical quality assurace method most commoly used. Usually, a complete test is ifeasible, because of the huge umber of possible iput situatios. Therefore, test cases have to be selected accordig to test hypotheses, e.g. each requiremet should be tested at least oce or every program brach should be executed durig the test. I most cases, test case desig is performed maually, requirig a cosiderable part of the project s resources. The method of evolutioary fuctioal testig facilitates the geeratio of test cases i order to detect fuctioal errors durig a directed search. The method of evolutioary fuctioal testig trasforms the test case desig process ito a optimizatio problem. Automated test result evaluatio is a prerequisite for this process. The evaluatio is doe by meas of the fitess fuctio, which assigs a umerical quality value to a test result.
This paper evaluates two differet approaches to the defiitio of fitess fuctios for the fuctioal testig of a autoomous parkig system. The fitess fuctios defied represet a quality metric ad ca automatically evaluate a parkig maeuver calculated by the parkig system, i.e. they retur a umerical value which describes the quality of the parkig maeuver. Both approaches are compared by umerical experimets for a prototype implemetatio of a autoomous parkig system. The results show that of both the criteria proposed i this paper, the area criterio ca idetify critical parkig maeuvers better tha the distace criterio itroduced i [1]. The area criterio provides a more efficiet method of error detectio i the parkig system. The structure of the paper is as follows: After a brief itroductio to evolutioary testig i the secod sectio, the autoomous parkig system is itroduced ad the applicatio of evolutioary testig to its fuctioal testig is explaied i the third sectio. Sectio 4 describes the differet fitess fuctios, whereas sectio 5 shows the experimetal results achieved by applyig these fitess fuctios. The paper closes with a short summary i the sixth sectio. 2 Evolutioary Testig Testig is aimed at fidig errors i the system uder test ad givig cofidece i its correct behavior by executig the system with selected iput situatios. Of all the test activities, test case desig is assiged decisive importace. Test case desig determies the type ad scope ad thus the quality of the test [2]. If relevat test cases are forgotte, the probability to detect errors i the system siks. Due to the cetral importace of test case desig, a umber of testig methods have bee developed to help the tester with the selectio of appropriate test data. Oe importat weakess of the testig methods available is that they caot be automated straightforwardly. Maual test case desig, however, is time-itesive ad error-proe. The test quality depeds o the performace of the tester. I order to icrease the effectiveess ad efficiecy of the test ad thus to reduce the overall developmet ad maiteace costs for systems, a test should be systematic ad extesively automatable. Both objectives are addressed by the method of evolutioary testig [3]. I order to trasform a test aim ito a optimizatio task a umeric represetatio of the test aim is ecessary, from which a suitable fitess fuctio for the evaluatio of the geerated test data ca be derived. Depedig o which test aim is pursued, differet fitess fuctios emerge for test data evaluatio. If, for example, the temporal behavior of a applicatio will be tested, the fitess evaluatio of the idividuals of evolutioary testig will be based o the executio times measured for the test data [3]. For safety tests, the fitess values are derived from pre- ad postcoditios of modules [4], ad for robustess tests of fault-tolerace mechaisms, the umber of cotrolled errors ca form the startig poit for the fitess evaluatio [5]. Applicatios of evolutioary testig to structural testig result i differet fitess fuctios agai [6], [7], [8], [9]. A overview o the differet applicatios of evolutioary testig ca be foud i [1].
If a appropriate fitess fuctio ca be defied, the evolutioary test proceeds as follows. The iitial populatio of test data is usually geerated at radom. Each idividual withi the populatio represets a test datum with which the system uder test is executed. For each test datum the executio is moitored ad the fitess value is determied with respect to the defied test aim. Next, populatio members are selected with regard to their fitess ad subjected to combiatio ad mutatio processes to geerate ew offsprig. These are the also evaluated by executig the system uder test with the correspodig test data. A ew populatio is formed by combiig offsprig ad paret idividuals, accordig to the survival procedures laid dow. From here o, the process repeats itself, startig with selectio, util the test objective is fulfilled or aother give stoppig coditio is reached. 3 Evolutioary Testig of the Autoomous Parkig System As a automobile maufacturer, DaimlerChrysler is cotiuously developig ew systems i order to improve vehicle safety, quality, ad comfort. Withi this cotext, prototypical vehicle systems are developed, which support autoomous vehicle parkig - a fuctio that is likely to be itroduced to the market i some years time. To describe the applicatio of evolutioary testig for the fuctioal testig of autoomous parkig systems a brief descriptio of the fuctioality of a autoomous parkig system is give first. The, the applicatio of the evolutioary test to the parkig system is explaied. I additio, the test eviromet developed for the test of the parkig systems ad the test data geerator are described. 3.1 Autoomous Parkig System The autoomous parkig systems dealt with i this paper are iteded to automate parkig legthways ito a parkig space, as show i Fig.1. For this purpose, the vehicle is equipped with evirometal sesors, which register objects surroudig the vehicle. O passig alog, the system ca recogize sufficietly large parkig spaces ad ca sigal to the driver that a parkig space has bee foud. If the driver decides to park i the parkig space detected, the vehicle ca do this automatically. Fig.1: Fuctioality of a Autoomous Parkig System Fig.2 shows the system eviromet ad the iteral structure of the autoomous parkig system. The iputs are sesor data, cotaiig iformatio o the state of the vehicle, e.g. vehicle speed or steerig positio, ad iformatio from the
evirometal sesors, which register objects o the left ad right had side of the vehicle. As for output the system possesses a iterface to the vehicle actors, where the vehicle's velocity ad steerig agle will be set. The parkig space detectio processes the data obtaied from the evirometal sesor systems ad delivers the recogized geometry of a parkig space if it has bee idetified as sufficietly large. The parkig cotroller compoet uses the geometry data of the parkig space together with the data from the vehicle sesors to steer the vehicle through the parkig procedure. For this purpose, velocity ad steerig agle are set for the vehicle actors. The parkig cotroller has to guaratee that the collisio area is ot etered by the cotrolled vehicle i order to avoid a high probability of causig damage to adjacet parkig vehicles or objects ad the cotrolled vehicle itself. Autoomous Parkig Cotroller vehicle sesors evirometal sesors vehicle data evirometal data Parkig Space Detectio parkig space geometry Parkig Cotroller actor data vehicle actors Fig.2: System Eviromet ad Sub-Compoets of the Autoomous Parkig System 3.2 Applyig Evolutioary Testig to the Autoomous Parkig System Comprehesive ad efficiet testig is essetial before releasig a system such as the automatic parkig system. As may tests as possible must be performed i a systematic ad efficiet way. Maual testig of the complete system is cost-itesive ad time-cosumig because every test case comprises buildig up a parkig sceario with real cars ad maual drivig of each maeuver. Furthermore, it is difficult to achieve a exact reproductio of the tests because the details of the test executio vary. I cotrast, automated tests have the potetial to perform a large umber of test cases with less effort i a reproducible maer. Therefore, automated fuctioal tests performed i a cotrolled simulatio eviromet should be itegrated ito the quality assurace process of the autoomous parkig system. Evolutioary fuctioal testig provides a way of automatig fuctioal tests as a complete process. Istead of selectig the test cases maually, a search for iterestig test cases is performed automatically. This is doe by traslatig the test case selectio ito a optimizatio problem. The possible iput situatios of the system uder test are mapped to the search space. O the oe had, the mappig should keep the size of the search space as small as possible, ad o the other had, the mappig should be able to produce all possible iput data for the system. If oe cosiders the whole iput rage durig the desig of the test data geerator it does ot mea that all
test cases i this rage will actually be tested, but it provides the possibility to geerate ay required test data. A appropriate model has to be desiged for this purpose. Both compoets of the autoomous parkig system have to be tested thoroughly. For the parkig space detectio, it has to be tested that suitable parkig spaces are idetified precisely whereas parkig spaces to small for a parkig maeuver have to be rejected. Usually, the parkig space detectio has to be tested i atural eviromets sice reliable simulatio models for the evirometal sesors are ot available. Therefore, it is ot cosidered i the subsequet sectios of this paper. Nevertheless, we pla to test the parkig space detectio by geeratig evirometal data i the future. For the testig of the parkig cotroller test cases describig differet parkig scearios are to be geerated. The evaluatio of the test cases is carried out by the fitess fuctio. For the test of the automatic parkig system, the fitess fuctio calculates a umerical fitess value for each parkig sceario geerated. This fitess value represets the quality of the correspodig test case ad iteds to lead the evolutioary search ito a directio of iput situatios which result i a error of the parkig cotroller. Therefore, the fitess fuctio is desiged to assig good fitess values to parkig scearios which lead the system to eter the collisio area or ed up i a iadequate parkig situatio. Bad fitess values are assiged to scearios which result i a good parkig positio with eough distace to the collisio area. 3.3 Test Eviromet The test eviromet of the automatic parkig system (Fig. 3) comprises the simulatio eviromet, a evolutioary algorithm toolbox, a implemetatio of the fitess fuctio ad the test data geerator which traslates idividuals ito actual parkig scearios. The test object is the cotrol uit of the vehicle with the implemetatio of the automated parkig system iside. fitess Fitess Fuctio parkig maeuver Evolutioary Algorithm Toolbox parkig space geometry idividuals Simulatio Eviromet sesor data actor data Cotrol Uit Test Data Geerator Fig.3: Desig of the Test Eviromet The GEA toolbox for Matlab (www.geatbx.com) was used to implemet the evolutioary algorithms used for the test of the autoomous parkig system. The
simulatio eviromet (built up o a Matlab R12.1 platform) simulates the properties of the vehicle as well as the surroudig eviromet. It rus with the cotrol uit "i-the-loop", meaig that the simulatio eviromet calculates the sesor data of the vehicle ad presets it to the parkig cotroller iside the cotrol uit. The cotrol uit processes this sesor data ad reacts o it with cotrol data for the simulatio eviromet. This loop simulates a complete parkig maeuver resultig for the geerated parkig space sceario. The parameters describig a parkig space sceario for the simulatio, such as the positio of the car ad the size of the parkig space, are outputs of the test data geerator (see 3.4.). After the simulatio of a parkig maeuver the fitess value is calculated usig the fitess fuctios described i the subsequet sectio ad the assiged to the geerated idividual. 3.4 Test Data Geerator The geometric data to characterize a parkig space comprises six poits P to, ad is referred to as parkig space geometry. The poits defie the border betwee the drivable ad impassable area of the parkig situatio. The model for the geeratio of this parkig space geometry is show i Fig.4. It is a simplified model, because the borders of the parkig space are always rectagular. The shape of the parkig space ca oly vary i legth ad depth. distace to space P P1 P2 space space psi gap collisio area Fig.4: Model for Geeratio of Parkig Space Geometry This model takes the values of five idepedet variables ad calculates from it the parkig space geometry. The idepedet variables defie legth ad width of the parkig space. I additio, the startig positio with the distace of the car to the parkig space (dist2space), the agle psi ad the gap betwee the vehicle ad the collisio area o the right side of the car are part of the parkig space scearios geerated. 4 Defiitio of Fitess Fuctios This sectio describes the defiitio of two differet strategies for the evaluatio of the fitess of a parkig sceario. Oe strategy uses the distace betwee vehicle ad
collisio area as a measure of the evaluatio of fitess [1], the other strategy works with the area betwee vehicle ad collisio area. Both strategies separate the parkig space ito two parts: (1) collisio with the precedet vehicle ad (2) collisio at the parkig side. The possible movemet of the vehicle ito the parkig space has limited degrees of freedom ad thus, if a collisio with the precedig vehicle or the parkig side happes, either the right rear edge or the right frot edge of the car must be ivolved. As a result from the positio of the collisio areas, the lies betwee - ad - have to be observed i order to idetify a collisio with the frot vehicle. To idetify a collisio with the side, the lie P2- has to be observed. The defiitio distiguishes betwee the observatio of a corer, defied by three poits, ad the observatio of a edge, defied by two poits. The fitess fuctio is iteded to assess a parkig sceario ad to assig a adequate fitess value to it. The fitess value should correspod to the quality of the parkig sceario. From the testig perspective it is a good parkig sceario if a collisio with other parkig cars occurs or the cotrolled vehicle touches the parkig side. Sice we are miimizig the fitess values durig the search process, the fitess value assiged to a parkig sceario decreases with the quality of the sceario. A iterestig parkig sceario achieves a smaller value tha a parkig sceario for which the automatic parkig system performs well. The fitess becomes egative whe a collisio occurs. 4.1 Distace Criterio Fitess Fuctio The distace criterio cosiders the closest distace betwee a vehicle edge ad the collisio border durig the parkig maeuver. I separate evaluatios the smallest distace of the collisio corer -- ad the smallest distace from the collisio side P2- are calculated. The followig subsectios describe, how the evaluatio of the collisio corer ad collisio side is carried out. Evaluatio of a Collisio Corer The collisio corer is defied by three poits --. All distaces will be calculated as polar coordiates with as origi (Fig. 5). The evaluatio of the collisio corer observes the sectio defied by -- ad the diagoal opposite sectio. Oly the poits withi these sectios are cosidered. As quality measure for the evaluatio of the parkig maeuver, the smallest distace betwee the vehicle path positios ad the poit is take. The value of the distace is positive, if the path is outside the collisio corer. The value is set to zero, if the path crosses. The value is measured as a egative value, if the path rus through the collisio corer (Fig. 6).
Fig.5: Selectio of Smallest Distace This strategy shall esure, that closer the path gets to the collisio corer, the lower the assiged fitess value becomes. For differet paths which cotiuously move ito the collisio corer the correspodig fitess values become cotiuously lower, as show i Fig.7, where D1 > D2 > D3. D1> D2= D3< Fig.6: Siged Distace Values Evaluatio of a Collisio Side The collisio side is defied by the straight lie betwee P2-. The distaces calculated are betwee this lie ad the path positios of the car. I this calculatio, oly path poits whose x-values are withi the rage of P2 ad are take ito accout. The selectio is doe by comparig the x-coordiates of the path poits with P2 ad. D< P2 D> P2 D= P2 Fig.7: Distace from Lie with Positive, Zero ad Negative Value The distace is calculated positive, whe the path is above the lie, ad it is set to zero, whe the path touches the lie. The distace is calculated as egative value, whe the path rus through the collisio area. From all calculated distace values, the miimum is take as fitess value for the geerated parkig space sceario.
4.2 Area Criterio Fitess Fuctio The area criterio fitess fuctio cosiders the area icluded betwee the path of the vehicle ad the parkig geometry. Here, the areas icluded i the collisio corer ad the collisio side are calculated i separate evaluatios. The followig subsectios describe how the evaluatio of the collisio corer ad the collisio side with the area criterio is carried out. Evaluatio of a Collisio Corer The area icluded betwee the corer lies ad the vehicle path is take as a measure of the evaluatio of the parkig sceario. To calculate this area, it is separated ito smaller segmets, appropriate to the poits of the vehicle s path through the corer, as show i Fig.8. The overall area A icluded i the collisio corer is the sum of all segmets together. P P A P A P P P Fig.8: Separatio of Icluded Area ito Small Segmets For a effective ad fast calculatio of the fitess value, a approximate descriptio of the area of each segmet ca be provided by a triagle. Whe the distaces betwee two poits of the path T ad T +1 is sigificatly smaller tha the distace to the agle alpha is very small. T +1 T +1 alpha alpha d T r T Fig.9: Approximatio of Segmet by a Triagle For small agles of alpha, the ratio d to r is approximately the agle i radiats. The area of oe segmet which is approximately described by a triagle is: 1 A = r d 2 ad with d r ( th th ) +1 (1) the area of a segmet ca be calculated thus: A 1 2 2 r +1 ( th th ). (2)
The agle th ad the radius r for each path poit T ca be easily obtaied, whe the path positios are trasferred to Cartesia coordiates ito polar coordiates with as origi of the coordiate system. The overall area of the corer is the sum of all segmets withi the corer A. (3) corer = A To lead the optimizatio towards a collisio, the corer -- symmetrical to poit is also take ito cosideratio. The path of the vehicle has to pass the poit ad the aim of the optimizatio is to brig that path ito the collisio area. The idea is to rate a area icluded i the opposite corer as a positive value ad a area icluded i the collisio corer as a egative value. Whe the vehicle path crosses through the corer poit the correspodig value is set to zero. Thereby, the areas show i Fig.1 become A1 > A2 > A3 whe the vehicle path keeps o shiftig ito the collisio corer. This leads to a gradual improvemet of the fitess value, depedig o how ear the vehicle path passes by the collisio corer or crosses ito it. A1> A2= A3< Fig.1: Siged Area Values Evaluatio of a Collisio Side The evaluatio of a collisio side takes the area icluded betwee the vehicle path ad the straight lie P2- as a measure. The calculatio of the area takes oly those poits ito cosideratio which have their x-coordiate i the rage betwee P2 ad. The calculatio is doe by approximatio with the rectagles defied through the path positios. The area of each rectagle ca be easily calculated by x ad the distace betwee path ad lie y. With a sufficiet umber of path poits i the rage betwee P2 ad, a good approximatio of the icluded area ca be achieved. A< P2 A> P2 A= P2 Fig.11: Area with Positive, Zero ad Negative Value Fig.11. (1) whe the path is above the lie, (2) the path touches the lie or (3) the path crosses the straight lie ito the collisio area. I the first case, the icluded area has a positive value, i the secod case the value for the area is set to zero. I the
third case the area below the lie has a egative value. The distictio draw betwee the three cases prevets a larger area above the lie compesatig small areas below the lie. It also prevets a touch of the lie from beig cocealed by adjacet areas so that they caot be observed. 5 Experimets The aim of the experimets is to aalyze ad compare the suitability of the fitess fuctios described i the previous sectio for the testig of autoomous parkig systems. A umber of experimets were performed to aalyze the fitess ladscapes for both fitess fuctios. For each experimet, two variables from the test data geeratio iput vector were varied withi a defied rage ad a defied umber of samples. The remaiig three variables were kept costat resultig i twodimesioal variatio plots of the fitess ladscapes. To calculate the fitess value for each parkig sceario, a parkig maeuver of the autoomous parkig system was simulated as described i sectio 3.2. Fig. 12 shows the results for the experimet i which the variables dist2space ad psi were varied, ad i which the legth ad width of the parkig space as well as the gap to the right of the vehicle are kept costat: legth was set to 8. m, width to 2.5 m ad the gap to.7 m. The axis to the right shows the distace to the parkig space i the rage of. m to 7. m, with a resolutio of 7 poits. The axis to the depth shows the agle psi from +1 o to 1 o, with a resolutio of 4 poits. Both fuctios retur egative fitess values whe agle psi reaches +1 o ad dist2space goes towards 7 m. A cosiderable differece betwee the shapes of both surfaces is that for fitess values greater tha zero the distace criterio returs costat small values where the area criterio desceds its values towards the border to zero. This appears as a flat plateau i the distace criterio surface (o the right had side of Fig. 12) where the area criterio surface slopes cotiuously (left had side of Fig. 12). 2.5.5 2 1.5.5 1 1 fitess value.5 fitess value 1.5 2 2.5.5 3 1 3.5 1.5 1 4 1 psi [deg] 1 1 2 3 4 5 6 7 psi [deg] 1 1 2 3 4 5 6 7 distace to parkig space [m] distace to parkig space [m] Fig.12: Fitess Ladscapes for Area Criterio (left) ad Distace Criterio (right) as Fuctio of Agle psi ad Distace to Parkig Space
Comparable results were foud i most experimets, e.g. Fig. 13 shows the plots for the variatio of dist2space ad parkig space legth. The rage of the legth, show o the axis to the right, is betwee 5 ad 1 m. Dist2space is show o the axis to the left ad rages from to 7 m. The remaiig variables are kept costat: width of parkig space was set to 2.5 m, the gap to the right of the vehicle was set to.7 m ad the agle psi was set to o. As i the precedig example, the surfaces for the area criterio ad the distace criterio differ. The distace criterio surface has a flat plateau of equal fitess values for low distaces to the parkig space ad loger parkig spaces. I cotrast, the area criterio shows a slopig characteristic for that domai. 6 Coclusio The experimets have show that both fitess fuctios have a slopig characteristic for collisio maeuvers, where their fitess value is less tha zero. The slope of the distace criterio fuctio is steeper i that domai tha that of the area criterio fuctio. Furthermore, the distace criterio fuctio shows a agle at the poit where the values eter the egative rage whereas the area criterio fuctio has a smooth trasitio for fitess values aroud zero. Both fitess fuctios guide the search to scearios where a collisio happes. The area criterio fuctio provides a slopig characteristic for maeuvers with positive fitess values, i cotrast to the distace criterio fuctio, which returs costat fitess values resultig i plateaus of equal fitess i the search space for scearios without a clash. It does ot differetiate betwee good ad better maeuvers. As a cosequece, the area criterio fuctio is better suited to direct the search towards collisio maeuvers, tha the distace criterio fuctio. It supports a faster fidig of test cases for which the autoomous parkig system does ot react correctly. Therefore, the area criterio will be used for the evolutioary testig of autoomous parkig systems i the future. 1.5.5 1.5.5 fitess value.5 1 fitess value 1 1.5 2 1.5 2.5 2 3 2.5 2 4 distace to parkig space [m] 6 8 5 6 9 8 7 legth of parkig space [m] 1 2 4 distace to parkig space [m] 6 8 5 6 9 8 7 legth of parkig space [m] 1 Fig.13: Fitess Ladscapes for Area Criterio (left) ad Distace Criterio (right) as Fuctio of Distace to ad Legth of the Parkig Space
Ackowledgmets The work described has bee performed withi the SysTest project., fuded by the EC uder the 5th framework program (GROWTH, project ref. G1RD-CT-22-683). Refereces 1. Buehler, O., Wegeer, J.: Evolutioary Fuctioal Testig of a Automated Parkig System. Proceedigs of the Iteratioal Coferece o Computer, Commuicatio ad Cotrol Techologies (CCCT '3) ad the 9th. Iteratioal Coferece o Iformatio Systems Aalysis ad Sythesis (ISAS '3), Florida, USA (23). 2. Grochtma, M., Grimm, K.: Classificatio-Trees for Partitio Testig. Software Testig, Verificatio & Reliability, vol. 3, o. 2, pp. 63-82 (1993). 3. Wegeer, J., Grochtma, M.: Verifyig Timig Costraits of Real-Time Systems by Meas of Evolutioary Testig. Real-Time Systems, vol. 15, o. 3, pp. 275-298 (1998). 4. Tracey, N., Clark, J., Mader, K.: The Way Forward for Uifyig Dyamic Test Case Geeratio: The Optimisatio-Based Approach. Proceedigs of the IFIP Iteratioal Workshop o Depedable Computig ad Its Applicatios, South Africa, pp. 169-18 (1998). 5. Schultz, A., Grefestette, J., Jog, K.: Test ad Evaluatio by Geetic Algorithms. IEEE Expert, vol. 8, o. 5, pp. 9-14 (1993). 6. Joes, B., Sthamer, H., Eyres, D.: Automatic Structural Testig Usig Geetic Algorithms. Software Egieerig Joural, vol. 11, o. 5, pp. 299-36 (1996). 7. Pargas, R., Harrold, M., Peck, R.: Test-Data Geeratio Usig Geetic Algorithms. Software Testig, Verificatio & Reliability, vol. 9, o. 4, pp. 263-282 (1999). 8. Michael, C., McGraw, G., Schatz, M.: Geeratig Software Test Data by Evolutio. IEEE Trasactios o Software Egieerig, vol. 27, o. 12, pp. 185-111 (21). 9. Tracey, N., Clark, J., Mader, K., McDermid, J.: A Automated Framework for Structural Test-Data Geeratio. Proceedigs of the 13th IEEE Coferece o Automated Software Egieerig, Hawaii, USA (1998). 1. McMi, P.: Search-based Software Test Data Geeratio: A Survey. To appear i Software Testig, Verificatio & Reliability (24).