Improving Dependability using Shared Supplementary Memory and Opportunistic Micro Rejuvenation in Multi-tasking Embedded Systems

Transcription

1 Improvig Depedability usig Shared Supplemetary Memory ad Opportuistic Micro Rejuveatio i Multi-taskig Embedded Systems Viaitheertha Sudaram * Sadip HomChaudhuri Sachi Garg Chadra Kitala Saurabh Bagchi Purdue Uiversity Motorola Software Group, Motorola Labs School of Electrical ad Computer Egg. Plot#5, 66/, Bagmae Tech Park, 465 Northwester Ave., West Lafayette IN CV Rama Nagar, Bagalore , Idia {vsudar,sbagchi}@purdue.edu, {sadip.homchaudhuri, sgarg, kitala}@motorola.com Abstract We propose a comprehesive solutio to hadle memory-overflow problems i multitaskig embedded systems thereby improvig their reliability ad availability. I particular, we propose two complemetary techiques to address two sigificat causes of memory-overflow problems. The first cause is errors i estimatig appropriate stack ad heap memory requiremet. Our first techique, called Shared Supplemetary Memory (SSM), exploits the fact that the probability of multiple tasks requirig more tha their estimated amout of memory cocurretly is low. Usig aalytical model ad simulatios, we show that reliability ca be cosiderably improved whe SSM is employed. Furthermore, for the same reliability, SSM reduces total memory requiremet by as much as 29.3%. The secod cause is the presece of codig Madelbugs, which ca cause abormal memory requiremet. To address this, we propose a ovel techique, called Opportuistic Micro- Rejuveatio, which whe combied with SSM, provide several advatages: prevetig critical-time outage, resource frugality ad depedability ehacemet. Keywords: stack overflow, heap overflow, embedded systems, software rejuveatio, resource costraied fault-tolerace. Itroductio Embedded systems are becomig icreasigly ubiquitous [2] ad i recet years, their complexity has grow expoetially. Specifically, the growth i requiremets for real-time, etworked ad multitaskig applicatios has led to icreased presece of MadelBugs/HeiseBugs [7] i the code. These are rare bugs which are triggered typically by the chage i the program ru-time eviromet or the (ivalid) iput data or due to ageig of the software ad are thus ot determiistic i ature. It has also made the task of accurately estimatig ru-time memory requiremet (stack ad heap) sigificatly harder. Iaccurate estimatio of memory requiremets ad MadelBugs cause memory overflow or out-of-memory, a grave problem i embedded systems. It leads to uexpected system crash [8] sice most embedded systems, as much as 95% accordig to Middha et al [4], do ot use virtual memory. The effect of a system crash ca be a slight icoveiece (e.g. crash of set-top boxes), loss of reveue (e.g. dropped call o mobile phoes), or eve loss of life (e.g. crash of aircraft cotroller). This has icreased the eed for desigig these systems with fault-tolerace mechaisms to ehace reliability, ot just from safety perspective but also from the userexperiece stadpoit. I this paper, we propose a comprehesive solutio that addresses both importat causes of memory overflow, amely, iaccurate estimatio of a task s memory requiremet, both stack ad heap, over its lifetime ad the presece of MadelBugs i the code. I additio to complex, iter-depedet ad sometime icomplete requiremets specificatio of real-time, etworked ad multi-taskig applicatios, the difficulty i estimatio arises from the memory requiremets depedece o iput data, certai laguage features like fuctio-poiters i C, ad the complexity of the software, which makes it prohibitively expesive to check all paths of program executio. Iaccuracies i estimatio compel the task desigers, as frequetly see i idustry, to deliberately over-estimate the stack ad the heap requiremets as a rudimetary form of robustess i the system. The first techique we propose is a uified approach to solve both stack ad heap overflow usig Shared Supplemetary Memory (SSM). The SSM is formed by combiig the supplemetary memory allocated to all tasks ad makig a global/shared use of the large memory so obtaied. There are several advatages to the uified approach.. It provides improved reliability for the same amout of total memory. I other words, it ca reduce total memory requiremet for the same * Correspodig Author. This work was doe whe the correspodig author was iterig at Motorola Idia Research Labs.

2 reliability. Reductio i memory requiremet directly traslates to cost savigs i ubiquitous embedded systems like mobile phoes. 2. It complemets the hadlig of Madel-Bugs as will be explaied later i Sectio By havig idepedet memory for each task except for the supplemetary part, it retais the advatages of fault-cotaimet. 4. The performace overhead i terms of accessig shared supplemetary memory is miimal as oly supplemetary memory is shared ad ot the estimated memory. The secod techique we propose is opportuistic micro-rejuveatio (OMR) to hadle Madel Bugs. Note that there is a differece betwee MadelBugs ad iaccurate estimatio ad a separate techique is eeded to tackle memory overflow due to MadelBugs. A memory leak or ivalid iput that causes otermiated recursio might use up the etire available memory over time ad hece, has to be hadled differetly from estimatio error. MadelBugs usually occur rarely ad it is hard to reproduce such bugs. Software Rejuveatio [] or rebootig the system at radom istats i time has bee show to be a effective techique to combat MadelBugs [,7] i cotiuously ruig servers (e.g. telecom billig). Micro-reboot/Micro-rejuveatio [3] is a fie-graied rejuveatio techique that reboots the idividual compoets istead of the machie (hardware reboot) ad thus provides icreased availability ad cotiuous service. We propose OMR that is ispired by both techiques described above ad adapted to resourcecostraied eviromets like embedded systems. The key idea is that istead of usig software rejuveatio for predictive maiteace, we rejuveate tasks at opportue istats. Also, we do ot rejuveate the etire system but just the tasks that are most likely to cause a overflow i the ear future. Sice we do fiegraied rejuveatio at opportue istats, we call our techique opportuistic micro-rejuveatio. The rejuveatio is doe whe the SSM usage is greater tha some threshold ad the task with most memory usage, i the SSM, is rejuveated. Sice SSM crossig the threshold is rare, the umber of times we rejuveate is miimal. The mai advatage of this techique is that by rejuveatig a task that is likely to cause a overflow ad its depedets, without rejuveatig other idepedet tasks i the system, reliability ad availability of the system is sigificatly improved. Aother advatage of OMR is its iheret faultcotaimet property. Moreover, i critical embedded systems, critical tasks are usually comprehesively tested as opposed to o-critical tasks. OMR esures that a Madelbug i a o-critical task does t affect the critical task i a embedded system. We have evaluated our solutio usig aalysis ad simulatio. We derived the reliability of three approaches of memory layout for stack overflow assumig ormal distributio for the memory requiremet of a task. Sice a closed form solutio was ot possible i the case of SSM, we resorted to simulatio. We show that our theoretical results ad simulatio results match for the cases for which closed form is possible. We show that SSM approach ca reduce the total amout of memory up to 29.3% for same amout of reliability. We have created a Stochastic Activity Network (SAN) model to aalyse the availability of opportuistic rejuveatio ad show slight (0.02%) improvemet i availability. The rest of the paper is orgaized as follows: Sectio 2 describes the related work; Sectio 3 describes the desig of the two proposed techiques, SSM ad OMR; Sectio 4 discusses the aalysis ad simulatio methodology; Sectio 5 cocludes the paper. 2. Related Work There have bee several approaches to tackle differet aspects of the memory overflow problem i embedded systems [4,, 2]. I the literature, stack ad heap overflow are hadled separately. To hadle the stack overflow problems, Nagauma et.al [2] suggested the possibility of keepig a uused auxiliary memory uit i the system which would serve to hadle the overflowig memory eeds of a faulty task. This auxiliary memory uit, however, comes at a additioal cost ad may ot be suitable for a costsesitive system. Moreover, o performace study was coducted o the proposal. Biswas et.al [] proposed hadlig a stack overflow by relocatig the overflowig memory ito the various dead global variables, uused array locatios ad uused heap of the faulty task; Data compressio was suggested as a viable techique to free up space i these areas. However, i practice, such reclaimed space is ofte too less to fulfil the eeds of a overflowig stack. Their results also idicate severe performace degradatio due to compressio. Middha et.al [4] proposed sharig free stack space from other tasks of the system i the evet of a overflow i oe task ad this techique is referred as Multi-Task Stack Sharig, or MTSS. Evaluatio of MTSS idicates sigificat memory savig i the stack segmet to achieve the same level of reliability. However, it is performace itesive, owig to multiple levels of stack page sharig. All the above schemes were desiged to address the first cause of memory overflow i the system, amely, iaccurate estimatio, ad would ot suffice to hadle the overflows caused by MadelBugs. Furthermore, oe

3 of the above solutios are geeric eough to exted to heap-overflow, as they are depedet o the regularity of the stack growth ad shrik model at each fuctio activatio; quite ulike a heap, where the teure of memory occupacy is u-predictable ad drive by exteral iputs. Accordigly, the mai cotributio i this paper is our solutio (SSM), which hadles stack ad heap overflow together ad the solutio (OMR), which hadles memory overflow due to MadelBugs i the cotext of a embedded system. Software Rejuveatio [] is a well kow faulttolerace techique to icrease the system reliability by proactively restartig a ageig software system. I embedded systems, however, the processig overhead of a health-moitorig module as well as full system restart is ot desirable. Micro-Reboot [3] is preferred with low overhead of compoet based restart. However, the primary premise of [3] was that a compoet restart i such applicatios is orders of magitude faster tha the complete reboot process, eed ecessarily ot hold true i these systems. I [5], the authors defied the cocept of Process Resurrectio as a fast reactive recovery mechaism to prevet a embedded system crash due to a sigle task failure. It was show that the hardware exceptios, viz. segmetatio fault etc, ca be itercepted ad used as a trigger for resurrectig the faulty process. However, they did ot address the memory overflow problem i the stack ad the heap. From the precedig discourse, it is clear that a uified approach to solve stack ad heap overflow has ot bee proposed ad memory overflow due to MadelBugs has ot bee tackled yet. 3. Desig ad Implemetatio I this sectio, we first provide a brief backgroud o memory orgaizatios i embedded systems ad the assumptios we have made. Next, we preset the detailed descriptio of our Shared Supplemetary Memory (SSM) ad Opportuistic Micro-Rejuveatio (OMR) techiques alog with implemetatio issues of these two techiques. 3. Memory Orgaizatio ad Assumptios The typical memory orgaizatio i a embedded system is show i Figure a. The physical memory is divided ito stack ad heap areas ad each of these areas are further subdivided. Each task pool, set of depedet tasks, has its ow stack ad heap allocated respectively i stack ad heap areas. For this paper s discussio, we ca cosider etire task pool as a sigle task without affectig the results. This memory orgaizatio is differet from geeral-purpose T T2 T3 T T2 T3 Heap Estimate Heap Segmet Shared Supplemetary Heap (SSH) obtaied by reducig all heap over-estimatios. Supplemetary Heap Heap Segmet T T2 T3 (a) (b) Stack Estimate Stack Segmet Stack Segmet T T2 T3 Shared Supplemetary Stack (SSS) obtaied by combiig all stack over-estimatios Supplemetary Stack Figure. Memory layout or orgaizatio i embedded systems (a) A typical orgaizatio of memory i embedded systems. (b) Proposed memory orgaizatio with shared supplemetary memory pool Code/Static Data Portio Code/Static Data Portio computers such as desktop/laptop systems, which use hardware-assisted virtual memory to give the illusio that a process ca use as much memory as it eeds. The hardware-assisted virtual memory swaps pages/frames ito secodary memory whe there is o space i the physical memory, thus hadlig memory overflow. Most embedded chips, as much as 95% accordig to Middha et al. [4], do ot provide hardware support for virtual memory as the performace hit every memory access icur i terms of eergy ca be prohibitively expesive [0]. Sice there is o swap device available i embedded systems, if a task s memory requiremet eve exceeds by a byte over its lifetime would cause a memory overflow, which leads to system crash. The tred of ot havig hardware-support for virtual memory i embedded processors will ot chage due to high eergy cost ad potetial for missig real-time guaratees [4]. Therefore, we assume that most embedded processor do ot use virtual memory. Each embedded system task desiger carefully estimates the stack requiremet by adoptig ay of the kow techiques [6]. However, estimatig correct stack size is complicated by several factors icludig recursive fuctios of ubouded legth, laguage features like virtual fuctios ad fuctio poiters, compiler features like allowig stack arrays to be of ru-time depedet size ad iterrupt hadlig. A detailed list of reasos ca be foud i [4]. Hece, the task desigers itetioally over-estimate the stack size i.e., add some supplemetary to the best estimate foud by their aalysis. Compile-time heap estimatio is iheretly difficult as actual heap usage depeds o ru-time iput values. The idustrial approach, therefore, is to ru a series of

4 test cases to evaluate the worst-case heap usage of the applicatios uder various scearios [4, 9]. As this method is ot exhaustive, heap, similar to stack, is deliberately over-estimated ad the supplemetary heap added to the best heap estimate is very high. Sice heap segmet is usually much larger tha stack segmet for ay task, eve a slight over-estimatio for each task ca result i large amout of memory beig used as supplemetary memory for all tasks. Therefore, we assume that accurate measuremet of stack ad heap requiremet is very difficult ad i curret systems, each task is provided with supplemetary memory for stack as well as heap to the best estimate obtaied. 3.2 Shared Supplemetary Memory We propose usig shared supplemetary memory (SSM) for hadlig stack ad heap overflow. The techique works as follows. The supplemetary memory from each task s stack (/heap) is removed ad combied to form a large chuk of shared supplemetary memory that ca be accessed by all tasks This is pictorially show i Figure b. Whe a task s stack (/heap) overflows, the stack is allowed to grow i SSM. We show by aalysis ad simulatios that havig a SSM ca improve reliability cosiderably. The ituitio is that sice the probability of multiple tasks overflowig the stack or heap is very low, SSM essetially provides more supplemetary memory to the overflowig task tha the o-ssm case ad thus the probability of overflow, caused by overflowig the SSM, is reduced sigificatly. The exact aalysis of the probability of multiple tasks overflowig their stack or heap memory is give i Sectio 4.. To implemet this solutio, we ca leverage the techique proposed i [4] to detect stack overflow. Heap overflow ca be easily detected durig memory allocatio. As memory protectio i embedded systems is desired to be abset ad ay task ca access ay part i the memory, havig a shared segmet i memory icurs a small overhead. To maage the SSM, a shared memory maager is required. The primary tasks of shared memory maager are allocatig or freeig memory for overflowig task ad removig the holes over time. Note that the shared memory maager comes ito play oly whe a task uses the SSM; durig ormal executio (i.e., whe o overflow occurs) the system would operate o a memory maager provided by the OS or by the respective tasks. As the probability of multiple tasks usig the shared memory cocurretly is rare, the performace overhead i fidig a free chuk of memory for a requestig task is always low. Note that compactig holes i SSM ca be doe as a low-priority task without affectig other tasks, whe the system has retured ito ormal operatio ad SSM is ot i use. 3.3 Opportuistic Micro-Rejuveatio Opportuistic micro-rejuveatio is a fie bled of classical software rejuveatio [] ad micro-reboot [3] obviatig the iappropriateess of both the schemes to the resource-costraied multi-taskig embedded systems. The key idea is that we reboot the tasks that are more likely to cause a overflow at opportue istats istead of the etire machie or idepedet tasks rebootig at radom istats for predictive maiteace, as evisaged i [] ad [3] respectively. Sice we do fie-graied rejuveatio at opportue istats, we call our techique opportuistic microrejuveatio. Whe OMR is used alog with SSM, the rejuveatio is doe whe the SSM usage is greater tha some threshold ad the task with most memory usage i the SSM is rejuveated. If the threshold is sufficietly large, say eighty percet of the size of SSM, the umber of times we rejuveate is miimal. There are several advatages to usig OMR ad are outlied i Sectio. The implemetatio of OMR is straightforward as the rejuveatio tasks ca ru as a backgroud task ad check for the threshold value or memory maager ca itimate the rejuveatio task whe the memory maager allocates memory above threshold. We do ot elaborate o the details i the iterest of space. 4. Evaluatio of SSM I this sectio, first, we evaluate the SSM techique usig aalysis ad simulatio. Evaluatig SSM ivolves comparig the three techiques to hadle iaccurate estimatio. First techique, referred to as BASE, is the curretly used techique where supplemetary memory is added idividually to each task. Secod techique, referred to as MTSS, is the multi-task stack sharig techique proposed by Middha et al [4], i which tasks lease stack space from other tasks whe there is memory overflow. Fially the third techique is the proposed SSM techique, i which, supplemetary memory of each task is combied to form a shared supplemetary memory. The metric used to compare the three techiques is reliability, which is defied as the probability of o memory overflow. Ituitively, we expect the performace of SSM to be i betwee BASE ad MTSS because these two represet the two extremes i how supplemetary memory is made available to a task upo overflow. Whe a task overflows its estimated memory, it has small supplemetary memory to grow i the BASE

5 case but potetially every task s free memory is available to grow i the case of MTSS. I SSM, sice the probability of multiple tasks overflowig their estimate is low, there is a large supplemetary memory available to grow ad thus probability of overflow is reduced. By aalysis ad simulatio, we cofirm the ituitio ad show that the reliability is improved cosiderably as it is closer to the MTSS case tha the BASE case. 4.. Aalysis Sice for the BASE ad the SSM case, stack overflow ad heap overflow are hadled i the same maer ad MTSS is proposed oly for stack overflow, we discuss the aalysis i the cotext of stack overflow. The same discussio is valid with respect to heap overflow except that MTSS is ot applicable. We make the followig assumptios. The amout of memory required by ay task is ormally distributed with mea μ ad stadard deviatio σ. The ratioale for this assumptio is that i commo cases the amout of memory required is more or less the same ad oly i rare iputs or rare program executio paths the amout of memory required is very high or very low. We use this assumptio to estimate the stack ad heap usage of a task. We also assume that the amout of memory required by a task is idepedet from other tasks i the system. Let there be tasks i the system ad each task be represeted usig a iteger betwee ad. Let X i be the Gaussia/Normal radom variable with mea μ i ad stadard deviatio σ i to deote the amout of stack memory required for a task i. Note that X i s for differet values of i are idepedet ad idetically distributed radom variables. Let α i deote the estimate of the stack memory usage by a task i. Let s i be the amout of supplemetary stack memory allocated to a task i. I the BASE case, α i + s i is the total amout of stack memory allocated to each task i. There is a stack overflow if ay of the task s stack requiremets grows larger tha the amout of stack memory estimated for the task. This ca be writte as i Equatio (0.). P{Stack Overflow i BASE} = - P{X } i α s i + i i= (0.) I the MTSS case, the tasks are allowed to share task space from memory ad there is stack overflow oly if the sum of all tasks stack requiremet is greater tha the total available stack memory for all the tasks i the system. This ca be writte as i Equatio (0.2). P{Stack Overflow i MTSS} = P{ X ( s )} i > α i + i i= i= (0.2) I the SSM case, task i will be allocated oly α i uits of memory ad a pool of stack memory called SSM that is shared amog all tasks is allocated. Note the task s ow space is ot shared i SSM. The size of the SSM is the sum of size of all idividual supplemetary memories ad is deoted by s. The stack overflow evet i this orgaizatio is ot as straight forward as i the other two cases. The stack overflow happes if space caot be allocated i task s stack or i the SSM. SSM is oly used whe the task s stack is exhausted. The SSM ca be used by oe or more tasks at the same time ad hece, stack overflow ca be due to a sigle task or ay combiatio of tasks. Takig ito accout all possible combiatios, the stack overflow evet s probability ca be writte as i Equatio (0.3). P{Stack Overflow i SSM} = P{ X > α + s X > α } * P{ X < α } i i i i j j i= j= j i r = 2 P{ X > α + s X > α }* P{ X < α } k k k k j j k { ( )} k k j= r j k P{ X > α + s X > α } k k k k k= k k + + (0.3) The idex k i the secod term of Equatio (0.3) represets oe combiatio ad it is boud for the etire term. For example, whe r is 2 ad is 3, k represet ay oe the followig combiatio: (, 2), (, 3), (2, 3). The two boudary terms i Equatio (0.3) ca also be take iside the summatio ad the expressio ca be rewritte as show i Equatio (0.4) P{Stack Overflow i System SSM}= P{ X > α + s }* { } X > α P X < α r= k k k k j j k k j= k { ( )} r j k (0.4) Sice a closed form for Normal CDF does ot exist, obtaiig a closed form for the Equatio (.4) is ot possible. Therefore, we resort to simulatio. We simulated all three cases ad foud that results from aalysis ad simulatio for the other two cases match. 4.2 Simulatio ad Results The simulatio is doe as follows: We geerate ormal radom variables usig Box-Muller method. The value of a geerated ormal radom variable represets the memory requiremet of a task. Each experimet cosists of geeratig memory requiremet of tasks ad checkig if ay oe of the cases would cause a overflow. I the BASE case, a overflow occurs if the ay oe of the tasks memory requiremet is greater tha the sum of task s stack memory ad supplemetary memory. Similarly, for MTSS case, a overflow occurs if the sum of all task s requiremet is

6 greater tha the total available memory. Note that i practice ot all available memory ca be used by MTSS due to iteral fragmetatio ad hece, the results obtaied is for MTSS ideal case. I SSM case, a overflow occurs if all possible combiatio of i tasks where i varies from to task s memory requiremet is greater tha the sum of tasks ow stack memory plus the SSM. Each simulatio ru repeats the experimets 00,000 times with differet radom seeds. The probability of overflow is calculated as umber of overflows divided by the total umber of experimets. Table : Parameters used i SSM simulatios Note memory uits are i k for stack ad 50k for heap. Parameter Name Number of tasks i the system () Expr. # Estimate of a task i ( α i ) 8.5 Supplemetary Memory of task i ( s i ) Mea of the task s istataeous memory requiremet ( μ i ) Stadard Deviatio of task s istataeous memory requiremet (σ i ) Shared Supplemetary Memory ( s) 8 8 Expr. #2.7, 8.5, ,.5, 3 7.4, 7, ,.5, The parameters used for the simulatio are give i the Table. I experimet, we assume for the sake of simplicity, that all tasks have equal memory requiremet ad thus estimated memory ad supplemetary memory are also equal for all tasks. The uit for memory ca be varied for the case of stack ad heap as stack segmet is usually much smaller tha the heap segmet. Note that i lie with our assumptios that we have made the estimate to be oe stadard deviatio away from the mea. This meas that the probability of usig supplemetary memory is reduced. I experimet 2, we choose oe-third of the task to have low memory requiremet, aother oe-third to have medium memory requiremets ad the oe-third to have high memory requiremet. We have plotted the total memory available i the system agaist the reliability ad are show i Figure 2. Sice the estimated memory does t vary i all the three cases, we vary the supplemetary memory. Therefore, the total memory i the system varies from sum of all estimated memory, that is, o supplemetary memory to sum of all estimated memory plus 400% supplemetary memory. Reliability BASE MTSS SSM BASE2 MTSS2 SSM Supplemetary Memory Figure 2: Variatio of reliability i three methods whe the supplemetary memory is icreased from 0% to 400%. Whe there are o supplemetary memories, SSM reduces to the BASE case. As the available supplemetary memory icreases, the reliability icreases expoetially i the case of SSM. This is because ot all tasks use the SSM cocurretly ad hece, few tasks have more supplemetary memory. However, i the BASE case, the supplemetary memory is ot combied ad task that uses supplemetary memory has oly its ow supplemetary memory ad it caot borrow supplemetary memory from other tasks. Note that for the same reliability, SSM requires much less supplemetary memory tha BASE ad MTSS requires the least. I the case of MTSS, the parameters chose for the simulatio provide eough available memory i other tasks estimated space ad the supplemetary memory was ot eeded. From the Figure 2, we see that the results for experimet 2, i which total memory is divided uequally amog the tasks, follow the same tred as experimet, i which total memory is equally divided amog the tasks ad thus geeralize the advatages explaied above. 5. Evaluatio of OMR Next, we use stochastic models to evaluate the OMR techique. Sice overflow failures ad hardware exceptios are rare, a real implemetatio-based evaluatio is ot feasible. Hece, we model our system usig Stochastic Activity Networks (SAN) [3, 4],which has bee successfully used to model ofuctioal aspects such as reliability ad availability of

7 may complex systems [4]. Followig the traditioal model used for software rejuveatio [], we model the system usig three states, amely, robust, vulerable, ad failed. The metric that we have used is availability of the system ad is defied as the average availability of all the tasks i the system. Let A i be the availability of the task i. The availability of the system of tasks, A, is defied as the average of tasks availabilities. That is, A = A i /. We compare OMR to the base case which is the existig system that reboots o a crash. Here we assume that the estimatio of memory is accurate ad memory overflow is oly due to Madelbugs. We show that sice MadelBugs are rare ad the recovery o a failure is usually very quick, there is oly slight icrease i availability. As expected, mea time to crash is very high i OMR as opposed to the base case. 5. SAN Model Followig the traditioal model used for software rejuveatio [], we model the system usig three states, amely, robust, vulerable, ad failed ad the trasitio times betwee these states are expoetially distributed. The Stochastic Activity Network (SAN) model for the base case ad the OMR case are show i Figure 3 ad Figure 4 respectively. Figure 3: SAN Model used for the base case where there is o rejuveatio. Figure 4: SAN Model used for the OMR case As the places ad trasitios are self-explaatory, we metio oly the importat aspects of the model. All the trasitios follow the expoetial distributio with rate show as i Table 2. I Figure 3, whe the Aoverflow activity fires, the marks i both places Probust ad Pvul are removed ad all marks are placed i Pfailure to emulate failure of the system. Similarly whe the Afailrecover activity fires, the place Probust is iitialized to total marks ad marks from failure is removed to emulate a system reboot. 5.2 Simulatio ad Results We solved the above described SAN models o Mobius tool[3] usig simulatio. The parameters used i the experimets are show i the Table 2 ad are chose so that they are close to real-world embedded systems. Table 2: Parameters used i the SAN simulatio. Parameter Base Case OMR OMR_SLOW Number of tasks/task pools Vulerability Rate ( i /hour ) Overflow after 0. to vulerability Rate ( i /hour ) - - Fail Recover Rate ( i /hour ) Rejuveatio Rate ( i /hour ) - 0. to 0. to I a embedded system, we assume that the ratio of restartig a system to rebootig a task to be 6. The vulerability rate is kept costat at oce every 00 hours ad the overflow rate is varied from oce every 0 hours to oce every hour. Oe order of magitude differece betwee vulerability rate ad overflow rate is because i embedded system oce vulerability happes, overflow is immiet due to the fact that amout of memory available i embedded systems is less. Note that a overflow happes oly after the system becomes vulerable ad therefore, the average umber of hours it takes to cause a overflow sice the start of the system is 0 to 0 hours. Figure 5 shows the steady state system availability as the average time to overflow after the vulerability is varied. The BASE, i Figure 5, represets the existig system that reboots o a system crash due to memory overflow. The simulatio result shows that micro-rejuveatio or task level rejuveatio icreases availability slightly compared to the base case. The reaso we see oly slight icrease i availability (0.02%) is because we look exclusively at rarely occurrig bugs. I practice, however, more memory overflows occur due to iaccurate estimatio. Sice we could t iclude the memory overflows caused by iaccurate estimatio ito our stochastic model, we are ot able to evaluate the combiatio of OMR ad SSM. Whe OMR combies with SSM, we ca show a sigificat improvemet i availability. The reaso is that the probability of a overflow is very small (less tha %) whe the threshold is 80% of SSM whe

8 OMR ad SSM are used together. However, i the base case the probability of overflow with the same threshold is high (aroud 30%). This is also the reaso why we call these two techiques complemetary. Aother reaso for oly slight improvemet is that we assume that every task ages which is ot the case ad thus it is a worst case improvemet for OMR. Availability BASE OMR OMR_SLOW Average time to overflow after vulerability ( i hours) Figure 5: Effect of availability whe average time to overflow after vulerability varies. The OMR_SLOW, i Figure 5 represets the case whe the recovery rate of OMR is reduced to be same as the base case ad hece, the tasks recover slower i this case. The OMR_SLOW shows that microrejuveatio is ot always better tha restart because i the OMR case tasks age eve whe other tasks rejuveate as opposed to reboot i which all tasks get rejuveated whe rebootig. 6. Coclusio ad Future Work I this paper, we idetified two mai causes of memory overflow, amely, iaccurate estimatio ad presece of MadelBugs.. We proposed two complemetary techiques amely Shared Supplemetary Memory (SSM) ad Opportuistic Micro Rejuveatio (OMR) to hadle iaccurate estimatio ad Madelbugs respectively. We showed that cosiderable icrease i reliability ca be obtaied usig SSM ad for the same reliability, the total memory requiremet ca be reduced cosiderably. Syergistic combiatio of OMR with SSM ca yield sigificat improvemet i reliability ad availability. I future work, we are curretly implemetig our techiques i ucliux, a ope source embedded operatig system. We are also lookig ito ways of developig stochastic petriet based models that ca hadle both idetified causes of memory overflow simultaeously. 7. Refereces [] Yeu Huag, Chadra Kitala, Nick Kolettis ad N. Dudley Fulto; Software Rejuveatio: Aalysis, Module ad Applicatios. I Proc of the 25th Itl. Symposium o Fault-Tolerace Computig (FCTS-25), Pasadea, CA, Jue 995 [2] Dr. Doug Locke; Real-Time & Embedded Systems: Past, Preset ad Future. I Keyote Talk at 0th IEEE RTAS Coferece, Toroto, 2004 [3] George Cadea, Shiichi Kawamoto, Yuichi Fujiki, Greg Friedma ad Armado Fox; Micro-reboot: A techique for cheap recovery. I Proc of the 6th coferece o Symposium o Operatig Systems Desig & Implemetatio Volume 6, CA, 2004 [4] Bhuva Middha, Matthew Simpso ad Rajeev Barua; MTSS: Multi Task Stack Sharig for Embedded Systems. I Proc of the ACM Iteratioal Coferece o Compilers, Architecture, ad Sythesis for Embedded Systems (CASES), Sa Fracisco, CA, September 25-27, 2005 [5] Kihwal Lee ad Lui Sha; Process Resurrectio: A Fast Recovery Mechaism for Real-Time Embedded Systems. I Proc of 2005 th IEEE Iteratio Coferece o Embedded ad Real-Time Computig Systems ad Applicatios [6] Joh Regehr, Alastair Reid ad Kirk Webb: Elimiatig stack overflow by abstract iterpretatio. I Proc of the 3rd Iteratioal Cof o Embedded Software, Philadelphia, PA, 2003 [7] Michael Grottke ad Kishor S. Trivedi. Fightig Bugs: Remove, Retry, Replicate, ad Rejuveate. I IEEE Computer February [8] G. V. Neville-Neil. Programmig without a et. ACM Queue: Tomorrow s Computig Today, (2):6 23,April [9] N.D.D. Brylow ad J. Palsberg. Stack-size estimatio for iterrupt-drive microcotrollers. Techical report, Purdue Uiversity, Jue [0] P.R. Pada, F. Catthoor, N. D. Dutt, K. Dackaert, E. Brockmeyer, C. Kulkari, A. Vadercappelle, ad P. G. Kjeldsberg. Data ad memory optimizatio techiques for embedded systems. ACM Trasactios o Desig Automatio Electroic Systems, 6(2):49 206, 200. [] Surupa Biswas, Thomas Carley, Matthew Simpso, Bhuva Middha ad Rajeev Barua. Memory Overflow Protectio for Embedded Systems usig Ru-time Checks, Reuse ad Compressio. I ACM Trasactios o Embedded Computig Systems (TECS), 5(4), pp , Nov 2006 [2] Nagauma Masayuki, Eto Takeshi. Method ad apparatus for cotrollig stack area i memory space. Published patet applicatio US [3] G. Clark, T. Courtey, D. Daly, D. Deavours, S. Derisavi, J. M. Doyle, W. H. Saders, ad P. Webster. The Möbius Modelig Tool. Proceedigs of the 9th Iteratioal Workshop o Petri Nets ad Performace Models, Aache, Germay, September -4, 200, pp [4] W. H. Saders ad J. F. Meyer. Stochastic Activity Networks: Formal Defiitios ad Cocepts. i E. Briksma, H. Hermas, ad J. P. Katoe (Eds.), Lectures o Formal Methods ad Performace Aalysis, First EEF/Euro Summer School o Treds i Computer Sciece, Berg e Dal, The Netherlads, July 3-7, 2000, Revised Lectures, Lecture Notes i Computer Sciece o. 2090, pp Berli: Spriger, 200