Checkng and Testng in Nokia RMS Process

Transcription

1 An Integrated Schedulng Mechansm for Fault-Tolerant Modular Avoncs Systems Yann-Hang Lee Mohamed Youns Jeff Zhou CISE Department Unversty of Florda Ganesvlle, FL 326 Advanced System Technology Group AlledSgnal Aerospace Columba, MD 245 youns, Abstract In ths paper, we present an effectve schedulng approach for a fault-tolerant IMA (Integrated Modular Avoncs)-based system. The system archtecture conssts of connected cabnets that are made of multple lne replaceable modules, such as core processor and I/O modules. To provde fault tolerance, the system s ncorporated wth fault reslent capablty and executes replcated tasks on dfferent cabnets. Thus applcaton output wll be ready after a task processng stage and a consstency checkng stage. To schedule the two-stage operatons at task processng nodes and at the voter, we adopt fxed prorty executves and nvestgate two prorty assgnment algorthms. Several experments have been conducted to measure the success ratos of fndng feasble schedules under varous condtons. The evaluaton results reveal a proper desgn space n whch feasble schedules can be found easly. TABLE OF CONTENTS. INTRODUCTION 2. SYSTEM ARCHITECTURE AND SCHEDULING MODEL 3. SCHEDULING ALGORITHMS 4. SCHEDULING EVALUATION AND EXPERIMENTS 5. CONCLUSION. INTRODUCTION Computer controllers are the core unts used n real-tme embedded systems. Such controllers or embedded processors may devate from general-purpose computer processors snce they are desgned for some specal applcatons and have substantally dfferent performance and mplementaton constrants. Varous hgh-end 32-bt processors have found a fast expanson n avoncs, telecommuncaton, mltary, aerospace, manufacture plants, and medcal montorng applcatons where computaton power and safe operaton are rgdly requred. These systems not only must be fault tolerant, but also must meet task executon deadlnes as ther applcatons are often mssonand safety-crtcal. Consder an example of the autolandng systems of wdeboded jumbo passage jets [] whch control crtcal functons of arcraft's moton n each axs,.e., roll, ptch, and yaw. They are requred to operate n all vsblty condton, ncludng zero horzontal vsblty and zero celng, and become truly crtcal n a 5 second nterval just before touchdown. To certfy the systems, a desgn must demonstrate a probablty falure less than -9 durng ths crtcal nterval. Ths hgh relablty requrement mples that the systems must tolerate most cases of component falures and must guarantee % tmng correctness. A cost-effectve desgn to reach ths strct requrement becomes extremely mportant consderng that a falure can be catastrophc and that the avoncs systems account for some 3% of the total cost of a new arplane [5]. One approach to reduce the desgn and mantenance cost of avoncs system s to take a modular approach. Tradtonal avoncs systems are mplemented wth autonomous and federated archtectures []. They consst of a number of nterconnected but functonally ndependent (or loosely coupled) subsystems. In order to mprove performance, flexblty, and avalablty of avoncs systems, t s necessary to avod possble duplcaton of functons and to allow resource sharng of system components and standard modules. One sgnfcant approach for cost-effectve avoncs systems s based on the ntegrated modular avoncs (IMA) approach n whch hardware and software systems are decomposed nto modules and then ntegrated for varous avoncs applcatons []. The IMA approach suggests an archtecture that conssts of a set of nterconnected cabnets. Each cabnet, contanng a standard backplane nterconnecton and multple lne replaceable modules (LRM), forms a common platform to house the executon of software modules. Wth standard nterfaces, hardware and software modules become nterchangeable and can be reused. They can also be upgraded usng new technology to add new functons. Thus, t s expected that the lfe-cycle cost of avoncs systems can be reduced, and the processes of system development, and mantenance can be smplfed.

2 As we gan the advantages of IMA approaches, t s expected that the whole system as well as the nteractons between modules must be consdered n the desgn and ntegraton process. Modules must be put together to collectvely perform applcaton functons wth a hgh relablty. In addton, the performance requrements must nclude a guarantee of responsveness. A cabnet that cannot schedule all crtcal tasks to meet ther deadlnes may cause a tmng error that can be catastrophc for tme-crtcal tasks. For nstance, a mss of deadlne n the control loop of the autolandng system may cause a crash. Thus n order to mplement the IMA approach for avoncs applcatons, we requre the hardware and software platforms to be able to:. allocate task modules of dfferent applcatons nto several canddate processors n whch tasks can be scheduled accordng to ther ndvdual tmng constrants. 2. provde fault tolerance mechansms such that replcated task executons and checkng operatons can be managed coherently. The desgn gudance report for IMA [] lsts several example archtectures that can utlze modular components and nstall fault tolerance mechansms at varous levels. As the applcaton modules are ntegrated n one or more cabnets, we may assume that operaton executves can dspatch ready tasks based on ether cyclc or prorty-drven schedules [4]. On the other hand, the fault tolerance capablty at system and cabnet levels can be establshed by ncorporatng redundant task executon at remote cabnets or at redundant processors. For ether of these fault tolerant arrangements, the executons should be done before a checkng process can verfy the correctness of the results, and the accumulated response tmes must be bounded wthn the task s deadlnes. Ths response tme requrement clearly ndcates that the schedulng of task replcatons and the result checkng process must be addressed altogether. The ssues of schedulng tasks n fault tolerant systems have been nvestgated n prevous research work. For nstance, Krshna and Shn devsed a ghost allocaton mechansm n order to generate backup tasks [9]. The algorthm assumes that there exsts a schedulng algorthm that checks the schedulablty. The optmal allocaton of replcated tasks under rate monotonc schedulng was studed n [6]. In addton, a dynamc schedulng and redundancy management approach was proposed n the Sprng system n whch replcatons can be created durng system operaton stages [6]. On the other hand, there are approaches to replcate dentcal subsystems. Then, smlar to the cyclc schedulng executves, task executons and checkng process are scheduled at specfc nstances [8, ]. In these studes, the man focus s the schedulng of prmary and backup tasks. In ths paper, we look nto two desgn ssues: how to embed a schedulng mechansm nto a fault tolerant IMA system, and how to mplement fxed-prorty schedulng algorthms for task executon and result checkng. Buldng fault reslence at system level can be an adequate approach for avoncs systems. Ths s due to the fault contanment ntroduced by the physcally dstrbuted cabnets that are often equpped wth ndependent power and clock sources. The task executon n each cabnet should meet a target tme whch s shorter than the task deadlne. Thus, the computaton results can be released to the followng checkng process n order to reach nteractve consstency. In fact, as the task processng s modeled by a two-stage ppelne, we need to fnd feasble prorty assgnments such that the sum of processng delays at the stages s bounded to the gven tmng constrant. Through an extendble tmng analyss, the system behavor under the fxed prorty schedulng algorthm can be predcted [8]. Most mportantly, the approach does not have to examne every executon nstance, thus makes t easy to accommodate any changes of system load. In the followng chapters, we frst present the system archtecture and descrbe the fault tolerance mplementaton wth a Redundancy Management System (RMS) unt. Also, we show the schedulng model for such a system. In Secton 3, we focus on the proposed algorthms to determne sutable prorty assgnments for a two-stage fxed prorty schedule. The performance of the prorty assgnment algorthms s evaluated n several experments. The success ratos of the algorthms are reported n Secton 4. Fnally, a short concluson follows n Secton SYSTEM ARCHITECTURE AND SCHEDULING MODEL Avoncs systems typcally consst of a number of cabnets that contans varous modules to perform applcaton processng, data communcaton, and local I/O operatons to sensors and actuators. Each cabnet s made of multple lne replaceable modules (LRMs) of dfferent types, such as CPU module, standard I/O and communcaton module, specal I/O module, power supply module, etc. Wth the consderaton of I/O requrement, wre length, mantanablty, and payload areas, cabnets are physcally dstrbuted throughout the arplane. For avoncs applcatons, ths set of cabnets can be vewed as a dstrbuted mult-computer system where applcaton tasks and ther redundant copes can be ntated n multple cabnets and/or multple modules. Thus, a falure of a cabnet or a LRM can be tolerated and the system functons contnuously wth a proper fault management scheme. In the followng, we wll present a fault tolerance desgn based on AlledSgnal s MAFT archtecture to provde system level fault reslence n a cabnet-based avoncs system. Then, we show a schedulng model sutable for ths archtecture. We assume that the cabnets are organzed accordng to the example archtecture C of the IMA report []. Under the archtecture C, sgnal I/O s handled by remote data concentrators, thus the processng resources are physcally ndependent of ther I/O data. At each CPU module, the software structure ncludes a sngle executve and multple applcaton tasks. Applcaton tasks are replcated across the redundant cabnets and a consstency

3 Broadcast Network Cabnet Bus RMS CPU CPU2 Bus Interface RMS CPU CPU2 Bus Interface RMS CPU CPU2 Bus Interface RMS CPU CPU2 Bus Interface Global Data Bus Fgure. The fault-tolerant cabnets for IMA systems checkng operaton ensures that correct results are always avalable gven a lmted degree of falure. Archtecture Model The archtecture model of our fault tolerant IMA system s shown n Fgure. In addton to typcal LRMs, each cabnet s equpped wth a Redundancy Management System (RMS) module whch provdes system executve functons such as synchronzaton and data votng. Wth ths quad-redundant archtecture, the system can tolerate a sngle Byzantne-type falure. The approach can let a system developer concentrate on system applcaton desgn and rely on the RMS module to acheve fault tolerance and redundancy management at the system level. The desgn of RMS s orgnated from the MAFT (Mult-computer Archtecture for Fault Tolerance) system [8, 9] and can be mplemented by a mx of hardware and software components. The prmary functon of the RMS modules s to provde a consstency checkng and votng mechansm. Wth a fully connected broadcast network, RMS performs votng on data values collected from replcated applcatons that are allocated throughout the redundant cabnets. Such data votng mantans consstency between the cabnets. In addton, t asssts n recoverng from transent and ntermttent faults by replacng any corrupted applcaton data wth the voted values. Moreover, RMS votes on ts nternal state and error reports to mantan a global consstent system vew of the system health status. In order to perform checkng and votng operatons, the RMS modules of multple cabnets must be synchronzed and a global clock must be mantaned n ths loosely coupled dstrbuted system. Each RMS has ts own clock and the system synchronzaton s acheved by exchangng the local tme among all RMS modules and correctng the local clock accordng to the cardnalty of clocks from all healthy RMS unts. A dstrbuted agreement mechansm s used to prevent any sngle pont of falure and a faulttolerant votng algorthm s used to protect the global system clock from falure by any type of faults ncludng Byzantne faults. Ths synchronzaton wll be nvoked perodcally so the system can lmt the clock skew and detect a falng cabnet mmedately. The ultmate goal of RMS s to prevent a system falure durng the duraton of a crtcal msson as a result of some error manfested by a fault on one node. After votng, RMS can detect, contan and recover from errors. By comparng the voted data values wth the data submtted by the cabnet, RMS detects errors and penalzes the faulty one. Snce all modules wll be usng the voted data values, errors can be tolerated and the faulty module wll get a chance to recover by usng the voted data. In addton, RMS supports dynamc system confguraton by excludng faulty modules and readmsson of recovered modules. From an applcaton s pont of vew, a task s replcated and statcally allocated to CPU modules of dfferent cabnets. The replcated nstances of a task are executed synchronously,.e. they must execute wth the smlar nput data and produce results before a scheduled votng nstance. When a result s generated by the task, t s passed from a host CPU module to the RMS module. When the RMS modules agree on the votng process on and completes the votng operaton, the result s verfed and become avalable for further computatons or I/O operatons. Thus, a set of CPU modules of dfferent cabnets can be regarded as a logcal processng node f they are runnng the same set of tasks. The RMS modules mantan the consstency between the replcated executons automatcally as long as enough executon and votng tmes are scheduled. Schedulng Model To buld the schedulng algorthms n the proposed system, we may nvestgate the task processng model depcted n the followng fgure. The system conssts of m processng nodes and each node has a proper degree of redundancy. Also ncluded n the system s a voter (mplemented by the RMS unts) whch verfes computaton results before they are put out. We assume that there are n tasks allocated to

4 processng node, where m. A task TS j wll be nvoked perodcally wth a perod T j. For each nvocaton, t takes a worst-case executon tme (WCET) C j at processng node and produces at most k j data tems n the votng queue. The votng process of these tems must be completed before the task s deadlne D j, where C j D j T j. processng nodes Task TS j arrves deadlne D j Votng ready tme VRT j Data tems (jon on-tme are ready arrvel queue) (jon early arrval queue) data tems are verfed votng overhead b VT votng cycle Fgure 3. The tmng dagram of votng operatons tasks votng queue voter Fgure 2 Schedulng model for task processng and votng To model the operatons n the RMS, we assume that votng cycles are ntated every VT seconds as shown n Fgure 3. Durng each votng cycle, the voter takes the ready tems from the votng queue and makes the verfed data avalable at the end of cycle. Gven b as the base overhead to perform the synchronzaton and to ntate votng actons, and τ as the votng tme for each data tem, the voter can verfy at most (VT b)/τ data tems n one cycle. The remanng tems n the queue and the newly arrved tems wll be voted n the subsequent cycles. Note that the votng queue s dvded nto two parts: an on-tme arrval queue and an early arrval queue, where the data tems n the on-tme arrval queue has a hgher prorty to jon the votng process than the data tems n the early arrval queue. For each task TS j, let VRT j be the target votng ready tme that ts output data tems enter the votng queue. Ths target votng ready tme can be vewed as the executon deadlne at the processng node and D j - VRT j becomes the deadlne of the votng processng for task TS j. If the task s completed earler than VRT j, the data tems can wat n the early arrval queue. On the other hand, f ts data tems have not been voted before the target votng ready tme, the tems are watng n the ontme arrval queue. Wth ths two-queue scheme, we can avod the mpact of arrval jtters to the schedulng process at the voter [3]. The worst-case arrval sequence to the voter occurs when all tasks release ther data tems at ther target votng ready tmes, thus formng perodc requests to the voter. In ths paper, we explore the feasblty of usng fxed prorty preemptve schedulng [3] at the processng nodes and the voter. Wth statc prorty assgnments at the processng nodes and at the voter, the task dspatchers can be easly mplemented and the choce of data tems to be voted n each cycle can be readly determned. Also, we can compute the worst-case response tmes for each task at the processng nodes and at the voter n order to check the schedulablty. Gven the executon prorty p (j) for each task TS j, the worst-case response tme RS j at processng node can be computed by the followng recursve equaton [8]: RS j RS j = C j + C Tl l l hp ( j) where hp (j) s the set of tasks of hgher prorty than task TS j at processng node. Smlarly, wth the votng prorty p v (,j) for task TS j, the maxmum votng delay VD j can be calculated as the sum of two parts: the ntal watng for the begnnng of a votng cycle and the votng response tme, VRS j : VRS j VD j = VT + VT VT VRS j VRS k VT b VRS VT VT j j = j + k + / τ τ Tl l hp v (, j) where hp v (,j) s the set of tasks of hgher votng prorty than task TS j. The equaton for VRS j assumes that the data tems from each task TS j arrve perodcally wth a perod T j. Ths case occurs when all task computatons are done at the target votng ready tmes. 3. SCHEDULING ALGORITHMS As the task processng s decomposed nto two stages at the processng nodes and the voter, a schedulng algorthm must provde the prorty assgnments p (j) and p v (,j). In addton, t must supply the target votng ready tme VRT j to begn the votng process. A feasble schedule s a set of (p (j), p v (,j), VRT j ) for all tasks TS j under whch RS j + VD j D 2 j. Among all statc prorty schemes, we adopt a deadlnemonotonc approach at each stage. Note that under a deadlne-monotonc prorty assgnment, task prortes are assgned nversely proportonal to the length of the deadlnes l Note that an arrval jtter caused by the varaton of executon tme can make the votng arrval process rregular. 2 An unnecessary strngent defnton of schedulablty s to have RSj RL j and VD j D j - RL j.

5 and such an assgnment s an optmal staton prorty scheme [3,2]. In fact, wth deadlne-monotonc approach, we only need to determne one of the three parameters, p (j), p v (,j), and RL j, and solve the other two based on ther dependency. Thus, f VRT j s known, we can use VRT j and D j -VRT j as the deadlnes at the processng node and the voter to assgn p (j) and p v (,j) accordng to a deadlne monotonc approach. Smlarly, f p (j) s known, we can compute the response tme RS j and assgn t to VRT j. Then, p v (,j) can be determned. The schedulng approach of determnng VRT j s smlar to solvng a 2-stage deadlne dstrbuton problem n whch D j s parttoned nto two parts n order to meet the end-to-end deadlne [7]. For a general deadlne dstrbuton problem, several heurstc approaches have been proposed for dstrbuted real-tme systems and dependent task sets. For nstance, the deadlne of a task can be evenly parttoned and used as the deadlnes of ts subtasks [4]. Search algorthms have also been appled for the problems, e.g., an teratve deadlne assgnment approach to mprove the schedulablty [7]. Recently, D Natale and Stankovc suggested a slcng technque, whch allocates slack tme 3 to the subtasks n the crtcal path of a task graph accordng to normalzed laxty and pure laxty metrcs [5]. Under the normalzed laxty metrc, slack tme s dstrbuted n proportonal to subtask executon tmes, whereas, under the pure laxty metrc, each subtask receves an equal share of slack tme. Consder the computaton and votng ppelne n our system. The votng processng tme s expected to be much smaller than the task executon tme at a processng node when the system has only one voter shared by all tasks. The pure laxty metrc cannot be effectve snce, wth an equal share of slack tme, the computaton deadlnes become tght and the schedulng at each processng node can be feasble only f the processor utlzaton s low. On the other hand, the approach wth normalzed laxty metrc assgns a bgger slce of slack tme to the computaton stage than to the votng stage. Ths can be a reasonable approach snce the schedulng of short votng processes s easer than the schedulng of long task executons. The approach uses D C VT k VT b VT VRTj C j j τ j /( ) = j ( + ) C j to set the deadlne of task computaton. Thus the prortes at the processng nodes and the voter, p (j) and p v (,j), can be assgned based on deadlne-monotonc assgnment. The schedulablty can then be determned once we compute RS j and VD j, and check that RS j + VD j D j. Instead of parttonng the deadlnes, an alternate schedulng approach s to determne p (j) and p v (,j) drectly. Let Φ and Θ be optmal prorty assgnments at the processng nodes and the voter. Denote RS j (Φ) and VD j (Θ) as the task response tmes at processng nodes and the votng delays at the voter for task TS j under the prorty assgnments Φ and Θ, respectvely. Gven that deadlne-monotonc s an optmal statc schedule [3,2], we can easly observe that Φ and Θ must be deadlne-monotonc prorty assgnments based on the deadlnes {D j - VD j (Θ)} and {D j - RS j (Φ)}, respectvely. Ths dependence suggests an teratve approach whch begns wth a prorty assgnment at the processng nodes Φ. After RS j (Φ ) s computed, a deadlne-monotonc prorty assgnment for the voter Θ can be obtaned. Then, a deadlne-monotonc prorty assgnment for the processng nodes Φ 2 can be defned based on VD j (Θ ). The teraton can contnue untl ether the prorty assgnments are feasble or there s no more mprovement n schedulablty. Ths schedulng algorthm, called DMA2, s gven n the Fgure 4. Algorthm DMA2: VD j = τk j / (VT - b) VT; old_tradness=; no_tres=; repeat for each processng node { for each task TS j ( j n ) { assgn p (j) nversely proportonal to D j - VD j ; } compute RS j for each task TS j ( j n ); } for each task TS j ("j"n and ""m) { assgn p v (,j) nversely proportonal to D j - RS j ; } compute VD j for each task TS j ( j n and m); tardness = max { RS j + VD j - D j }; f (tardness >= old_tardness) no_tres++; old_tardnes s= tardness; untl { tardness <, # schedulable or no_tral = a maxmal threshold } # too many unsuccessful trals Fgure 4. The DMA2 algorthm for prorty assgnments Note that, n DMA2 algorthm, the computaton starts wth a zero votng delay ntally. Ths ntal settng leads to a deadlne-monotonc assgnment of p (j) almost smlar to the one based on the deadlne D j. In fact, as the votng delay s much smaller than the task computaton tme, ths ntalzaton selects an assgnment that focuses on the task delay caused n the processng nodes. Also, DMA2 algorthm search for the prorty assgnments that are mutually deadlne-monotonc. It may not be able to fnd such a par of assgnment or fal to fnd a feasble one. In these cases, the algorthm wll termnate after the number of trals reaches a threshold. 3 A task s slack tme s defned as the dfference between ts deadlne perod and ts total executon tme.

6 4. SCHEDULING EVALUATION AND EXPERIMENTS We performed several experments to determne the performance of the slcng approach based on the normalzed laxty metrc and DMA2 prorty assgnment algorthm. Snce the performance of the schedulng algorthms s lkely affected by varous parameters, we ntend to set the values of the parameters n varous ranges. The expermental envronment has 4 processng nodes and each node has 6 tasks. Thus, data tems generated by a total of 24 tasks wll be voted. For each experment, we collect the success ratos of the schedulng algorthms among 5 random cases. To defne varous parameters n the experments, we frst assume the votng overhead per each votng cycle,.e., b, s a constant and s equal to one unt of tme. The votng cycle s set to a perod of 2, except the experment that the perod s a control parameter. The task perods are dstrbuted unformly n the range of [5, 5]. Then, for a gven utlzaton at each processng node, we assgn the task executon tmes such that the fracton of processor tme spent n executng each task s randomly dstrbuted n the range of [5%, 25%]. To determne votng processng tmes, we frst assgn the votng processng tme for task TS j to be unformly dstrbuted between [5%, 45%] of the task s executon tme at the processng node. Then, all votng processng tmes are adjusted proportonally such that the overall utlzaton of the voter (ncludng the votng ntalzaton overhead) s n the range of 5 to.9. Our frst experment s to examne the performance of slcng technque and DMA2 algorthm under dfferent utlzaton. As shown n Fgure 5, two sets of curves are plotted n whch the utlzaton of each processng node s set to 5 and.65, respectvely. The cases wth a utlzaton of.65 probably have the hghest utlzaton we need to consder snce t s slghtly less than the theoretcal utlzaton bound (.e. ln 2) of rate monotonc algorthm n a sngle server [3]. The experments assume that the task deadlnes are equal to ther perods, and the votng perod s 2. The utlzaton at the voter vares from 5 to.9. The fgure reveals several nterestng propertes of the two schedulng algorthms. When the utlzaton at the voter s less than 5%, both algorthms can reach a success rato of % even f the utlzaton of each processng node reaches.65. Caused by the small votng processng tmes, the voter s lkely to complete the votng process for any output data n one cycle and mpose at most two cycle delay n verfyng the correctness of output data. Ths delay doesn t make many dsturbances n selectng a good schedulng at the processng nodes. However, when the utlzaton of the voter ncreases, the success ratos of the algorthms begn to dverge. A sgnfcant dfference can be observed when the utlzaton of the voter s.75 and the utlzaton of the processng nodes s.65. The DMA2 algorthm has a success rato of.98 whereas the slcng approach can make only 2% of the cases feasble Fgure 5 also shows substantal dfferences n the change of success ratos for each algorthm when the utlzaton of the success rato voter utlzaton 5, DMA2 5, slcng.65, DMA2.65, slcng Fgure 5. The success ratos of slcng and DMA2 algorthms measured n Experment processng node vares from 5 to.65. Wth a voter utlzaton larger than.55, the success rato under the slcng approach drops drastcally wth an ncrease of processor utlzaton. It suggests the settng of votng ready tmes does not wegh the ncreased delays at both the processng node and the voter. Conversely, DMA2 algorthm can adjust the prortes properly, thus avod the adverse effort caused by the ncrease of utlzaton at the processng nodes. Ths schedulng mplcaton can be explaned n a smple example. Consder two tasks wth slghtly dfferent deadlnes. If ther ratos of the executon tme at the processng node to that at the voter are smlar, the deadlnes are dstrbuted to each stage wth the same proporton. Then, the task wth a shorter deadlne wll be assgned a hgher prorty at both the processng node and the voter. It can be completed much earler n the expense of the task wth a longer deadlne that may mss ts deadlne. On the other hand, under DMA2 algorthm, the task recevng a hgher prorty at the processng node s lkely assgned wth a lower prorty at the voter. Thus, the total delays of these two tasks are lmted due to the shuffle of prortes, and can meet the deadlnes as long as the utlzaton s not hgh. To reassess the prorty shufflng under DMA2 algorthm, we conduct the second experment that compares the success ratos of two cases. The utlzaton at the processng nodes and the votng cycle are fxed at 5 and 2, respectvely. In the frst case, we set task deadlnes randomly n the range of 6% to % of the task perods, whereas the second case assumes task deadlnes are dentcal to the task perods. As the deadlne shrnks, we expect that the tasks wth long response tme and votng delay wll mss ther deadlnes. Thus, by examnng the success ratos, we can detect the exstence of long task delays. The results of the experment

7 success rato voter utlzaton D=[.6,]T, DMA2 D=[.6,]T, slcng D=T, DMA2 D=T, slcng Fgure 6. The success ratos of slcng and DMA2 algorthms measured n Experment 2 are shown n Fgure 6. The reducton of deadlnes leads to a bgger decrease of success rato for the slcng approach than for DMA2 algorthm. Ths dfference mples that the slcng approach may result n a delay dstrbuton that has a long percentle. Also, the effect of prorty shufflng under DMA2 algorthm can elmnate the long percentle and allow ndvdual tasks to meet ther deadlnes. As the frst two experments assume that all processng nodes have the smlar utlzaton, we look nto the cases that the loads n processng nodes are not balanced n our thrd experment. We set the average utlzaton of processng node to.55. Under the balanced case, each node has an equal utlzaton, whereas, n the unbalanced case, the utlzaton s set to 5 and.65 for two pars of nodes. The success ratos wth the slcng approach and DMA2 algorthm are shown n Fgure 7. The success ratos under DMA2 algorthm are almost dentcal n both balanced and unbalanced cases. Apparently, the prorty assgnment at the voter adapts to the utlzaton of each processng node by comparng D j - RS j. Thus, the task whch experences a long computaton delay n a heavly loaded processng node can stll meet ts deadlne by holdng a hgh prorty at the voter. On the contrary, the slcng approach fals to make any compensaton to the varaton of node utlzaton as t does not check the loadng stuaton at all n the determnaton of the target votng ready tmes. Our next experment s to nvestgate the effect of varous perods of votng cycles. To make the synchronzaton between RMS unts easy, the votng s ntated perodcally by a hardware clock. Ths results n a perodc votng server and an ntaton overhead per each cycle. Consder the mpact of votng cycle tmes to the schedulablty. We can conjecture that a short perod may ncrease the utlzaton of the voter, and a long perod may brng up addtonal watng tme to the votng requests. These suppostons are llustrated n Fgure 8 that plots the success ratos wth the perods vary from 5 to 5. Note that, n all cases, the utlzaton due to votng process (excludng the synchronzaton overhead) s set to.55. By addng the synchronzaton overhead per cycle, the net utlzaton at the voter vares from.75 to.57. In addton, the fgure also confrms the stablty of the DMA2 algorthm aganst any change of processng load and votng cycle. Comparng the cases of experments and 4, we may observe that feasble schedules can be found easly when the votng perod s short. In experment, we have a case that the utlzaton of the voter s.75 and the votng perod equals to 2. Wth the same utlzaton, we have a case n experment 4 n whch the votng perod equals to 5. The resultng success ratos are qute dfferent,.e., much hgher for the cases n the experment 4 than that n the experment. As our voter begns a votng process perodcally, the votng delay ncreases lnearly as we prolong the votng perod. A good desgn of the voter should mantan a utlzaton less that 65% whle choosng a small votng perod. success rato voter utlzaton balanced, DMA2 balanced, slcng unbalanced, DMA2 unbalanced, slcng Fgure 7. The success ratos of slcng and DMA2 algorthms measured n Experment 3 Our last experment s to nvestgate the effect of task perods to the schedulablty of these two algorthms. In the prevous experments, the task perods are unformly dstrbuted between 5 to 5. Thus, the maxmal perod s lkely several tmes more than the mnmal perod. If we change the range n whch the task perods are dstrbuted, t may get dffcult to fnd a feasble schedule as that the most dffcult schedulng condton occurs when all perods are less than the twce of the mnmal perod [3]. We assume that the mean of task perods s 275. The range s set to 275- task_perod_range to 275+task_perod_range, where

8 success rato votng cycle task_perod_range vares from 9 to 225. The experment also assumes that the utlzaton of the processng node s equal to 5. Two sets of curves are plotted n Fgure 9 to represent the cases that the voter utlzaton s equal to.7 and.75, respectvely. They llustrate the property that the schedulng algorthms result n dfferent success ratos as the range of task perods changes. For nstance, wth a voter utlzaton of.75, the success ratos of varous perod dstrbutons may drop more than 5% and 4% under the slcng approach and DMA2 algorthm, respectvely. The results from the 5 experments clearly ndcate that DMA2 algorthm outperforms the slcng approach n determnng feasble schedules, and s robust under varous condtons. In fact, the experments show that feasble schedules can be found even f the utlzaton at the processng node and the voter closes to the theoretcal bound of rate-monotonc algorthm. Ths fndng s nterestng snce, n order to make a schedule feasble, we expect there s a need to lmt the utlzaton at the two processng stages such that the sum of the experenced delays at both processng stages s less than the task deadlne. However, when DMA2 algorthm s able to shuffle the prortes at the two stages, no task needs to undergo long delays at both stages and feasble schedules can be obtaned when the utlzaton s not hgh. 5. CONCLUSION 5, DMA2 5, slcng.65, DMA2.65, slcng Fgure 8. The success ratos of slcng and DMA2 algorthms measured n Experment 4 In ths paper, we have proposed an effectve schedulng mechansm that can be ncorporated n fault-tolerant IMA systems. The emphases are placed n the ssues of desgnng fault reslence at system level and schedulng consstency checkng operatons along wth task executon. To fnd feasble solutons for fxed prorty schedulng, we have examnes two prorty assgnment schemes and evaluated success rato.6 ther correspondng performance through a set of experments. In addton, the measured data ndcates a sutable desgn space n terms of the utlzaton at each processng node and the RMS unt. Ths result wll be extremely useful n preparng system requrements n the early stage of the desgn process. To contnue the research work n the ntegrated mechansms for schedulng and fault tolerance, we plan to nvestgate the schedulng algorthms n the APEX (applcaton/executve nterface) envronment. The APEX envronment calls for a parttonng approach to set up fault contanment [2]. The temporal partton lmts the executon of a task set to specfc partton wndows of a major tme frame. Apparently, ths determnstc approach favors a cyclc task schedulng whch n turn results n a determnstc checkng process. In ths context, we need to look nto effcent approaches of sharng the checkng mechansms among parttons, and to schedule parttons, the task set wthn each partton, and consstency checkng operatons. REFERENCES task_perod_range.7, DMA2.7, slcng.75, DMA2.75, slcng Fgure 9. The success ratos of slcng and DMA2 algorthms measured n Experment 5 [] Desgn gudance for ntegrated modular avoncs, ARINC Report 65, Aeronautcal Rado Inc., Annapols, Maryland, Oct. 99. [2] Avoncs applcaton software standard nterface, ARINC Specfcaton 653, Aeronautcal Rado Inc., Annapols, Maryland, Jan [3] N. Audsley, A. Burns, M. Rchardson, and A. Wellngs, Hard real-tme schedulng: the deadlne-monotonc approach, Eghth IEEE Workshop on Real-tme Operatng Systems and Software, 99, pp

9 [4] R. Bettat and J. W.-S. Lu, End-to-end schedulng to meet deadlnes n dstrbuted systems, Proc. Of the IEEE Int l Conf. on Dstrbuted Computng Systems, 992, pp [5] R. P. G. Collnson, Introducton to Avoncs, Chapman & Hall Publsher, 996. [6] O. Gonzalez, H. Shrkumar, K. Ramamrtham, and J. A. Stankovc, Adaptve fault tolerance and graceful degradaton under dynamc hard real-tme schedulng, To appear n Proc. IEEE Real-Tme Systems Symposum, Dec [7] J. Jonsson and K. Shn, Deadlne assgnment n dstrbuted hard real-tme systems wth relaxed localty constrants, Proc. Of the IEEE Int l Conf. on Dstrbuted Computng Systems, 997, pp [8] R. Keckhafer, C. Walter, A. Fnn, and P. Thambdura, The MAFT archtecture for dstrbuted fault tolerance, IEEE Trans. on Computers, Vol. 37, No. 4, 988, pp [9] C. M. Krshna and G. Shn, On schedulng tasks wth a quck recovery from falure, IEEE Trans. on Computers, Vol. 35, No. 5, 986, pp [] J. H. Lala and R. E. Harper, Archtectural prncples for safety-crtcal real-tme applcatons, Proceedngs of the IEEE, Vol. 82, No., Jan. 994, pp [] L. Lamport, R. Shostak, and M. Pease, The Byzantne Generals Algorthm, ACM Trans. `Programmng Languages and Systems, No. 4, 982, pp [2] J. Leung and J. Whtehead, On the complexty of fxedprorty schedulng of perodc real-tme tasks, Performance Evaluaton, Vol. 2, No. 4, 982, pp [3] C. L. Lu and J. W. Layland, Schedulng algorthms for multprogrammng n hard real tme envronment, J. Assoc. Comput. Mach., Vol. 2, No., 973, pp [4] C. D. Locke, Software archtecture for hard real-tme applcatons: Cyclc executves vs. Fxed prorty executves, The Journal of Real-Tme Systems, Vol. 4, No., 992, pp [5] M. D Natale and J. Stankovc, Dynamc end-to-end guarantees n dstrbuted real-tme systems, Proc. IEEE Real-Tme Systems Symposum, Dec. 994, pp [6] Y. Oh and S. Son, Enhancng fault-tolerance n ratemonotonc schedulng, Real-tme Systems, Vol. 7, 994, pp [7] M Saksena and S. Hong, An engneerng approach to decomposng end-to-end delays on a dstrbuted realtme system, Proc. of the IEEE Workshop on Parallel and Dstrbuted Real-tme Systems, 996, pp [8] K. Tndell, A. Burns, and A. Wellngs, An extendble approach for analyzng fxed prorty hard real-tme tasks, Real-tme Systems, Vol. 6, 994, pp [9] C. J. Walter, Evaluaton and desgn of an ultra-relable dstrbuted archtecture for fault tolerance, IEEE Trans. on Relablty, Vol. 39, No. 4, 99, pp Yann-Hang Lee s an assocate professor wth the Computer and Informaton Scence and Engneerng Department, Unversty of Florda. Hs research nterests nclude real-tme systems, fault-tolerant computng, communcaton networks, computer archtecture, and performance evaluaton. He co-edted two specal ssues n Real-Tme Systems n IEEE Computer (May 992) and IEEE Proceedngs (Jan. 994), and co-chared the Real- Tme system Symposum, 996. He receved hs Ph.D. degree n Computer, Informaton, and Control Engneerng from the Unversty of Mchgan, Ann Arbor, MI, n 984. Mohamed F. Youns receved the B.S. degree n computer scence and the M.S. n engneerng mathematcs from Alexandra Unversty n Egypt n 987 and 992, respectvely. In 996, he receved hs Ph.D. n computer scence from New Jersey Insttute of Technology. Dr. Youns s currently a research scentst wth the AlledSngal Advanced Systems Technology Group, Columba MD, where he s leadng multple projects for buldng ntegrated fault tolerant avoncs. Hs techncal nterests are n fault tolerant computng, system ntegraton, real-tme dstrbuted systems and comple-tme analyss. Jeffrey Zhou, Sr. Manager of AlledSgnal Advanced Systems Technology Group, receved hs B.S. degree n 98 from Shangha Scence and Technology Unversty, M.E. degree n 983 from Shangha Jao-Tong Unversty, and Ph.D degree n 989 from the Unversty of Florda. Hs major s computer engneerng specalzed n real-tme and fault-tolerant computng systems, complex computerbased systems and avoncs systems. He joned AlledSgnal Aerospace n 99 where he was nvolved n the development of the real-tme engne smulator FAST. He also partcpated n the Shuttle Man Engne Control and Health Montorng System (EC&HMS) program funded by NASA and was a key contrbutor for the development of the Real-Tme Executve Module (RTEM), a real-tme and faulttolerant operatng system for the EC&HMS program. Snce 996, he has been leadng an AlledSgnal team n developng the Redundancy Management System whch s a key component for the Vehcle and Msson Computer currently under the development for NASA s X-33 space launch vehcle. Dr. Zhou has served dfferent commttees and has numerous publcatons n hs research feld.