A Loadabl Task Excution Rcordr for Hirarchical Schduling in Linux Mikal Åsbrg and Thomas Nolt MRTC/Mälardaln Univrsity PO Box 883, SE-721 23, Västrås, Swdn {mikalasbrg,thomasnolt@mdhs Shinpi Kato Carngi Mllon Univrsity Dpartmnt of Elctrical and Computr Enginring Shinpi@ccmudu Abstract This papr prsnts a Hirarchical Schduling Framwork (HSF) rcordr for Linux-basd oprating systms Th HSF rcordr is a loadabl krnl modul that is capabl of rcording tasks and srvrs without rquiring any krnl modifications Hnc, it complis with th rliability and stability rquirmnts in th ara of mbddd systms whr provn vrsions of Linux ar prfrrd Th rcordr is built upon th loadabl ral-tim schdulr framwork RESCH (REal-tim SCHdulr) W valuat our rcordr by comparing th ovrhad of this solution against anothr (patchd) rcordr Also, th tracing accuracy of th HSF rcordr is tstd by running a mdia-procssing task togthr with priodic ral-tim Linux tasks in combination with srvrs Th tsts ar rcordd with th HSF rcordr, and th Ftrac rcordr, in ordr to show th corrctnss of th xprimnts and th HSF rcordr itslf Systm CPU Global schdulr Intrfac Intrfac Intrfac Local schdulr Local schdulr Srvr Task Srvr Indx Trms ral-tim systms, hirarchical schduling, rplay dbugging, xcution visualization Fig 1 Hirarchical Schduling Framwork I INTRODUCTION Introduction Th rsarch that w conduct is primarily focusd on th dvlopmnt of hirarchical schduling [1], [2], [3] Our prvious and ongoing work within hirarchical schduling includs practical (implmntation) aspcts of this kind of schduling [4], [5], th applicability/usag [6], [7] of it, as wll as applying formal mthods [8] on it In srvrbasd schduling (th prdcssor of hirarchical schduling), tasks (a squnc of instructions) ar only allowd to xcut whnvr thir srvr (th virtual task which thy blong to) runs Th srvr itslf xcuts according to som schduling schm (global schduling) which is indpndnt of th tasks Th advantag is that it can improv th rspons tim (th tim lngth btwn task activation and compltion) of vnt triggrd tasks, and still kp th schduling dtrministic sinc th srvr schduling paramtrs ar known and includd in th schdulability analysis Furthr, introducing a schdulr within ach srvr (local schduling) maks it mor gnral sinc it supports tim triggrd tasks as wll This can b gnralizd vn furthr by rprsnting a task as a st of tasks togthr with a schdulr Whn w hav sparat schduling insid a srvr, i both global and local schduling, thn w rfr to hirarchical schduling or a Hirarchical Schduling Framwork (HSF), this is illustratd in Figur 1 Hirarchical schduling has svral advantags, bsids improving rspons tim of vnt triggrd tasks It nabls paralll dvlopmnt of systm parts (subsystms), simplifis intgration of subsystms (analysis), supports runtim tmporal partitioning and saf xcution of tasks tc Howvr, xcpt for ARINC653 [9], [10] compliant oprating systms that ar commonly found in avionics applications, hirarchical schduling is rarly an intgratd part of an oprating systm (OS) Indd, thr is a nd to dvlop/implmnt nw schduling algorithms, such as hirarchical schduling, in th ara of mbddd and/or ral-tim systms [6] A motivation of this can b found in our schduling xampl in th valuation (Sction IV), whr w lt a mdia-procssing task (which dos a movi playback) xcut within a srvr (srvr-basd schduling) Th srvr xcuts with a crtain frquncy, giving (guaranting) th mdia task an vn amount of CPU powr which improvs th playback quality of th movi, vn though it xcuts among othr tim triggrd tasks Th mdia task has an unknown xcution pattrn, i, th rlass ar undfind Still, w gt prdictability (sinc w can analyz th bhavior) from both th mdia tasks point of viw, and th tim triggrd tasks Also, w avoid (tmporal) intrfrnc at runtim, maning that w gt a saf xcution nvironmnt for th tasks bcaus tmporal rrors do not propagat btwn th mdia task and th tim triggrd tasks From a practical point of viw, it is an advantag if hirarchical schduling can b implmntd asily/fficintly and without modifying th krnl Th lattr maks it asir
for both dvloprs and usrs sinc thr is no nd to maintain/apply krnl modifications vry tim th krnl is rplacd or updatd Morovr, kping th schdulr isolatd in a krnl modul, without modifying th krnl, simplifis dbugging and potntial crtification of its corrctnss (componnt-basd dvlopmnt advantags) W s that th RESCH schduling framwork [11] is usful bcaus it has th advantags mntiond, sinc it dos not nd any krnl modifications Also, it maks schdulr dvlopmnt asir bcaus it simplifis th schduling intrfac to th usr and it supports th dvlopmnt of schdulrs (plugins) which run as indpndnt krnl moduls Howvr, whil th dvlopmnt of schdulrs ar simplifid with this framwork, it lacks support for dbugging th schdulrs That is why w hav dvlopd a HSF rcordr, which can asily b pluggd in to a srvr-basd/hirarchical schdulr, dvlopd in RESCH Th rcordr dos not rquir krnl modifications and it is of cours also suitabl for analyzing th runtim bhavior of tasks/srvrs sinc th rcordd trac can b visualizd graphically with th Tracalyzr [12] or Grasp [13] visualization tools In turn, ths tools can prsnt valuabl trac data such as xcution- and rspons-tim Th HSF rcordr is abl to rcord th following schduling vnts during run-tim: 1) Th tim instanc whn a task/srvr is rlasd (vn though it might not start to xcut) 2) Th tim instanc whn a task/srvr starts to xcut 3) Whn thr is a task/srvr contxt switch, th rcordr distinguishs btwn prmption and non-prmption 4) Th tim instanc whn a task/srvr finishs its xcution Contribution Th main contributions of this papr ar: 1) W hav implmntd a task/srvr rcordr with th us of RESCH, i, it dos not rquir any krnl modifications Th rcordr nabls dbugging at task and srvr lvl, in Linux basd ral-tim/gnral-purpos OSs 2) W hav valuatd our HSF rcordr by implmnting yt anothr rcordr (Sction II-C), using th tchniqu prsntd in [14], and compard th ovrhad of this rcordr, with th HSF rcordr 3) W hav tstd our rcordr by running a mdiaprocssing task togthr with tim triggrd tasks and srvrs Th xampl shows how th playback quality gts improvd by putting th mdia-procssing task in a srvr Th HSF rcordr is usd in this xampl to dbug and display th runtim bhavior Outlin Th outlin of this papr is as follows: Sction II prsnts prliminary background, in Sction III w dscrib th HSF-rcordr implmntation Sction IV valuats th ovrhad and tracing accuracy of th HSF rcordr Sction V prsnts rlatd work, and finally, Sction VI concluds II PRELIMINARIES A Systm modl W assum fixd-priority, prmptiv, schduling of priodic tasks, according to th priodic task modl [15] A task i is prsumd to hav th following paramtrs, T i, WCET i,d i,pr i, whr th priod T i rprsnts th frquncy in which th task is rlasd for xcution, WCET i is th worst cas xcution tim of th task, th rlativ dadlin D i (within th priod) is whn th task must complt its xcution (RESCH monitors this) and pr i is th task priority (lowr valu rprsnts highr priority) Also, all tasks ar assumd to xcut indpndntly of achothr and on th sam cor, i, singl cor Thsrvrsaralsoassumdtohavfixdpriorityand thy ar schduld prmptivly and priodic A srvr j has similar paramtrs as tasks, i P j,q j,pr j,whrp j is th srvr priod, Q j is dfind as a budgt (which is th tim givn at ach priod P j to th tasks within th srvr) and pr j is th srvr priority (lowr valu rprsnts highr priority) B RESCH W hav bn dvloping a loadabl ral-tim schdulr framwork, RESCH [11], dsignd to work with th POSIXcompliant SCHED FIFO schduling policy implmntation RESCH has prviously bn usd as th basis for anothr schdulr calld AIRS [16] - a multi-cor CPU schdulr for intractiv ral-tim applications As mntiond prviously, RESCH is a modification-fr schduling framwork for Linux It supports priodic tasks which can b schduld in a fixd-priority prmptiv mannr RESCH is simply composd of xtrnal krnl moduls and usr-spac libraris for asy installation It givs both an intrfac to th usrs in usr spac (g a task spcific intrfac lik rtwaitforpriod()) as wll as in th krnl spac Th krnl spac API (Application Programming Intrfac) has th intrfac shown blow: 1) task run plugin( ) 2) task xit plugin( ) 3) job rlas plugin( ) 4) job complt plugin( ) Ths functions can b implmntd by a RESCH plugin (Figur 3), i, a krnl modul that has accss to th RESCH krnl API Ths functions ar calld in th RESCH cor at crtain vnts which ar illustratd in Figur 2 Functions 1) and 2) ar xcutd vry tim a task rgistrs/unrgistrs to RESCH With rgistr w man that th task dos a RESCH API call, transforming it to a RESCH task, which crats a RESCH TCB (Task Control Block) and puts it in th RESCH rady-quu tc A RESCH TCB has, among othr ral-tim spcific data, a rfrnc to its corrsponding Linux task TCB (taskstruct) Onc th task is rgistrd in RESCH, it will b schduld priodically (and prmptiv) according to its ral-tim priority Th primitivs 3) and 4) ar calld whnvr a RESCH task is rlasd for xcution or whn it has finishd its xcution Th plugins gt ths schduling notifications and can thrby affct schduling, trac tasks tc Th plugin notifications ar shown in Figur 2 Whn a task notifis RESCH that it has finishd its xcution in its currnt priod, th RESCH cor will inform any plugin about this vnt and st a timr for th rlas of th tasks nxt priod
As a last stp, it will call th Linux krnl to r-schdul anothr task Th nxt running task might b a RESCH task or any othr Linux procss RESCH task usr lvl krnl lvl RESCH cor RESCH plugin intrrupt contxt krnl-thrad contxt Linux krnl jobcompltplugin() rtwaitforpriod() Fig 2 modtimr() schdul() r tim RESCH control flow j o b r l a s p l u g i n ( ) w a k u p p r o c s ( ) () to h itc s w Whn th krnl rsponds to th corrsponding timout (task rlas), a handlr in th RESCH cor will gt notifid about this vnt Th handlr will notify any plugin about th task rlas and thn call th krnl to wak up th task Application Application Linux Krnl Schdulr Ral-tim task Timrs Fig 3 Application RESCH library RESCH cor RESCH task HSF plugin RESCH framwork In Linux, sinc krnl vrsion 2623 (Octobr of 2007), tasks can b ithr a fair or a ral-tim task Th lattr group has highr priority (0-99 whr 0 is highst) than fair tasks (100-140) A task that rgistrs to RESCH is automatically transformd to a ral-tim task RESCH is rsponsibl for rlasing tasks, and tasks rgistrd to RESCH must notify whn thy hav finishd thir xcution in th currnt priod In this way, RESCH can control th schduling RESCH uss an absolut-tim clock, i, it dos not wrap around Also, rlas tims ar stord as absolut valus, so rlas pattrns ar xact Th cost of having a modification-fr solution is that RESCH can only s schduling vnts rlatd to its rgistrd tasks Ral-tim tasks with highr priority than RESCH tasks (i tasks that ar not rgistrd in RESCH) can thrby Krnl Spac Usr Spac intrfr with RESCH tasks without th RESCH cor bing abl to dtct it A simpl solution to this problm is to schdul all ral-tim tasks with th RESCH framwork C Task-switch hook patch Our prvious work [14] includs an implmntation of a taskswitchhook function (Figur 4), rsiding in a krnl modul, which is calld by th Linux schdulr at vry schdulr tick In this way, it is possibl to rcord task schduling vnts This solution rquirs modification of two cod lins in two sparat krnl sourc fils (schdrtc and schdfairc) Th modification of fil schdrtc is illustratd in Figur 4 (a similar chang is don in schdfairc) Linux has (sinc krnl vrsion 2623) two schduling classs, namly th fair and th ral-tim schduling classs Whn a nw task should b rlasd, th Linux schdulr itrats through its schduling classs (first th ral-tim class, scondly th fair class) in ordr to find th nxt task to rlas Th modification (Figur 4) maks it possibl to r-dirct a schduling class function pointr picknxttask to point to a usr dfind function (i, our function taskswitchhook), instad of th original function picknxttaskrt Our function will instad point to picknxttaskrt, in this way, w do not altr th krnl functionality othr than xcuting our function taskswitchhook (which contains usr dfind cod) just bfor picknxttaskrt starts to xcut Our function (hook) can b insrtd and rmovd during runtim A task rcordr can asily b implmntd (as a krnl modul) and us th taskswitchhook function to rgistr task contxt switchs, howvr, th krnl must b modifid Static const struct schdclass rtschdclass = { patch picknxttask = picknxttaskrt, schdrtc r-compil krnl Fig 4 Hook patch III IMPLEMENTATION Linux krnl rtschdclass picknxttask picknxttaskrt bfor Loadabl krnl modul Excution tim monitor aftr taskswitchhook Th implmntation of th HSF rcordr is basd on th schdulr plugin HSF which in turn is basd on th schduling framwork RESCH Figur 5 shows that th HSF schdulr uss primitivs xportd by RESCH and xports ths, as wll as srvr spcific primitivs, to th rcordr Ths primitivs ar usd to rgistr srvr and task contxt switchs Not that th flxibl structur allows for nw schdulr plugins to rus th rcordr as long as thy xport th sam primitivs
RESCH cor Fig 5 HSF rcordr plugin srvrcomplt srvrrlas HSF plugin jobcompltplugin jobrlasplugin HSF-rcordr plugin For th rcording to work corrctly, it is assumd that no highr priority ral-tim Linux tasks, which ar not rgistrd by RESCH, ar xcutd Th currnt implmntation dos not support load balancing (a function in Linux that migrats tasks to othr CPUs basd on load) This is bcaus th RESCH schdulr cannot dtct task migrations mad by th Linux schdulr Each rcordd vnt has 2 rcords: ID of th nxt task/srvr to xcut Timstamp of th vnt Th ID of th nxt task/srvr is usd to calculat th prvious task/srvr Th 4 hook functions (Figur 5) ar usd by th rcordr to sav schduling rcords in mmory (this is a circular implmntation) Th rcordr flushs th rcordd data to disk whn it gts unloadd by th usr Th rcording format can asily b convrtd to match any visualization tool W hav succssfully convrtd th format to fit with th Tracalyzr [12] and th Grasp [13] visualization tools W us Grasp in th valuation (Sction IV) in ordr to visualiz th trac of th HSF rcordr sinc it also supports hirarchical schduling in addition to rgular (flat) schduling Figur 6 illustrats how th HSF rcordr gts triggrd As can b sn, th HSF schdulr gts triggrd by its own timrs as wll as by th RESCH cor Th HSF schdulr rlays task rlass and compltions to th HSF rcordr whn th HSF schdulr itslf is triggrd by th RESCH cor Whnvr th HSF schdulr gts triggrd by a timr, it automatically calls its srvr rlas/compltion plugin, which in turn starts th rcordr Th figur also shows that th HSF rcordr xcuts mostly in intrrupt contxt This maks it lss xpnsiv in trms of contxt-switch ovrhad IV EVALUATION W hav valuatd our HSF rcordr by rcording a st of tasks and srvrs (Tabl I and II) In our xampl, task rttask1 blongs to srvr Srvr0, rttask2 and rttask3 dos not blong to any srvr whil rttask4 blong to srvr Srvr1 and rttask5 to Srvr2 Th valuation shows two aspcts: th masurd ovrhad (sction IV-A) of th HSF rcordr compard to th patchd rcordr [14], and an xampl of how th Quality of Srvic (QoS) of multimdia tasks can b improvd with hirarchical schduling as wll as how our HSF rcordr can assist in this work (sction IV-B) In th multimdia xampl w usd our HSF rcordr and th Ftrac [17] rcordr During our xprimnts, th two rcordrs wr rcording th tasks and srvrs simultanously Task-nam T WCET D Prio Srvr rt task1 80 9 80 0 Srvr0 rt task2 200 75 200 1 - rt task3 105 9 105 2 - rt task4 500 100 500 3 Srvr1 rt task5 - - - 4 Srvr2 TABLE I TASKS USED IN THE EVALUATION Srvr-nam P Q Prio Srvr0 40 6 1 Srvr1 90 23 2 Srvr2 25 8 0 TABLE II SERVERS USED IN THE EVALUATION A Ovrhad masurmnts In ordr to stimat th ovrhad impact, w masurd th xcution tim of th patchd and th HSF rcordr, running simultanously and rcording th sam trac W also notd th amount of data (in kilo byts) that th two rcordrs producd (out of curiosity w also masurd Ftrac) W implmntd an optimizd vrsion of th patchd rcordr, Patch (Tabl III) so that it only savd rcordd data of th tasks that w wr intrstd in rcording In this way, th comparison to th HSF rcordr bcam fair sinc it is only triggrd at task/srvr vnts rlatd to th tasks/srvrs w ar intrstd in rcording (RESCH rlatd task and srvrs) Rcordr Exc tim (µs) Rc data (KB) HSF 45 105 Patch 1246 174 Ftrac - 8886 TABLE III MEASUREMENTS OF THE RECORDERS Th valus listd in Tabl III ar th avrag masurd valus of 10 runs and th rcordrs rcordd about 4 sconds at ach run W s that th HSF rcordr has a ratio of 43 µs/kb whil Patch has 716 µs/kb Th conclusion is that th HSF rcordr producs lss ovrhad than th patchd rcordr, comparing th xcution-tim/data ratio Th small amount of rcordd data compard to Ftrac suggsts that our rcordr might b a bttr option if th usr is only intrstd in a subst of tasks Having a small amount of ovrhad is attractiv for rcordrs sinc thy can rmain activ in shippd
RESCH task usr lvl krnl lvl rtwaitforpriod() RESCH srvr rlas modtimr() stop rlas RESCH cor RESCH HSF HSF rcordr srvrrlas() modtimr() jobcompltplugin() intrrupt contxt krnl-thrad contxt schdul() srvrcomplt() timr jobrlasplugin() wakupprocss() switchto() Linux krnl Fig 6 HSF rcordr control flow products (without wasting too much rsourcs), and thrby liminating th prob ffct B Multimdia xampl Th purpos of this xampl is to show how a multimdia task (procssing a movi) can bnfit from hirarchical schduling in such a way that th movi playback runs mor smoothly Th HSF schdulr has nvr bn valuatd (and dbuggd) as proprly as th xampl w ar about to show, so this is a good cas study for th HSF rcordr W run th multimdia task in diffrnt stups (with and without hirarchical schduling), and masur its prformanc Th hirarchical schduling givs th multimdia task an vn amount of CPU powr, and thrby improvs th movi playback Not that all of this is don, including th rcording, without modifying th krnl Th HSF rcordr plays a ky rol sinc knowldg of th schduling bhavior is important in ordr for th rsult of this valuation to b corrct For xampl, th rcordr shows that th tasks and srvrs gt th amount of CPU that w spcify (i, that tasks run within thir srvrs) and that th tasks/srvrs run according to th spcifid frquncy and WCET /Q During our xprimnts, th rcording showd that th HSF cannot kp tasks within thir srvr if thy do a lot of blocking (g multimdia tasks) Thrfor, w st lowst priority to th multimdia task and add idl tasks with highr priority than th multimdia task This will kp th multimdia task within its srvr, thrby guaranting th uppr limit on its rsourc supply This was confirmd by th rcording of our HSF rcordr A scond rcordr (Ftrac) was also usd in ordr to show that th HSF rcordr rcordd corrctly W usd th Grasp tool [13] to visualiz our rcordings (for both th HSF rcordr and Ftrac), sinc it can display both tasks and srvrs In this xampl, w hav 5 tasks, i, rttask1 to rttask5 (Tabl I) Tasks rttask1 to rttask4 ar dummy tasks, i, thy just loop (rttask1 in Figur 7) rttask5 dos a movi playback, its task body is shown in Figur 7 // rt task1 int main(int argc, char *argv[ ]) { for (i = 0; i < NR OF JOBS; i++) { for (j = 0; j < USEC UNIT; j++) { if (!rt wait for priod()) { printf( dadlin is missd!\n ); // rt task5 int main (int argc, char *argv[ ]) { libvlc mdia playr play(playr); Fig 7 Task bodis rttask5 usd th libvlc 1 for movi playback and th library itslf has th nic proprty that th movi procssing can b xcutd by a task running in ral-tim mod W xcutd rttask5 in 4 diffrnt stups: 1 libvlc http://wikividolanorg/libvlc
1) rttask5 with lowst priority and tasks rttask1 to rttask4 with priority ordr as in Tabl I 2) rttask5 with mdium priority (in btwn rttask2 and rttask3) andtasksrttask1 to rttask4 with priority ordr as in Tabl I 3) rttask5 with highst priority and tasks rttask1 to rttask4 with priority ordr as in Tabl I 4) rttask5 xcutd in srvr Srvr2, and rttask1 and rttask4 in srvr Srvr0 and Srvr1 rspctivly (rttask2 and rttask3 was not includd in this stup) Givn ths 4 stups, task rttask5 will gt diffrnt amount/distribution of CPU powr and th procssing of movi imags (frams) will thrfor also b affctd Th movi procssing is masurd in amount of producd frams pr scond (FPS) Th CPU utilization (prcantag of CPU tim) of task rttask5 is shown in Tabl IV as wll as th fram rat of which rttask5 is procssing a movi W masurd th FPS by timstamping th bginning and nd of th movi playback systm call and dividing th amount of frams of th movi with th masurd tim Th amount of frams is 91 and this valu was gnratd by Mplayr 2 (using th bnchmark flag) It is important to not that th CPU utilization givn in Tabl IV is th availabl CPU tim, it dos not man that task rttask5 uss this CPU tim Th FPS valus may not considrd to b 100% accurat, but it shows th approximat fficincy For xampl, running rttask5 with 100% CPU should of cours not giv wors FPS valu than running it with 32% CPU Ths valus ar of cours affctd by ovrhad from th Linux krnl tc W ran th th xprimnts on an Intl Pntium Dual- Cor (E5300 2,6GHz) platform, quippd with a Linux krnl vrsion 26319, running with load balancing disabld Th rcordd tasks (and srvrs) ran on th sam cor, i, all tasks wr migratd to CPU #0 at initialization phas Stup CPU utilization (%) FPS Lowst prio 2265 2255 Mdium prio 5125 2357 Highst prio 100 2548 HSF 32 2566 TABLE IV FPS OF TASK rttask5 Th conclusion basd on Tabl IV is that th distribution of CPU powr influncs th fram frquncy a lot and that utilization alon is not sufficint for dtrmining this For xampl, giving task rttask5 5125% of th CPU producs lss FPS than giving it 32% Th 32% CPU is guarantd (no mor no lss) and it is distributd vnly as can b sn by th rcording of HSF rcordr in Figur 8 (visualizd with th Grasp tool [13]) 2 Mplayr http://wwwmplayrhqhu/dsign7/nwshtml Apparntly, (during our xprimnts) task rttask5 must hav bn activ whn othr highr priority tasks wr occupying th CPU, thrby tmporarly gtting lss than 5125% CPU This is not th cas whn running th multimdia task in its srvr, sinc it is always supplid 32% Srvr2 Srvr0 Srvr1 Srvr3 idl rttask5 s3idl s1idl rttask4 s0idl rttask1 8 4 0 6 3 0 23 12 0 2250 1125 0 0 50 100 150 200 250 300 350 Fig 8 Tasks and srvrs rcordd with th HSF rcordr Figur 9 shows th sam trac as in Figur 8, but rcordd with th Ftrac rcordr As can b sn, th HSF rcordr rcords corrctly, also, it shows that task rttask5 dos not consum CPU continuously (i, it blocks oftn) idl rttask5 s3idl s1idl rttask4 s0idl rttask1 0 50 100 150 200 250 300 350 Fig 9 Tasks rcordd with th Ftrac rcordr Figur 10 shows a trac by our HSF rcordr whn task rttask5 was running with lowst priority, without HSF As can b sn, th CPU availability for task rttask5 is highly dpndant on whn highr priority tasks xcut Our xampl shows that it is difficult to fin tun th CPU supply for a multimdia task, i, w can only do it by changing th priority of th task sinc it is not priodic Howvr, it is possibl to do tuning by stting srvr priod, budgt and priority, whn using HSF Th main contribution of this xampl is th trac (Figur 8) mad by th HSF rcordr which shows th corrctnss of th CPU distribution, mad by HSF, to ral-tim tasks (with mdia procssing) W hav also tstd th corrctnss of th HSF rcordr by comparing
0 50 100 150 200 250 300 350 idl rttask5 rttask4 rttask3 rttask2 rttask1 Fig 10 Tasks rcordd with th HSF rcordr its trac rsults with th Ftrac rcordr, i, th trac in Figur 8 is idntical with th trac in Figur 9, which shows that it rcords corrctly Also, th trac in Figur 8 shows th amount of unusd CPU tim (slack tim) at both srvr lvl and within ach srvr, sinc th diffrnt idl tasks rprsnt this For xampl, srvr Srvr3 (which has lowst priority) and its task s3idl rprsnt slack tim at srvr lvl, whil s0idl rprsnt unusd tim in Srvr0 Th conclusion is that th HSF rcordr can b a good tool for dbugging hirarchical schdulrs in RESCH, sinc it rcords accuratly and with low ovrhad Furthr, this xampl shows that our (HSF) rcordr and schdulr rcords (and schduls) corrctly, vn though w do not modify th krnl V RELATED WORK Th ida of our solution is basd on th rplay dbugging approach [18], which rcords systm vnts onlin and rplays thm offlin In latr work [19], th rplay dbugging has bn xtndd to b compilr- and OS-indpndnt Whil th rplay dbugging works with off-th-shlf compilrs for application-lvl dbugging, our solution is slf-containd softwar using Grasp [13] for OS-lvl dbugging, and it is primarily focusd on ral-tim schdulr dbugging Th SCHED DEADLINE projct [20], which is in charg of th EDF schdulr implmntation for Linux, has usd th schd switch tracr providd by th Ftrac toolkit [17] to output th rcordings of contxt switchs Th output logs ar latr convrtd to th Valu Chang Dump (VCD) format so that GtkWav can visualiz th task xcution tracs Th trac can of cours b convrtd to othr trac formats, such as th Tracalyzr [12] or th Grasp [13] format Givn that Ftrac is supportd by th Linux community, it is rasonabl to us this toolkit to trac task xcutions for krnl dbugging, but it is ddicatd to th Linux krnl, so it is not ncssarily suitabl for ral-tim schdulr dbugging in gnral For instanc, schd switch dos not catch job rlass, howvr, contxt switchs ar prcisly tracd, and it can distinguish btwn task compltions and task prmptions Our solution is mor flxibl and intgratd in that it is availabl not only for th Linux krnl, but also for othr OSs, onc th RESCH framwork is portd to othr platforms Our prvious work [21] includs a simpl task rcordr in Linux (basd on RESCH) which supports th Tracalyzr [12] and th Grasp [13] format Furthr, w hav also implmntd a task rcordr [14] (in Linux) which is abl to rcord all task schduling vnts, but it rquirs modifications to th krnl DTrac [22], SystmTrap [23], LTT [24], and LTTng [25] ar advancd tools for OS dbugging Thy ar orintd for tracing ntir krnl vnts, so it is rquird that th dvloprs undrstand how to us thm Manwhil, our solution is mor simplifid by focusing on ral-tim schdulr dbugging, and it is vry asy to us in practic Ral-Tim Application Intrfac for Linux (RTAI) [26] is a collction of loadabl krnl moduls and a krnl patch which togthr provids a rich ral-tim API to th usr It givs th possibility to add/dlt hooks for vry task-start, task-switch and task-dlt Ths hooks giv th possibility to monitor task xcution in a dtaild lvl Tracalyzr [12] is a visualization and analysis tool for mbddd systms It can visualiz task tracs as wll as task communication Rcordrs implmntd in th OSs VxWorks, OSE, Rubus and RTXC support th Tracalyzr format VI CONCLUSION W hav prsntd th implmntation and valuation of a task/srvr rcordr basd on th RESCH (REal-tim SCHdulr) framwork in Linux RESCH is a schduling framwork for Linux which support schdulr plugins, i, multi-, unicor, flat-, srvr-basd-schduling tc Our rcordr implmntation is a plugin on top of an alrady xisting hirarchical schdulr plugin calld HSF (Hirarchical Schduling Framwork) This framwork supports fixd-priority prmptiv schduling of srvrs as wll as tasks Th HSF rcordr uss schduling primitivs supportd by RESCH itslf, and HSF, in ordr to rcord schduling vnts Th RESCH framwork, th HSF schdulr plugin as wll as our HSF rcordr rquir no modification of th krnl and this is th main contribution of this approach To th bst of our knowldg, this is th first attmpt to prform task tracing (within hirarchical schduling) in Linux, without krnl modifications Th valuation of th HSF rcordr includs two parts: Ovrhad comparison against an optimizd vrsion of our prviously implmntd task-switch patch [14] Th corrctnss of th HSF rcordr (as wll as th HSF schdulr) is tstd with a mdia procssing xampl Th tracing capability and accuracy of th HSF rcordr is compard against th main-lin Linux rcordr Ftrac [17] Our HSF rcordr producs vry low ovrhad, in trms of CPU consumption, compard to th task-switch patch Th amount of rcordd data is also much smallr than Ftrac, suggsting that th HSF rcordr could b a bttr choic if only a subst of Linux tasks is of intrst to monitor Th mdia-procssing xampl shows 5 ral-tim tasks running with, and without srvrs, i, with th HSF schdulr activatd and with only RESCH In th xampl, w show that on of th tasks (which is procssing a movi) producs highr fram rat with thortically lowr CPU utilization (using th HSF schdulr) than with highr CPU utilization
(using only RESCH) Th rason for this is that HSF givs th mdia-procssing task bttr CPU rsourc distribution In this xampl, th HSF rcordr contributs by showing that th mdia task uss only its allocatd CPU rsourc, thrby showing that th xampl is corrct It also shows a waknss with th HSF schdulr in that it has problms with kping mdia tasks (and similar tasks which blocks oftn) within its srvr Howvr, non-blocking ral-tim tasks ar shown to b proprly containd insid thir srvrs All tracs from th HSF rcordr, in this xampl, ar don in paralll with th Ftrac rcordr, thrby showing th accuracy (and corrctnss) of our HSF rcordr Th conclusion is that th HSF rcordr could b a good tool for dbugging hirarchical schdulrs in RESCH Th rcordr can, togthr with a visualization tool, such as Grasp [13], visualiz th xcution of tasks and srvrs as wll as display worst-cas, bst-cas and avrag valu of both xcutionand rspons-tim of tasks In cas that th Linux krnl is configurd with Ftrac, thnitcouldbusfultousalso, sinc it complmnts our rcordr wll Our rcordr can rcord srvr vnts and task rlass, whil Ftrac can rcord th contxt switchs btwn th RESCH ral-tim tasks and othr Linux tasks Futur work includs mrging Ftrac and th HSF rcordr to gt mor dtaild and complt tracs W will also continu with improving th HSF schdulr plugin as wll as dvloping nw srvr-basd schdulrs (Bandwidth Sharing Srvr, Constant Bandwidth Srvr, Sporadic Srvr tc) and support for multi-cor schduling (and tracing) REFERENCES [1] P Goyal, X Guo, and H M Vin, A Hirarchical CPU Schdulr for Multimdia Oprating Systms, in Proc of th 2nd USENIX Symposium on Oprating Systms Dsign and Implmntation, 1996 [2] Z Dng and J W-S Liu, Schduling Ral-tim Applications in an Opn Environmnt, in Proc of th 18th IEEE Intrnational Ral-Tim Systms Symposium, 1997 [3] J Rghr and J A Stankovic, HLS: A Framwork for Composing Soft Ral-Tim Schdulrs, in Proc of th 22nd IEEE Intrnational Ral-Tim Systms Symposium, 2001 [4] M Åsbrg, M Bhnam, T Nolt, and R J Bril, Implmntation of Ovrrun and Skipping in VxWorks, in Proc of th 6th Intrnational Workshop on Oprating Systms Platforms for Embddd Ral-Tim Applications, 2010 [5] M Bhnam, T Nolt, I Shin, M Åsbrg, and R J Bril, Towards Hirarchical Schduling on top of VxWorks, in Proc of th 4th Intrnational Workshop on Oprating Systms Platforms for Embddd Ral-Tim Applications, 2008 [6] M Åsbrg, M Bhnam, F Nmati, and T Nolt, Towards Hirarchical Schduling in AUTOSAR, in Proc of th 14th Intrnational Confrnc on Emrging Tchnologis and Factory Automation, 2009 [7] M Åsbrg, T Nolt, and P Pttrsson, Prototyping and Cod Synthsis of Hirarchically Schduld Systms using TIMES, Journal of Convrgnc (Consumr Elctronics), vol 1, no 1, pp 77 86, 2010 [8] M Åsbrg, P Pttrsson, and T Nolt, Modlling, Vrification and Synthsis of Two-Tir Hirarchical Fixd-Priority Prmptiv Schduling, in Proc of th 23rd Euromicro Confrnc on Ral-Tim Systms, 2011 [9] ARINC, ARINC 653: Avionics Application Softwar Standard Intrfac (Draft 15) Airlins Elctronic Enginring Committ (AEEC), 1996 [10] ARINC/RTCA-SC-182/EUROCAE-WG-48, Minimal Oprational Prformanc Standard for Avionics Computr Rsourcs RTCA, Incorporatd, 1828 L Strt, NW, Suit 805, Washington DC 20036, 1999 [11] S Kato, R Rajkumar, and Y Ishikawa, A Loadabl Ral-Tim Schdulr Suit for Multicor Platforms, Tchnical Rport CMU- ECE-TR09-12, 2009 [Onlin] Availabl: http://wwwcontribandrw cmudu/ shinpi/paprs/tchrp09pdf [12] Editors: T Maragria and B Stffn, Lvraging Applications of Formal Mthods, Proc of th 1st Intrnational Symposium On Lvraging Applications of Formal Mthods, Vrification and Validation Springr, pp 140 141, 2004 [13] M Holndrski, M M H P van dn Huvl, R J Bril, and J J Lukkin, Grasp: Tracing, Visualizing and Masuring th Bhavior of Ral-Tim Systms, in Proc of th 1st Intrnational Workshop on Analysis Tools and Mthodologis for Embddd and Ral-tim Systms, 2010 [14] M Åsbrg, T Nolt, C M O Prz, and S Kato, Excution Tim Monitoring in Linux, in Proc of th WIP sssion in th 14th Intrnational Confrnc on Emrging Tchnologis and Factory Automation, 2009 [15] C Liu and J Layland, Schduling Algorithms for Multi-Programming in a Hard-Ral-Tim Environmnt, ACM, vol 20, no 1, pp 46 61, 1973 [16] S Kato, R Rajkumar, and Y Ishikawa, AIRS: Supporting Intractiv Ral-Tim Applications on Multicor Platforms, in Proc of th 22nd Euromicro Confrnc on Ral-Tim Systms, 2010 [17] T Bird, Masuring Function Duration with Ftrac, in Proc of th Japan Linux Symposium, 2009 [18] H Than and H Hansson, Using Dtrministic Rplay for Dbugging of Distributd Ral Tim Systms, in Proc of th 12th Euromicro Confrnc on Ral-Tim Systms, 2000 [19] H Than, D Sundmark, J Huslius, and A Pttrsson, Rplay Dbugging of Ral-Tim Systms Using Tim Machins, in Proc of th 17th Intrnational Paralll & Distributd Procssing Symposium, 2003 [20] D Faggioli and F Chcconi, An EDF Schduling Class for th Linux Krnl, in Proc of th 11th Ral-Tim Linux Workshop, 2009 [21] M Åsbrg, J Kraft, T Nolt, and S Kato, A Loadabl Task Excution Rcordr for Linux, in Proc of th 1st Intrnational Workshop on Analysis Tools and Mthodologis for Embddd and Ral-tim Systms, 2010 [22] B Cantrill, M Shapiro, and A Lvnthal, Dynamic Instrumntation of Production Systms, in Proc of th USENIX confrnc, 2004 [23] V Prasad, W Colhn, F Eiglr, M Hunt, J Kniston, and B Chn, Locating Systm Problms Using Dynamic Instrumntation, in Proc of th Ottawa Linux Symposium, 2005 [24] K Yaghmour and M Dagnais, Masuring and Charactrizing Systm Bhavior Using Krnl-Lvl Evnt Logging, in Proc of th USENIX confrnc, 2000 [25] M Dsnoyrs and M Dagnais, Th LTTng Tracr: A Low Impact Prformanc and Bhavior Monitor of GNU/Linux, in Proc of th Ottawa Linux Symposium, 2006 [26] D Bal, E Bianchi, L Dozio, S Hughs, P Mantgazza and S Papacharalambous, RTAI: Ral Tim Application Intrfac, Linux Journal, vol 29, no 10, 2000