Telemedicine assessment framework

Transcription

1 ASSIST Assesse ad Evaluao Tools fo Teleede Teleede assesse faewok Theoes ad ehods appled o eleede assesse Fal daf 13 Sepebe 2012 Page 1 of 34 July 2012

2 Doue foao Poe le ASSIST -Assesse ad Evaluao Tools fo Teleede ESA oa o 23001/09/NL/AF ESA ehal offe Faeso Fela Ogasao esposble fo he delveable epa Gesellshaf fü Koukaos ud Tehologefoshug bh, Bo, Geay Auhos Rehad Haeshd, epa To Joes, epa Delvey dae: 13 Sepebe 2012 Dsseao level x P Publ C Cofdeal, oly fo ebes of he osou ad ESA Veso hsoy Veso Dae Chages ade By Se o V Veso fo publao RH webse Saee of ogaly Ths delveable oas ogal upublshed wok exep whee lealy daed ohewse. Akowledgee of pevously publshed aeal ad of he wok of ohes has bee ade hough appopae ao, quoao o boh. Page 2 of 34 July 2012

3 Table of Coes Table of Coes Ioduo Ths doue Evaluao faewok Buldg o ehods used exsg ehodologes Theoeal foudaos of poposed ehodology The evaluao odel Qualave aalyss Daa pu ad daa olleo Quaave aalyss alulaos Pefoae easues Glossay ad abbevaos Bblogaphy Page 3 of 34 July 2012

4 1 Ioduo Aeweh ad de Keze [1] oe ha evaluao of healh foao syses ad seves has o deal wh a ulude of heeogeeous vaables, lke aos he people, he aefas he ehology, he evoe whh s pleeed as well as wh he eaos - plus, os poaly, he oupus ad ouoes. Oe ould also say ha eleede has edal, ehal, psyho-soal, ogasaoal, busess ad soeal aspes. Assessg eleede eas o opoae all hese aspes. The eseah of Aeweh [1] o he evaluao ehods of 1,035 sudes healh foao syses shows ha hee s a huge badwdh of evaluao ehods, bu ay, f o os, ae o suable o deal wh a opehesve, hols evaluao of a eleede seve as hey have ueous ehodologal shoogs. To avod ad addess hese shoogs a full evaluao of he ouoes ad pas of a eleede seve eeds o fulfl he followg equees: The pupose of assesse Beefs ad Coss A assesse should always be he aswe o a spef queso ad desged o f hs pupose. I eal lve he develope of eleede seves goes hough vaous eaos ludg ueous desos deeg he deo ad fally suess of a eleede seve. Seveal evaluao appoahes suh as healh ehology assesse HTA fous o he healh syses pespeve yg o aswe he queso f a ehology should be used by a healh seve o ebused by suaes. Thee s aohe pespeve seppg eale, whh s ake by he ASSIST assesse faewok, fousg o he develope ad povee ad askg how a seve a be ade susaable. Wh he oex of povee hee ae ueous ohe quesos o aswe so whehe olou valy s good eough fo exaple. Fo a opehesve, paal assesse, howeve, edal o ehal aspes wll eed o be udged by he pa hey ause o a sakeholde he oex of he eal pleeao evoe. The oupus of he ehal syse aloe ay o ay o have a pa. The assesse faewok us be able o defy ad aou fo a hage of uly o beefs fo he sakeholdes a posve as well as a egave deo. Seveal ehods oly aou fo posve, beefal hages whh egle ha ewly odued syses ad poesses oe a a os. Negleg oss s espeally poblea f he sakeholde eevg he beef s o he sae who has o bea he oss. We all hs beef shf. These beef shfs ae a poa easo fo syse falue. I essee s he su of oss ad beefs,.e. he e-beef, ha a assesse ehod eeds o deee. I addo, seveal assesse faewoks, espeally faal oes, do o aou fo agble beefs, beause hey ae dfful o easue. I ou opo hs s o a good easo fo exludg he. As soe sudes pove hee ae a ube of ehods ha help o esae agble beefs lke he wllgess o pay appoah [2]. Fuhe o hs agble oss ad beefs ae o as easy o easue as he quaave, oeay e gh sugges. Ofe oss labelled wh he sae ae a lude hghly vayg oeps. A good exaple s saff oss. Does he alulao aou fo pa e eployees, does lude o exlude ogasaoal oveheads ad soal seuy obuos. Measued ove a peod of e ad wha o do wh hages he eae? The dffules of easuee bg us o he eed fo esg gou. Obevy ad valdy I soal sees hee ae fou ooly appled qualy ess a ehod should pass [3]: Page 4 of 34 July 2012

5 Cosu valdy: esablshg oe opeaoal easues fo he oeps beg suded Ieal valdy: esablshg a ausal elaoshp, wheeby ea odos ae show o lead o ohe odos, as dsgushed fo spuous elaoshps Exeal valdy: esablshg he doa o whh a sudy s fdgs a be geealzed Relably: deosag ha he opeaos of a sudy suh as he daa olleo poedue a be epeaed, wh he sae esuls. These bas pples ae poa ad should be appled. Howeve hee wll ea a degee of ueay ad falue, f a ehod s bul upo seveal pu vaables ad hese pu vaables ae ake fo a ube of soues usg vaous ehods. Theefoe a assesse faewok eeds o povde povso o es he gou of s esuls ad aou fo he ueay eessaly luded, e.g. by ayg ou sesvy aalyss. Evaluaos have a kow edey o udesae oss ad ovesae beefs. Ths edey eases whee he bass of esao eles oe o udgee ha fas ad whee he peso delveg he esae has a eve o ovesae pefoae. The UK Gee Book[4] alls hs ops bas ad poposes o adus by deeasg beefs ad easg oss. A assesse faewok should povde eas o oue ops bas. Susaably: ex-ae assesse, busess odellg, affodably, sk assesse Susaably s a ao oe fo eleede poes ad he fudes. Chsophe Godo desbes susaably as efeg o he ably o oue ay gve avy o he fuue wh he lkely exsg esoues of a ogasao, as pa of s ogog budgeay ad aagee poesses [5]. I he oex of assesse, a seve odel fo eleede s susaable whe povdes ogasaos wh hs above desbed ably o oue aves. Theefoe susaably ao be egaded as soehg sa. I eques a ogog poess ha adaps he seve o hagg evoeal odos lke ew ehologal eds, behavou of opeos he ake ad hages o egulaos o usoe equees. I he oveall poly oex of he ehealh ake, susaably of a eleede seve would also ply he ably of povdg he seve ove he log u ad whou he ave suppo of eseah fuds o subsdes esed he duao. Fo hs defo we daw fou equees o eleede assesse: The ehod us be able o assess he hsoy of a seve ad o poe s fuue. A poeo eques a seve odel o whh assupos a be based. Ths also ples a opoae saegy ad he develope of a busess pla. A appasal us assess he affodably of a udeakg, whh eas ha he eeded aou of ash, ad ash flow s avalable ad e eus also ake o aou he os of fag e.g. he ees ae of a bak loa. The ehod should povde eas o assess sks, e.g. hose of ake develope assupos, opeo behavou e. Mulple sakeholdes Mos of he publshed evaluao sudes fous o assessg he pa of eleede o a sgle sakeholde. Assesse a dffe wdely depede o he po of vew fo whh s pefoed. Teleede elaed aalyses ake o exepo. A sudy showed ha paes beefed exesvely fo a ehealh applao, bu ha he healhae povde was he a ey o fae he ehealh vese [6]. Whle hs s o osse wh a oal ake seg whee he a beefaes ed o pay, s eveheless a oo osellao healhae. Hee he suao s ofe oe whh pvae beefs o he veso do o povde a suffe eve fo vese, bu soal beefs o all sakeholdes do. So hs s o be efleed he assesse ehod by esug ha all sakeholdes pespeves ae beg aalysed. The assessg pay wll lea whh sakeholde ga ad Page 5 of 34 July 2012

6 whh lose. I he has a bass o akle veo playes adapg he pleeao saegy e.g. by hagg he busess odel. Healh syse, ake ad opeaoal evoe The assesse ehod eeds o aou fo he evoe of a eleede seve a dffee levels. A eleede seve ao be assessed solao of he ogasao s pleeed. Ths opeaoal evoe,.e. s ogasaoal suues ad ulues, has a ual fluee o he adopo ad beefs of a eleede soluo. I s obvous ha suessful goveae odels ae based o he ulue ad ogasaoal suue of he hospal [7]. Appaely, sog leadeshp suppo fo ealsg poeal effey gas ad a suue suppog fee flow of foao ae peodos fo suessful pleeao [8]. The espeve aoal o egoal healh syse ses poa faewok odos. Espeally he effe of paye ehass o he dsbuo of faal ad o-faal beefs eeds o be ake o aou. Geeally, he paye syse affes affodably ad a lead o suffe faal eves fo vese. Ude ea paye ehass, soe epoed beefs a be ealsed by povdes, o hey ay eve suffe a faal loss adopg EHRs [9]. The ake evoe - paulaly he suue of he eoo seo/dusy ses he odos fo suessfully ag he ake. I eoos, he oep of a ake eas ay suue ha allows buyes ad selles o exhage goods, seves ad foao. The ake evoe s a sogly lg fao o busess saeges ad e ala deees he pofably of a veue [10, 11]. The ake evoe osss of busess paes ad value syse pe-supples, he dvdual ad soal pefeees, buyg powe e. of usoes, de ad de opeos, laws ad egulaos fo he espeve seo, faal akes ad he bakg syse ad he powe of poly suos o egulae he ake, paulaly opeo. Ths se of odos s dffee fo dusy o dusy ad fo ao o ao. Ths s espeally ue he healhae seo whee a lo of egulaos exs. I ay oues o egos healh seve povso s a govee duy. Suh akes have oly oe usoes a oopsoy leadg o vey spef odos. Fo he assesse faewok hs eas ha eeds o dep he value syse/ he value ha whh aybe dffee fo seve o seve. Page 6 of 34 July 2012

7 Fgue 1 - The ake evoe Busess ulue Regulaos Soal pefeees Busess Paes Supple s busess odel Ifasuue Podu Cusoe Iefae Cusoe Subsupples Faal Aspes De Ide Copeos Copeos epa 2008 Te deso Soe ehods explly exlude he evoe fo he easuees, ode o be able o oe up wh wdely geealsable, exeally vald esuls. Bu hs seveely ls he usefuless of he ouoes ay spef busess suao. I ou opo he evoe s suh a poa odo o be efleed whe assessg he susaably of ay eleede seve ha us o be exluded. Ths howeve eas ha he esul ao be sly geealsed beyod he espeve evoe osdeed. Mos podus o seves udego a spef lfe yle fo he fs dea o oue use, ad fally hey ae supplaed by aohe oe appopae o ew seve [12]. Ipoa sages he ealy phase of develope ae plos whh ae ofe fuded by eseah agees o dusy. Ths ake valdao phase s deally followed by a sale up o a oue seve. Ths lfe yle has wo plaos: Fsly eas ha values suh as he ube of uses ae o fxed bu vay ove e. Seodly fo he assesse o be eagful, eeds a fowad lookg, foave evaluao o allow fo ouse oeg aos o be ake. Ths efes o he geeal a of addessg povee as has bee elaboaed above. Copaably of easues ad opos Deso akg eques he dsusso of opos ha gh o be opaable a fs sgh lke opag a ask doe by a use o ha asfeed o a IT syse. The assesse ehod we ae lookg fo eeds o ake hose opos opaable. Mul-fao aalyss A assesse of eleede eeds a hols appoah o be able o ove s ul faeed haae. Tehal ad fuoal pefoae as well as egao o healhae foao syses ae, howeve, o values by s ow. They ae dely valued by he uses ad also pefeably easued hs way. Page 7 of 34 July 2012

8 Coseess of odel Coseess s aohe poa equee of he assesse ehod espeally beause he ASSIST assesse faewok shall o age aly a aed healh eooss bu poe leade who should be bough a poso o self assesses he seve. Thus he faewok eeds o be easoable oplex o ove all he equees lsed hee a suffe way, bu also eeds o edue he ube of pu daa ad oupu easue so ha: Daa povso s possble fo sakeholdes heselves a paga way. The fgues asked fo eed o be avalable he ogasao o easly gaheed. Ohewse daa wll be epoed opleely o wogly. Also he oupu easues eed o be balaed bewee easy o gasp ad oplex eough o deal wh ha ul-faeed op. The ehod has o wegh bewee he wo sks o a odel: ove-splfao ad ove-oplexy. Applably o eleede & spae ehology Teleede s a e appled o a age of aves ad seves. The oo hee s a edal avy suppoed ove a dsae, ye hs a be a vey heeogeeous feld. Thus he assesse faewok eeds o aou fo he wdes possble se of eleede applaos he wdes possble ypes of healhae segs, ludg pay, seoday ad eay ae, ad vla, oeal, usodal, lay ad dsase eovey aves. A possble appoah o edue he oplexy s he gudg pple of he lassfao of eleede applaos o doas of sla aue. The ehod evew odued shows ha assesse ehods ha have pevously bee suessfully appled o spae based eleede have advaages ove oe geeal appoahes fo vese he publ seo, beause hey have developed soe oex-spef easues. 1.1 Ths doue Chape wo of hs epo s devoed o he dealed peseao of he ASSIST assesse faewok. Based o he exsg expeee he hape addesses he heoeal foudaos of he faewok, he poposed evaluao odel suue wh s fou bas opoes qualave aalyss, daa pu, quaave aalyss ad pefoae easues. Ths doue s he oupu of a poe fuded by he Euopea Spae Agey. I doues he sae of wok a he ed of he poe. Fuhe wok s evsaged ad pu o hs s weloed. Page 8 of 34 July 2012

9 2 Evaluao faewok The a o suppo he aso fo poes o oue, susaable seves eques he ASSIST assesse faewok o aou fo boh, eleede seves geeal, ad hose akg use of spae asses paula. The faewok was desged o be gee as fa as possble o be applable o a vaey of seves poeally also fo ohe doas suh as Eegy. The ool ad s ue saao howeve wee desged o auaely f healhae ogasao o healhae ogasao eleede seves as well as pae/foal ae o healhae ogasao eleede. Developed o hese spef doa exaples, he ehodology ad odel should be suable fo adapao o ohe eleede doas. Sll afe falsg he ASSIST poe hee s soe oo fo povee of boh assesse faewok ad ool. The assesse faewok efles he saus of debae he sef ouy. The sudy ea wll obue o hs debae aeas whh see o eed fuhe hough espeally wha egads he luso of agble beefs. The followg seos elaboae o he deals of he ASSIST assesse faewok, ulsg he exsg expeee based o a leaue evew ad dealed aalyss of 16 assesse faewoks [13], buldg o sghs fo oopeao wh valdao ses, ad dawg fo he sudy ea s expeee evaluag pas ad ue poes. 2.1 Buldg o ehods used exsg ehodologes A evew of 16 dffee eleede evaluao aves oluded ha oe of he exsg faewoks s eadly suable fo he puposes of he ASSIST poe [1, 13-23]. All lak soe elees, sub-ehods o ehques ha ake he usable fo he pupose a had. Neveheless, he sae-ofhe-a povded a good sag po fo developg a faewok ha ees he eeds of he evaluao of pas eleede. The gee ehealh IMPACT/EHR IMPACT evaluao ehodology was defed as he bes sag po fo developg he eleede evaluao odel ad s assoaed ools [24]. Ths ehodology does o ove all equees of a goous, f fo pupose ehodology addessg all sublees of he equees foulaed above, bu has poved o delve paal esuls due o s applao seveal ehealh ases ad was heefoe hough o povde a good foudao. 2.2 Theoeal foudaos of poposed ehodology I ode o povde lea, eleva ad paal foao o be used fo evdee-based deso akg oeg he possble gao of he seves developed by eleede poes o eal opeaoal udeakgs [25], he ehodology has wo feaues. Oe s a obeve, paal assesse of he pa ad poeal of poe-geeaed seves. The ohe s a eagful opaso of fuue poeal bewee dffee opos. These wo feaues gve evdee-based suppo o f ad, oe poaly, how o poeed wh pos-plo pleeao ad how o odfy he plaed seve o ede oe suessful. The heoeal foudao of he ASSIST ehodology s value heoy, ad paula, he oep of value added. Value added eoos s he addoal value esulg fo asfoaos of faos of poduo o a eady podu. A s sples, s he dffeee bewee he value of a podu Page 9 of 34 July 2012

10 ad he aggegae value of s dvdual opoes povded by ohe papas he value syse. Ove he las deade, value added has bee a wdely used appoah suppog vese deso akg. I he oex of a hols ASSIST assesse, he effes ad ouoes of a eleede seve a be defed as value-added o soey, ehe pa o as a whole, by pleeg ad usg eleede seves. Ths sadpo, alled he soal plae s pespeve, eopasses he pas o all affeed aos ad s osse wh he eoedaos of he Haove Cosesus [26]. The valueadded equals he oal value of a healh seve povded wh he suppo of eleede less he oal value of a healh seve povded whou hs kd of suppo. value added fo eleede = value of healh seve wh eleede value of healh seve whou eleede Ths soeal pespeve ludes all sakeholdes ad aggegaes he espeve gas ad losses, o beefs ad oss. Posve effes, o beefs, eae value, egave effes, o oss, ou whe value s edued. The oal value added s he su of posve ad egave value added, whh s also efeed o as e beef. Ths soeal pespeve a be dsaggegaed o he beefs ad oss of eah sakeholde goup. Fuheoe, wha ay be a beef o oe goup ay be a os o aohe goup, ad he aggegae soe of he ay ael ou. The aalyss us expose hese shfs value ode o povde a easoable aou of he pa of eleede o dvdual sakeholdes as well as soey as a whole. Beyod hs, a egaed healh seves syse ay have eege haaess, whh lead o beefs he aggegae. Fo exaple, shaed aess o opehesve pae daa falaed by a egaed healhae foao syse ao be eaped by dvdual sakeholdes aloe. Cos beef aalyss CBA s he appoah fo ug hese heoeal foudaos o a paga evaluao ool [24]. Aog ohes he UK Teasuy s Gee Book [4], Geay s WBe [27] ad he Whe House Offe of Maagee ad Budge [28] spefy he CBA ehodology as a appopae ool fo aalysg he pa of veses ad aves doas of publ ees, ludg healhae. CBA eables he pa o all sakeholdes o be luded a soo-eoo evaluao, ove he seleed esales, ad he defao of he aowe faal opoes wh he oss ad beefs, also fo dvdual sakeholde goups. These subses a lude he daa used fo Cos Aalyss CA, Cos Effeveess Aalyss CEA ad Cos Uly Aalyss CUA. CBA s geeal he oe opehesve oep wh he hallege, howeve, of oeasg he beefs. Due o beg bedded o a oee healh syse evoe, ay eleede poe evaluao has o ely o a bespoke aalyss wh wo sa pos. Oe s developg a udesadg of he healhae eeds, dagos, lal, healhae ad ogasaoal seg whh a eleede seve opeaes. Whe he evaluao s pefoed by poe ad pogae aages, hs pa a be assued as gve. The ohe s defyg eleva pas ove e, based o a al hypohess abou he expeed ad aed fo pas. Wh hs seod pa, he qualave aalyss bulds up o a quaave valuao of eah dao by assgg oeay values o. The esul of suh a assesse, he e pa peseed oeay us, oples wh he wo hghly desable odos of absolue assesse ouoe ad opaably bewee opos. The lae s gve by a de opaso of e pa values. I hs way, CBA leads o a lea foao base fo fal deso akg, whh a be eaed ad, f deeed eessay, odfed by he deso ake wh ow esaes. 2.3 The evaluao odel The evaluao odel defes he suue of he ool fo esag he oss ad beefs of a eleede poe ove e. The e hozo sas fo he po e whe fs veses Page 10 of 34 July 2012

11 have bee ade o he poe ad he way ae has bee odued bega o hage. I exeds o he fuue, ode o help poe aages wh desos o fuhe vese Suue The evaluao odel follows fou subseque seps as show Fgue 1: Qualave aalyss, defg he sope ad oex of he evaluao Quaave daa pu Calulaos of esaed oss, beefs, ad e beefs ove e, ludg suay aalyses fo eah sakeholde goup Resuls suay ad aalyses peseao Fgue 2: Suue of he ASSIST evaluao odel Qualave aalyss Segs, ludg: Teles, oex vaables, dsou ae Seve odell, value ha, sakeholde aalyss Daa pu Daa olleo of values Fxed, befoe ad afe, dya ove e Calulaos Cos ad beef alulaos pe sakeholde Pefoae suay pe sakeholde Level 1 pefoae easues, fo he soal plaes pespeve: Cuulave soo-eoo eu ae ove e Resuls Level 2 pefoae easues, fo sakeholde pespeves: soo-eoo eu, eu o vese, e agble eu Soue: epa 2012 The fs sep he ASSIST ehodology s defg he sope ad oex of he evaluao. Ths ludes seg bas boudaes of g, healh syse seg, dsou faos, ad he eablg o obsug fluee of he egulaoy evoe. The sakeholdes ad he elao o eah ohe he fo of a value ha ad seve odel ae a poa pequse ad defg feaue of he qualave assesse. I ses he ube ad ype of sakeholdes o be aoued fo. The seod sep volves he olleo ad pu of daa, whh lae feed all alulaos he odel. Daa pu ludes he pae populao, saffg, u oss, oeay values, ad assupo shedules used fo esaes whee aual daa s o avalable. Fo he wo doas of eleede a Page 11 of 34 July 2012

12 pe-defed se of daos s avalable ha a be ulsed depedg o he oex ad sakeholde aalyses pus. Chape Fehle! Vewesquelle koe h gefude wede. hs doue deals wh oe deals o he op of easuee daos ad poedues. The hd pa of he odel daws fo all pevous foao avalable. The value of oss ad beefs s alulaed usg obaos of daa sepaaely fo eah sakeholde goup. The alulaos lude aduses fo ogees, ad dsoug. Daa ae assged o oe of hee ypes of pas: faal, esoue o agble pas. The fal sep of he assesse s he alulao of pefoae easues, whh ae eded fo deso suppo ad as bass fo oeve ao. The daa allows geg ovevews of he oveall soo-eoo pefoae ad sub-aalyses of he pefoae aggegae ad fo eah sakeholde, as desbed seo 2.7. The a s o help deso akes defy affodably aspes, faal ad ohe sks, ad po of hghes pa. The followg seos povde oe dealed foao abou eah of he fou levels of he ASSIST assesse faewok. The despos ae gudg he eade hough he log of he odel fo he use s pespeve, whee he fou sages of he ehodology ae addessed hoologally. 2.4 Qualave aalyss The qualave aalyss of he seve queso ad s evoe s al. May beefs, as well as egave pas, ae ggeed o edued by exeal faos ad eal haaess Coex of he eleede seve ad sope of aalyss A esseal pespeve fo ay eleede poe s a udesadg of he healhae ad ogasaoal segs ad heewh he seve as well as busess odel. I defes he uses ad ohe sakeholdes of he seve ad he ehal fuoaly o be developed. These alogehe also defe he evaluao sope. Healhae syses usually have spef, aoal egoal ad loal feaues. The exe o whh a seve s, o a be egaed o he healhae syse has a pa o he pobably of suessful log-e seve povso. Udesadg he ogasaoal bakgoud helps o defy he vese oves ad develope pah of he poe, as well as o defy eleva uses ad ohe sakeholdes. Ebeddg a eleede syse he healhae seg s eag a ICT suppoed seve. The ICT syses ehal ad fuoal pefoae lke vdeo fae aes o olou fdely ay be pvoal fo he usably ad susaably of he soluo. Whou a egaed seve odel, howeve, s o suffely desbed o be assessed. Fo aalyal soudess, he ASSIST assesse faewok leaves ou eay pas. Seoday pas ae oly dely geeaed based o pay pas ad eessaly eah level of pas would add ueay o a ode of agude. I addo, would be dfful o defe a lea elao bewee ause ad effe. A exaple ay be feed saff e, a seoday effe ha beae avalable due o oe effeve ae, he pay ause. The queso ow s wha o do wh he feed saff e? Ia be used o lay off saff, ake ove ohe asks o edue ues o allow opesag saff shoage. Wha wll be doe wh he e we would all a eay ode pa ad leave ou o he quaave sde. I wll, howeve, be a eessay pa of he qualave evaluao o addess hese. Eve oe de effes lke he level of eploye a seo of he eooy ae hkable bu eve oe doubful o elae o he ause. The qualave aalyss has o defy poess hages, ludg dffee ad ew wokflows, lal paes, ad wokg paes. These lay he bass fo evealg posve ad egave effes fo Page 12 of 34 July 2012

13 usg he eleede syses. A he ex level of assesse, all of hese faos, lke uses eaos, boh obeve sgs of behavoual hage ad subeve opos, play a sgfa ole assessg ad assgg quaave beefs ad oss. The ASSIST ool, developed o he bass of hs faewok, seves he pupose of gudg he evaluao hough defg he sope ad oex ad he defyg he eleva pas o all sakeholdes. As sad elsewhee, foao syse odellg s ogasao odellg [29] ad hus, defg he segs of he eleede seve s al o he evaluao. Paly he ool suppos uses o adus ad fe-ue he ool segs es of eleva aos ad he ouee, o o, of pedefed pas Sakeholde aalyss The sakeholde aalyss volves a pese aalyss of he aual people ad ogasaos affeed by he eleede seve. Ths s al fo esug a hols appoah o evaluao. A poa pa s he defao of poeal veo-playes who a, hough he polal o ogasaoal powe, sop a ave, as well as dvg playes, who have a s ees oveso of plo applaos o oue seves. Sakeholdes o eessaly eed o be ave ages he seve. They ay also be affeed by hages aused hough he eleede seve lke a shf fo hospal ae o pay ae esulg fewe adssos o he hospal. The ASSIST assesse faewok povdes a suue fo he sakeholde aalyss. The ea has defed fve hgh-level sakeholde goups wh seveal subgoups. A sakeholde goup has a oo ole ad elaed o hs a oo se of daos o easue pas o hs goup. The ASSIST ool povdes he possbly o dvdualse he se of sakeholde goups ad subgoups as f fo he spef ase. Ths ludes he expaso of he ool o dffee sall sakeholde goups wh he pedefed aegoes. Fo exaple, he healhae saff volved a eleede seve a lude dffee ypes of doos, uses, heapss, ad ohe pofessoals. The assesse faewok defes he eed o pay aeo o healhae saff as sepaae sakeholde goups, bu he evaluao wll have o defy how ay hese goups ae, ad wha ypes of pofessoals hey volve. The sakeholde goups ad subgoups pe-defed he assesse ae desbed below Idvduals Paes Idvduals lude wo subgoups paes ad foal aes. Paes a be hoally o auely ll, o boh. They ay eed eveos, ay be sube o eveos, o have us opleed eveos ad eed oog. They ay be ave papas of a seve o ases a heapy. Paes ay also be healhy people a elavely hgh sk of beog ll. A healhy eldely peso wh a faly hsoy of hea dseases, who dedes o o a hoe oog pogae o peveo gouds, falls ude he sakeholde goup paes. So does a solde ued a lay ao. Paes ae usually expeed o beef fo eleede seves. Beefs lude he pae s valuao of he seve expessed as bee healh saus, fewe ad oe effeve eveo ad bee qualy of lfe. O he ohe had, soe busess odels eque paes o beoe oe ave he ow ae o pay fo he seve, whh ould be peeved as a egave pa. Soees s eagful o subdvde he goup of paes oe spef goups. Fo exaple, f eleede s fo paes wh dabees, you ay eed o defy he ubes of ype 1 ad ype 2 paes. You ay wa o lassfy he by ypes of oplao, suh as ophhal, eal, euologal ad ulaoy, o by sul depede o o-sul depede. Physas should defe a lassfao ha s eleva fo effes expeed fo he eleede seve fo he paes as well as he effos wh hese dffee goups of paes. Page 13 of 34 July 2012

14 Ifoal aes Ifoal aes ae usually faly ebes, eghbous, olleagues, o feds who povde ae o paes fo fee. They a be affeed by hages he ae delvey poesses, as soe of he ask ay beoe obsolee o ay have o be aleed o eplaed by ohe oles Healh Povde Ogasaos HPOs ad saff Healh povde ogasaos lude all kds of pofessoal ae seve povdes, fo lassal pay ae paes, seoday ae hospals, hough soal ae seves a hoe o a usg hoe. HPOs ae he oe of os seve ad busess odels fo eleede seves. HPOs eploy healhae pofessoals, ad pay fo soe of he ohe esoues eeded fo healhae oo, suh as dugs, ls, oupae fales ad ohe hospal fales suh as heaes ad phaaes. They also eeve oe fo paes, govees, healh sues ad ohe hd paes. Teleede seves ay hage soe of hese ash flows. Soe HPOs pay fo eleede equpe, sallg, aag ad ag people o use. Hospals ay lose oe whe paes avod adssos o say fo fewe days. They ay ease he oe fo oe oupae appoes. These ash flows usually hage ove e. HPOs ae busess ees o aohe way depede ees. ASSIST dffeeaes bewee fopof ad o-fo-pof ees. No-fo-pof ees have a lowe dsou ae. Tele seve ees Tele seve ees povde healhae seves. Mos eleede seves lude a ey ha akes hs ole. They a be pa of a bgge ey lke a hospal. Fo he puposes of eleede seve assesse ad he poae of ele seve povso he seve odels, he ee should be sepaae fo aalyal puposes ad hadled as a busess u eve f hs s uely o he ase. I os ees, qualfed uses ad physas povde adve o paes. I hs way he ele seve ee a also be dffeeaed fo eleede supple, whh oly povdes ehal ad ogasaoal suppo bu o edal seves. Pay ae ogasaos Pay ae ogasaos lude sgle paes of geeal paoes GPs, goup paes, o lage pay ae ees. Speals ae ogasaos Hospals These lude he povso of seoday seves ousde of hospals. I soe oues, adologss, deaologss, ad ohe spealss u he ow sugey, o o foes a goup pae fo a paula spealy. Mlay edal us a be luded hs goup, boh o-obave ad ofl ad wa oles. Hospals ae he oveoal seoday ad eay ae ogasaos, bu a also lude vla ad lay ehablao ees. Fo he puposes of assessg eleede seves, we us dffeeae bewee pae, oupae, ad eegey seves. Eah ype of seve a be affeed dffeely by he oduo of oue eleede seves. The obvous pa hee elaes o he volue of wok, bu ohe effes lke hages oles ad esposbles us also be ake o aou. Page 14 of 34 July 2012

15 Medal aspoao seves Nusg hoes These lude abulae a seves bu also edal evauao ad aspoao by plae. They a be povded ehe by a hospal, he a foe, o by a depede ogasao. Thee ae dffee ypes of abulae seves, suh as eegey ha asfe paes fo see, eegees ha asfe paes bewee hospals ad fs o see spealss. Thee ae also oue pae asfe seves. Nusg hoes povde lvg fales fo eldely ad fagle people, wh o-dead as well as egula usg seves. They a povde usg ae, soal ae, o boh. Nusg hoes a be uses of eleede osulaos wh GPs o spealss, ad a be affeed by a hage dead due o eleede eablg people o say a he ow hoe. Couy Nusg Seves CNS Cae pofessoals Couy usg s haaesed by paes lvg a hoe, who eeve egula vss fo uses. I ay evaluao ehodologes, ae saff ae oly egaded as eployees he soal ae syses. Howeve, hey a be poweful dves o veo-playes as dvduals. Ulaely, hey ae de uses ad povdes of eleede seves ad hus he susaably of a ave depeds o he aepae ad edosee of he ew odel of ae. The goup ludes all doos, uses, heapss, lay edal pesoel, soal ae wokes, ad suppo saff. They ae eployed by a HPO. The pesoal aude owads a eleede seve depeds o he pa has o he daly aves. Spefally, he peepo of whehe eleede akes he lfe ease o oe dfful has a bg pa o ob sasfao, ovao, ad fally also he qualy of lfe Payes Healhae payes Soal ae payes Payes ae ogasaos ha aage he faal flows of he healh ad soal ae syse. Depedg o he ouy, hese a be pvae o publ suae opaes, o-fo pof fuds, loal, o aoal auhoes. The ds feaue of payes s ha hey olle obuos fo he populao soe fo, pool he sk of ae eed, ad ebuse HPOs fo ae seves. The obuos a also be de, va he geeal ax syse. Rebusee o HPOs a also be va budges. Healhae payes deal wh pay, seoday, eay, ad lay healhae, ad ove he healhae value ha fo peveo, hough dagoss, eae, ad ehablao. I soe oues, egaed ae pogaes ha go o he eal of hoe ae a be also oveed by healhae payes. Soal ae payes usually fous o ae fo eldely, dsabled, ad ohewse soally dsadvaaged people, who ae o eessaly sk. Hoe oog seves ofe addess boh, healh ad soal ae eeds, whh soees leads o ueaes as of who should be payg. Soal seuy offes Soal seuy offes ae esposble fo he adsao of ueploye beefs, ludg beefs ase of pologed sk leave. The lae a be eleva fo eleede, se fase Page 15 of 34 July 2012

16 eaes ad oe depede lvg opos fo ho paes ould pa o he faal bude of he offes No-ICT ogasaos Ths sakeholde goup s affeed by eleede, bu s usually o dely volved eleede poes. Phaaeual dusy All eployes Phaaeual opaes ould obseve hages dead fo he podus, whh a be a effe of pleeg eleede seves. These hages a go ehe way. The pa s expeed o be o he busess sde of opaes, ad spefally o he podu x. I should be oed ha ode o obseve a easuable pa, he sale of eleede seves ad assoaed hages dug eges us be sgfa. Ths goup oss of opaes who eploy people ha ae also paes he seve. Regadless of he spef busess seo, eployes have a ees a healhy wokfoe. I soe ases, eleede a suppo hs by peveo o eale eu o he wokplae ICT dusy Teleo supples The ICT dusy has a de sake eleede seves, se ICT vedos ae supples of he ehology eeded fo gaheg ad exhage of daa. The ees eleede s o develope ad sellg of podus. Ohe supples, eal o lage ogasaos suh as lay auhoes, also have a ees eleede, as hey ae ofe volved he ehology se up ad aeae. The followg sub-goups a pese boh, exeal vedos ad eal depaes wh ogasaos, depedg o he spef poe. Teleouao supples ae esposble fo he asfe of daa fo oe plae o aohe. They ae o oeed wh he oe of he daa. Teleo supples lude also SaCo seve povdes. Teleede supples Sofwae supples povde he opoes eeded ode o aage he daa befoe ad afe s asfe. Hadwae supples dsbue deves, fo PCs ad oos, o pesoal healh ools lke ECG, blood pessue, o gluose easuee sesos. Moe ad oe eleede supple s povde a egaed ehal seve ludg aeae, sallao ad hosg. 2.5 Daa pu ad daa olleo Ths s a bef ovevew of he ypes of daa equed. A dao usually osss of seveal daa es ove he peod of assesse. The fal daos a be foud hape Fehle! Vewesquelle koe h gefude wede.. ASSIST a be used o assess ad evaluae eleede seves dug he plag phase, whle seg up, plog o allg ad whe s oue use. Fo eah phase, esaes of daa wll be eeded. The aou of esao edues as pleeao ad sale up pogesses. Howeve soe effes espeally lal pas ae dfful o easue ad effo fo auae easuee s hgh. Page 16 of 34 July 2012

17 Theefoe s aepable o paly ely o esaes also whle oue use. I hape we expla how he ASSIST assesse faewok deals wh hese uavodable ueaes. Plag Roue use Plog Esao Measug Moeay values Resoues Iagble pas The pefeable soue of daa ae healhae povde ad eleede seve ogasaos. They a povde daa fo he oue seves. Whee suh daa does o exs, esao by evaluaos wll be eessay. Ifoao o oeay values of all eleva oss ad beefs s seldo eadly avalable fo HPOs, beause he sasal ad faal eods usually do o eod hese ouely. The evaluaos wll eed o suvey hs daa fo eah dvdual assesse. Moeay values of oss ad beefs wll usually be esaed a osa pes ove he whole vese lfe-yle of desg ad develope, egagee, esg, pleeao, opeao ad hage. All values wll be based o pes fo he yea of evaluao ad fo he ouy whh he seve s povded. Oly oues whee flao s a ao oe ad would osdeably bas hese daa, flao-adused daa a be used. Epal evdee has show ha ee yeas, flao was o a ao oe os oues, ad slgh aduses wll have vually o effe o ouoes. VAT should be dedued fo pes whee possble. Esaes of all sakeholdes volvee wll ely o esaos of he e alloaed o hese aves. Pofessoals e edeployed fo ohe aves ad addoal oss, suh as ew poe eas ae exaples. Redeployed e fo healh povde ogasaos s oeazed based o he salay of eployees pe wokg e. Esag he oeay value of agbles pas apples seveal ehques. Revealed pefeee s used o esae e savgs of zes. A aveage salay s used as a poxy hee. Fo he valuao of he seve a saed pefeee appoah was used. Due o he haae of ASSIST as a self assesse ool, wllgess o pay ould o be used as eeds a aed ses o se up a egoao evew. Isead of ha we assged oeay values o a valuao quesoae. Xxxexpla WTP ool Daa olleo Daa olleo a be a hape s ow ad he ssues elaed o daa olleo wll oly be oughly ouhed hee. I ppal all daa a be used a aalyss agg fo bes guess o daa fo lal als. The bee he valdy of he daa he oe vald wll be he fal esuls of a ASSIST aalyss. Page 17 of 34 July 2012

18 Soe appopae es a be foud ogasaos ahves ad foao syses. These lude soe lal daa ad sass, eleede seve ulsao sass, wokload sass, hages apal ad opeaoal expedue, ad pes. Soe oss ad os beefs have o ely o esaes ad assupos. Ideally, assesses should pefo dealed obsevaoal sudes o esablsh pese hages lal paes, e alloaos o asks o qualy of ae. Howeve, gve epoal ad budgeay osas, se-suued evews a povde boh, qualave olusos ad soe of he foao eeded o ake esaes ad assupos eeded fo quafyg he pa [3, 30]. Ievews ae eoeded as he pay foao gaheg ehque, oly opleeed by dsbug dealed quesoaes. The lae a be oo gd, leavg lle oo fo elaboao o ga kowledge o he bakgoud, oex, ovaos, dves, ad he eveual pa of dvdual aves [3]. Fully suued quesoaes offe led sope o apue spoaeous eaos o suble affes, o eluae by sakeholdes, ad ae esoue-esve. Qualave ehods usg se-suued goup evews offe sope o seek osse foao ad o efle spef healhae segs, suh as hages o lal ad wokg paes. They ae also fuful ad ope eough o eludae sakeholdes pespeves, o ove a wde age of opos ad he segh of opos held [31]. Thus he poess of evaluao s ofe as sghful as he eveual esuls ad poe ad pogae aages should ulse o hs oppouy. Whe akg a deso o he use of evews ad suveys, evaluaos should ake o aou he sks of ove-splfao ad ove-oplexy. The ul-faeed haae of eleede akes hs doa geeal vey oplex, whh eas ha eah aea has ay opos. Ths akes sadadsed suveys vey elaboae ad dfful o osu. A huge ube of opos eed o be aowed dow o he opo applable o he spef ase. As he assesses wll volve ovave poes, hee wll always be he poble ha he poe ea has foud a ew way of dog hgs, whh s o efleed a gd quesoae. To pu he ohe way aoud: We a oly osu a quesoae fo ssues we kow abou. Tyg o deal wh all aspes akes a daa gaheg sue gd ad ove-oplaed. Ove-splfyg would o ove eough deals abou he paula eleede seve. Eve daa fo supposedly vey goous ehods lke lal als a delve sleadg esuls whe fo exaple he subes have bee seleed wh speal ae e.g. a ubalaed dsbuo of degees of sevey of he lless. I addo plos ae ofe odued ees of exellee ha ayhow delve a hgh sadad of ae ad ovaed ad sklled paes ed o papae lal als. Wha sees o be oe poa ha aual valdy of he daa s akg aspae he level of valdy ad he oeed vaae of esuls. Valdy of daa should be poved eavely depedg o he fluee o he ed esuls. I hape we fuhe exploe he elae of ASSIST o dsbuos. 2.6 Quaave aalyss alulaos The quaave aalyss he assesse ehodology efles ad bulds o he qualave wok doe befoe. The gee odel allows fo a vaey of pa daos o be luded. These faos affe boh beef ad os esaes. Ths ul-fao appoah esues ha he assesse povdes a opehesve easue of he pefoae ad poeal of eleede seves Maheas The aheas behd he ASSIST assesse faewok allows he oveso of daa ad qualave sghs o a quaave, oeay se of esuls ha a gude deso-akg. Page 18 of 34 July 2012

19 Beefs + + = Faal beefs Redug ash oulays Lbeaed esoues Te saved fo exsg aves Iagbles Coveee Toal sooeoo beefs - Coss + + = Faal oss Need fo exa ash Redeployed esoues Te ake away fo ohe aves Iagbles Ioveee Toal sooeoo oss = Reu + + = Ne ash eu/ Reu o vese Ne eoo eu Ne agbles Soo-eoo eu k k A he os bas level, he aalyss s fouded o dvdual daos fo beefs, b ad oss,. These a be of hee aegoes: faal, edeployed ad lbeaed esoues, ad agble pas. The ak k { f,, } daes he espeve aegoy of he beef o os dao. k f deoes puely faal, o ash pas. These ae beefs ad oss ha ae pa of ash flows. A exaple s fees fo seves of ay kd. k deoes daos fo beefs ad oss epeseg edeployable faos of poduo, suh as e ad ohe esoues ha do o dely affe aoug epos ad dvduals puses. A exaple s saved e of pofessoals wokg a hospal, whh hey a edeploy o ohe asks. Redeployable esoues a be oveed o, o daw fo, ash seas, bu hs eques aageal desos ad ofe ogasaoal hage. k peses beefs ad oss of agble, soal aue, whh ae puely o-faal pas. A exaple s eased wok-sasfao of eployees The ses of peae values, p p, p, p,..., ad of e sees values s s, s, s,..., k povde he bass fo alulag he oeay value of eah beef dao b ad eah os k dao. The oeay values ae fuos of he vaables p ad s fo he eleva yea of alulao, ad he ogey fao σ: k b f s, p, 1 k g s, p, 2 Evaluaos a use a osse ehodology, bu he odels eed osug fo eah ase o efle he spef segs. Spef fuos ae eaed fo eah dvdual dao. The evaluao defes he e peod whh a spef pa dao s applable o he poe a had. The exe o whh he dao apples o a paula evaluao depeds o he seg. Ths s a he level of os ad beef alulaos of he odel. Thee ae seveal ehques fo esag a paula beef o os dao [32, 33]. Exaples of suh ehques lude: Te hages of zes elyg o esaes of he value of e. Chages avel oss elyg o esaes of avel oss. Chages of e o ubes of edal poedues a be esaed fo u os alulaos. Qualy gas a be bee-foed paes, eless of ae, effeveess of ae, pae safey ad sealed ae. Soe of hese a be esaed usg u os alulaos, suh as avoded hospal adssos. Page 19 of 34 July 2012

20 Iagble beefs, suh as he value o paes ad ogasaos, ely o sakeholde behavou evealed pefeee o saed pefeee. Fo he os sde, a llusao s he oss o zes fo povdg ose, a os ha a ou ay saes elaed o soage ad assso of pesoal daa. Czes who wsh he daa o be shaed aoss healhae povdes have o gve he expl foed ose a egsao. Ths s a puely o-faal effo of aegoy k=so fag evey egseed pae oe. A poxy fo he value of hs effo s he e akes o olle foao ad povde he ose. The e s ehe pesely easued, o esaed by healhae saff povdg he dealed foao ad asweg ay quesos. Fo hs llusao, a aveage e esae s 5 ues pe pae. Ths apples o all ew egsaos he eleva yea, se we fous o he egsao ose, povded oly oe. Le us assue ha 200,000 paes egse wh he syse fo a gve yea. I ealy, hs ube s fo he eal sass of eah se ea. The fal vaable s he oeay value of e. Gve ha he age populao s he aveage ze, a appopae poxy fo he oeay value of e s aveage oe. A easoable value s 20 pe hou. Eah evaluao eseahes oes o efle he aual levels. I ode o oplee he os fuo s leaes fo, we have o assg a ogey fao. Takg a aduse of 10% eas a fao σ of 1.1. Ths gves he followg os fuo fo he effo by zes o povde up-fo, oe-off, foed ose fo egsao, fo yea, whh 200,000 ew paes ae egseed: paeo se e fo ose hous x ube of paes yea x aveage houly oe x ogey fao = 5/60 x 200,000 x 20 x 1.1 = 366,667 3 The ube 366,667 hs ase s us a oeay epeseao of he esaed value of he equed effo, o a faal oulay fo paes ad heewh belogs o he aegoy of agble oss, o aegoy k. As aleady sessed, he value s a poxy, ad as suh oly a esae. Ohe fuos a be uh sple ad pese. A sple exaple s gve by sagh eduos oss pe pae, whee he oduo of a eleede syse poves he faal poso of a healhae povde ad/o a hd pay paye. Ths beef fao, o be epoed by eal sudes wh he ogasao, s a agble, faal pa ad hus of aegoy k f. The oespodg beef fuo fo a paula yea s he aual exa ash saved ulpled by he espeve ogey fao. The value of Aual Beef AB yea of eah aegoy k s defed as he su of he dvdual k beef b daos, as show by equao 4. The value of Aual Coss AC of eah aegoy k s deved oespodgly, deped by equao 5. Fo beef daos ad os daos, he aual beef ad os fo aegoy k ae: AB AC k k k b k The Pese Value PV of he Aual Beef fo aegoy k yea of he ave s he su of he dvdual beef daos fo aegoy k dsoued by he dsou ae. Equao 6 shows he aheaal epeseao: PV of AB k = 1 1 b k k = 1 b 1 6 Page 20 of 34 July 2012

21 Page 21 of 34 July 2012 Beause he base yea fo dsoug s he yea of evaluao, say 2010, a addoal vaable α deoes he e o hs yea. α beoes egave whe esag fuue pefoae. The os dsoug woks he sae way. Equao 7 shows he pese value of he aual Ne Beef NB of aegoy k yea, whh s he dsoued dffeee bewee he aual beef ad aual os: PV of aual NB k = 1 k k b The PV of he uulave e beef, o he Ne Pese Value NPV of aegoy k of he seve, s he su of dsoued aual e beefs of eah yea, up o yea T, he ed of he hozo. The aheaal fuo s show by equao 8: NPV k = T k k b We ou, he NPV of he hee aegoes faal, edeployable ad agble ae llusaed by equaos 9 o 11: Cash NPV = T f f b Redeployable NPV= T b Soal NPV = T b The eoo e beef s defed as he su of faal ad edeployable eoo esoues. Usg he dsoued values, hs effevely eas addg equaos 9 ad 10: Eoo NPV = Cash NPV + Redeployable NPV T f f b b The soo-eoo pa osss of all hee aegoes, addg he soal deso o he eoo oe. I a dsoued fo, hs eas addg equaos 11 ad 12: Soo-Eoo NPV = Eoo NPV + Soal NPV f f b b b Equaos 14 ad 15 deal wh alulaos of eu aes. Fs, he Eoo Reu o Ivese ROI s defed equao 14. I volves boh eoo dao aegoes, ash k ad edp k. The eoo ROI s opaable o a adoal eu fo a vese, say a oveoal busess poe, ye does o eque he sep of oveg edeployable esoues o ash. I s alulaed as follows:

22 Page 22 of 34 July 2012 Poxy eoo ROI= T f T f f b b I he fal sep, equao 15 alulaes he Soo-Eoo Reu SER of he vese, whh s he ao of dsoued uulave e beefs ad uulave oss: SER = T f f f b b b The SER s he pay ad os opehesve pefoae paaee used fo assesse of ehealh veses ASSIST. I povdes a opehesve easue of value fo oey, aoug fo all soal ad eoo pas elao o he oss assoaed wh hose pas. 1 Sub-aalyss, suh as he ae of eu ROI puely eoo es o ash flow pas fo busess ees follow he sae log as peseed above, albe wh a aowe age of os ad beef daos, aly deved fo aoug defos ad lked o he ees vese. Fo suh aalyses, equaos 4 ad 5 lude oly eoo faos of poduo o oly faal faos, espevely. Ths peseao povdes he bas oeps behd he aheas of he ASSIST assesse faewok. Dsoug Dsoug s a oo eoo oep efleg a dvdual s pefeee o have oey o esoues ow o eploy he ad ga value fo he as opposed o lae. Whehe soe agble beefs should be dsoued s a ae of debae, as he uly of healh s peeved dffee fo hose of oey o podus. Ul fuhe lay s eahed we follow he adve by Duod o do so, whh eas ha all dao ae dsoued he sae way [18]. The ASSIST assesse faewok dsous posve ad egave pas o e pese values. The base yea s he fs yea of aalyss. The defaul dsou ae a be se a 3.5%, efleg a aveage fao of ue offal aes foud aoss Euope [34]. The ASSIST ool allows evaluao o adus hs ae. The defaul dsou ae s a soal e pefeee ae. Fo vese desos oe would expe a hghe ae sla o opoae bods. ASSIST dffeeaes bewee fo-pof ad o-fopof sakeholdes. No-fo-pof sakeholdes ae alulaed wh he soal e pefeee ae Ueaes evaluao Thee ae seveal ways of dealg wh ueaes evaluao. ASSIST offes wo sues dealg wh ueaes: ogees ad a sesvy aalyss usg Moe Calo. Ueaes oe o evaluao beause pas wee fogoe, ae ulea o dfful o addess. VAT s a exaple ha s dfful o addess. Aohe aspe s ueay esaos o easuees. Ops bas s a exaple ha ous poes whe fuue oss ad beefs eed o be esaed. I s a deosaed, sysea edey fo poe appase o be ovely ops [4]. 1 Fo oe deal o he advaages of SER ad he dfeees o ROI, see seo 3.7 below.

23 Ne Beefs ASSIST Cogees The evaluao ehques peseed above povde basele esaos of oss ad beefs. Cogey aduses efle he elae o esao. They ease oss ad edue beefs. Cogees a be as hgh as 70% fo soe basele oeay values. The odel has a defaul value fo eah goup of pa faos, whh a be adused f eessay. Pobably, vaae ad Moe Calo Iepeg he ouoes of he evaluaos should ely o he ode of agude, o he absolue values. Fo hs easo, he odel akes use of Bayesa pobables, whh epe he oep of pobably as "a easue of a sae of kowledge", oas o epeg as a fequey o a physal popey of a syse[35, 36]. I obao wh epeaed ado saplg followg Moe- Calo sulao ehods, he esul s assesse ouoes he fo of ages. The pedve powe of a age s lage ha he powe of a spef po wh ha age. The peseao s llusaed fgue 2. Fgue 3: Exaple of a age-based assesse ouoe 30,000, ,000, Pobably Dsbuo of Ne Beefs Rage of pobable ouoes: 20,000, ,000, ,000, % 99% 5,000, ,000, Yeas Mea Soue: epa 2009 The llusave fa-ha would sugges ha he oss of he poe wll ealy be exeeded by he beefs. The uve of he ea pa a seve as a gude, bu eay s gve oly by he age. Adusg pu values a allow poe aages o defy aos wh he powe, whh shf he age he desed deo. 2.7 Pefoae easues The poposed ehodology povdes a ose, bu led aswe o a spef queso lke whehe s woh o oue vesg. I a also be used as a poweful ool ha suppos deso akes he log-e vsos, as well as poe ad pogae aages he gudae of ug poes. Pefoae easues ae show he fo of has peseg vaous pefoae faos ove e. The seleo of pefoae daos was guded by wo pay quesos fo he pespeve of deso akes: Page 23 of 34 July 2012

24 Is woh gog ahead wh he poe, ad dag he sep o oue seve povso? Wha eeds o be haged ode o sueed gog ahead? Addessg hese wo quesos esues susaably of fuue developes ug ad eegg poes. The seod queso apples equally o he po of deso o wha o do afe a poe ad o vaous deso pos dug he lfee of develope ad pleeao poes. I he lae ase, hese quesos a be he fudaeal pa of egula hek pos, poeally ggeg hages he deo of aves ad he ulae aual SER. The easues ha eed oog ae of wo levels. The fs level povdes he bg pue, akg he pespeve of he soal plae. The oep of he soal plae has s oos welfae eoos ad efes o a hypohe deso-ake who aeps o aheve he bes esul su. Thus, level 1 pefoae easues lude aggegae paaees fo soey as a whole, pog o poeally ususaable developes. Level 1 easues fous o he followg hee ypes of paaees: Soo-eoo eu SER Eoo eu o vese eoo ROI Cash flow The deals ae dsussed o seo below. The seod level of pefoae easues povdes oe deals o spef pas ad helps defy pos of hgh posve pa as well as poeal affodably gaps fo dvdual sakeholdes. Level 2 pefoae easues addess dvdual po of vew of key sakeholde goups ad sub-goups. These easues allow deso akes o defy auses of poeal halleges defed by he aalyss of level 1 easues, ad hus ae oeve ao. The daos a hs level ae: SER fo eah sakeholde Eoo ROI fo eah sakeholde Cash flow fo eah sakeholde Level 1 pefoae easues Level 1 pefoae easues ae he hghes level of esuls peseao, suasg he pefoae alog he hee daos SER, ROI, ad ash flow ove e. -. Soo-eoo eu The SER s defed as he ao of e beef o oss ove e. Ths easue opses all posve ad egave pas, ludg faal ad ohe eoo esoues, agble hages qualy of lfe ad wok, as well as agble, ofe eooal pas. A posve ao daes a wohwhle edeavou fo a soo-eoo pespeve. A ao of zeo equals a pl beak-eve po a whh he oveall soo-eoo pa s zeo. The SER easue seves wo a puposes. Fs, s a opehesve easue of all soo-eoo faos, ludg o-eoo daos. Ths eables a opaso bewee he eoo ROI pefoae ad he soo-eoo pa SER of a vese. Seodly, he SER easue, peseed peeages, allows dffee vese opos o be opaed o he elave eu, ahe ha o he absolue values of pa. The advaage of usg eu aes sead of absolue values s llusaed by he followg exaple. If oe vese s expeed o aheve uulave e beef of 200 llo ove a ea peod, ad aohe s eug 250 llo ove he sae peod, he absolue values would sugges hoosg he seod vese. Howeve, hs ay be a sub-opal hoe. The fs vese ay bg 450 llo woh of beefs a he os of 250 llo, whle he seod opo eeds 500 llo ode o ealse 750 llo of beef value. The SERs ae 80% ad 50% espevely. Despe he lowe absolue value of e beefs, soey s bee off wh he fs opo, as ges bee value fo oey. Aohe easo o pese he soo-eoo pa as a ae s o avod ofuso bewee he oeay value of soal pas ad aual faal effes. The peseao of e soo-eoo beefs Page 24 of 34 July 2012

25 Cuulave SER % ASSIST oeay us, suh as Euo, has led o sepeao of esuls ad espeve oveops expeaos egadg he faal eu of ehealh veses, ludg eleede aves. The fs ha peses he uulave SER ove e, depg he eds ad akg expl how log akes o aheve a posve eu. As aleady oed, he SER s show a age, efleg possble ueaes he pu vaables of he odel. Fgue 4 - Illusao of aggegae uulave SER of a vese Te Soue: epa 2010 Posve SER s a eessay, howeve o suffe, odo fo log e susaably of seves. Eoo eu o vese The ROI easue ludes oly eoo esoues, adoally defed as lad, labou, ad apal. I ay saes, suh as saff e ad expedue o ehology, he value of esoues s he oppouy os. The esoues ould be eployed fo ohe puposes, f o used fo eleede seves. Howeve, he ealsao of he poeal ROI o eus efleed aoug epos s a hallegg aageal ask. Fo exaple, whe egh doos save oe hou a day eah, esoue a he value of oe full e equvale FTE s lbeaed. The deso whehe o dsss oe doo, o assg ohe asks o all doos s sepaae fo he value of beef obseved. The eoo ROI would lude a ga of oe FTE egadless of how he lbeaed esoue s used. Page 25 of 34 July 2012

26 ROI % ASSIST Cha 1: Illusao of aggegae eoo ROI of a vese Te Soue: epa 2010 I oas o he SER poso, ROI s expeed o ake loge o be ealsed, as ay beefs fo eleede ae agble ad o-eoo. Whle he ROI s vey poa fo ogasaos, espeally hose whou a soal adae, dvduals do o eessaly eed a eoo eu ode o suppo a ave. I ay ases, dvduals ae pepaed o ves eoo values, suh as oey ad e, ode o eeve agble beefs, suh as ofo, healh, ad geeal wellbeg. Whle hese lae beefs ae aoued fo he soo-eoo paaee of SER, hey ae o pa of he puely eoo oep of ROI. The ROI easue s poa aly fo geeal aages esposble fo ogasaos. I defes he odos fo he busess ases of pvae ad publ ogasaos. Cash flow Cash flow efes o he puely faal pas of pleeg eleede seves. A posve e eu efles he geeao of exa ash, whh s suffe o ove he equed ash oulays. The oe oo poso s oe whee he aalyss defes how uh oey eeds fag ad whe. A llusave aggegae pefoae s show Cha 5, depg boh ash flow beefs ad ouflow oss fo eah peod. Page 26 of 34 July 2012

27 ASSIST Cha 2: Illusao of aggegae ash flow pa Cash Ouflow Cash Iflow Te Soue: epa 2010 The ash flow aalyss s he foudao fo assessg affodably of veses. Howeve, sakeholdes ae affeed dffeely wh espe o ash, ad ay ases expedues of oe sakeholde s oe o aohe. Ths ay ael ou he aggegae vew of level 1 pefoae easues bu be vsble level 2 pefoae. Theefoe he hgh level vew s o suffe fo deso akg, so aages wll have o fous o he seod level pefoae daos ode o udesad he deals ad he eed fo ao Level 2 pefoae easues A he seod level, he easues beoe oe dealed ad allow oe spef defao of aduse equees ad aduse opos. The peseao of esuls ludes sx ses of has ad ables: Cuulave SER fo eah sakeholde Cuulave ROI fo eah sakeholde Aual ash flow fo eah sakeholde Hgh pa beefs ae he posve pas The fs hee ses ae se he sae foa as desbed above fo he hgh-level pefoae daos. Hgh pa beefs ae he posve pas ha obue he os o he oveall pefoae. Wh vew of asfeg plo poes o oue seves, hese beefs eveal he value poposo he busess odel. Hgh pa oss ae hose os es ha have a sgfa effe o he boo le eu easues.. A sesvy aalyss should defy he espeve pa daos of hghes wegh. The ASSIST ool povdes he wegh fo eah dao. Hgh pa o pefoae s assoaed wh hgh sk. O he egave sde, hese oss us be oolled ghly, as oveus would edage he whole poe. O he gas sde, whe hgh pa beefs ae o ealsed he e pefoae s a sk. Thus, he sesvy aalyss of he odels falaes he defao of sk faos ad he gao of oveall sk. Page 27 of 34 July 2012

28 Glossay ad abbevaos AB: aual beef AC: aual oss A&E: ade & eegees Asses: hese ae ghs o ohe aess o fuue eoo beefs oolled by a ey as a esul of pas asaos o eves. Sofwae a be luded wh hadwae as a asse ASSIST: poe by epa, Telbos ad IRER o assesse ad evaluao ools fo eleede fuded by he ESA Beefs: he oeay value of he gas a aoug peod fo a vese of esoues poduve seves. Beefs a be agble, ad deeed by pe o asaos, o agble, whe shadow pes ay be used o used o esae oeay values. Beefs fo ehealh a aue o seveal sakeholdes, ludg paes, aes, zes, healhae pofessoals, healhae ogasaos ad hd pay payes Capal expedue: expedue o a poe ha has a beef o a ey fo oe ha oe of s aoug peods. A exaple s a oe-off expedue o se up a lage opue o ouaos faly ha s expeed o opeae fo fve yeas befoe eeds o be eplaed Cash flow: he aou of oey ehe pad ou o eeved by a ey a aoug peod, o goup of aoug peods, ad ehe fo apal es o aual opeag aves. Cash flow s ofe lassfed as payes ad eeps CBA: os-beef aalyss CNS: ouy usg seves Cos: he oeay value of he esoues safed a aoug peod o ealse beefs ad eveue fo poduve seves. Coss a be agble, ad deeed by pe o asaos, o agble, whe shadow pes ay be used o used o esae oeay value. Coss of ehealh a ued by seveal sakeholdes, ludg paes, aes, zes, healhae pofessoals, healhae ogasaos ad hd pay payes Depeao: he easue of he os o evalued aou of he eoo beefs of agble fxed asses osued dug a aoug peod. Capal expedue ad he value of leases exludg ees payes, ae usually depeaed eah yea of he useful lfe of he assoaed asses o efle s use eah aoug peod, ad so he edued value a he ed of eah aoug peod. Depeao s a e of oss Desk eseah: as opposed o feld eseah leaue ad ee eseah ae used o ga foao o a spef op ECG: abbevao fo eleoadoga ad eleoadogaph Expedue: he aou of oey spe a aoug peod, ad elaed o he aves he aoug peod, o a goup of aoug peods, usually he faal yea of he ey. Exaples ae he salaes pad o saff ad / o aual lee fees pad o ehealh supples fo he use of he sofwae. Ths ype of expedue s also efeed o as eveue expedue, dsgushg fo apal expedue lke buyg a ew buldg Reveue: he aou of oey eeved a aoug peod, ad elaed o he sales o aves he aoug peod, o a goup of aoug peods, usually he faal yea of he ey. I should o be ofused wh eveue expedue. A exaple s he oey eeved fo seves povded o paes Page 28 of 34 July 2012

29 Ipa faos: pa s a eual despo of a effe. I does o udge o he deo of pa whh a be, posve as well as egave Ivese: expedue o a avy, o poe, o aheve a eu ove s poposed lfe-yle, suh as expedue o a ehealh applao ad hage aagee o aheve a povee healhae qualy, aess o effey. I s ofe opsed of es of apal expedue, ad a also lude addoal o e-alloaed eveue expedue. A exaple s a ehealh poe ha ludes he expedue o hadwae, sofwae ad hagg he lal ad wokg paes eeded o ealse he beefs Dsoug: esaed values of oss ad beefs of a avy o poe ae adused usg a ees ae ha efles a ypal eu ha a be aheved fo a busess, seve avy o publ seve. I s he ehque by whh oeay values fo dffee pos e ae oveed o opaable easues a he e of aalyss. Usually, s absolue oeay values he fuue ha ae edued ode o show he pese value, efleg he oppouy os of e, aly ees, uly fo osupo ow sead of lae. Dsoug s paulaly poa evaluag log-e veses whee he beefs ase uh afe he po of vese expedue. I also eables oss of poes wh dffee lfe yles o be opaed ehealh vese pofle: he apal ad eveue expedue uve ove he lfe-yle of a poposed ehealh vese o lude plag ad develope, pleeao ad opeao ehealh IMPACT: sudy fo he Euopea Cosso o he eoo pa of ehealh ehealh ulsao: easues of he use of ehealh by a appopae u of avy, suh as he ube of ave uses, he ube of asaos, o he ube of lve eods EHR IMPACT: sudy fo he Euopea Cosso o he eoo pa of eleo healh eods ad epesbg syses ESA: Euopea Spae Agey Ex-ae: 'befoehad', 'befoe he eve'. Based o po assupos. A foeas Ex-pos: 'afewads', 'afe he eve'. Based o kowledge of he pas. Measue of pas pefoae Faal esoues: he volue of oey fo he vaous soues of fae fo a vese, o poe. I he oex of ehealh, hey ae used o pay fo he esoues eeded fo a poe, suh as a ICT ea, hadwae ad sofwae, ove s esale Fag: a aagee ha povdes fuds o pay fo a plaed o aual avy, o poe. Two segs fo he fag ehealh sudy ae fag healhae ad fag ehealh. Povdg fuds fo healhae s aheved by hee gee odels, publ oey, suh as fo axao, healh suae, ehe soal o voluay, o de paye. Eah of hese a have ay dffee aagees, as foud he EU Mebe Saes. They all opeae by asfeg fuds fo zes o ebuse healhae povdes, ofe hough hd pay payes. ehealh fag a be pa of ebusee, o lude aess o addoal fuds, ad a be see as a aagee fo pay fo ehealh veses, o poes, ove he whole lfe. I a have seveal phases, suh as fag fo oe-off o o-eug oss of develope, pleeao o hage aagee, ad fag fo ouous vese, suh as a ease aual expedue o opeag he ehealh seves. Fag aagees a have pese aagees fo aagg he sks of a vese o poe Fudg: he po, o pos, of oey ha a be aessed o fae a poposed vese, o poe. I wll be avalable fo he soue of fag, ad o he es ageed fo s use FTE: full e equvales GP: geeal paoes Page 29 of 34 July 2012

30 Ga: asssae he fo of asfes of ash o asses o a eepse eu fo pas o fuue oplae wh ea odos elag o he opeag aves of he eepse. Reveue ad oss should be ahed wh oe aohe. I ay ases, he ga-akg body has he gh o eove all o pa of a ga pad f he eepse has o opled wh he odos ude whh he ga was ade. IAS 20 Aoug fo Govee Gas ad Dslosue of Govee Asssae ses ou fuhe explaao HPO: healh povde ogasao ICT foao ad ouao ehology Ipae: os oues a pae s defed as a hospal pae who oupes a bed fo a leas oe gh he ouse of eae, exaao, o obsevao Iagble beefs: f o-faal faos Make pe: a eoo oep whee he pe of a good o seve s egoaed bewee he ake papas, suh as a buye ad selle Moeay Value: esag oss ad beefs whee pes ae o avalable fo oeal asaos. Tehques suh as shadow pg ad wllgess o pay a used o help esag oeay values Leasg: aess o he use of asses hough a lease, whh s a oa fo a spef asse whee he oweshp of he asse eas wh he ey povdg he fae fo he lease, ad he use pays a eal fee fo he ghs o use he asse. A fae lease asfes os, o all he sks of usg he asse o he use, a opeag lease a shae he sks. IAS 17 Aoug fo leases ses ou fuhe explaao Lfe-yle oss: he oal os of owg o usg fales, asses o seves, expessed as es of apal o eveue expedue eeded fo a vese, o poe, ove s whole e peod. Fo a ehealh poe, a exed fo plag, develope, pleeao ad opeao, ad lude expedue o obsolesee, efeshe, upgadg, ad hages o apay o efle sgfa hage ulsao Lables: hese ae a ees oblgaos o asfe eoo beefs as a esul of pas asaos o eves Moeay us: a u of value ad oey of a ouy, esp. he ao o sadad u e.g. Euo o Dolla Moe-Calo-Sulao: Moe Calo sulao s a ehod fo eavely evaluag a dees odel usg ses of ado ubes as pus. Ths ehod s ofe used whe he odel s oplex, olea, o volves oe ha us a ouple uea paaees NB: e beef s he dffeee bewee beefs ad oss NPV: e pese value Ne pa: ouoe of a CBA o os-effeveess sudy No-faal esoues of eoo value: hages faos of poduo lke e.g. a eduo hous spe o a ask No-faal faos: agble, ofe eooal pas suh as qualy of lfe o oveee Oupae: opposed o pae says fo a day eae, exaao o day ase oly Peae value: a value ha s assued o say osa dug he e sale of he assesse Pefoae easues: wh ASSIST hey ae oupus eablg a deso o he whehe a elehealh poe a be asfeed o a fully opeaoal udeakg Page 30 of 34 July 2012

31 PV: pese value QALYs: qualy adused lfe yeas Real esoues: ludes he people who ae eeded fo a avy, o poe, he equpe ad fales eeded houghou he whole e peod of a poe Relably: deosag ha he opeaos of a sudy suh as he daa olleo poedue a be epeaed, wh he sae esuls. Sesvy aalyss s a poa sue fo esug elably Resoue alloao: esoue alloao s he dsbuo of esoues usually faal - aog opeg goups of people o pogas Rsk: hs s ueay as o he aou of beefs, ludg poeal fo ga ad exposue o loss. ROI: eoo eu o vese ROI. The ROI easue ludes oly eoo esoues, adoally defed as lad, labou, ad apal. I ay saes, suh as saff e ad expedue o ehology, he value of esoues s he oppouy os. The esoues ould be eployed fo ohe puposes. Howeve, he ealsao of he poeal ROI o eus efleed aoug epos s a hallegg aageal ask. Fo exaple, whe egh doos save oe hou a day eah, esoue a he value of oe full e equvale s lbeaed. The deso whehe o dsss oe doo, o assg ohe asks o all doos s sepaae fo he value of beef obseved. The eoo ROI would lude a ga of oe FTE egadless of how he lbeaed esoue s used. The ROI easue s poa aly fo geeal aages esposble fo ogasaos. I defes he odos fo he busess ases of pvae ad publ ogasaos SER: soo-eoo eu SER. The SER s defed as he ao of e beef o oss ove e. Ths easue opses all posve ad egave pas, ludg faal ad ohe eoo esoues, agble hages qualy of lfe ad wok, as well as agble, ofe eooal pas. A posve ao daes a wohwhle edeavou fo a soo-eoo pespeve. A ao of zeo equals a pl beak-eve po a whh he oveall soo-eoo pa s zeo Soal plae: akes a dealsed pespeve ag a axsg value fo soey as a whole Soues of fag: vaous ypes of aagees ha a be avalable o pay fo he oss of a vese, o poe, ove s whole lfe. Exaples lude a ease spedg o ehealh by he ey, a ease he budge fo a eal ICT ea o pay fo a ehealh develope o hage aagee oss; aagees wh baks ad suos o pay fo he addoal apal o eveue expedue of a poe, suh as he Euopea Ivese Bak EIB, leasg aagees ad ehealh supples ha povde seves fo a aual fee ha ludes develope ad apal expedue spead ove he lfe of a oa whee he payes ae lassfed as eveue expedue Sakeholde: dvduals ad ogasaos ha ae avely volved o passvely affeed by a eleede seve, e.g. paes, healhae saff dffee ae ogasaos, hd pay payes, aagee ogasaos, healh auhoes, ehology ad eleouaos povdes, sevg opaes Teleede: a e appled o a age of aves ad seves wh a oo hee of a edal avy suppoed ove a dsae. Hee s used also fo elehealh ad eleoog as well as fo doo o doo eleede Te sees value: a value ha s assued o vay subsaally ove he e hozo of he assesse. U oss: he alulaed os of a gve u of a podu, ofe usg a osg odel Valdy: esablshg oe opeaoal easues fo he oeps beg suded Valdy, eal esablshg a ausal elaoshp, wheeby ea odos ae show o lead o ohe odos, as dsgushed fo spuous elaoshps Valdy, exeal: esablshg he doa o whh a sudy s fdgs a be geealzed Page 31 of 34 July 2012

32 Value added: he aou of he e beefs, so beefs us oss, usually of a poposed poe o vese, ove a spefed e, ad ofe easued by os beef aalyss ad dsoued o a e pese value. I a be used o defy a opal opo avalable o a ey. Exaples ae whee he esaed beefs o paes ad healhae povdes fo a ehealh poe exeed s esaed oss ove he poe s vese lfe-yle Veo-playes: who a, hough he polal o ogasaoal powe, sop a ave, as well as dve, ad who have a s ees oveso of plo applaos o oue seves WTP: wllgess o pay, a eoo oep fo esag oeay values fo whh o ake pe exss Page 32 of 34 July 2012

33 Bblogaphy 1. Aeweh, E. ad N. de Keze, A veoy of evaluao sudes of foao ehology healh ae: eds evaluao eseah Sud Healh Tehol Ifo, P 2: p Olse, J.A. ad R. Sh Who Have Bee Asked o Value Wha? A Revew of 54 Wllgess-o-Pay Suveys Healh Cae Y, R.K., Case sudy eseah - desg ad ehods. Appled Soal Reseah Mehods Sees2003: Thousad Oaks. 4. UK HM Teasuy, The Gee Book - Appasal ad Evaluao Ceal Govee, 2003, TSO: Lodo. 5. Godo, C. LabfoCulue Glossay. [ed ]; Avalable fo: hp:// 6. Joes, T., E-healh - faal ad eoo ase sudes, A.o.C.C.A. ACCA ad E.C.I.S.D. EC-ISD, Edos Davs, M., Sage 6 Hospals: The Jouey ad he Aoplshes, 2007, HIMSS Aalys. 8. Gado, T., e al., Effe of eleo healh eods abulaoy ae: eospeve, seal, oss seoal sudy. BMJ, : p Xu, S., Advag eu o vese aalyss fo eleo healh eod vese. Ipas of paye ehass ad publ eus. J Healh If Maag, : p Poe, M., The fve opeve foes ha shape saegy. Havad Busess Revew, 20081: p Poe, M.E. ad E.O. Tesbeg, Redefg healh ae : eag value-based opeo o esuls2006, Boso, Mass.: Havad Busess Shool Pess. xv, 506 p. 12. The Lew Goup, Assesse of Appoahes o Evaluag Teleede, 2000, Offe of he Asssa Seeay fo Plag ad Evaluao, US Depae of Healh ad Hua Seves. 13. ASSIST, Suay of Teleede Evaluao Iaves, 2010, epa: Bo. 14. Balow, J., e al., A sysea evew of he beefs of hoe eleae fo fal eldely people ad hose wh log-e odos. J Teleed Teleae, : p Beefs Realsao & Ahevee Ieaoal Newok, Whe pape: easug beefs a healhae oex, Cusak, C.M., e al., Healh Ifoao Tehology Evaluao Toolk: 2009 Updae, 2009, Agey fo Healhae Reseah ad Qualy - U.S. Depae of Healh ad Hua Seves. 17. Davalos, M.E., e al., Eoo evaluao of eleede: evew of he leaue ad eseah gudeles fo beef-os aalyss. Teleed J E Healh, : p Duod, M.F., Mehods fo he eoo evaluao of healh ae pogaes. 3d ed. Oxfod edal publaos2005, Oxfod ; New Yok: Oxfod Uvesy Pess e-healh Ipa. Sudy o Eoo Ipa of e-healh. [ed ]; Avalable fo: hp:// 20. EUeHTA. HTA Coe Model Hadbook [ed ]; Avalable fo: hps://fo.sakes.f/haoe/hadbook.hl. 21. Kdhol, K., e al., Faewok fo ASsesse of Teleede - FAST - A faewok based o he MehoTeleed poe, So, R.E., e al., Naoal Telehealh Ouoe Idaos Poe [NTOIP] - Poe Ifoao Doue ad a Syhess of Telehealh Ouoes 2003, Healh Teleas U, Uvesy of Calgay. 23. Ued Saes Geeal Aoug Offe, Measug Pefoae ad Deosag Resuls of Ifoao Tehology Iveses, EHR IMPACT, Mehodology fo evaluag he soo-eoo pa of eopeable EHR ad epesbg syses, 2008: Bo. 25. Euopea Spae Agey, Saee of Wok fo he Resuls Assesse of ESA-sposoed Teleede Poe, vo de Shulebug, J., e al., Deushe Epfehluge zu gesudhes-ökooshe Evaluao - de ud akualsee Fassug des Haoveae Koses. Gesudh öko Qual aag, : p Röhg, P., ICT Ivesoe begüde - Wshaflhkesbeehuge de WBe-Kozep, 2009, WBe: Wea. 28. Whe House Offe fo Maagee ad Budge, Cula No. A-94 - Gudeles ad Dsou Raes fo Beef-Cos Aalyss of Fedeal Pogas, Page 33 of 34 July 2012

34 29. Bo, O., A. Tesappe, ad D.P. Peshe, Modelg, Sulag, ad Vsualzg Ifoao Syse Afas Healhae - A Geeal Appoah o Aalyss ad Desg, Wesbook, J.I., e al., Mulehod evaluao of foao ad ouao ehologes healh he oex of wked pobles ad sooehal heoy. J A Med Ifo Asso, : p Kukaz, Z., e al., Qualave Evaluao. De Eseg de Paxs2008, Wesbade: VS Velag fü Sozalwsseshafe. 32. Gae, ehealh fo a Healhe Euope! oppoues fo a bee use of healhae esoues, Caada Healh Ifoway, Dagos Iagg Beefs Evaluao, Soeso, C., M. Duod, ad P. Kaavos, Esug value fo oey healh ae - The ole of healh ehology assesse he Euopea Uo, Obsevaoy Sudes Sees, Euopea Obsevaoy o Healh Syses ad Poles, Edo Jayes, E., Pobably Theoy: The Log of See2003: Cabdge Uvesy Pess. 36. de Fe, B., Theoy of pobably Vol , New Yok: J. Wley & Sos, I.,. 37. ACCA, Teleadology Ialy: beefs fo a eleede ewok oeg ho paes, geeal paoes ad healhae povde ogasaos, So, R.E., e al., Telehealh ouoes: a syhess of he leaue ad eoedaos fo ouoe daos. J Teleed Teleae, Suppl 2: p Yok Healh Eoos Cosou, Evaluao of he Teleae Develope Pogae, va Sel, H.F. ad E. Buskes, Copaso of he SF-6D ad he EQ-5D paes wh ooay hea dsease. Healh Qual Lfe Ouoes, : p Cola, A. ad D. Cafloe, geasat - Fal Repo, Page 34 of 34 July 2012