Developing an Employee Evaluation Management System: The Case of a Healthcare Organization

Transcription

1 FINANCIAL ENGINEERING LABORATORY Techncal Unversty of Crete Developng an Employee Evaluaton Management System: The Case of a Healthcare Organzaton Evangelos Grgorouds Constantn Zopounds Workng Paper 20 August 200 Workng papers are n draft form. They are dstrbuted for purposes of comment and dscusson only. The papers are expected to be publshed n due course, n revsed form. They may not be reproduced wthout permsson of the copyrght holder(s). Copes of workng papers are avalable at

2 FINANCIAL ENGINEERING LABORATORY Department of Producton Engneerng & Management Techncal Unversty of Crete Lst of Workng Papers Edtoral Commttee Constantn Zopounds, Mchael Doumpos, Fotos Pasouras Modellng bankng sector stablty wth multcrtera approaches C. Gagans, F. Pasouras, M. Doumpos, C. Zopounds 20 Bank productvty change and off-balance-sheet actvtes across dfferent levels of economc development A. Lozano-Vvas, F. Pasouras Developng multcrtera decson ad models for the predcton of share repurchases D. Androsopoulos, C. Gagans, F. Pasouras, C. Zopounds 20 Developng an employee evaluaton management system: The case of a healthcare organzaton E. Grgorouds, C. Zopounds

3 Developng an employee evaluaton management system: The case of a healthcare organzaton E. Grgorouds * and C. Zopounds Techncal Unversty of Crete, Department of Producton Engneerng and Management Unversty Campus, Kounoupdana, 7300 Chana, Greece Abstract The long-term vablty of a busness organzaton depends on ts ablty to evaluate the performance of the employees and to examne the contrbuton of ts personnel n achevng the assessed goals. In ths context, the evaluaton of employees may provde a quanttatve measure of ther apprasal amng at determnng the degree of conformance between the job output and the defned standards. The man am of ths study s to present the development of an employee evaluaton system n a healthcare organzaton. The proposed approach s based on multcrtera analyss and consders the complexty of the dfferent job profles. In partcular, the appled quanttatve model consttutes a varant of the UTA method, takng nto account the strategy of the organzaton and the preferences of the management. The man advantage of ths approach focuses on ts ablty to use absolute performance measures and to develop an evaluaton system that can handle qualtatve (ordnal) nformaton. Moreover, usng the proposed approach, employees are evaluated on a set of dfferent but specfc job dmensons, provdng the ablty to perform dfferent types of comparson analyses. Keywords: Health servce, Multcrtera analyss, Case study, Performance measurement, Balanced scorecard, Busness strategy. Introducton Employee evaluaton or employee performance evaluaton s a process ncorporated n every actvty of modern busness organzatons. Usually, through ths process, the organzaton evaluates how well employees perform ther job when compared to a set of standards, and communcates the evaluaton results to those employees (Maths and Jackson, 2007). The mportance of an effectve employee evaluaton system s unversally recognzed snce t can promote both the nsttutonal development of the organzaton and the personal development of employees. The mpact of employee performance or, more generally, the mpact of human resource management on organzatonal performance has been studed n numerous prevous research efforts (see for example Becker and Gerhart, 996; Ostroff and Bowen, 2000; Guest et al., 2003; Sels et al., 2006). In fact, the ablty of an organzaton to evaluate the performance of ts employees and examne the contrbuton of ts personnel n achevng the assessed goals s consdered crucal for ts long-term vablty. Usually, an employee evaluaton system s wdely used for admnstratng wages and salares, gvng performance feedback, and dentfyng ndvdual strengths and weaknesses. As noted by Maths and Jackson (2007), an employee evaluaton has two general roles n organzatons: * Correspondng author: Tel , Fax , Emal: vangels@ergasya.tuc.gr

4 Makng admnstratve decsons about employees (compensaton, promoton, dsmssal, downszng, layoffs, etc). Identfyng and plan employees growth opportuntes (dentfy strengths or areas for growth, coach, develop career, etc). Often, these two dfferent roles are conflctng (Maths and Jackson, 2007), whle employee evaluaton systems are usually strongly related wth other human resource management actvtes (e.g. employee selecton or recrutment). Moreover, employee performance evaluaton s ncorporated n many busness operatons, whle subjectvty s a crtcal pont n every employee evaluaton system. Fnally, many factors can affect the performance of employees (e.g. sklls, motvatons, support they receve, nature of work, relatonshp wth organzaton). For all these reasons, developng, mantanng, and mprovng an employee evaluaton system s nether easy nor straghtforward. When developng an employee evaluaton system, the management of the organzaton should take a seres of mportant decsons: Who should desgn the evaluaton process? Who should evaluate whom? Who should revew evaluaton results? How these results could be exploted? Several alternatves may be found for each one of the prevous decsons. For example, the assessment of the evaluator may nclude the followng possbltes (Maths and Jackson, 2007):. Supervsory ratng of subordnates (supervsors who rate ther employees) 2. Employee ratng of managers (employees who rate ther superors) 3. Team/Peer ratngs (team members who rate each other) 4. Outsde raters (ratngs based on outsde sources) 5. Self-ratngs (employees who rate themselves) 6. Multsource or 360 degree ratng (feedback from all around the employee,.e. superors, subordnates, peers, customers, or self-apprasal) Moreover, dfferent methods can be used for measurng employee performance. These methods may be categorzed nto the followng major groups (Maths and Jackson, 2007): Category ratng methods (graphc ratng scale, checklst) Behavoral/Objectve methods (behavoral ratng approaches, management by objectves) Comparatve methods (rankng, forced dstrbuton) Narratve methods (crtcal ncdent, essay, feld revew) Multple crtera decson analyss approaches have been also appled n the employee evaluaton problem (see for example Spyrdakos et al., 2000; Saaty, 2005; Cheng and L, 2006). It should be emphaszed that employee performance evaluaton s drectly lnked wth the strategy of the organzaton. As shown n Fgure, strategy should drve performance management practces, lke the dentfcaton of expected performance levels, the measurement of ndvdual performance, the communcaton of evaluaton results, etc. On the other hand, performance outcomes (e.g. productvty, advancement, dscplne, pay rases) are lnked to organzatonal results, whch n turn are the man feedback for the strategy of the organzaton. The man objectve of the presented study s to develop an employee evaluaton system n a healthcare organzaton. The proposed approach s based on multcrtera analyss and consders the complexty of the dfferent job profles. Furthermore, n order to take nto account the strategy of the organzaton and the preferences of the management, a varant of the UTA method s proposed. The man advantage of ths approach s the ablty to use absolute performance measures and develop an evaluaton system that can handle qualtatve 2

5 (ordnal) nformaton. Fnally, usng the proposed approach, employees are evaluated on a set of dfferent but specfc job dmensons, provdng the ablty to perform dfferent types of comparson analyses. Organzatonal Strategy Performance Management Practces Employee Performance Performance Management Outcomes Organzatonal Results Fgure : Lnkng performance management wth strategy and results (Maths and Jackson, 2007) The presented real-world applcaton concerns a prvate general hosptal n Greece, whch has been sgnfcantly grown durng the last years. The applcaton s used n order to llustrate the methodology appled for the development of the employee evaluaton system and pnpont ts dstngushed characterstcs n a healthcare organzaton. The paper s organzed n four more sectons. Secton 2 presents the modfed UTASTAR model and gves the general mplementaton steps for the development of the proposed employee evaluaton system. The detals of the real-world applcaton are gven n secton 3, ncludng the presentaton of the healthcare organzaton and the assessment of the evaluaton crtera. Secton 4 presents the results of the multcrtera method, focusng on the crtera weghts and the employee evaluaton scores. Fnally, secton 5 summarzes some concludng remarks, and dscusses potental extensons of the research. 2. Methodology 2. Multcrtera method The multcrtera model appled n the present study s based on the UTASTAR method but consders addtonal crtera mportance preferences. The man am of the model s to analyze the behavor and the cogntve style of the Decson Maker (DM) (.e. to mprove the DM s knowledge about the decson stuaton and hs/her own judgment polcy that s entaled for a consstent decson to be acheved). The UTASTAR method s a regresson based approach that adopts the aggregatondsaggregaton prncples. Proposed by Sskos and Yannacopoulos (985), t s a varaton of the UTA method (Jacquet-Lagrèze and Sskos, 982), whch ams at nferrng a set of addtve value functons from a gven rankng on a reference set of actons (alternatves). In the context of the method, the addtve value functon u s assumed to have the followng form: n u( g ) = u ( g ) σ + + σ () = under the followng normalzaton constrants: u( g* ) = 0 n * u( g ) = = =, 2,, n (2) * where g = { g, g2,, g n } s the set of crtera, [ g, g * ] s the crteron evaluaton scale wth g * and g the worst and the best level of the -th crteron, u ( =, 2,, n) are the margnal * 3

6 value functons normalzed between 0 and, σ + and σ are the overestmaton and the underestmaton error, respectvely, and n s the number of crtera. It should be noted that the UTASTAR method, as an aggregaton-dsaggregaton approach, s focused on the nference of preference models from gven global preferences (Jacquet- Lagrèze and Sskos, 200; Sskos et al., 2005). Jacquet-Lagrèze and Sskos (200) propose the use of a set of reference actons n order to derve these global preferences. Usually, ths set can be a set of past decson alternatves, a subset of decson actons (partcularly when the orgnal set of alternatves s very large), or even a set of fcttous actons, consstng of performances on the crtera, whch can be easly judged by the DM (.e. the management of the healthcare organzaton) to perform global comparsons. In any case, these gven global preferences consttute a weak-order preference structure on a set of actons, so the problem s to adjust addtve value or utlty functons based on multple crtera, n such a way that the resultng structure would be as consstent as possble wth the ntal structure. Followng the UTASTAR algorthm (Sskos and Yannacopoulos, 985; Sskos et al., 2005), the man steps of the appled multcrtera method n the presented employee evaluaton problem are as follows:. Assumng a consstent famly of evaluaton crtera g = { g, g2,, g n }, a set of employees that can play the role of reference acton AR = { a, a2,, am}, and the evaluaton g( a k) of every employee a k accordng to each crteron g, the management of the organzaton s asked to rank the set A R from the best to the worst acton (.e. employee). A R s also rearranged n such a way that a s the head of the rankng (best employee) and a m ts tal (worst employee). Snce the rankng has the form of a weak order R, for each par of consecutve actons ( ak, a k + ) t holds ether ak a k + (preference) or ak a k + (ndfference). * 2. In the next step, the nterval [ g, g * ] s cut nto ( α ) equal ntervals n order to estmate the correspondng margnal value functons n a pecewse lnear form. Followng Jacquet-Lagrèze and Sskos (982), the margnal value of an acton a k s j j approxmated by a lnear nterpolaton, and thus, for g( ak) [ g, g + ] the followng relatonshp holds: g ( a ) g u g a u g u g u g j [ ( k)] = ( j k j+ j ) + ( ) ( ) j j + g g In addton, at the end of ths step, the margnal values u( g ) should be wrtten n terms of the model varables: u( g) = 0 =, 2,, n j j u ( g ) = wt =,2,, n and j = 2,3,, α t= where the transformaton varables w j are ntroduced n order to reduce the sze and the complexty of the model, snce the monotoncty condtons for u can be replaced by non-negatve constrants for w j (Sskos and Yannacopoulos, 985) 3. Usng formula () and ntroducng two error varables σ + and σ for each par of consecutve actons n the rankng, the followng expressons should be wrtten: ( a, a ) = u[ g( a )] u[ g ( a )] (5) k k+ k k+ (3) (4) 4

7 4. The DM has the ablty to express addtonal preferences regardng the mportance of * crtera. Snce n UTASTAR u( g ) have the role of mportance coeffcents (weghts), f formula () s wrtten n a weghted form, these addtonal preferences may be modeled as follows: If crteron g h s consdered at least as mportant as crteron α h αq * * h ( h ) q ( q ) ht qt t= t= u g u g w w g q then: (6) If crtera gh gh g are consdered at least as mportant as crtera gq gq g, wth gh g Q = then: α h αq * * uh ( gh ) uq ( gq ) wht wqt gh gh gq gq gh gh t= gh gh t= (7) 5. In ths step, the followng Lnear Program (LP) s solved: m + [mn] z = [ σ ( ak) + σ ( ak)] k = subject to ( ak, ak+ ) δ f ak ak+ k ( ak, ak ) = + 0 f ak ak+ n α wj = = j= addtonal constrants of type (6) or (7) + wj 0, σ ( ak ) 0, σ ( ak ) 0, j and k wth δ beng a small postve number. 6. The fnal step concerns the stablty analyss of the estmated results, where the exstence of multple or near optmal solutons of the LP (8) s examned. In case of non unqueness, the mean addtve value functon of those (near) optmal solutons may be calculated whch maxmze the objectve functons: α * ( ) = j =, 2,, j= u g w n (9) on the polyhedron defned by the constrants of the LP (8) bounded by the new constrant: m + * σ ( ak) + σ ( ak) z + ε (0) k = where * z s the optmal value of the LP (8) and ε s a very small postve number. An analytcal presentaton of the UTASTAR algorthm s gven by Jacquet-Lagrèze and Sskos (200) and Sskos et al. (2005) ncludng dscusson on the stablty analyss (postoptmalty analyss) of the results and several extensons and varants of the method. Fnally, t should be emphaszed that crtera mportance preferences may be conflctng, and thus, the constrants (6) and/or (7) should be used wth care (the examnaton of these preferences for possble nconsstences s consdered necessary). On the other hand, addng these constrants n LP (8) may ncrease the stablty of the results, although the fttng of the model may be decreased. A dscusson about the effect of such constrants n the context of (8) 5

8 robustness analyss may be found n Sskos and Grgorouds (200). Ιt s also mportant to note that the weghts n such addtve models are value trade-offs among the assessed crtera. 2.2 Consstency evaluaton As mentoned before, the modfed UTASTAR algorthm s able to provde a preference model based on the aforementoned data. Ths evaluated preference model s as consstent as possble wth the DM s preferences. The consstency evaluaton framework appled n the current study adopts the general phlosophy of aggregaton-dsaggregaton approaches. As shown n Fgure 2, the whole procedure starts wth the analyss of the decson problem, the modelng of the crtera, and the collecton of necessary decson data, ncludng the DM s global preferences (.e. rankng of the reference set of actons). The next steps refer to the development and the mplementaton of the multcrtera model. In case of consstency between the assessed preference model and the DM s global preferences (.e. between DM s and model rankngs) the preference model s accepted and model results are extrapolated nto the whole set of actons A. Otherwse, the DM s asked f he/she s wllng to accept the results of the model and modfy hs/her ntal rankng (partcularly when dfferences between the aforementoned rankngs are relatvely small). If the DM s wllng to modfy hs/her preferences, then agan the model results are accepted and the procedure ends. In the opposte case, a learnng process referrng to the re-modelng of the decson problem takes place. Ths learnng process s termnated when complete consstency s acheved, and ncludes a seres of feedbacks concernng the modfcaton of (Sskos et al., 993):. The problem formulaton or the crtera modelng. 2. The DM s judgment polcy. 3. The multcrtera model (.e. reference set, model parameters, alternatve optmalty crtera or post-optmalty analyss technques). Sskos et al. (993) also propose a trade-off process, whch s based on the acceptance of the DM s weak order and the a posteror modfcaton of the assessed preference model. Ths trade-off process ams to elmnate nconsstences usng vsual technques and guded by an expert system. Problem formulaton and crtera modelng No DM s judgment polcy (.e. rankng) No Multcrtera model development No DM accept model results? Yes No Multcrtera model results (ncludng model rankng) Same rankng? Yes Accept results and extrapolate nto the whole set of actons Fgure 2: Consstency evaluaton framework 6

9 2.3 Implementaton process The mplementaton process appled n the employee evaluaton problem s schematcally presented n Fgure 3. Although the presented applcaton concerns a healthcare organzaton, ths procedure may be adopted by every busness organzaton, consderng, however, ts partcular characterstcs. The frst step of the mplementaton procedure refers to the prelmnary analyss of the health care organzaton. Durng ths stage, mportant nformaton about the structure and the culture of the organzaton should be examned, so that organzaton s vson and strategy may be taken nto account n the next steps of the employee evaluaton process. Addtonally, analytcal nformaton about job descrptons should be collected, n order to assst n the assessment of the employee evaluaton crtera. The next steps concern the determnaton of the evaluaton process and the development of the evaluaton methodology. Durng ths stage mportant decsons about the sources/herarchy of the evaluaton process should be taken, snce several alternatve approaches may be used (e.g. self-evaluaton, 360 degree evaluaton). In the present study, the next lne supervsor or the department head s chosen as the man evaluator manly because the healthcare organzaton does not have a prevous systematc employee evaluaton experence. On the other hand, the development of the evaluaton methodology concerns the assessment of the evaluaton crtera and scales (see secton 3.2), the development of the evaluaton forms, as well as the collecton of addtonal preferences expressed by the management of the organzaton about the mportance of the evaluaton crtera (see secton 2.). The last two steps of the mplementaton process refer to the data collecton and analyss. Usng the forms developed n the prevous step, employee evaluaton data are collected through an nternal survey. Durng ths survey a drect communcaton between the evaluator and the employee s mantaned, supervsed by the management of the organzaton. Fnally, the analyss of the collected data s based on the aforementoned multcrtera model and the man results are able to determne potental mprovement actons. Prelmnary analyss Evaluaton process Evaluaton methodology Data collecton Data analyss and results Fgure 3: Employee evaluaton process 7

10 3. Applcaton 3. The healthcare organzaton The healthcare organzaton of the study s a prvately-owned general hosptal located n Chana, Greece. It was offcally founded n 977 as an obstetrcs-gynecology Clnc, but t has been sgnfcantly grown durng the last twenty years. The organzaton now operates as a modern dagnostc, therapeutc, surgcal, and research Center wth a capacty of 70 beds. The facltes of the organzaton occupy more than 3,200 m 2, whle sgnfcant nvestments have been made n medcal equpment durng the last fve years. In order to provde ntegrated medcal servces, the general hosptal operates 5 man departments and more than 35 medcal unts:. Pathology department (general pathology unt, ncludng pneumonology, obesty clnc, sleep clnc, ant-smokng clnc, and bronchoscopy, pathologcal oncology unt, cardology unt). 2. Surgery department (general surgery, obstetrcs/gynecology, orthopedcs, urology, ophthalmology, plastc surgery, neurosurgery, otolaryngology). 3. Specal unt department (ntensve care unt, hgh care unt, one day clnc, emergency room, outpatent clnc, patent transfer unt). 4. Clncal laboratores (endoscopy, cardology, neurology, pneumonology, urology, otolaryngology). 5. Dagnostc laboratores (MRI, 64-slce CT, radology/radodagnostc laboratory, ultrasound laboratory, mammography, osseous densty/osteopoross, mcrobology, bochemstry, endocrnology, mmunology, hematology, cardac catheterzaton laboratory, dgtal angography, gamma camera). Furthermore, the hosptal owns 4 ambulances, 2 of whch comprse moble unts, whle there are 6 operatng rooms, 3 delvery rooms, and 9 beds n the ntensve care and hgh care unts. Durng last year the hosptal was able to provde prmary and secondary level medcal servces to approxmately 50,000 patents and to perform more than 44,000 lab tests. The personnel workng n the hosptal accounts more than 00 employees and refers to dfferent job postons (medcal, nursng, admnstratve, auxlary, and techncal), whle t s mportant to menton that the organzaton cooperates wth more than 00 external physcans. The organzaton s vson s focused on the provson of hgh qualty dagnostc and operatve servces and the coverage of medcal needs of western Crete resdents, as well as vstors (both foregn and Greek). Thus, the recent strategy objectves nclude: Investment on modern medcal equpment Adopton of a customer-focused phlosophy Promoton of medcal research 3.2 Evaluaton crtera Several performance dmensons may be consdered when assessng the set of employee evaluaton crtera. These crtera may nclude the quantty, qualty or tmelness of output, the presence at work, and the cooperatveness, snce performance evaluaton focuses on how much the employees contrbute to the organzaton (Maths and Jackson, 2007). However, t s not possble to assess a unversal set of employee evaluaton crtera because each job poston has specfc job performance dmensons. The nformaton that usually managers receve about the performance of employees can be of the followng dfferent types (Maths and Jackson, 2007): 8

11 . Trat-based nformaton (subjectve character trats lke ntatve, creatvty, etc) 2. Behavor-based nformaton (specfc behavors that lead to job success) 3. Results-based nformaton (what the employee has done or accomplshed) The assessment of the evaluaton crtera s based on prevous relevant studes (see for example DeCenzo and Robbns, 2005; Torrngton, et al., 2008; Glddon, 200), as well as on an nteractve communcaton process wth the management of the healthcare organzaton. It should be emphaszed that durng the assessment process, sgnfcant effort has been made so that the evaluaton crtera are adapted n the partcular characterstcs of the hosptal,.e. reflect the organzaton s vson and take nto account the dfferent aspects of job postons. The evaluaton crtera are grouped nto the followng man dmensons (Fgure 4):. Work content: employee sklls, wllngness to be nformed and follow progress on work subject, qualty of work output, and leadershp. 2. Work practce: employee consstency, relablty, and ntatves. 3. Work effcency: on tme completon of tasks, employee s response when workng under pressure, receptvty to gudance, and adaptablty. 4. Work qualty/communcaton: patent orentaton, wllngness to mprove organzaton s mage, cooperaton wth admnstratve departments, and communcaton wth patents. Overall employee evaluaton Work content Work practce Work effcency Work qualty/ Communcaton Sklls Consstency On tme task s completon Patent orentaton Inform/follow progress Relablty Work under pressure Organzaton s mage Qualty of work output Intatve Receptvty to gudance Cooperaton Leadershp Adaptablty Communcaton wth patents Evaluaton crtera common to all job postons Evaluaton crtera that dffer accordng to each job poston Fgure 4: Employee evaluaton crtera As shown n Fgure 4, most of the evaluaton crtera are common to all employees, although they are adapted to the specal characterstcs of dfferent job postons (e.g. the meanng of work qualty may dffer f the evaluaton concerns nursng, admnstratve, or techncal personnel). Ths common set of crtera gves the ablty to develop a unform evaluaton framework and to perform addtonal comparson analyses. However, there are some crtera that dffer accordng to each job poston. For example, the attrbute of leadershp concerns only the heads of the departments. These crtera have been measured usng a 5-pont ordnal scale of the followng type: Poor, Far,, Very good,. 9

12 4. Results 4. Applyng the multcrtera model The multcrtera model presented n secton 2. s used n order to evaluate the employees of the healthcare organzaton. The proposed approach s focused on the evaluaton of the DM s preference system, as ths s reflected n the estmated addtve/margnal value functons and crtera weghts. In order to apply the varaton of the UTASTAR method, a set of reference actons s necessary n order to derve these global preferences. Snce the orgnal set of actons s very large, a set of characterstc scenaros have been developed (.e. fcttous employees wth dfferent performance levels on the assessed evaluaton crtera). As noted by Sskos et al. (999) the followng should be taken nto account when selectng the reference set: the actons n the reference set should be famlar to the DM so as the expresson of hs/her global judgment polcy comes from a known stuaton, and the reference set should reflect the global mage of the problem state. In the present survey, specal expermental desgn technques were used durng the process of developng these scenaros, and partcularly the orthogonal desgn concept, so that to avod domnated actons (.e. fcttous employees). Other alternatve approaches for the selecton of the reference set may also be consdered (see for example cluster analyss technques n Tabachnck and Fdell, 989 and graph approaches n Sskos et al., 999). In the present study, n addton, the management of the hosptal decded to evaluate the followng 8 dfferent job poston:. Fnancal department (managers) 2. Nursng department (managers) 3. Customer servces (CS) department/secretarat (managers) 4. Fnancal department (personnel) 5. Nursng department (personnel) 6. Lab personnel 7. Customer servces (CS) department/secretarat (personnel) 8. Techncal personnel Based on the aforementoned approach, a set of 8 dfferent characterstc scenaros for each one of the prevous job postons was developed. Then the management of the hosptal was asked to rank these scenaros and express addtonal crtera mportance preferences for each job poston. These preferences, whch are summarzed n Table, reflect the strategy of the organzaton, and refer to the assessed evaluaton dmensons or the analytcal evaluaton crtera. Based on the prevous nformaton the modfed UTASTAR method was appled n order to evaluate the DM s preference system. The consstency of the estmated preference model was examned, comparng the ntal rankng of scenaros gven by the DM to the rankng provded by the multcrtera method. In the present study, the consstency of the estmated preference models was relatvely hgh. As shown n Fgure 5, Kendall s tau between the aforementoned rankngs s relatvely hgh, varyng from to n the dfferent scenaros that were examned (wth an average of 0.934). In every case the DM accepted the results of the model. 0

13 Job poston Table : Crtera mportance preferences for dfferent job postons Importance preferences All The most mportant dmenson s the work content The most mportant crteron s the qualty of the work output Fnancal/Nursng/Customer Servce department (managers) Fnancal personnel Customer Servce department (personnel) The second most mportant dmenson s the work practce (wth all crtera havng equal mportance) The thrd most mportant dmenson s the work effcency (wth work under pressure beng the most mportant crteron) The least mportant dmenson s the work qualty/communcaton The second most mportant dmenson s the work qualty/ communcaton (wth patent orentaton beng the most mportant crteron) The thrd most mportant dmenson s the work practce The least mportant dmenson s the work effcency Nursng/Lab personnel The second most mportant dmenson s the work practce The thrd most mportant dmenson s the work qualty/ communcaton (wth communcaton wth patents beng the most mportant crteron) The least mportant dmenson s the work effcency Techncal personnel The second most mportant dmenson s the work practce The thrd most mportant dmenson s the work effcency The least mportant crteron s the cooperaton Techncal personnel 0.98 Customer Servce personnel Nursng/Lab personnel Fnancal personnel Fnancal/Nursng/Customer Servce dept (managers) Fgure 5: Kendall s tau for the dfferent scenaros Fnally, a stablty analyss was performed for the provded results, usng the post-optmalty approach presented n secton 2.. The fnal results nclude the estmated value functons for every crteron and job poston (an example of the estmated margnal value functons for the managers of the fnancal and customer servce departments s gven n the appendx), the weghts of the evaluaton crtera, and the overall and margnal evaluaton scores for every employee (after extrapolatng the results to the whole set of employees).

14 4.2 Crtera weghts One of the most mportant results provded by the multcrtera method refers to the crtera weghts, whch s an mportant facet of the DM s preference system. The estmated weghts respect the preferences expressed by the management of the healthcare organzaton (see Table ), whle they are able to reflect the vson and the strategy of the hosptal. The estmated weghts regardng the managers of the hosptal s departments are shown n Table 2 and the fndngs may be summarzed n the followng:. The crtera weghts are almost dentcal for these job postons, snce the expressed mportance preferences do not dffer for the departments managers (see Table ). The only excepton refers to the crtera of cooperaton (t s not taken nto account for the managers of the nursng department) and communcaton wth patents (t s not taken nto account for the managers of the fnancal and customer servce departments). 2. The most mportant evaluaton crteron for these postons s the qualty of the work output wth a relatve mportance of 5.62%. The crteron of sklls s also consdered mportant (weght 9.40%). These results are consstent wth the man strategc objectve of the organzaton for provdng hgh qualty medcal servces. 3. On the other hand, the evaluaton crtera wth lower relatve mportance are the adaptablty (weght 4.02%) and the mprovement of organzaton s mage (weght 4.05%). However, t should be noted that the majorty of the crtera weghts s relatvely balanced, snce n general there are no sgnfcant dfferences among them. 4. As shown, the management of the hosptal places greater emphass on the work content and practce rather on the work effcency and communcaton, probably because of the pressure to acheve specfc results n a daly bass. Crtera Table 2: Crtera weghts for departments managers (%) Fnancal department (managers) Customer Servce department (managers) Nursng department (managers) Sklls Inform/follow progress Qualty of work output Leadershp Consstency Relablty Intatve On tme tasks completon Work under pressure Receptvty to gudance Adaptablty Patent orentaton Organzaton s mage Cooperaton Communcaton wth patents

15 Smlarly, the estmated crtera weghts for the rest of the job postons are presented n Table 3. Accordng to these results, the most mportant fndngs may be focused on the followng:. The most mportant crteron for all these job postons s the qualty of the work output wth a weght varyng from 6% to 27% approxmately. Moreover, the crteron of adaptablty s the least mportant crteron (weght 4% approxmately for all job postons). These results are consstent wth the preferences expressed by the DM regardng the mportance of the evaluaton crtera. 2. For the personnel workng n the fnancal department, the crteron of sklls and the dmenson of work practce (consstency, relablty, and ntatve) are also consdered mportant (weght 8%). On the other hand the crteron referrng to the mprovement of organzaton s mage s consdered less mportant wth a relatve mportance of 4.05%. 3. Regardng the personnel workng n the customer servce department, the patent orentaton s also consdered mportant (weght.90%). 4. For the personnel workng n the nursng and the lab departments the crteron of communcaton wth patents and the dmenson of work practce have also a relatvely hgh mportance (weghts from 8.3% to 9.36%). 5. Concernng the techncal personnel, the sklls and the wllngness to be nformed and follow progress on ther work subject are also mportant wth a weght of 2.66% and 0.08%, respectvely. On the contrary, the management of the hosptal gves lower mportance to the cooperaton crteron for ths partcular job poston (weght 4.2%). Crtera Table 3: Crtera weghts for other job postons (%) Fnancal personnel Customer Servce personnel Nursng/Lab personnel Techncal personnel Sklls Inform/follow progress Qualty of work output Leadershp Consstency Relablty Intatve On tme tasks completon Work under pressure Receptvty to gudance Adaptablty Patent orentaton Organzaton s mage Cooperaton Communcaton wth patents

16 4.3 Evaluaton scores The extrapolaton of the prevous results to the whole set of employees gves the ablty to calculate overall and margnal scores for every employee. These scores are bascally the overall and margnal values estmated by the modfed UTASTAR model and normalzed between 0 and. Table 4 presents the number of employees evaluated n each job poston, the average overall evaluaton scores for these employees, and the correspondng mnmum/maxmum value and standard devaton. As shown, there are sgnfcant dfferences n these results. In partcular, the overall evaluaton of the fnancal department managers and the fnancal, nursng, and lab personnel s relatvely hgh, varyng from 0.8 to 0.85 approxmately. On the other hand, the managers and the lower level personnel of the customer servce department, as well as the techncal personnel have a rather low overall evaluaton score (from to 0.536). However, the hgh standard devaton observed n the techncal personnel shows that there are sgnfcant dfferences n the overall scores of employees workng n ths department (.e. some employees have a relatvely hgh evaluaton score, whle the overall score of others s extremely low). The same varablty s also observed for the managers of the nursng department, but the relatve average score s not low. The prevous average evaluaton scores show that the management of the hosptal s rather satsfed from the departments managers and the personnel workng n the fnancal and nursng/lab departments. On the other hand, t seems that there s a problem n the customer servce personnel, gven the sgnfcant low evaluaton scores of ths partcular group of employees. Ths last result may be justfed by the unfavorable current workng condtons (.e. work load, staff shortages). Job poston Fnancal department (managers) Nursng department (managers) Customer Servce department (managers) Table 4: Overall evaluaton scores for dfferent job postons Number of employees Average score Mn score Max score Standard devaton Fnancal personnel Nursng personnel Lab personnel Customer Servce personnel Techncal personnel Smlarly, the appled multcrtera method can provde analytcal scores for each employee and evaluaton crteron, based on the estmated margnal value functons. Usng these results, the management of the organzaton has the ablty to dentfy the strengths and the weaknesses for each employee or job poston. Moreover, t s possble to perform 4

17 comparatve analyses for the employees of a partcular department, as well as for the dfferent job postons. The strengths and weaknesses for each job poston are presented n Table 5. These results are not based only on the margnal scores, but they take nto account the weghts of the evaluaton crtera. Based on ths table, the most mportant fndngs nclude the followng:. The managers of the departments appear to have dfferent strengths and weaknesses, probably because ther job postons present large dfferences. 2. The consstency and the qualty of the work output are the most mportant common strengths for all the employees workng n the fnancal department. On the other hand ther weaknesses manly refer to the crtera of ntatve and work under pressure. 3. Regardng the nursng department, dfferent strengths and weaknesses are found between the hgher and the lower-level personnel. 4. The most mportant common strengths for all the employees of the lab departments are focused on the consstency and the qualty of the work output, whle the crteron of ntatve s ther man weakness. 5. Fnally, patent orentaton s the most mportant common strength for all the employees workng n the customer servce department, whle ther man weakness concerns the crteron of qualty of work output. Table 5: Strengths and weaknesses based on employees evaluaton Job poston Strengths Weaknesses Fnancal department (managers) Nursng department (managers) Customer Servce department (managers) Qualty of work output Consstency Relablty Consstency Relablty Intatve Organzaton s mage Intatve Work under pressure On tme tasks completon Leadershp Qualty of work output Sklls Patent orentaton Qualty of work output Fnancal personnel Consstency Qualty of work output Receptvty to gudance Nursng personnel Qualty of work output Organzaton s mage Lab personnel (mcrobology) Lab personnel (radology) Customer Servce personnel Communcaton wth patents Qualty of work output Consstency Communcaton wth patents Inform/Follow progress Qualty of work output Consstency Patent orentaton Inform/Follow progress Techncal personnel Consstency Intatve Intatve Work under pressure Intatve Work under pressure Intatve Work under pressure On tme tasks completon Relablty Intatve Inform/Follow progress Qualty of work output Relablty Intatve Work under pressure Qualty of work output Inform/Follow progress 5

18 Consequently, the man weaknesses of all the employees workng n the organzaton refer to the crtera of ntatve and work under pressure. On the other hand, the strengths dffer among the dfferent job postons, although the performance of consstency s relatvely hgh n many employees. Another mportant pont to menton s that the crteron of qualty of work output appears both as strength and as weakness n the evaluated job postons. 5. Concludng remarks Modern busness organzatons evaluate the performance of ther employees by examnng the contrbuton of ther personnel n achevng the assessed goals. Thus, employee evaluaton s the crtcal lnk between organzatonal strateges and results. However, developng, mantanng, and mprovng an employee evaluaton system s a complex task, gven the dfferent alternatves approaches that can be adopted, the lnkage wth other busness functons, and the subjectvty of every evaluaton process. The presented study proposed a multcrtera method for the development of an employee evaluaton system. The method s based on the UTASTAR algorthm but consders addtonal preferences expressed by the DM. The man advantage of the appled multcrtera method s that the strategy of the organzaton s drectly taken nto account, through the evaluaton crtera mportance preferences gven by the management of the organzaton. The method has also the ablty to use absolute performance measures and handle qualtatve (ordnal) nformaton. Moreover, the proposed approach consders the complexty of the dfferent job profles and evaluates employees on a set of dfferent but specfc job dmensons. Fnally, the method can help the management of the organzaton to mprove hs/her knowledge about ths partcular decson stuaton. Although the study focuses on the mplementaton of the multcrtera method and the analyss of the provded results, an ntegrated employee evaluaton approach should also examne addtonal ssues, lke communcatng results on employees, determnng and mplementng mprovement actons (e.g. tranng), revsng the evaluaton system etc. In any case, t s crtcal for every busness organzaton to adopt a culture based on contnuous evaluaton and mprovement and ncorporate the employee evaluaton management system nto every-day practce. In addton, when employee evaluaton s consstent wth the strategc msson of the organzaton, t can help the development of an effectve performance management system. In ths context, several researchers emphasze that t s not mportant whch method s used n an employee evaluaton approach, but the key s whether managers and employees understand that the evaluaton process s a mean to mprove the organzaton. These are the reasons why an employee evaluaton system s usually consdered as both an opportunty and a danger. References Becker, B.E. and B. Gerhart (996). The mpact of human resource management on organzatonal performance: Progress and prospects. Academy of Management Journal, 39 (4), Cheng, E.W.L. and H. L (2006). Job performance evaluaton for constructon companes: An analytc network process approach, Journal of Constructon Engneerng and Management, 32 (8),

19 DeCenzo, D.A. and S.P. Robbns (2005). Fundamentals of human resource management, 8 th edton, John Wley and Sons, New Jersey. Glddon, D.G. (200). Performance management systems, n R. Watkns and D. Legh (eds.), Handbook of mprovng performance n the workplace: Selectng and mplementng performance nterventons (vol. 2), Pfeffer, San Francsco, Guest, D.E., J. Mche, N. Conway, and M. Sheehan (2003). Human recourse management and corporate performance n the UK, Brtsh Journal of Industral Relatons, 4 (2), Jacquet-Lagrèze, E. and Y. Sskos (982). Assessng a set of addtve utlty functons for multcrtera decson makng: The UTA method, European Journal of Operatonal Research, 0 (2), Jacquet-Lagrèze, E. and Y. Sskos (200). Preference dsaggregaton: 20 years of MCDA experence, European Journal of Operatonal Research, 30 (2), Maths, R.L. and J.H. Jackson (2007). Human resource management, 2 th edton, South- Western College Pub, Cncnnat. Ostroff, C. and D.E. Bowen (2000). Movng HR to a hgher level: HR practces and organzatonal effectveness. n: K.J. Klen and S.W.J. Kozlowsk (eds.), Multlevel theory, research, and methods n organzatons: Foundatons, extensons, and new drectons, Jossey- Bass, San Francsco, Saaty, T.L. (2005). The analytc herarchy and analytc network processes for the measurement of ntangble crtera and for decson-makng, n: J. Fguera, S. Greco, M. Ehrgott (eds.), Multple crtera analyss: State of the art surveys, Sprnger, New York, Sels, L. S. De Wnne, J. Delmotte, J. Maes, D. Faems, and A. Forrer (2006). Lnkng HRM and small busness performance: An examnaton of the mpact of HRM ntensty on the productvty and fnancal performance of small busnesses, Small Busness Economcs, 26 (), Sskos, J., A. Spyrdakos and D. Yannacopoulos (993). MINORA: A multcrtera decson adng system for dscrete alternatves, Journal of Informaton Scence and Technology, 2 (2), Sskos, Y. and D. Yannacopoulos (985). UTASTAR: An ordnal regresson method for buldng addtve value functons, Investgação Operaconal, 5 (), Sskos, Y. and E. Grgorouds (200). New trends n aggregaton-dsaggregaton approaches, n: C. Zopounds and P.M. Pardalos (eds.), Handbook of multcrtera analyss, Sprnger, Hedelberg, Sskos, Y., A. Sprdakos and D. Yannacopoulos (999). Usng artfcal ntellgence and vsual technques nto preference dsaggregaton analyss: The MIIDAS system, European Journal of Operatonal Research, 3 (2), Sskos, Y., E. Grgorouds, and N.F. Matsatsns (2005). UTA methods, n: J. Fguera, S. Greco, M. Ehrgott (eds.), Multple crtera analyss: State of the art surveys, Sprnger, New York, Spyrdakos, A., Y. Sskos, D. Yannakopoulos and A. Skours (2000). Multcrtera job evaluaton for large organsatons, European Journal of Operatonal Research, 30 (2), Tabachnck, B.G. and L.S. Fdell (989). Usng multvarate statstcs, Harper/Collns, New York. Torrngton, D., L. Hall, and S. Taylor (2008). Human resource management, 7 th edton, Prentce Hall, Harlow. 7

20 Appendx: Margnal value functons (managers of fnancal/customer servce department) Inform/Follow progress Sklls Qualty of work output Leadershp Consstency Relablty Intatve On tme tasks' completon