Portfolios as a method of student assessment

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Portfolios as a method of student assessment"

Transcription

1 Porfolios as a mehod of suden assessmen AMEE Medical Educaion Guide No 24 This AMEE Educaion Guide was firs published in Medical Teacher: Friedman Ben David M, Davis M H, Harden R M, Howie P W, Ker J and Pippard M J (2001) AMEE Medical Educaion Guide No 24: Porfolios as a mehod of suden assessmen. Medical Teacher 23(6). The Auhors Miriam Friedman Ben David is Professor of Medical Educaion, formerly Co-Direcor of he Clinical Skills Cerificaion Programme a he Educaional Commission for Foreign Medical Graduaes (ECFMG) and Associae Dean a he Medical College of Pennsylvania, Philadelphia. Professor Friedman Ben David is currenly a he Cenre for Medical Educaion, Dundee, Scoland. Her main areas of ineres are physician and suden assessmen, programme evaluaion and developmen of leadership in he medical profession. Margery Davis is Senior Lecurer in Medical Educaion a he Universiy of Dundee s Cenre for Medical Educaion. She has provided educaional suppor for he Dundee Medical School since Ronald M Harden is Direcor of he Cenre for Medical Educaion, Universiy of Dundee and Direcor of he Scoish Council for Posgraduae Medical and Denal Educaion s Educaion Developmen Uni. Peer Howie is Professor of Obserics and Gynaecology a he Universiy of Dundee and Chairman of he Scoish Council for Posgraduae Medical and Denal Educaion. He has had a long-sanding ineres in medical educaion and he developmen of porfolio assessmen in paricular. Jean Ker is a Clinical Lecurer in Medical Educaion in he School of Medicine, Universiy of Dundee. She is a pracising General Praciioner and provides suppor o he undergraduae curriculum in clinical skills and assessmen. Marin J Pippard has been Professor of Haemaology in Ninewells Hospial and Medical School, Universiy of Dundee, since He rained in Oxford UK and Seale USA and was formerly Consulan Haemaologis a he MRC Clinical Research Cenre and Norhwick Park Hospial, Harrow UK. As convenor for Phase 2 (years 4 and 5) of he Dundee undergraduae medical curriculum, he has coordinaed he implemenaion of his phase of a new curriculum over he las 3 years. Guide Series Edior: Pa Lilley Producion and Deskop Publishing: Lynn Bell AMEE 2001 ISBN: Copies of his guide are available from: AMEE, Cenre for Medical Educaion, Tay Park House, 484 Perh Road, Dundee DD2 1LR, UK Tel: +44 (0) ; Fax: +44 (0) ;

2 Conens Purpose of his guide Page Summary A: Porfolios as a mehod of suden assessmen Background o he use of porfolios for assessmen purposes Wha are porfolios? Types of porfolios Wha porfolios migh conain Why use porfolios? The assessmen of professionalism B: The Dundee case sudy C: Psychomeric issues relaing o use of porfolios for assessmen Summary of he porfolio s main assessmen feaures Raers consisency Decision consisency Sampling The consensus approach Forms of validiy Exernal vs. inernal examiners D: Implemenaion of porfolio assessmen Defining he purpose Deermining compeences o be assessed Selecion of porfolio maerial Developing a marking sysem Selecion and raining of examiners Planning he examinaion process Suden orienaion Developing guidelines for decisions Esablishing reliabiliy and validiy evidence Designing evaluaion procedures E: Conclusions References

3 Purpose of his guide This guide informs medical eachers abou he use of porfolios for he purpose of suden assessmen. I looks a: Wha are porfolios? Why use porfolios and he conribuions ha porfolios can make o suden assessmen and he psychomeric issues relaing o use of porfolios in assessmen. Guidelines for he implemenaion of porfolio assessmen. The educaional/heoreical consideraions presened in his guide along wih principles of pracical applicaions should enable he reader o design and implemen a porfolio assessmen. -3 -

4 Summary This guide is inended o inform medical eachers abou he use of porfolios for suden assessmen. I provides a background o he opic, reviews he range of assessmen purposes for which porfolios have been used, idenifies possible porfolio conens and oulines he advanages of porfolio assessmen wih paricular focus on assessing professionalism. The experience of one medical school, he Universiy of Dundee, is presened as a case sudy. The curren sae of undersanding of he echnical, psychomeric issues relaing o porfolio assessmen is clarified. The final par of he paper provides a pracical guide for hose wishing o design and implemen porfolio assessmen in heir own insiuions. Five seps in he porfolio assessmen process are idenified: documenaion, reflecion, evaluaion, defence and decision. I is concluded ha porfolio assessmen is an imporan addiion o he assessor s oolki. Reasons for using porfolios for assessmen purposes include he impac ha hey have in driving suden learning and heir abiliy o measure oucomes such as professionalism ha are difficul o assess using radiional mehods. A: Porfolios as a mehod of suden assessmen 1 Background o he use of porfolios for assessmen purposes Recen changes in medical educaion and raining emphasise he developmen of eaching and learning programmes which mee he needs of he medical profession and sociey as a whole. Professional organisaions and he public a large demand demonsraion of professional aribues which ensure docors finess for pracice while adhering o high sandards of care (GMC 1997). The medical schools and posgraduae raining programmes have responded by inroducing new eaching and learning sraegies which will enhance docors accounabiliy. One imporan innovaion in curriculum reform is he developmen of an oucome-based educaional framework (Harden e al 1999). Learning oucomes of undergraduae and posgraduae raining are defined broadly o allow sudens o recognise heir progress along he scope and deph of heir professional compeences. The use of such frameworks faciliaes he developmen of radiional learning oucomes, such as clinical skills, as well as learning oucomes defining he docor as a professional which, in he pas, have been ignored. The growing ineres and concern wih docors professional aribues is shared worldwide as indicaed by he vas number of medical educaion papers published on he opic of professionalism during he year 2000 alone (Ginsburg 2000). Furhermore, he concep of professionalism has filered ino eaching and learning sraegies where sudens are encouraged o ake responsibiliy for heir own learning, and personalise heir learning experiences. Suden-cenred programmes increasingly incorporae he learners needs, as well as lisen o he learners voice. Consequenly, medical schools curricula are seeking o broaden suden experiences inside and ouside he hospials o allow beer appreciaion of he muli-conex sysem wihin which docors are now pracising. The new direcion of medical schools curricula resuls in somewha less srucure and more auhenic experiences for he rainee which, in urn, increases self-direced learning. Concern wih he lack of coninuiy beween undergraduae and posgraduae educaion is anoher imporan facor. Transiion from he undergraduae o he posgraduae phase of medical educaion should be consisen wih progression from echnical discree abiliies o full inegraion of professional compeencies, which is he ulimae oucome of medical raining. Concurren o hese educaional reforms, new assessmen sraegies are being developed o mee he needs of recen innovaions in he healh professions. The search for new assessmen ools is a reacion agains exising mehods of assessmen, which ofen have had adverse effecs on he learner, he eacher and he curriculum as a whole. I is recognised ha assessmen ools should enhance and suppor learning as well as measure performance. Much curren ineres is in auhenic, performancebased assessmen (Korez e al, 1998) which also encourages learners o ake responsibiliy for heir own learning and guides he learners o accumulae evidence of learning, while incorporaing a crierionreferenced inerpreaion of heir performance. -4 -

5 The porfolio is an aemp o counerac he limiaions of a reducionis approach o assessmen. I faciliaes assessmen of inegraed and complex abiliies and akes accoun of he level and conex of learning. I provides an assessmen soluion for a curriculum ha designs learning along broad educaional and professional oucomes. I personalises he assessmen process while incorporaing imporan educaional values. I suppors he imporan principle of Learning Through Assessmen (Friedman 2000). This guide describes he use of porfolios as a ool for assessmen and explores he conribuions hey can make. I examines how hey can be implemened in pracice and looks a heir psychomeric properies. A previous AMEE guide, Porfolio-based Learning and Assessmen (Snadden e al 1999), inroduced he concep of porfolios in undergraduae, posgraduae and coninuing medical educaion and focused on heir conen and consrucion. 2 Wha are porfolios? Definiions The idea of a porfolio is drawn from he sudy of ar, where he erm signifies a purposeful collecion of work. Secher 1998 A porfolio is a collecion of papers and oher forms of evidence ha learning has aken place. Davis e al 2001 I is a collecion of suden work ha exhibis he suden s effors, progress and achievemens in one or more areas This collecion represens a personal invesmen on he par of he suden an invesmen ha is eviden hrough he suden s paricipaion in he selecion of he conens, he crieria for selecion, he crieria for judging he meri of he collecion and he suden s self reflecion. Gisselle O, Marin-Kniep 2000 These definiions imply ha he purpose of suden assessmen will guide he conen of he porfolio, i.e. bes work, ypical work, mos diverse work, work on a heme, or work on one projec as i evolves over ime (Secher 1998). Definiion of purpose, from an assessmen perspecive, will deermine he porfolio conen, he process of creaing i and he inerpreaion of he evidence. Therefore, porfolios will vary from conex o conex depending on heir specialised purpose. Mos porfolios will share a common characerisic of being cumulaive, i.e. conain work compleed over a period of ime relevan o sudens ongoing work or sudies. 3 Types of porfolios Porfolios are used in all sages of educaion: elemenary, higher educaion and in professional and vocaional programmes. Porfolios are used in elemenary school classrooms as par of ongoing assignmens (Herman & Winer 1994). Sudens prepare draf wrien assignmens over ime and, a a pre-deermined dae, choose he bes work for a show case. Such porfolios could be scored a a school level or a sae level (a a sae scoring sie) - for high sakes examinaions or for qualiy conrol procedures. In he USA, he Sae of Vermon inroduced in 1992 a large-scale porfolio assessmen for grades 4 o 8 in wo subjecs, mahemaics and wriing. Sudens selec from heir on-going assignmens in mahemaics, five o seven bes pieces of work which are scored on seven dimensions of performance, hree peraining o communicaion and four o problem solving. The sudens submi a porfolio wih one bes piece of work and oher specified pieces. The bes piece was scored separaely and he remaining pieces were scored as a se. The wriing pieces were scored on five dimensions: purpose, organisaion, deails, voice/one, usage/mechanism/ grammar. Porfolio maerials were scored by eachers o provide individual scores and by he Sae o provide school profiles in hose wo subjecs (Korez 1998). The Sae of Kenucky developed a similar largescale porfolio assessmen in wriing for 4 h grade sudens. Over a period of one year, he sudens compiled a porfolio comprising six pieces: a personal narraive, a poem, a play, a piece of ficion, one informaion or persuasive piece, one piece from any subjec area oher han English and language ars, a bes piece and a leer o he reviewer abou he bes piece and suden s growh as a wrier (Korez 1998). In he professions, porfolios are used, for example, in recording nurses career and professional developmen hrough formal and experienial learning for periodic regisraion wih he UK Cenral Council (Jasper 1995). In medicine, porfolios are used o sudy criical hinking and self-direced learning in he daily pracice of general praciioners (Al Shehri 1995). The Royal College of Physicians and Surgeons of Canada inroduced a learning porfolio, The Mainenance of Compeence Program (MOCOMP), using compuer sofware for physicians o creae a personal learning porfolio. The collecion of -5 -

6 informaion shows how physicians learn, which facors affec changes in pracice and wha sources of learning physicians use. I also creaes daabases for esablishing sandards of physicians performance and for coninuing medical educaion programmes (Bashook e al 1998). The Naional Council for Vocaional Qualificaions esablished assessmen procedures for qualificaion in England, Wales and Norhern Ireland in which porfolios play a cenral role. I is aimed for young people in upper secondary schooling and young people and aduls in work or in governmen raining. Wolf (1998) repors ha he porfolio approach o assessmen has affeced he desired learning syles, bu major problems have arisen relaing o he manageabiliy of he approach and reliabiliy of assessors judgemens. The General Medical Council in UK has pu forward a revalidaion proposal for pracising physicians (GMC 2000). The 3-sage proposal suggess a porfolio (folder) which will include informaion and evidence regarding docors cumulaive performance o be assessed by groups of docors. 4 Wha porfolios migh conain Porfolios conain sudens ongoing work over ime which may provide evidence for learning and progress owards educaional and professional oucomes or learning objecives. These may include mainenance of compeences, keeping up o dae wih professional pracice, finess for pracice, adherence o professional sandards and more. Porfolios may also conain suden s or candidae s reflecion on he submied evidence. By definiion any maerial which provides evidence for he above menioned educaional and work relaed crieria can be included in porfolios. This may include: bes essays wrien repors or research projecs samples of evaluaion of performance, e.g. uor repors from clinical aachmens video apes of ineracions wih paiens or wih peers records of pracical procedures underaken (log books) annoaed paien records leers of recommendaion CVs wrien reflecion on he evidence and on professional growh. Evidence included in porfolios is limied only by he degree of he designer s creaiviy. 5 Why use porfolios? The use of porfolios for sudens assessmen enables sudens and eachers o engage in a process of learning hrough assessmen. This implies ha he assessmen procedure no only measures and reinforces he desired learning oucomes bu raher enhances he developmen of sraegies, aiudes, skills and cogniive processes essenial for life long learning. Consequenly, he use of porfolios no only broadens he scope of assessmen bu also inroduces a number of educaional benefis. Porfolios conribuion o assessmen The assessmen of learning oucomes: This includes oucomes no easily assessed by oher mehods, e.g. personal growh, self-direced learning, reflecive abiliy, self-assessmen of personal growh, professionalism and more. An exended discussion of he use of porfolios for assessmen of professionalism appears a he end of his secion. The provision of evidence of performance: This evidence may be colleced from a range of sources, e.g. suden s on-going work over ime, over seings and over subjec maers. The represenaion of evidence colleced over a period of ime: Porfolios provide evidence of suden developmen over ime and no jus a snapsho a one ime or a series of snapshos, as in radiional assessmen mehods. Suden progression owards he learning oucomes: Porfolios allow assessmen of progress owards he learning oucomes by using chronological work samples colleced a differen poins of ime. Summaive and formaive assessmen: The evaluaion of he porfolio conen generaes summaive saemens regarding suden performance for promoion or pass/fail decisions. I also provides an in-deph profile of suden abiliies. Summaive evaluaion saemens may also conain informaion regarding sudens srenghs and weaknesses, hus inegraing summaive decisions wih formaive profiles. The curren challenge facing porfolio assessmen is o judge he qualiaive and quaniaive evidence and ye mainain reliabiliy and validiy -6 -

7 of he judgemen. Secion C in his guide expands he discussion of his issue. In addiion o he conribuion ha he porfolio makes o suden assessmen, i also reinforces oher imporan educaional aspecs cenral o effecive eaching and learning. Focus on personal aribues Provides personal and professional educaional evidence for suden learning (Glen and High, 1992). Makes provision for sudens o receive feedback abou heir own personal values, feelings, syles of ineracion and ways of handling significan experiences (Beaie 1991) Provides a personalised sensiive porrai of wha sudens know and are able o do (Herman & Winer, 1994). Conexualises learning and links experience wih personal inerpreaion. Enhances ineracions beween sudens and eachers Allows dialogue beween sudens and educaors and ensures individual proeced ime. Reminds sudens ha learning is a wo-way process beween learner and educaor. Mirrors sudens and eachers work and simulaes eachers o re-assess eaching sraegies (Aschlacher 1993). Raises eachers expecaions in relaion o hinking abiliy and problem solving (Finlay e al 1998). Simulaes he use of reflecive sraegies Faciliaes he use of pas experiences o define learning and o recognise progress. Simulaes he use of reflecive skills in order o analyse and synhesise experiences. Sudens can also describe and analyse learning sraegies hus engaging in meacogniive processes. Allows educaors o separae he qualiy of evidence from he suden abiliy o reflec on he evidence. Expands undersanding of professional compeence The variaions in suden percepion and inerpreaion of heir experiences increases undersanding of sudens professional growh. Those newly acquired insighs inform medical educaion and pracice (Herman & Winer 1994). Encourages a holisic and inegraive approach o medical pracice. 6 The assessmen of professionalism Why assess professionalism? The acquisiion of he professional behaviour required o pracise medicine is slowly emerging as a cenral focus in he new undergraduae medical curriculum (Cohen 2001) and wih i increasing consensus on professional behaviour (Swick 2000). The inroducion of new mehods of assessmen o expand professional horizons (Friedman 2000) presens he profession wih he opporuniy o realign curricula o incorporae he developmen of professionalism as a cenral oucome and find ways o assess professionalism and professional growh. Wha is professionalism? One of he unique problems for medicine in defining professional ideniy is ha i develops from pracice which in recen imes has seen such enormous changes boh in biomedical advances and in shifing role boundaries in he delivery of care (Shapiro 1999). There has been lile appreciaion ha professionalism is a dynamic concep which needs o be defined in he conex of changing expecaions of sociey and changing imes. Recenly, however, clearer definiions of professionalism have sared o emerge. For example, reflecive abiliy is increasingly being idenified as a key componen of medical professionalism (Grundy 1987, Lundberg 1991, Parboosingh 1996, Wear 1997, Haffery 2001). This builds on Schon s work in relaion o he developmen of he reflecive praciioner for professional pracice (1983). Reflecion requires boh cogniive and humaniarian aribues. I is a process ha enables praciioners o deermine heir own acions, criically review hese acions and ac on he oucome in he bes ineres of he clien or paien. Projec Professionalism, which was inroduced by he American Board of Inernal Medicine (ABIM) (1994), defined medical professionalism as an ideology consising of he following elemens: a commimen o: The highes sandards of excellence in he pracice of medicine and in he generaion and disseminaion of knowledge (cogniive aribues) Susain he ineress and welfare of paiens Be responsive o he healh needs of sociey -7 -

8 These elemens were furher defined as he humaniarian qualiies of alruism, accounabiliy, excellence, duy, honour, inegriy and respec for ohers. These are required of all candidaes seeking cerificaion and recerificaion from he ABIM. The Associaion of American Medical Colleges (AAMC) (1998), in heir Medical Schools Objecives projec, have also focused on professional developmen in relaion o boh humaniarian and cogniive qualiies. They have idenified four core aribues ha all medical sudens should be able o demonsrae on graduaion. The four aribues include alruism, knowledge (cogniive) skill and duifulness. They furher idenify learning objecives for each of hese aribues. In relaion o alruism AAMC include characerisics such as compassion, honesy, inegriy, eam relaions, commimen o paiens ineress, undersanding hreas o medical professionalism and an awareness of one s own limiaions. In he UK, he Scoish Deans Medical Curriculum Group (2000), which consiss of five medical schools has idenified a Scoish medical graduae as a compeen and reflecive praciioner. In heir new oucome based model, hey have idenified an apiude for personal and professional developmen as one of he welve curriculum oucomes of a Scoish docor a graduaion. The key feaures of his oucome include self awareness, (developing reflecive pracice o be aware of one s own compeence), self learning (idenifying one s own srenghs and weaknesses) self care (being aware of inernal and exernal facors on one s own professional developmen, career choice and commimen). Why are porfolios a useful means of assessing professionalism? The developmen of muliple sampling in he assessmen of clinical pracice is receiving increasing aenion paricularly in relaion o he assessmen of professional behaviour. Turnbull (2000) has used a sysem of clinical work sampling o assess sudens during a roaion and found i o be a reliable, valid and feasible form of assessmen. Porfolios, which sample evidence over ime, also demonsrae a poenial and reliable ool for he assessmen of acquisiion of knowledge, skills, aiudes, undersanding and achievemens and...reflec he curren sages of developmen and aciviy of he individual (Brown 1995). The use of porfolio assessmen has recenly been inroduced a he Universiy of Dundee School of Medicine (Davis 2001) as a key componen in he final undergraduae degree examinaion, in order o assess aspecs of medical pracice such as professional developmen. Porfolios presen an ideal mehod of assessing professional developmen as hey enable he assessmen of he docor s progress mached agains professional crieria over ime. Porfolios also enable professional developmen o be assessed a differen sages of a praciioner s career. The sampling of work-relaed aciviy reflecs he realiy or auheniciy of an individual s professional clinical pracice. Having a variey of auhenic evens recorded in he porfolio enables he examiner o idenify any cogniive or humaniarian deficiencies ha candidaes may have in relaion o heir professional developmen. The porfolio assessmen process allocaes ime for he examiner o focus on he sudens developmen, during he oral componen of he assessmen, by asking he examinees o reflec on heir professional role and relaed experiences. The porfolio provides an ideal conex o assess reflecive abiliy as a key componen of professional developmen. Time is provided during he porfolio assessmen for examiners o assess he abiliy of sudens o reflec in relaion o heir porfolio evidence. In addiion, reflecive wriing for a porfolio assessmen demonsraes he examinee s experience separaed from his/her personal inerpreaion of he experience. Organising sudens experiences hrough reflecive wriing modifies heir percepion of he experience and enables i o be inegraed ino prior learning or used o iniiae new learning (Eisner 1991). The porfolio assessmen process also provides a safe opporuniy o explore errors in professional medical pracice and sraegies o avoid fuure errors. These sensiive maers can be discussed wihin he oucome of an apiude for personal and professional developmen. -8 -

9 B: The Dundee case sudy Dundee medical school moved o oucome-based educaion in he academic year wih 12 learning oucomes idenified. A hree-circle classificaion model was developed for he presenaion of he learning oucomes (Harden e al 1999). The inner circle describes wha he docor is able o do and his includes seven compeencies: 1) clinical skills, 2) pracical procedures, 3) invesigaing a paien, 4) paien managemen, 5) healh promoion and disease prevenion, 6) communicaion skills and 7) informaion handling and rerieval. The middle circle describes how he docor approaches he ask and i includes 3 compeencies: 8) undersanding of basic, clinical and social sciences, 9) appropriae aiudes, ehical undersanding and legal responsibiliies, 10) appropriae decision making, clinical reasoning and judgemen. The ouer circle describes he docor as a professional and i includes: 11) he role of he docor wihin he healh service and 12) apiude for personal developmen. In Dundee Universiy School of Medicine, he learning oucomes provide he framework for suden learning, eaching and assessmen. Individual sudens progress owards he 12 oucomes, is he focus of he porfolio assessmen. In heir porfolios he sudens provide he evidence ha hey have achieved he oucomes a he appropriae level expeced of a graduaing suden. The oucome definiions are he crieria for he porfolio assessmen. The final examinaions were redesigned o mee he needs of he Dundee new curriculum (Davis e al 2001). The curriculum incorporaed an inegraed, ask-based approach ha is inerwoven wih he oucome model. The final examinaions are adminisered in wo pars: Par 1 occurs in he end of year 4 when sudens have compleed heir core clinical aachmens. I includes a knowledge and problem solving componen measured by exended maching iems (EMI), and consruced response quesions (CRQ); and an OSCE o assess clinical skills. Par 2 of he final examinaions akes place owards he end of year 5 by employing a porfolio assessmen. Sudens and faculy were given he opporuniy o evaluae, in a holisic approach, suden progress owards all he oucomes including hose dealing wih he docor as a professional and personal developmen which are no easily assessed by radiional assessmen mehods. The conen of he porfolio includes sudens summaries of progress owards each of he oucomes and a variey of maerials prepared during years 4 and 5 (Table 1). The noes o examiners and sudens conain deailed descripions of suden work which are lised in Table 1. The porfolio examinaion in Dundee is par of a sequenial assessmen process in ha sudens need o pass he year 4 final examinaions before hey are permied o si for he porfolio. End of year 5 sudens, who pass he porfolio examinaion, require no furher assessmen and hus fulfil requiremens for graduaion. Sudens who presen wih minor or major deficiencies are referred eiher for furher remedial work or for a year 5 final examinaion comprising an OSCE and review of he porfolio afer appropriae correcion or addiions. Figure 1 oulines he process of porfolio examinaion in Dundee from he examiners perspecive. Two examiners independenly read each suden porfolio a few days prior o he dae of he porfolio oral review wih he suden. The wo examiners independenly grade he porfolio conen employing crieria and guidelines se ou by he medical school. They use a marking shee, he porfolio assessmen scoring shee (PAS) o inser heir oucome grades based on heir reading of sudens porfolio work. The examiners gradings of he 12 oucomes are independen bu ake accoun of he already pre-marked suden work. They focus on he overall paerns of achievemen and progress over he wo years raher han he grading of specific pieces of evidence. They make judgemens on how he differen ses of porfolio evidence relae o he suden progress owards he 12 oucomes. In a prereview session on he day of he oral review, he wo examiners mee and discuss he sudens porfolios. They agree on he sudens srenghs and weaknesses and specific issues ha should be explored during he oral review of he porfolio wih he suden. During he oral review of he porfolio wih he suden, which lass 40 minues, he examiners pursue he issues of concern and, a he end, hey independenly assign grades for each oucome based on sudens performance during he oral review. The examiners hen reach consensus on a hird se of oucome grades which consiues he final grading based on which he pass/referred decisions are made. They also make recommendaions for disincion. A he end of each examinaion day, all examiners for his day mee as an Examiners Commiee o consider he referred and disincion sudens, o challenge he decisions -9 -

10 Suden work Year Number Pre-marked* Suden s personal summary of progress owards each oucome No Paien presenaions: shor summaries of paiens seen by he suden, seleced from he 100 core clinical problems on which eaching and learning is based on his phase of he curriculum Someimes marked Case discussions: repors of approximaely 1500 words, each analysing a paien s hisory and findings in erms of one of he curriculum hemes. 5 7 Yes Year 4 assignmen: a projec repor wih a grade awarded by he projec supervisor ogeher wih feedback for he sudens. 4 1 Yes Record of achievemen: a record of procedures ha he suden was expeced o have compleed or observed during he phase. 4&5 1 Signed by Faculy GP special sudy module assessmen form: a repor on he suden by heir general pracice supervisor wih a grade awarded. 5 1 Yes Clinical special sudy module assessmen form: heme special sudy module assessmen form: a repor on suden performance during he module wih a grade awarded for relevan oucomes. 5 2 Yes PRHO appreniceship in medicine assessmen form: a learning conrac beween he suden and his/her educaional supervisor wih grades awarded for each learning oucome. 5 1 Yes PRHO appreniceship in surgery assessmen form: a learning conrac beween he suden and his/her educaional supervisor wih grades awarded for each learning oucome. 5 1 Yes Elecive repor: a repor compleed by he suden afer he elecive period. This was read by one of he wo members of saff responsible for elecive sudies and wrien feedback provided. 5 1 Yes * The pre-marked suden work is scored using he Universiy grading sysem (A-G) and conains commens provided by Faculy. Table 1: Descripion of porfolio conens -10 -

11 Porfolios as a mehod of suden assessmen Porfolio maerial Porfolio reading process Broad sampling of sudens work (graded & no graded) Examiner I Independen grading of oucome Examiner II Independen grading of oucome Pre-porfolio oral review Agreemen on srenghs and weaknesses, o be explored during he porfolio oral review wih he suden Oral review of porfolio by examiners wih he sudens Pos-porfolio oral review Examiner I Independen oucome grading Examiner II Independen oucome grading Consensus on 12 oucome grades by Examiners I and II Consensus by group of examiners Pass Disincion Referred Figure 1: The porfolio examinaion process: he examiners perspecive and reach consensus for he final lis of he passed, referred and disincion sudens. Grades awarded for each oucome correspond o he following crieria: Grade A excellen Grade B very good Grade C saisfacory Grade D borderline pass Grade E marginal fail Grade F definie fail Grade G bad fail Following he Examiners Commiee meeing, sudens are allocaed o one of he following caegories: Exemp from furher pass/fail oucome-based assessmen, (all grades are D or above). Exemp from furher pass/fail oucome-based assessmen, bu o be considered for disincion. These sudens will si for a disincion examinaion which will deermine who is awarded a disincion

12 Poenial exempion from furher examinaion (only one oucome is graded E wih all ohers graded D or above). Furher work will be assigned o address he limied deficiency idenified in he porfolio examinaion. Proceeds o furher examinaion (abou 6 weeks laer) if wo or more oucomes a Grade E or below or one or more oucomes a Grade F or G. Those sudens will si an OSCE and a furher porfolio assessmen. In summary, he porfolio examinaion in Dundee is one sequenial componen of he final examinaion process which sars a he end of year 4. Sudens need o pass he year 4 final examinaion componens before hey can si he porfolio assessmen. Sudens who pass he porfolio assessmen, mee he final examinaion requiremens. Sudens who fail he porfolio assessmen proceed o eiher remediaion or furher esing via an OSCE and furher review of he porfolio. Figure 2 demonsraes he porfolio process from he suden perspecive. For each phase of he process here is an associaed educaional value which indicaes how he porfolio process suppors suden learning. In he beginning of year 4 sudens learn in advance wha is expeced of hem. As hey selec he evidence during years 4 and 5, hey decide wha informaion hey need in order o mee he 12 oucome crieria. When hey compile he evidence in he porfolio, hey demonsrae heir organisaional skills, ime managemen and prioriisaion abiliy. The 12 oucome summary shee allow sudens o reflec on heir progress and prepares hem o hink on heir fee during he oral review. Sudens are asked o complee an evaluaion form a he end of he porfolio assessmen which allows hem o furher reflec on he whole process. Sudens who pass he porfolio complee he cycle of demonsraing performance achievemens. Sudens who were seleced for disincion have furher opporuniies o demonsrae problem solving abiliy, informaion handling and rerieval and knowledge. The ones who were referred o furher work or esing have o ac on heir own deficiencies. Educaional value Task Time Advance organiser Mee learning needs wihin he oucome framework Organisaional skills, ime managemen and prioriisaion Reflecive abiliy Read orienaion book + aend inducion Sar of Year 4 Selecion of he evidence Sar of Years 4&5 Compiling evidence in he porfolio Year 5 Wrie 12 oucome summary shees Year 5 Think on one s fee. Demonsrae professional behaviour Reflecion on he whole process. Have curriculum inpu. Oral review. Defending he evidence Year 5 Complee evaluaion forms Year 5 Pass Mee requiremen Disincion Demonsrae problem solving abiliy, informaion handling and rerieval and knowledge Referred Ac on own deficiencies Figure 2: The porfolio process: suden perspecive -12 -

13 C: Psychomeric issues relaing o he use of porfolios for assessmen 1 Summary of he porfolio s main assessmen feaures Table 2 provides a summary of he porfolio s main assessmen feaures. 1 Formaive and summaive 2 Qualiaive and quaniaive 3 Personalised 4 Sandardised 5 Auhenic Table 2: The porfolio s main assessmen feaures Formaive and summaive The main assessmen characerisics of porfolios focus on he FORMATIVE and SUMMATIVE aspecs in which sudens collec evidence of heir own performance for pass/fail decisions, feedback, diagnosic purposes and inervenion. The formaive value of he porfolio, when linked o summaive decisions, presen a very powerful assessmen ool. In his secion, a forward sraegy is discussed for he use of porfolios for summaive decisions such as promoion, graduaion, cerificaion and oher high sakes applicaions. Qualiaive and quaniaive An imporan feaure of porfolios is he combined QUALITATIVE and QUANTITATIVE approach, unlike many oher performance ess ha rely mainly on he quaniaive aspec of performance. The porfolio conains descripive recorded maerial as well as graded evidence. This approach o assessmen allows he inegraion of qualiaive judgemen wih quanified informaion o generae a more comprehensive inerpreaion of suden achievemen. Personalised The STUDENT CENTERED aspec of porfolios ensures ha he suden assumes an acive role in direcing he evidence while eacher involvemen is essenial in monioring he process. By personalised, we imply he suden individualised selecion of evidence, he individualised experience conained in he evidence, he review process by which he examiner learns he unique suden s characerisics and he opporuniy given o sudens o reflec and defend heir work. This process creaes a highly PERSONALISED assessmen mehod. When he porfolio approach is conrased wih he common forms of an OSCE clinical saion, he OSCE will focus mainly on STANDARDISATION of he encouner while providing an individual assessmen profile. The porfolio seeks o balance he personalised approach o assessmen and he sandardisaion of he process. Sandardised I is difficul o achieve maximum conrol of sandardisaion for he porfolio conen and process. I is possible o deermine he ype of work o be submied (case sudies, projecs, essay, ec). However, human performance wih regard o srucure, deph, volume, maerial exhibiion is highly variable (Pis e al 2001). In heir argumens Pis e a (2001) fear ha a oo srucured approach o developing guidance for using a porfolio as an assessmen ool migh diminish is srengh. The dilemma is how o mainain he personalised aspec of porfolio bu a he same ime sandardize he approach o enhance he porfolio s reliabiliy. Some suggesions for sandardisaion are provided in Table 3. 1 Same porfolio s unis of evidence are assigned o all sudens. 2 Tasks and crieria for assessmen are defined and made clear. 3 Insrucions o sudens provide clear guidelines. 4 The porfolio reading process and raing of maerial follow sandardised guidelines mainly by wrien insrucions and raining workshops for examiners. 5 The probing in an oral review of he porfolio wih he suden follows sandardised guidelines. 6 Pass/fail decisions follow a pre-deermined policy. Table 3: Suggesions for sandardisaion of porfolio assessmen -13 -

14 In cases where porfolio assessmen incorporaes an oral porfolio review componen (noe his is no a radiional oral examinaion) one has o consider sraegies o avoid he many pifalls of oral examinaion (Missin 1985). Facors which accoun for problems in oral examinaions include: axonomic level of quesions (Des Marchais 1995), he subjeciviy of examiners, he lack of sandardised crieria employed by examiners eiher for probing or for grading, lack of shared inerpreaion of examinees behaviour and varied levels of expecaions from examinees. All may conribue significanly o lack of sandardisaion and low reliabiliy. However, some of hese problems could be recified by raining and by exensive reading of individual suden s porfolios which helps prepare he examiners for a conexualised and personalised oral encouner while mainaining a holisic framework of suden performance. The sandardisaion process is cenral o he reliabiliy and he validiy of he porfolio. The validiy of grading relaes o he inerpreaion by he examiners of sudens work, he expecaions, shared undersanding of crieria for assigning he grade and he use of he resuls (Messick 1994). Auhenic Auheniciy is defined as he exen o which he oucomes measured represen appropriae, meaningful, significan and worhwhile forms of human accomplishmens. (Archibald and Newmann 1988). Characerisics of auhenic achievemens as judged in porfolio assessmen are provided in Table 4. 1 Producion of knowledge raher han reproducion or ciing ohers work 2 Disciplined enquiry, dependen on: A prior knowledge base (o be used o help produce knowledge). In-deph undersanding Inegraion he producion of knowledge requires he abiliy o organise, synhesise and inegrae informaion in new ways. 3 Value beyond assessmen aesheic, uiliarian or personal value. 4 Higher order hinking. Table 4: Characerisics of auhenic achievemens as judged in porfolio assessmen (Cumming e al 1999) The porfolio is auhenic in he sense ha i incorporaes pas academic and work-relaed experiences which form he foundaions for he above-menioned crieria. Messick (1994) saes ha a porfolio is an auhenic simulaion in ha i does no simulae a paricular ask, bu raher requires ha he examinees by concepualizing abou he case will generalize o oher occasions. The auhenic aspec of he porfolio is cenral o he validiy of he porfolio maerial and he scores inerpreaions. I is also an essenial aspec of developing professional aiudes while reflecing on one s own experiences (Schon 1987). For his reason Case (1994) saes her concern for he loss of porfolio auheniciy in large scale, more sandardised, less personalised examinaions. The personalised approach of porfolio assessmen is he unique componen which conribues o is auheniciy, i.e. is validiy. The lack of i may increase is reliabiliy bu may also decrease is validiy. 2 Raers consisency Discussions of reliabiliy are ofen cenered around erms of generalisabiliy ; ha is, he exen o which performance is consisen across various sources of error labelled faces. (Korez 1998). In he porfolio case, raers are considered as one face of error ha hreaens generalizabiliy of performance. The raers effec includes issues of raers sabiliy over ime (Tes - rees), sabiliy over raers (Inerraer) and reproducibiliy or decision consisency of pass/fail marks (Livingson 1995). Given he complexiy of porfolio assessmen, he amoun of ime involved in processing he es and he acual knowledge learned abou examinees from he firs adminisraion, presen difficulies for sudying raers sabiliy over a relaively shor ime. As for iner raer agreemen, Le Mahier e al (1993) repored.60 o.70 iner-raer correlaions and generalizabiliy esimaes beween wo raers for each uni of porfolio evidence in he range of.80. Herman e al (1995) repored average correlaions beween pairs of raers of.82. Percenages of absolue agreemens of all pairs averaged resuled in 98%. Korez (1998) saes ha much of he oal error in scores was aribued o facors oher han raers in he porfolio. High agreemen depends on clear crieria, adequae examiners raining, communicaing crieria o sudens, good suden orienaion maerials, examiners familiariy wih he conex and shared undersanding of expeced -14 -

15 suden performance and he assessmen purpose. In he Dundee case sudy, as in oher porfolio assessmens, he sabiliy over raers is mainly relaed o he consisency of heir inerpreaion of paerns of performance over ime and over asks, raher han agreemen on a single uni of performance or on a single grade. The shared undersanding of suden progress owards he curriculum learning oucomes in Dundee is he focus of examiners raining. Auhenic crieria, which incorporae progressional benchmarks owards he global oucomes, is he Dundee curren challenge for achieving high inerraer reliabiliy. We will furher discuss developmen of crieria in he secion regarding implemenaion. However, one needs o quesion he meaning of inerraer consisency in porfolio assessmen. For example, in a porfolio which conains one sample of bes work, for he purpose of assessing an abiliy such as wriing, high absolue iner-raer agreemen will be essenial if decisions are based on he wriing score aained. However, for porfolios which conain broad, complex, muliple asks, one needs o quesion he likelihood of wo independen examiners arriving a idenical grading of individual ses of evidence. In his case, i migh be imporan ha examiners agree on he consequences of he grading raher han on he acual grade obained. 3 Decision consisency Misclassificaion of sudens ino pass/fail or referred caegories due o raers source of error, is a main concern regarding disagreemen among raers. The lack of reproducibiliy as a funcion of low inerraer agreemen, is a major hrea o he reliabiliy of he porfolio examinaion in high sakes examinaion or any porfolio examinaion wih summaive consequences. Iner-raer agreemens for unis of porfolio may resul in high correlaions. However, wha is imporan is he reproducibiliy of pass/fail decisions among raers. For example, do he pass/fail sandards/ crieria or oher policy decisions assigned o he marks produce he same classificaion consequences across raers? If correlaions are high among raers, while one raer consisenly raes lower, sudens who obained he low raings may fail based on he examinaion sandards, while sudens who achieved higher raings may pass even hough he wo raers grading correlaed very highly. Consequenly, agreemens expressed in correlaion saisics are no sufficien if pass/fail sandards are employed. Correlaion saisics and percen agreemens should accompany he following quesions: 1) Wha is he exen of agreemen for individual unis among raers in reaching he same pass/fail resuls? (Livingson 1995). 2) Irrespecive of he grades assigned, do raers reach he same pass/fail decision? Daa available from he firs year of he Dundee porfolio adminisraion, shows 98% pass/referred agreemen beween wo pairs of examiners, who independenly examined he sudens. Generalizabiliy coefficiens will define he amoun of error aribued o raers. I will also assis in esimaing he number of raers needed o achieve high reproducibiliy (Brennan 1983). 4 Sampling Sampling of asks is anoher imporan face of measuremen error in porfolio assessmen. In fac, given he complexiy of behaviours measured by he porfolio assessmen, and in order o achieve consisency of performance across a number of asks, sampling needs o be exensive. Linn (1994) raises he issue of limied sampling in performance assessmen and urges he use of sampling of suden work over ime and over asks o reduce sampling error due o ask, by employing porfolios and/or muliple sources of performance assessmen. Appropriae sampling is a common problem in performance assessmen due o limied examinaion ime. The Dundee approach allows such sampling hrough he preparaion of evidence over ime, over occasions, employing muliple asks while addressing he 12 oucomes. This approach is differen from a porfolio in which one uni of bes work is presened as evidence. In he bes work case, when a limied number of compeences are assessed, he suppor provided by more evidence of he same abiliy as in he Vermon example, will increase he porfolio validiy due o broader sampling. 5 The consensus approach In he Dundee porfolio examinaion, judgemens on suden performance are generaed by incorporaing a consensus approach. Can one employ a consensus approach for pass/fail decisions? How scienific and accurae is he consensus process beween wo raers or among commiee members? There is evidence ha scienific heories are deermined by a consensus approach raher han by a criical hinking process, or by consisency among independen research findings (Hand 1999). I is common pracice in assessmen cenres o incorporae a consensus approach for judging rainees performance (Gagler e al 1987). In pracice, he selecion of rainees and oher higher level -15 -

16 personnel is conduced by selecion commiees which employ a consensus process raher han inerraer agreemens. Juries also operae on a consensus principle. In boh cases evidence is presened, he panel argues he case and decision is graned following a consensus process. Relevan o personalizing assessmen in a performance domain, is he issue of how one guards agains subjeciviy of raers. The porfolio approach, by is complex naure, calls for judgemen of muliple performance aspecs, cogniive, aiudinal, professional and echnical skills ha are bes expressed a he personalised level. The hreas o reliabiliy among raers, in such a complex and personalised underaking, raises he quesion: should consensus procedures be recognised as a legiimae and reliable assessmen sraegy and can consensus procedures replace independen raings? Linn (1994) menions he work of Moss (1994) on validiy versus reliabiliy and brings examples of search commiees examining he credenials of candidaes and arriving a an inegraed decision. Delandshere and Perosky (1994) alk abou confirmaion of prior judgemen raher han replicaion, analogous o a second opinion from a physician. Linn (1994) saes ha he confirmaion approach may be a useful way of looking a suden porfolios and exhibiions? However, in such cases examinee performance is judged by inernal examiners who are familiar wih he educaional conex and he oucome framework as i is pracised, for example, in Dundee. Due o he highly qualiaive naure of porfolios, i seems reasonable o mainain independen raing and srive for high iner-raer agreemen. A he same ime, we may ask: how can we equip examiners wih he righ ools and he appropriae definiions o judge examinees independenly and furher submi hese judgemens o a rigorous consensus process? The use of examples and evidence o suppor raers judgemens agains pre-deermined crieria is one way o achieve a meaningful raings and consensus process. Jusificaion of grading wihin defined frameworks and shared crieria may creae a meaningful and reliable consensus process. In he Dundee example, wo sages of independen raings provide he basis for he final consensus process, which leads o an overall judgemen and agreemen beween he wo raers. I seems ha he hree forms of reliabiliy: agreemen among independen raings, reproducibiliy of pass/fail decisions and rigorous consensus procedures may all provide an inegraed soluion for achieving high reliabiliy and for he use of porfolios for summaive purposes. When raers provide heir independen grading hey own his decision. In he consensus process hey mus negoiae he grades by supplying evidence of suden performance and explain heir inerpreaion of he performances. Iner-raer agreemens could be sudied as well as reproducibiliy of pass/fail decisions o esablish he overall reliabiliy of he porfolio examinaion. A sronger design may incorporae wo pairs of examiners, each providing individual raings as well as consensus judgemen. This approach was implemened in Dundee in he firs year of he porfolio examinaion. Each pair of examiners conduced a 20-minue oral review of he porfolio wih he suden. However, i was fel ha some sudens needed more ime wih each pair and, on many occasions, sudens faced he same quesions by he wo pairs albei concerning differen componens of he porfolio. There was also an issue of manpower. Currenly, Dundee employs only one pair of examiners for he 40 minue oral review of he porfolio wih he suden, bu his policy is under review. One proposal is o have wo pairs of examiners wih a suden for 30 minues each. Work needs o be done o furher sudy he validiy and reliabiliy of he consensus approach as complemenary o oher forms of reliabiliy. 6 Forms of validiy Discussions of validiy include he appropriaeness of score inerpreaion and he specific inferences made from he es (Messick 1994). The validiy of he inferences depends also on he reliabiliy of he es. If es scores suffer from low accuracy due o low iner-raer agreemen or poor sampling, inferences canno be made. The auheniciy aspec of he porfolio srenghens he predicive validiy of he porfolio assessmen. If porfolio examinaions are adequae o measure high level, complex abiliies applicable in real life siuaions, hen is predicive validiy should demonsrae srong relaionship wih laer professional or career performance. Such long-erm sudies were no found. Finlay e al (1998) conduced a randomized conrolled sudy and has shown beer resuls for he porfolio (experimenal) group on Oncology facual knowledge. Those submiing porfolios had overall higher marks han hose who have seleced no o submi. This sudy may also indicae differences in moivaion. As for consruc validiy, Korez e al (1993) found moderae correlaions ( ) beween wriing -16 -

17 porfolio scores and oher direc wriing assignmen. Where pass/fail decisions are considered, he dilemma of low correlaions beween differen measures becomes more problemaic. One has o ensure ha sudens will achieve masery on boh he porfolio and oher forms of wriing abiliy o esablish comparabiliy of measures. Also i was found ha differen scoring approaches o he porfolio may produce differen classificaion resuls. (Korez 1993). Currenly, a cenral validiy aspec of he porfolio assessmen is is face validiy. The process, he evidence and he personalised approach are aracive o boh sudens and educaors. However, work is needed o furher esablish is predicive validiy and is relaionship o oher measures of suden performance. 7 Exernal vs inernal examiners I is argued ha porfolio assessmen is more suiable for inernal examiners (Linn 1994) where familiariy wih he educaional programme and wih suden progression ensures consisency of marking. The personalised approach of porfolio assessmen requires inernal knowledge of he educaional milieu. Exernal examiners a he sae or disric level may be recognised more as examiners for qualiy conrol of he porfolio process. This may have implicaions for judging porfolio work a a cenral sie wihou seeing he examinee or lack of familiariy wih he educaional conex. The purpose of such cenral marking of porfolio assessmen should be clarified and he crieria employed for judgemen should incorporae sandardisaion procedures as well as generic crieria suiable o all examinees independen of educaional programme. Exernal examiners who si in a personalised porfolio may differ in heir inerpreaion of examinee performance due o a differen educaional philosophy in heir own schools. Exernal examiners, however, are crucial in he assessmen sysems of UK medical schools. When exernal examiners paricipae in porfolio examinaions, seps should be aken o ensure ha hey do no bias he resuls. Specific raining programmes should be designed for exernal examiners o familiarise hem wih he curriculum, suden assessmen and educaional philosophy of he school. The conexual effec on porfolio performance as well as he personalised approach o sudens are forces which aemp o localize he porfolio raher han use i for exernal esing. The large number of sudies on porfolio implemenaion indicae ha inernal as well as exernal uses are repored. The difference in he applicaion lies in he purpose of he porfolio and he crieria employed for judgemen. Exernal uses may employ general crieria for sudens work which is independen of educaional or work-relaed conex. The exernal marking of suden work may generae group daa on school profiles, sudens level of achievemen and he qualiy conrol of educaional programmes. D: Implemenaion of porfolio assessmen Implemenaion of porfolio assessmens commonly incorporaes a sequence of seps. These seps are idenified in Table 5. For each sep we will bring examples o illusrae some pracical consideraions. 1 Defining he purpose 2 Deermining compeences o be assessed 3 Selecion of porfolio maerial 4 Developing a marking sysem 5 Selecion and raining of examiners 6 Planning he examinaion process 7 Suden orienaion 8 Developing guidelines for decisions 9 Esablishing reliabiliy and validiy evidence 10 Designing evaluaion procedures Table 5: Seps in developing porfolios 1 Defining he purpose Porfolios are used for summaive and formaive decisions, for differen rainees levels and in a variey of seings. The variaion in is conen, process and applicaion requires ha he purpose will be clearly defined. The non-radiional assessmen approaches employed by porfolios may inroduce confusion and uncerainy regarding is purpose. The clearer he purpose, he higher he qualiy of he porfolio assessmen. An example of a purpose definiion The purpose of he porfolio assessmen is o deermine, in line wih he GMC guidelines, ha he graduaing suden is ready o proceed o he Preregisraion House Office (PRHO) year and ha he -17 -

18 suden has achieved he required leaning oucomes. The porfolio is only one componen of he final examinaion. Successful performance on he porfolio is a requiremen for graduaion. The purpose saemen direcs he es designers o consider he compeencies essenial for enering he PRHO year, o deermine which compeencies are already measured by he oher final examinaion componens, and how he porfolio should supplemen he oher assessmen mehods. In his saemen, he porfolio is one hurdle which necessiaes a summaive decision in order o fulfil graduaion requiremens. The purpose defines he assessmen sysem and he relaionship beween he porfolio and he oher assessmen componens. 2 Deermining compeences o be assessed Faculy, medical educaors, assessmen commiee and oher medical school saff will mee o discuss he purpose of porfolio assessmen. They will review he exising assessmen procedure and will idenify gaps in assessmen which should be supplemened by he porfolio. In Dundee, where he porfolio is defined as a requiremen owards graduaion, he firs 11 oucomes are assessed a he knows and knows how level (Miller 1991) by EMIs and CRQs, and a he shows how level by he OSCE. A he level of does, however, oucome 12 is only assessed by he porfolio. Table 6 shows he grid of final examinaion and he associaed mehods for each oucome. The porfolio, hrough sudens cumulaive work, idenifies srenghs and weaknesses in all oucomes wih emphasis on oucomes The porfolio designers may also emphasise he inerrelaions among he oucomes and sudens overall progress owards all oucomes. The process of idenificaion of compeencies o be assessed signifies a sysemic and sysemaic approach o assessmen (Friedman 1999) which ensures a comprehensive sraegy o deermine qualificaion for graduaion. The principle ha only one assessmen mehod is inadequae o deermine finess for pracice guides his design. 3 Selecion of porfolio maerial The idenificaion of compeencies o be measured is a saring poin for selecion of porfolio maerial. I is imporan o idenify key performance behaviours relevan o he compeency measured. The idenified behaviours will guide he porfolio conen selecion. For example, in assessing rainees paien managemen skills, one of he seleced performance behaviours could be paien educaion. The porfolio designers will search for a work sample of paien educaion. Evidence migh include wrien ouline of a paien educaion programme in he communiy, or a video of individual paien educaion session wih paien discussing smoking cessaion. I is imporan o remember ha evidence is seleced from ongoing rainee s work and is no creaed in an arificial way jus for he porfolio examinaion. I is also essenial ha he compilaion of evidence will serve as learning and educaional experience for he rainee. For example, in selecing a video for paien educaion, rainees will employ crieria for selecing he bes paien educaion encouner. In defending he selecion, he rainee demonsraes undersanding of crieria for opimal performance. For oucome 9, appropriae aiudes, ehical and legal undersanding and responsibiliy, he porfolio maerial in Dundee conains he elecive repor which provides evidence if he suden shows ehical undersanding of issues inheren in he elecive. In a case discussion on ehics, he sudens will provide evidence of ehical judgemen and moral reasoning and could be quesioned abou he case. In he Dundee case sudy, where he purpose of assessmen is progress owards he 12 oucomes, he evidence should indicae progress over ime. A selecion of samples of suden work will faciliae discussion of paerns of performance and overall srenghs and weaknesses. 4 Developing a marking sysem In he Vermon applicaion of porfolios, generic crieria were developed specifically for grade 4 in mahemaics. Sudens work is judged by crieria which will specify he level of heir academic achievemens and will deermine heir progress owards sae sandards. This approach is possible when one uni of porfolio conen is evaluaed. However, in he Dundee case sudy, he 12 broadly defined oucomes do no allow he applicaion of specific crieria, bu raher general saemens regarding suden performance relaive o he oucome specificaion. For example, under he oucome Informaion Handling he following dimensions are specified: he docor is compeen in recording, rerieving and analysing informaion using a range of mehods including compuers. Crieria for marking may include evaluaion of -18 -

19 Oucomes EMI 1 CRQ 2 OSCE Porfolio 1 Clinical skills xxx xxx xxx xx 2 Pracical procedures xx xx xxx xx 3 Invesigaing a paien xx xx xxx xx 4 Paien managemen xxx xxx xxx xx 5 Healh promoion and disease prevenion xx xx xxx xx 6 Communicaion skills x x xxx xxx 7 Informaion handling xx xx x xxx 8 Undersanding of basic, clinical and social sciences xxx xxx x xxx 9 Appropriae aiudes ehical and legal responsibiliy xx xx xx xxx 10 Decision making, clinical reasoning xxx xxx xxx xxx 11 Role of he docor x x xx xxx 12 Personal developmen x x xx xxx 1 EMI (Exended Maching Iems), 2 CRQ (Consruced Response Quesions) x considered for use, xx appropriae and is currenly used, xxx mos appropriae assessmen mehod Table 6: Final examinaion grid in Dundee oucome-based curriculum paien records (including specific marks for qualiy of records). Crieria for search sraegies (including marks for formulaing research quesions and generaing key words for muliple daabases) and crieria for analysing informaion (including marks for criique of a research paper). The porfolio maerial should direc he examiner o consider suden progress according o he oucome specificaion and should enable he examiner o idenify srenghs and weaknesses. As menioned earlier, highly specific crieria could be employed if one compeence is assessed. For muliple compeences, general sandards should be developed. I is also possible as in he Dundee case sudy o assign grades o suden global work. However, prior o grades assignmen, examiners mus share undersanding of he specific or general crieria and consisenly negoiae wha consiues an A level grade (excellen) versus an F level (fail) and furher define and refine he poins on he coninuum. Schools may be bound by universiy grading sysems. The bes approach will be o follow he above procedures. Definiion of crieria for performance and idenificaion of level of performance which will correspond o he universiy grading sysem. 5 Selecion and raining of examiners If one considers only an inernal examiners sysem for porfolio assessmen, selecion of examiners will follow he inernal srucure and philosophy of he educaional program. Each porfolio according o is purpose will dicae he appropriae examiners. These will include a wide range of saff; eachers in he basic sciences and laboraory-based disciplines, clinicians, faculy who indicae special ineres in educaion and in suden developmen, faculy who are moivaed o spend he exra ime needed and faculy who are good oral examiners (mainly suden cenred). Anoher selecion issue relaes o he senioriy of examiners. Pairing of new examiners wih more senior colleagues wih experience of porfolio assessmen has been found o be helpful in Dundee. Whaever he selecion process employed, he raining of faculy examiners and mainaining hem in he examiners pool is a key poin for he success of he programme. Naurally he amoun of ime and effor invesed in he examiners pool hrough raining and orienaion and heir accumulaed experience wih porfolio examinaion, should be preserved and reinforced. Consequenly, hey should be reaed accordingly (dinners, lunches) and given help o form a group wih recognised academic respec and appreciaion. The fac ha a faculy member says I can join you for omorrow s meeing I am a porfolio examiner, should bring pride o he speaker and respec from he lisener. Faculy in general appreciae heir paricipaion in porfolio examinaions (Davis 2001). I provides an opporuniy o know he sudens and o share opinions wih heir colleagues. They obain beer perspecives on suden accumulaed work and progress. Alhough he amoun of reading migh be -19 -

20 excessive (a problem if one decided o mainain he broad sampling principle) realisic approaches o reading plans can keep his burden a manageable levels. 6 Planning he examinaion process Forunaely, he use of OSCEs has inroduced he concep of individualised suden assessmen. This concep, ogeher wih he ime needed, faculy manpower, logisics, adminisraion issues and availabiliy of suppor saff are all issues shared wih porfolio examinaions. In Dundee, wih a class of 150, each pair of examiners sees five sudens for a oal examinaion ime of 200 minues (40 minues for each oral review of he porfolio wih he suden). If we include lunch, coffee breaks, discussion ime beween candidaes, filling up forms and he Examiners Commiee meeing a he end of he day five sudens per day is he maximum number. If 10 pairs of examiners are recruied for each day all 150 sudens could si he oral review examinaion in hree days wih differen ses of examiners. The number of examiners needed for such an assessmen operaion is 60 wih each examiner being commied for jus one day. There is, of course, he addiional ime examiners spend reading he porfolios before he meeing wih he sudens, approximaely one hour per porfolio in Dundee. Faculy needs o buy ino he concep of porfolio o be willing o paricipae in such an imporan underaking. As wih all aspecs of reform, srong leadership is needed wihin he medical school o faciliae he launch of he firs adminisraion and keep he momenum in subsequen years in spie of any reservaions among faculy, sudens and adminisraors. when he guidelines are oo broad. However, in he Dundee case sudy, sudens use heir on-going work for selecion of maerial and usually, if hey demonsrae good progress in heir achievemens, hey have confidence in heir abiliy o pass he porfolio. The more informaion given o sudens he more posiive hey become owards he porfolio. Following he porfolio examinaion, sudens are pleased ha senior examiners have given deailed aenion o heir work over heir las wo years of medical school. The sudens, in general, like his personalised, suden cenered, conexualised assessmen approach. 8 Developing guidelines for decisions Once a grading sysem is developed, he porfolio designers mus develop policy and guidelines for sandards of performance. If he porfolio is used for summaive pass/fail decisions, sandards should specify wha consiues a fail or a pass. If porfolios are used for formaive decisions, sandards for srenghs and weaknesses should be defined along wih due process for inervenion and remediaion. The process should also incorporae he procedures for evaluaing he inervenion and he decision mechanisms o furher promoe or o hold back/expel he suden. The decision making process may be oulined by he following flow diagram. Porfolio examinaion Pass Fail Referred Si for anoher exam Undergo remediaion 7 Suden orienaion Sudens mus be informed a he beginning of he undergraduae course abou he porfolio examinaion and kep up o dae wih any changes in subsequen years. Pass Graduae Fail Commiee review Fail Pass Graduae A special orienaion bookle for sudens is issued o inform hem of he purpose and conen of porfolio building, he porfolio assessmen process, he examinaion day, he marking sysem and he use of resuls. The more complex he ask of porfolio building, he more anxious he suden becomes. Problems may occur if he guidelines and crieria for judging performance are no enirely clear as may happen Figure 3: Flow diagram for he decision-making process 9 Esablishing reliabiliy and validiy evidence I is imporan o deermine, prior o he implemenaion of he porfolio, wha will consiue good reliable evidence and plan he examinaion -20 -

USE OF EDUCATION TECHNOLOGY IN ENGLISH CLASSES

USE OF EDUCATION TECHNOLOGY IN ENGLISH CLASSES USE OF EDUCATION TECHNOLOGY IN ENGLISH CLASSES Mehme Nuri GÖMLEKSİZ Absrac Using educaion echnology in classes helps eachers realize a beer and more effecive learning. In his sudy 150 English eachers were

More information

Performance Center Overview. Performance Center Overview 1

Performance Center Overview. Performance Center Overview 1 Performance Cener Overview Performance Cener Overview 1 ODJFS Performance Cener ce Cener New Performance Cener Model Performance Cener Projec Meeings Performance Cener Execuive Meeings Performance Cener

More information

Reporting to Management

Reporting to Management CHAPTER 31 Reporing o Managemen Inroducion The success or oherwise of any business underaking depends primarily on earning revenue ha would generae sufficien resources for sound growh. To achieve his objecive,

More information

Course Outline. Course Coordinator: Dr. Tanu Sharma Assistant Professor Dept. of humanities and Social Sciences Email:tanu.sharma@juit.ac.

Course Outline. Course Coordinator: Dr. Tanu Sharma Assistant Professor Dept. of humanities and Social Sciences Email:tanu.sharma@juit.ac. Course Name : HUMAN RESOURCE MANAGEMENT Course Code: 10B1WPD75 Course Credi: (-0-0) Semeser: VII Course Type: Elecive (All B. Tech. sudens) Deparmen: Humaniies and Social Sciences Course Coordinaor: Dr.

More information

Idealistic characteristics of Islamic Azad University masters - Islamshahr Branch from Students Perspective

Idealistic characteristics of Islamic Azad University masters - Islamshahr Branch from Students Perspective Available online a www.pelagiaresearchlibrary.com European Journal Experimenal Biology, 202, 2 (5):88789 ISSN: 2248 925 CODEN (USA): EJEBAU Idealisic characerisics Islamic Azad Universiy masers Islamshahr

More information

Appendix D Flexibility Factor/Margin of Choice Desktop Research

Appendix D Flexibility Factor/Margin of Choice Desktop Research Appendix D Flexibiliy Facor/Margin of Choice Deskop Research Cheshire Eas Council Cheshire Eas Employmen Land Review Conens D1 Flexibiliy Facor/Margin of Choice Deskop Research 2 Final Ocober 2012 \\GLOBAL.ARUP.COM\EUROPE\MANCHESTER\JOBS\200000\223489-00\4

More information

Chapter 1.6 Financial Management

Chapter 1.6 Financial Management Chaper 1.6 Financial Managemen Par I: Objecive ype quesions and answers 1. Simple pay back period is equal o: a) Raio of Firs cos/ne yearly savings b) Raio of Annual gross cash flow/capial cos n c) = (1

More information

PROFIT TEST MODELLING IN LIFE ASSURANCE USING SPREADSHEETS PART ONE

PROFIT TEST MODELLING IN LIFE ASSURANCE USING SPREADSHEETS PART ONE Profi Tes Modelling in Life Assurance Using Spreadshees PROFIT TEST MODELLING IN LIFE ASSURANCE USING SPREADSHEETS PART ONE Erik Alm Peer Millingon 2004 Profi Tes Modelling in Life Assurance Using Spreadshees

More information

Duration and Convexity ( ) 20 = Bond B has a maturity of 5 years and also has a required rate of return of 10%. Its price is $613.

Duration and Convexity ( ) 20 = Bond B has a maturity of 5 years and also has a required rate of return of 10%. Its price is $613. Graduae School of Business Adminisraion Universiy of Virginia UVA-F-38 Duraion and Convexiy he price of a bond is a funcion of he promised paymens and he marke required rae of reurn. Since he promised

More information

The Grantor Retained Annuity Trust (GRAT)

The Grantor Retained Annuity Trust (GRAT) WEALTH ADVISORY Esae Planning Sraegies for closely-held, family businesses The Granor Reained Annuiy Trus (GRAT) An efficien wealh ransfer sraegy, paricularly in a low ineres rae environmen Family business

More information

Nowadays, almost all health organizations do not have

Nowadays, almost all health organizations do not have Original Aricle The Developmen of Nurse Residency Program OBJECTIVE: To sudy new graduae nurses work problems; o design raining program courses as well as he experimenal evaluaion of he effeciveness of

More information

Chapter 8: Regression with Lagged Explanatory Variables

Chapter 8: Regression with Lagged Explanatory Variables Chaper 8: Regression wih Lagged Explanaory Variables Time series daa: Y for =1,..,T End goal: Regression model relaing a dependen variable o explanaory variables. Wih ime series new issues arise: 1. One

More information

CAREER MAP HOME HEALTH AIDE

CAREER MAP HOME HEALTH AIDE CAREER MAP HOME HEALTH AIDE CAREER MAP HOME HEALTH AIDE Home healh aides are one of he fases growing jobs in New York Ciy. Wih more educaion, home healh aides can move ino many oher ypes of jobs in healh

More information

Morningstar Investor Return

Morningstar Investor Return Morningsar Invesor Reurn Morningsar Mehodology Paper Augus 31, 2010 2010 Morningsar, Inc. All righs reserved. The informaion in his documen is he propery of Morningsar, Inc. Reproducion or ranscripion

More information

Initiative to Transform Medical Education

Initiative to Transform Medical Education American Medical Associaion Iniiaive o Transform Medical Educaion American Medical Associaion Recommendaions for change in he sysem of medical educaion Iniiaive o Transform Medical Educaion Recommendaions

More information

PROFIT TEST MODELLING IN LIFE ASSURANCE USING SPREADSHEETS PART TWO

PROFIT TEST MODELLING IN LIFE ASSURANCE USING SPREADSHEETS PART TWO Profi Tes Modelling in Life Assurance Using Spreadshees, par wo PROFIT TEST MODELLING IN LIFE ASSURANCE USING SPREADSHEETS PART TWO Erik Alm Peer Millingon Profi Tes Modelling in Life Assurance Using Spreadshees,

More information

Market Liquidity and the Impacts of the Computerized Trading System: Evidence from the Stock Exchange of Thailand

Market Liquidity and the Impacts of the Computerized Trading System: Evidence from the Stock Exchange of Thailand 36 Invesmen Managemen and Financial Innovaions, 4/4 Marke Liquidiy and he Impacs of he Compuerized Trading Sysem: Evidence from he Sock Exchange of Thailand Sorasar Sukcharoensin 1, Pariyada Srisopisawa,

More information

Risk Modelling of Collateralised Lending

Risk Modelling of Collateralised Lending Risk Modelling of Collaeralised Lending Dae: 4-11-2008 Number: 8/18 Inroducion This noe explains how i is possible o handle collaeralised lending wihin Risk Conroller. The approach draws on he faciliies

More information

In a class I teach for prospective administrators,

In a class I teach for prospective administrators, SEDL Leer APRIL 2007 11 The Professional Teaching and Learning Cycle: Implemening a Sandards-Based Approach o Professional Developmen In a class I each for prospecive adminisraors, a suden shared he sory

More information

GoRA. For more information on genetics and on Rheumatoid Arthritis: Genetics of Rheumatoid Arthritis. Published work referred to in the results:

GoRA. For more information on genetics and on Rheumatoid Arthritis: Genetics of Rheumatoid Arthritis. Published work referred to in the results: For more informaion on geneics and on Rheumaoid Arhriis: Published work referred o in he resuls: The geneics revoluion and he assaul on rheumaoid arhriis. A review by Michael Seldin, Crisopher Amos, Ryk

More information

Identify and ranking the factors that influence establishment of total quality management system in Payame Noor University of Lordegan

Identify and ranking the factors that influence establishment of total quality management system in Payame Noor University of Lordegan Idenify and ranking he facors ha influence esablishmen of oal qualiy sysem in Payame Noor Universiy of Lordegan Farhad Farhadi MA Suden, Deparmen of Managemen, Najafabad Branch, Islamic Azad Universiy,

More information

Advise on the development of a Learning Technologies Strategy at the Leopold-Franzens-Universität Innsbruck

Advise on the development of a Learning Technologies Strategy at the Leopold-Franzens-Universität Innsbruck Advise on he developmen of a Learning Technologies Sraegy a he Leopold-Franzens-Universiä Innsbruck Prof. Dr. Rob Koper Open Universiy of he Neherlands Educaional Technology Experise Cener Conex - Period

More information

Relative velocity in one dimension

Relative velocity in one dimension Connexions module: m13618 1 Relaive velociy in one dimension Sunil Kumar Singh This work is produced by The Connexions Projec and licensed under he Creaive Commons Aribuion License Absrac All quaniies

More information

Individual Health Insurance April 30, 2008 Pages 167-170

Individual Health Insurance April 30, 2008 Pages 167-170 Individual Healh Insurance April 30, 2008 Pages 167-170 We have received feedback ha his secion of he e is confusing because some of he defined noaion is inconsisen wih comparable life insurance reserve

More information

Child Protective Services. A Guide To Investigative Procedures

Child Protective Services. A Guide To Investigative Procedures Child Proecive Services A Guide To Invesigaive Procedures The purpose of his brochure is o help you undersand he Child Proecive Services (CPS) reporing and response process. Please conac your CPS worker

More information

Distributing Human Resources among Software Development Projects 1

Distributing Human Resources among Software Development Projects 1 Disribuing Human Resources among Sofware Developmen Proecs Macario Polo, María Dolores Maeos, Mario Piaini and rancisco Ruiz Summary This paper presens a mehod for esimaing he disribuion of human resources

More information

SKF Documented Solutions

SKF Documented Solutions SKF Documened Soluions Real world savings and we can prove i! How much can SKF save you? Le s do he numbers. The SKF Documened Soluions Program SKF is probably no he firs of your supplier parners o alk

More information

METHODS OF LEARNING IN STATISTICAL EDUCATION: A RANDOMIZED TRIAL OF PUBLIC HEALTH GRADUATE STUDENTS 3

METHODS OF LEARNING IN STATISTICAL EDUCATION: A RANDOMIZED TRIAL OF PUBLIC HEALTH GRADUATE STUDENTS 3 5 METHODS OF LEARNING IN STATISTICAL EDUCATION: A RANDOMIZED TRIAL OF PUBLIC HEALTH GRADUATE STUDENTS 3 FELICITY BOYD ENDERS, PHD Mayo Clinic, Division of Biosaisics enders.feliciy@mayo.edu MARIE DIENER-WEST,

More information

METHODS OF LEARNING IN STATISTICAL EDUCATION: A RANDOMIZED TRIAL OF PUBLIC HEALTH GRADUATE STUDENTS 3

METHODS OF LEARNING IN STATISTICAL EDUCATION: A RANDOMIZED TRIAL OF PUBLIC HEALTH GRADUATE STUDENTS 3 5 METHODS OF LEARNING IN STATISTICAL EDUCATION: A RANDOMIZED TRIAL OF PUBLIC HEALTH GRADUATE STUDENTS 3 FELICITY BOYD ENDERS, PHD Mayo Clinic, Division of Biosaisics enders.feliciy@mayo.edu MARIE DIENER-WEST,

More information

Cannibalization and Product Life Cycle Management

Cannibalization and Product Life Cycle Management Middle-Eas Journal of Scienific Research 19 (8): 1080-1084, 2014 ISSN 1990-9233 IDOSI Publicaions, 2014 DOI: 10.5829/idosi.mejsr.2014.19.8.11868 Cannibalizaion and Produc Life Cycle Managemen Ali Farrukh

More information

Market Analysis and Models of Investment. Product Development and Whole Life Cycle Costing

Market Analysis and Models of Investment. Product Development and Whole Life Cycle Costing The Universiy of Liverpool School of Archiecure and Building Engineering WINDS PROJECT COURSE SYNTHESIS SECTION 3 UNIT 11 Marke Analysis and Models of Invesmen. Produc Developmen and Whole Life Cycle Cosing

More information

MACROECONOMIC FORECASTS AT THE MOF A LOOK INTO THE REAR VIEW MIRROR

MACROECONOMIC FORECASTS AT THE MOF A LOOK INTO THE REAR VIEW MIRROR MACROECONOMIC FORECASTS AT THE MOF A LOOK INTO THE REAR VIEW MIRROR The firs experimenal publicaion, which summarised pas and expeced fuure developmen of basic economic indicaors, was published by he Minisry

More information

BALANCE OF PAYMENTS. First quarter 2008. Balance of payments

BALANCE OF PAYMENTS. First quarter 2008. Balance of payments BALANCE OF PAYMENTS DATE: 2008-05-30 PUBLISHER: Balance of Paymens and Financial Markes (BFM) Lena Finn + 46 8 506 944 09, lena.finn@scb.se Camilla Bergeling +46 8 506 942 06, camilla.bergeling@scb.se

More information

Graphing the Von Bertalanffy Growth Equation

Graphing the Von Bertalanffy Growth Equation file: d:\b173-2013\von_beralanffy.wpd dae: Sepember 23, 2013 Inroducion Graphing he Von Beralanffy Growh Equaion Previously, we calculaed regressions of TL on SL for fish size daa and ploed he daa and

More information

Investigation of Human Resource Management Practices (HRM) in Hospitals of Jalgaon District

Investigation of Human Resource Management Practices (HRM) in Hospitals of Jalgaon District BUSINESS AND TECHNOLOGY (IJSSBT), Vol. 1,.2, FEBRUARY 213 Invesigaion of Human Resource Managemen Pracices (HRM) in Hospials of Jalgaon Disric Saroj B. Pail*; Dr. P.T.Choudhari** *Research Scholar, rh

More information

Building an E- Commerce Strategy for the Office Equipment and Printer Marketplace. CAPt CAP VENTURES

Building an E- Commerce Strategy for the Office Equipment and Printer Marketplace. CAPt CAP VENTURES Building an E- Commerce Sraegy for he Office Equipmen and Priner Markeplace The Inerne is creaing a revoluion in he disribuion of office equipmen. New conceps and business models abound, and i seems cerain

More information

TEMPORAL PATTERN IDENTIFICATION OF TIME SERIES DATA USING PATTERN WAVELETS AND GENETIC ALGORITHMS

TEMPORAL PATTERN IDENTIFICATION OF TIME SERIES DATA USING PATTERN WAVELETS AND GENETIC ALGORITHMS TEMPORAL PATTERN IDENTIFICATION OF TIME SERIES DATA USING PATTERN WAVELETS AND GENETIC ALGORITHMS RICHARD J. POVINELLI AND XIN FENG Deparmen of Elecrical and Compuer Engineering Marquee Universiy, P.O.

More information

Documenting Uncertain Times: Post-graduate Transitions of the. Academically Adrift. Cohort. Josipa Roksa

Documenting Uncertain Times: Post-graduate Transitions of the. Academically Adrift. Cohort. Josipa Roksa Documening Uncerain Times: Pos-graduae Transiions of he Academically Adrif Cohor Richard Arum Esher Cho Jeannie Kim Josipa Roksa Acknowledgmens This research projec is organized by he Social Science Research

More information

This is the author s version of a work that was submitted/accepted for publication in the following source:

This is the author s version of a work that was submitted/accepted for publication in the following source: This is he auhor s version of a work ha was submied/acceped for publicaion in he following source: Debnah, Ashim Kumar & Chin, Hoong Chor (2006) Analysis of marine conflics. In Proceedings of he 19h KKCNN

More information

Impact of scripless trading on business practices of Sub-brokers.

Impact of scripless trading on business practices of Sub-brokers. Impac of scripless rading on business pracices of Sub-brokers. For furher deails, please conac: Mr. T. Koshy Vice Presiden Naional Securiies Deposiory Ld. Tradeworld, 5 h Floor, Kamala Mills Compound,

More information

INTRODUCTION TO FORECASTING

INTRODUCTION TO FORECASTING INTRODUCTION TO FORECASTING INTRODUCTION: Wha is a forecas? Why do managers need o forecas? A forecas is an esimae of uncerain fuure evens (lierally, o "cas forward" by exrapolaing from pas and curren

More information

ARCH 2013.1 Proceedings

ARCH 2013.1 Proceedings Aricle from: ARCH 213.1 Proceedings Augus 1-4, 212 Ghislain Leveille, Emmanuel Hamel A renewal model for medical malpracice Ghislain Léveillé École d acuaria Universié Laval, Québec, Canada 47h ARC Conference

More information

The Greek financial crisis: growing imbalances and sovereign spreads. Heather D. Gibson, Stephan G. Hall and George S. Tavlas

The Greek financial crisis: growing imbalances and sovereign spreads. Heather D. Gibson, Stephan G. Hall and George S. Tavlas The Greek financial crisis: growing imbalances and sovereign spreads Heaher D. Gibson, Sephan G. Hall and George S. Tavlas The enry The enry of Greece ino he Eurozone in 2001 produced a dividend in he

More information

Quantity Surveyors career potential as construction project managers

Quantity Surveyors career potential as construction project managers December, 2. 6. 2013 Quaniy Surveyors career poenial as consrucion projec managers Michelle Burger Deparmen of Consrucion Economics Universiy of Preoria Preoria, Souh Africa michelle.burger@up.ac.za Absrac

More information

Academic Advising. Ultimately the college experience you build is your responsibility. Office: Phone: Web: Advising Hours:

Academic Advising. Ultimately the college experience you build is your responsibility. Office: Phone: Web: Advising Hours: C C M T a s h g i e H w e N o k o o b e id u G n io a r is g e R d n a g in is v d A, n io a n ie r Suden O ! C C M T o e m o c l We y. ne ur jo l na io a uc ed ur yo ou gh ou hr o be used ed nd e in is

More information

GUIDE GOVERNING SMI RISK CONTROL INDICES

GUIDE GOVERNING SMI RISK CONTROL INDICES GUIDE GOVERNING SMI RISK CONTROL IND ICES SIX Swiss Exchange Ld 04/2012 i C O N T E N T S 1. Index srucure... 1 1.1 Concep... 1 1.2 General principles... 1 1.3 Index Commission... 1 1.4 Review of index

More information

Measuring macroeconomic volatility Applications to export revenue data, 1970-2005

Measuring macroeconomic volatility Applications to export revenue data, 1970-2005 FONDATION POUR LES ETUDES ET RERS LE DEVELOPPEMENT INTERNATIONAL Measuring macroeconomic volailiy Applicaions o expor revenue daa, 1970-005 by Joël Cariolle Policy brief no. 47 March 01 The FERDI is a

More information

Multiprocessor Systems-on-Chips

Multiprocessor Systems-on-Chips Par of: Muliprocessor Sysems-on-Chips Edied by: Ahmed Amine Jerraya and Wayne Wolf Morgan Kaufmann Publishers, 2005 2 Modeling Shared Resources Conex swiching implies overhead. On a processing elemen,

More information

Term Structure of Prices of Asian Options

Term Structure of Prices of Asian Options Term Srucure of Prices of Asian Opions Jirô Akahori, Tsuomu Mikami, Kenji Yasuomi and Teruo Yokoa Dep. of Mahemaical Sciences, Risumeikan Universiy 1-1-1 Nojihigashi, Kusasu, Shiga 525-8577, Japan E-mail:

More information

PRACTICES AND ISSUES IN OPERATIONAL RISK MODELING UNDER BASEL II

PRACTICES AND ISSUES IN OPERATIONAL RISK MODELING UNDER BASEL II Lihuanian Mahemaical Journal, Vol. 51, No. 2, April, 2011, pp. 180 193 PRACTICES AND ISSUES IN OPERATIONAL RISK MODELING UNDER BASEL II Paul Embrechs and Marius Hofer 1 RiskLab, Deparmen of Mahemaics,

More information

Relationships between Stock Prices and Accounting Information: A Review of the Residual Income and Ohlson Models. Scott Pirie* and Malcolm Smith**

Relationships between Stock Prices and Accounting Information: A Review of the Residual Income and Ohlson Models. Scott Pirie* and Malcolm Smith** Relaionships beween Sock Prices and Accouning Informaion: A Review of he Residual Income and Ohlson Models Sco Pirie* and Malcolm Smih** * Inernaional Graduae School of Managemen, Universiy of Souh Ausralia

More information

Vida Mojtahedzadeh*, Seyed Hossein Alavi Tabari and Maryam Asgharzadeh Badr. Abstract

Vida Mojtahedzadeh*, Seyed Hossein Alavi Tabari and Maryam Asgharzadeh Badr. Abstract A Comparison beween Accouning, Business Adminisraion and Theoreical Economics Sudens on he Impac of Their Majors on Personal Saisfacion in Acquiring General Skills Vida Mojahedzadeh*, Seyed Hossein Alavi

More information

SELF-EVALUATION FOR VIDEO TRACKING SYSTEMS

SELF-EVALUATION FOR VIDEO TRACKING SYSTEMS SELF-EVALUATION FOR VIDEO TRACKING SYSTEMS Hao Wu and Qinfen Zheng Cenre for Auomaion Research Dep. of Elecrical and Compuer Engineering Universiy of Maryland, College Park, MD-20742 {wh2003, qinfen}@cfar.umd.edu

More information

INVESTIGATION OF THE INFLUENCE OF UNEMPLOYMENT ON ECONOMIC INDICATORS

INVESTIGATION OF THE INFLUENCE OF UNEMPLOYMENT ON ECONOMIC INDICATORS INVESTIGATION OF THE INFLUENCE OF UNEMPLOYMENT ON ECONOMIC INDICATORS Ilona Tregub, Olga Filina, Irina Kondakova Financial Universiy under he Governmen of he Russian Federaion 1. Phillips curve In economics,

More information

PolicyCore. Putting Innovation and Customer Service at the Core of Your Policy Administration and Underwriting

PolicyCore. Putting Innovation and Customer Service at the Core of Your Policy Administration and Underwriting PolicyCore Puing Innovaion and Cusomer Service a he Core of Your Policy Adminisraion and Underwriing As new echnologies emerge and cusomer expecaions escalae, P&C insurers are seeing opporuniies o grow

More information

Management problems of electronic information resources: A case study of UNN Library

Management problems of electronic information resources: A case study of UNN Library Vol. 5(5), pp. 126-133, June, 2013 DOI: 10.5897/IJLIS011.097 ISSN 2141 2537 2013 Academic Journals hp://www.academicjournals.org/ijlis Inernaional Journal of Library and Informaion Science Full Lengh Research

More information

Model Formulation to Estimate Manpower Demand for the Real-Estate Construction Projects in India

Model Formulation to Estimate Manpower Demand for the Real-Estate Construction Projects in India Formulaion o Esimae Manpower Demand for he Real-Esae Consrucion Projecs in India Anil L. Agarwal NICMAR; 25/1, Balewadi, Pune (India) anilagarwal@nicmar.ac.in B. L. Rajpu NICMAR; 25/1, Balewadi, Pune (India)

More information

Information Systems for Business Integration: ERP Systems

Information Systems for Business Integration: ERP Systems Informaion Sysems for Business Inegraion: ERP Sysems (December 3, 2012) BUS3500 - Abdou Illia, Fall 2012 1 LEARNING GOALS Explain he difference beween horizonal and verical business inegraion. Describe

More information

The NHSL Medical Education Department

The NHSL Medical Education Department The NHSL Medical Educaion Deparmen professional resusciaion appraisal resources developmen online ucion skills faculy professional resusciaion appraisal resourc elopmen online inducion skills faculy professional

More information

WATER MIST FIRE PROTECTION RELIABILITY ANALYSIS

WATER MIST FIRE PROTECTION RELIABILITY ANALYSIS WATER MIST FIRE PROTECTION RELIABILITY ANALYSIS Shuzhen Xu Research Risk and Reliabiliy Area FM Global Norwood, Massachuses 262, USA David Fuller Engineering Sandards FM Global Norwood, Massachuses 262,

More information

NYWEA Asset Management Task Force White Paper

NYWEA Asset Management Task Force White Paper The New York Waer Environmen Associaion, Inc. The Waer Qualiy Professionals 525 Plum Sree Suie 102 Syracuse, New York 13204 (315) 422-7811 Fax: 422-3851 www.nywea.org e-mail: pcr@nywea.org NYWEA Asse Task

More information

Fourier series. Learning outcomes

Fourier series. Learning outcomes Fourier series 23 Conens. Periodic funcions 2. Represening ic funcions by Fourier Series 3. Even and odd funcions 4. Convergence 5. Half-range series 6. The complex form 7. Applicaion of Fourier series

More information

MANAGEMENT CERTIFICATE PROGRAMS. tr aini ng.ua.edu. Management Certificate Programs To Improve Your Competitive Edge!

MANAGEMENT CERTIFICATE PROGRAMS. tr aini ng.ua.edu. Management Certificate Programs To Improve Your Competitive Edge! MANAGEMENT CERTIFICATE PROGRAMS r aini ng.ua.edu o u ch i n g l i v e s Bryan Conference Cener Tuscaloosa, Alabama Managemen Cerificae Programs To Improve Your Compeiive Edge! Inroducory-Advanced Managemen

More information

The Application of Multi Shifts and Break Windows in Employees Scheduling

The Application of Multi Shifts and Break Windows in Employees Scheduling The Applicaion of Muli Shifs and Brea Windows in Employees Scheduling Evy Herowai Indusrial Engineering Deparmen, Universiy of Surabaya, Indonesia Absrac. One mehod for increasing company s performance

More information

Why Do Real and Nominal. Inventory-Sales Ratios Have Different Trends?

Why Do Real and Nominal. Inventory-Sales Ratios Have Different Trends? Why Do Real and Nominal Invenory-Sales Raios Have Differen Trends? By Valerie A. Ramey Professor of Economics Deparmen of Economics Universiy of California, San Diego and Research Associae Naional Bureau

More information

Can Individual Investors Use Technical Trading Rules to Beat the Asian Markets?

Can Individual Investors Use Technical Trading Rules to Beat the Asian Markets? Can Individual Invesors Use Technical Trading Rules o Bea he Asian Markes? INTRODUCTION In radiional ess of he weak-form of he Efficien Markes Hypohesis, price reurn differences are found o be insufficien

More information

Prostate Cancer. Options for Localised Cancer

Prostate Cancer. Options for Localised Cancer Prosae Cancer Opions for Localised Cancer You or someone you know is considering reamen opions for localised prosae cancer. his leafle is designed o give you a shor overview of he opions available. For

More information

DOES TRADING VOLUME INFLUENCE GARCH EFFECTS? SOME EVIDENCE FROM THE GREEK MARKET WITH SPECIAL REFERENCE TO BANKING SECTOR

DOES TRADING VOLUME INFLUENCE GARCH EFFECTS? SOME EVIDENCE FROM THE GREEK MARKET WITH SPECIAL REFERENCE TO BANKING SECTOR Invesmen Managemen and Financial Innovaions, Volume 4, Issue 3, 7 33 DOES TRADING VOLUME INFLUENCE GARCH EFFECTS? SOME EVIDENCE FROM THE GREEK MARKET WITH SPECIAL REFERENCE TO BANKING SECTOR Ahanasios

More information

CVA calculation for CDS on super senior ABS CDO

CVA calculation for CDS on super senior ABS CDO MPRA Munich Personal RePEc Archive CVA calculaion for CDS on super senior AS CDO Hui Li Augus 28 Online a hp://mpra.ub.uni-muenchen.de/17945/ MPRA Paper No. 17945, posed 19. Ocober 29 13:33 UC CVA calculaion

More information

Impact of Debt on Primary Deficit and GSDP Gap in Odisha: Empirical Evidences

Impact of Debt on Primary Deficit and GSDP Gap in Odisha: Empirical Evidences S.R. No. 002 10/2015/CEFT Impac of Deb on Primary Defici and GSDP Gap in Odisha: Empirical Evidences 1. Inroducion The excessive pressure of public expendiure over is revenue receip is financed hrough

More information

TSG-RAN Working Group 1 (Radio Layer 1) meeting #3 Nynashamn, Sweden 22 nd 26 th March 1999

TSG-RAN Working Group 1 (Radio Layer 1) meeting #3 Nynashamn, Sweden 22 nd 26 th March 1999 TSG-RAN Working Group 1 (Radio Layer 1) meeing #3 Nynashamn, Sweden 22 nd 26 h March 1999 RAN TSGW1#3(99)196 Agenda Iem: 9.1 Source: Tile: Documen for: Moorola Macro-diversiy for he PRACH Discussion/Decision

More information

Establishing Prefabricated Wood I-Joist Composite EI

Establishing Prefabricated Wood I-Joist Composite EI July 008 Esablishing Prefabricaed Wood I- Composie EI INTRODUCTION Composie (glued/nailed) floors are common in boh residenial and commercial consrucion, and have been successfully designed by Prefabricaed

More information

The Relationship between Moral Intelligence with Organizational Development

The Relationship between Moral Intelligence with Organizational Development Inernaional Academic Insiue for Science and Technology Inernaional Academic Journal of Organizaional Behavior and Human esource Managemen Vol. 3, No. 6, 2016, pp. 31-38. ISSN 2454-2210 Inernaional Academic

More information

Factors Affecting Initial Enrollment Intensity: Part-Time versus Full-Time Enrollment

Factors Affecting Initial Enrollment Intensity: Part-Time versus Full-Time Enrollment acors Affecing Iniial Enrollmen Inensiy: ar-time versus ull-time Enrollmen By Leslie S. Sraon Associae rofessor Dennis M. O Toole Associae rofessor James N. Wezel rofessor Deparmen of Economics Virginia

More information

CAPt. Print e-procurement: Changing the Face of the Printing Industry CAP VENTURES. Market Forecast for Web-Based Print e-procurement

CAPt. Print e-procurement: Changing the Face of the Printing Industry CAP VENTURES. Market Forecast for Web-Based Print e-procurement Prin e-procuremen: Changing he Face of he Prining Indusry Marke Forecas for Web-Based Prin e-procuremen Buyers Crieria for Web Procuremen Opporuniies and Advanages for Prin Service Providers Profiles of

More information

Georgia State University CIS 8000 IT Project Management. Upon completion of the course, students should be able to:

Georgia State University CIS 8000 IT Project Management. Upon completion of the course, students should be able to: Georgia Sae Universiy CIS 8000 IT Projec Course Descripion This course examines he defining characerisics of IT projecs, especially involving he developmen of sofware inensive sysems, and inroduces he

More information

A Brief Introduction to the Consumption Based Asset Pricing Model (CCAPM)

A Brief Introduction to the Consumption Based Asset Pricing Model (CCAPM) A Brief Inroducion o he Consumpion Based Asse Pricing Model (CCAPM We have seen ha CAPM idenifies he risk of any securiy as he covariance beween he securiy's rae of reurn and he rae of reurn on he marke

More information

Why Did the Demand for Cash Decrease Recently in Korea?

Why Did the Demand for Cash Decrease Recently in Korea? Why Did he Demand for Cash Decrease Recenly in Korea? Byoung Hark Yoo Bank of Korea 26. 5 Absrac We explores why cash demand have decreased recenly in Korea. The raio of cash o consumpion fell o 4.7% in

More information

Molding. Injection. Design. GE Plastics. GE Engineering Thermoplastics DESIGN GUIDE

Molding. Injection. Design. GE Plastics. GE Engineering Thermoplastics DESIGN GUIDE apple GE Plasics GE Engineering Thermoplasics DESIGN GUIDE Wall Thickness Paring Lines Ejecion Appearance Pars Ribs/Gusses Bosses Holes Depressions Radii, Filles and Corners Molding Design Injecion s for

More information

AP Calculus BC 2010 Scoring Guidelines

AP Calculus BC 2010 Scoring Guidelines AP Calculus BC Scoring Guidelines The College Board The College Board is a no-for-profi membership associaion whose mission is o connec sudens o college success and opporuniy. Founded in, he College Board

More information

State Machines: Brief Introduction to Sequencers Prof. Andrew J. Mason, Michigan State University

State Machines: Brief Introduction to Sequencers Prof. Andrew J. Mason, Michigan State University Inroducion ae Machines: Brief Inroducion o equencers Prof. Andrew J. Mason, Michigan ae Universiy A sae machine models behavior defined by a finie number of saes (unique configuraions), ransiions beween

More information

CLASSIFICATION OF REINSURANCE IN LIFE INSURANCE

CLASSIFICATION OF REINSURANCE IN LIFE INSURANCE CLASSIFICATION OF REINSURANCE IN LIFE INSURANCE Kaarína Sakálová 1. Classificaions of reinsurance There are many differen ways in which reinsurance may be classified or disinguished. We will discuss briefly

More information

SPEC model selection algorithm for ARCH models: an options pricing evaluation framework

SPEC model selection algorithm for ARCH models: an options pricing evaluation framework Applied Financial Economics Leers, 2008, 4, 419 423 SEC model selecion algorihm for ARCH models: an opions pricing evaluaion framework Savros Degiannakis a, * and Evdokia Xekalaki a,b a Deparmen of Saisics,

More information

YTM is positively related to default risk. YTM is positively related to liquidity risk. YTM is negatively related to special tax treatment.

YTM is positively related to default risk. YTM is positively related to liquidity risk. YTM is negatively related to special tax treatment. . Two quesions for oday. A. Why do bonds wih he same ime o mauriy have differen YTM s? B. Why do bonds wih differen imes o mauriy have differen YTM s? 2. To answer he firs quesion les look a he risk srucure

More information

Chabot College Physics Lab RC Circuits Scott Hildreth

Chabot College Physics Lab RC Circuits Scott Hildreth Chabo College Physics Lab Circuis Sco Hildreh Goals: Coninue o advance your undersanding of circuis, measuring resisances, currens, and volages across muliple componens. Exend your skills in making breadboard

More information

Journal Of Business & Economics Research September 2005 Volume 3, Number 9

Journal Of Business & Economics Research September 2005 Volume 3, Number 9 Opion Pricing And Mone Carlo Simulaions George M. Jabbour, (Email: jabbour@gwu.edu), George Washingon Universiy Yi-Kang Liu, (yikang@gwu.edu), George Washingon Universiy ABSTRACT The advanage of Mone Carlo

More information

THE LAW SOCIETY OF THE AUSTRALIAN CAPITAL TERRITORY

THE LAW SOCIETY OF THE AUSTRALIAN CAPITAL TERRITORY Complee he form in BLOCK LETTERS Provide deails on separae shees if required To Responden Address THE LAW SOCIETY OF THE AUSTRALIAN CAPITAL TERRITORY Personal Injury Claim ificaion pursuan o he Civil Law

More information

Small and Large Trades Around Earnings Announcements: Does Trading Behavior Explain Post-Earnings-Announcement Drift?

Small and Large Trades Around Earnings Announcements: Does Trading Behavior Explain Post-Earnings-Announcement Drift? Small and Large Trades Around Earnings Announcemens: Does Trading Behavior Explain Pos-Earnings-Announcemen Drif? Devin Shanhikumar * Firs Draf: Ocober, 2002 This Version: Augus 19, 2004 Absrac This paper

More information

The Impact of Promotional Mix Elements on Consumers Purchasing Decisions

The Impact of Promotional Mix Elements on Consumers Purchasing Decisions Inernaional usiness and Managemen Vol. 8, No. 2, 2014, pp. 143-151 DOI:10.3968/4800 ISSN 1923-841X [Prin] ISSN 1923-8428 [Online] www.cscanada.ne www.cscanada.org The Impac of Promoional Mix Elemens on

More information

LEASING VERSUSBUYING

LEASING VERSUSBUYING LEASNG VERSUSBUYNG Conribued by James D. Blum and LeRoy D. Brooks Assisan Professors of Business Adminisraion Deparmen of Business Adminisraion Universiy of Delaware Newark, Delaware The auhors discuss

More information

EDUCATION POLICIES AND STRATEGIES

EDUCATION POLICIES AND STRATEGIES EDUCATION POLICIES AND STRATEGIES Naional Educaion Secor Developmen Plan: A resul-based planning handbook 13 Educaion Policies and Sraegies 13 Educaion Policies and Sraegies 13 Naional Educaion Secor Developmen

More information

Principal components of stock market dynamics. Methodology and applications in brief (to be updated ) Andrei Bouzaev, bouzaev@ya.

Principal components of stock market dynamics. Methodology and applications in brief (to be updated ) Andrei Bouzaev, bouzaev@ya. Principal componens of sock marke dynamics Mehodology and applicaions in brief o be updaed Andrei Bouzaev, bouzaev@ya.ru Why principal componens are needed Objecives undersand he evidence of more han one

More information

policies are investigated through the entire product life cycle of a remanufacturable product. Benefiting from the MDP analysis, the optimal or

policies are investigated through the entire product life cycle of a remanufacturable product. Benefiting from the MDP analysis, the optimal or ABSTRACT AHISKA, SEMRA SEBNEM. Invenory Opimizaion in a One Produc Recoverable Manufacuring Sysem. (Under he direcion of Dr. Russell E. King and Dr. Thom J. Hodgson.) Environmenal regulaions or he necessiy

More information

ANALYSIS AND COMPARISONS OF SOME SOLUTION CONCEPTS FOR STOCHASTIC PROGRAMMING PROBLEMS

ANALYSIS AND COMPARISONS OF SOME SOLUTION CONCEPTS FOR STOCHASTIC PROGRAMMING PROBLEMS ANALYSIS AND COMPARISONS OF SOME SOLUTION CONCEPTS FOR STOCHASTIC PROGRAMMING PROBLEMS R. Caballero, E. Cerdá, M. M. Muñoz and L. Rey () Deparmen of Applied Economics (Mahemaics), Universiy of Málaga,

More information

A Re-examination of the Joint Mortality Functions

A Re-examination of the Joint Mortality Functions Norh merican cuarial Journal Volume 6, Number 1, p.166-170 (2002) Re-eaminaion of he Join Morali Funcions bsrac. Heekung Youn, rkad Shemakin, Edwin Herman Universi of S. Thomas, Sain Paul, MN, US Morali

More information

LINKING STRATEGIC OBJECTIVES TO OPERATIONS: TOWARDS A MORE EFFECTIVE SUPPLY CHAIN DECISION MAKING. Changrui Ren Jin Dong Hongwei Ding Wei Wang

LINKING STRATEGIC OBJECTIVES TO OPERATIONS: TOWARDS A MORE EFFECTIVE SUPPLY CHAIN DECISION MAKING. Changrui Ren Jin Dong Hongwei Ding Wei Wang Proceedings of he 2006 Winer Simulaion Conference L. F. Perrone, F. P. Wieland, J. Liu, B. G. Lawson, D. M. Nicol, and R. M. Fujimoo, eds. LINKING STRATEGIC OBJECTIVES TO OPERATIONS: TOWARDS A MORE EFFECTIVE

More information

Implementing 130/30 Equity Strategies: Diversification Among Quantitative Managers

Implementing 130/30 Equity Strategies: Diversification Among Quantitative Managers Implemening 130/30 Equiy Sraegies: Diversificaion Among Quaniaive Managers Absrac The high degree of correlaion among he reurns of quaniaive equiy sraegies during July and Augus 2007 has been exensively

More information

Vector Autoregressions (VARs): Operational Perspectives

Vector Autoregressions (VARs): Operational Perspectives Vecor Auoregressions (VARs): Operaional Perspecives Primary Source: Sock, James H., and Mark W. Wason, Vecor Auoregressions, Journal of Economic Perspecives, Vol. 15 No. 4 (Fall 2001), 101-115. Macroeconomericians

More information

Contrarian insider trading and earnings management around seasoned equity offerings; SEOs

Contrarian insider trading and earnings management around seasoned equity offerings; SEOs Journal of Finance and Accounancy Conrarian insider rading and earnings managemen around seasoned equiy offerings; SEOs ABSTRACT Lorea Baryeh Towson Universiy This sudy aemps o resolve he differences in

More information

THE FIRM'S INVESTMENT DECISION UNDER CERTAINTY: CAPITAL BUDGETING AND RANKING OF NEW INVESTMENT PROJECTS

THE FIRM'S INVESTMENT DECISION UNDER CERTAINTY: CAPITAL BUDGETING AND RANKING OF NEW INVESTMENT PROJECTS VII. THE FIRM'S INVESTMENT DECISION UNDER CERTAINTY: CAPITAL BUDGETING AND RANKING OF NEW INVESTMENT PROJECTS The mos imporan decisions for a firm's managemen are is invesmen decisions. While i is surely

More information