!!!! Global&Mapping&of& Health&Policy&and& Systems&Research& Training&

Global&Mapping&of& Health&Policy&and& Systems&Research& Training& Tara&Tancred& Meike&Schleiff&& David&H.&Peters& Dina&Balabanova& & & February&2015& 1

TheTeachingandLearningHealthPolicyandSystemsResearchThematicWorkingGroup, HealthSystemsGlobal Leadauthors:TaraTancred,MeikeSchleiff,DavidH.Peters,andDinaBalabanova Othercontributors:TaghreedAdam,PreaGulati,andMaryamBigdeli 2

Acknowledgements& TheanalysispresentedinthisreportwascarriedoutbyHealthSystemsGlobal'sThematic WorkingGrouponTeachingandLearningHealthPolicyandSystemsResearch(ofwhichthe authorsaremembers)withfinancialsupportfromtheallianceforhealthpolicyand SystemsResearch. Wewouldliketothankallrespondentswhotooktimetocontributeandenrichthereport. WearealsogratefultothemembersoftheThematicWorkingGrouponTeachingand LearningHealthPolicyandSystemsResearchwhoprovidedinspirationandhelpedto extendthereachofthestudy.thereportalsobenefitedfromadviceandreviewbymany HealthSystemsGlobalmembers,towhomweareindebted. 3

Contents& Tables&and&figures&...&5& Executive&summary&...&6& Introduction&...&9& Study&aim&and&objectives&...&11& Study&methods&...&12& Results&...&16& Respondentoverview...16 InstitutionalRlevelresults...17 Competencies...19 CourseRlevelresults...20 Geographicaldistributionofcourses...20 Formatofcoursedelivery...21 Coursetypeandtiming...22 Languageofinstruction...23 Teachingformats...24 Disciplines...25 Researchmethodstaught...30 Targetaudiences...33 Evaluationofstudents...35 PostRtrainingfollowRupofstudents...36 ChallengesinteachingHPSR...37 OpportunitiesforexpandingcapacityinHPSR...38 Conclusions&...&40& ThegapsinHPSRtraining...40 Studylimitations...42 Thewayforward...43 References&...&44& Appendices&...&45& InRdepthinterviewguide...45 OnlineSurvey...49 4

Tables&and&figures& Table1NumberofHPSRcoursesperinstitution,byinstitutiontype...18 Table2ExistenceofinstitutionalHPSRcompetencies,byinstitutiontype...19 Table3Formatofcoursedeliverybytypeofinstitution...21 Table4Coursedurationbytypeofinstitution...22 Table5CountandpercentageofdisciplinarybackgroundsfromwhichHPSRcoursesare offered,bycountryincome...26 Figure1Overviewofonlinesurveyrespondents...16 Figure2Mapoflocationsofrespondentinstitutions...17 Figure3Institutiontypebycountryincome...18 Figure4MapoflocationsofHPSRcourses(subsetofFigure2)...20 Figure5PercentageofHPSRtrainingcourses,byregion...20 Figure6Percentageofcoursesofferedaspartofasemester/termorasashortcourse,by countryincome...23 Figure7LanguageofHPSRcourseinstruction...23 Figure8Mostcommonlyreported(amongLMICrespondents)teachingformatsinHPSR courses...24 Figure9MostcommonlyreporteddisciplinaryorientationofHPSRcourses...26 Figure10MostcommonlyreportedcontentinHPSRcourses...28 Figure11MostcommonlyreportedlearningobjectivesofHPSRcourses...29 Figure12Percentageofrespondentsreportinganyquantitativeorqualitativemethods taughtintheirhpsrcourse...31 Figure13Mostcommonlyreported(amongLMICrespondents)quantitativemethodstaught inhpsrcourses...32 Figure14Mostcommonlyreported(amongLMICrespondents)qualitativeandmixed methodstaughtinhpsrcourses...33 Figure15Mostcommonlyreported(amongLMICrespondents)studentaudiencesofHPSR courses...34 Figure16Mostcommonlyreported(amongLMICrespondents)evaluationmethodsusedin HPSRcourses...35 Figure17MostcommonlyreportedmethodsoffollowRupforalumni...36 5

Executive&summary& Manyinternationalinstitutionsareincreasinglyrecognizingtheimportanceofdeveloping healthpolicyandsystemsresearch(hpsr)capacitytosupporthealthsystems.thealliance forhealthpolicyandsystemsresearch(ahpsr)hasprioritizedsupportfortraininginhpsr toexpandcapacity,andiscurrentlydevelopingaprogrammeofworktoprogresstowards thisgoal.tosupportthisprocess,ahpsrcommissionedaglobalmappingstudyofcurrent teachingandtrainingprogrammesfocusedonhpsrrelevanttolowrandmiddlerincome countries(lmics).itintendedtoassesstheirreachanddiversityintermsofcontentand modalities,identifyingmajorgapsandopportunitiestoexpandhpsrteachingcapacity.the studywasconductedthroughthehealthsystemsglobalthematicworkinggroupon TeachingandLearningHealthPolicyandSystemsResearch. Thestudyemployedanonlinesurveyandsubsequentlysoughttointerpretitskeyfindings throughinrdepthkeyinformantinterviews.inthesurvey,191respondentsprovided informationabouthpsrcourses,representing169differentorganizationsfrom59countries. Themajorityofrespondentswerefromacademicorganizations(60%ofrespondentsfrom LMICsand79%ofrespondentsfromhighRincomecountries(HICs)werefromacademic institutions).fromthese,112respondents someofwhomwereinvolvedinmorethanone course reportedon152hpsrcourses. Institutional+levelresponses FromrespondentswhoseinstitutionsofferedcoursesrelevanttoHPSR,59%(110)reported thattherewereinstitutionalrlevelcompetenciesrelatedtohpsr.manyofthesecourses wereofferedaspartofmasterofpublichealthprogrammes. Course+levelresponses Geographical,distribution,, HPSRcourseswerereportedlytaughtin42countries,representingeveryWHOregion(see Figure4):19%ofcourseswerebasedinAfrica,24%intheAmericas,1%intheEastern Mediterranean,31%inEurope,16%inSoutheastAsia,and9%intheWesternPacific. Format,of,course,delivery, Courseswereprimarilyofferedonsite(80%),withonly12%ofcoursesbeingofferedonline; 8%ofcourseswereofferedbothonsiteandonline. Course,type,and,timing, ThemajorityofHPSRcourses(66%)wereofferedduringanacademictermaspartofa largerdegreeprogramme.however,thisfindingwasmorecommoninhics,where79%of courseswereterm/semesterrlongcomparedtoonly54%inlmics,wheremorecourses wereofferedasshortcourses(22%)thaninhics(10%). Language,of,instruction, CoursesweretaughtprimarilyinEnglish(76%),thoughsomeweretaughtintwolanguages withenglishastheprimarylanguage(8%),and16%wereofferedonlyinotherlanguages., 6

7 Teaching,format,used, Themajorityofcoursesusedtraditionalteachingformatsinacademicsettings.Classroom lecturespredominated,followedbyindependentreading,casestudies,smallgroup seminars,andgroupprojects.insightsfromkeyinformantinterviewsaddedfurther explanationofthesefindings,suggestingthat,despiteafocusontraditionalteaching methods,theseofteninvolvedrealrlifeexamples,andanefforttodemonstratethepractical applicationofthematerialtaughtasmuchaspossible. Disciplines,from,which,HPSR,courses,are,offered, Trainingwasofferedfromawiderangeofdisciplines,mostnotablypublichealth,health policy,andglobal/internationalhealth. Course,content, HPSRcoursecontentwasverydiverse.Manyrespondentsdescribedcoursecontentthat focusedonunderstandingormanaginghealthsystems,andnotnecessarilyonresearch relatedtohealthpolicyandhealthsystems.however,themostcommonlyreportedcourse contentincluded:healthsystemsresearchmethods,healthsystemsmanagement,policy analysis,researchprojectsinhpsr,andanoverviewofhealthsystems. Learning,objectives,, SeventyRsixpercentofrespondentsprovidedlearningobjectivesforthehealthpolicyand systemsresearchcoursestheyarepartof.theabilityto:framehealthsystemsresearch questions,developaresearchproposal,conductpolicyanalysis,usedata/evidence,and identifymethodstoanswerresearchquestionswerethetopfivemostcommonlycited objectives. Research,methods,taught, Themajorityofbothqualitativeandquantitativeresearchmethodstaughtwereeither genericresearchmethods,ormethodsusuallytaughtwithinspecificdisciplines(e.g. epidemiologyorethnography)ratherthandistincthpsrmethods.eightyrthreepercentof coursestaughtatleastonequantitativeresearchmethodandatleastonequalitative researchmethod.inlmics,respondentsreportedquantitativeresearchmethodsbeing taughtin75%ofthecourses,whichissimilartothatofhics(69%). Target,audiences, TheprimaryaudienceofmostHPSRcourseswasmasters students,followedbydoctoral students,healthorganizationmanagers,policyrmakers,andpractisingresearchers.key informantsfeltthathealthministrystaff,publichealthprofessionals,andpractising physiciansareaudiencesthatshouldbetargetedthroughtraininginhpsr. Evaluation,of,students, Presentations,exams,researchprojects,reports,andessayswerethemostcommonlyused meansofevaluatingstudents.keyinformantselaboratedonthesefindings,suggestingthat manyinstructorsaimtoassesswhetheractualcapacitytoapplyknowledge ratherthan simplymemorizeinformation hadbeenachieved.,,

FollowCup,and,mentorship,of,students, Only40%ofrespondentsreportedfollowingRupwithstudentsaftertheyhadcompleted theirstudiesinacourserelevanttohpsr.alumnisurveysandnetworkswereusedmost frequentlytofollowrupwithformerstudents.keyinformantshighlightedthatthisareais felttobeofparticularimportanceandisonethatmanyinstructorswouldliketodevelop. Challenges,in,teaching,HPSR, KeyinformantinterviewswereusedtoexplorechallengesinteachingHPSR.Twomain challengeswereidentified:thefirstisthelackofasupportiveinstitutionalenvironmentto offertraining,whichincludesaninsufficientnumberofinstructorsandpotentialphd supervisors.thesecondisthesensethathpsrlackslegitimacybeyonditsowncircles, resultinginlittledemandforit. Opportunities,for,expanding,capacity,in,HPSR, KeyinformantsalsohighlightedstrategiestoexpandcapacityinHPSR,whichincluded developingfurtherresearchnetworksthatlinkinstitutions,bothtodoresearchand exchangeideasandmaterialsaroundteachingpractices.additionally,creatingdemandfor HPSRwasseenascriticalforexpandingcapacity,bothamongconsumersofHPSR(e.g. policyrmakersandresearchfunders)andwithinacademicinstitutions. Conclusion:thegapsinHPSRcourses, Competencies:thereisalackofanagreeduponscopeofHPSRcompetenciesthat trainingshouldaimtobuild. Geographicalandlanguagegaps:SomeregionsareparticularlyunderRrepresented regardinghpsrtraining,mostnotablytheeasternmediterraneanaswellaslatin America,CentralandEasternEurope,andtheformerSovietUnion.Thereisalsoaclear gapintermsoftheexistenceofnonrenglishlanguagehpsrcourses. Institutionalgaps:Manyinstitutionslackasufficientnumberofinstructorswiththe capacitytotrainhsprandsupervisedoctoralstudents.collaborationacrossdisciplines wasrecognizedasbeingveryvaluable,butchallengingtoachieve.finally,thelackof demandforhpsrwasreiterated. Coursecontent:traininginthemethodologicalbreadththatHPSRnecessitatesis challengingandfurtherincreasestheneedformultidisciplinarycollaboration;academic andothertraininginstitutionsmaylacktheincentivestoseeksuchpartnerships. Targetaudiences:Thereisaneedtotargetfrontlineworkersandministrystaffwhomay notbeabletoundertakeafullmaster sdegree.differentmeansofprovidinghpsr trainingforhealthprofessionalswithfullrtimejobsneedtobeconsidered. The,way,forward, ThisreportsuggeststhatdefiningindividualHPSRcompetenciesandorganizational capacitiesisanimportantstepinbuildingthefieldoftrainingforhpsr.furthermore, trainingcoursesandresourcesneedtobedevelopedinlanguagesotherthanenglish. AnupdatedlistofHPSRcoursesgloballyandarepositoryofteachingmaterialsshouldbe developedsuchthatinstructorsofhpsrcandrawontheseresources.finally,networksof exchange,crossrlearning,andmutualsupportamonginstructorsofhpsrshouldbe promotedtofacilitateengagementaroundbestteachingpractices. 8

Introduction& & Healthsystemsplayacriticalroleinthehealthandwellbeingofindividuals.Healthpolicy andsystemsresearch(hpsr)isconsideredessentialtohelpbuildcapacitytoassesshealth systemfunctioningand,ultimately,toimprovehealthoutcomesandaccesstohealthcare. 1 TheWorld,Health,Report,2013emphasizedthecriticalroleofhealthsystemsresearchfor guidingthemovetowardsuniversalhealthcoverage. 2 ThefieldofHPSRisoftendistinguishedbytheresearchquestionsthatareasked,regardless ofthedisciplineandthespecificthematicareaoffocusthatresearchersmightcomefrom. HPSRteachingmaybefromtheperspectiveofdisciplinessuchasepidemiology,policy analysis,oranthropology,amongothers.however,theresearchquestionsdictatethestudy designandmethodologicalandanalyticalapproachesused.thesequestionsareoften concernednotjustwith what istheeffectofcertainpoliciesandinterventionsbutalso why and how thesehavebeenimplemented,andmaybeassociatedwithparticular outcomes. 3 Thesequestionsmayencompassissuesrelevanttohealthservices,health promotion,andthepoliciesandpoliticsofhealthsystems strengthening. 4 Finally,HPSRis oftenintendedtorespondtoimmediatepolicyandpracticeconcernsandgeneratelessons thatcanbeapplied. ThereisaperceptionthattherearesignificantcapacitygapsinHPSRteachinggloballyand thatavailablecoursesarenotadequatelypreparingstudentstoconducthpsr.moreover, thereislimitedsupportfordevelopingcoursesandbuildingteachingcapacities,andvery littlesharingofexistingexperiencetocontributetoenhancedlearning.atthesametime,a tremendousamountofuntappedexpertiseandinnovationhasemergedinrecentyears, especiallyinlowrandmiddlerincomecountries(lmics),whichmaybebeneficialacross othersettings,includinginhighrincomecountries(hics).althougheffortshavebeenmade todevelopnewhpsrcoursesandtrainingprogrammesworldwide,muchofthisworkis occurringinisolation. Manyinternationalinstitutionsareincreasinglyemphasizingtheimportanceofdeveloping HPSRcoursestosupporthealthsystems.TheAllianceforHealthPolicyandSystems Research(AHPSR)recognizesthatthesupportoftrainingaroundHPSRiscriticalfor expandingcapacitytoconducthpsrgloballyandiscurrentlyseekingtodevelopa programmeofworkinthisarea.thenewlyestablishedinternationalsocietyforhealth SystemsResearch(HealthSystemsGlobal),anditsThematicWorkingGrouponTeaching andlearninghasreceivedconsiderablefeedbackfromitsmembersandothersonthe urgentneedforsupportofstrategiestoimprovehpsrtraininginvariouscontextsandfor varioususers. WhilethereisasharedunderstandingthataccesstoHPSRtrainingandteachingresources shouldbeexpandedandreciprocallearningpromoted,currentlythereisapaucityof evidenceastowherethegapsandopportunitiesinhpsrteachingare.inresponsetothis lackofevidence,theahpsrcommissionedastudythatwasconductedbythethematic WorkingGrouponTeachingandLearningofHealthSystemsGlobal,exploringthelandscape 9

ofhpsrtrainingandseekingtoidentifyconstraintsandopportunitiesaroundwhichaction canbetaken. ThefindingsofthisstudywillfeeddirectlyintotheworkoftheAHPSRanditsestablishment ofaprogrammededicatedtohpsrtraining.thethematicworkinggroupwillalsobenefit, gaininginsightsthatwillenhanceitshpsrteachingstrategiesandinformactivitiesthatare beneficialtohealthsystemsglobalmembers,andtomanyotheractorscommittedto healthsystemstrengthening. 10

Study&aim&and&objectives Theoverallgoalofthestudywastoconductaglobalmappingofteachingandtraining programmesfocusedonhpsrrelevanttolmics. Thisstudyhadthreemainobjectives: 1. Toconductaglobalmappingstudyofteachingandtrainingcurriculaandsyllabi developedbyacademic,educational,and/orresearchinstitutionsforhpsrrelevant tolmics,identifyingthevarioustargetaudiences,content,andmodalities. 2. Toidentifythemajorgapsintrainingandtoinvestigatethereasonsforthesegaps. 3. Toformulatepossiblestrategiesforaddressingthesegaps. Informationfromthisstudyprovidedabasisofdiscussionduringanexpertconsultation assembledbytheahpsrinnovember2014inordertoidentifyhowtheahpsrcan contributetoaddressingandmovingbeyondthegapsidentifiedhere. 11

Study&methods& Definitions InrecentyearstherehavebeeneffortstobuildthefieldofHPSRanditsmethods,anda widerangeofdefinitionshavebeendebated.thisstudyadoptedadefinitionofhpsr proposedwithintheahspr,drawingonasynthesisofthecurrentliterature.hpsris definedasresearchthat: [seeks]tounderstandandimprovehowsocietiesorganizethemselvesinachieving collectivehealthgoals,andhowdifferentactorsinteractinthepolicyandimplementation processestocontributetopolicyoutcomes [tocreate]acomprehensivepictureofhow healthsystemsrespondandadapttohealthpolicies,andhowhealthpoliciescanshape andbeshapedby healthsystemsandthebroaderdeterminantsofhealth. 4 Thestudyemployedmixedmethods;datawerecollectedthroughanonlinequantitative surveyandsemirstructuredinterviewswithkeyinformants. Sampling Respondentstotheonlinequantitativesurveywerecontactedthroughavarietyofmethods, seekingtoimprovethereachofthestudy.first,thesurveywasdistributedthroughseveral keymailinglistsaccompaniedbyinformationaboutthesurveyandtheintendeduseof findings.apreliminaryassessmentofthelistswasconductedtoidentifythosewithan explicitlystatedobjectivetopromoteresearchand/orbuildcapacityinhpsr.these included: emaillistofcurrentmembersofhealthsystemsglobal; membersofrelevantthematicworkinggroupsofhealthsystemsglobal; membersoftheconsortiumforhealthpolicyandsystemsanalysisinafrica (CHEPSAA) emaillistsof: o AHPSR; o AfroRNets; o HealthSpaceAsia; o HealthSystemsResearchIndiaInitiative; o EuropeanObservatoryonHealthSystemsandPolicies. Allrespondents(includingthosewhodidnotcompletethesurveyastheydidnothave coursestoreportonorwhoseinstitutionsdidnotofferhpsrtraining)wereaskedto identifyoneormorefollowrupcontactstowhomtheyfeltthesurveywouldberelevant.as such,snowballsamplingwasusedtoidentifyover100additionalrespondentsworldwide, andtheseindividualswerethencontacteddirectlytocompletethesurvey. Additionally,thestudyinvestigatorsidentifiedotherrespondentsthroughtheirpreRexisting contactsandnetworksofindividualsinvolvedinhpsrtraining,aswellasthroughtheir respectiveinstitutionsandinstitutionalnetworks.thisadditionalsamplingwasdoneto ensurethatkeyinstitutionsinvolvedinhpsrwerenotmissedduetotheirlackofinitial responsetorecruitmentemails,forexample. 12

13 Finally,amanualonlinesearchofpotentialcoursesgloballywasconducted.Wesearched Googleforcoursesinhealthsystemsorhealthpolicyusingavarietyofsearchterms [[ healthpolicy OR healthsystems ORHSRORHPROR healthpolicyandplanning ]AND [courseormoduleorworkshoporseminarorclassorlectureorshortcourse]and research]]andreviewedcoursesoffered,especiallyatschoolsofpublichealth,acrosseach region.potentialrespondentsfromthesesearcheswerecontacteddirectlyandaskedto completethesurvey. Allthesamplingstepsdescribedabovewereconductedinparallel.Afteridentifyingthat responseratewaslowinparticularregions,toenhanceresponserateandensurethatas representativeasampleaspossiblehadbeenobtained,over120followrupemailsand reminderstopotentialsurveyrespondentsweresentout.followrupemailsweresentuntil wecollecteddatafromrespondentsfromallregionsandkeyinstitutionsknownbythe studyinvestigatorstoofferhpsrcourses. Survey Tools,and,key,subject,areas, TheonlinesurveywasadministeredinEnglishusingtheSurveyMonkeyplatform,between JulyandSeptember2014.Thesurveywasdividedintotwoparts,thefirstobtaining informationonfeaturesoftheinstitutionsofferingteachinginhpsr,andthesecond exploringinmoredetailindividualhpsrcoursesthatrespondentsweredirectlyinvolvedin. Intheinstitutionalsection,participantswereaskedtoidentifygenerallyhowmanyHPSR courseswereofferedattheirinstitution.hpsrwasdescribedas: ResearchthatasksquestionsthatrelatetorealRworldsituationsandissuesthatfocuson healthservicesandpromotinghealth.itsupportsappliedresearchthatexplicitlyseeksto influencepolicyandhasthepotentialtoleadtohealthsystemdevelopment.healthsystems researchisseenasparticularlywellrsuitedtoanswerwhyorhowacertainmixofstructures, policies,orinterventionsmayhaveshapedoutcomes.itcandrawononeormoredisciplines andmethodstoanswerthesequestions. Issuesstudiedincludehowhealthcareisfinanced,organized,delivered,andused;how healthpoliciesareprioritized,developed,andimplemented;andwhysomehealthsystems achievetheirgoalsandothersdonot.itenablesacomprehensiveanalysisofhowhealth systemsrespondandadapttohealthpolicies,andhowhealthpoliciescanshape andbe shapedby healthsystemsandbroaderdeterminantsofhealth.bothunintended consequencesandcomplexityinachievingintendedeffectsareoftenexamined. ThesubsequentquestionsfocusedonthecompetenciesthatHPSRRrelatedprogrammes withintheirinstitutionaimedtobuildintrainingparticipants,thetargetaudiencesforhpsr training,andifandhowtrainingparticipantsarefollowedrupaftercompletinghpsrtraining. Thesurveysoughttodistinguishbetweencompetenciesandlearningobjectivesby providingdefinitions: Acompetencyistheabilitytoapplyasetofrelatedknowledge,skills,andabilities neededtosuccessfullyperformimportantworkfunctionsortasks.aneducational competencyisusuallydefinedatthelevelofadegreeorprogramme.incontrast, learning&objectivesaremorespecificanddescribewhatthelearnershouldbeable

14 toachieveattheendofacourse.learningobjectivessaywhatwewantthelearners toknowandcompetenciesindicatehowwecanbecertaintheyknowit. Asasimplifiedexample,acompetencywithinpublichealthprogrammesofferedby aninstitutionmightbeforstudentsto demonstrateculturalawarenessand sensitivity.alearningobjectiveofacoursewithinapublichealthprogrammewith thatcompetencymightbeto identifytherangeofsocialandenvironmental determinantsofhealthamongindigenouspopulationsinnorthernalberta. Thesecondpartofthequestionnairefocusedonspecificcourses.Asthecourseswerethe unitofanalysis,weidentifiedinclusioncriteriaforthecourseswewishedtogainmore informationon.weincludedcoursesorprogrammesfromanytypeofinstitution(e.g. academic,government,ornonrgovernmentalorganizations(ngos))thatmetthefollowing criteria: HPSRcourses,seminars,practicums,andotherformallyrecognizededucational formatsthatincludehpsrandrelatedterms(i.e.healthsystemsresearch,heath servicesresearch,healthpolicyresearch,implementationresearch,operational research;orevaluationofthesamehealthpolicyandsystemsterms)inthetitle, objectives,ordescriptions; or coursesorprogrammeswithanexplicitcomponentonhpsr(participantsselfr identifiedthispointafterreadingthedefinitionofhpsrprovided),butthatmay fallwithinaneducationalprogrammeonhealthservicesresearchmethods, biostatistics,epidemiology,orqualitativeresearchmethods; and coursesininstitutionsincountriesofallincomelevels,whichhaveexplicitly statedrelevancetohpsrinlmics. Toensurethatthesurveyresponseswereobtainedforcoursesthatmettheinclusion criteria,participantswereaskedthreescreeningquestionsbeforeproceedingtothecourser specificpartofthesurvey.thesewerewhetherthecourse(s):1.fittedthespecified definitionofhpsr;2.taughtresearchmethodologies;and3.hadrelevancetolmiccontexts. Ifparticipantsanswered no toanyofthosequestions,theywerenotincluded.further screeningtookplaceasresponseswerereviewedandsurveydatawerecleaned,again ensuringthatonlycoursesthatmatchedtheinclusioncriteriawereretained. In+depthinterviewswithkeyinformants Oftheparticipantswhocompletedthesurveyandindicatedwillingnesstobecontactedfor followrup,wepurposivelysampledkeyinformants.weaimedtoidentifyparticipants representingabroadrangeofregions,institutiontypes(e.g.academicinstitutions, governmentministries,ngos),hpsrcoursecontent,androleswithincourses.wealso includedindividualswhowerefrequentlysuggestedbyotherparticipants,throughsnowball sampling,andwhohadindicatedbeinginvolvedintwoormorecoursesrelevanttohpsr. WeaimedtointerviewidentifiedHPSR champions ineachregion,whowefeltwouldbe abletoshareperspectivesthatwererepresentativeoftheirrespectiveregions.

Analysis ThequantitativeonlinesurveydatawerecleanedusingExcel.Crosstabulationsofthekey variablesbywhogeographicregion,typeofinstitution,andincomelevel(hicandlmic) wereconductedinspss,withstatisticalsignificanceassessedbychirsquaretestsandfisher exacttestsforsmallsamples,withabenchmarkofp<0.05.openrendedresponseswere summarizedintablesandmajorthemeswereidentified.quantitativedatawere summarizedthroughpiecharts,bargraphs,orpresentedastablesasappropriate. TranscriptsofinRdepthinterviewswereanalysedthematicallyusingNVivo10software. TranscriptswerereRreadseveraltimes.Afterfamiliarizationwithalltranscripts,anoverall codingframewasdeveloped.codeswereaddedasthescriptswerereviewedlinerbyrline. Relationshipsweredrawnbetweencodestogeneratethemesfromthedata.Furtherdata wascollectedtoexploredivergentcasesoremergingthemesmorefully,andinterviews werecontinueduntiltheoreticalsaturationhadbeenreached.keythemesthatemerged fromthedatawerethenillustratedinthefindingssectionthroughrepresentative quotations. Ethicalconsiderations TheEthicsCommitteeoftheLondonSchoolofHygieneandTropicalMedicineassessedthe protocolanddeemedit nothumansubjectsresearch andexemptfromfullethicsreview. Participantsweretoldthattheirconsentintheonlinesurveywasimplicitthroughtheir participationinthesurvey. TheEthicsCommitteereviewedandapprovedtheprotocolforthequalitativepartofthis study(reference#8485).informedconsentwassoughtfromallparticipants.alltranscripts wereanonymizedbyttandidentifyinginformationwasremoved;transcriptsweretreated asconfidential. 15

Results& Respondentoverview Overall,306respondentscompletedtheonlinesurvey,ofwhom191wentontoprovide informationabouttheirrespectiveinstitutionsofferingcoursesrelevanttohpsr.ofthese respondents,140metourinclusioncriteria.afterremovingincompleteentries,112 respondentswentontoprovideinformationabout152specifichpsrcoursestheywere directlyinvolvedwith(figure1). 306 respondents overall Less 48 incomplete responses 258 respondents with complete entries Less 67 with no institutional HPSR courses 191 respondents with institutional information Less 18 respondents not personally involved in a HPSR course 173 respondents Less 17 respondents whose course had no methodologies component 156 respondents Less Less 16 respondents whose course was not relevant to LMICs 140 respondents 28 incomplete course responses 112 respondents providing detail about 152 courses Figure&1&Overviewofonlinesurveyrespondents& TwentyRsevenindividualsweretheninterviewedaskeyinformants.Therespondentswere from23countriesindifferentgeographicalregions(theamericas:theunitedstates,cuba, Chile,andTrinidadandTobago;Africa:Ethiopia,Kenya,Rwanda,SouthAfrica,Uganda;the EasternMediterranean:Turkey,Lebanon,Iran;Europe:Switzerland,Denmark,Sweden,the UnitedKingdom,Italy,Germany,andIreland;SoutheastAsia:BangladeshandIndia;andthe WesternPacific:ChinaandAustralia). 16

Institutional+levelresults Inthesurvey,respondentswereaskedtofirstanswerquestionsabouttheirinstitutionsand thetypesofhpsrrrelevantcoursesthatareofferedthere.the191respondentswho providedinstitutionalrlevelquestionsrepresented169differentorganizations(16 institutionshavingmorethanonerespondent)from59countries.themapinfigure2 indicatesthelocationsofinstitutionsforwhichdatawereobtained. Figure&2&Mapoflocationsofrespondentinstitutions(coloursindicateWHOregions:pink=the Americas,green=Africa,yellow=Europe,blue=EasternMediterranean,purple=SoutheastAsia, orange=westernpacific)& ThemapinFigure2showsthatsomeregionsappearedtohaveconsiderablegapsinHPSR training,havingeitherfewornoinstitutionsofferinghpsrrrelevantcourses.themost notablegapswereseeninthemiddleeastandnorthernafrica(encompassedbythe EasternMediterraneanRegion),theWesternPacific,andpartsoftheAmericas,namely LatinAmerica.MuchofEasternEuropeandtheformerSovietUnionalsostoodoutas almostentirelylackinginstitutionsofferingsuchcourses. Mostrespondents(122)werebasedatuniversitiesandotheracademicinstitutions(71%), buttherewerealsorespondentsfromnonracademicorganizations,includingwho(40, 23%),governmentministries(7,4%),andassociations(4,2%).Associations groupsor networksofinstitutions(e.g.chepsaa,whichiscomprisedofmultipleinstitutions) were createdasacategoryastherewereindicationsthattheircharacteristicsandhpsrtraining capacitiesmaybeslightlydifferentfromthoseoftraininginthecontextofasingle institution. 17

ThereweredifferencesbetweenHICsandLMICsintermsofwhattypeofinstitutionswere representedinthissurvey,althoughtheabsolutenumbersofinstitutiontypesbyincome levelwererelativelysmall.hpsrtraininginlmicstendedtobehostedlessoftenin academicinstitutions(49,61%)comparedtohics,wherealargemajorityofrelevant trainingtookplaceinsuchsettings(73,80%).lmicshadahigherpercentageofnonr academicandgovernmentalorganizations(26,32%)offeringhpsrcourses. 2% 3% 15% 32% 80% 2% 5% 61% & Figure&3&Institutiontypebycountryincome& Overall,themajorityofrespondents(152,79%)reportedthattheirinstitutionofferedmore thanonecoursethatwasrelevanttohpsr.almosthalf(79,46%)oftherespondents indicatedthattheirinstitutionofferedfourormorecoursesrelevanttohpsr(seetable1). & Table&1NumberofHPSRcoursesperinstitution,byinstitutiontype & Institution&type& & Total&#(%)& Number& of&hpsr& courses& HICs Academic& #(%)& Association& #(%)& Government& agency&#(%)& LMICs NonVacademic& (including& WHO)&#(%)& 1& 23(19%) 1(25%) 1(14%) 11(28%) 36(21%) 2& 16(13%) 2(50%) 1(14%) 7(18%) 26(15%) 3& 12(9%) 0( s%) 0(0%) 5(13%) 15(9%) 4&or&more& 63(54%) 0(0%) 2(29%) 12(30%) 79(46%) Don t& know& 5(4%) 1(25%) 3(43%) 5(13%) 14(8%) Total& 121(100%) 4(100%) 7(100%) 40(100%) 172(100%) FourtyRonesurveyrespondentsrepresentedinstitutionsthatwerenotcurrentlyoffering coursesrelevanttohpsr,butindicatedthattheywouldbeinterestedinofferingsuch coursesiftheopportunityarose.someofthereasonsforthisinterestthatwerereported included: aneedtoaddresstoday scommonandcomplexpublichealthproblems; enhancingtheevidencebaseforhealthpoliciesandprogrammes; helpingtobuildacommonlanguageforhealthsystems actorstoavoid misunderstandingandpotentialconflict; 18

helpingtoenhanceandsustaintheprofessionaldevelopmentofhealthsystems actors;and preparingstudentsandyoungprofessionalsforthekindsofworkandchallenges theyarelikelytoface. Competencies Thesurveydefinedcompetenciesforrespondents.Thehypothesisforinvestigating competencieswasthatmanyinstitutionswillseecompetencyrbasededucationasan importantwaytotrainpeopleinhpsr.despiteprovidingdefinitionsandexamplesto respondentsonthesurvey,confusionseemedtohaveremainedaboutwhatcompetencies areandhowtheydifferfromlearningobjectives.thirteenpercentofrespondents answeringinstitutionalinformationprovidedonlyagenericinstitutionalwebsitethatdid notindicateaclearsetofcompetenciesbutrather,verygeneralprogrammeorcourse information.thesewebsiteswerenavigatedtothegreatestextentpossibletoseeif competencieswereindicatedanywhere,whichtheywerenot.additionally,11%ofcourse respondentsprovidedthesameinformationforbothcompetenciesataninstitutionallevel andlearningobjectivesatacourselevel. OftherespondentsfrominstitutionsofferingcoursesrelevanttoHPSR,110(59%)from92 differentinstitutionsreportedhavingprogrammerspecificcompetencies. Table&2ExistenceofinstitutionalHPSRcompetencies,byinstitutiontype & Institution&type& & Academic& #(%)& Association&#(%)& Government& agency&#(%)& NonVacademic& (including&who)& #(%)& Total&#(%)& No& 10&(8%)& 1&(25%)& 1(14%)& 10(25%)& 22(13%) Competencies& Yes& 97(81%) 2(50%) 6(86%) 26(65%) 131(76%) Don t&know& 13(11%) 1(25%) 0(0%) 4(10%) 18(11%) Total& 120(100%) 4(100%) 7(100%) 40(100%) 171(100%) Surveyrespondentswereaskedtoindicatecompetenciesthatwereidentifiedbytheir institutionsasbeingkeytobuildamongtheirstudentsreceivingtraininginhpsr.awide rangeofinformationwasprovidedrelatedtocompetenciescurrentlyestablishedwithin organizationsandprogrammesofferinghpsrcourses.however,asindicatedabove,after analysingtheseresponses,itwasprobablethattheywerenotwellunderstood,asmany responseswouldnotbeconsideredcompetencies,suchas togainageneralbackgroundin publichealth, togainpracticalexperience,or tounderstandpolicy.thefollowingwere themostcommonlystatedcompetencyareas: - Toapplyleadershipskills - Todesignhealthfinancingarrangementsandinvestigatehealthfinancingschemes - TocriticallyevaluateHPSRliterature - Touseparticipatoryresearchapproaches Duringkeyinformantinterviews,manyofthesecompetencyareaswerereiterated, althoughtheroleofapplyingcriticalthinkingwasemphasizedtoagreaterextentthanwhat wasseeninthesurveyresults.pleaseseepages28and29formoredetailsaboutcourser specificlearningobjectives. 19

Course+levelresults Geographical,distribution,of,courses, Outofatotalof191participantswhoindicatedthatHPSRcourseswereofferedattheir institutions,112reportedthattheywereinvolvedwithandprovidedinformationonatleast onecourse.ofthose,40wereinvolvedinmorethanonecourse therefore,these112 respondentsprovidedinformationabout152courses.thefollowinganalysesaretaken fromthissubsetofrespondents.responsesonhpsrcourseswerereceivedfrom42 countries,representingeverywhoregion(seefigure4). & Figure&4MapoflocationsofHPSRcourses(subsetofFigure2) ThemajorityofHPSRcourseswereofferedbyEuropeaninstitutions(47,31%),with institutionsfromtheeasternmediterraneanregionofferingthefewestcourses(2,1%), illustratedinfigure5. WesternPacific 9% SoutheastAsia 16% Africa 19% Europe 31% Figure&5PercentageofHPSRtrainingcourses,byregion Americas 24% Eastern Mediterranean 1% 20

Withineachregion,capacitywasoftenconcentratedinoneortwocountries.Thecountries fromwhichthehighestnumberofcourseswerereportedincludedindia(19,13%),the UnitedStates(21,14%),theUnitedKingdom(11,7%),andSouthAfrica(7,5%).Similarly, theinstitutionsfromwhichthegreatestnumberofsurveyrespondentswereinvolvedin coursesincludedwho(6,2% mostfromwho sspecialprogrammeforresearchand TraininginTropicalDiseases),AzimPremjiUniversity(5,2%),KarolinskaInstitute(4,1%), andjohnshopkinsuniversity(3,1%),whileeightotherinstitutionshadtworespondents. Format,of,course,delivery, HPSRcourseswereofferedlargelyonsite(faceRtoRface),butsomehadonlinecomponents orwereofferedentirelyonline. Table&3Formatofcoursedeliverybytypeofinstitution & Institution&type& Total&#(%)& Academic& Association& Government& agency& NonVacademic& organization& Course& interface& Both& 8(8%) 1(20%) 0(0%) 1(5%) 10(8%) Onsite&& 82(82%) 2(40%) 4(80%) 19(82%) 107(80%) Online& 10(10%) 2(40%) 1(20%) 3(13%) 16(12%) Total& 100(100%) 5(100%) 5(100%) 26(100%) 133(100%) Theproportionofcoursesofferedonsite,online,orviabothmodalitiesbetweenHICand LMICsettingswasalmostidentical.Thisfindingsuggeststhatbandwidthorotherresource constraintsmaynotrepresentasignificantbarriertoonlineandinteractiveteaching modalitiestotheextentthatmighthavebeenexpected. Keyinformantswereaskedtoreflectontheirexperiencesusingonsiteversusonline teachingorviceversa,whereverapplicable.acommonlymentionedconcernrelatedto onlineteachingwasthelackofimmediateadaptabilityofthecoursedeliverybasedonhow studentsunderstandandengagewithcontent,whichisanassetofonsiteteaching. Additionally,whiletherewereindicationsofcontextRspecificproblems(e.g.studentsfacing difficultiesinhavingregularaccesstotheinternet),themajorconstrainttoofferingthese typesofcoursesrelatedtoalackoftimeandincentivesfortheinstructors.findingtrained professionalswiththeskillsandtimetofacilitatethesecoursesaswellassupportforthe designand/orupdateofinteractiveonlinematerialswasperceivedasamajorchallenge. Furthermore,itwasunderstoodthatonlinelearningrequiresadifferenttypeof commitmentfromthestudentsintermsoftimemanagementandselfrteaching,whichis sometimeschallenging. WhatInotedinmyexperienceisthereisgoingtobealotofhandRholding.Themaindifferenceis commitmenttodoingvirtuallearning,bothfromthetutorandthementee.becauseyouwillfind thatthesepeopleareverybusy.iftheygototheclassroom,thatistimeoutfromworkandthey havepermission,youknow,explicitpermissionfromtheirbossestoactuallygo Ittakesalotof commitmentandsothetendencyistodropit. (LMICrespondent,university) 21

However,itwasnotedthatonlineteachingdramaticallyincreasesaccessibilityofcourses andenablestargetingoflearnerpopulationswhomightotherwisenothavetheopportunity toreceivetraininginhpsr.furthermore,itwasperceivedthatonlinecourseswereagood platformforinstitutionaland/ormultidisciplinarycollaborationaroundstrengtheninghpsr training. Ithinkwhereonlinecoursesbecomeausefultooliswhenwehavepopulationsthatcan thave thatinrpersoncontactatall.andithinktherearehugepopulationsaroundtheworldthatdon t haveaccesstothattraining sointheabsenceofthat,onlinecoursesbecomeausefultool In HPSR,there sacomponentofthetrainingthatisjustintroducingvocabularyandconcepts. (LMICrespondent,NGO/university/MinistryofHealth),, Imean,oneoftheotheradvantagesofonlineprogrammesisthatthey rescalableandtheycan beusedbyothersinanothercontext.so,it sideallysuitedforpartnership. (HICrespondent,onlineteachinginstitution) Despitetheconstraintsofonlineteaching,manykeyinformantssawblendedlearning that is,coursesofferedwithbothonsiteandonlinetrainingcomponents asideal,capitalizing onthebenefitsofbothmodalities., Ithinkthequestion[is]howwediscoverabalancedapproachwhereyoudeliversomecontent onlineandatthesametimeyouallowforthatinteractive,participativeprocess,becausepeople learnfromeachotheraswell,particularlywhenyouhavestudentssittinginthesameroom. (LMICrespondent,university) Coursetypeandtiming HPSRcoursesvariedinduration.Themajorityweresemester/termRlength,oftenofferedas partofadegreeprogramme.shortcoursesweremorecommonlyofferedinlmics,most likelyreflectingthefindingthatfeweracademicinstitutionsofferedhpsrcoursesinthese settingscomparedtohics,asacademicinstitutionsaremorelikelytoembedsuchcourses withindegreeprogrammes(figure6). Table&4Coursedurationbytypeofinstitution & Institution&type& Total&#(%)& Course& timing& Academic& Association& Government& agency& NonVacademic& organization& InVservice& 2(2%) 0(0%) 0(0%) 3(13%) 5(4%) Independent/flexible& 5(5%) 1(14%) 2(40%) 1(4%) 9(6%) Other& 8(7%) 1(14%) 0(0%) 1(4%) 10(7%) Semester/term/degree& 73(70%) 4(57%) 2(40%) 14(60%) 93(66%) Short&courses& 17(16%) 1(14%) 1(20%) 4(17%) 23(16%) Total& 105(100%) 7(100%) 5(100%) 23(100%) 140(100%) 22

Percentage of courses 90 80 70 60 50 40 30 20 10 LMIC HIC 0 During semester/term Short courses Figure&6Percentageofcoursesofferedaspartofasemester/termorasashortcourse,bycountry income& Languageofinstruction CoursesweretaughtprimarilyinEnglish(76%ofallcourses),thoughsomeweretaughtin twolanguageswithenglishastheprimarylanguage,andsomewereofferedonlyinother languages(figure7).thesecondlanguagesforcoursestaughtintwolanguageswithenglish asprimaryincludedfrench(4),hindiorgujarati(2),thai(2),georgian(2),andodia(1). MostcoursesnotofferedinEnglishatallwereconductedinSpanish(8)orPortuguese(6). ThatHPSRcourseswereofferedsopredominantlyinEnglishhighlightsanimportantgapin teaching,butitcouldhavebeenanartefactgiventhatthesurveywasconductedinenglish anddistributedthroughenglishrspeakingnetworks. 8% 16% English Both 76% Other Figure&7LanguageofHPSRcourseinstruction ThepredominanceofEnglishwithinHPSRtraining,publication,andspheresofacademic exchangewasdiscussedasaconstrainttoteachingwithinthefield,particularlyamongkey informantsfromlmicsettingswhereenglishisnotwidelyspoken. One[barriertoHPSRteaching]isthelanguage,butthesecond[barrier]isaboutanacademic culture.wearenotsocollaborative. (HICrespondent,university) 23

Teachingformats TheHPSRteachingcoursescapturedbythestudyusedavarietyofteachingformats.The majorityreflectedtraditionalteachingmethodsestablishedinacademicsettings.thus, classroomlecturespredominated,followedbyindependentreading,casestudies,small groupseminars,andgroupprojects.awidevarietyofteachingmethodswereusedfor studentstoapplyknowledgetoparticularproblems,suchasresearchprojects(48%), problemsets(33%),andsimulations(notshown,16%). TherewasnodifferenceintheuseofthesemethodsinLMICorHICsettings,withthe exceptionof flippedclassrooms (inwhichstudentsaregivenmaterialsrelatedtocontent aheadoftimeandareexpectedtoselfrstudy oftenwithonlinematerialssuchasrecorded lectures andtimeintheclassroomisoftenspentengagingindiscussions,activities,and peerlearning)(lmics:13%;hics:32%;p=0.006).otherteachingformatsnotshownonthe graphbelowincluded:practicumsorinternships(lmics:15%;hics:12%);useofclosedr accessonlinematerials(lmics:12%;hics:20%);andopenraccessonlinematerials(lmics: 5%;HICs:12%). 90 Percentage of respondents reporting format used 80 70 60 50 40 30 20 10 LMIC HIC 0 Classroom lectures Case studies Independent reading Small group seminars Group projects Research projects Problem sets Figure&8Mostcommonlyreported(amongLMICrespondents)teachingformatsinHPSRcourses Thequalitativedataaddedimportantnuancestothesefindings.Whereasclassroom lecturesmaybeseeminglydidactic,theyoftenaimtooperationalizeandapplyabstract concepts.mostkeyinformantsemphasizedtheimportanceofbringingrealrlifeexamples intohpsrteaching.someexamplesofteachingformatsthatwereperceivedtobeofvalue bykeyinformantsincludedexposingstudentsto thefield,enablingthemtoidentifytheir ownquestionorproject,andencouragingthemtoengagewithpolicyactors often throughsimulationsorrolerplay helpingthemtoseehowhpsrmayinformpolicyand practice. Ithinkfieldwork,othergroupworkandthosethingsthatyoumentionedareusefulofcourse,but goingtothefieldandactuallyseeinghowahealthsystemworksismoreimportant.ordoing somerealhealthsystemswork,understandingthedrugmarket,howitisproducedand distributed thosesortsofthingsareveryuseful.orjustgoingtoahospitalorprimarycentre andseeinghowitactuallyfunctionsinreallifeandwhataretheconstraints. (LMICrespondent,university/NGO), 24

, KeyinformantsprovidedalotofilluminatingdetailabouthowtheyincorporaterealRlife scenariosintotheirteaching.rolerplayingwascitedregularly. We realsorunningahalfrdayrolerplayforthatcourseandit sbasedonaseriesofarticles publishedbywhoanditkindofrepeatsarealmeetingthattookplace,wherewebringtogether thetechnicalexperts thisisaboutsettingupaglobalactionplanforpneumoniacontrol.soit s bringingtogethertechnicalexpertsofdifferentkindsofinterventions,soit sdoctorsorandrsoon nutritionandthehealtheconomistwithdifferentperspectives.andatthesametimeitbrings togetherthefundersaswell:worldbank,billandmelindagatesareusuallypresentinourroler play notinreallifewho,unicef it sreallyinteresting.theygettoseewhattheycanfund andwhattheycannotfund,whattheycansupportandwhattheycannotsupport.andyouread itandseethestruggle,whichisverymuchlikereallife,betweenthescientistsontheonehand withtheirownvestedinterestsandthepolicyrmakersandthefundersontheotherhandwith theirspecificinterests. (HICrespondent,university) Furthermore,recognizingthatmanystudentswithinHPSRcoursesbringinaconsiderable amountofexpertise,peerrtorpeerlearning oftenthroughvariousformsofinteractive groupwork wasseenasbeingofgreatimportancewithinthefieldofhpsr.working collaborativelyacrossdisciplineswasalsoseenasanessentialprincipleunderlyingeffective HPSRtraining therefore,equippingstudentswiththeskillstodosowasseenasimportant, andgroupworkwasakeyfacilitatorofdevelopingthoseskills., Ifyoudohavepeoplewhobringrelevantexperienceandifyoudostructureyourteachingin waysthatdrawonthat,itmaynotbeasgoodastheperfectfieldstudies,butit salotbetter thanthetextbookrbasedteachinginthisarea (HICresearcher,university) Disciplines AkeyfindingisthatHPSRcoursesarethoughtofasmultidisciplinaryandoftenfallwithina broaderdegreeprogrammeorsetofcoursesthatteachcontentinformedbymany disciplines.respondentsweregiventheopportunitytoidentifymorethanonedisciplineor fieldofstudythatcontributestothecorecontentofthehpsrtrainingtheyareinvolvedin. Figure9indicatesthemostcommondisciplinesunderpinningtheHPSRcoursesnotedinthe survey.respondentsindicatedthatmanycourseswereofferedfromanhpsrdisciplinary perspective,however,thatwastypicallyinadditiontoselectingoneormorefieldofstudy likepublichealth,ormorespecificdisciplineslikeepidemiologyoranthropology, highlightingtheinterdisciplinarynatureofhpsr. 25

140 120 100 Count of courses 80 60 40 20 0 Figure&9MostcommonlyreporteddisciplinaryorientationofHPSRcourses & Ofnoteisthat,althoughadisciplinelikepublichealthisverybroadandencompassesother subrdisciplineslikeepidemiologyorbiostatistics,insomeinstancescourseswereclearly offeredfromoneofthelattermorespecificsubrdisciplines.assuch,wewereinterestedin seeingthesedisciplinesadequatelydrawnout.additionally,althoughhpsrisnot consideredbymanytobeastandalonedisciplineasofyet,participantsweregiventhe optiontoselectoneormoredisciplinarybackgroundsforthecoursestheyprovided responsesfor.manyparticipantsselectedmorethanonediscipline,includinghpsr, suggestingthatthis discipline,asitemerges,isreallyonethatismultirdisciplinaryin nature,asreflectedbythebreadthofdisciplinesfromwhichitistaught. Table&5CountandpercentageofdisciplinarybackgroundsfromwhichHPSRcoursesareoffered,by countryincome Country&income& Total& Discipline& HIC& LMIC& Anthropology& 12(19%) 10(12%) 22(15%) Biostatistics& 11(17%) 20(25%) 31(21%) Epidemiology& 19(27%) 29(36%) 48(33%) Global/international&health& 35(55%) 29(36%) 64(44%) Health&economics& 29(45%) 25(31%) 54(37%) HPSR& 62(98%) 71(91%) 133(94%) Political&science& 14(22%) 11(14%) 25(17%) Public&health& 39(61%) 59(73%) 98(68%) Sociology& 13(20%) 14(17%) 27(19%) Other& 9(14%) 7(8%) 16(11%) 26

27 Keyinformantinterviewstouchedontheissueofbringingmultidisciplinaryviewpointsinto theirteachingaroundhpsr.manyindicatedthatthiswasdifficultforanumberofreasons, includingthattheyalreadyfeltthattheyhadtoolittletimeforthehpsrrrelevantmaterials theyaimedtocoverwithoutbringinginmaterialsacrossdifferentdisciplines.alackof connectionswithintheirinstitutionsorwithpotentialcollaboratorsfromotherdisciplines wasanadditionalchallenge. Ihavetosaywe renotdoingmuchonthat,andwe renot thisisoneofthegapsmaybe we re nothelpingoutstudentsonhowtoframequestionstakingintoconsiderationdifferent disciplines. (LMICrespondent,university) Furthermore,thepressuretopublishwithinaparticulardisciplineandgenerallybeing conditionedtoworkingfromwithintheirownsilosofexpertisethroughtheirinstitutional environmentswerealsoidentifiedasbarrierstomultidisciplinaryworkandteachingthese differentviewpoints. Thereisaparticularapproachtomethodsandrigourthathasbeendevelopedoverdecadesin epidemiologythathasveryclearrules.soinawayyou reaskingpeopletoquestionthoserulesin somedomainsofresearch.theneveninapproachestoqualitativeresearch,it sbecomingvery rulerbound.what shappenedinpracticeinourcommunityhereisthatpeoplehavecomefrom differentbackgroundsandhavegravitatedintoafieldratherthanremainingboundbya discipline. (LMICrespondent,university) Moreacademicinstitutionschallenge[multidisciplinarywork]becausealltheacademiccareers arebasedondisciplinesandevenmorenowbecauseit sverymuchlinkedtopublicationsin certaindisciplinesandithinkthisisachallengingfactor. (HICrespondent,university/researchinstitution) Almostallkeyinformantsindicatedthattheywouldliketoincreasetheircapacitytowork acrossdisciplinesandtobringinmultidisciplinaryviewpoints,seeingtheenormousvalue theybringwithinhpsr. Coursecontent TherewasamarkeddiversityinthecontentofferedbyHPSRcourses,indicatingthatHPSRis anewlydevelopingfieldwithfluidboundaries.keyinformantsoftenraisedtheambiguityof whatwouldorwouldnotconstituteanhpsrcourse,suggestingthatthisreflectsalackofa unifiedsetoftheoriesorgroundingcoursematerials. Inthecaseofhealthpolicyandsystemsresearch,wedon thaveacommontheory.mainly becauseweareexplainingthetopicbasedonourownexperiences.so,ifyouneedtogotoa bookforhealthpolicyandsystemsresearch,whatwillyoufind?oneaboutengland,onebook aboutafrica,onebookabouttheoecdcountries,highrincomecountries youhavethat problem. (HICrespondent,university) Figure10highlightssomeofthemostcommonresponsesrelatedtocontentwithinHPSR courses.ascanbeseen,althoughmostisveryrelevanttohpsr,somecontenthasaless

directlink,despitethedefinitiongiveninthesurveytoolandtheinclusioncriteriadescribed inthemethodssection.forexample,epidemiologyandbiostatisticsskills,althoughuseful forhpsr,relatedmorebroadlytopublichealth. 40 Count of courses focusing on content 35 30 25 20 15 10 5 0 Figure&10MostcommonlyreportedcontentinHPSRcourses Whatconstitutes orwhatshouldconstitute HPSRtrainingwasexploredinthekey informantinterviews.keyinformantsreiteratedthebreadthofcoursecontent.many furtherpointedoutthattherelativeexpertiseateachinstitutionwouldvaryandthat teachingaroundhpsrwouldsimilarlyvaryaccordingtothebackgroundsandexperiencesof theinstructors.manyrespondentssuggestedthatabasicunderstandingofhealthsystems andsystemsthinking,oftenunderpinnedthroughtheuseofframeworksortheories,would beparticularlybeneficialforstudents. Gettingpeopletocometoacommongroundonaframeworkofconceptsanddefinitionsand approachestohealthsystemsseemstobearealprerequisite,becausethenyoucanstartusing thatasawaytoreachintothesystemasawaytotalkaboutit,understandit,appreciatethe complexityandtheinterrelationshipsofthecomponents. (HICrespondents,university) Inthisoverviewofhealthsystems,introducingstudentstothecomplexityofsuchsystems wasseentobeofcriticalimportance.furthermore,thecontextrspecificityofhealthsystems functioningwasperceivedtobenecessarytorelaytostudents. HealthpolicyresearchisacrossRcuttingdiscipline.Ittouchesonsocialissues,politicalissues,and developmentissues.soifatrainingprogrammeisdesigned,ithastoconsiderthesebroader issuessothatthetraineescanseehowhealthsystemsworkfromabroaderperspective. (LMICrespondent,university) 28

[Students]needtounderstand whatmakes[healthsystemsindifferentcountries]tick,the politics,thesocialsetting,theculturalbarriersandfacilitators. (HICrespondent,university) Fromthisbase,manykeyinformantsthensawanopportunitytointroduceothertopicsas required.however,asmanyhpsrcoursesareembeddedwithindegreeprogrammes,itwas acknowledgedthatdevelopingthefullspectrumofknowledgeandskillstodohpsrcomes notfromoneortwospecificcourses,butthroughoutafulldegreeprogramme. Wewouldfocusmuchlessonmethodsanddatacollectionthanonoveralldesignbecausewe assumethatpeoplewoulddoothercoursesonqualitativeandquantitativemethods Because wehaveseveraldifferentofferingsinanenvironmentwherepeoplehaveanumberofdifferent choices it sdoneelsewhere. (LMICrespondent,university) Learningobjectivesandcapacities, Surveyrespondentsprovideddataoncourselearningobjectivesorcapacitiesaswell.Ofthe respondentsdirectlyinvolvedwithatleastonecourse,80(76%)providedcourserspecific objectivesfor101courses. Figure11indicatesthetop10learningobjectivesthatrespondentsreportedtheircourses trytobuildamongparticipants.thethreemostcommonobjectiveswere:theabilityto frameahealthsystemsresearchquestion,theabilitytoconstructaresearchproposal,and theabilitytodopolicyanalysis,withtherestreceivingabroadlysimilarnumberof responses. Count of course capacities or learning objectives 40 35 30 25 20 15 10 5 0 Figure&11&MostcommonlyreportedlearningobjectivesofHPSRcourses 29

DuringinRdepthinterviews,keyinformantswerepromptedtothinkofwhattheircourses aimtoequipstudentstobeabletodoafterfinishingtheirstudies,thatis,thecapacities thatthesecoursesaimtodevelopinparticipants.verycommonly,theconceptsofcritical thinking abouthealthsystemsand/orpolicy,appropriateresearchquestions,andavailable literature alongsidetheabilitytoevaluateandconductresearchfromahealthsystems perspective,werementionedbyparticipantsasparticularlyimportantcapacities. Theideathereistoinstilinthemcriticalthinking.Criticallyassessingthecurrentstate,critically assessingtheevidenceinsupportofdifferentapproaches thenkindofbringingtogetherand criticallyanalysingthisevidencepuzzleatdifferentlevels,bothatthetechnicalinterventions levelandattheimplementationofpoliciesandprogrammeslevel. (HICrespondent,university) Themostimportantcapacitiesshouldbeacomprehensiveknowledgeandskillsconcerning healthsystemresearchprinciplesandplanningprocesses,todevelopthebasiccompetencies relatedtoproblemrsolving,dataanalysis,resultscommunicationanddecisionrmaking,which shouldbebuiltonsuchevidencerbasedresults. (LMICrespondent,university), Takentogether,criticalthinkingandsystemsthinkingwerebelievedtobeimportanttohelp studentstranscendtheirindividualspecializationsandgainskillsforeffectivehpsr. Inmostofthecourses,theyaremoresoftcourses;theyaremoreaimedattryingtotransfer someinformation,butdevelopingcapacitytothinkcriticallyaboutdifferenttopicsordifferent waysoflookingathealthsystems. (HICrespondent,university/researchinstitution) Ithinktheskillisreallycriticalthinkingabouttheirownspecializationifit snothealthsystems researchinordertotakeasystematicandsystemichealthsystemsapproachtowhateveritis theyaredoingsothateveniftheirresearchisverynarrow,theyunderstandthatit snot implementedinisolation. (HICrespondent,university), Researchmethodstaught SeventyRtwopercentofHPSRcoursestaughtatleastonequantitativemethod,compared to76%ofhpsrcoursesteachingatleastonequalitativemethod.eightyreightpercentof coursestaughtatleastonequantitativeresearchmethodandatleastonequalitative researchmethod.theteachingofthesemethodsdidnotvarywidelyacrossinstitutiontype. TherewerenosubstantiveorstatisticallysignificantdifferencesbetweenLMICandHMIC coursesintermsofprobabilityofteachingquantitativeorqualitativeresearchmethods. 30

Percentage of respondents indicating any method taught 100 90 80 70 60 50 40 30 20 10 0 Any quantitative methods taught Any qualitative methods taught LMICs & Figure&12Percentageofrespondentsreportinganyquantitativeorqualitativemethodstaughtin theirhpsrcourse Intheinterviews,manykeyinformantsindicatedthatanoverviewofbothquantitativeand qualitativeresearchmethodswouldbevaluablecoursecontenttoincludeinhpsrtraining. Therewereanumberofrespondentswhoindicatedadditionalresearchmethodsthat wouldbeusefultoteachinhpsrcourses,astheyhaveparticularrelevancetoanswering researchquestionsinhpsr. We vegotquitearesearchmethodsfocusaswellbecausewedoquitealotofparticipative researchhere.forexample,inthecoursei mworkingon,wehaveamoduleonsocialscience researchmethodsinhealth,andthat sreallyakindofparticipativefocus,sowelookat participatorymapping,matrixproviderpreferences,institutionalmapping,thingslikethat. (HICrespondent,developmentinstitution), Althoughsomerespondentsfeltthattrainingintheunderstandingandapplicationofboth qualitativeandquantitativeresearchmethodsisessentialwithinthesecourses,othersfelt thatthemostimportantcoursecontentwasinidentifyingappropriateresearchquestions. Mostrespondentsemphasizedthatitwasnecessarytoconveytostudentsthattheresearch methodsusedshouldreflectthetypesofresearchquestionasked. Theapproachtoresearchstrategiesandstudydesignwerefertoasflatratherthanhierarchical. Actually,asanappliedfield,we reinterestedinresearchthataddressesaparticularproblemand thatthemethodologyfollowsfromthat,sothemethodologycanbequalitativeorquantitative,it canbeactionresearch,itcanbeobservational,itcanbeexperimental actuallyfroma methodologicalpointofview,we refairlyagnosticisuppose (LMICrespondent,university) Manykeyinformantsfeltthatthesecourseswerestilldominatedbyepidemiologicaland/or biomedicalapproachesandwerethereforemorelikelytoemphasizequantitativeresearch methods.however,thesurveyfindingssuggestedthattherewasnodifferenceinthe frequencyofcoursesthatusedqualitative(76%)orquantitative(72%)methods. IngeneralIdothinkthattherearefewerqualitativemodulesormethodsthataretaughtthan quantitativeones.everywherei vebeenactually,weusequitealotofquantitativestuff: epidemiology,variousaspectsofepidemiology,randomizedcontrolledtrials,tostartwiththese methods.there sdefinitelyabiastowardsquantitativemethodsinteaching. (HICrespondent,university) HICs 31

Quantitative,methods, Percentage of responents reporting method taught in course 50 45 40 35 30 25 20 15 10 5 0 LMIC HIC Figure&13Mostcommonlyreported(amongLMICrespondents)quantitativemethodstaughtinHPSR courses AlthoughdifferentquantitativeresearchmethodsweretaughtinHPSRcourses,randomized controlledtrials,householdsurveys,healthfacilitysurveys,systematicreviews,cohort studies,andcasercontrolstudieswerethemostcommonlyindicatedmethods.throughkey informantinterviews,methodslikesocialnetworkanalysiscameacrossmorestronglyas beingintroducedinhpsrcoursesbecausetheywereparticularlyrelevanttothefield.few quantitativemethodswerementionedwithrespecttotheirspecificitytohpsr. InLMICs,householdandhealthfacilitysurveys(taughtby46%and43%ofLMIC respondentsrespectively)werethemostcommonlytaughtquantitativeresearchmethods, withsimilarpercentagesseeninhics(indicatedby39%and41%ofhicrespondents respectively).theteachingofcohortanalysis(36%),casercontrolstudies(36%),pragmatic trials(25%),andsystemsdynamicsmodelling(13%)weresimilarinhicsasinlmics(27%, 27%,14%,and5%respectively). 32

Qualitative,and,mixed,methods, Percentage of responents reporting method taught in course 60 50 40 30 20 10 0 LMICs HICs Figure&14Mostcommonlyreported(amongLMICrespondents)qualitativeandmixedmethods taughtinhpsrcourses Themostfrequentlyindicatedqualitativeresearchmethodstendedtoinvolvecommonly usedsocialsciencesqualitativeresearchmethods:qualitativecasestudies,focusgroup discussions,inrdepthinterviews,andkeyinformantinterviews.increasingly,bothcase studiesandparticipatoryactionresearchusingquantitativeandqualitativemethodsare usedinhpsrcourses mixedmethodsresearchappearstobeacommonfeatureofthese courses.inrdepthinterviews(43%)andkeyinformantinterviews(46%)weresimilarlytaught inlmicsandhics(38%,38%,and28%respectively),whereasthedifferenceforrealist reviewsnearlyreachedstatisticalsignificance(p=0.08)(19%inhicscomparedto10%in LMICs),aswellasforparticipantobservation(p=0.09)(37%inLMICscomparedto28%in HICs).Notably,participatoryactionresearchwasindicatedbyaconsiderablenumberofkey informantsasamethodthatwasofparticularvalueinhpsr.again,therewasnodifference betweenlmics(28%)andhics(25%)inhowfrequentlythismethodwastaught. Targetaudiences HPSRcoursesweremostcommonlyaimedatmastersstudents,thoughothertypesof learnerswerealsocommonlyreached.thesurveydididentifythefactthatmanyofthe studentshavemultipleroles forexample,manymastersstudentsalsohadongoingroles ashealthorganizationmanagersorpractitioners.however,asthesurveyrespondentswere instructors,theymaynothavehadafullgraspofthemultiplerolesplayedbytheirstudents otherthananecdotally,suggestingthattherewere differenthatsworn bymanyofthem. MastersstudentswerethemostcommonlytargetedstudentaudienceinbothHICandLMIC settings(70%and73%respectively).thisfindingissurprisinggiventhatinlmicsettings therewerefeweracademicinstitutionsrepresentedamongthoseofferinghpsrcourses andmoreshortcourses,whichwouldsupportthepropositionthatmorenonrtraditional students,suchaspolicyrmakersandhealthorganizationmanagers,mightberepresented amongthetargetedstudents. 33

ResponsesfurtherindicatedthatinHPSRcourses,thereweresimilarproportionsof doctoralstudents(48%),policyrmakers(45%),healthorganizationmanagers(50%),and practisingresearchers(37%)inhicsandlmics(37%,31%,37%,and28%respectively all comparisonsprvalue>0.2).figure15doesnotshowundergraduatestudentsandeducators, whowereindicatedastargetaudiencesbylessthan20%ofrespondentsinbothlmicand HICsettings. Percentage of rspondents reporting group as a primary audiece of their course 80 70 60 50 40 30 20 10 0 LMICs HICs Figure&15&Mostcommonlyreported(amongLMICrespondents)studentaudiencesofHPSRcourses Keyinformantsconfirmedsurveyfindingsandfurtheridentifiedstudentaudiencesthatare currentlynottargetedthroughhpsrcourses,includingpractisingphysiciansandpublic healthprofessionals,aswellasministryofhealthstaffwhoaredirectlyresponsiblefor implementingpolicy.itwasfeltthat,bybeingembeddedwithinhealthsystemsandwith thecapacitytoimpacthealthsystemdesignandpolicydevelopment,thesegroupsof individualswouldbeimportanttotarget. Ithinkyou reclearlymissingdoctorsandotherhealthprofessionals.medicaldoctorscarryoutan enormousamountofresearch theyseethepatients,theyarecollectinganenormousamount ofdata.theyareworkingintheirsilosandinisolationsotheyneedtounderstandhowhealth systemswork. (HICrespondent,university) IfwecangetmoreandmorepeoplewhoareinthepolicyRmakingprocesses,peoplewhoare reallypolicyimplementers,peoplefromcivilsocietygroups Ithinkcertainlywewouldmake muchmoreimpact. (LMICrespondent,university/NGO), AsHPSRcoursesaretypicallyofferedaspartofadegreeprogramme,someaudienceswho mayhavethemosttogainfromhpsrtrainingareprecluded,asitisunlikelythattheycould findtheopportunitytogetinvolvedinthesecoursesgiventheirworkingschedules. Peoplewhomaybecountrydeskofficersorpeoplewhoareheadquartersofficers.Theydon t havealotoftimetodoamasters,butifwecouldgetthemintoadiplomacourseorsomekindof shortcourse,wecouldhooktheminthatwayasonewayofgettingattheotherproblemiwas talkingabout:peopleareaskingformagicandaren treallyunderstandingwhattheyareasking for. (HICrespondent,university/researchinstitution) 34

Evaluationofstudents Mostrespondents(93%)reportedevaluatingtheirstudentstakingHPSRcourses. Presentations,exams,researchprojects,essays,andreportswereusedmostcommonlyto evaluatestudents.withtheexceptionofresearchprojects,evaluationmethodsthat requiredapplicationofknowledgeandskillsinrealrlifesettings(e.g.practicums inwhich studentsareexpectedtoengageinworkorresearchrelatedtohealthsystemsorhpsr, rolerplaying,writingpolicybriefs,conductingliteraturereviews,etc.)wereusedtoalimited extent.reportsweremorefrequentlyusedinlmicscomparedtohics(40%vs.22%, p=0.03)aswellasresearchprojects(44%vs.31%,p=0.04). Manykeyinformantsalsoreiteratedthatitiscrucialtoevaluatewhethercapacitieshad actuallybeenbuiltamongstudentstothegreatestextentpossible.assuch,whether throughmore traditional methodsofevaluationsuchaspresentationsandexams,or otherslikepracticalassignments(figure16),itwasfoundthatappliedknowledgeand criticalthinkingwereassessed. Percentage of respondents reporting ecvaluation type used 60 50 40 30 20 10 0 LMICS HICs & Figure&16Mostcommonlyreported(amongLMICrespondents)evaluationmethodsusedinHPSR courses, Thewaytheseassignmentsaregraded,theygetcontentsubstancepoints,buttheyalsoget pointsofanyindicationofcriticalthinkingandanalysisthatimentionedearlier,andevidence intopolicyandwhetherthey veimplementedthat. (HICrespondent,university) Moststudentassessmentisbasedonclassroomexamsandtermpapersalthoughwedohave a componentthatisprojectrbased.itisnotembeddedinanyofthecoursemodules.itisa separateproject,aguidedstudy,usuallythestudentswillhaveasupervisor.itcanbeas researchryasanyofthepublishablepapers,sobasedonsoliddataandstatisticalanalysis. (LMICrespondent,university) So,[students]haveactuallytodevelopinthecourseadevelopmentreportandtheyhave actuallytocomeandpresentitbacktothepolicyrmakersandalsotoapanelofresearchersand professorswheretheysitandlistentothem,andtheyhavetoactuallyshowthevalueandthe changethattheyhavebroughtasaresultoftheirwork. (LMICrespondent,university), 35

Post+trainingfollow+upofstudents Only40%oftherespondentsreportedfollowingRupwithstudentsaftertheyhadcompleted theirstudiesinanhpsrrrelevantcourse.twentypercentoftherespondentsdidnotknow whetheranyfollowrupoccurred,and40%reportednocurrentattemptstofollowrupwith students. ThereweremultipleplatformstofollowRupwiththesestudents,butfollowRupsurveys, alumninetworksandgroups,informalcontactwithalumni,conferences,andfollowrup emailswerecommonlyused.however,itisnotknownwhetherthesearespecifictothe courseorpartofbroaderinstitutionalengagementwithalumni. 30 Count of Courses Using Method 25 20 15 10 5 0 Follow-up Method Figure&17MostcommonlyreportedmethodsoffollowRupforalumni InRdepthinterviewshighlightedthatmanyinstructorsofHPSRwouldbeinterestedtodo morefollowrupofthestudentswhohavecompletedtheircourses,butthattheyare constrainedindoingso,oftenduetoinstitutionalrlevelbarriersoralackoftimeand/or resources. Nowwehavemassproduction[ofgraduates],wecannotfollowRupwhattheyaredoing.From themomenttheyleavetheuniversity,wedon tknowwheretheyare thisshouldbetakenasa specialareawherewecouldcreateanetwork Thisisveryimportant,becauseiftheygraduate andtheyareforgotten,thentheygobacktosquareone. (LMICrespondent,university) Agoodprocessheredoesbringrenewalwithinit,andIthinkwedon tdothatverywellandit s quitedifficulttodoitwhereyou redealingwithadispersedpopulation Particularlyinthisarea wheretheresearchmethodsareevolvingandchangingprettyfast,ithinkit simportantfor peopletohaveacontinuinginterest. (HICrespondent,university), 36

37 Althoughalumniassociationsandsurveyswerecommonlyreportedbysurveyrespondents, keyinformantshighlightedthevalueoflinkingalumniintospecificresearchnetworks,and furthertothat,tryingtofacilitateongoingacademicandprofessionaldevelopment opportunities. Wetrytomakesurethattherearealwaystrainingprogrammesthatpeoplecanapplytosothat theyalwayshavesomethingtoaspireto,andthenwehaveresearchgroupsthattheycanbea partof teamsofpeoplewithvaryingexperience. (LMICrespondent,university/NGO/MinistryofHealth), OneimportantcommentthatalsoaroseduringinRdepthinterviewswasthatreversefollowR up,inwhichthestudentscontacttheinstructorortheinstitutiontoseekoutfurther opportunitiesforcareerdevelopmentalsooccurs. Wehearbackfromthestudentsalotandalotofthestudentscomebackformore.Whenthey getstuckintheirproblems,theycomeback.oftenitcomesbackintheformofajointproject,so ourgraduatesactuallygetintoasituationwheretheycangetprojects.wehelpthemgetmoney andwekeepaworkingrelationship.itgoesoninsomecasesfordecades.wehelpthemin startingupcourses.sothereissomefollowrupbutithinkit smoreadrhoc. (HICrespondent,university/researchinstitution), ChallengesinteachingHPSR Keyinformantswereaskedtoidentifysomeofthemostsignificantchallengestheyfacein teachinghpsr.althoughmanyofthechallengesemergedthroughouttheinterviews,the majorityofkeyinformantsmentionedtwooverarchingchallenges.thefirstisthelackofa supportiveinstitutionalenvironmenttooffertraininginhpsr,whichincludesaninsufficient numberofpeopletrainedinhpsrtotakeonteachingandsupervisoryroles,especiallyfor studentsenteringhigherrlevelstudiessuchasphdsorpostdoctoralfellowships.asaresult, therearefewincentivestoteachhpsr. ThesecondisthesensethatHPSRlackslegitimacy,bothwithinacademicinstitutionsand beyond,resultinginlittledemandforhpsr.onthispoint,keyinformantsoftenrecognized thedifficultyinmainstreaminghpsrteachinginacademicinstitutions,andalsodifficultyin gettingresearchintopolicyduetoahistoricaltendencytonotalwaysincorporatean evidencebaseintopolicyrmaking.asanoffshootofthissecondchallenge,hpsrisunderr funded,whichalsocarriesintothefirstchallenge,makinghpsranunattractivefieldfor studentstopursuebecausetheyfeartherewillnotbegrantorjobopportunitieswithinit. It sparticularlydifficulttogetprogrammaticrtypefundingsothatyoucanreallybuildasetof people.becausetheotherrealchallengeinthispartoftheworldisthatwehaveourown challengesasaresearchinstitute,butintermsofsustainablecareers,healthsystemsresearchis notpartofthefabricofacademicinstitutionshere.souniversitiesreallydon thavethosekinds ofpeopleyet.andthey renotlikelytogetthemiwouldsayforquitesometimeunlessthereare profitablewaystorunmasterscourses. (LMICrespondent,researchinstitution)

Ihavedoctorsallthetime[who]sayafterourcourses, thatwasreally,reallyinterestingandwe mightevenwanttodoathesisonitoraprojectonitinthemedicalschool.thentheycome backandsay, wewereadvisedthattherearenosupervisors,weneedtofocusonour specializationandthatdoesn tincludehealthsystems. (HICrespondent,university) QuitealotneedstobedonetodeRlegitimizeonRtheRhoofpolicyRmakingasitveryoftenoccurs Alotoftheusefulsupportsystemsfordevelopingresearchcapacitiesarepeopleactuallythinking there ssomechancethattheirresearchwillmakeadifferenceorthattherearewaysof facilitatingresearchintopractice. (HICrespondent,university) OpportunitiesforexpandingcapacityinHPSR Keyinformantswerealsoaskedtosharetheirperspectivesonpossiblewaystoexpand capacityaroundhpsr,whetherthroughtrainingorothermeans.therewereanumberof suggestionsthatwerefrequentlymentioned: NetworkssuchasCHEPSAA,thatlinkinstitutionsandfacilitatecapacitydevelopmentin doinghpsr,butalsointeachinghpsr,shouldbeencouraged. [CHEPSAAhas]createdacertainenergyandinnovationandsetofmaterialsandbranding andsoon.ithinkweneedmoreofthatkindofthing.theconveningpowertobring togetherschoolswhoaredevelopingcurriculaandworkingoncurriculawithmentorship thatpeoplelike[chepsaacoordinators]provideisagreatmodel. (HICrespondent,university/researchinstitution) Ithinkwhatwouldbeveryimportantwouldbeaglobalcommunityofpractice.Peoplewho areinvolvedinhealthpolicyandsystemsresearchshouldhaveaplatform.therehastobea listservforthecommunityofpracticewhereyoucansharegloballyexperiencesand strategiesforhealthpolicyandsystemsresearch. (LMICrespondent,university) Becausewe verecognizedthatwehavealotofexpertiseinlearningandteaching particularlyaroundhealthpolicyandsystemsthatcouldbeusedtodevelopother institutionswhoaredeliveringthatkindoftrainingorteachinginothersettings,soithink thereistherecognitionthatit sthedirectionwewanttomoveinanditwillbeimportantto moveinto RegionalnetworksthathaveanorthRsouthdimension,weneedmore. (HICrespondent,university/researchinstitution), 38

CreatingdemandforHPSR,especiallyfrompolicyRmakers,butalsowithinacademic institutionsthemselves. Ithinkoneofthebitsthat smissingisrecognitionfromgovernmentthatthesekindsof areasareimportant.mostkindsofafricancountrieshavegotsomekindofhealthinstitute ormultipleresearchinstitutions,buttheyveryrarelyhaveanythingthatapproacheshpsr soithinkactuallythat sabigglobalissueandithinktheyneedtoworktoultimatelyengage withlocaluniversitieswhocanprovidethebedrockfordevelopingtheseareasand obviouslybeinvolvedintheresearch.theymayverywellbeininstitutionslikeourownas well,butintermsofsustainablecapacityrbuilding,it sgottobethroughlocalinstitutions. (LMICrespondent,researchinstitution) Ithinkweneedtogooutandmakearealcaseaboutwhatahealthsystemisandisnot,and alsointhereusesomegoodexamplesofwhatpeoplewhoareskilledinhealthsystemscan do Inasense,theyarestartingtomakeacleardistinctionbetweenwhat[a]healthsystem doesandwhathealthpolicydoesasopposedtootherdisciplinesinpublichealth. (LMICresearcher,university), CreatingamentorshipdatabasewherestudentsinLMICsmightbelinkedtomentors, especiallythosewhohavelongstandingexpertiseinhpsr. Engagingspecialiststobringintheirknowledge.Suggestionstobringinspecialist perspectivesincludedfieldvisits,havingshortvideospreparedaroundspecialisttopic areas,andgenerallyworkingacrossdisciplinesoreveninstitutionswhendeveloping curricula. Continuallyattemptingtoupdateresources,especiallytobringinrealRtime,realRworld workingexamplesforstudentstolearnfrom.innovatingwithinteaching,suchas increasinglyusingblendedlearningorflippedclassrooms,wasseentobeofvaluein trainingstudentsinappliedcontentsuchashpsr. 39

Conclusions& ThismappingstudywasconductedinordertoassessthereachanddiversityofHPSR coursesofferedgloballyandtoidentifymajorgapsandopportunitiestoexpandhpsr teachingcapacity.basedonourfindings,whichhavebeensummarizedabove,wehave notedthegapsinhpsrtrainingandpossiblewaysforwardintermsofexpandingcapacityin HPSR. ThegapsinHPSRtraining Geographical,and,language,gaps, Thestudydemonstratedtheexistenceofgapsintermsofgeographicalcoverageandthe languageofthecourses.theinstitutionalandcoursemapsidentifiedanumberof blank spots andsparselycoveredregions.someregionsareparticularlyunderrrepresented regardinghpsrtraining,mostnotablytheeasternmediterranean,whichincludesnorth AfricaandtheMiddleEast.CentralandEasternEuropeandtheformerSovietUnionalso hadveryfewcoursesoffered.latinamericaalsoappearedtohavefewercoursesoffered thanexpected.althougheffortshavebeenmadetoaugmentthesamplingintheseregions toincreaseresponsetothesurvey,thesefindingscouldhavebeenanartefactofthe samplingandrecruitmentstrategies. Additionally,thedominanceofEnglishastheprimarylanguageofinstructionmaypreclude teachingincountrieswhereenglishisnotthemajorspokenlanguageandenglishlanguage skillsarenotstrong.thisconsiderationisimportant,ashpsrisarapidlydevelopingfield, withnewtermsandconceptsemergingconstantly,andkeyliteraturethatmayinform coursecontenttendstobepublishedinenglishrspeakingjournals. InstitutionalClevel,gaps, ThisstudyhighlightedthecriticalroleofinstitutionalsupportindevelopingHPSRtraining. Thissupportisnotonlyfinancialbutalsorelatestoallocatingtimefordevelopingand runningtrainingandprovidingaccesstotechnicalexpertise,especiallyinthecaseofonline coursedevelopment.manykeyinformantsspokeoffeelingliketheyexistwithinasilo,not beingfullyintegratedintothesupportiveinfrastructureoftheinstitution.otherinstitutional challengesincludedobtainingfundingfordevelopmentofnewcoursesandgraduate studentsbeingconcernedthatstudyinghpsrwouldnotenhancetheirjobprospects. Havingfewindividualswiththecapacitytoprovidementorshipandpeersupportwithin HPSRwasindicatedbykeyinformantsasabarriertoadequatelysupportingPhDand postdoctoralstudentsinparticular.offeringmastersdegreesorphdsdedicatedtohpsr, havingmorehpsrcoursesavailable,andhavingmoreteachingstafftodeliverhpsrcourses weresuggestedasviablestrategiestoincreaseinstitutionalsupportofhpsr.therewasthe perceptionthatexistinginstitutionalstructuresandprocessesdidnotencourage multidisciplinarycollaborationwhentrainingstudents.furthermore,itwasfeltthathpsrr relatedcourseswereoftendeliveredwithlittleintegrationbetweendepartments. 40

41 AmorefundamentalissuethatwasnotedwasaroundtheperceptionofHPSRasanewfield thatlackslegitimacy.hpsrlackingrecognitionasadistinctthematicarearequiringskillsand competenciesbeyondtraditionalresearchmethodswasanadditionalissuethatwasraised bykeyinformants. Gaps,in,course,content, AwiderangeofmethodsweretaughtaspartofHPSRcourses,withqualitativeresearch methodsoftenconsideredacoreelementoftraininginparallelwithquantitativeandmixed researchmethods.hpsrcourseswerealsoofferedfromawiderangeofdisciplinary backgrounds.however,anumberofkeyinformantsnotedthatlackingthemethodological capacitytoanswerthehpsrquestionsthattheywouldliketointheirresearchwasa recurrentproblem.thisparticularissuealsostemsfromthenatureofhpsrcomparedto otherfields,relyingonresearcherstohaveaverybroadyethighlydevelopedskillset,which isoftenunrealistic.assuch,collaborationbetweenresearchersspecializinginparticular methodswithinhpsrtrainingwasseenasanimportantundertaking,butonethatmanyfelt wasachallengeinpractice. Ingeneral,HPSRtrainingthatfocusedonrealRworld,realRtimehealthsystemproblemswas seenasapriority,althoughwasnotalwayspossibletoestablish. Training,in,HPSRCspecific,methods, AlthoughthereisacommontendencytousemixedmethodswhentrainingHPSR,thestudy didnotidentifyresearchmethodologiesthatarespecifictohpsrtraining.rather,thestudy discoveredthatadiverserangeofmethodsweretaught,oftenfromdifferentdisciplines, whichcanbeappliedtohealthsystemsinlmics.teachingofhpsr,particularlyatmore advancedlevels,maydependonmethodologicallyrdrivencoursesthatarenotrestrictedto HPSRapplications,particularlyasmanyorganizationslackthecriticalmassofteachersand studentstoofferhpsrrspecificcourses.hpsrteachingprogrammesthenhavethechoiceof focusingonanyofthefollowing: 1. TeachingaspectsofresearchthatarespecifictoHPSR(e.g.definingHPSRresearch questions;identifyingtheconceptualframeworksandappropriateresearch methods;andworkingwithhpsrstakeholders). 2. TeachingselectedresearchmethodsthataremostcommonlyappliedinHPSR. 3. ConductingresearchonHPSRproblemsusingaselectedsetofresearchmethods. ThefindingsofthisstudyalsosuggestthatcoreHPSRcompetenciesarenotwelldeveloped. DefiningcorecompetenciesinHPSRisparticularlychallenginggiventhatprogrammeshave awidesetofaudiences,withgraduatesplayingrolessuchaspolicyrmakers,managers, researchers,andadvocates.giventhediversityofcompetenciesthatwereidentifiedinthis study,itwouldbevaluabletogeneratedialoguearoundhpsrcompetenciesbetween individualsengagedinhealthpolicyandhealthsystemsandtheresearcherswhoteach HPSRcourses.,,,

Gaps,in,course,modalities,and,audiences, Althoughtherewasvariationindurationandtiming,mostHPSRcourseswereofferedas partofadegreeprogrammewithinatermorsemester.onsiteteachingwasthe predominantmediumbywhichcourseswereoffered.embeddinghpsrcourseswithina master sdegree,withsignificantemphasisonteachingcontentdeliveredfacertorface, drawsinaspecificstudentaudiencewhoareeithernotworkingorareabletotakea considerableamountoftimeoutoftheirprofessionalworkinglifetoundertaketraining. ManymidRcareerorseniorprofessionals,especiallythoseworkingasfrontlinehealthcare providersoraspolicyrmakerswithinministriesofhealth,arenotlikelytohavethis opportunity. AlthoughthisstudyfoundthatcurrentHPSRcoursesaregenerallytargetedatamixof graduatestudents,researchers,andpractitionerssuchasorganizationmanagers, respondingtotheneedsofthesegroupstoincreaseuptakeofhpsrtrainingmayrequire moreflexibleteachingmodelstobeoffered.thesemodelsmightincludeaseriesofshort coursestailoredtospecificstudentaudiences,onrtherjobtraining,practicums,andblended onsiteandonlineteachingformats.furthermore,trainingneedstodrawmoreextensively ontheexperiencesofthestudentsthemselves;theincreasinglypopularflippedclassroomis anexampleofapromisingteachingformatthatmaybeparticularlyusefulindeveloping HPSRtrainingandfullyengagingparticipants. Studylimitations Severallimitationsofthestudyneedtobeacknowledged.Boththerecruitmentemailsused insamplingandthesurveytoolwereinenglish,whichmayhaveaffectedtheresponserate andbiasedtheresults.furthermore,despitemakingeffortstoensurewewerecapturing relevantcoursesacrossregions,minimalwebpresenceofcourseswithinregionslikethe MiddleEastandLatinAmericamayhaveledtofewerrecruitmentemailssenttorelevant individualsintheseareas,whichmayhaveledtotheirunderrrepresentation. AlthoughadefinitionofHPSRwasprovidedinthesurveytoindicatethetypeofcourses thatshouldbeincluded,respondentsprovidedanswersonsomecoursesthat,although relevantforhpsrinlmics,werenotnecessarilyappliedtohpsrinlmics.responsesfor coursesthatmadenoexplicitmentionofresearchmethodswerenotincludedinthe analysis.however,ifcoursesengagedwithresearchmethodsinsomeway,withoutthese beingacentralcomponentofthecourse,thoseresponseswereincluded.asaresult,we mayhavereceivedresponseswhereresearchandresearchmethodswerediscussedvery superficially,andagain,notappliedwithinthecontextofhpsr.owingtopractical constraints,itwasnotpossibletofollowrupwitheveryrespondenttoconfirmtheextentto whichresearchandresearchmethodswereincludedintheircourses,andtheextentto whichthesewerespecificallyappliedtohpsr. 42

Thewayforward Thisstudyhelpedtoidentifyanumberofsuggestionsformovingforwardindevelopingand supportinghpsrtrainingcapacitywithparticularrelevancetolmics.thesecanbegrouped inseveralcategories: Definingthefield:competencies ThisreportsuggeststhatdefiningindividualHPSRcompetenciesisanimportantstepin buildingthefieldofhpsrtrainingcapacityworldwide.thescopeandnatureofhpsr trainingandtheprerequisitesforhpsrcoursesneedtobedebatedwithinemerging communitiesofpractice,suchasthroughhealthsystemsglobalandtheahpsr,aswellas throughregionalnetworksandorganizations.thesediscussionsshouldinvolveinstitutions providinghpsrcoursesandotherswhoareseekingtodevelopthese. OperationalizinginstitutionalcompetenciesinHPSRisausefulexercisetobuildacommon platformforhpsrtrainingdevelopment.asthisreportsuggests,hpsrisseentoencompass adiversityofspecializedand soft skills.assuch,anabilitytoworkacrossdisciplinesand overcomeinstitutionalandprofessionaldivisionsandexpectationsiscentraltopromoting coherenthpsrtrainingprogrammes. ExpandedgeographicalandlanguagecoverageofHPSRcourses AclearrecommendationidentifiedfromthestudyistheneedtoexpandHPSRtrainingin particularregionsthatarelackingcapacity,especiallythemiddleeast,partsofafrica,latin America,andtheformerSovietUnion.Coursesandresourcesneedtobedevelopedin languagesotherthanenglish.todoso,dedicatedtrainingcapacitydevelopmentregionally andengagementwithdonors,governments,andnetworkswithaninterestinhpsrin underrrepresentedregionsisessential. Accesstoinformation RespondentsexpressedinterestinhavingaccesstoanupdatedlistofHPSRcourses,their objectives,andageneraloverviewoftheircontent.sucharesourcewouldassistindividuals developing,updating,oradaptingcoursesindrawinguponwhatisbeingdoneelsewhere. Anotherfrequentrequestwasfortheestablishmentofarepositoryofcoursematerialsthat areopenaccessorcanbeshared. KnowledgeRsharingandcrossRlearning TheabilitytointeractandlearnfromotherinstructorsofHPSRanddrawinglessonsfrom theirexperiencesiscriticalindevelopinghpsrtraining.therewassignificantinterestin engagingpractisingandprospectiveteachingstaffthrough webinars and/orother exchangesthatmightallowforthesharingofinnovativeandeffectiveteachingpractices. Giventheincreasingneedforflexibletraining,itisimportanttolearnaboutinnovationin teachingapproachesandtoidentifyfeaturesthataretransferrablebetweensettings. Encouragingnetworksandpartnershipsgloballymayhelptotranscendweaknessesin particularinstitutions.forexample,onlinetrainingplatformscandrawonthestrengthsof diverseinstitutionsandhelptoscaleupteachingmodels. 43

References& 1. EuropeanCommissionDirectorate,EconomicPolicyCommittee.Joint,report,on, health,systems.brussels,europeanunion,2010. 2. DyeC,BoermaT,EvansD,HarriesA,LienhardtC,McManusJ,et,al.The,world,health, report,,research,for,universal,health,coverage.geneva,worldhealthorganization,2013. 3. SheikhK,GilsonL,AgyepongIA,HansonK,SsengoobaF,BennettS.Buildingthefield ofhealthpolicyandsystemsresearch:framingthequestions.plos,medicine,2011, 8(8):e1001073.Epub23August. 4. AllianceforHealthPolicyandSystemsResearch,WorldHealthOrganization.Health, policy,and,systems,research:,a,methodology,reader.geneva,worldhealthorganization, 2012. 44

Appendices& In+depthinterviewguide Over,the,past,few,months,,we,have,been,conducting,a,global,mapping,of,training,in,Health, Policy,and,Systems,Research.,We,began,with,a,quantitative,survey,that,yielded,several, hundred,responses,including,187,full,responses,from,institutions,offering,hpsr,training,as, well,as,information,on,over,140,courses,representing,every,region,of,the,world.,the, following,questions,build,upon,and,seek,to,help,us,interpret,the,findings,from,the,survey,, and,we,are,reaching,out,to,leaders,like,you,in,the,hpsr,field,to,help,us,do,that.,, WedefineHPSRcoursesasthosethatemphasizehowtodoresearcharoundhealthsystems issues.theyshouldseektounderstandandimprovehowsocietiesorganizethemselvesin achievingcollectivehealthgoals,andhowdifferentactorsinteractinthepolicyand implementationprocessestocontributetopolicyoutcomes [tocreate]acomprehensive pictureofhowhealthsystemsrespondandadapttohealthpolicies,andhowhealthpolicies canshape andbeshapedby healthsystemsandthebroaderdeterminantsofhealth. Content, 1. Whatcoursecontent,inyourview,constitutesagoodqualityHPSRtraining? Wefoundthatthecoursecontentwasquitediverse,butsomeofthemostcommonly taughtcontentwas: & Policyanalysis/development/planning Healthsystemsstrengthening/analysis/dynamics Operationsresearch/implementationscience Healthsystemsresearch/socialscienceresearchmethods Healtheconomics(econometrics,costeffectiveness,etc.) Governance,leadership,andmanagement Ethics Epidemiologyandmeasurement/samplingissuesforhealthsystems Healthpromotion/literacy DothesefindingsagreewithwhatyoudoinyourownHPSRteachingandhoware theysimilar/different?isthereanothercontentareathatyouthinkshoulddefinitely bepartofanhpsrcourse? 2. HowdoesyourcoursehandlethechallengeofmultiRdisciplinaryviewpoints, particularly,differingworldviews,anddifferentwaysofapproachingresearch (epistemologies),thatmayorientthepurposeandapproachofhealthpolicyand systemsresearchdifferently? Note:,If,their,course,follows,only,one,discipline,but,their,institution,offers,different, approaches,,followcup, 45

3. Howdoesyourcoursereflectissuesaroundsocietalvaluesandethicswithinhealth systems?, Explain,that,all,health,systems,are,underpinned,by,values,that,shape,their,aims,and, operation.,for,example,,some,systems,are,concerned,with,equity,,others,with,individual, choice.,, Capacities,and,competencies, Wedefineacompetencyastheabilitytoapplyasetofrelatedknowledge,skills,and abilitiesneededtosuccessfullyperformimportantworkfunctionsortasks.aneducational competencyisusuallydefinedatthelevelofadegreeorprogram.incontrast,learning objectivesaremorespecificanddescribewhatthelearnershouldbeabletoachieveatthe endofacourse.learningobjectivessaywhatwewantthelearnerstoknowand competenciesindicatehowwecanbecertaintheyknowit. 4. Inyourexperience,whataresomeofthemostimportantcapacities(thosethatarebeing trainedthroughcourses)relatedtohealthpolicyandsystemsresearchthat,ideally,should bebuiltamongstudentsparticipatinginhpsrcourses?doesthiscorrespondtowhatyoudo inyourowncourse? Howdothesecapacities/learningobjectivesrelatetoanycompetenciesrelevanttoHPSRif definedbyyourinstitution? & Student,audiences,and,teaching,formats, 5. WehavefoundthatthemostcommonaudienceforHPSRcoursesismaster sstudents, followedbyhealthorganizationmanagers,doctoralstudents,policyrmakers,andngostaff. Doesthisfindingreflectyourexperience?Whoshouldbetheprimaryaudienceinyour view? WhatothergroupsshouldbetargetedforacourseinHPSR? 6. ThemostcommonlyusedapproachesinHPSRteachingincludeclassroomlecturesfollowed byindependentreading,casestudies,smallgroupdiscussions,andgroupprojects.whydo youthinkthisisthecase? WhichapproachesdoyouthinkwouldbemostvaluableinteachingHPSR?, Probe:,what,approach,helps,to,explore,real,life,problems?, 7. OnRsiteHPSRteachingwasusedmuchmorefrequentlythanonlineteaching.Atyour institution,whichisused,andinyourviewswhatmakeonemoreusedthantheother? Whatarethestrengthsandweaknessesof(whateverapproachtheyindicate)? 8. Whatareconstraintsbeyondyourinstitutionformoreonlinelearning? 46

9. ThemostcommonlyusedquantitativeresearchmethodsusedinHPSRcoursesinclude householdandhealthfacilitysurveys,systematicliteraturereviews,randomisedcontrolled trials,andcasercontrolstudies.doesthisfindingreflectyourexperience? Whichquantitativeresearchapproachesandmethodsdoyouthinkarethemostvaluableto teachinhpsr?why?, 10. Wefoundthatcasestudies,focusgroupdiscussions,keyinformantinterviews,inRdepth interviews,andparticipantobservationwerethemostcommonlytaughtqualitative researchmethodsincoursesrelevanttohpsr.(theyarealsoverycommonqualitative researchmethodsingeneral).couldyoucommentonthesefindings? Whichqualitativeresearchapproachesandmethodsdoyouthinkarethemostvaluableto teachinhpsr?why? Evaluation,and,followCup, 11. Inthisstudy,themajorityofparticipantswhoreportedevaluatingstudentsinHPSRcourses didsothroughpresentations,exams,researchprojects,essays,andreports.couldyou commentonthesefindings? Probe:,what,are,possible,advantages,and,disadvantages,to,using,these,evaluation, approaches? 12. Whatotherevaluationmethodsdoyouthinkaremostvaluableforstudentstoensurethey havebuiltcapacitiesinhpsr?, 13. Wefoundthatonly40%ofrespondentsprovidinginstitutionalinformationwerefollowing upwithstudentsoncetheyhadcompletedtheirtraining(and20%ofrespondentsdidnot knowwhetherstudentswerefollowedup).someofthewaysthatstudentswerefollowed upinclude: Surveys/Alumnisurveysbytraininginstitutionsoremployers Alumniandmentoringnetworksandevents,includingconferences Continuingeducationprograms Annuallettersandnewsletters TowhatextentdoyouthinkitisvaluabletofollowstudentsaftertheyhavecompletedHPSR training?why?whatdoyoufeelarethebestpossiblewaystofollowupwithstudents? Broader,analysis,and,suggestions, 14. Movingonfromthesurvey,whatdoyouthinkarethebestapproachestoexpandcapacity inhpsr?(insettingsyouarefamiliarwithorworldwide) 15. WhatdoyouthinkaresomeofthegreatestchallengesinteachingHPSR? 16. Howdoyouthinkthechallengesyouindicatedcouldbeovercome? 47

17. TowhatextentareyouwillingandabletocollaboratewithotherinstitutionsaroundHPSR training?whataboutyourinstitution? 18. Doyouhaveanyothercommentsonthestudyfindingsthatyouwouldliketoshare?Are anyofthesefindingssurprisingforyou?survey 48

Mapping of Health Policy and Systems Research Courses We are conducting global mapping of current teaching and training programmes with a focus on HEALTH POLICY AND SYSTEMS RESEARCH relevant to low- and middle-income countries. We aim to identify major gaps in training and promote access to HEALTH POLICY AND SYSTEMS RESEARCH training and teaching resources. We are going to ask you some questions about your institution first. We are then going to focus on specific HEALTH POLICY AND SYSTEMS RESEARCH courses that you are familiar with. By continuing and answering the questions, you are giving permission to participate in the survey. We appreciate you taking the time to share your important work with us 1. Name 2. Email 3. Telephone number (please include country and area codes) 4. Name of organization/department, if applicable Country City 5. Your primary location Introduction As an emerging area of research, HEALTH POLICY AND SYSTEMS RESEARCH asks questions that relate to realworld situations and issues that focus on health services and promoting health. It supports applied research that explicitly seeks to influence policy and has potential to lead to health system development. Health systems research is seen as particularly well suited to answer WHY and HOW a certain mix of structures, policies, or interventions may have shaped outcomes. It can draw on one or more disciplines and methods to answer these questions. Issues studied include how health care is financed, organised, delivered and used;; how health policies are prioritised, developed and implemented;; and why some health systems achieve their goals and others do not. It enables a comprehensive analysis of how health systems respond and adapt to health policies, and how health policies can shape and be shaped by health systems and the broader determinants of health. Both unintended consequences and complexity in achieving intended effects are often examined. Your Institution First we will ask some questions about HEALTH POLICY AND SYSTEMS RESEARCH training at your institution as a whole. Page 1

Mapping of Health Policy and Systems Research Courses 6. Are there courses relevant to HEALTH POLICY AND SYSTEMS RESEARCH offered at your institution? "Course" encompasses all capacity development initiatives for HEALTH POLICY AND SYSTEMS RESEARCH, including short-term in-service trainings, full semester-long courses, modules and short courses, online courses, special studies, dissertation mentoring, and others. Yes No I do not know Your Institution 7. Do you feel that courses relevant to HEALTH POLICY AND SYSTEMS RESEARCH should be offered at your institution? Yes No I do not know 8. Please explain your response: Your Institution 9. How many courses that have relevance to HEALTH POLICY AND SYSTEMS RESEARCH are offered at your institution? 1 2 3 More than 3 I do not know Page 2

Mapping of Health Policy and Systems Research Courses 10. A competency is the ability to apply a set of related knowledge, skills, and abilities needed to successfully perform important work functions or tasks. An educational competency is usually defined at the level of a degree or program. In contrast, learning objectives are more specific and describe what the learner should be able to achieve at the end of a course. Learning objectives say what we want the learners to know and competencies indicate how we can be certain they know it. As a simplified example, a competency within public health programs offered by an institution might be for students to, "demonstrate cultural awareness and sensitivity". A learning objective of a course within a public health program with that competency might be to "identify the range of social and environmental determinants of health among indigenous populations in Northern Alberta". Does your institution have educational competencies specified for any teaching relevant to HEALTH POLICY AND SYSTEMS RESEARCH? Yes No I do not know Your Institution 11. Please list competencies here or provide a URL to the appropriate page on your institution's website or academic handbook. If competencies apply to a specific program, please also indicate that program here. Your Institution Page 3

Mapping of Health Policy and Systems Research Courses 12. What are the target (student) audiences for training relevant to HEALTH POLICY AND SYSTEMS RESEARCH at your institution? Please check all that apply. Undergraduate students Master s students Doctoral students Policy-makers Health organization managers Practicing researchers Educators (teachers, trainers, professors) Physicians NGO staff I do not know Other (please specify) 13. Does your institution follow the professional development and career progression of those who have completed training relevant to HEALTH POLICY AND SYSTEMS RESEARCH? (E.g. Career surveys sent via alumni networks) Yes No I do not know Your Institution 14. How are these individuals followed-up after they complete their training? Your Institution Page 4

Mapping of Health Policy and Systems Research Courses 15. If you cannot provide an institutional perspective on HEALTH POLICY AND SYSTEMS RESEARCH training, can you identify a contact person at your institution who has a lead role in the oversight of academic programs and could answer questions about institutional involvement and capacity for teaching HEALTH POLICY AND SYSTEMS RESEARCH? Yes No Your Institution 16. Contact information (name, title, email, phone number) of the person you recommend to provide an institutional perspective of HEALTH POLICY AND SYSTEMS RESEARCH training: Specific Health Policy and Systems Research Courses We will now ask you some information about courses relevant to HEALTH POLICY AND SYSTEMS RESEARCH that you may be involved with. The survey will guide you through the same series of questions about each course for up to two different courses. Remember, for the purposes of this survey, "course" encompasses all capacity development initiatives for HEALTH POLICY AND SYSTEMS RESEARCH, including short-term in-service trainings, full semester-long courses, modules and short courses, online courses, special studies, dissertation mentoring, and others. Specific Health Policy and Systems Research Courses 17. Are you involved in at least one course that is relevant to HEALTH POLICY AND SYSTEMS RESEARCH? Yes No I do not know Specific Health Policy and Systems Research Courses Page 5

Mapping of Health Policy and Systems Research Courses 18. Do(es) the course(s) teach or appraise research methodologies? Yes No Yes, but teaching or appraising research methodologies is not central to the course(s) Specific Health Policy and Systems Research Courses 19. Do(es) the course(s) have application to a low- or middle-income country context? Yes No Specific Health Policy and Systems Research Course 1 20. What is the title of the first course relevant to HEALTH POLICY AND SYSTEMS RESEARCH that you are directly involved in? 21. What is your role in this course? (E.g. Course coordinator, module organizer, regular lecturer, etc.) 22. Please provide a brief description of the course including major topics/methods covered. (Suggestion: you may wish to copy and paste parts of the course syllabus here) Page 6

Mapping of Health Policy and Systems Research Courses 23. Does this course have core learning objectives? Yes No I do not know Specific Health Policy and Systems Research Course 1 24. What are the core learning objectives for this course? Specific Health Policy and Systems Research Course 1 25. What formats of teaching do you use in your course relevant to HEALTH POLICY AND SYSTEMS RESEARCH? (Check all that apply) Classroom lectures Small group seminars (<20 students) Case studies Problem sets Group projects Independent reading Simulations Online courses (closed to program participants) Blended learning (on-site and online;; flipped classrooms --online lectures with person-to-person discussion, case studies, problem sets, or simulations) Open access online courses (e.g. Coursera, OpenCourseWare) Practicum or internships Research projects Other (please specify) Page 7

Mapping of Health Policy and Systems Research Courses 26. How is this course primarily offered? On-site (face-to-face) Online Both Specific Health Policy and Systems Research Course 1 27. What is the primary on-site (face-to-face) location of the course? (Please provide the city and country) Specific Health Policy and Systems Research Course 1 28. What language(s) is this course offered in? (Please specify) 29. When is the course offered? (Please specify the usual timing and duration of the course and when it is repeated) Page 8

Mapping of Health Policy and Systems Research Courses 30. What are the target (student) audiences for this course relevant to HEALTH POLICY AND SYSTEMS RESEARCH? (Check all that apply) Undergraduate students Master s students Doctoral students Policy-makers Health organization managers Practicing researchers Educators (teachers, trainers, professors) Physicians NGO staff Other (please specify) Page 9

Mapping of Health Policy and Systems Research Courses 31. Which QUANTITATIVE research methods and study designs are taught for this course relevant to HEALTH POLICY AND SYSTEMS RESEARCH? (Check all that apply) Randomised controlled trials Cohort analysis Case-control Time & Motion Studies Non-random concurrent comparison studies Cluster or community trials Pragmatic trials Household surveys Health facility surveys Longitudinal modeling Multi-level statistical modeling Network analysis (social network analysis) Optimization/forecasting modeling Systems dynamics modeling Agent-based modeling Systematic reviews Quantitative meta-analysis None Other quantitative methods (please specify) Page 10

Mapping of Health Policy and Systems Research Courses 32. Which QUALITATIVE research methods and study designs are taught for this course relevant to HEALTH POLICY AND SYSTEMS RESEARCH? (Check all that apply) Ethnography Phenomenology Narratives In-depth interviews Key informant interviews Case studies Realist reviews Focus group discussions Participant observation Historical review Scenario-building Delphi design for opinions Participatory action research None Other qualitative methods (please specify) 33. Which MIXED research methods and study designs are taught for this course relevant to HEALTH POLICY AND SYSTEMS RESEARCH? (Check all that apply) Effectiveness-Implementation trials Mixed methods case studies None Other mixed methods (please specify) Page 11

Mapping of Health Policy and Systems Research Courses 34. What discipline does this course relevant to HEALTH POLICY AND SYSTEMS RESEARCH fall under? (Check all that apply) Anthropology Biostatistics Epidemiology Global/International Health Health Economics Health Policy or Health Systems Evaluation Health Policy or Health Systems Research Political Science Public Health Sociology I do not know Other (please specify) 35. Do you assess individuals during this course relevant to HEALTH POLICY AND SYSTEMS RESEARCH? Yes No Specific Health Policy and Systems Research Course 1 36. How are individuals assessed? (Please select all that apply) Exams Essays Literature reviews Policy briefs Practical assessments/role-playing Presentations Reports Research projects I do not know Other (please specify) Page 12

Mapping of Health Policy and Systems Research Courses Specific Health Policy and Systems Research Course 1 37. Are teaching materials for this course relevant to HEALTH POLICY AND SYSTEMS RESEARCH available via open access? Yes No 38. Is there a URL or hyperlink available for these teaching materials? Yes No I do not know 39. URL for open course teaching materials 40. Are you able to share non-open access materials for this course relevant to HEALTH POLICY AND SYSTEMS RESEARCH (including a description, outline, syllabus, or learning materials)? Yes No I do not know Not applicable (all materials open access) Specific Health Policy and Systems Research Course 1 Thank you Please send materials for this course relevant to HEALTH POLICY AND SYSTEMS RESEARCH to: hpsrmapping@gmail.com We will not share anything further without your explicit permission. Additional Courses 41. Are you involved in any other courses relevant to HEALTH POLICY AND SYSTEMS RESEARCH? Yes No Page 13

Mapping of Health Policy and Systems Research Courses Specific Health Policy and Systems Research Course 2 42. What is the title of the second course relevant to HEALTH POLICY AND SYSTEMS RESEARCH you are directly involved in? 43. What is your role in this course? (E.g. Course coordinator, module organizer, regular lecturer, etc.) 44. Please provide a brief description of the course including major topics/methods covered. (Suggestion: you may wish to copy and paste parts of the course syllabus here) 45. Does this course have core learning objectives? Yes No I do not know Specific Health Policy and Systems Research Course 2 46. What are the core learning objectives of this course? Page 14

Mapping of Health Policy and Systems Research Courses Specific Health Policy and Systems Research Course 2 47. What formats of teaching do you use in your course relevant to HEALTH POLICY AND SYSTEMS RESEARCH? (Check all that apply) Classroom lectures Small group seminars (<20 students) Case studies Problem sets Group projects Independent reading Simulations Online courses (closed to program participants) Blended learning (on-site and online;; flipped classrooms --online lectures with person-to-person discussion, case studies, problem sets, or simulations) Open access online courses (e.g. Coursera, OpenCourseWare) Practicum or internships Research projects Other (please specify) 48. How is this course primarily offered? On-site (face-to-face) Online Both Specific Health Policy and Systems Research Course 2 49. What is the primary on-site (face-to-face) location of the course? (Please provide the city and country) Specific Health Policy and Systems Research Course 2 50. What language(s) is this course offered in? (Please specify) Page 15

Mapping of Health Policy and Systems Research Courses 51. When is the course offered? (Please specify the usual timing and duration of the course and when it is repeated) 52. What are the target (student) audiences for this course relevant to HEALTH POLICY AND SYSTEMS RESEARCH? Please check all that apply. Undergraduate students Master s students Doctoral students Policy-makers Health organization managers Practicing researchers Educators (teachers, trainers, professors) Physicians NGO staff Other (please specify) Page 16

Mapping of Health Policy and Systems Research Courses 53. Which QUANTITATIVE research methods and study designs are taught for this course relevant to HEALTH POLICY AND SYSTEMS RESEARCH? (Check all that apply) Randomised controlled trials Cohort analysis Case-control Time & Motion Studies Non-random concurrent comparison studies Cluster or community trials Pragmatic trials Household surveys Health facility surveys Longitudinal modeling Multi-level statistical modeling Network analysis (social network analysis) Optimization/forecasting modeling Systems dynamics modeling Agent-based modeling Systematic reviews Quantitative meta-analysis None Other quantitative methods (please specify) Page 17

Mapping of Health Policy and Systems Research Courses 54. Which QUALITATIVE research methods and study designs are taught for this course relevant to HEALTH POLICY AND SYSTEMS RESEARCH? (Check all that apply) Ethnography Phenomenology Narratives In-depth interviews Key informant interviews Case studies Realist reviews Focus group discussions Participant observation Historical review Scenario-building Delphi design for opinions Participatory action research None Other qualitative methods (please specify) 55. Which MIXED research methods and study designs are taught for this course relevant to HEALTH POLICY AND SYSTEMS RESEARCH? (Check all that apply) Effectiveness-Implementation trials Mixed methods case studies None Other mixed methods (please specify) Page 18

Mapping of Health Policy and Systems Research Courses 56. What discipline does this course relevant to HEALTH POLICY AND SYSTEMS RESEARCH fall under? Anthropology Biostatistics Epidemiology Global/International Health Health Economics Health Policy or Health Systems Evaluation Health Policy or Health Systems Research Political Science Public Health Sociology Other (please specify) 57. Do you assess individuals during this course relevant to HEALTH POLICY AND SYSTEMS RESEARCH? Yes No Specific Health Policy and Systems Research Course 2 58. How are individuals assessed? (Please select all that apply) Exams Essays Literature reviews Policy briefs Practical assessments/role-playing Presentations Reports Research projects I do not know Other (please specify) Specific Health Policy and Systems Research Course 2 Page 19

Mapping of Health Policy and Systems Research Courses 59. Are teaching materials for this course relevant to HEALTH POLICY AND SYSTEMS RESEARCH available via open access? Yes No Specific Health Policy and Systems Research Course 2 60. Is there a URL or hyperlink available for these teaching materials? Yes No I do not know Specific Health Policy and Systems Research Course 2 61. URL for open course teaching materials Specific Health Policy and Systems Research Course 2 62. Are you able to share non-open access materials for this course relevant to HEALTH POLICY AND SYSTEMS RESEARCH (including a description, outline, syllabus, or learning materials)? Yes No I do not know Not applicable (all materials open access) Specific Health Policy and Systems Research Course 2 Thank you Please send materials for this course relevant to HEALTH POLICY AND SYSTEMS RESEARCH to: hpsrmapping@gmail.com We will not share anything further without your explicit permission. Additional Courses 63. Are you involved in any other courses relevant to HEALTH POLICY AND SYSTEMS RESEARCH? Yes No Page 20

Mapping of Health Policy and Systems Research Courses 64. It appears that you are involved in a number of courses relevant to HEALTH POLICY AND SYSTEMS RESEARCH. Can we contact you directly to ask you further questions about additional courses not mentioned here? Yes No 65. Thank you Please provide us with your preferred contact information: Preferred email address: Preferred phone number (please include country and area codes): 66. Could you please provide us with contact information for someone else who is able to answer similar questions regarding the additional course(s) relevant to HEALTH POLICY AND SYSTEMS RESEARCH that you are directly involved in and have not answered questions about yet in this survey (please provide name, title, email, and/or phone number): Forthcoming Courses 67. Even if courses relevant to HEALTH POLICY AND SYSTEMS RESEARCH are or are not currently offered at your institution, are you aware of any plans to offer any (additional) courses in the future? Yes No I am not aware Forthcoming Courses Page 21

Mapping of Health Policy and Systems Research Courses 68. Please describe any forthcoming courses relevant to HEALTH POLICY AND SYSTEMS RESEARCH: Follow-up Information 69. Please identify names and contact information for coordinators or course organizers for any other courses relevant to HEALTH POLICY AND SYSTEMS RESEARCH offered at your institution or beyond. First contact name and title First contact email address and phone number Second contact name and title Second contact email address and phone number Third contact name and title Third contact email address and phone number 70. Would you agree to be contacted by a member of the Teaching and Learning Thematic Working Group for a follow-up interview on some of the responses you ve provided? Yes No 71. Thank you Please provide us with the following information and someone will be in touch with you within one-to-two weeks: Preferred e-mail address Preferred phone number (please provide country and area codes) Page 22

Mapping of Health Policy and Systems Research Courses 72. Thank you very much for answering our screening question(s). Even though it appears that you are not directly involved with a course or an institution that meet the inclusion criteria for this study, we are seeking to gain as broad of a scope of courses as possible. Do you know anyone else at your institution or beyond that you think we should contact about relevant courses? First contact name and title First contact email and phone number Second contact name and title Second contact email and phone number Third contact name and title Third contact email and phone number Thank you for your responses We would like to invite you to the two sessions organised by the Teaching and Learning Thematic Working Group of the Alliance for Health Policy and Systems Research taking place at the Third Global Symposium on Health Systems Research in Cape Town (September 30 October 3): 1. Our satellite session entitled Health Policy and Systems Research Teaching and Learning 'Clinic', on Tuesday September 30 from 13:30 17:00 2. Our session on Health Policy and Systems Research. Please check the conference programme this summer: http://hsr2014.healthsystemsresearch.org/ for details Results from this study will be shared with participants in advance of the symposium. Page 23