The Current Utilization and Impact of Curriculum Mapping within Physician Assistant (PA) Education: A National Survey Project

Transcription

1 Proposal Number 3 Impact of Curriculum Mapping Page 1 of 12 The Current Utilization and Impact of Curriculum Mapping within Physician Assistant (PA) Education: A National Survey Project PI s Collaboration Statement: As principal investigator I have sought to lead our team in a manner that promotes collaboration between the sponsoring institution of [name of program number 1, blinded] Department of PA Studies and the [name of program number 2, blinded] Division of Physician Assistant Studies. The collaborative effort blends the extensive curriculum mapping experience of [name of program number 2, blinded] with the inquisitiveness of [name of program number 1, blinded] as they seek to add curriculum mapping features to their novel learning management system [acronym, blinded]. The collaboration also promotes the sharing of strong research experience in the areas of lead principal investigator and mentoring of survey instrument development of [names of two Co-PIs at PA program number 2, blinded] with the scholarly writing and project implementation experience of [name of PI at PA program number 1, blinded]. This has created a situation where [name of PI at PA program number 1, blinded] is being appropriately mentored and challenged in his initial lead investigator role. Key skills being developed include: networking to promote strong intra- and interinstitutional collaboration, budgeting and budget justification, survey instrument creation, effectively use of biostatistical consultation and navigating institutional requirements for budget approval and IRB. Funding Statement: The principal investigator and co-principal investigators have never been funded through the PAEA Research Grants Program.

2 Proposal Number 3 Impact of Curriculum Mapping Page 2 of 12 Abstract The quick turnover in medical knowledge necessitates PA educators continously assess and update curricula. Curriculum mapping has been described as an effective model for curriculum evaluation in the medical education literature. Little is known about the implementation of this tool within PA education. This project will survey PA program directors nationally to describe the current status of curriculum mapping usage within PA educational programs, evaluate the impact of curriculum mapping on the programs, assess programs level of satisfaction with mapping tools, and identify common barriers to adoption of curriculum mapping. Analyses will include descriptive statistics, bivariate plots and regression analyses. The results of this proposed survey will create a baseline assessment of the status of curriculum mapping within PA education. In addition, the results may help promote or discourage the adoption of curriculum mapping, based on the experiences of current users. The results may identify barriers that are amenable to faculty development initiatives or provide programs with data on which to base the decision to invest or redirect current program resources.

3 Proposal Number 3 Impact of Curriculum Mapping Page 3 of 12 The Current Utilization and Impact of Curriculum Mapping within PA Education Study Rationale: Rapid evolution in the biomedical sciences conveys a short half life to medical information 1. This quick turnover in current, relevant knowledge necessitates physician assistant (PA) educators continuously assess and update their curricula. Curriculum mapping has been used in medical education during the construction of integrated testing models. Gjerde 1 described how curriculum mapping can lead to the development of a test blueprint or table of specifications, which helps test planners to consider their content emphasis and their cognitive level emphasis when designing a test. Gjerde 1 also asserted that A mapping analysis can assess the degree of congruence within a curriculum and identify areas which need closer attention [thus] can be a useful model for curriculum evaluation. Drawing insight from the backward design model proposed by Wiggins and McTighe 2, PA programs could tie curriculum mapping or instructional planning to the NCCPA Examination Content Blueprint 3 as an initial or basic framework for curriculum development or evaluation. Key benefits of curriculum mapping include identification of instructional gaps, unintentional redundancies, inadequate breadth or depth of coverage, and disparity between intended and delivered instruction. Other uses for curriculum mapping noted in the medical literature have included its use to support quality enhancement for e-curricula and online instruction and as a tool for effective integration of new or specialty-focused information into an existing curriculum (i.e. genetic basis of disease, sexand gender-based sciences) 4,5. Dehn described eloquently the potential role mapping could have in an editorial note within an article by Holycross 6 in 2006 when he stated, In a curriculum set as complicated, concentrated, and integrated as core PA education, one tool that would provide great value would be a curriculum language and communication system understandable to all educators. Curriculum mapping may evolve to serve this purpose. The role of curriculum mapping was first described in the PA education literature in 2006 as a potentially useful tool for the development and evaluation of curricula 6. Essary and Statler 7 built upon this introductory article in 2007, when they illustrated how curriculum mapping can aid PA programs in assessing how they teach and evaluate students in regards to the six core competencies of the PA profession. During the 2009 PAEA Education Conference, the need for faculty development in the area of curriculum mapping was clearly communicated. At the close of that conference s Academic

4 Proposal Number 3 Impact of Curriculum Mapping Page 4 of 12 Coordinator s Retreat, curriculum mapping was added to the list of retreat topics for future discussion at the upcoming 2010 conference. These recent articles and events have introduced the role of mapping, illustrated its use, touted its promise and acknowledged the need for faculty development with this tool. A key purpose of this study is to assess if PA programs have adopted and benefited from curriculum mapping. If so, how effective is mapping at identifying curricular gaps or unintentional redundancies, aligning instruction with evaluation, fostering thoughtful sequencing of instruction, supporting continuous quality improvement of curriculum, communicating curriculum to stakeholders, and improving program efficiency? Research Questions: 1. What is the current state of curriculum mapping utilization within PA educational programs? 2. What has been the impact of curriculum mapping on programs? 3. How satisfied are curriculum mapping adopters with this process/tool? Study Aims: The aim of this study it to survey PA educational programs to describe the current status of curriculum mapping usage within PA educational programs, evaluate the impact of curriculum mapping on the programs, assess programs level of satisfaction with mapping tools, and identify common barriers to adoption of curriculum mapping. Methodology: Survey Development: The target audience of the survey is program directors or their designees responsible for curriculum evaluation components of continuous self-study. The survey will include the following elements: Demographic characteristics of the program. For example, geographic location, category of institutional sponsor, number of students per class, length of program, degree awarded and number of principal faculty with formal training as an educator (i.e. master s or doctorate degree in education; completion of an educational residency or fellowship). Describe the current status of curriculum mapping within PA educational programs. For example, the programs use of software to support curriculum mapping, type of software

5 Proposal Number 3 Impact of Curriculum Mapping Page 5 of 12 used, cost of curriculum mapping software, nature of software technical support, identification of additional software features Evaluate the impact of curriculum mapping on the program. For example, ability of mapping process to identify 1) instructional gaps, 2) unintentional redundancies in instruction, 3) topics not being covered in adequate depth within the curriculum, 3) disparity between intended and delivered instruction and 4) markers of quality of instruction. Ability of the curriculum mapping process to support continuous self-study. For example, capturing of useful quality assessment/improvement data and ease of report generation. Level of satisfaction with curriculum mapping software. For example, ability of curriculum mapping software to improve communication of the curriculum between faculty and program stakeholders or administrator perceptions of cost-effectiveness. Barriers to implementation. Curriculum mapping software adopters and non-users will be asked what they perceive to be the largest challenges or barriers to adopting software. Previously established barriers will be examined including cost, availability of commercial product amenable to PA education, inadequate time of staff or faculty to manage curriculum mapping, inadequate technology skills of staff or faculty, lack of institutional resources, unfamiliarity with curriculum mapping, and no perceived benefit to adopting curriculum mapping software. An opportunity for open ended comments of other potential barriers will also be provided for completeness. Survey Peer Review: The attached survey has undergone expert review by faculty trained in public health research and biostatistics. The survey was subsequently refined to enhance clarity, minimize or create controls for potential confounders, and facilitate seamless statistical analysis. A pilot survey will be conducted with a small sample (approximately 10) of program directors. Peer review will be conducted again after the pilot trial to determine if any additional refinements are warranted before nationwide administration to all PA educational program directors. Survey distribution: The survey will be distributed to all program directors of ARC-PA accredited programs as of September 2010 via the web-based survey tool, Survey Monkey. An

6 Proposal Number 3 Impact of Curriculum Mapping Page 6 of 12 automated reminder will be sent to all non-responders after two weeks. The researchers reserve the right to conduct telephone reminders or written requests to complete the survey instrument if the automated delivery and reminder system do not yield an adequate response rate for statistical analysis. Evaluation Plan: Analyses will include descriptive statistics consisting of frequency tables and percents for categorical variables and means, medians, standard deviations, ranges for continuous variables. Bivariate plots will be used to describe simple associations among continuous variables. Regression analyses will be used to investigate relationships between the categorical (logistic regression) and continuous (linear regression) outcomes and important characteristics. Demographic characteristics will be used to ensure survey respondents are representative of the larger PA program population. Statistical analyses will be conducted using SAS v Potential Significance: The results of this proposed survey will create a baseline assessment of the status of curriculum mapping within PA education. In addition, the results may help promote or discourage the adoption of curriculum mapping, based on the experiences of current users. The results may identify barriers that are amenable to faculty development initiatives or provide programs with data on which to base decisions to invest or redirect current program resources.

7 Proposal Number 3 Impact of Curriculum Mapping Page 7 of 12 References 1. Gjerde CL. Curriculum mapping: Objectives, instruction, and evaluation. Journal of Medical Education. 1981;56: Wiggins G, McTighe J (2006). Understanding by Design (2nd edition). Alexandria: Association for Supervision and Curriculum Development. 3. National Commission on Certification of Physician Assistants. Examination Content Blueprint. Available at: Accessed July 27, Weiss LB, Levison SP. Tools for integrating women s health into medical education: clinical cases and concept mapping. Academic Medicine. 2000;75(11): Fleiszer DM, Posel NH, Steacy SP. New directions in medical e-curricula and the use of digital repositories. Academic Medicine. 2004;79(3): Holycross J. Curriculum mapping: an essential tool for curriculum development. The Journal of Physician Assistant Education. 2006;17(4): Essary AC, Statler PM. Using a curriculum map to link the competencies for the PA profession with assessment tools in PA education. The Journal of Physician Assistant Education. 2007;18(1): Product Notes SurveyMonkey TM is a copyrighted product of SurveyMonkey, Inc. Information is available at: SAS Version 9.2 Software is a copyrighted product of SAS Institute, Inc. Information is available at:

8 Proposal Number 3 Impact of Curriculum Mapping Page 8 of 12 Budget Budget and Timeline Line items Description Description cont'd Personnel Funding Sought In- Kind Total Cost [PI, blinded name] 2% with dept match* [Co-PI, blinded name] 1% with dept match* [Co-PI, blinded name] via service contract** 1200 Biostatistical support stipend 500 Web based survey 160 $4756 $2896 $7652 *based on annualized base salary; **project support via service contract has been negotiated at the rate noted above for [one of the Co-PIs] Timeline November through December, 2010 Biostatistical consultation, pilot and refine survey. Distribution of survey to program directors via web-based January through February, 2011 survey. Follow-up request for survey completion. March through April, 2011 Data preparation and analyses. Summarize results and prepare for manuscript creation. Submit abstract for annual PAEA Education Conference. May through June, 2011 Prepare final report to PAEA Research Institute.

9 Proposal Number 3 Impact of Curriculum Mapping Page 9 of 12 PAEA Research Grant Proposal Survey The Current Utilization and Impact of Curriculum Mapping within PA Education Hello PAEA Colleague, Survey Cover Letter We are requesting your assistance by completing this brief anonymous survey on the role of curriculum mapping within your PA program. We anticipate it will take x minutes to complete. Your participation is voluntary and will provide valuable insights into the current status, benefits, costs and impact curriculum mapping has within PA education. If you have any questions or concerns about the survey please contact the principal investigator, [name and contact information of PI, blinded]. This survey has been approved by the IRB at [name of PA program, blinded]. Questions regarding the IRB may be directed to the [name and phone number of PA IRB office, blinded]. We appreciate your consideration of this request and look forward to sharing our research results with you. Sincerely, [PI, blinded name] [Co-PI, blinded name] [Co-PI, blinded name] Program demographics Survey Instrument 1. Which of the following PAEA geographic regions is your PA program located in: a. East b. Heartland c. Midwest d. Northeast e. Southeast f. West 2. Which of the following best describes your sponsoring institution? a. Private b. Public 3. Which of the following best describes your sponsoring institution? a. University: Non-Academic Health Center b. 4-year College c. Academic Health Center

10 Proposal Number 3 Impact of Curriculum Mapping Page 10 of 12 d. Community College e. Other 4. Please list how many students matriculate into your professional program during the current year? [ _] 5. What is the highest credential your entry-level PA program awards? a. Certificate b. Associate s c. Baccalaureate d. Master s e. Doctorate 6. What is the length of your PA program? [ months] 7. How many principal faculty within your program have formal training in an educationrelated field (e.g., baccalaureate, master s or doctorate in education, post-graduate teaching fellowship)? [ ] 8. What is the total number of principal faculty in your program? [ ] Describe the current status of curriculum mapping within your PA educational programs. 9. Does your program map its curriculum? If no, skip to question #19. a. Yes b. No 10. Does your program use a software program to support curriculum mapping processes? a. Yes b. No 11. If yes, what type of software does your program use for curriculum mapping? a. Locally developed program for curriculum mapping b. Database software (e.g. Excel, Access, Lotus 1-2-3, iworks Numbers) c. Product designed specifically for curriculum mapping d. Other, please specify 12. What is the cost of your curriculum mapping software? a. Purchase price: $, _ b. If applicable, cost per year: $, _ 13. Who provides technical support for your curriculum mapping software? Select all that apply. a. Institution b. Commercial Vendor c. PA Program Staff d. PA Program Faculty

11 Proposal Number 3 Impact of Curriculum Mapping Page 11 of 12 e. Other, please specify 14. Which of the following additional features does your curriculum mapping software? Select all that apply. a. Allow instruction planning b. Student performance tracking c. Evaluation of teaching effectiveness d. Reporting to show linkages of curricular events to The Standards, PA Competencies or PANCE task areas/content e. Measure depth of coverage (i.e. introduced, developed or reinforced) f. Allow tracking of the type of evaluation tools used for a particular topic/skill (i.e. MC test, essay, observation, simulation, SP/OSCE) g. Create graphical representation of curriculum elements and relationships h. Other 1, specify i. Other 2, specify The Impact of Curriculum Mapping on Program Effectiveness 15. Please rate how well your program s curriculum mapping software supports identification of: (1 = strongly agree to 5 = strongly disagree) a. Redundant instruction b. Omitted topics c. Disparity between planned and delivered instruction d. Inadequate quality of instruction Please rate how well your program s curriculum mapping software improves your ability to: a. Perform continous self-study b. Create useful reports c. Communicate your current curriculum to the principal faculty d. Communicate your current curriculum to adjunct faculty or guest lecturers 17. I am satisfied with the benefit of curriculum mapping relative to the financial and time expenditures required to use it. a. Strongly Agree b. Agree c. Neutral d. Disagree e. Strongly Disagree 18. Overall, I am satisfied with our curriculum mapping software. a. Strongly Agree b. Agree c. Neutral

12 Proposal Number 3 Impact of Curriculum Mapping Page 12 of 12 d. Disagree e. Strongly Disagree Barriers to implementation. Please identify which of the following that was (or is) a barrier to implementing curriculum mapping software. (Select all that apply.) 19. Cost 20. Availability of commercial product 21. Staff time 22. Faculty time 23. Program director time 24. Technology skills of staff 25. Technology skills of faculty 26. Technology skills of program director 27. Institutional resources 28. Lack of familiarity with curriculum mapping 29. No perceived benefit to adopting curriculum mapping software.