DISCIPLINARY ACTION BY MEDICAL BOARDS AND PRIOR BEHAVIOR IN MEDICAL SCHOOL

Transcription

1 DISCIPLINARY ACTION BY MEDICAL BOARDS AND PRIOR BEHAVIOR IN MEDICAL SCHOOL By Maxine A. Papadakis, M.D., Arianne Teherani, Ph.D., Mary A. Banach, Ph.D., M.P.H., Timothy R. Knettler, M.B.A., Susan L. Rattner, M.D., David T. Stern, M.D., Ph.D., J. Jon Veloski, M.S., Carol S. Hodgson, Ph.D. ABSTRACT Background Evidence supporting professionalism as a critical measure of competence in medical education is limited. In this case control study, we investigated the association of disciplinary action against practicing physicians with prior unprofessional behavior in medical school. We also examined the specific types of behavior that are most predictive of disciplinary action against practicing physicians with unprofessional behavior in medical school. Methods The study included 235 graduates of three medical schools who were disciplined by one of 40 state medical boards between 1990 and 2003 (case physicians). The 469 control physicians were matched with the case physicians according to medical school and graduation year. Predictor variables from medical school included the presence or absence of narratives describing unprofessional behavior, grades, standardized-test scores, and demographic characteristics. Narratives were assigned an overall rating for unprofessional behavior. Those that met the threshold for unprofessional behavior were further classified among eight types of behavior and assigned a severity rating (moderate to severe). Results Disciplinary action by a medical board was strongly associated with prior unprofessional behavior in medical school (odds ratio, 3.0; 95 percent confidence interval, 1.9 to 4.8), for a population attributable risk of disciplinary action of 26 percent. The types of unprofessional behavior most strongly linked with disciplinary action were severe irresponsibility (odds ratio, 8.5; 95 percent confidence interval, 1.8 to 40.1) and severely diminished capacity for self-improvement (odds ratio, 3.1; 95 percent confidence interval, 1.2 to 8.2). Disciplinary action by a medical board was also associated with low scores on the Medical College Admission Test and poor grades in the first two years of medical school (one percent and seven percent population attributable risk, respectively), but the association with these variables was less strong than that with unprofessional behavior. Conclusions In this case control study, disciplinary action among practicing physicians by medical boards was strongly associated with unprofessional behavior in medical school. Students with the strongest association were those who were described as irresponsible or as having diminished ability to improve their behavior. Professionalism should have a central role in medical academics and throughout one s medical career. The importance of professionalism in medical school is receiving renewed attention. 1-6 A fundamental assumption in medical education is professional students become professional physicians. However, the data to support this assumption are limited. 2 In a pilot study of physician graduates of the University of California, San Francisco (UCSF), we found disciplinary action taken against physicians by the Medical Board of California was associated with prior unprofessional behavior when the physicians were students. 7 We also identified three types of unprofessional behavior that were of particular concern: irresponsibility, diminished capacity for self-improvement, and poor initiative. 8 We undertook this case control study, involving three medical schools, to determine whether these findings could be generalized to all medical students and state medical boards. METHODS Selection of Physicians Who Were Disciplined The physicians who had been disciplined were graduates of JOURNAL OF MEDICAL LICENSURE AND DISCIPLINE vol 92 Number Page 11

2 three medical schools since 1970: the University of Michigan Medical School in Ann Arbor, Jefferson Medical College of Thomas Jefferson University in Philadelphia and UCSF School of Medicine. These schools were chosen for reasons of geographic diversity and to provide representation of both public and private institutions. In addition, complete records for their graduates were available. The physicians from the University of Michigan and Jefferson Medical College included all graduates disciplined by any state medical board in the United States between 1990 and The physicians from UCSF included all graduates disciplined by any state board other than the Medical Board of California during the same period. UCSF graduates disciplined by the Medical Board of California were excluded from this study, because they had been described previously. 7 All physicians were identified through a search of public records maintained in databases by the Federation of State Medical Boards. The disciplinary actions taken against physicians are available to the public 9-13 according to individual state laws. The disciplinary actions range from public reprimand to revocation of the medical license. According to the Federation of State Medical Boards, even behavior that results in the least severe disciplinary action public reprimand may adversely affect patients. 13 Three persons, two of whom were staff members at the Federation of State Medical Boards, classified the disciplinary actions of the state boards into three categories: unprofessional behavior, incompetence and violation with the category not determined. Selection of Control Physicians In the analysis, each physician who was disciplined was paired with two control physicians who had graduated within one year of the disciplined physician and for whom no disciplinary actions were recorded in the database of the Federation of State Medical Boards. In the pilot study, the reports of unprofessional behavior among the control physicians differed among medical specialties. 7 To control for specialty in this study, the specialty of one of the two control physicians was matched to that of the disciplined physician. Information regarding specialties was obtained from the American Medical Association masterfile 14 and the database of the American Board of Medical Specialties. Measurements The graduates academic records from their medical schools contained applications for admission, course grades, evaluation narratives, scores from licensing examinations, administrative correspondence, and the dean s letter of recommendation to a residency program. Research assistants and academic investigators for this study gathered the data from these records while blinded to the case or control status of the physicians. Negative excerpts about professional behavior were culled from reports of admission interviews, course evaluations (including check marks in designated boxes on rating forms and narrative comments), deans letters of recommendation to residency programs, and any documents in the students files dated before graduation. The course-evaluation forms contained items intended to capture the entirety of professional behavior. Overall Unprofessional Rating The excerpts containing information about unprofessional behavior were compiled and assigned a severity rating for such behavior by at least two investigators. The definition of unprofessional behavior was based on our previously established criteria. 15,16 The rating scale for unprofessional behavior included the five categories evaluated in the UCSF pilot study: none, trace, concern, problem, and extreme. 7 An a priori decision was made that the ratings would be dichotomized, with the categories of concern, problem, and extreme meeting the threshold of unprofessional behavior. The investigators who assigned ratings could refer back to the academic file to provide a context for the excerpts. In the ratings of the negative excerpts, the interobserver agreement was 91 percent; the interobserver correlation was 95 percent for severity ratings of none or trace as compared with concern, problem, or extreme. Consensus was reached on all discordant rankings. Types of Unprofessional Behavior An analysis of the content of the negative excerpts was performed to characterize the types of behavior that were deemed unprofessional. The items from the UCSF Professionalism Evaluation Form and from our pilot study were used to develop a set of software-related search terms (with the use of QSR NVivo software, version 2.0) for eight types of unprofessional behavior. 8,15,16 Two of the academic investigators reviewed all comments coded by the software; search terms were either added or removed by consensus. An NVivo listing of the total number of search terms per type of behavior per physician was uploaded into an SPSS statistical program. The severity of unprofessional behavior was ranked on the basis of the frequency of the search terms (none = 0; one or two times = moderate; three or more times = severe). Page 12 Journal of Medical Licensure and Discipline vol 92 Number

3 Other Predictor Variables Other variables included age, sex, undergraduate gradepoint average (GPA) for science courses, scores on the Medical College Admission Test (MCAT), grades for medical school courses and clerkships, and scores on the examination of the National Board of Medical Examiners (NBME), Part I, or on the United States Medical Licensing Examination (USMLE), Step 1. The scaled scores based on different versions of the MCAT were transformed to z scores with the use of the means and standard deviations for each subtest of each version of the MCAT. The mean z score of the subtests for each student was used as the independent variable. For students who repeated the MCAT, the mean of the first two scores was used. 17 The three medical schools used numerical, letter, and pass fail grades. To standardize these measures, we dichotomized the grades on the basis of the inability to pass a course on the first attempt (as indicated by a number grade below 70 points, a letter grade of D or F, or a provisional nonpass or fail). Raw scaled scores from NBME Part I and USMLE Step 1 were changed to z scores with the use of the mean and standard deviation for the year in which the test was taken. Statistical Analysis The demographic characteristics of the disciplined and control physicians were analyzed with the use of the chi-square test for proportions. The associations of predictor variables with case and control status were first examined with the use of conditional logistic-regression models (SAS software, version 8) that adjusted only for specialty, as required by the sampling design. 18 We then examined the association between the predictor variables and disciplinary action using unadjusted and adjusted conditional logistic-regression analyses. Variables in the multivariate model included sex, MCAT z scores, the number of medical- school courses not passed on the first attempt, the overall measure of unprofessional behavior, and the specialties of the physicians (categorized as internal medicine, family practice, obstetrics and gynecology, pediatrics, or all other specialties). We subsequently evaluated the eight types of unprofessional behavior as predictors of disciplinary action using unadjusted conditional logistic-regression analyses. These eight types of behavior (each categorized as 0, 1, or 2) then competed for inclusion in a conditional logistic-regression model that predicted the risk of disciplinary action. The two types of behavior found to be significant in the logisticregression analysis and a third behavior that almost reached statistical significance replaced the variable for overall unprofessional behavior in a multivariate model that adjusted for all the variables listed above. We then repeated the multivariate conditional logistic- regression analysis and replaced the three types of behavior with their scores for severity of behavior (moderate or severe). The proportion of disciplinary action attributable to a variable was calculated with the use of population attributable risk 19 according to the following equation (with PAR denoting population attributable risk, Pd the proportion of the exposure in the cases and RR the adjusted relative risk): PAR = [Pd x (RR-1)] RR. Continuous variables (e.g., MCAT z scores) were dichotomized (as the proportion of cases in the bottom quartile vs. others). The frequency distribution of specialties represented by the physicians who had been disciplined was compared with that of the specialties of all graduates of the three medical schools, to determine whether the specialties were similarly distributed. Evidence indicates that physicians who have been in practice for more than 20 years are at increased risk for disciplinary action. 20,21 We investigated whether this was true in our study sample by dichotomizing the disciplined physicians according to the year of graduation before 1980 and 1980 or later. The institutional review boards of UCSF, the University of Michigan, and Jefferson Medical College approved this study, and none required informed consent from the graduates. The Federation of State Medical Boards approved and collaborated with the investigators of this study. To protect confidentiality, we did not list the number of disciplined physicians according to medical school, year of graduation, or state in which disciplinary actions occurred. RESULTS Records for 235 of the 243 physicians who were disciplined and 469 of the 486 control physicians were available. Each of these 704 physicians graduated from one of the three medical schools between 1970 and One or more of 40 state medical boards disciplined the case physicians; unprofessional behavior was the basis for at least 74 percent of the violations (Table 1). Most physicians who were disciplined committed multiple violations; for 94 percent of those who were disciplined, one or more violations involved unprofessional behavior. The disciplined physicians had a slightly lower mean JOURNAL OF MEDICAL LICENSURE AND DISCIPLINE vol 92 Number Page 13

4 undergraduate science GPA than did the control physicians (Table 2). MCAT scores were also slightly lower among the disciplined physicians, as were NBME Part I scores and USMLE Step 1 scores. There was no difference in the findings for physicians who took the MCAT twice. Disciplined physicians were roughly twice as likely as control physicians not to have passed at least one course on the first attempt in both the preclinical and clinical years of medical school. Overall Unprofessional Behavior Twice as high a proportion of disciplined physicians as of control physicians demonstrated unprofessional behavior in medical school (Table 3). In unadjusted analyses, disciplined physicians were more likely than control physicians to display the following types of unprofessional behavior while in medical school: irresponsibility, diminished capacity for self-improvement, poor initiative, impaired relationships with students, residents, and faculty, impaired relationships with nurses, and unprofessional behavior associated with being anxious, insecure, or nervous. The multivariate analysis revealed three variables with regard to medical school that independently predicted disciplinary action. Unprofessional behavior was associated with an increase, by a factor of three, in the risk of subsequent disciplinary action, and it accounted for the largest population attributable risk (26 percent) (Table 4). Low MCAT scores and low grades in the first two years of medical school were also significant predictors, with a population attributable risk of disciplinary action of one percent and seven percent, respectively. Types of Unprofessional Behavior We evaluated the types of unprofessional behavior and the frequency of their occurrence during medical school Page 14 Journal of Medical Licensure and Discipline vol 92 Number

5 (Table 3). Examples of irresponsibility were unreliable attendance at clinic and not following up on activities related to patient care. Examples of diminished capacity for self-improvement were failure to accept constructive criticism, argumentativeness and display of a poor attitude. Poor initiative was characterized by a lack of motivation or enthusiasm or by passivity. Two types of unprofessional behavior independently predicted disciplinary action: irresponsibility and diminished capacity for self-improvement. The odds of receiving disciplinary action increased as the frequency of unprofessional behavior increased; students who were severely irresponsible (as indicated by three or more search terms) or who were described as severely unable to improve their behavior had odds ratios of 8.5 (95 percent confidence interval, 1.8 to 40.1) and 3.1 (95 percent confidence interval, 1.2 to 8.2), respectively, for subsequent disciplinary action. JOURNAL OF MEDICAL LICENSURE AND DISCIPLINE vol 92 Number Page 15

6 Unprofessional behavior associated with being anxious, insecure, or nervous (three or more search terms) approached statistical significance (P=0.06). Other Analyses The major predictor variable, overall unprofessional rating, remained significantly associated with disciplinary action when it was analyzed within subgroups. Disciplined physicians were compared with control physicians matched by specialty (odds ratio, 3.1; 95 percent confidence interval, 1.8 to 5.3) and with control physicians not matched by specialty (odds ratio, 3.1; 95 percent confidence interval, 1.7 to 5.8), as well as physicians stratified according to year of graduation 1970 to 1979 (odds ratio, 2.9; 95 percent confidence interval, 1.6 to 5.2) and 1980 to 1999 (odds ratio, 3.5; 95 percent confidence interval, 1.6 to 7.7). Two variables (undergraduate science GPA and z scores on NBME Part I and USMLE Step 1) were deleted from the final model because these variables were missing for nearly 30 percent of the study subjects. Had the two variables remained in the Page 16 Journal of Medical Licensure and Discipline vol 92 Number

7 final model, they would not have been associated with disciplinary action (odds ratio for undergraduate science GPA, 0.8; 95 percent confidence interval, 0.4 to 1.5; odds ratio for z scores on the NBME Part I and USMLE Step 1 board tests, 0.9; 95 percent confidence interval, 0.6 to 1.3). However, the association of the overall unprofessional rating with disciplinary action would have persisted (odds ratio, 5.2; 95 percent confidence interval, 2.6 to 10.1). The comparison of the distribution of specialties among the disciplined physicians with that among the graduates of the three medical schools is shown in Table 5. The specialties of family practice and obstetrics and gynecology were overrepresented among disciplined physicians, and pediatrics was underrepresented. The UCSF graduates who were disciplined outside of California and were subjects in this study were similar to the previously reported UCSF graduates who were disciplined within California 7 : chi-square analyses showed no difference between these two groups in terms of sex distribution (P=0.11), the frequency of unprofessional behavior (none or trace vs. concern, problem, or extreme; P=0.36), or distribution of specialties (P=0.17). DISCUSSION In this case control study, we found that physicians who were disciplined by state medical licensing boards were three times as likely to have displayed unprofessional behavior in medical school than were control students. This association was observed among graduates of three geographically diverse medical schools, both public and private, and among 40 state licensing boards. Unprofessional behavior as a student was by far the strongest predictor of disciplinary action. Furthermore, the types of unprofessional behavior displayed by students were associated with subsequent disciplinary actions. Among students who were subsequently disciplined, the most irresponsible had a risk of later disciplinary action eight times as high as that for control students, and those who were the most resistant to self-improvement had a risk of later discipline three times as high as that for controls. Among students who were subsequently disciplined, students with low MCAT scores and those with low grades in the first two years of medical school were also at risk for future disciplinary action, but these were associated with, at most, only one quarter of the risk attributed to unprofessional behavior. Recent objectives for undergraduate and graduate medical education provided by the Association of American Medical Colleges and the Accreditation Council for Graduate Medical Education include professionalism as a core competency. 22,23 Our study provides empirical support for its inclusion and also provides concrete data regarding what is meant by unprofessional behavior. In previous studies, physicians practicing in the areas of obstetrics and gynecology, general practice, psychiatry and family medicine were more likely to receive disciplinary action, and those practicing in pediatrics and radiology were less likely to be disciplined. 20,21 The practices of internal medicine, surgery and anesthesiology were not predictive of disciplinary action. In our study, similar patterns of discipline according to specialty were seen in five of the seven largest specialties (internal medicine, family practice, pediatrics, surgery and obstetrics and gynecology); these patterns support the generalizability of our findings. In contrast to earlier studies, we did not find male sex to be a risk factor. 20,21 Our study design precluded a full assessment of age as a risk factor for disciplinary action. The maintenance of complete student files since 1970 on the part of the three medical schools included in this study JOURNAL OF MEDICAL LICENSURE AND DISCIPLINE vol 92 Number Page 17

8 afforded a unique opportunity for investigation. Nonetheless, the limitations of this study include its retrospective design and the absence of data, because of incomplete medical school files, for disciplined physicians who graduated before Also, there may have been additional types of unprofessional behavior in medical school that led to disciplinary action that can best be identified with the use of multidimensional assessments (360-degree multisource feedback i.e., from peers, patients and coworkers) of professional competency. 24,25 The national rate of disciplinary action among the approximately 725,000 physicians practicing in the United States is 0.3 percent. 13 Actions taken by state medical boards may reflect only the most extreme forms of unprofessional behavior. Despite this possibility, our study revealed a strong association between disciplinary action on the part of 40 state medical boards and unprofessional behavior among students. What should be done with the findings of this study? Technical standards for admission to medical school and outcome objectives for graduation should be reviewed to make certain they contain explicit language about professional behavior. Standardized instruments should be implemented that assess the personal qualities of medical school applicants and that predict early medical school performance. 26 Professionalism can and must be taught and modeled. 5,27-29 Improved systems of evaluation are needed to monitor the development of professional behavior and to document deficiencies. 30 Providing students with feedback guided by evidence may motivate and direct remediation strategies, but the best practices for the remediation of deficiencies in professional behavior need development A recent study showed medical students who lack thoroughness and are unable to perceive their weaknesses in the first two years of medical school are more likely than those who do not have these deficiencies to be identified as unprofessional in the clinical years. 33 Our study extends this finding by demonstrating, among some students, unprofessional behavior is sustained over decades. However, disciplinary action by state medical boards occurs much less frequently than does unprofessional behavior in medical school. Not only do these two assessments have different thresholds, but physicians are also likely to improve in terms of professionalism with training and experience. 34 Our study supports the importance of identifying students who display unprofessional behavior. A prospective study looking at the later performance of these students could assess the effect of interventions on professional development. Page 18 Journal of Medical Licensure and Discipline vol 92 Number

9 ACKNOWLEDGEMENTS Supported in part by an Edward J. Stemmler, M.D., Medical Education Research Fund grant from the National Board of Medical Examiners (NBME). This study does not necessarily reflect NBME policy and NBME support includes no official endorsement. No other potential conflict of interest relevant to this article was reported. We are indebted to Toni Conrad and Bonnie Hellevig for their assistance with data abstraction; to Amanda Gilbert at the Association of American Medical Colleges for providing national mean MCAT scores; to Rachel Glick, M.D., for her assistance in the organization of the study; to Eric Vittinghoff, Ph.D., Michael G. Shlipak M.D., M.P.H., and Gretchen Guiton, Ph.D., for the statistical analyses; and to Robert Galbraith, M.D., and Stephen Clyman, M.D., of the NBME and James N. Thompson, M.D., of the Federation of State Medical Boards for their support. Copyright 2005 Massachusetts Medical Society. Reprinted with permission from pages of the December 22, 2005, issue of the New England Journal of Medicine. REFERENCES 1. Arnold L. Assessing professional behavior: yesterday, today, and tomorrow. Acad Med 2002;77: Stern DT, ed. Measuring Medical Professionalism. New York: Oxford University Press, Veloski JJ, Fields SK, Boex JR, Blank LL. Measuring professionalism: a review of studies with instruments reported in the literature between 1982 and Acad Med 2005;80: ABIM Foundation, ACP-ASIM Foundation, European Federation of Internal Medicine. Medical professionalism in the new millennium: a physician charter. Ann Intern Med 2002;136: Cruess SR, Johnston S, Cruess RL. Profession: a working definition for medical educators. Teach Learn Med 2004;16: Cohen JJ. Measuring professionalism: listening to our students. Acad Med 1999; 74: Papadakis MA, Hodgson CS, Teherani A, Kohatsu ND. Unprofessional behavior in medical school is associated with subsequent disciplinary action by a state medical board. Acad Med 2004;79: Teherani A, Hodgson CS, Banach M, Papadakis MA. Domains of unprofessional behavior during medical school associated with future disciplinary action by a state medical board. Acad Med 2005;80:Suppl:S17- S California Law. California Business and Professions Code, ch. 5, 2227, 803.1, 2027: Sacramento: Legislative Counsel of California. (Accessed Nov. 23, 2005, at gov/calaw.html.) 10. Michigan Department of Community Health. Michigan Public Health Code. P.A. 368 of 1978, as amended. (Accessed Nov. 23, 2005, at michigan.gov/mdch/0,1607, _ ,00.html.) 11. Disciplinary actions: Pennsylvania Department of State. (Accessed Nov. 23, 2005, at state.pa.us/bpoa/cwp/view.asp?a=1104&q=432631&b poanav=.) 12. Federation of State Medical Boards. DocInfo: the premier physician disciplinary history report service. (Accessed Nov. 23, 2005, at Idem. Summary of 2003 board actions. April 4, (Accessed Nov. 23, 2005, at pdf/fpdc_summary_boardactions_2003.pdf.) 14. American Medical Association physician masterfile. Chicago: American Medical Association, 2003, Papadakis MA, Osborn EH, Cooke M, Healy K. A strategy for the detection and evaluation of unprofessional behavior in medical students. Acad Med 1999;74: Papadakis MA, Loeser H, Healy K. Early detection and evaluation of professionalism deficiencies in medical students: one school s approach. Acad Med 2001;76: Hojat M, Veloski JJ, Zeleznik C. Predictive validity of the MCAT for students with two sets of scores. J Med Educ 1985;60: Logistic regression for matched case control studies. In: Hosmer DW, Lemeshow S. Applied logistic regression. New York: John Wiley, 1989: Rockhill B, Newman B, Weinberg C. Use and misuse of population attributable fractions. Am J Public Health 1998;88: Kohatsu ND, Gould D, Ross LK, Fox PJ. Characteristics associated with physician discipline: a case-control study. Arch Intern Med 2004;164: Morrison J, Wickersham P. Physicians disciplined by a state medical board. JAMA 1998;279: Learning objectives for medical student education guidelines for medical schools: report I of the Medical School Objectives Project. Acad Med 1999;74: Accreditation Council for Graduate Medical Education. Outcome Project: general competencies. (Accessed Nov. 23, 2005, at org/outcome/comp/compfull.asp.) JOURNAL OF MEDICAL LICENSURE AND DISCIPLINE vol 92 Number Page 19

10 24. Epstein RM, Hundert EM. Defining and assessing professional competence. JAMA 2002;287: Ramsey PG, Wenrich MD, Carline JD, Inui TS, Larson EB, LoGerfo JP. Use of peer ratings to evaluate physician performance. JAMA 1993;269: Eva KW, Reiter HI, Rosenfeld J, Norman GR. The ability of the multiple miniinterview to predict preclerkship performance in medical school. Acad Med 2004; 79:Suppl:S40-S Learning and transfer. In: National Research Council. How people learn. Washington, D.C.: National Academy Press, 2000: Brufee KA. Collaborative learning: higher education, interdependence, and the authority of knowledge. 2nd ed. Baltimore: Johns Hopkins University Press, Cruess SR, Cruess RL. Professionalism must be taught. BMJ 1997;315: Dannefer EF, Henson LC, Bierer SB, et al. Peer assessment of professional competence. Med Educ 2005;39: National Research Council. How people learn. Washington, D.C.: National Academy Press, 2000: Glick TH. Evidence-guided education: patients outcomes data should influence our teaching priorities. Acad Med 2005;80: Stern DT, Frohna AZ, Gruppen LD. The prediction of professional behaviour. Med Educ 2005;39: Ginsburg S, Regehr G, Hatala R, et al. Context, conflict, and resolution: a new conceptual framework for evaluating professionalism. Acad Med 2000;75: Suppl:S6-S11. Page 20 Journal of Medical Licensure and Discipline vol 92 Number