MCAT Update. Judith Byrne Director, MCAT Administration & Reporting. Jen Page Director, MCAT Preparation Products

Transcription

1 MCAT Update Judith Byrne Director, MCAT Administration & Reporting Jen Page Director, MCAT Preparation Products Scott Oppler Director, MCAT Development & Psychometrics November 5, 2011

2 MCAT Update Test Administration 2011 Review 2012 Preview Resources for Examinees Test Scores and Research 2011 Test Scores Using MCAT Data in Student Selection Upcoming MR5 Sessions 2

3 Test Administration: 2011 in Review 3

4 We saw overall increases Scores Released 79,344 82,033 85,754 Unique Examinees 68,823 70,941 75,313 1 Attempt 58,768 60,297 64,869 2 Attempts 9,686 10,225 10,018 3 Attempts

5 Number of unique examinees increased by 6% in ,000 78,000 76,000 75,313 74,000 72,000 70,941 70,000 68,823 68,000 66,000 64,000 62,

6 Number of scores increased by 5% 88,000 86,000 85,754 84,000 82,000 82,033 80,000 79,344 78,000 76,

7 Number of voids increased by 5% in ,000 5,000 4,000 4,218 4,897 5,120 3,000 2,000 1,

8 14% of examinees retested within ,000 64,869 60, ,000 40,000 30,000 20,000 10, , Attempt 2 Attempts 3 Attempts

9 Shaking things up 28 administrations in 2011 (Jan Sept) PLUS one additional exam on October 7 th to accommodate earthquake examinees from August 23 rd 112 examinees tested on October 7 th Scores for October 7 th released on October 27 th

10 Numbers of 2011 examinees varied by race/ethnicity , , ,654 6, American Indian and Alaskan Native Asian Black or African American Hispanic or Latino White 10

11 Numbers of examinees in each group increased, some more than others Percent Group Change American Indian and Alaskan Native % Asian 16,574 17,760 7% Black or African American 6,185 6,654 8% Hispanic or Latino 5,632 6,446 14% White 32,640 34,025 4% 11

12 35% increase in accommodations applications approved in

13 Still a very small percentage of examinees tested with accommodations <.5% testers with accommodations Most commonly provided accommodations include: Extra testing time Time-and-a-quarter a Time-and-a-half Double-time Extra Breaks Separate testing rooms Specifically for disability To allow other accommodations, such as food and drink

14 Additional accommodations offered Food and drink, medication, other personal medical items Zoom Text or other magnification Adjustable workstations and other seating adaptations Extra breaks Paper exams Readers and scribes

15 FAP approvals have gone up by approximately 8% in ,000 7,000 6,000 6,203 6, ,000 4,827 4,000 3,000 2,000 1,

16 Benefits to FAP Recipients in 2011 Reduced MCAT registration fee of $85 (from $235) $500 to help defray costs of re-evaluations for those seeking accommodations Free copy of The Official Guide to the MCAT Exam Free copy of MSAR 14 free AMCAS applications

17 Test-Day Survey Results 100% 90% 80% 84% 84% 86% 70% 60% 50% Satisfied 40% Dissatisfied 30% 20% 10% 0% 3% 3% 4%

18 Test Administration: 2012 Preview 18

19 No major changes for 2012 Planning for analogous volume increases S&R opened October 12 th for the 2012 testing ti season Over 4k registrants t on opening day Same distribution of test dates Base registration fee has increased by $5 No major policy changes, but have updated web and print material to communicate better 19

20 New Additions New roles, but not new faces: Jeremy Manus, M.A. Supervisor, MCAT Examinee Services (202) Oscar Whiteman Supervisor, Accommodated Services (202) New to AAMC: Kathryn Bugbee, Ph.D. Director, Accommodated Testing (202)

21 Resources for Examinees Updates to existing resources & Updates to existing resources & new resources planned

22 What do we know about student preparation? p How did you prepare for the examination you took today (select all that apply)? Used an MCAT preparation book 77% Took one or more MCAT practice tests online 69% Took an MCAT preparation course provided by a private company 44% Took one or more paper-based MCAT practice tests 25% Took an MCAT preparation course provided d by your (or another) university 8% No Response 3%

23 Preparing for the MCAT Resources from AAMC: The Official Guide to the MCAT Exam MCAT Practice Tests New Resources

24 The Official Guide to the MCAT Exam Updated content includes: Current data on MCAT scores & GPA Updated info on changes in MCAT scores after retesting 2 new writing samples, with sample essays Buy online for $30 at:

25 Official Guide: Two Sections Narrative: 7 chapters Practice: new questions with detailed analysis

26 Part I: Narrative Overview of the MCAT Exam MCAT Fundamentals Test-Day Process and Procedures All About MCAT Scores Your MCAT Score as Part of the Admissions Decision To Retake or Not Retake THAT is the Question The Strategy t Mystique: Debunking the Myths Practice Effectively

27 Part II: Practice Content and Skill Classifications Tips Explanation of correct & incorrect answers Difficulty Level: Medium (45%)

28 The Official Guide to the MCAT Exam Great for getting familiar with the exam! 140 previously administered items, none found on practice tests Overview of everything MCAT Available online for $30 at: g *free to FAP recipients

29 MCAT Practice Tests Updated look, same tests Free practice test available to all 7 additional tests for sale $35 each

30 MCAT Practice Tests Simulates actual testing environment Real MCAT questions Customizable practice modes Solutions by test t developers Detailed score reports

31 Detailed Score Reports

32 What s next? The Official MCAT Self-Assessment Package Coming in Spring Self-Assessment Tests Physical Sciences Biological i l Sciences Verbal Reasoning Results compare individual id performance to the average performance on the same items Provides guidance on where to focus test preparation efforts Help students use practice data to guide study

33 Data to Support Individualize Prep The Official MCAT Self-Assessment Package Detailed analysis by: Exam section Content category within a section Self-rated confidence pre & post test Items self-identified as guessed Cognitive skills tested Item difficulty and type (discrete or passage-based) Practice Test Data to Track Progress Baseline test, before preparation begins Follow-up tests, to track improvements

34 New Additions Creating more resources Abby Thomsen, M.A. Instructional Designer, MCAT Preparation Products (202)

35 Test Scores and Research

36 Test Scores and Research Comparison of 2010 and 2011 test scores Using MCAT Data in Medical Student Selection Looking to the future: MR5 and the new MCAT

37 Test Scores and Research Comparison of 2010 and 2011 Comparison of 2010 and 2011 scores

38 How did examinees score on the MCAT in 2011? Results in 2011 were very similar to those reported in 2010 Total scores Section scores 38

39 2010 vs Total scores (Mean=25.0) 2011 (Mean=25.1) Percentag ge Total Score 39

40 2010 vs MC Section Scores MCAT Section Physical Sciences Biological Sciences Verbal Reasoning

41 2010 vs Writing Sample Scores Percentile th percentile M M 50 th percentile O O 75 th percentile Q Q

42 Test Scores and Research Using MCAT Data in Medical Using MCAT Data in Medical Student Selection

43 Using MCAT Data in Medical Student Selection Expanded from version of document sent to admissions offices last year Purpose is to provide admissions committees with information relevant to the use of MCAT scores and other applicant information in the selection of this year s medical school applicants Please take one with you (and keep it handy)! We want to hear your feedback

44 Questions addressed: What does the MCAT exam measure? How is the MCAT exam scored? How well do examinees score on the MCAT exam? How accurate are examinees MCAT scores and how should they be interpreted? Why are some MCAT scores marked as non- standard? 44

45 Questions addressed (cont d): How do examinees scores change when they retake the MCAT exam and how do admissions officers use scores for applicants who test more than once? [EXPANDED] How do admissions officers use MCAT scores and other application data in the holistic review of applicants qualifications? [EXPANDED] How well do UGPAs and MCAT scores predict students performance in medical school? [EXPANDED] 45

46 How do examinees scores change when they retake the MCAT exam? MCAT examinees can test up to three times in one calendar year, with no lifetime limit In 2010, approximately 47% of applicants reported scores from multiple test administrations in their applications On average, the lower the initial iti MCAT score, the greater the amount of score improvement However, the degree (and direction) of score change varies considerably across individuals 46

47 47 Changes in MCAT Total Scores for First-time Examinees Retesting in Same Year ( )

48 How well do UGPAs and MCAT scores predict students performance in medical school? In this year s document, we have provided d an expanded d set of grids depicting the relationships between UGPA, MCAT Total scores, and a variety of medical education outcome variables, including: 4- and 5-year graduation rates Withdrawing i or being dismissed i d for academic reasons First attempt and eventual pass rates for Step 1, Step 2-CK, and Step 2-CS exams (see Appendix) 48

49 Percentage of Students who Graduated from Medical School in 4 Years, by MCAT and UGPA Ranges Series 1 4 Series 2 Series 3 3 Series 4 Series 5 2 Series Category 1 Category 2 Category 3 Category 4

50 Percentage of Students who Graduated from Medical School in 5 Years, by MCAT and UGPA Ranges Series 1 4 Series 2 Series 3 3 Series 4 Series 5 2 Series Category 1 Category 2 Category 3 Category 4

51 Percentage of Students Passing Step 1 Exam on the First Attempt, by MCAT and UGPA Ranges Series 1 4 Series 2 Series 3 3 Series 4 Series 5 2 Series Category 1 Category 2 Category 3 Category 4

52 Percentage of Students Eventually Passing Step 1 Exam, by MCAT and UGPA Ranges Series 1 4 Series 2 Series 3 3 Series 4 Series 5 2 Series Category 1 Category 2 Category 3 Category 4

53 Test Scores and Research Looking to the Future: MR5 and the Looking to the Future: MR5 and the New MCAT

54 The 5 th Comprehensive Review of the MCAT After 3 years of dedicated effort, the MR5 committee has made its final recommendations regarding changes to the MCAT exam Among these recommendations are: The addition of a new section that will include content from psychology, sociology, and biology The addition of content typically taught in first semester introductory biochemistry courses The elimination of the Writing Sample section 54

55 The 5 th Comprehensive Review of the MCAT The committee has recommended that the new MCAT be introduced in the Spring of 2015 To support the development of new test content, the AAMC plans to begin administering an optional un-scored section at the end of the current exam, beginning in January 2013 To prevent an increase in the length of the testing day, the decision has been made to stop administering the Writing Sample at that time 55

56 The 5 th Comprehensive Review of the MCAT For more information about the recommendations of the MR5 committee, please join us at the following sessions: MR5 Update (Today,12:00 1:00 p.m., Convention Center, Rooms 406/407) AAMC Focus Session: Rethinking Admissions: Gateway to the Profession of Medicine (Tuesday, 8:00 9:30 a.m., Convention Center, Rooms 205/207) 56

57 And last but not least.. Please join us in welcoming the most recent addition to the MCAT team: Dr. Cindy Searcy, Director of MCAT Research Phone number: (202)

58 58 Questions?

59