ASSESSMENT REPORT CMDS Master s 04 I. CMDS Master s Degree (MS/MCD) student performance on the National Examination in Speech Language Pathology Expected Outcome: Students nearing the completion of their graduate coursework will demonstrate their knowledge and skill in speech language pathology by passing the National Examination (PRAXIS in Speech Language Pathology). Assessment: Method A. The PRAXIS pass rate for AU Master s in Communication Disorders (MS/MCD) students will be calculated based on the reported scores collected across the academic year. B. Average subtest performances within AU MS/MCD student cohort of reported scores will be calculated within the academic year. The seven main subtest categories of the PRAXIS in Speech Language Pathology are: I. Basic Human Communication Processes, II. Phonological and Language Disorders, III. Speech Disorders, IV. Neurogenic Communication Disorders, V. Audiology and Hearing, VI. Clinical Management and VII. Professional Issues, Psychometrics and Research. These averages from our AU MS/MCD students will be compared to the national data offered by the test service for the subtest performance range of the middle 50% of students who took the examination during AY 03 04. C. Demonstration of Knowledge and skills by our graduate students is defined as maintenance of an 80% pass rate on this National PRAXIS Exam in Speech Language Pathology. This metric is required by the American Speech Language Hearing Association s Council on Academic Accreditation. D. To further enhance our program s self study and improvement. We are targeting each subtest performance average (across cohort as data reporting permits) to fall within or above the national middle 50% performance range provided from the testing service. Assessment: Findings The pass rate for AU MS/MCD students during AY 0 03 was 00% (n = 5). A. Average performances within Subtest and National (middle 50% performance range): a. Subtest I. Basic Human Communication Processes: i. AU average performance: 7% correct ii. National Mid 50% range: 58% 77% iii. National Mid 50% midpoint: 68% b. Subtest II. Phonological and Language Disorders: i. AU average performance: 8% correct ii. National Mid 50% range: 7% 85%
iii. National Mid 50% midpoint: 79% c. Subtest III. Speech Disorders i. AU average performance: 76% correct ii. National Mid 50% range: 63% 80% iii. National Mid 50% midpoint: 70% d. Subtest IV. Neurogenic Communication Disorders: i. AU average performance: 7% correct Note: within range but below national mid 50% midpoint ii. National Mid 50% range: 63% 80% iii. National Mid 50% midpoint: 7% e. Subtest V. Audiology and Hearing: i. AU average performance: 77% correct ii. National Mid 50% range: Not calculated iii. National Mid 50% midpoint: Not calculated f. Subtest VI. Clinical Management i. AU average performance: 74% correct ii. National Mid 50% range: 63% 77% iii. National Mid 50% midpoint: 69% g. Subtest VII. Professional Issues, Psychometrics, & Research i. AU average performance: 8% correct ii. National Mid 50% range: 7% 88% iii. National Mid 50% midpoint: 80% Assessment: Use of Findings for Improvement A. Our pass rate: 00% exceeded expectations of 80%. B. In AY 03 04, we chose to address two specific subtest topic areas where the AU performance average for AY 0 03 fell slightly below the midpoint of the middle 50% range. In AY 03 04 the average performance fell within the middle 50% range. Subtest VI. Clinical Management Subtest AY 0 3 AY 03 4 AU average performance: 64% correct 74% National Mid 50% range: 55% 7% 63 77% National Mid 50% midpoint: 63% 69% Subtest VII. Professional Issues, Psychometrics, and Research AU average performance: 7% correct 8% National Mid 50% range: 6% 8% 7% 88% National Mid 50% midpoint: 7.5% 80% C. All the AU average performances fell within the National Mid 50% range, however, the AU average for Subtest VI. Neurogenics (7%) fell just below the calculated National Mid 50% midpoint of 7%. At our Annual Faculty Retreat (August 7, 04) we decided to track the performance in AY 04 055 to see if this continued. Any Additional Comments? None
II. Clinical Evaluation Skill Development Expected Outcome: Students will demonstrate the ability to administer and score diagnostic tests in communication disorders and to determine accurate diagnoses along with providing an appropriate prognosis. Assessment: Method Three essential evaluation components will be monitored across the fourth semester of clinical practicum, when the students are assigned to conduct evaluations of clients in the AU Speech and Hearing Clinic. We are defining demonstration of ability as all fourth semester graduate students meeting expectations and 50% would exceed expectations. We will track the performance of Graduate Student Clinicians who are enrolled in CMDS 7500 Clinical Practicum, at Level 4: (fourth and final semester of CMDS 7500 Clinical Practicum and semester before enrolling in CMDS 7940 Field Experience) who have been assigned to evaluate an AUSHC client s communication disorders. [PLEASE SEE ATTACHMENT: Minimum Performance Requirements: Level 4: Fourth Semester Graduate]. As result of last year s data we removed the Key Skill: Determining Diagnosis/Prognosis from our goals and replaced it with tracking another challenging and critical skill, taking a Case History. The focus will be upon three challenging and critical performance skills needed to conduct such client evaluations: Case History, Test Administration, and Test Scoring. [PLEASE SEE ATTACHMENTS: Clinical Grading Form: Evaluation AND Explanation of Clinic Supervision (Evaluation) and Grading Sheet.] Certified Clinical Supervisors and Clinical faculty in the AU Speech and Hearing Clinic will monitor the students assigned to evaluate their clients using the following Rating Scale: 0 = Unsatisfactory = Needs Improvement = Meets Expectations 3 = Exceeds Expectations Assessment: Findings Note: Clinical evaluation skill development was tracked for 4 Level 4 graduate clinicians for the Fall Semester 03: Key Skill: Case History: 57% met expectations and 43% exceeded expectations [target = 50% exceed]. The average rating was.4. 3
Key Skill: Test Administration: 7% met expectations and 9% exceeded expectations [target = 50% exceed]. The average rating was.3. Key Skill: Test Scoring: 64% met expectations and 36% exceeded expectations [target = 50% exceed]. The average rating was.4. Assessment: Use of Findings for Improvement New Goal 03 4: Key Skill: Case History: Target 50% exceed expectations; Achieved 43% Key Skill: Test Administration: Target 50% exceed expectations; Achieved 9% 03 4 [3% 0 3] Key Skill: Test Scoring: Target 50% exceed expectations; Achieved 36% 03 4 [38% 0 3] Plan: Discuss performance in Annual Faculty Retreat Part : August 7, 04. Request that clinical and academic faculty teaching graduate courses in speech language pathology emphasize the importance of taking a complete case history, administering tests correctly and accuracy in test scoring in the evaluation of each specific disorder in their courses and stress the necessity of rehearsal practice with peers prior to the actual diagnostic session. Anticipate additional emphasis on test administration and test scoring be placed on the projects assigned in CMDS 7500. Any Additional Comments? None III. Exit Interview/Survey Data: Expected Outcome: Students will be satisfied with their professional preparation in the academic program, clinical program and internship setting. Assessment: Method: (PLEASE SEE ATTACHMENT: End of Program Survey in Speech-Language Pathology: Master s Degree Program) Student satisfaction is defined here as 50% of the students will select a rating of 5 = Very Adequate for to each skill area surveyed. We considered 45% as rounding up to 50% for this year. This is only the second year we have set such a high standard on our student satisfaction ratings. Previous years combined ratings of 4 & 5. 4
As a result of your entire graduate program (including academic, clinical and internship portions) how well prepared do you feel to perform each of the following skills? Conduct screening and other prevention procedures Collect case history information and integrate information from clients/patients,family, caregivers,teachers,relevant others, and other professionals Select and administer appropriate evaluation procedures, such as behavioral observations, nonstandardized and standardized tests and procedures Adapt evaluation procedures to meet client/patient needs Interpret, integrate, and synthesize all information to develop diagnoses and make appropriate recommendations for intervention Develop setting appropriate intervention plans with measurable and achievable goals that meet clients/patients needs. Collaborate with clients/patients and relevant others in the planning process Implement intervention plans(involve clients/patients and relevant others in the intervention process) Select or develop and use appropriate materials and instrumentation for prevention and intervention Measure and evaluate clients/patients performance and progress Modify intervention plans, strategies, materials, or instrumentation as appropriate to meet the needs of clients/patients Collaborate with other professionals in case management Ver y Inadequate Inadequate 3 Neutral 4 Adequate 5 Very adequate 55% 45% 5% 3% 64% 5% 4% 55% 55% 45% 9% 64% 7% 5% 59% 36% 55% 45% 45% 55% 50% 50% 50% 50% 5% 55% 4% 5
Provide counseling regarding communication and swallowing disorders to clients/patients, family, caregivers, and relevant others 8% 50% 3% Assessment: Use of Findings for Improvement: 7% [target = 50%] Rated their ability to Interpret, integrate, and synthesize all information to develop diagnoses and make appropriate recommendations for intervention as 5 = Very Adequate. PLAN: Discuss performance in Annual Faculty Retreat Part : August 7, 04. Note: Level 4 clinicians typically meet or exceed their supervisors expectations in their ability to determine the diagnosis/prognosis. Perhaps boosting their confidence in this challenging skill across all the entire program of coursework and clinical experience will improve their self perception. 36% [target = 50%] Rated their ability to Develop setting appropriate intervention plans with measurable and achievable goals that meet clients/patients needs. Collaborate with clients/patients and relevant others in the planning process as 5 = Very Adequate. PLAN: Discuss performance in Annual Faculty Retreat Part : August 7, 04. Track goal setting skills within treatment and encourage faculty to offer assignments within the various specific disorder courses that require goal writing to enhance the graduate students competence and confidence in goal writing. 4% [target = 50%] Rated their ability to Collaborate with other professionals in case management as 5 = Very Adequate. PLAN: Discuss performance in Annual Faculty Retreat Part : August 7, 04. Two projects are already underway to further our collaboration with nursing students in goal writing and in patient care plans. In addition, continue to request that off campus placement sites to encourage attendance in team conferencing/planning meetings where possible at those sites. 3% [target = 50%] Rate their ability to Provide counseling regarding communication and swallowing disorders to clients/patients, family, caregivers, and relevant others as 5 = Very Adequate. PLAN: Discuss performance in Annual Faculty Retreat Part : August 7, 04. Reinforce topics areas already provided in CMDS 7970 Special Topics Seminar Counseling and ask faculty teaching courses in communication disorders to highlight the aspects of counseling and role play activities in their classes to provide practice in counseling scenarios. Any Additional Comments? None 6
FOUR ATTACHMENTS Minimum Performance Requirements Clinical Grading Form Evaluation Explanation of Clinic Supervision (Evaluation) and Grading Sheet End of Program Survey in Speech-Language Pathology: Master s Degree Program 7
Minimum Performance Requirements: Undergraduate and First Semester Graduate Level The student will identify and gather specific information in client files, text books, administrative manuals and treatment materials as instructed by the supervisor. Second Semester Graduate Level The student will identify and gather information as specified in clinical competency I (CC I). The student will prepare an analysis of treatment and/or diagnostic options relevant to the client s needs for presentation to the supervisor and implement accordingly. Third Semester Graduate Level 3 Students who have not met expectations for CC I & CC II with a minimum of 75% accuracy or above will not be assigned to an off-campus clinical placement and/or diagnostics. The student will analyze, implement, and modify with assistance information and treatment options. (May be considered off-campus and/or diagnostics assignments). Fourth Semester Graduate Level 4 The student will perform on-going analysis and assessment of all elements of therapy in light of client progress and recommend timely and appropriate modifications to the supervisor. Students must pass level 4 with a minimum of 75% accuracy in order to participate in an externship. Fifth Semester Graduate (field experience) Level 5 The student will independently implement all modifications to therapy approved by the supervisor according to established professional standards of practice. Supervision will be provided according to ASHA supervisory requirements (5% for Tx/50% for Dx). Students must pass this level with a minimum of 75% accuracy in order to pass the externship. Note: Students may perform functions at any level with the assistance of the clinical supervisor, but are expected to do so independently at the level described for each semester in clinic. Ability to perform at levels that exceed or do not meet minimum requirements will be reflected in the student s grade. 8
Students must pass Levels -3 with a minimum of 75% accuracy in order to advance levels. Students must pass Level 4 with 75% accuracy in order to be eligible for an externship. Students must pass Level 5 with 75% accuracy in order to satisfactorily pass the externship. CLINIC SUPERVISION GRADING FORM DIAGNOSTICS Student Level Clinical Instructor Client Semester EVALUATION Unsatisfactory Needs Improvement Meets Expectations Exceeds Expectations Not Applicabl e. Pre evaluation Planning/Test 0 3 NA Selection based on history. Case history: Student reviews 0 3 NA appropriate information and formulates appropriate follow up questions with interview executed effectively. 3. Test administration/ 0 3 NA Equipment Use 4. Patient Interaction 0 3 NA 5. Test Scoring /Test 0 3 NA Interpretation 6. Diagnosis/Prognosis 0 3 NA 7. Client Summary and Ability to Communicate summary to Client/Caregiver 8. Appropriate recommendations and referrals 0 3 NA 0 3 NA 9
9. Report preparation appropriate to facility standards: Pertinent, accurate 0 3 NA Organized 0 3 NA Professional writing style 0 3 NA Spelling, grammar 0 3 NA Appearance 0 3 NA Report 0 3 NA Average 0. Utilizes feedback from clinical 0 3 NA instructor, team members and other professionals. TOTAL x = Please give information on any score rated less than (meets expectations): PROFESSIONAL CHARACTERISTICS Unsatisfactory Needs Improvement Meets Expectations. Meets client on time 0. Appropriate attire for situation 0 3. Documentation (reports, plans) submitted on time 0 4. Complies with billing policies 0 5. Keeps working files in order 0 6. Maintains accurate records for ASHA hours/daily log 0 7. Attends meetings on time 0 8. Complies with Universal Precautions 0 9. Maintains equipment 0 0. Maintains client confidentiality 0 TOTAL x = 0
Please give information for any score rated less than (meets expectations) GRAND TOTAL = GRADE Extra Credit (3 points maximum, point per activity NSSLHA Attendance at the NSSLHA Symposium or other conference Extra Clinical duties Interaction with other professionals Other Fund Raising Explanation of Clinic Supervision Evaluation and Grading Sheet Evaluation Complex Client Material Development Extensive client/family education. Pre-Evaluation Planning: The clinician will review the client folder and compose appropriate interview questions prior to the initial meeting with the supervisor. The clinician will be able to present the proposed assessment plan including standardized testing, material set-up and equipment to be used (if needed) to the supervisor. Test selection should be based on normative data and the reported concern. The clinician will contact the client and follow-up with any necessary questions in a professional manner.. Case History: The clinician will discuss the case history with the client and will ask additional questions during the interview when appropriate. 3. Test Administration/Equipment Use: The clinician will administer tests following test criteria and protocol in an efficient and organized manner. The clinician will be prepared to conduct an informal assessment in addition to the formal assessment. The clinician will be familiar with the operation of any equipment that may be necessary for the evaluation. 4. Client Interaction: The clinician will attempt to establish rapport with the client and caregiver, as well as demonstrating effective communication skills and counseling when needed. The clinician will conduct the evaluation in a professional yet sensitive manner with an awareness of cultural diversity. The clinician will inform the client and caregiver of the assessment procedures using clear and concise terminology. 5. Test Scoring and Interpretation: The clinician will score tests following test criteria and protocol while demonstrating comprehension of the results. 6. Diagnosis/Prognosis: Based on formal and informal assessment, the clinician is able to assess severity and type of disorder. Based on formal/informal assessment, case history information, and diagnosis, the clinician is able to make a reasonable prediction of the client s rehabilitative potential. 7. Client Summary and Ability to Communicate results to Client/Caregiver: The clinician will demonstrate the ability to relate diagnosis and prognosis to the client and or caregiver using clear
and concise terminology. The clinician will attempt to respond to client/caregiver questions pertaining to the evaluation with the assistance of the supervisor as needed. 8. Appropriate recommendations (goals) and referrals: The clinician will make appropriate recommendations regarding the need for additional evaluation, treatment or a referral to other professionals. The clinician will develop appropriate goals based on assessment results. The clinician will follow-up with the client and/or caregiver education when appropriate, as well as referrals to other professionals. 9. Report preparation appropriate to facility standards: The clinician will compose an evaluation report and other required paperwork following the evaluation. The report should be written and submitted according to facility requirements. 0. Utilizes Feedback: The clinician will utilize supervisor, team member, and allied professional feedback and formulate questions regarding information that is unclear. The clinician will incorporate the supervisor s suggestions made during the pre-evaluation planning session and during the evaluation. End of Program Survey in Speech-Language Pathology Master s Degree Program In order to ensure that we are providing high quality academic and clinical preparation, the program in speech-language pathology uses several mechanisms for ongoing and systematic evaluation. This anonymous survey of graduate students as they complete the program is an important part of that evaluation process. Please rate the adequacy of your academic preparation in each of the following areas Articulation/Phonology Fluency Voice and resonance Receptive and expressive language: Children Receptive and expressive language: Adults Hearing, including the impact on speech and language Swallowing Cognitive aspects of communication (attention, memory, sequencing, problem-solving, executive functioning) Social aspects of communication (including challenging behavior, ineffective social skills, lack of communication opportunities) Communication modalities (including oral, manual, augmentative, and alternative communication techniques and assistive technologies) Standards of ethical conduct including knowledge of, appreciation for, and ability to interpret the ASHA Code Very Inadequate Inadequate 3 Neutral 4 Adequate 5 Very Adequate
of Ethics. Principles of basic and applied research and research design. Accessing sources of research information and relating research to clinical practice. Knowledge of contemporary professional issues including professional practice, academic program accreditation standards, ASHA practice policies and guidelines, and reimbursement procedures. Knowledge about certification, specialty recognition, licensure, and other relevant professional credentials. Please rate the quality of the advising you received from your departmental graduate advisor 3
Please rate the adequacy of your clinical experience with each of the following disorder types. Be sure to consider your entire clinical experience (i.e. on-campus, off-campus and internship). Articulation/Phonology Motor speech disorders (dysarthria, apraxia etc.) Fluency Voice and resonance Receptive and expressive language: Children Receptive and expressive language: Adults Hearing, including the impact on speech and language Swallowing Cognitive aspects of communication (attention, memory, sequencing, problem-solving, executive functioning) Very Inadequate Inadequate 3 Neutral 4 Adequate 5 Very Adequate As a result of your entire graduate program (including academic, clinical and internship portions) how well prepared do you feel to perform each of the following skills? Conduct screening and other prevention procedures Collect case history information and integrate information from clients/patients, family, caregivers, teachers, relevant others, and other professionals. Select and administer appropriate evaluation procedures, such as behavioral observations, non-standardized and standardized tests, and instrumental procedures Adapt evaluation procedures to meet client/patient needs. Interpret, integrate, and synthesize all information to develop diagnoses and make appropriate recommendations for intervention. Develop setting-appropriate intervention plans with measurable and achievable goals that meet clients'/patients' needs. Collaborate with clients/patients and relevant others in the planning process. Implement intervention plans (involve clients/patients and relevant others in the intervention process). Select or develop and use appropriate materials and instrumentation for prevention and intervention Measure and evaluate clients'/patients' performance and progress. Modify intervention plans, strategies, materials, or instrumentation as appropriate to meet the needs of clients/patients. Collaborate with other professionals in case management. Provide counseling regarding communication and swallowing disorders to clients/patients, family, caregivers, and relevant others. 4