WebXam End-of-Course Field Testing

Transcription

1 WebXam End-of-Course Field Testing Learn about Ohio Career-Technical Education technical tests being field tested during the school year. Know the distinction between continuation field testing and new field testing. Revised April 8, 2015

2 Purpose This document explains procedures for End-of-Course (EOC) field testing, an important part of psychometric quality assurance for technical testing programs. The purpose of field testing is to examine both individual items and entire tests. This quality assurance process helps CETE staff to select the best items to create operational test forms. Field test data is also used to adjust cutoff scores when finalizing operational test forms. Field testing may be either continuation (year 2) or new (year 1). Important Reminders Field-tested Pathway Tests now appear on the Ohio CTE Assessment Matrix as required for juniors who have received instruction in that content. Field testing closes earlier than operational testing so quality assurance can be completed before the end of the year. Once final forms are created, final scores are generated to provide accountability data to local districts in the same year as field testing. Field Tested Pathway Tests Available: October 2014 through May 1, 2015 Note that some of the field tested Pathway Tests from last year needed additional data, so these are being carried over this year as continuation field testing. During both secondary and postsecondary instructors from across Ohio participated in itemwriting workshops at CETE at The Ohio State University. Preliminary EOC tests for selected pathways listed below were created at these workshops. Beginning in October 2014, these items will be field tested with students as part of the process for creating two forms of 40 items. Agricultural and Environmental Systems 46MD Biotechnology Human Services 21MC Cosmetology and Barbering Information Technology 81MD Information Support and Services 82MD Network Systems 83MD Programming and Software Development 84MD Interactive Media Transportation Systems Career Field 09MC Air Transportation 10MC Ground Transportation Page 2

3 Information for All Pathways Transition to Courses The new field-tested Pathway Tests for will be replacing older tests that were end-of-program or modular tests. The new EOC tests are intended to be administered upon the completion of courses. For more information on the Ohio CTE move from a program-based to a course-based system, see this web page: Test Administration CTE Pathway Tests, including those being field tested, are administered at no cost to Ohio local education agencies via WebXam ( the online test delivery system maintained by CETE. Tests are not available in paper-and-pencil format. District personnel with WebXam accounts can set up tests. Help is available at the WebXam Information Resources web page at For more information about CTE Testing, contact WebXam Support by at webxam@osu.edu or by phone at NOTE: Field tested EOC Tests will be available online until the cutoff date of May 1, 2015 so rescoring can be accomplished (this is about one month earlier than the end of regular post-testing). Availability Tests undergoing field-testing are available to all students enrolled in the corresponding career-technical education programs. The larger the number of examinees participating in the field test, the more reliable the results; therefore, all teachers and school districts with relevant programs are encouraged to participate. Students should complete EOC Tests only if 1. the student is enrolled in the corresponding course and has completed the related coursework related to that EOC Test (post-test usage), 2. or the student will be taking the coursework in a future school year, and the district is using the EOC Test as a pretest. To determine whether students need to take this new Pathway Tests or an old test, please consult the Ohio Department of Education s Career Technical Assessment Matrix at Student Scores Scores for Pathway Tests field tested in will be reportable through Information Technology Centers (ITC) to local districts for transmission to EMIS, Ohio s statewide data-collection system. However, test results will be analyzed and scores adjusted before reporting by CETE at the end of the school year. During the field-testing period, students will only receive a raw score (# right, %) for each EOC Test taken. Students will not receive a pass/fail indication for field-tests when they test. Field testing (and only field testing) will end on the cutoff date of May 1, At this time, CETE staff and ODE pathway program specialists will analyze the field-test results. Poorly performing items will be dropped, and final test forms will be selected. Every student s test result will then be rescored based on the final set of items chosen. The percentage score may increase or decrease from the raw score shown to the student during field testing. By May 29, 2015 CETE staff will provide final scores with pass/fail indications for all field- Page 3

4 tested tests. Districts will be able to run reports on the WebXam website, and field-test results will be included in the end-of-year score output provided to ITCs for reporting purposes. Test Structure In general, each EOC Test contains items at two levels of challenge: C1 and C2. Approximately 30 percent of items will be at the C2 level (higher in Bloom s taxonomy mainly application, analysis, or evaluation). The remaining items are at the C1 level (knowledge and comprehension). In addition, approximately 30 percent of items for each test are scenario-based. A scenario is a short situation, story, or passage of text that provides information needed to answer several associated items. Graphics (tables, charts, figures, maps) may be included as well to provide information needed to answer the test item. Each EOC Test corresponds to a course as defined by the Ohio Department of Education. Specific information regarding each individual pathway test EOC Tests to administer, EOC Tests included in each Pathway Test, and number of items in each EOC Test follows in the next section. To find the competencies in the Ohio Career Field Technical Content Standards covered by each EOC Test, go to the WebXam Information Resources page at Students are expected to test for each course in which they are enrolled. They are not required to take all EOC Tests available for the entire program. Because programs vary in length and the minimum number of courses approved for a program is four, CTE Concentrators cannot take less than three EOC Tests and still receive a Pathway Test score. Students with an insufficient number of EOC Test scores will be reported with an Incomplete test score by WebXam. Contact WebXam webxam@osu.edu Phone: Web: and Page 4

5 NEW Field Testing Agricultural and Environmental Systems Career Field 46MD Biotechnology and Food Science and Technology of Food Food Marketing and Research Meat Science and Technology Applications of Food Science and Safety Animal and Plant Biotechnology Principles and Practices of Bioscience Technologies Genetics of Plants and Animals Bioresearch 84 Students taking the 46MD Pathway Test will also have access to other EOC Tests in the Agricultural and Environmental Systems Career Field that have already been field tested and published in previous years. For more information, see FY2015 Available Ohio CTE Pathway Tests on WebXam at Engineering and Science Technologies Career Field 26MC Engineering Design DC and AC Electronic Circuits Analog Based Electronic Devices 95 Health Science Career Field 25MC Allied Health and Nursing Health Science and Technology Dental Radiography Medical and Dental Office Technology 96 Page 5

6 Human Services Career Field 21MC Cosmetology and Barbering Pathway Test Hair Microbiology and Infection Control Trichology Hair Fundamentals of Cutting and Styling Hair Advanced Cutting and Styling Hair Fundamentals of Chemical Services Hair Advanced Chemical Services Hand and Foot Treatment Fundamentals and Enhancements Skin Care Fundamentals and Enhancements Salon Operations and Communications 84 Information Technology Career Field 81MD Information Support and Services Pathway Test Information Technology Web Design Computer and Mobile Applications Computer Hardware Computer Software Networking Network Operating Systems Programming Object Oriented Programming Visual Programming Systems Analysis and Design Database Administration Database Applications Development Video and Sound 86 Page 6

7 82MD Network Systems Pathway Test Information Technology Web Design Computer Hardware Computer Software Networking Network Operating Systems Network Management Network Security Routing and Switching Programming Object Oriented Programming Visual Programming Database Administration Video and Sound 86 83MD Programming and Software Development Pathway Test Information Technology Web Design Computer and Mobile Applications Programming Object Oriented Programming Visual Programming Systems Analysis and Design Database Administration Database Applications Development Game Design Multimedia and Image Management Techniques Video and Sound Animation D Techniques Interactive Application Development 85 Page 7

8 84MD Interactive Media Pathway Test Information Technology Web Design Computer and Mobile Applications Programming Object Oriented Programming Visual Programming Systems Analysis and Design Game Design Design Techniques Creating and Editing Digital Graphics Multimedia and Image Management Techniques Video and Sound Animation D Techniques Interactive Application Development 85 Transportation Systems Career Field 09MC Air Transportation Pathway Test Aviation Aviation Maintenance General Aviation Structure and Design Aviation Airframe Systems and Components Aviation Powerplant Theory and Maintenance Aviation Powerplant Systems and Components Aviation Meteorology Aviation Airport Management Aviation Pilot Training Aviation Air Traffic Control 80 Page 8

9 10MC Ground Transportation Pathway Test Ground Transportation Maintenance Ground Transportation Engine and Power Train Ground Transportation Electrical/Electronics Automotive Braking, Suspension and Steering Systems (Undercarriage Systems) Ground Transportation HVAC Truck Braking, Suspension and Steering Systems (Undercarriage Systems) Automotive Engine Performance Truck Diesel Engines Sports/Recreational Power Systems Collision Electrical and Mechanical Systems Collision Structural Inspection and Repair Collision Nonstructural Inspection and Repair Collision Painting and Refinishing 77 Continuation Field Testing This part of the document lists only EOC Tests in Continuation Field Testing: i.e., EOC Tests that are being field tested for a second year. Other EOC Tests are available for these Pathway Tests. For a complete listing, see FY2015 Available Ohio CTE Pathway Tests on WebXam at Construction Technologies Career Field 15MC Construction Design and Management Pathway Test Mechanical, Electrical and Plumbing Systems Construction Electrical Systems Heating and Cooling Systems Plan Reading Architecture Design Structural and Mechanical/Electrical/Plumbing Architecture Design Site and Foundation Plans Construction Management Remodeling/Renovation Facility and Building Maintenance Custodial Services Heavy Equipment Operations Construction Site Preparation Interior Design 90 Page 9

10 27MC Mechanical, Electrical and Plumbing Pathway Test Mechanical, Electrical and Plumbing Systems Construction Electrical Systems Commercial and Industrial Construction Electrical Systems Plumbing and Pipefitting Systems Residential and Commercial Plumbing Systems Heating and Cooling Systems HVAC Refrigeration Sheet Metal Telecommunications/Low Voltage Systems Alternative Power Generation Systems Powerline/Hi-Voltage Power Transmission Plan Reading Construction Management Facility and Building Maintenance 93 45MC Structural Systems Pathway Test Mechanical, Electrical and Plumbing Systems Masonry Concrete and Residential Masonry Construction Electrical Systems Residential and Commercial Plumbing Systems Plan Reading Construction Management Remodeling/Renovation Heavy Equipment Operations Construction Site Preparation 75 Engineering and Science Technologies Career Field 26MC Engineering Design Pathway Test Robotics Computer Integrated Manufacturing Mechanisms and Drives Plan Reading Architecture Design Structural and Mechanical/Electrical/Plumbing Architecture Design - Site and Foundation Plans 82 Page 10

11 Health Science Career Field 22MC Health Information Management Pathway Test Health Information Technology Health Information Management Billing and Coding 79 23MC Exercise Science and Sports Medicine Pathway Test Bio-Statistics in Exercise Science and Sports Medicine Fitness Evaluation and Assessment Sports Exercise Psychology 82 25MC Allied Health and Nursing Pathway Test Human Pathophysiology Mental Health Surgical Support Dental Technology Oral Diagnosis and Treatment Planning Pharmacology Respiratory Technology Opticianry and Vision Care 89 47MC Medical Bioscience Pathway Test Human Pathophysiology Biochemistry of Health Biotechnology for Health and Disease 85 Page 11

12 Manufacturing Career Field 28MC Manufacturing Operations Pathway Test Robotics Computer Integrated Manufacturing Computer Numerical Control Technology with Industrial Mills and Lathes 86 Revision History 04/08/2015: Added five additional field tested EOC Tests in Engineering and Health, announced at Page 12