WebXam 2014-2015 End-of-Course Field Testing Learn about Ohio Career-Technical Education technical tests being field tested during the 2014-2015 school year. Know the distinction between continuation field testing and new field testing. Revised April 8, 2015
Purpose This document explains procedures for End-of-Course (EOC) field testing, an important part of psychometric quality assurance for technical testing programs. The purpose of field testing is to examine both individual items and entire tests. This quality assurance process helps CETE staff to select the best items to create operational test forms. Field test data is also used to adjust cutoff scores when finalizing operational test forms. Field testing may be either continuation (year 2) or new (year 1). Important Reminders Field-tested Pathway Tests now appear on the Ohio CTE Assessment Matrix as required for juniors who have received instruction in that content. Field testing closes earlier than operational testing so quality assurance can be completed before the end of the year. Once final forms are created, final scores are generated to provide accountability data to local districts in the same year as field testing. Field Tested Pathway Tests Available: October 2014 through May 1, 2015 Note that some of the field tested Pathway Tests from last year needed additional data, so these are being carried over this year as continuation field testing. During 2013-2014 both secondary and postsecondary instructors from across Ohio participated in itemwriting workshops at CETE at The Ohio State University. Preliminary EOC tests for selected pathways listed below were created at these workshops. Beginning in October 2014, these items will be field tested with students as part of the process for creating two forms of 40 items. Agricultural and Environmental Systems 46MD Biotechnology Human Services 21MC Cosmetology and Barbering Information Technology 81MD Information Support and Services 82MD Network Systems 83MD Programming and Software Development 84MD Interactive Media Transportation Systems Career Field 09MC Air Transportation 10MC Ground Transportation Page 2
Information for All Pathways Transition to Courses The new field-tested Pathway Tests for 2014-2015 will be replacing older tests that were end-of-program or modular tests. The new EOC tests are intended to be administered upon the completion of courses. For more information on the Ohio CTE move from a program-based to a course-based system, see this web page: http://ohiocareertech.wordpress.com/2013/08/17/ohio-career-tech-goes-from-program-basedto-course-based-system/ Test Administration CTE Pathway Tests, including those being field tested, are administered at no cost to Ohio local education agencies via WebXam (www.webxam.org), the online test delivery system maintained by CETE. Tests are not available in paper-and-pencil format. District personnel with WebXam accounts can set up tests. Help is available at the WebXam Information Resources web page at https://www.webxam.org/asp/documentation/info_docs.asp. For more information about CTE Testing, contact WebXam Support by email at webxam@osu.edu or by phone at 614-292-4988. NOTE: Field tested EOC Tests will be available online until the cutoff date of May 1, 2015 so rescoring can be accomplished (this is about one month earlier than the end of regular post-testing). Availability Tests undergoing field-testing are available to all students enrolled in the corresponding career-technical education programs. The larger the number of examinees participating in the field test, the more reliable the results; therefore, all teachers and school districts with relevant programs are encouraged to participate. Students should complete EOC Tests only if 1. the student is enrolled in the corresponding course and has completed the related coursework related to that EOC Test (post-test usage), 2. or the student will be taking the coursework in a future school year, and the district is using the EOC Test as a pretest. To determine whether students need to take this new Pathway Tests or an old test, please consult the Ohio Department of Education s Career Technical Assessment Matrix at http://education.ohio.gov/topics/career-tech/cte-assessments-and-industry-credentials. Student Scores Scores for Pathway Tests field tested in 2014-2015 will be reportable through Information Technology Centers (ITC) to local districts for transmission to EMIS, Ohio s statewide data-collection system. However, test results will be analyzed and scores adjusted before reporting by CETE at the end of the school year. During the field-testing period, students will only receive a raw score (# right, %) for each EOC Test taken. Students will not receive a pass/fail indication for field-tests when they test. Field testing (and only field testing) will end on the cutoff date of May 1, 2015. At this time, CETE staff and ODE pathway program specialists will analyze the field-test results. Poorly performing items will be dropped, and final test forms will be selected. Every student s test result will then be rescored based on the final set of items chosen. The percentage score may increase or decrease from the raw score shown to the student during field testing. By May 29, 2015 CETE staff will provide final scores with pass/fail indications for all field- Page 3
tested tests. Districts will be able to run reports on the WebXam website, and field-test results will be included in the end-of-year score output provided to ITCs for reporting purposes. Test Structure In general, each EOC Test contains items at two levels of challenge: C1 and C2. Approximately 30 percent of items will be at the C2 level (higher in Bloom s taxonomy mainly application, analysis, or evaluation). The remaining items are at the C1 level (knowledge and comprehension). In addition, approximately 30 percent of items for each test are scenario-based. A scenario is a short situation, story, or passage of text that provides information needed to answer several associated items. Graphics (tables, charts, figures, maps) may be included as well to provide information needed to answer the test item. Each EOC Test corresponds to a course as defined by the Ohio Department of Education. Specific information regarding each individual pathway test EOC Tests to administer, EOC Tests included in each Pathway Test, and number of items in each EOC Test follows in the next section. To find the competencies in the Ohio Career Field Technical Content Standards covered by each EOC Test, go to the WebXam Information Resources page at http://www.webxam.org/asp/documentation/info_docs.asp. Students are expected to test for each course in which they are enrolled. They are not required to take all EOC Tests available for the entire program. Because programs vary in length and the minimum number of courses approved for a program is four, CTE Concentrators cannot take less than three EOC Tests and still receive a Pathway Test score. Students with an insufficient number of EOC Test scores will be reported with an Incomplete test score by WebXam. Contact WebXam Email: webxam@osu.edu Phone: 614-292-4988 Web: www.webxam.org and http://news.webxam.org Page 4
NEW Field Testing Agricultural and Environmental Systems Career Field 46MD Biotechnology and Food 011010 Science and Technology of Food 83 011015 Food Marketing and Research 82 011020 Meat Science and Technology 86 011030 Applications of Food Science and Safety 84 012010 Animal and Plant Biotechnology 87 012015 Principles and Practices of Bioscience Technologies 86 012020 Genetics of Plants and Animals 84 012025 Bioresearch 84 Students taking the 46MD Pathway Test will also have access to other EOC Tests in the Agricultural and Environmental Systems Career Field that have already been field tested and published in previous years. For more information, see FY2015 Available Ohio CTE Pathway Tests on WebXam at https://www.webxam.org/asp/documentation/octca_assessments.asp. Engineering and Science Technologies Career Field 26MC Engineering Design 175011 DC and AC Electronic Circuits 104 175012 Analog Based Electronic Devices 95 Health Science Career Field 25MC Allied Health and Nursing 072001 Health Science and Technology 97 072076 Dental Radiography 97 072155 Medical and Dental Office Technology 96 Page 5
Human Services Career Field 21MC Cosmetology and Barbering Pathway Test 174000 Hair Microbiology and Infection Control 83 174005 Trichology 89 174010 Hair Fundamentals of Cutting and Styling 85 174015 Hair Advanced Cutting and Styling 84 174020 Hair Fundamentals of Chemical Services 85 174030 Hair Advanced Chemical Services 85 174035 Hand and Foot Treatment Fundamentals and Enhancements 85 174040 Skin Care Fundamentals and Enhancements 86 174045 Salon Operations and Communications 84 Information Technology Career Field 81MD Information Support and Services Pathway Test 145005 Information Technology 81 145010 Web Design 80 145020 Computer and Mobile Applications 85 145025 Computer Hardware 81 145030 Computer Software 79 145035 Networking 83 145040 Network Operating Systems 86 145060 Programming 86 145065 Object Oriented Programming 85 145070 Visual Programming 86 145075 Systems Analysis and Design 84 145080 Database Administration 84 145085 Database Applications Development 88 145110 Video and Sound 86 Page 6
82MD Network Systems Pathway Test 145005 Information Technology 81 145010 Web Design 80 145025 Computer Hardware 81 145030 Computer Software 79 145035 Networking 83 145040 Network Operating Systems 86 145045 Network Management 83 145050 Network Security 85 145055 Routing and Switching 85 145060 Programming 86 145065 Object Oriented Programming 85 145070 Visual Programming 86 145080 Database Administration 84 145110 Video and Sound 86 83MD Programming and Software Development Pathway Test 145005 Information Technology 81 145010 Web Design 80 145020 Computer and Mobile Applications 85 145060 Programming 86 145065 Object Oriented Programming 85 145070 Visual Programming 86 145075 Systems Analysis and Design 84 145080 Database Administration 84 145085 Database Applications Development 88 145090 Game Design 85 145105 Multimedia and Image Management Techniques 84 145110 Video and Sound 86 145115 Animation 84 145120 3D Techniques 86 145125 Interactive Application Development 85 Page 7
84MD Interactive Media Pathway Test 145005 Information Technology 81 145010 Web Design 80 145020 Computer and Mobile Applications 85 145060 Programming 86 145065 Object Oriented Programming 85 145070 Visual Programming 86 145075 Systems Analysis and Design 84 145090 Game Design 85 145095 Design Techniques 84 145100 Creating and Editing Digital Graphics 85 145105 Multimedia and Image Management Techniques 84 145110 Video and Sound 86 145115 Animation 84 145120 3D Techniques 86 145125 Interactive Application Development 85 Transportation Systems Career Field 09MC Air Transportation Pathway Test 177013 Aviation 75 177014 Aviation Maintenance General 84 177015 Aviation Structure and Design 84 177016 Aviation Airframe Systems and Components 82 177017 Aviation Powerplant Theory and Maintenance 85 177018 Aviation Powerplant Systems and Components 84 177019 Aviation Meteorology 81 177020 Aviation Airport Management 81 177021 Aviation Pilot Training 79 177022 Aviation Air Traffic Control 80 Page 8
10MC Ground Transportation Pathway Test 177000 Ground Transportation Maintenance 86 177001 Ground Transportation Engine and Power Train 84 177002 Ground Transportation Electrical/Electronics 83 177003 Automotive Braking, Suspension and Steering Systems (Undercarriage Systems) 85 177004 Ground Transportation HVAC 91 177005 Truck Braking, Suspension and Steering Systems (Undercarriage Systems) 84 177006 Automotive Engine Performance 85 177007 Truck Diesel Engines 85 177008 Sports/Recreational Power Systems 84 177009 Collision Electrical and Mechanical Systems 81 177010 Collision Structural Inspection and Repair 85 177011 Collision Nonstructural Inspection and Repair 92 177012 Collision Painting and Refinishing 77 Continuation Field Testing This part of the document lists only EOC Tests in Continuation Field Testing: i.e., EOC Tests that are being field tested for a second year. Other EOC Tests are available for these Pathway Tests. For a complete listing, see FY2015 Available Ohio CTE Pathway Tests on WebXam at https://www.webxam.org/asp/documentation/octca_assessments.asp. Construction Technologies Career Field 15MC Construction Design and Management Pathway Test 178002 Mechanical, Electrical and Plumbing Systems 87 178007 Construction Electrical Systems 85 178012 Heating and Cooling Systems 80 178019 Plan Reading 84 178020 Architecture Design Structural and Mechanical/Electrical/Plumbing 86 178021 Architecture Design Site and Foundation Plans 82 178022 Construction Management 85 178023 Remodeling/Renovation 86 178024 Facility and Building Maintenance 93 178025 Custodial Services 82 178026 Heavy Equipment Operations 84 178027 Construction Site Preparation 75 178028 Interior Design 90 Page 9
27MC Mechanical, Electrical and Plumbing Pathway Test 178002 Mechanical, Electrical and Plumbing Systems 87 178007 Construction Electrical Systems 85 178009 Commercial and Industrial Construction Electrical Systems 83 178010 Plumbing and Pipefitting Systems 92 178011 Residential and Commercial Plumbing Systems 84 178012 Heating and Cooling Systems 80 178013 HVAC Refrigeration 80 178014 Sheet Metal 80 178015 Telecommunications/Low Voltage Systems 89 178016 Alternative Power Generation Systems 82 178017 Powerline/Hi-Voltage Power Transmission 91 178019 Plan Reading 84 178022 Construction Management 85 178024 Facility and Building Maintenance 93 45MC Structural Systems Pathway Test 178002 Mechanical, Electrical and Plumbing Systems 87 178005 Masonry 85 178006 Concrete and Residential Masonry 85 178007 Construction Electrical Systems 85 178011 Residential and Commercial Plumbing Systems 84 178019 Plan Reading 84 178022 Construction Management 85 178023 Remodeling/Renovation 86 178026 Heavy Equipment Operations 84 178027 Construction Site Preparation 75 Engineering and Science Technologies Career Field 26MC Engineering Design Pathway Test 175004 Robotics 85 175006 Computer Integrated Manufacturing 79 175008 Mechanisms and Drives 83 178019 Plan Reading 84 178020 Architecture Design Structural and Mechanical/Electrical/Plumbing 86 178021 Architecture Design - Site and Foundation Plans 82 Page 10
Health Science Career Field 22MC Health Information Management Pathway Test 072135 Health Information Technology 86 072140 Health Information Management 85 072145 Billing and Coding 79 23MC Exercise Science and Sports Medicine Pathway Test 072005 Bio-Statistics in Exercise Science and Sports Medicine 79 072020 Fitness Evaluation and Assessment 67 072030 Sports Exercise Psychology 82 25MC Allied Health and Nursing Pathway Test 072045 Human Pathophysiology 81 072065 Mental Health 81 072070 Surgical Support 83 072075 Dental Technology 82 072080 Oral Diagnosis and Treatment Planning 85 072085 Pharmacology 84 072090 Respiratory Technology 78 072095 Opticianry and Vision Care 89 47MC Medical Bioscience Pathway Test 072045 Human Pathophysiology 81 072120 Biochemistry of Health 85 072125 Biotechnology for Health and Disease 85 Page 11
Manufacturing Career Field 28MC Manufacturing Operations Pathway Test 175004 Robotics 85 175006 Computer Integrated Manufacturing 79 176007 Computer Numerical Control Technology with Industrial Mills and Lathes 86 Revision History 04/08/2015: Added five additional field tested EOC Tests in Engineering and Health, announced at http://news.webxam.org/2015/01/five-additional-tests-published/ Page 12