NATIONAL. Response. RFT(s) Section. Responses have been. Research Study. Laptops/PCs are supported) What. Services. All. assessments using the

Transcription

1 CONTRACTS IN RELATION TO NATIONAL ASSESSMENT AND SURVEYS ONLINE PROGRAM (NASOP) QUESTIONS FOR ACARA Responses have been provided to the below questions in relation to each of the following National Assessment and Surveys Online Program (NASOP) Requests for Tender (RFT): 2012/06 NAP Civics And Citizenship Online Trial and Main 2012/07 NAP-CC Online Pilot 2012/08 NASOP - Pilot (Reading, Language Conventions, Numeracy) 2012/09 NASOP - Pilot (Writing) RFT(s) Section RFT provision Question Response Part A: The Schedule Contract details All 2 Services What are the technical requirements that school IT systems must have to be able to administer the assessments using the online platform developed by ESA? Laptops/PCs are supported (ipads are not Mac OS X MS Windows XP Firefox 3.6 Safari 4.0 MS Internett Explorer 7.0 Adobe Flash Player 10 Shockwave Player 11 Java 1.6 QuickTime PDF plug in 5.0 Pop ups enabled Screen resolution 1048 x 768 All 2 Services Does the ESA platform allow administration on the range of devices such as PCs, Notebooks, Apple and tablets (eg. ipad)? Laptops/PCs are supported (ipads are not

2 All 2 Services Is it intended that schools will be able to have students take the items on the range of hardware devices including, for example PCs, Notebooks, Apple, Tablets (like the ipad)? Laptops/PCs are supported (ipads are not 2012/07 NAP CC Online Pilot 2012/ /09 (Writing) 2012/06 NAP Civics And Citizenship Online Trial and Main 2 Services Can the registration of students in the ESA system be done by central batch importing, for example, a class list of students? 2 Services Can the registration of students in the ESA system be done by central batch importing, for example, a class list of students? No, this functionality is not possible for this release. It is envisaged that this functionality would be available; however, it is not yet confirmed. All 2 Services Please can you describe the full nature of how items need to be entered into the ESA system and the associated quality assurance processes that are required of a contractor? Contractors are not required to load test items in the system. Contractors would be required to create tests using the items available in the Item bank and quality assure the test instrument. They will also be required to create student accounts in the system.

3 All 2 Services What is the format for the training of invigilators on the delivery platform that ESA will perform and for how many invigilators and where? ESA will provide System User training for up to 20 contract staff. ESA would prefer that the contractor uses this as a train the trainer model and conducts subsequent training on their own. Training would be provided in Melbourne and computers during training will be available. All 2 Services Who will produce invigilator training materials for test administration on the ESA system? Is it the responsibility of ESA or the contractor? ESA would produce only the train the trainer material on how to use the system. It would be contractor s responsibility to produce the invigilator training and associated material. 2012/07 NAP CC Online Pilot 2012/ /09 (Writing) 2012/06 NAP Civics And Citizenship Online Trial and Main 2 Services Does the ESA platform include automated scoring of multiple choice and constructed response items? 2 Services Does the ESA platform include automated scoring of multiple choice and constructed response items? No. The responses will be provided., but scoring and collating the data is the contractor s responsibility and outside the system provided by ESA. Tenderers should assume that only multiple choice would be scored by the new system. However, additional functionality for automated scoring of constructed response items may be added at a later stage.

4 : NAP : NAP : NAP 2 Services refers to There will be two test forms in total for this study, 40 items in total for each test form, comprising two fixed order clusters of items that contain 20 horizontal items, and 20 vertical items. The testing modes (paper based and computerbased) will be embedded into the design of the forms and there will be a counterbalanced administration of the tests. Could you please elaborate on the above? If there are only two forms, how are they linked vertically and horizontally? If both tests are the same level and there are horizontal links, does that mean there is no vertical linking? If the two tests are different levels then where would be the horizontal link? Are there two forms for each year level? 2 Services refers to The design will yield information from 400 students per item. The test will be delivered to 800 students in each Year Level (6 and 10). Could you please provide clarification on how this would be achieved? If each of 1600 students take one of four disjoint tests then there would be 400 responses for each item but if there are common items, it is assumed there must be more responses per item 2 Services refers to marking and coding by hand Could you please clarify this requirement? Does this preclude onscreen marking? There is one test per year level, comprising two forms. Table 1 on page 24 of the RFT shows the possible sequences of test form administration for two students in a year level. Each test will be linked horizontally to the common historical item pool through the 20 horizontal items. The two tests will be vertically linked by the set of items, common to both years 6 and 10 (i.e., the vertical items). The design will yield information from 400 students per test form. The test will be delivered to 800 students in each year level. No, onscreen marking is acceptable.

5 : NAP 3 Required Contract Material 3.4 Technical report refers to The Contractor will provide a fully documented database, data sets and technical report describing the analysis procedures and the outcome of the pilot study, including final recommendations regarding the administration of the NAP CC test items in different modes. Is there a preferred format for the database? The preferred format for data sets is fixed ASCII text file format. In addition, where suitable, a data file format used by mainstream statistical software (SPSS or similar) can be supplied. It is also expected that the database will contain all relevant command scripts used to run Rasch Model and other statistical analyses Services 2.4 refers to The successful Contractor will be required to: In collaboration with ESA, prepare computerbased assessment instruments, using existing items that will be provided by ACARA. Could you please elaborate on what exactly this requirement entails for the Contractor, i.e. the preparation of computer based assessment instruments? 2 Services refers to marking and coding by hand Could you please clarify this requirement? Does this preclude onscreen marking The Contractor will be required to provide assistance to ESA in activities such as rendering paper based items onto an online form, and ensuring that appropriate tests are available/ready for administration to students participating in the research study. No, onscreen marking is acceptable.