Accreditation of Initial Teacher Education Programs in Australia

Transcription

1 Accreditation of Initial Teacher Education Programs in Australia Standard 3: Program Entrants Year 12 Study Score Results as proxy indicators of personal literacy and numeracy July 2013

2 This study was a collaboration of the Australian Institute for Teaching and School Leadership (AITSL) and six members of the Australasian Curriculum, Assessment and Certification Authorities (ACACA) the body for CEOs of assessment agencies. This project was funded by the Australian Institute for Teaching and School Leadership. The Australian Institute for Teaching and School Leadership was formed to provide national leadership for the Commonwealth, state and territory governments in promoting excellence in the profession of teaching and school leadership with funding provided by the Australian Government. July

3 Australian Capital Territory The Accreditation of Initial Teacher Education Programs in Australia: Standards and Procedures requires that all entrants to initial teacher education will successfully demonstrate their capacity to engage effectively with a rigorous higher education program and to carry out the intellectual demands of teaching itself. To achieve this, it is expected that applicants levels of personal literacy and numeracy should be broadly equivalent to those of the top 30 per cent of the population. Analysis of some Australian Capital Territory subjects included in a study 1 has shown how these and other results in senior secondary subjects can be used as proxy indicators of levels of personal literacy or numeracy. This means that students results in this initial, preliminary, list can be used to guide judgments about the risk that, on entry, an applicant to a teacher training course may not have levels of personal literacy and numeracy broadly equivalent to the top thirty per cent of the population. In this study, these levels of personal literacy and numeracy have been approximately defined by level 4 in the Australian Core Skills Framework (ACSF). The table below lists the eight senior secondary subjects in Australian Capital Territory used on this occasion and, for each subject, a grade (one of five possible grades: E, D, C, B and A, where A is the highest). The grade in the table is the result at which it is highly likely 2 that the work of a student in this subject with that result would be seen by a reasonable person as sufficient evidence that the student has the skills required by Australian Core Skills Framework level 4 in the listed domain (literacy or numeracy), the level broadly equivalent to the top 30 per cent of the population. Subject Domain Indicative result English literacy B History literacy B Geography literacy B Physics numeracy B Mathematical Applications numeracy A Chemistry numeracy B Mathematical Methods numeracy B Table 1: Australian Capital Territory subjects in first phase study, grade indicating likely personal levels of literacy or numeracy at ACSF4 1 A collaboration of the Australian Institute for Teaching and School Leadership (AITSL) and six members of the Australasian Curriculum, Assessment and Certification Authorities (ACACA) the body for CEOs of assessment agencies. 2 Four times more likely than not, for someone who has only just met the requirements for this grade. For someone with a higher result, even more likely. For someone with a lower grade, less likely, though for results just below this grade it is still very likely. This level has been chosen so that the ratio of false negatives to false positives is about four to one: we are giving the wrong impression about four students who are really ok (but just below the grade listed) for everyone who might not be ok. This does not take into account measurement error the uncertainty in any assessment process. 3

4 A person who enters a teaching preparation course with a grade in one of these subjects at or above the grade listed in the table is highly likely to have personal levels of literacy or numeracy skills at least at ACSF level 4 well below the grade in the table and no grade above the relevant threshold in any other subject in the final complete table is much more likely to need additional and specific support and help to ensure the development of appropriate literacy or numeracy skills before the end of the course. It is not appropriate to use the results of this study to exclude individuals from initial teacher education. They may, however, be used as an indicator of the risk that a person may not have the capacity to engage effectively with the program and with the intellectual demands of teaching. The grades in the table are results in specific senior secondary subjects. They represent achievement in these subjects and are not assessments of literacy or numeracy skills defined by ASCF level 4. It is therefore possible to have skills at or above this ACSF level and to have a low result in a subject because of a failure to demonstrate knowledge and understanding of the content of the subject. A result in, for example, Physics is based on showing understanding of Physics, not on elements of the ACSF numeracy standards. A result in a study of English literature is based on showing understanding of literature, not on the elements of the ACSF literacy standards. There are other senior secondary subjects in Australian Capital Territory that can clearly provide evidence of levels of personal literacy and numeracy. These have yet to be studied. The results in the table do not compare achievement standards in corresponding subjects across jurisdictions. 3 The fact that HA appears as the threshold result for one subject in another table and B appears as the threshold result for a similar subject in Australian Capital Territory does not tell us that the HA represents the same standard of knowledge and understanding of the subject as the B does. 3 ACACA currently has a project examining processes for establishing and monitoring such comparability. This study does not contribute meaningfully to that task. 4

5 Queensland The Accreditation of Initial Teacher Education Programs in Australia: Standards and Procedures requires that all entrants to initial teacher education will successfully demonstrate their capacity to engage effectively with a rigorous higher education program and to carry out the intellectual demands of teaching itself. To achieve this, it is expected that applicants levels of personal literacy and numeracy should be broadly equivalent to those of the top 30 per cent of the population. Analysis of some Queensland subjects included in a study 4 has shown how these and other results in senior secondary subjects can be used as proxy indicators of levels of personal literacy or numeracy. This means that students results in this initial, preliminary, list can be used to guide judgments about the risk that, on entry, an applicant to a teacher training course may not have levels of personal literacy and numeracy broadly equivalent to the top thirty per cent of the population. In this study, these levels of personal literacy and numeracy have been approximately defined by level 4 in the Australian Core Skills Framework (ACSF). The table below lists the five senior secondary subjects in Queensland used on this occasion and, for each subject, a level of achievement (one of five possible levels of achievement: Very Limited Achievement (VLA), Limited Achievement (LA), Sound Achievement (SA), High Achievement (HA) and Very High Achievement (VHA), where VHA is the highest). The level of achievement in the table is the result at which it is highly likely 5 that the work of a student in this subject with that result would be seen by a reasonable person as sufficient evidence that the student has the skills required by Australian Core Skills Framework level 4 in the listed domain (literacy or numeracy), the level broadly equivalent to the top 30 per cent of the population. Subject Domain Indicative result English literacy HA Modern History literacy HA Physics numeracy HA Mathematics A numeracy HA Mathematics B numeracy SA Table 1: Queensland subjects in first phase study, level of achievement indicating likely personal levels of literacy or numeracy at ACSF4 4 A collaboration of the Australian Institute for Teaching and School Leadership (AITSL) and six members of the Australasian Curriculum, Assessment and Certification Authorities (ACACA) the body for CEOs of assessment agencies. 5 Four times more likely than not, for someone who has only just met the requirements for this level of achievement. For someone with a higher result, even more likely. For someone with a lower level of achievement, less likely, though for results just below this level of achievement it is still very likely. This level has been chosen so that the ratio of false negatives to false positives is about four to one: we are giving the wrong impression about four students who are really ok (but just below the level of achievement listed) for everyone who might not be ok. This does not take into account measurement error the uncertainty in any assessment process. 5

6 A person who enters a teaching preparation course with a level of achievement in one of these subjects at or above the level of achievement listed in the table is highly likely to have personal levels of literacy or numeracy skills at least at ACSF level 4 well below the level of achievement in the table and no level of achievement above the relevant threshold in any other subject in the final complete table is much more likely to need additional and specific support and help to ensure the development of appropriate literacy or numeracy skills before the end of the course. The levels of achievement in the table are results in specific senior secondary subjects. They represent achievement in these subjects and are not assessments of literacy or numeracy skills defined by ASCF level 4. It is therefore possible to have skills at or above this ACSF level and to have a low result in a subject because of a failure to demonstrate knowledge and understanding of the content of the subject. A result in, for example, Physics is based on showing understanding of Physics, not on elements of the ACSF numeracy standards. A result in a study of English literature is based on showing understanding of literature, not on the elements of the ACSF literacy standards. There are other senior secondary subjects in Queensland that can clearly provide evidence of levels of personal literacy and numeracy. These have yet to be studied. The results in the table do not compare achievement standards in corresponding subjects across jurisdictions. 6 The fact that B appears as the threshold result for one subject in another table and HA appears as the threshold result for a similar subject in Queensland does not tell us that the B represents the same standard of knowledge and understanding of the subject as the HA does. 6 ACACA currently has a project examining processes for establishing and monitoring such comparability. This study does not contribute meaningfully to that task. 6

7 South Australia The Accreditation of Initial Teacher Education Programs in Australia: Standards and Procedures requires that all entrants to initial teacher education will successfully demonstrate their capacity to engage effectively with a rigorous higher education program and to carry out the intellectual demands of teaching itself. To achieve this, it is expected that applicants levels of personal literacy and numeracy should be broadly equivalent to those of the top 30 per cent of the population. Analysis of some South Australian subjects included in a study 7 has shown how these and other results in senior secondary subjects can be used as proxy indicators of levels of personal literacy or numeracy. This means that students results in this initial, preliminary, list can be used to guide judgments about the risk that, on entry, an applicant to a teacher training course may not have levels of personal literacy and numeracy broadly equivalent to the top thirty per cent of the population. In this study, these levels of personal literacy and numeracy have been approximately defined by level 4 in the Australian Core Skills Framework (ACSF). The table below lists the six senior secondary subjects in South Australia used on this occasion and, for each subject, a grade level (one of fifteen possible grade levels: E-, E, E+, D-, D, D+, C-, C, C+, B-, B, B+, A-, A and A+, where A+ is the highest). The grade level in the table is the result at which it is highly likely 8 that the work of a student in this subject with that result would be seen by a reasonable person as sufficient evidence that the student has the skills required by Australian Core Skills Framework level 4 in the listed domain (literacy or numeracy), the level broadly equivalent to the top 30 per cent of the population. Subject Domain Indicative result English Communications literacy B Modern History literacy B+ English Studies literacy B- Mathematical Applications numeracy B+ Mathematical Studies numeracy C+ Physics Stage 2 numeracy C+ Table 1: South Australia subjects in first phase study, grade level indicating likely personal levels of literacy or numeracy at ACSF4 7 A collaboration of the Australian Institute for Teaching and School Leadership (AITSL) and six members of the Australasian Curriculum, Assessment and Certification Authorities (ACACA) the body for CEOs of assessment agencies. 8 Four times more likely than not, for someone who has only just met the requirements for this grade level. For someone with a higher result, even more likely. For someone with a lower grade level, less likely, though for results just below this grade level it is still very likely. This level has been chosen so that the ratio of false negatives to false positives is about four to one: we are giving the wrong impression about four students who are really ok (but just below the grade level listed) for everyone who might not be ok. This does not take into account measurement error the uncertainty in any assessment process. 7

8 A person who enters a teaching preparation course with a grade level in one of these subjects at or above the grade level listed in the table is highly likely to have personal levels of literacy or numeracy skills at least at ACSF level 4 well below the grade level in the table and no grade level above the relevant threshold in any other subject in the final complete table is much more likely to need additional and specific support and help to ensure the development of appropriate literacy or numeracy skills before the end of the course. The grade levels in the table are results in specific senior secondary subjects. They represent achievement in these subjects and are not assessments of literacy or numeracy skills defined by ASCF level 4. It is therefore possible to have skills at or above this ACSF level and to have a low result in a subject because of a failure to demonstrate knowledge and understanding of the content of the subject. A result in, for example, Physics is based on showing understanding of Physics, not on elements of the ACSF numeracy standards. A result in a study of English literature is based on showing understanding of literature, not on the elements of the ACSF literacy standards. There are other senior secondary subjects in South Australia that can clearly provide evidence of levels of personal literacy and numeracy. These have yet to be studied. The results in the table do not compare achievement standards in corresponding subjects across jurisdictions. 9 The fact that HA appears as the threshold result for one subject in another table and B appears as the threshold result for a similar subject in South Australia does not tell us that the HA represents the same standard of knowledge and understanding of the subject as the B does. 9 ACACA currently has a project examining processes for establishing and monitoring such comparability. This study does not contribute meaningfully to that task. 8

9 Tasmania The Accreditation of Initial Teacher Education Programs in Australia: Standards and Procedures requires that all entrants to initial teacher education will successfully demonstrate their capacity to engage effectively with a rigorous higher education program and to carry out the intellectual demands of teaching itself. To achieve this, it is expected that applicants levels of personal literacy and numeracy should be broadly equivalent to those of the top 30 per cent of the population. Analysis of some Tasmanian subjects included in a study 10 has shown how these and other results in senior secondary subjects can be used as proxy indicators of levels of personal literacy or numeracy. This means that students results in this initial, preliminary, list can be used to guide judgments about the risk that, on entry, an applicant to a teacher training course may not have levels of personal literacy and numeracy broadly equivalent to the top thirty per cent of the population. In this study, these levels of personal literacy and numeracy have been approximately defined by level 4 in the Australian Core Skills Framework (ACSF). The table below lists the seven senior secondary subjects in Tasmania used on this occasion and, for each subject, an award (one of five possible awards: Preliminary Achievement (PA), Satisfactory Achievement (SA), Commendable Achievement (CA), High Achievement (HA) and Exemplary Achievement (EA), where EA is the highest). The award in the table is the result at which it is highly likely 11 that the work of a student in this subject with that result would be seen by a reasonable person as sufficient evidence that the student has the skills required by Australian Core Skills Framework level 4 in the listed domain (literacy or numeracy), the level broadly equivalent to the top 30 per cent of the population. Subject Domain Indicative result English Communications literacy CA English Writing literacy CA Modern World History literacy CA Ancient History literacy CA Mathematics Applied numeracy CA Mathematics Methods numeracy CA Physics numeracy HA Table 1: Tasmania subjects in first phase study, award indicating likely personal levels of literacy or numeracy at ACSF4 10 A collaboration of the Australian Institute for Teaching and School Leadership (AITSL) and six members of the Australasian Curriculum, Assessment and Certification Authorities (ACACA) the body for CEOs of assessment agencies. 11 Four times more likely than not, for someone who has only just met the requirements for this award. For someone with a higher result, even more likely. For someone with a lower award, less likely, though for results just below this award it is still very likely. This level has been chosen so that the ratio of false negatives to false positives is about four to one: we are giving the wrong impression about four students who are really ok (but just below the award listed) for everyone who might not be ok. This does not take into account measurement error the uncertainty in any assessment process. 9

10 A person who enters a teaching preparation course with an award in one of these subjects at or above the award listed in the table is highly likely to have personal levels of literacy or numeracy skills at least at ACSF level 4 well below the award in the table and no award above the relevant threshold in any other subject in the final complete table is much more likely to need additional and specific support and help to ensure the development of appropriate literacy or numeracy skills before the end of the course. The awards in the table are results in specific senior secondary subjects. They represent achievement in these subjects and are not assessments of literacy or numeracy skills defined by ASCF level 4. It is therefore possible to have skills at or above this ACSF level and to have a low result in a subject because of a failure to demonstrate knowledge and understanding of the content of the subject. A result in, for example, Physics is based on showing understanding of Physics, not on elements of the ACSF numeracy standards. A result in a study of English literature is based on showing understanding of literature, not on the elements of the ACSF literacy standards. There are other senior secondary subjects in Tasmania that can clearly provide evidence of levels of personal literacy and numeracy. These have yet to be studied. The results in the table do not compare achievement standards in corresponding subjects across jurisdictions. 12 The fact that B appears as the threshold result for one subject in another table and HA appears as the threshold result for a similar subject in Tasmania does not tell us that the B represents the same standard of knowledge and understanding of the subject as the HA does. 12 ACACA currently has a project examining processes for establishing and monitoring such comparability. This study does not contribute meaningfully to that task. 10

11 Victoria The Accreditation of Initial Teacher Education Programs in Australia: Standards and Procedures requires that all entrants to initial teacher education will successfully demonstrate their capacity to engage effectively with a rigorous higher education program and to carry out the intellectual demands of teaching itself. To achieve this, it is expected that applicants' levels of personal literacy and numeracy should be broadly equivalent to those of the top 30 per cent of the population". Analysis of some Victorian subjects included in a recent study 13 has shown how results can be used as proxy indicators of levels of personal literacy or numeracy. This means that students' results in this initial, preliminary, list can be used to guide judgments about the risk that, on entry, an applicant to a teacher training course may not have levels of personal literacy and numeracy broadly equivalent to those of the top thirty per cent of the population. In this study, these levels of personal literacy and numeracy have been approximately defined by level 4 or above in the Australian Core Skills Framework (ACSF). The table below lists the seven senior secondary subjects in Victoria used on this occasion and, for each subject, a study score range (study scores are numbers with, for each study, the mean set to 30 and the maximum set to 50). The study score range in the table is the range of study scores for which it is highly likely 14 that the work of a student in this subject with a study score in that range would be seen by a reasonable person as sufficient evidence that the student has the skills required by Australian Core Skills Framework level 4 in the listed domain (literacy or numeracy). ACSF level 4 or above describes the level of skills broadly equivalent to those of the top 30 per cent of the population. Subject Domain Indicative result English literacy EALD literacy History: Revolutions literacy Literature literacy Accounting numeracy Further Mathematics numeracy Mathematical Methods (CAS) numeracy Victoria VCE subjects in the first phase study, study score range indicating likely personal levels of literacy or numeracy at ACSF4 13 A collaboration of the Australian Institute for Teaching and School Leadership (AITSL) and six members of the Australasian Curriculum, Assessment and Certification Authorities (ACACA) the body for CEOs of assessment agencies 14 Four times more likely than not, for someone who has a study score around the middle of this range. For someone with a higher study score, even more likely. For someone with a lower study score, less likely, though for results just below this study score range it is still very likely. This level has been chosen so that the ratio of false negatives to false positives is about four to one: we are giving the wrong impression about close to four students who are really ok (but just below the grade listed) for everyone who might not be ok (but has the result listed in the table). This does not take into account measurement error the uncertainty in any assessment process, which changes the uncertainty in both directions. 11

12 A person who enters a teaching preparation course with a study score in one of these subjects above the study score range listed in the table is almost certain to have personal levels of literacy or numeracy skills at least at ACSF level 4, which indicates a capacity to engage effectively with the literacy and numeracy component of an initial teacher education program well below the study score range in the table and no study score at or above the relevant study score range in any other subject in the final complete table is much more likely to need additional and specific support and help to ensure the development of appropriate literacy or numeracy skills before the end of the course. It is not appropriate to use the results of this study to exclude individuals from initial teacher education. They may, however, be used as an indicator of the risk that a person may not have the capacity to engage effectively with the program and with the intellectual demands of teaching. The study score ranges in the table are results in specific senior secondary subjects. They represent achievement in these subjects and are not assessments of literacy or numeracy skills defined by ASCF level 4. It is therefore possible to have skills at or above this ACSF level and to have a low result in a subject because of a failure to demonstrate knowledge and understanding of the content of the subject. There are other senior secondary subjects in Victoria that clearly can provide evidence of levels of personal literacy and numeracy. These have yet to be studied. The results in the table do not compare achievement standards in corresponding subjects across jurisdictions. 15 The fact that B appears for one subject in the table for another jurisdiction and a range of appears as the study score range for a similar subject in Victoria does not tell us that the B represents the same standard of knowledge and understanding of the subject as the range of does. It should be noted that Victorian study scores cannot be compared across different subjects. 15 ACACA currently has a project examining processes for establishing and monitoring such comparability. This study does not contribute meaningfully to that task. 12

13 Western Australia The Accreditation of Initial Teacher Education Programs in Australia: Standards and Procedures requires that all entrants to initial teacher education will successfully demonstrate their capacity to engage effectively with a rigorous higher education program and to carry out the intellectual demands of teaching itself. To achieve this, it is expected that applicants' levels of personal literacy and numeracy should be broadly equivalent to those of the top 30 per cent of the population". Analysis of some Western Australian subjects included in a recent study 16 has shown how results can be used as proxy indicators of levels of personal literacy or numeracy. This means that students' results in this initial, preliminary, list can be used to guide judgments about the risk that, on entry, an applicant to a teacher training course may not have levels of personal literacy and numeracy broadly equivalent to those of the top thirty per cent of the population. In this study, these levels of personal literacy and numeracy have been approximately defined by level 4 or above in the Australian Core Skills Framework (ACSF). The table below lists the five senior secondary subjects in Western Australia used on this occasion and, for each subject, a WACE Band (one of five possible WACE Bands: Inadequate Achievement (IA), Limited Achievement (LA), Satisfactory Achievement (SA), High Achievement (HA) and Excellent Achievement (EA), where EA is the highest). Other courses will contain certain elements that if achieved at Satisfactory Achievement (SA) could be used as proxy indicators of levels of personal literacy or numeracy. The WACE Band in the table is the result at which it is highly likely 17 that the work of a student in this subject with that result would be seen by a reasonable person as sufficient evidence that the student has the skills required by Australian Core Skills Framework level 4 in the listed domain (literacy or numeracy). ACSF level 4 or above describes the level of skills broadly equivalent to those of the top 30 per cent of the population. Subject Domain Indicative result English Stage 3 literacy SA Literature Stage 3 literacy SA Modern History Stage 3 literacy SA Mathematics Stage 3 numeracy SA Physics Stage 3 numeracy SA Western Australia - subjects in the first phase study, WACE Band indicating likely personal levels of literacy or numeracy at ACSF4 16 A collaboration of the Australian Institute for Teaching and School Leadership (AITSL) and six members of the Australasian Curriculum, Assessment and Certification Authorities (ACACA) the body for CEOs of assessment agencies 17 Four times more likely than not, for someone who has met the requirements for this WACE Band. For someone with a higher WACE Band, even more likely. For someone with a lower WACE Band, less likely, though for results just below this WACE Band it is still very likely. This level has been chosen so that the ratio of false negatives to false positives is about four to one: we are giving the wrong impression about close to four students who are really ok (but just below the WACE Band listed) for everyone who might not be ok (but has the result listed in the table). This does not take into account measurement error - the uncertainty in any assessment process, which changes the uncertainty in both directions. 13

14 A person who enters a teaching preparation course with a WACE Band in one of these subjects at or above the WACE Band listed in the table is highly likely to have personal levels of literacy or numeracy skills at least at ACSF level 4, which indicates a capacity to engage effectively with the literacy and numeracy component of an initial teacher education program well below the WACE Band in the table and no WACE Band at or above the relevant WACE Band in any other subject in the final complete table is much more likely to need additional and specific support and help to ensure the development of appropriate literacy or numeracy skills before the end of the course. Western Australia reserves the right to review the indicators of personal levels of literacy and numeracy once WACE 2015/16 is introduced. It is not appropriate to use the results of this study to exclude individuals from initial teacher education. They may, however, be used as an indicator of the risk that a person may not have the capacity to engage effectively with the program and with the intellectual demands of teaching. The WACE Bands in the table are results in specific senior secondary subjects. They represent achievement in these subjects and are not assessments of literacy or numeracy skills defined by ASCF level 4. It is therefore possible to have skills at or above this ACSF level and to have a low result in a subject because of a failure to demonstrate knowledge and understanding of the content of the subject. The results in the table do not compare achievement standards in corresponding subjects across jurisdictions. 18 The fact that B appears for one subject in the table for another jurisdiction and SA appears as the WACE Band for a similar subject in Western Australia does not tell us that the B represents the same standard of knowledge and understanding of the subject as the SA does. 18 ACACA currently has a project examining processes for establishing and monitoring such comparability. This study does not contribute meaningfully to that task. 14

15 New South Wales The Accreditation of Initial Teacher Education Programs in Australia: Standards and Procedures requires that all entrants to initial teacher education will successfully demonstrate their capacity to engage effectively with a rigorous higher education program and to carry out the intellectual demands of teaching itself. To achieve this, it is expected that applicants' levels of personal literacy and numeracy should be broadly equivalent to those of the top 30 per cent of the population. In NSW this will equate to the achievement of at least Band 5 in the NSW Higher School Certificate or equivalent, in at least three subjects one of which must be English. Subject Domain Indicative result English (Advanced) literacy Band 5 English (Standard) literacy Band 5 English Extension 1 literacy Band E3 English Extension 2 literacy Band E3 Mathematics numeracy Band 5 Mathematics Extension 1 numeracy Band E3 Mathematics Extension 2 numeracy Band E3 General Mathematics numeracy Band 5 New South Wales Other subjects will contain certain elements that, if achieved at Band 5, can contribute to overall standards of literacy and numeracy. For a person who enters a teaching preparation course: a Band 5 result in at least three subjects, one of which must be an English course listed in the table, is almost certain to have personal levels of literacy or numeracy skills at least at ACSF level 4, although no comparison study has been undertaken; well below Band 5 results indicates a likely need for additional and specific support and help to ensure the development of appropriate literacy or numeracy skills before the end of the course. Band 5 results represent achievement in subjects and are not assessments of literacy or numeracy skills defined by ACSF. 15

16 The results in the table do not compare achievement standards in corresponding subjects across jurisdictions 19. The fact that B appears for one subject in the table for another jurisdiction and a Band 5 appears as the study score range for a similar subject in New South Wales does not tell us that the B represents the same standard of knowledge and understanding of the subject as Band 5 does. 19 ACACA currently has a project examining processes for establishing and monitoring such comparability. This study does not contribute meaningfully to that task. 16