Spanish as Foreign Language Teachers conceptions of assessment: preliminary results from an internet inquiry. Introduction

Transcription

1 Spanish as Foreign Language Teachers conceptions of assessment: preliminary results from an internet inquiry. Ana Remesal Universidad de Barcelona, Spain Gavin T.L. Brown University of Auckland, New Zealand Introduction This study intends to fill a gap in current research on conceptions of assessment. Indeed, conceptions of assessment have been focus of systematic research in the last decade in different cultural contexts (e.g., Brown, Kennedy, Fok, Chan & Yu, 2009; Brown, Lake & Matters, 2011; Coll & Remesal, 2009; Delandshere & Jones, 1999; Remesal, 2011). These studies refer mostly to in-service teachers, both active in primary or secondary education. Other recent studies, such as Brown & Remesal (2012) turned attention to prospective teachers. However, up to now, we have very little information-if any at allabout teachers conceptions of assessment in educational contexts other than compulsory or basic schooling. We gathered data from teachers of Spanish as Foreign Language (ELE) working in four different educational contexts: compulsory school, extra-school teaching, adult education, and in-company training. Research method and design We presented a questionnaire with a positively packed rating scale (Brown, 2004) to the members of an online professional community of teachers of ELE (Comunidad Todoele, teachers altogether were invited to respond to the questionnaire. The questionnaire got a response ratio of 7%; hence, the sample is 492 respondents (margin of error = 4.72%), in a three month period with three reminder invitations. Demographic characteristics of the sample are presented in table 1. The questionnaire was built around a theoretical model of conceptions of assessment, drawn from a previous interpretive research (Remesal, 2006). This model rests on a definition of conceptions as the subjective sum of individual beliefs, which, in turn, are assumptions about objects and phenomena that people take as true (often without intellectual contrast) (Green, 1971; Pajares, 1992). The model refers to four basic dimensions within a bipolar continuum of conceptions. It states that conceptions about assessment are the result of a combination of beliefs about how assessment affects (a) teaching, (b) learning, (c) the certification of learning, and (d) the account giving of teaching. Hence, depending on the combination of particular beliefs, a teacher s conception can be defined as more inclined towards understanding assessment as an

2 instrument for regulation of educational processes or rather as an instrument for societal control. The following variables were considered for further comparison: professional practice context and type of initial training. Results Participants in this study were asked to report personal characteristics about themselves (Table 1). The sample was characterised by teachers who were predominantly female, middle-aged, and with considerable teaching experience. The majority had their initial training in linguistic (e.g., translation, philology, linguistics) rather than educationoriented (e.g., pedagogy, didactics, psychology) disciplines. Just over half taught in adult-level contexts (e.g., higher education or in businesses), while the remainder taught children and adolescents in schools or after-school contexts. Confirmatory factor analysis of the data was used to evaluate the proposed 6 factor, bifactoral structure (Figure 1). In this model, beliefs about assessment refer to four aspects of the teaching and learning process that may be affected by the participant s sense of whether assessment regulates or does not regulate them. The factors then were teaching, learning, certifying of learning, and accountability of teaching and two tendencies of using assessment, either pro- or anti-formatively. Each item is predicted by one of these aspects and tendencies. The fit of the model was acceptable (χ 2 = , df=486, [χ 2 /df=2.141, p=.14]; CFI=.81; gamma hat=.94; RMSEA=.048, 90% CI= ; SRMR=.052). While not every path was statistically significant, the combined weight of paths produced percentage of variance explained ranging from 2% to 47% (M=25%, SD=12.5%) (see Table 2). Mean scores for each factor were calculated by averaging all items contributing to the factor (Table 3). These values indicated that the sample as a whole endorsed the regulation of learning most, though the differences among the four aspects were small. The Pro-Formative tendency was much more strongly endorsed than the Anti-Formative tendency. Beliefs concerning the formative regulation of learning and teaching were endorsed most strongly; and teachers gave the weakest agreement to the Anti-Formative tendency of these same two aspects. As a whole, then, the teachers conceptions of assessment were positive towards Formative uses of assessment for the improvement of teaching and learning. In order to carry out a multiple analysis of variance of the effect of key demographic variables on conceptions, eight interactive factor scores were used (i.e., Learning-Pro, Teaching-Pro, Accountability-Pro, Certifying-Pro, Learning-Anti, Teaching-Anti, Accountability-Anti, and Certifying-Anti). The predictor variables used were initial training and working context (i.e., students were either children/adolescents or adults). Statistically significant differences were found only for the main effect of working context (Wilks λ=.96; F (8,478) =2.71, p=.006) (i.e., interaction and main effect for training were not statistically significant). Inspection of univariate differences for the eight factor scores showed that statistically significant differences existed for only three

3 factors (i.e., Accountability-Anti, p=.006, d=.26; Certifying-Anti, p=.031, d=.20; and Learning-Anti, p<.001, d=.38). In all three cases, teachers in adult contexts gave lower scores, though the differences were small to moderate. Multi-group confirmatory factor analysis was carried out to determine whether the measurement model was statistically equivalent across the two working contexts. Constraining the measurement weights to be the same for the two groups resulted in a difference in CFI=.011 which exceeds the standard of <.01, supporting the conclusion that the two groups responded to the inventory in different ways. Thus, we can conclude that teachers of Spanish in these two different contexts are samples of two different populations. Further analyses of the regression weights and inter-factor correlations is needed to understand how the teaching context impacts beliefs about assessment. Discussion and conclusion The results of this study show that, as proposed by the original model (Remesal, 2006), teacher beliefs about assessment are a consequence of two interacting elements; that is, teacher beliefs about the effects of assessment and teacher s evaluation as to whether assessment may have formative or non-formative uses. On the whole, the teachers of Spanish as a foreign language conceived of assessment primarily in terms of its regulating effects on teaching and learning and were strongly positive towards formative assessment purposes. In other words, they agreed mostly that assessment existed to serve the improvement of teaching and learning. Differences based on context of work identified that those working in adult contexts were less positive about the nonformative aspects of assessment, a result that may be explained by the greater responsibility of care for students that teachers in schooling contexts are expected to exercise. This result is coherent with the theoretical model which declares the twofold functional nature of assessment in the basic school system (Remesal, 2011), which by default imposes both, formative and non-formative (i.e., accrediting and accountability) purposes of assessment. Within schooling contexts, teachers have to accept that assessment results are also used to evaluate students, monitor teacher effectiveness, and judge school quality. In contrast, it appears that teachers working outside the basic school system have a lesser attachment or commitment to non-formative purposes; perhaps this arises because adult learning and teaching is more driven by interest and personal motivation and hence, attracts a lesser sense of accountability and evaluation. Further analysis of the non-equivalent aspects of the measurement model across working contexts is needed to see if further insights can be gained. Nonetheless, this study provides internal validation evidence for the questionnaire and advances our understanding of the contingencies in teachers belief systems about assessment. Acknowledgements

4 Ana Remesal and Gavin Brown want to acknowledge the contribution of Patrick S.G.Kerr to the analysis of the reported data. References Brown, G. T. L. (2004). Measuring attitude with positively packed self-report ratings: Comparison of agreement and frequency scales. Psychological Reports, 94, Brown, G. T. L., Kennedy, K. J., Fok, P. K., Chan, J. K. S., & Yu, W. M. (2009). Assessment for student improvement: Understanding Hong Kong teachers conceptions and practices of assessment. Assessment in Education: Principles, Policy & Practice, 16(3), Brown, G. T. L., Lake, R., & Matters, G. (2011). Queensland teachers conceptions of assessment: The impact of policy priorities on teacher attitudes. Teaching and Teacher Education, 27(1), Brown, G.; Remesal, A. (2012). Prospective teachers conceptions of assessment: a cross-cultural comparison. The Spanish Journal of Psychology, vol.15 (1, onlinefirst). Coll, C., & Remesal, A. (2009). Concepciones del profesorado de matemáticas sobre la evaluación del aprendizaje en la educación obligatoria: dimensiones y funciones. [Mathematics teachers conceptions of learning assessment in compulsory education: dimensions and functions]. Infancia y Aprendizaje,32(3), Delandshere, G., & Jones, J. H. (1999). Elementary teachers beliefs about assessment in mathematics. A case of assessment paralysis. Journal of Curriculum and Supervision, 14(3), Green, T. F. (1971). The activities of teaching. New York: McGraw-Hill. Pajares, M. F. (1992). Teachers beliefs and educational research: Cleaning up a messy construct. Review of Educational Research, 62(3), pp Remesal, A. (2006). Los problemas en la evaluación del área de matemáticas en la educación obligatoria: perspectiva de profesores y alumnos. Tesis Doctoral presentada en la Universidad de Barcelona. Director: Dr.César Coll Salvador. Remesal, A. (2011). Primary and secondary teachers conceptions of assessment: A qualitative study. Teaching and Teacher Education, 27(2),

5 Figure 1. Bifactoral measurement model of 492 Spanish as a Foreign Language (ELE) teachers responses to inventory

6 Table 1. Demographic characteristics of sample Category N % Age < > Sex male female Teaching experience <2 years years years >10 years Initial training Linguistic-oriented Educational-oriented Student addressees Children-adolescents Adults

7 results from an internet inquiry. Paper presented at International Conference on Assessment: Linking Multiple Perspectives on Table 2. Item statistics by predictor factors item Spanish English Purpose β Formative β SMC M SD ac01 Ante unos malos resultados de un grupo de alumnos, la mejor opción es bajar el nivel o incluso repetir la evaluación In face of bad assessment results, the best option is to lower the standards or even to repeat the test Certifying 0.28 Anti ac02 ac03 ac04 ac05 ac06 ac07 ac08 ac09 ac10 Una bajada de nivel y hacer la vista gorda nunca es la solución cuando nos encontramos con malos resultados de evaluación en una clase El avance de los alumnos siempre se debe valorar con el mismo rasero para todos, según los objetivos del curso planteados El punto de partida individual es la referencia imprescindible para valorar adecuadamente el avance de cada alumno Es preferible comunicar los resultados en forma estrictamente numérica, para evitar malentendidos Las calificaciones numéricas (0-10, 6-1, 1-15, ), o categoriales básicas (A,B,C; aprobado, suspenso ) son, por lo general, poco informativas Para aprobar, el alumno debe alcanzar un dominio mínimo, indicado en el currículo oficial vigente Para aprobar, el alumno debe demostrar que ha avanzado bastante desde su punto de partida al inicio del curso La comunicación de resultados siempre debe ser pública para que cada alumno se ubique en el grupo-clase La comunicación de resultados debe ser siempre privada para evitar comparaciones Lowering standards and blinking an eye are never the solution to face bad assessment results in a course Students learning progress must be always measured with the same scale for everyone, according to the course objectives The individual starting point is an indispensable reference to properly evaluate the individual student s progress It is always better to communicate assessment results in a numerical form, in order to avoid misunderstandings Numerical grading or basic categories are, generally speaking, uninformative. In order to pass the course, the students must reach a minimal competence, indicated in the official curriculum In order to pass the course, the student must demonstrate a sufficient progress considering his own starting point Communicating assessment results must always happen in public, so every student can locate himself in the class Communicating assessment results must always be private, to avoid comparisons among students Certifying 0.24 Pro Certifying 0.53 Anti Certifying 0.07 Pro Certifying 0.53 Anti Certifying Pro Certifying 0.56 Anti Certifying 0.44 Pro Certifying 0.39 Anti Certifying 0.03 Pro ap01 Los exámenes sorpresa son un buen elemento Surprise exams are a good motivational Learning 0.32 Anti

8 results from an internet inquiry. Paper presented at International Conference on Assessment: Linking Multiple Perspectives on item Spanish English Purpose β Formative β SMC M SD motivador para los alumnos tool for students ap02 El alumno siempre debe estar informado de la The student must always be informed about Learning 0.29 Pro ap03 ap04 ap05 ap06 ap07 ap08 intención evaluativa del profesor Es imprescindible que los alumnos se conciencien del nivel que van alcanzando según el baremo establecido Es necesario que cada alumno sea consciente de su propio avance desde su propio punto de partida Evaluar el aprendizaje no les supone a los alumnos una ocasión de aprendizaje nueva Cada ocasión de evaluación es también una ocasión de posible nuevo aprendizaje Si no hubiera evaluación del aprendizaje sería imposible motivar a los alumnos para estudiar La evaluación vista como amenaza de castigo o promesa de premio no ayuda al aprendizaje the assessment intention of the teacher It is indispensable that the students get aware of their learning level according to the standards It is necessary that the students get aware of their own learning improvement, considering their particular starting point Learning assessment does not imply a new learning chance. Each assessment occasion is also a new opportunity for learning If there was not learning assessment, it would be impossible to motivate students to learn Assessment understood as a threat of punishment or promise of Price does not help to learn Learning 0.48 Anti Learning 0.04 Pro Learning 0.20 Anti Learning Pro Learning 0.41 Anti Learning Pro ap09 Lo mejor para el alumno tras una evaluación The student s best option after bad negativa es hacer borrón y cuenta nueva assessment results is to make a clean slate Learning 0.40 Anti ap10 Cuando hay malos resultados de evaluación, el If there are bad assessment results, the alumno debe reflexionar sobre sus errores students must reflect on their mistakes Learning 0.24 Pro e01 La evaluación del aprendizaje y la enseñanza Learning assessment shouldn t be mixed up no se deben mezclar with teaching Teaching 0.47 Anti e02 La evaluación del aprendizaje y la enseñanza Learning assessment and teaching must be deben ser coherentes la una con la otra coherent Teaching Pro e03 La evaluación del aprendizaje supone una Assessment implies a loss of time and quite a pérdida de tiempo y bastante estrés stress Teaching 0.13 Anti e04 La evaluación del aprendizaje de los alumnos Learning assessment is the indispensable es la brújula imprescindible de la enseñanza compass of teaching Teaching 0.10 Pro e07 Las actividades de evaluación deben Assessment activities and learning activities diferenciarse claramente de las actividades de must be clearly differentiated aprendizaje Teaching 0.50 Anti e08 Cualquier actividad de aprendizaje puede ser Any learning activity may be as well used for Teaching Pro

9 results from an internet inquiry. Paper presented at International Conference on Assessment: Linking Multiple Perspectives on item Spanish English Purpose β Formative β SMC M SD también actividad de evaluación the purpose of assessment e09 e10 r05 r06 r07 r08 r09 r10 La evaluación del aprendizaje no debe afectar a otras decisiones docentes (objetivos, contenidos, recursos, métodos...) Cada vez que se evalúa el aprendizaje de los alumnos es también ocasión de revisar diversos aspectos de la enseñanza (objetivos, contenidos, recursos, métodos...) La evaluación del dominio de ELE, pero particularmente la evaluación externa mediante los exámenes oficiales, es imprescindible para tener un control Los exámenes oficiales de nivel de ELE ayudan a cada ciudadano a demostrar su propia competencia, por ejemplo ante futuros empleadores El docente es el único responsable de elaborar o seleccionar actividades de evaluación y corregirlas posteriormente para los alumnos Los alumnos son co-partícipes con el docente en todos los pasos de la evaluación, incluyendo la preparación de actividades y la corrección de resultados El currículo oficial determina los objetivos inexorables que cada docente debe cumplir Cada docente debe ajustar su planificación de curso a los alumnos de su grupo Learning assessment must not affect other teaching decisions (objectives, contents, resources, methods ) Every time the students learning is assessed, it is also the time to revise different aspects of teaching (objectives, contents, resources, methods). Assessment of Spanish as Foreign Language, but particularly external assessment by means of official examinations, is indispensable to control. Official SFL examinations help each citizen to demonstrate his own competence, for instance in front of likely employers The teacher is the only responsible of designing or selecting assessment activities and evaluating them afterwards for the students Students collaborate with the teacher in every step of the assessment process, including the preparation of activities and evaluating results The official curriculum determins the learning goals that every teacher must pursue The teacher must adjust his course plan to his group of students Teaching 0.50 Anti Teaching Pro Accountability 0.65 Anti Accountability 0.44 Pro Accountability 0.42 Anti Accountability Pro Accountability 0.67 Anti Accountability Pro Note. Items in plain type reflect pro-formative tendency, while items in italics reflect anti-formative tendency; SMC=squared multiple correlate; β=standardised regression weight.

10 ELE Teacher Conceptions of Assessment 10 Table 3. Mean scores for each factor Factor M se SD Aspects of teaching and learning Learning Accountability Certifying Teaching Tendency Pro-Formative Anti-Formative Aspect*Tendency Interaction Learning-Pro Teaching-Pro Accountability-Pro Certifying-Pro Accountability-Anti Certifying-Anti Learning-Anti Teaching-Anti