Using Computer-based Text Analysis to Integrate Qualitative and Quantitative Methods in Research on Collaborative Learning

Transcription

1 Using Computer-based Text Analysis to Integrate Qualitative and Quantitative Methods in Research on Collaborative Learning Pre-print draft copy of: Wegerif, R., and N. Mercer. (1997) Using computer-based text analysis to integrate quantitative and qualitative methods in the investigation of collaborative learning. Language and Education, Vol. 11. (4): ISSN: Rupert Wegerif and Neil Mercer Centre for Language and Communications, School of Education, Open University, Milton Keynes, MK7 6AA, UK. Abstract This paper argues that there are great potential benefits in incorporating computer-based text analyses into methods for researching talk and educational activity in classrooms. The first part of the paper discusses the strengths and weaknesses of some existing approaches to the study of talk and collaborative activity. The second part of the paper suggests ways that computer-based analysis of transcribed talk can integrate qualitative and quantitative methods, and in so doing overcome some of their respective weaknesses. This integrated approach is illustrated by a recent study of primary school children's talk and joint activity while working at the computer. Key Words Classroom Discourse; Collaborative learning; Computer-based Analysis; Discourse Analysis; Methodology; Reasoning. Introduction Computer-based text analysis is one of the fastest growing areas of linguistics (Stubbs, 1996). Some have even described its impact on research in linguistics as a revolution (Baker, Francis, and Tognin-Bonelli 1993). To date computer-based analysis has mainly been applied to the study of large corpora of written texts. In this paper, we propose that such methods are also of great value for research on children's talk and the joint construction of knowledge in the classroom.

2 The paper begins with a brief discussion of the strengths and weaknesses of some commonly-used methods for analysing classroom talk. One such method involves the use of coding schemes to analyse talk into functional categories. Coding schemes enable researchers to deal with large amounts of data, and normally use explicit, publicly-verifiable criteria to make categorisations. However, they are of limited value for exploring the construction of knowledge because they are not sensitive to the ways that the content and context of talk develops over time. We will refer to such methods, which depend upon coding and quantifying data according to a pre-established set of categories, as quantitative. On the other hand, there are also well-established qualitative methods of discourse analysis which generally eschew the categorical classification and quantification of data and rely essentially on the interpretative analysis of transcribed speech. These qualitative methods of discourse analysis can be sensitive to content and context, but have been criticised for the apparent reliance of their exponents on the detailed interpretation of short excerpts selected from larger, unseen speech corpuses. We believe that the incorporation of computer-based methods into the study of talk offers a way of combining the strengths of quantitative and qualitative methods of discourse analysis while overcoming some of their main weaknesses. We will illustrate our claims for the value of computer-based discourse analysis by applying it to the study of talk amongst pupils in primary school classrooms. For this study we used a software package specifically developed for analysis of transcripts of talk which offers the researcher the possibility of moving quickly between a detailed focus on a small section of transcript and an analysis of the contexts of all occurrences of selected linguistic features throughout the whole transcript or series of transcripts. In the research described, this software was used to link a detailed analysis of the quality of children's talk with quantitative data of the same children's scores on a test of reasoning. Features of the talk of groups of children were related to their success in working together to solve reasoning problems. In this way the old dichotomies of process and product, quantitative and qualitative, were at least to some extent transcended. The Paradigm Debate in Classroom Discourse Analysis Coding and counting There are methods for analysing classroom talk, which have developed from the well-established methodological tradition commonly called 'systematic observation (Croll, 1986), in which talk data is reduced to coded categories which are then statistically compared. The particular set of categories employed vary according to

3 the focus of the research study. Teasley (1995) offers a recent example of this type of method, applied to the study of collaborative learning. In Teasley s study the talk of children working in pairs on a problem-solving task was transcribed and each utterance attributed to one of fourteen mutually exclusive categories. These categories included such functions as 'prediction' and 'hypothesis'. Transcripts were coded independently by two coders and the level of agreement measured to ensure reliability. A count of categories of talk in different groups was correlated with outcome measures on the problem solving activity in order to draw conclusions about the kinds of utterances which promote effective collaborative learning. There have been many other studies of collaborative learning which have used some version of this coding approach to analysing talk. King (1989), for example, used measures such as length of utterance as well as pragmatic functional categories to investigate variables affecting the success of collaborations. Kruger (1993) counted utterances considered indicative of 'transactive reasoning' and correlated their incidence with measures of the success of children's problem solving. Barbieri and Light (1992) similarly measured the incidence of plans and explanations expressed in talk, while Azmitia and Montgomery (1993) looked for talk features indicative of scientific reasoning. And, drawing on the neo-piagetian concept of 'socio-cognitive conflict' (Doise and Mugny, 1984, and Perret-Clermont, 1980), Joiner (1993) counted the number and type of disagreements in interactions and related these to problem-solving outcome measures. These and other studies using similar coding methods have produced interesting and valuable results. Their strength, as opposed to more qualitative methods discussed below, lies in their capacity to handle large corpora of data, to offer explicit criteria for comprehensively categorising the whole of a data set, to offer a basis for making systematic comparisons between the communicative behaviour of groups of children and similarly, to offer a basis for relating this behaviour to measures of the outcomes of collaborative activity. However, critics of such approaches can point to certain weaknesses and limitations, which we will now summarise. Critiques of coding approaches to the analysis of talk Both the usefulness and the validity of coding methods have been questioned. Edwards and Mercer point out (1987, p11) that coded analysis is often presented as a fait accompli and that the prior interpretative movement that generated the codes from the data is often obscured or forgotten. Draper and Anderson (1991) go further to identify four specific kinds of problem:

4 1) Utterances are often ambiguous in meaning, making coding difficult or arbitrary. 2) Utterances may have indeed often have multiple simultaneous functions, which is not recognised by most coding schemes which normally involve the assignment of utterances to mutually-exclusive categories. 3) The phenomena of interest to the investigator may be spread over several utterances, and so any scheme based on single utterances as the unit of analysis may not capture such phenomena. 4) Meanings change and are re-negotiated during the course of the ongoing conversation. It might be thought that using two or more independent coders and measuring their level of agreement overcomes the question of validity raised as the first problem above. Indeed, coding schemes are often used in preference to other discourse analysis methods because they appear to offer a more 'objective' basis for validity claims. But, as Potter and Wetherell (1994) point out, this widely-held belief confuses the reliability of a measure with its validity. That two or more coders can consistently agree on how to code different classes of ambiguous utterances tells us only that they have a shared way of interpreting utterances - it tells us little if anything about the validity of their way of interpreting utterances. If, as is often the case, such researchers offer no examples of the utterances they have coded in their original discursive contexts, readers of their research reports have to take the validity of any interpretations entirely on trust. Moreover, Potter and Wetherell argue that talk is inevitably and necessarily ambiguous in its meanings because it is a means by which shared meaning is negotiated. Crook (1994) emphasises that this limitation with coding schemes is particularly serious in the study of collaborative learning. He argues that the study of collaborative learning should be the study of the development of shared knowledge over time. Coding schemes for talk fail to capture this crucially important temporal dimension in co-operative activity, and tend to reduce collaborations into atemporal inventories of utterances (ibid. p 150). In research on collaboration, coding schemes are often used to search for correlations between the incidence of some kinds of talk behaviour and particular outcomes of joint activity (for example, success or failure in solving problems). But while coding methods can show a statistical relationship between two events in time, i.e. that event B generally follows event A, they are not good at demonstrating causal

5 relations between two events, i.e., how and why event A led to event B. For example, King's (1989) finding that there is a statistical correlation between the incidence of task-focused questions and group success in problem-solving is interesting, and suggestive of a causal link; but that kind of analysis does not in itself explain how such a link is achieved. To explain such a relationship, a researcher would have to show exactly how asking questions helped the groups of learners to solve the problems. Qualitative Approaches to the Analysis of Talk and Collaborative Activity in Classrooms Douglas Barnes (Barnes 1976; Barnes and Todd 1978; 1995) was amongst the first researchers to devise an analytic method for studying collaborative learning in classrooms which was sensitive to context and to the temporal development of meanings. In contrast to the coding approaches described above, Barnes has used detailed classroom observation and the interpretation of transcribed talk of children engaged in normal classroom tasks to explore the processes through which knowledge is shared and constructed. His approach is allied to ethnography in that it incorporates intuitive understanding gained through discussions with teachers and children and participation in the contexts described. His usual method of reporting his research is to demonstrate and illustrate his analysis by including transcribed extracts of talk, on which he provides a commentary. Since Barnes' pioneering work, many other educational researchers have developed similar methods of discourse analysis, and some have applied them to the study of children's talk and joint activity (e.g. Lyle 1993; Maybin, 1994; Swann, 1994; Mercer, 1995). Critique of qualitative discourse analysis In their comprehensive review of methods for researching talk in classrooms. Edwards and Westgate argue that the strength of Barnes early work lay in making easily taken for granted aspects of classroom life visible and so available for reflection and that the value of this can be seen in the recognition his insights gained immediately from many teachers (Edwards and Westgate 1994, p58). However they also quote many critics of such 'insightful observation' methods (ibid. p108). It is easy, they write, to pull transcript evidence out of context in order to illustrate a case already made and so to offer only the illusion of proof. Stubbs similarly argues that while studies based on the presentation of fragments of recorded talk can be insightful and plausible they raise problems of evidence and generalisation (Stubbs 1994). It is often not clear, Stubbs continues, how such studies could be replicated

6 and compared or how they could lead to cumulative progress in the field. While we are not convinced that Stubbs' own sociolinguistic analyses are appropriate to the investigation of shared understanding, his criticisms of fragment-based discourse analysis are particularly relevant to our concerns here - and all the more so because they lead him to advocate the use of computer-based text analysis. Qualitative discourse analysis in the tradition of Barnes must rely on presenting short selected texts. Yet educational research often seeks generalisations, and evaluative comparisons, which cannot rest only on these samples. This is why, as Hammersley has argued, qualitative analysis can be effective for generating theories but not so effective for rigorously testing them (Hammersley 1992). In contrast, the quasi-experimental research designs which are often associated with the use of coding schemes and other quantitative measures can offer explicit tests of hypotheses and systematic comparisons between the communicative behaviour and outcomes of 'target' and 'control' groups. The strengths of quantitative and qualitative methods of discourse analysis in the field of collaborative learning appear on the surface to be complementary. However, as Snyder points out in her discussion of integrating multiple perspectives in classroom research, (Snyder 1995) different methodologies can be taken to embody different views of the nature of meaning. The act of coding a text into a limited number of discrete categories, for example, tends to imply a view of the meaning of utterances as relatively stable, unambiguous and independent of context. Many qualitative researchers, on the other hand, insist that the meaning of any utterance depends upon the way it is interpreted by participants in the collaboration and so is not only highly sensitive to context but also necessarily always ambiguous (Potter and Wetherell, 1994; Graddol, Maybin and Stierer, 1994). In the next section we argue that computer-based transcript analysis offers a way beyond this apparent divide through enabling a more context sensitive approach to combining qualitative analysis with systematic comparison and evaluation. Interrelating Different Levels of Data Computer-based text analysis Computer-based tools such as concordancers are increasingly used in linguistics to explore changes in word meaning and create modern dictionary entries (Graddol, Cheshire, and Swann 1994). Software is being developed making it possible to apply similar techniques to transcripts of discourse without the need for complex coding.!kwictex, for example, a concordancer designed by Graddol, allows the researcher

7 to move almost instantly between full transcripts and lists of utterances, or other contextual unit, containing key words in context. Graddol argues that the use of Key Word In Context (KWIC) searches of electronically stored text can dramatically speed up the iterative cycle of exploration and testing involved in any analysis of discourse (Graddol, in preparation). This iterative cycle often combines close exploratory qualitative work with generalisation and testing of hypotheses about linguistic features across the whole of a text or series of texts. The use of such techniques for written transcripts of spoken language offers the possibility of systematic comparisons of language use in different settings without losing sight of the relationship between particular linguistic features and their context within transcripts. Different levels and types of abstraction in the data Data can be more or less abstract: more or less concrete. A count of different functions of utterances in a transcript is more abstract, in the strict sense, than the full transcript itself. That transcription is in turn an abstraction from an audio recording which may in turn be an abstraction from a full video and audio recording. Any video/audio recording of events is the most concrete level of data available to language researchers after an event. However, recorded data is evidently itself abstracted from a more concrete original event. The very concept of recording assumes that there is a more concrete original event or events which is recorded through being abstracted in some way. The nature of this original level of actual events, a level which we posit as degree zero on the scale of concrete to abstract, is a matter for philosophical debate which need not concern us here. In research we are always dealing with data which has some degree of abstraction. Not all types of data can be easily placed in relation to each other on this continuum. A brief narrative account of a week s training course in problem solving, for example, is not evidently any more or less abstract than the numerical results of a questionnaire survey taken at the end of the course. Here each type of data draws out different aspects of the concrete event. The time required for analysis and the space required for presentation mean that there is a de facto relationship between degree of abstraction useful in the data and the sample size of a study or the degree of generalisation. More concrete data such as video-recordings of events cannot be used to generalise across a range of events without abstracting and focusing on some key features from each event.

8 The proposed method We have seen that quantitative coding methods which have emerged from the 'systematic observation' tradition of research on classroom talk have been criticised for their failure to handle the development of contextualized meanings which is at the essence of talk. Qualitative methods of discourse analysis, on the other hand, have been criticised for not incorporating a systematic basis for abstracting and generalising from specific episodes of discourse. It was also suggested that while coding methods (when incorporated into experimental or quasi-experimental research designs) offer an efficient means for correlating features of observed events with outcomes, they were weak in explaining any process by which such correlations might arise. Qualitative methods, in contrast, are good for showing how specific events unfold and arrive at their actual conclusions, but are not easily used to demonstrate the superiority of one theory, or generalisable account of such events, over another. It seems, then, that the difficulty in effectively combining the strengths of quantitative and qualitative methods of discourse analysis in the study of collaborative learning is the problem of integrating different levels of abstraction in the data. Coding operates at too abstract a level of data, and so the explanatory power of an analysis of the quality of the talk in particular events is lost; while insightful observation and other similar methods of discourse analysis lack a systematic means of moving beyond the concrete and particular. The value of computer-based text analysis, we suggest, lies in its ability to perform a mediating role between these two positions. Concordancing software such as!kwictex enables a rapid movement between different levels of abstraction. Units of analysis such as words, utterances or conversational turns can be abstracted from the transcript to form a separate list. At the same time, these words or utterances are highlighted in the main text, which can be returned to instantly at any time. Clicking on an abstracted conversational turn in a list, for example, returns the analyst immediately to its place in the full transcript. At one extreme this software offers the possibility of a quantitative breakdown of speakers conversational turns, or other selected features such as the occurrence of particular key words or types of utterance; while at the other it offers the possibility of carrying out a close, detailed analysis of full transcripts. The use of such a tool therefore facilitates the inter-relation of different levels of abstraction in the data. An essential and attractive feature of the method we are proposing here is that it maintains a connection between relatively concrete data, such as recordings of events, and relatively abstract data such as word counts or test scores. In this way the quantitative analysis of test scores, for example, does not replace a qualitative analysis of collaborative interactions but both co-exist and contribute to an overall

9 understanding. With regard to the dissemination of research, this new methodology enables dicourse analysts to present their findings in ways that should be more explicit and convincing to a critical audience. Any qualitative analysis is not represented only by the illustrative interpretation of a very few selected events but is supported by a more systematic analysis of the data as a whole, and possibly also by a demonstration of statistical relationships between features of linguistic events and other kinds of measures of collaborative activity (in this case, scores on group reasoning tests). A study illustrating the proposed method To support the claim we are making for the value of combining the use of computer-based text analysis with more established qualitative methods of discourse analysis it is necessary to look at how this approach works in practice. To that end we present some aspect of a recent research project which used this approach in the context of evaluating an intervention programme. The intervention programme consisted of a series of eight lessons coaching exploratory talk, defined as talk in which reasons are given for assertions and reasoned challenges made and accepted within a co-operative framework orientated towards agreement (as discussed in more detail in Mercer 1995; see Wegerif and Mercer, 1996, for a fuller account of this study). Each lesson integrated three stages. They began with the teacher explaining and modelling the type of interaction being taught continued through small group exercises designed to encourage the children to practice the type of interaction being taught and finished with whole class discussions both reflecting back on what had been learnt and using exploratory talk in a realistic context. The series of lessons began with exercises to encourage cooperation and listening to each other. One early lesson, for example, had the children sitting back to back in such a way that only one could see an object which she had to describe to her partner in order that her partner could draw it. Later lessons coached ground-rules of exploratory talk, for example that reasons should be given for assertions, that all should be asked for their views and listened to with respect or that several alternative possibilities should be generated and discussed before an answer to a problem is reached. The intervention programme included some computer software designed to present information and problems in a way that encouraged children to formulate hypotheses, share information, question assumptions and reach joint decisions where the content of their discussions was directly relevant to the National Curriculum for England and Wales. In the area of citizenship, for example, moral dilemmas were presented on the screen with a number of different possible views in the form of the thought of participants to

10 support the children in debating alternatives and embedded within an engaging narrative frame. In the area of science the software both encouraged the children to formulate predictions before changing variables in a simulation and then asked them for reasons why their predictions succeeded or failed. (Some principles behind the design of this software are put forward in Wegerif, 1996a, and the software itself is described more fully in Wegerif, 1996b). The evaluation of this intervention programme combined an analysis of classroom talk and interaction throughout the programme with the use of a pre- and post-intervention comparison of children s problem solving (as described below). This pre- to post-intervention comparison used two kinds of data: scores from a group reasoning test and an analysis of the recorded talk of certain focal groups of children video-taped while doing this reasoning test. The use of video made it possible to relate the children s talk to the answers they gave to particular problems in the test. With this research design it was possible to statistically link changes in test score measures to changes in linguistic features, in a similar way to many coding and counting studies, but it was also possible to relate extracts of transcripts of groups talking together to their work on specific problems, the sort of study normally found in the qualitative discourse analysis tradition. A control class of children of the same age in a neighbouring school were also given the group reasoning test, before and after the intervention had taken place in the target school. Pre- and post-intervention reasoning test results The reasoning test given to both the target class and a control class, at the beginning of the intervention programme and again at the end, consisted of problems taken from a wisely used psychological test of reasoning, Raven s Standard Progressive Matrices. Each Raven s problem involves matching shapes and patterns. (Wegerif and Mercer, 1996, contains more information about this test). The children (all aged 9 and 10 years old) worked together in groups of three. In the target class nine groups produced comparable pre- and post- intervention tests, while in the (smaller) control class five groups produced comparable pre- and post-intervention tests. There was one question sheet and answer sheet per group and children were encouraged to talk together to reach a joint solution. In each case the first two questions were used to explain the tests and the tests did not begin until it was clear that all in the class understood the procedure. All the group scores in both target and control classes increased over the period of the intervention programme. The target class group scores increased by 32% while the control class group scores increased by 15%. The differences between the pre-

11 and post-intervention test scores for all groups in the target class were compared to the differences between the pre- and post-intervention-test scores for all groups in the control class and it was found that this difference was significant (Z = p = One-tailed Mann-Whitney test, corrected for ties). (These results are presented in more detail in Wegerif, 1996). Figure 1. Comparing the pre- to post-intervention change in the means of the target and control group reasoning tests Transcript evidence The test results appeared to show that group effectiveness in problem solving had increased. However, on their own, these highly abstract statistics do not tell us why this change has occurred. To demonstrate a connection between the outcome measures and the coaching of exploratory talk it is necessary to look at changes in quality of the interactions of the groups. Video-recordings of three focal groups of children in the target class (i.e., the class which had experienced the intervention

12 programme) were made when they did the initial, pre-intervention reasoning task and again when they did the same task again after the intervention It was thus possible to study the talk of the groups as they solved problems the second time that they had failed to solve the first time. The following two transcripts give an illustration of this more concrete and contextualised type of analysis. Figure 2: A11 The pre-intervention talk around problem A11 John: (Rude noise) Elaine: How do you do that? Graham: That one look All: It s that (Elaine rings 1 as answer for A9) Elaine: No, because it will come along like that (Elaine rings 5 as answer for A11) John: Look it s that one (Elaine rings 2 as answer for B1) The post-intervention talk around problem A11 John: Number 5 Graham: I think it s number 2

13 John: No, it s out, that goes out look Graham: Yeh but as it comes in it goes this Elaine: Now we re talking about this bit so it can t be number 2 it s that one Elaine: It s that one it s that one Graham: Yeh 'cos look Elaine: 4 Graham: I agree with 4 (Elaine rings 4 as answer for A11) Commentary. In the pre-intervention task question A11 was answered wrongly in the context of a series of several problems which were moved through very rapidly. The other problems in this short series were answered correctly. Elaine s second utterance No, because it will come along like that implies that one of the other two group members had just pointed to a different answer on the page. She gives a reason to support her view and this is not challenged. There is no evidence that agreement is reached before the answer is given. The group move on to the next problem. Looking at the full transcript it is apparent that the children do not take the task set very seriously and much of their talk is off-task. In the post-intervention task episode much more time is spent by the group on A11. Two alternatives are considered and rejected before the right answer is found and agreed on. This is crucial. In the pre-intervention task example only one alternative was considered and rejected before a decision was reached. The structure of the problem is such that, to be sure of a right answer it is necessary to consider at least two aspects of the pattern. John first spots the pattern of the dark vertical lines moving outwards and so suggests answer 5. Graham contradicts John saying the answer must be 2 presumably basing this on the pattern of the lighter horizontal lines. Just as Graham s reason means number 5 is wrong so John s reason means that number 2 is wrong. In proposing number 4 Elaine is building on these two earlier failed solutions. Graham sees that she is right and points to confirming evidence on the page. In the context of John s vocal objections to previous assertions made by his two partners his silence at this point implies a tacit agreement with their decision. Both episodes appear to contain talk of the kind that we have called exploratory. That is, challenges are offered, reasons are given and the group appear to be working co-operatively towards a shared goal. However the second episode includes a much longer sustained sequence of exploratory talk about the same shared focus.

14 It was generally found to be the case that the problems which had not been solved in the pre-intervention task and were then solved in the post-intervention task, leading to the marked increase in group scores, were solved as a result of group interaction strategies associated with exploratory talk and coached in the intervention programme. Key Word in Context (KWIC) Analysis We have presented above, in some detail, our analysis of how one group of children solved one problem in the post-intervention task which they had failed to solve in the pre-intervention task. We have used this analysis to support a claim that the children s use of exploratory talk was a significant factor in their finding the solution of the problem. However, for reasons discussed in the early part of this paper, this kind of claim (even if elaborated) may not be convincing to some educational researchers, who might argue that we simply chose such an example of talk becuase it supported, rather than tested, our claim. This is where the value of using a computer-based analysis to augment a qualitative study can be appreciated. The software we used,!kwictex, enabled us to extract all uses of specific linguistic features across the full transcripts of this group of children in the immediate context of utterances in which they occur. In the extracts shown above, focal group 1 solving problem A11, the use of the word 'because' or 'cos'' was found in both the pre-intervention and the post-intervention example. In the following transcripts all the occurrences of this key word in the pre-intervention task and the post-intervention task for this group are listed within the context of one conversational turn. Focal group 1 pre-intervention task use of 'úcos or because Elaine: It isn t 'cos look that s a square Graham: No 'cos look watch there all down there and they are all at the side and they are all up there Elaine: Wait wait wait its that one 'cos look its them two and them two () and them two John: 'Cos look that goes out like that - Elaine: 'Cos look that goes in John: 'Cos look that goes too far out Graham: Look 'cos that s got 4

15 Elaine: John: Elaine: Elaine: Elaine: (12) No... not that one not that one because its got a little bit like that its that one look - it goes in and then it goes out No its isn t because its there No because it will come along like that Could be that one because look stops at the bottom and look It isn t It isn t because look Focal group 1 post-intervention task use of 'cos or because Graham: Number 6 'cos 6 stops in there 'cos look if you Elaine: It can t be there 'cos look if you done that Elaine: It is look if that goes like that and then it has another one 'cos those two make Elaine: He doesn t say what they are 'cos he might be wrong Graham: Yeh 'cos look Elaine: 'Cos it would go round John: It is 'cos it goes away 'cos look that one goes like that Elaine: No it can t be 'cos look... with the square with the triangle you take away the triangle so you re left with the square so if you do just this and then again take that away it s going to end up, like that isn t it? Graham: Actually 'cos that s got a square and a circle round it John: Yeh 'cos it goes like that and then it takes that one away and does that Elaine: No 'cos look Elaine: Probably one in the circle 'cos there are only two circles Graham: 'Cos if they are lines and then they are going like that it is because they are wonky isn t it Graham: No actually it ain t 'cos then Elaine: Yeh its number 8 because those ones - those two came that those two make that John: No because 1, 2, 3 1, 2, 3 John: No because that goes that way and that goes that way Graham: No because it s that one (21) Commentary. For the same group of children, there were more uses of 'cos or because to link reasons to claims in the post-intervention task than in the pre-intervention task. In the pre-intervention task two-thirds (8) of the usages are collocated with look - that is either because look or 'cos look. In the post-intervention task this collocation is less frequent. It occurs 6 times which is less than one third of the total uses. When

16 collocated with look, because' or 'cos has a deictic function akin to non-verbal pointing. In this case the reason or warrant is usually to be found outside of the talk in the physical context, i.e. on the problem book page. In these utterances there is a noticeable shift between the pre-intervention condition and the post-intervention condition from using because/'cos in this way to using it to link claims to verbally elaborated reasons which are more evident in the talk. In other words, there is a shift from reliance on the physical context to greater use of the linguistic context which the children construct together. Our key word in context analysis took one linguistic features which a detailed qualitative analysis had indicated was associated with one group s solution of one reasoning problem, and tested the generality of this association across the transcripts of this group in the pre-intervention and post-intervention tests. The generalisation was made possible through the partial abstraction of listing the utterances in which the key word occurred. This showed a marked shift in the number of times this key word was used from the pre- to the post-intervention condition. It also showed an evolution in the way this key term was used in context. The use of!kwictex in this way makes it possible to explore rapidly the contextualized use of such linguistic features. A Count of Key Usages Our key word in context' analysis showed how the general occurrence of specific words can be tracked over a set of related transcripts (in this case, those representing one group of children working on one task). In disseminating research, it is difficult to present many such analyses in a way that does not take up much space and overburden the reader. To generalise the analysis further to include several key terms hypothesised as marking the occurrence of exploratory talk, and to do so for all the transcripts available from the three focal groups, it is necessary to move up a level of abstraction and make a count of key usages. A key usage is not simply a key word but a key word being use to serve a particular function. A key word count alone is not adequate for words which can have many functions and where there is reported speech in transcripts, for example words being read from instructions. The use of!kwictex to look at a word in its immediate context makes it easier to ascribe a function to that word. In the context of the pre- and post-intervention group reasoning tasks the following list of key usages were found, through qualitative analysis of the kind shown above, to be indicative of exploratory talk: if used to link a reason to an assertion

17 so used to link a reason to an assertion because/'cos used to link a reason to an assertion questions used both to challenge, as in why? questions, and to be socially inclusive, as in what do you think? type questions. Table 1 shows counts of these key usages for the talk of the three focal groups when doing the standard group test together. Table 1. Key usage count for the pre- and post-intervention tests of the focal groups in the target class Pre-test Post test Gp1 Gp2 Gp3 Total Gp1 Gp2 Gp3 Total Test score Questions Because/'cos So If Total words Summary of the key word in context analysis results Computer-based text analysis using!kwictex demonstrates a difference between the post-intervention task talk of the three focal groups and their pre-intervention task talk. This difference takes the form of more talk, more reason clauses using the linguistic forms because/'cos, so and if and more questions being asked. This marked change in the use of language occurs in association with an improvement in test-scores which was in line with the mean improvement in all the groups of the target class (see figure 1). Summary In the first half of this paper we proposed a method for the study of collaborative learning which was illustrated in the second half of the paper. The basis of the proposed method was interrelating different levels and types of data to produce an overall interpretation which integrated qualitative and quantitative dimensions. It was argued that the use of computer-based methods of text analysis facilitated this

18 integration through allowing the researcher to move rapidly between different levels of abstraction in dealing with transcript data. The illustration of this method given in the second half of the paper used two kinds of data and four levels of abstraction. The most concrete level of data was represented by the full transcript of each recorded event, (in the full study this was enhanced by other information from video-recording and field notes). The use of specialised computer software facilitated generation of data at the next two levels. These were the selected key words and other linguistic features in the utterances of one focal group, and a count of key usages for all three focal groups in the target class. The final level of abstraction and generality, was afforded by the results of the group reasoning tests. Interrelating these four levels of analysis enabled us to draw two conclusions. First, that the exploratory features of talk responsible for solving the problem illustrated through a qualitative analysis were features generally found more in the post-intervention talk of the children doing a reasoning test task than in the pre-intervention talk of the children doing the same task. Second, that this change in the style of talk of the children towards exploratory talk was matched by an increase in group reasoning test scores. Conclusion Using computer-based transcript analysis to help combine qualitative and quantitative methods in the study of collaborative learning can produce an overall interpretation which is more convincing than either qualitative or quantitative accounts can be if used alone. Computer-based transcript analysis has a special role to play in bringing data representing different levels of abstraction together into dynamic relationship, in which different kinds of analysis reflect upon and inform each other. The computer-based method we propose can facilitate this approach because it enables the abstraction of different levels of linguistic data without ever leaving behind the original linguistic contexts of the actual words spoken. References Azmitia, M, and R Montgomery. (1993) Friendship, transactive dialogues, and the development of scientific reasoning. Social Development 2 ( 3): Baker, M, G Francis, and E Tognin-Bonelli, eds. (1993) Text and Technology. Amsterdam and Philadelphia: Benjamin's.

19 Barbieri, M. and Light, P..(1992) Interaction, gender and performance on a computer-based task. Learning and Instruction 2 ( ): Barnes, D. (1976) From Communication to Curriculum. Harmondsworth: Penguin Books. Barnes, D, and F Todd. (1978) Communication and Learning in Small Groups. London: Routledge & Kegan Paul. Barnes, D., and F. Todd. (1995) Communication and Learning Revisited. New York: Heinemann. Croll, P. (1986) Systematic Classroom Observation. Lewes, Sussex: The Falmer Press Crook, C. (1994) Computers and the collaborative experience of learning. London and New York: Routledge. Doise, W, and G Mugny. (1984) The Social Development of Intellect. Oxford: Pergamon Press. Draper, S, and A Anderson. (1991) The significance of dialogue in learning and observing learning. Computers and Education 17 (1): Edwards, A, and D Westgate. (1994) Investigating Classroom Talk. London: Falmer Press. Edwards, D, and N Mercer. (1987) Common Knowledge: The development of understanding in the classroom. London: Methuen/Routledge. Graddol, D. (In preparation)!kwictex a computer-based tool for discourse analysis. Occasional Paper: Centre for Language and Communication, Open University. Graddol, D, J Cheshire, and J Swann. (1994) Describing language. Milton Keynes: Open University Press. Graddol, D., Maybin, J. and Stierer, B. (1994) Researching Language and Literacy in Social Context. Clevedon: Multilingual Matters. Hammersley, M. (1992) What's Wrong with Ethnography. London: Routledge.

20 Joiner, R. (1993). A dialogue model of the resolution of inter-individual conflicts: Implications for computer-based collaborative learning. Unpublished Ph.D. thesis, The Open University. King, A. (1989) Verbal Interaction and Problem Solving within Computer Aided Learning Groups. Journal of Educational Computing Research 5:1-15. Kruger, A. (1993) Peer collaboration: conflict, cooperation or both? Social Development 2 ( 3). Maybin, J (1994) Children's voices: talk, knowledge and identity. In Graddol, Maybin, and Stierer (eds.) Researching Language and Literacy in Social Context. Clevedon: Multilingual Matters. Mercer, N. (1995) The Guided Construction of Knowledge: talk amongst teachers and learners. Clevedon: Multilingual Matters. Mercer, N and R. Wegerif (In press) Is exploratory talk productive talk? In Light, P and K. Littleton (eds.) Productive interactions around computers. London: Routledge Perret-Clermont, A N. (1980) Social interaction and cognitive development in children. London: Academic press. Potter, J, and M Wetherell. (1994) Discourse Analysis and Social Psychology. London: Sage. Lyle, S. (1993) An Investigation into Ways in Which Children Talk Themselves into Meaning. Language and Education 7 (3): Snyder, I. (1995) Multiple perspectives in literacy research: integrating the quantitative and qualitative. Language and Education 9 (1). Swann, J. (1994) What do we do about gender? In B. Stierer and J. Maybin (eds.) Language. Literacy and Learning in Educational Practice. Clevedon: Multilingual Matters. Stubbs, M. (1994) Grammar, Text and Ideology: computer-assisted methods in the linguistics of representation. Applied Linguistics 15 (2):

21 Stubbs, M. (1996) Text and Corpus Analysis. Oxford: Blackwell Teasley, S. (1995) The role of talk in Children's Peer Collaborations. Developmental Psychology 31 (2): Wegerif, R. (1996a) Collaborative learning and directive software, Journal of Computer Assisted Learning, 12, 1, Wegerif, R. (1996b). Using computers to help coach exploratory talk across the curriculum, Computers and Education, Vol 26, 1-3, Wegerif, R. and Mercer, N. (1996) Computers and reasoning through talk in the classroom, Language and Education, 10, 1,