RUNESTONE, an International Student Collaboration Project

Transcription

1 RUNESTONE, an International Stuent Collaboration Project Mats Daniels 1, Marian Petre 2, Vicki Almstrum 3, Lars Asplun 1, Christina Björkman 1, Carl Erickson 4, Bruce Klein 4, an Mary Last 4 1 Department of Computer Systems, Uppsala University, Sween 2 Centre for Informatics Eucation Research, Faculty of Mathematics an Computing, Open University, U.K. 3 Department of Computer Sciences, University of Texas at Austin, Texas, USA 4 Computer Science & Information Systems, Gran Valley State University, Michigan, USA Abstract- Our stuents will eventually work in a global market; what better preparation can we provie for international collaboration than international collaboration? The RUNESTONE project is eveloping an evaluating the notion of incorporating international group projects into the unergrauate Computer Science curriculum. RUNESTONE as new imensions to stuent teamwork, requiring stuents to hanle collaboration that is remote, cross-cultural, an linguistically challenging. RUNESTONE is a three year project, with the prototype version running in winter 1998 with stuents at Uppsala University, Sween, an Gran Valley State University, Michigan, USA. The 1998 pilot stuy will be followe by a full-scale implementation in 1999 an another in Introuction The RUNESTONE project involves stuents an faculty at Uppsala University (Sween), an Gran Valley State University (Michigan, USA), an researchers from the Open University (UK) an the University of Texas at Austin (Texas, USA). The project s primary aim is to introuce real international experience into unergrauate Computer Science eucation in a way that has value for all participants. Group projects (typically 5-10 stuents per team, 5-10 weeks per project) will be incorporate into courses at Uppsala University an Gran Valley State University. These stuents will collaborate closely with their foreign counterparts using appropriate communications an computing technology to solve a given problem. Because the stuents come from ifferent specializations (all CS majors), they have ifferent knowlege to contribute to the project. Problems will be esigne to cover the spectrum of backgrouns. RUNESTONE s seconary aim is to ientify effective support structures for remote international collaboration, encompassing strategies for communication, management, an technology use. RUNESTONE will evaluate peagogical an technical solutions for collaboration, will examine the costs, both in time an money, an will investigate how stuents learn in such a setting an what they learn. This paper introuces the RUNESTONE project, escribes the support an peagogic mechanisms use, an presents preliminary observations from the first, pilot year. Further Aims The RUNESTONE project aims to: Give stuents international contacts an experience with teamwork with people from a foreign culture. Give stuents experience of collaboration with a group having a ifferent eucational backgroun. Encourage learning through peer-teaching. Give stuents experience with the use of Information Technology in problem solving. Use the foreign experience to ai stuents in proucing a superior prouct locally. Benefit staff by close collaboration with other universities, giving insights to other epartments an ieas for new teaching methos. Gain experience with use of new techniques in the running of a course. Another goal is to create a well-organize setting with courses that, after the initially higher start-up costs, run at normal or lower costs. One example of cutting costs without compromising quality is the use of stuent peer-learning, which can reuce the eman for staff hours. Another example is that the costs for renewing the course can be istribute among the epartments involve. In carrying out the RUNESTONE project, we will establish results that aress the issue of transferability to other epartments an institutions. For this reason, the evaluation will aim to istinguish between omain-specific an general lessons, particularly with respect to the impact of international collaboration on group interaction an personal evelopment, the extent of peer-learning, an the costs of using this form of eucation. For example, the project shall examine questions such as how much time is spent on becoming acquainte with new techniques for communication an in what ways (if any) using non-native language impairs learning. Peer-learning Base on anecotal evience from our own experience as teachers, we believe that having stuents explain concepts an solutions to one another is a powerful learning technique. Our conjecture is that there will be plenty of

2 occasions for the stuents involve with the RUNESTONE project to help each other with activities such as explanation, clarification, sharing knowlege or rehearsal of ieas. Occasions for peer-teaching can be formal or informal. Formal occasions arise when stuents at site X present information for the stuents at site Y. Informal occasions inclue questions that arise uring ay-to-ay or simple stuy sessions. The RUNESTONE project will examine peer-teaching an -learning systematically by consiering which settings ten to encourage or iscourage peer-teaching as well as factors that contribute to the effectiveness of peer-learning in these situations. One of our hypotheses is that the rather ifferent eucational backgrouns of the two sets of stuents involve in the project will encourage peer-teaching an - learning. The ifferences in backgrouns shoul motivate the stuents to articulate their reasoning, rather than assuming that there is mutual tacit unerstaning between them an their foreign counterparts. The pilot stuy From early January through late March 1998, the RUNESTONE project ran a pilot stuy, which involve a group of eight stuents: four in Uppsala an four in Michigan. All of the stuents were in their thir or fourth year of university stuies. For the Sweish stuents, the group project was part of a course that starte in September, whereas for the Americans it was the major part of a course that starte in early January. The problem specifie for the group project was fairly avance, involving stuy areas such as real-time systems, networking, an istribute systems. (See the appenix for a etaile escription of the group project.) A major goal of the RUNESTONE project is to examine the influence of the group project on learning, an espectially to ientify how particular factors in the project set-up affect what the stuents learn an how they learn it. Data collection Data in a variety of forms was collecte uring the pilot stuy. This paper raws on that ata to make some preliminary observations about what occurre an how to run the collaboration in the coming year. Data collection was carrie out throughout the group project an covere all types of interaction between the stuents except their informal face-to-face meetings (which were covere by the project logs kept by the stuents). For etaile exposition of these an other techniques see, for example, [1, 2 & 3]. Entry questionnaires: All stuents were aske to complete a questionnaire covering their backgrouns, expectations for the course, attitues, an learning styles. Vieo-conferences: The first meeting between the stuents on both sies of the Atlantic was via vieoconference, with both ens recore on vieotape. While we ha planne to hol a secon vieo-conference with all of the stuents after the project was over, this meeting was abanone ue to problems in synchronizing scheules. Weekly ebriefings: Each week, the teachers at both sites hel a meeting with their local group of stuents, where they reflecte on how the project ha gone uring that week. The ebriefing followe a stanar script, but was sufficiently flexible to allow the teachers to immeiately explore the stuents' observations an any new evelopments. The meetings were auio-recore. At the en of these meetings, each stuent fille out a quick, one-page questionnaire about the meeting. The questionnaire aske about the meeting organization an the outcomes (ecisions, learning, conflict resolution, clarifications, etc.), as well as about the responent's satisfaction with the proceeings, both overall an in terms of their own role in the meeting. IRC logs: The whole trans-atlantic project group hel weekly meetings using Internet Relay Chat (IRC). Logs of those meetings were collecte. Electronic mail an Web ocuments: Much of the stuent interaction about the project was via electronic mail an ocuments share on the Web. All stuent mail relating to the RUNESTONE project was collecte, an the Web site was monitore. Weekly project logs: The stuents complete weekly project logs where they kept a aily log of their time on the project, their activities an interactions uring that time, an the outcomes. Other stuents in the Sweish project course, i.e., those in groups that consiste of only Sweish stuents, were also aske to keep project logs, in orer to provie a basis for comparison. (This was not feasible for the rest of the American contingent.) Teachers' journals: Each teacher kept a journal of their observations, particularly with respect to peer-teaching an -learning, culture clashes or eveloping sensitivities, collaboration, effective or ineffective proceures, an technology issues. This collection of ata will be analyse for emergent patterns such as ecision strans, stuent roles, evience of peerlearning, an cultural factors that affect outcomes. The analysis will be ata-riven in the first instance, albeit with special attention to the state topics, an will be use to generate an analysis protocol for the subsequent years. The value of the international project will be juge primarily in terms of stuents experience, as reflecte in their recore behaviour uring the project an their questionnaire responses. It will be juge as well in terms of teachers experience (again, base on qualitative ata) an in terms of observe costs. Preliminary observations: Is the international project a goo eucation form? The funamental question for RUNESTONE is whether an in what respects this is a goo eucation form, meaning that: The syllabus is covere at least as well as through conventional methos.

3 The actual time the stuents spen on the course is relate to the allotte time. The time staff spen on the course is relate to the course size an is comparable to other ways of elivering the course. The cost of running a class is not higher than other forms. The course contributes to stuents personal evelopment. The form is motivating to stuents. We aress each of these points below. Because the pilot stuy was just that a pilot stuy any observations we make are necessarily limite an preliminary. Moreover, etaile analysis of the ata is not yet complete; the comments given below are base on on-going examination of the ata an a first-pass, topic-base review of the material, as well as on a more extensive examination of the ata generate on the Sweish sie. Performance (syllabus coverage) The coverage of the syllabus is a special case here, because the primary aim of this part of the course in Sween is to provie experience in the use of concepts covere in earlier, more theoretical parts. Hence, the completion of the project task is perhaps a better measure. Base on their performance on previous projects, the Sweish stuents involve in RUNESTONE are strong stuents.uner normal circumstances, their project woul have been preicte to have been among the first complete an best prouce by the class. This was not the case here an was, in our opinion, ue to ifficulties in coorination an synchronization among the stuents involve. Time spent on task The project logs of the Sweish stuents in the international group showe that they spent roughly the expecte number of hours on the course: the equivalent of three weeks of full time stuies, i.e., 120 hours. The American stuents spent on average somewhat less, i.e., roughly 100 hours, but this was in line with the expectations for the course the American stuents followe. It was interesting to see how these hours were actually spent, especially compare to each stuent's iniviual estimates from the backgroun questionnaire. One question ha aske the stuent to estimate, for courses taken prior to the pilot stuy, the percent of their total course time they generally spent stuying alone an in groups. For these summary figures, ing was consiere as working alone. Recategorizing ing as a group activity woul make the focus on group work even stronger. The, on average, lower procentage spent working in a group among the Americans was ue to a higher rate of local group work among the Swees. This was not surprising, because the Swees knew each other well before this course. Two stuents (one Swee an one American) provie incomplete ata an so are not reporte here. Swee 2 s reporte time inclue consierable time searching the web for useful information, which was both time-consuming an solitary. Percent of stuytime spent alone vs. in group Alone In a group estimate reporte estimate reporte Swee Swee Swee averages American American American averages Staff time an costs Because this course require new evelopment, staff time spent on the course cannot be consiere typical. The greatest evelopment cost was in setting up the project, which is stanar overhea for any project course. This offering certainly involve fewer lectures than usual an less involvement from teaching assistants. There were some special costs, for example running the vieo-conferences an obtaining special harware for the project. None of the costs was iscouraging. Stuent evelopment It is too early to say much about the effect of recent project work on stuents' personal evelopment. It is likely that the project outcomes for the stuents were not what they woul have been ha the project been iniviual or purely local. In either case, the stuents woul have expecte to complete the project, an some of them to excel. Hence, we speculate that the outcomes in personal evelopment are likely to be ifferent in kin from those of a conventional project. Our experience as teachers suggests that the experience an frustration of working in a relatively large group with unknown persons is likely to be counte as a key lesson in the long term. The stuents have ha to eal with problems that were ifferent, an in many cases more inter-personal, than usual. Each stuent appeare to reflect on his or her iniviual responsibility for communication or other problems with the project. For some, insurmountable frustration an failure to complete a project were new experiences. After the course, both the American an Sweish stuents talke about lessons in project an time management, ieas for improving the experience inclue alternative group structures, more milestones, an better inicators of progress. The American stuents escribe a lack of closure : they knew some parts of the project worke, but they han t seen it working an in t know if it worke. (The Sweish stuents, on the other han, were certain that it in t.) The stuents all realize the value of communication skills (incluing how to conuct a meeting an set an agena); perhaps the clearest lesson for the stuents was the nee to acknowlege all an to answer promptly. All of the stuents rate the project as

4 being more successful in terms of acquiring knowlege an experience than in terms of proucing a prouct. Hence, the early inicators are that the project i contribute to stuents personal evelopment. Stuent motivation Three factors enhance the initial motivation of stuents in this international group: There was a project to o. The team was international. The international project was part of an experiment. In the initial meetings, some stuents state that the real challenge was to make the group work as a team, an to emonstrate the viability of the experiment; others cite both the teamwork an the challenge of the project itself. During the project, motivation was neither constant nor evenly istribute; stuents cite ifferences in expectations an motivation within the groups as one of the main problems. At times the awkwarness of physical separation an ifferent time zones impaire stuent motivation an enthusiasm. Nevertheless, seven of the eight stuents report that they woul be willing to participate in such a project again. Aitional observations Discrepancies between the groups Much of the observe frustration can be attribute to iscrepancies between the two groups of stuents, in terms of expectations, sense of urgency, time available, local cohesion (an hence local group ynamics), technical skill, an access to a key, charismatic lecturer (an American working for the year in Uppsala). The American stuents felt that they were a step behin all the way. The Sweish stuents felt that the American stuents lacke passion. One American stuent expresse regret at not being able to contribute to the extent wante, for the reason that there was too much else (i.e., job an family commitments) going on. The American stuents perceive the RUNESTONE project as bigger than those they normally unertake; they felt that future international projects shoul make clear that all stuents must participate fully in orer for the project to succee. Stuent-ientifie problem areas Motivation, commitment: The Sweish stuents perceive the iscrepancies of commitment an motivation as the biggest problem; everything else, they believe, woul have been surmountable ha all the stuents been working with the same passion. Technical issues: The American stuents thought that technical issues were the biggest problem, especially writing coe for harware that was locate on the other sie of the Atlantic. Testing was ifficult when the local platforms iffere from the target platform. Communication: The American stuents in t perceive communication as a problem, while the Sweish stuents ientifie communication as one of the biggest problems. All the stuents were frustrate by slow or lacking responses to messages an IRC questions. The stuents cite multiple misse ealines as a major problem, although they argue that this might not have happene ha the communication been really effective. Programming language knowlege: All stuents mentione that some extra programming language competence (C or Java) woul have been esireable. Problem efinition: All stuents sai that unerstaning the problem to be solve in the project was something that cause extra work. The American stuents rarely use opportunities to ask the Sweish-base teachers for clarification; the Sweish stuents were unable to iagnose the misunerstanings of the American stuents quickly. Single physical version of equipment: There was only one physical version of the equipment, which was locate in Sween. This, an the fact that the teachers most involve in the course were in Sween, put the American stuents at a isavantage. (This will be change next year.) Communications technology None of the stuents consiere the communication meia as problematic. We trie a number of ifferent forms of collaboration; IRC an were the preferre moes of communication. [4 & 5] report experiences with tools that support project-base learning. IRC: All of the stuents felt that IRC was the best way of communicating with their counterparts. It contribute a liveness to the communication an often left the stuents feeling that they ha mae progress. They appreciate being able to have ifferent conversations going simultaneously an to review the log. Other benefits of the IRC were that it allowe time to think before answering, an that silence wasn t awkwar compare to more irect forms of collaboration, e.g., vieo an auio. As one of the stuents observe: is great it s as goo as you make it. The American stuents observe that woul have been their main meium even if the project were American-only. Web pages: The stuents use Web pages to share ocuments an to keep recors. However, they rejecte other possible Internet-base communication tools; electronic whiteboars an CUSeeMe were iscusse but not trie. Vieo-conference: The initial vieo-conference was not particularly useful, largely because the connection was quite poor. The fact that the stuents requeste a project-en vieo-conference shows that they foun some value in this meium. Experience reporte elsewhere [6] inicates that an initial social kick-off helpe subsequent communication. Auio-conference: Provision was mae for weekly auioconferences between the stuents (using speaker phones). The first try was not useful, an no further attempts were mae. Language Language per se was not a barrier for these stuents. The Sweish stuents are highly competent English speakers (with 8-9 years of stuy an English usage require in many university courses), although they are not necessarily fully confient. The stuents an IRC logs are full of jokes

5 but the stuents expresse low confience that their jokes were unerstoo. Everyone was fiercely polite. National culture The stuents notice a few cultural ifferences between the two groups, specifically in these areas: Eucational backgroun (e.g., lack of knowlege of C an use of functional programming) Age (the Swees being oler: vs. 20). External obligations; the Americans perceive that they ha more job an family obligations, although some of the Sweish stuents work as consultants, i.e., the groups actually worke uner similar conitions. Nevertheless, the stuents were emphatic that culture was a non-problem ; each group escribe their counterparts as being "just like" or "pretty much like" them. It seems likely that the pilot stuy simply i not reveal sufficient clues here to inicate the role which culture plays; it is ifficult with a small sample to istinguish between iniviual ifferences an group trens. Team coherence an roles The American cohort was a collection of iniviuals, whereas the Sweish cohort worke in concert as a team. The American stuents escribe a sense of working on an iniviual basis. While roles were assigne rather late, there was a goo international istribution of responsibilities. The groups recommene that in the future we appoint a stuent at each site as local project leaers. Designate responsibilities for these two stuents woul inclue acting as principal liaison an watching for problems within the local cohort or the overall interaction. This monitoring function might catch problems earlier an help to efuse them; for example, this year the Sweish stuents helpe one another with programming an technical ifficulties, preventing these factors from becoming problems. Social interaction There was relatively little social interaction between the cohorts; the stuents felt that they in t know their counterparts very well, an the project in t help them to get to know each other. Some interactions woul probably have been more efficient if the participants ha known each other better. Social interaction jokes an talk about personal topics increase towar the en, uring the hectic efforts to make the project fly. Yet, for each of the stuents, some part of the process or of their counterparts actions or interpretations remaine mysterious. Peer-learning Peer-learning between the cohorts was limite; it was largely relate to craftsman skills, e.g., better technical solutions. This may be accounte for by the lack of familiarity between the stuents an possibly by the nature of the project, which coul be sub-ivie in a way that avoie the nee to learn about what the others were oing. Some of the Swees reporte peer-learning within the Sweish cohort, but this occurre largely in face-to-face interactions about which no ata was collecte. Conclusion No reliable conclusions can be rawn from a small pilot stuy. However, this trial oes suggest changes for the next phase in the RUNESTONE project. Some example changes inclue having equal resources at the two sites, incorporating a clearer technical an project management briefing in orer to achieve a faster an appropriately structure project start, making clearer suggestions about targets an milestones, an recommening a ifferent team management structure. Before next year, we will investigate other means of collaboration. For example, we are intrigue by the "CoWeb" (Collaborative Website) concept from Georgia Tech (see /myswiki). In a CoWeb, any page (incluing both text an graphics) can be eite by any user. There is still much to unerstan about how international collaboration influences the learning process. We hope that the more etaile analysis of the full ata will reveal more about factors that affecte the nature of the social interactions within the team an will provie examples of peer-learning opportunities taken an misse. The ata collection schemes themselves will be evaluate, with the goal of introucing new methos that are faster to carry out an more attune to this particular project. Scaling the project up to inclue the whole class in Uppsala next winter appears feasible. Overall, the pilot stuy was a qualifie success: the technology an interaction were emonstrate to be feasible. However, the project was not quite complete, an the stuents experience frustration associate with group interaction (particularly with the international interaction). Nevertheless, the stuents all report that they learne a great eal, an all but one reporte that they woul volunteer again for an international group project. Interestingly, the frustrations were largely attribute to iniviual ifferences (style, personality, commitment, an expectation), an a perceive imbalance of resources (key resources being locate only in Sween), rather than language, technical, or cultural factors although this perception in particular will require re-examination when more stuents are involve. The nee for faculty on both sies who know the technical content of the project was apparent, but woul be interesting to fin ways aroun this in future offerings. Acknowlegements The RUNESTONE project woul not be possible without the generous participation of the stuents; we are grateful to all the stuents who participate. We thank the Sweish Council for Renewal of Unergrauate Eucation, which sponsors the RUNESTONE project.

6 References [1] Isaac, S., an Michael, W.B. (1989) Hanbook in Research an Evaluation for Eucation an the Behavioral Sciences. EITS Publishers. [2] Gall, M.D., Borg, W.R., an Gall, J.P. (1996) Eucational Research: An Introuction. (Sixth eition) Longman. [4] Kehoe, C., Guzial, M., an Turns, J. (1997). What We Know About Technological Support for Project-Base Learning, In Proceeings of IEEE Frontiers in Eucation Conference. Pittsburgh, PA: IEEE. [5] Guzial, M., an Turns, J. (1997). Supporting sustaine iscussion in computer-supporte collaborative learning: The role of anchore collaboration. Journal of the Learning Sciences (Submitte). [3] Denzin, N.K., an Lincoln, Y.S. (Es.) (1994) Hanbook of Qualitative Research. Sage. The actual project in the RUNESTONE pilot stuy was to navigate a steel ball through a maze by tilting the maze in two imensions with stepper motors. The user submits a navigation algorithm, efines a path for the ball to follow, requests the server to execute the algorithm, then waits for access to the game. When the user gains access, the game server resets the ball in the maze, executes the user's navigation algorithm, then provies feeback to the user on the result of the run. Feeback inclues information on how the navigation coe execute, an a graphical isplay of the path which the ball trace through the maze. The input to the navigation algorithm is the position of the ball. The output is the rotational positions of the motors as a function of time. Vieo images of the maze an ball are available from a black an white igital vieo camera. The harware components available for the projects were locate in Uppsala. The central piece consiste of a esktop computer, with a black an white igital vieo camera attache to its parallel port an an Ethernet connection to a laptop computer (the Sweish stuents each ha access to a personal laptop). One of the laptops was use to communicate with the two rotational stepping motors via its serial port. The camera was permanently mounte over the Brio maze game as was a light source. A secon laptop, connecte via Ethernet, was use to run as a client computer with a web browser for playing the game. The software components inclue a C library of coe to rea vieo signals from the parallel port, control camera settings, Motif app (Ximprov) for viewing camera ata, experimenting with camera settings, an example C program using camera ata (Ximprov), an Apache HTTP server for Linux, an Linux JDK 1.1 with RMI support The starting point for playing the game (running the maze) was a website. This website ha to: Display the currently installe maze boar. Allow user to efine a path for the ball to follow. Accept user's navigation algorithm to execute. Give feeback to user uring the run of the maze. Optional extras were to: Provie information to a user on eveloping navigation algorithms. Appenix: The Group Project [6] Petre, M., Carswell, L., Price, B., an Thomas, P. (1997) Innovations in large-scale supporte istance teaching for the Internet: transformation, not just translation. In Proceeings of IEEE Frontiers in Eucation Conference. Pittsburgh, PA: IEEE. Notify a user about game queue, estimate waiting time, etc.. Use RMI technology appropriately. Display graphical representation of a run, superimpose on selecte path. Display a full vieo image of the maze. The game server neee to be a concurrent system, either multiple processes or multiple threas, an ha to: Maintain a queue of users who wish to play, insuring mutually exclusive game semantics. Accept navigation algorithms an selecte paths from clients. Provie feeback to client on the success/failure of navigation coe. Provie ata to the client on the ball's movement uring a run of the maze. Provie a framework in which navigation coe executes preictably an safely. Be able to reliably reset the ball to the ocumente starting position. Drive the stepper motors via a serial port interface. Use priority to scheule the concurrent entities properly. Fetch ball position information from the vieo server an make available to navigation coe. Optional extras were to: Provie information to clients about game queue, estimate waiting time, etc.. Provie a ocumente framework in which navigation coe executes. The vieo server ha to consist of one or more processes which must: Rea vieo frames from the camera as fast as possible. Reuce vieo ata to an x,y location of the ball on the maze. Make ball position information available via a network connection. Optional extras were to: Provie a grayscale vieo image of the maze via a network connection. Provie as many positions upates per secon as possible.