i-feedback: Getting Instant Feedback From Students to Improve Learning and Teaching

Transcription

1 i-feedback: Getting Instant Feedback From Students to Improve Learning and Teaching C.K. Leung, PhD Assistant Professor, Department of Electronic and Information Engineering Hong Kong Polytechnic University, Hong Kong Josephine Csete, PhD Senior Educational Development Officer, Educational Development Centre Hong Kong Polytechnic University, Hong Kong Collecting (and using) formative feedback to improve students learning is a good idea that is difficult to do well in actual practice. This paper reports on an online instant feedback system that is designed for teachers to quickly and easily customize and administer an online questionnaire survey for students. The feedback results are collected and analyzed by the system. Based on the survey results, which are made available in a timely manner by the instant feedback system, the teacher can make prompt course improvements for the same class of students who have given the feedback. Key words: formative evaluation, system demonstrations Introduction Scenario 1 Scenario 2 It is the last lecture in the semester. As the clock on the wall is ticking towards 3:10 P.M. on a drowsy afternoon, Alan, an assistant professor who has joined the department for barely one year, is anxious. Alan s anxiety is not unfounded: his class will be filling out the Student Feedback Questionnaire (SFQ) in a few minutes' time, commenting on his teaching. In contrast, the students are experiencing neither anticipation nor expectation. Filling out the SFQ has long become a routine exercise in every class they take. They have never received feedback to the responses they make, nor will they benefit from the improvements they suggest. If changes are made, it will be in the next class, which literally means next year. The students focus is on filling out the form as fast as possible so they can hand it in and get out of the classroom for some afternoon tea in a nearby cafe. Billy is a second year university student. While he is chatting with his friend on MSN, he receives an from his subject lecturer inviting him to go to a certain Web page. Billy knows that this is an online questionnaire tailor-made by Prof. Davies to solicit comments on how class is going and on the professor s teaching. Billy loses no time in filling out the online questionnaire, commenting that the pace of the lecturing has been too quick for him to understand. He also points out, in the open-ended comments section, that the previous two assignments have been very time consuming. Billy s responsiveness is understandable: he knows his professor will read his comments and Billy will potentially enjoy the changes the teacher makes to the rest of the semester. The potential benefits to him are well worth the few minutes it takes to fill out the anonymous online questionnaire. Which of the two scenarios are more familiar to you? The first scenario is typical for collecting summative feedback. This type of evaluation is often conducted in pencil-and-paper form via a questionnaire and typically administered at the end of the semester (or academic year). Authenticity of the filled-out questionnaire is important because the purpose is judgmental. The process usually and The Annual Conference on Distance Teaching and Learning 1

2 involves many stages (starting from the point of printing off the questionnaires) and hence takes a long time to complete. Changes can only be implemented for the next class offering. There is no feedback from the teacher to the students. Provided the validity and reliability of the questionnaire is high, this type of feedback collection is appropriate for judging the teacher s performance as seen by the students. The second scenario is more suitable for collecting formative feedback. The purpose is developmental rather than judgmental. The major objective is for prompt improvement or adjustment in course delivery. The process is usually administered in early or mid-course and must be simple and time-efficient for both the teacher and the students. With ICT (Information and Communication Technology) facilities (web server, personal computers, web browser software) widely available, online administration of questionnaires is an effective way to achieve these purposes. This paper describes an Instant Feedback (i-feedback) system, which has been built and put into practice for formative evaluation purposes at a university. Setting the Scene When an instant feedback system is to be designed, it is important to prescribe the major purposes and assumptions on which the system is based. The major purposes of our instant feedback system are: formative feedback, with prompt closing of the feedback loop (students teachers) developmental (aiming at improvement rather than verdict) time-efficient (quick to set up, administer, and fill-in, for both teachers and students) short time frame (from teacher set up to feedback to students) convenient (for both teachers and students -- access any-time, any-place) The basic assumptions are: (I) students will give their genuine opinions; (II) teachers are competent in judging the validity of students comments; and (III) reliability of the instrument is not an issue (different classes will give different feedback). Assumption (I) is founded on the notion that the system is designed to collect feedback and to implement improvements promptly. There is no reason for the students to lie when giving their opinions as they know that they are likely to benefit from the improvements they suggest. Assumption (II) is important for an anonymous feedback system. For comments like the notes are not well organized, or we need more support in doing the assignments, the teacher should be competent in judging their validity (or there should be alternate channels to obtain other facts/opinions for triangulation). Assumption (III) is based on the fact that every cohort of students will have a different context and potentially different issues. Hence it is not necessarily effective for an earlier cohort s advice to be applied to a course another cohort is taking later. Online Feedback The design of an online instant feedback system is made possible by capitalizing on the new paradigm of computing with the advent of the Internet and client/server mode of operation. Were the feedback system designed with the old paradigm of computing, each user would interact directly with a stand-alone computer (or a questionnaire form). He or she would input the data and then the computer would process the data and store the results. If the system were designed to collect comments from a group of users, the data collection and analysis process could be quite tedious, posing difficulty in designing prompt improvements to the teaching and learning process. But with the client/server model, the data collection process can be deployed on a single server. The different users interact with the process remotely through the Internet and the Web browser program in their local computer. Since data are collected centrally, they can be consolidated and analyzed conveniently. Practically, each user of the system becomes an operator to input his or her own data. These two paradigms of computing are best compared and contrasted in Figures 1(a) and (b), as depicted below. and The Annual Conference on Distance Teaching and Learning 2

3 The i-feedback System The i-feedback system has been designed along the lines as illustrated in Fig.1 (b). In this system, the teacher designs an online feedback form with a template in step 1. The teacher can choose between design convenience and feedback form customization. In the extreme case of convenience, the teacher just makes several mouse clicks and a more or less standardized HTML form is put onto a web server for the students to fill-in. The process to design the online form is basically an HTML form compiler. The students are notified about the location of the online feedback form by the teacher. In step 2, the students go to a specified URL address to input their comments and opinions. The online process collects the students input and gives some response (e.g. a thank you message) to them in step 3, and analyzes the input data and archives the results in a database in step 4. The teacher then collects the analyzed results in step 5 and designs the improvements to the course according to students comments. The feedback loop is closed in step 6 when the teacher communicates back to students their compiled feedback and the planned course changes. Examples of the log-in screen, form design template, feedback form as viewed by the students online and the real time report that the teacher can view are shown in Figure 2(a) and Figure2(b) below: 4. System analyzes data Server Process Database 3. Student gets response Internet (Web) 1. Teacher designs a form 5. Teacher gets results Input Students Student A 2. Student gives comments Student B 6. Closing the feedback loop Teacher Fig. 1(a) Fig. 1(b) and The Annual Conference on Distance Teaching and Learning 3

4 (from Instant Feedback Log in Screen Instant Feedback Form Design Template Figure 2(a). i-feedback System Screen-Shots. Resultant Online Feedback Form and The Annual Conference on Distance Teaching and Learning 4

5 Figure 2(b). i-feedback System Screen-Shots. Report in MS Excel Format The Future With personal computers and the Internet being used more and more in the support of learning by students, an instant online feedback system can be readily available for the collection of comments and opinions for prompt improvement to course delivery. Prompt feedback from students is needed for course adjustments to be administered in a timely manner. This will in turn provide satisfaction to students who have given their feedback, recognizing the value of their comments and opinions. This can in turn attract more students in future uses of the instant feedback system to give their genuine opinion, thus creating a positive spiral of comment-adjustment cycles. The success of the system is thus founded upon the value that the system brings about to both the teachers and the students. Once the value is established and recognized, the system will be self-sustaining (attracting enough users to spread its values) and capable of evolving. With the adoption of outcome-based education and criterion-referencing assessment, there is a stronger need for evidence about how the students have learnt. An online instant feedback system, once established and accepted by its users, can potentially gather such data for analysis. To fully exploit the potential of the online client/server computing paradigm, the online feedback system can also be further developed to cater for individual differences in learning. An example is an online quiz system that tests students knowledge and understanding. A student s feedback can be a reference for remedial actions or directions to be prescribed to individual students. This instant feedback system, called the i-feedback system, has been implemented at The Hong Kong Polytechnic University and is available to all teachers to use on a voluntary basis. Anyone wishing to trial the system for non-profit educational purposes is welcome to access the system at the URL Trial use of the system will be available through December 31, and The Annual Conference on Distance Teaching and Learning 5

6 Biographical Sketches Dr. Josephine Csete has been working at The Hong Kong Polytechnic University since 1995 and is the Section Leader of the e-learning Development and Support Section ( ). Address: Educational Development Centre The Hong Kong Polytechnic University Hunghom, Kowloon Hong Kong SAR, China. Phone: Fax: Dr. C.K. Leung is an Assistant Professor in the Department of Electronic and Information Engineering at The Hong Kong Polytechnic University. C.K. is Programme Leader for an undergraduate programme and his current research focus is on the design, development and evaluation of web-assisted teaching and learning, as well as the impact of community service on students learning. Address: Department of Electronic and Information Engineering The Hong Kong Polytechnic University Hunghom, Kowloon Hong Kong SAR, China. Phone: Fax: and The Annual Conference on Distance Teaching and Learning 6