Retrofitted Materials for WebCT: Guidelines for Authors, Web Designers, and Users

Transcription

1 Tony Houston 657 Retrofitted Materials for WebCT: Guidelines for Authors, Web Designers, and Users TONY HOUSTON University of Missouri-Rolla ABSTRACT With the advent of web-based programs that automate the scoring of exercises, materials are now being developed from the outset with the capabilities of the web in mind. Materials developed without consideration of the web s capabilities, on the other hand, must be retrofitted. Texts that are retrofitted to the web have the advantage of being pedagogically, not technologically, driven. With adequate attention paid to the pedagogy of adapting such texts to the web, these texts can compete on their own merits with products that are designed from the outset for use on the web. This paper discusses some of the complications of working with retrofitted materials and presents some solutions to common problems. KEYWORDS Web, Retrofitting Materials, WebCT, Pedagogy INTRODUCTION A frequently heard mantra in the teaching profession is that pedagogy, not technology, should drive our decisions. With the advent of web-based programs that automate the scoring of exercises, materials developers are now bearing in mind the capabilities of the web from the outset. Materials developed without consideration of the web s capabilities, on the other hand, must be retrofitted. As Feustle (2001) observes, the question of whether retrofitted electronic versions of workbooks are superior to their paper counterparts remains to be seen. In order to tap into the potential of the web, he advises that commercial publishers consider the possibilities and limitations of the web when designing materials so that these materials do not have to be retrofitted later. Although retrofitting can yield an awkwardly adapted product, it has the advantage of making materials available for the web that are pedagogically, not technologically, motivated. Web-based course management programs can liberate instructors from the tedious task of grading student workbooks, saving valuable instructor time better CALICO Journal, 24 (3), p-p CALICO Journal

2 658 CALICO Journal, Vol. 24, No. 3 spent on research, curriculum development, and lesson planning. Although there are many such programs available, WebCT and Blackboard have been the leading choices. Deciding which program to use is usually in the hands of administrators, not instructors. For those instructors who have been using WebCT, the present paper offers guidelines on the use of the quiz module, especially in conjunction with retrofitted materials. It also offers suggestions for ways in which authors and the designers who retrofit their materials to the web can work together to improve materials. Two important advantages of web-based programs and materials are to liberate instructors from tedious grading so that their talents can be applied to developing more effective ways of teaching and to free up class time that might otherwise be spent on reviewing homework. Despite these advantages, there a number of limitations in such programs. In this paper, I present a number of technical challenges to retrofitting texts to the web and describe what adapters and adopters of these materials can do to overcome some of the limitations of the WebCT quiz tool. The examples that follow are primarily from a customized version of Feustle s (2004) WebCT adaptation of Sabías que?. The customization involved modifications that are described in the sections that follow. Although the examples come primarily from one text in one course management tool, many of the challenges apply to other texts and to Blackboard as well. TECHNICAL CHALLENGES Technical Challenge #1: Free Expression Questions The question formats useful for language instruction are matching, multiple choice, short answer, and paragraph. Grading is automated with all of these except the paragraph format. Feustle (2001) observes that course management tools such as WebCT and Blackboard are ill suited to deal with free expression in the target language. Although the grading of these questions cannot be automated, Feustle points out that typed assignments are easier to read than hand-written ones. Another advantage is that once the score is assigned, it transfers automatically to the online grade book. The disadvantage to having free expression questions online relates back to the issue of instructor time. The tedium of correcting free written expression online can be prohibitive. This problem is exacerbated when lower level instruction is relegated to teaching staff whose university-provided computer equipment is slow or obsolete. To grade free expression questions, instructor must click on each submission, wait for it to load, read it, enter a grade, and submit the assignment. It may actually be more efficient to grade such exercises on paper. To customize the publisher s WebCT product for use at Saint Louis University, we took assignments that require instructor intervention off line and copied them to an organizer page. Because free expression activities in Sabías que? tend to be labeled para entregar to hand in, we used that same label for the organizer page. On the organizer page are links to the activities in Word. If the purpose of using an online workbook is to save instructor time, not using the WebCT

3 Tony Houston 659 paragraph format is an option worth considering. Instructors who prefer to grade such exercises on paper can still require learners to type their responses, although learners may find the expense of printing daily homework objectionable. We have retained these activities in the electronic workbook for instructors who prefer online grading. Adopters working with retrofitted materials who are interested in generating more automated material may wish to convert some paragraph questions to short answer, matching or multiple choice. To change the format of a question, it is necessary to generate a new question in the desired format. Using the cut and paste function, it is possible to transfer the direction lines to the new question without losing the hypertext that encodes any formatting or links to sound files. It is crucial to transfer any associated images, shown in the image dialogue box, to the new question. Figure 1 shows a link to a sample paragraph question. Figure 1 Sample Paragraph Question When converting a paragraph question to an automated format, it may be necessary to select the setting Release the score once the quiz has been submitted. In the WebCT products we used, the designer selected the setting Release the score once the quiz has been submitted and all the questions have been graded with paragraph questions. The designer has also programmed the submission message Your instructor will have to grade this quiz. It may be necessary to delete any such submission messages that are no longer relevant when automating quizzes. We allowed two attempts for each activity and accepted the average of the two scores. Students see the correct answer after their first attempt, so most manage

4 660 CALICO Journal, Vol. 24, No. 3 to score 100% on the second attempt. By accepting the average score, we encouraged learners to make a genuine attempt the first time rather than guessing the first time just to trigger the answer prompts. Because paragraph questions require instructor intervention, we allowed only one attempt for quizzes containing paragraph questions. Figure 2 shows the link to question settings. Figure 2 Quiz Settings It is sometimes possible to replace paragraph questions with automated question formats. The following activity was rendered as a paragraph question in the original WebCT product. It asks learners to state their names and the names of their instructor and another classmate. Lección preliminar Quién eres? ACTIVIDAD A Nuevos amigos You have probably spent the first day or two of Spanish class getting to know people in the class. Can you remember some names? Answer the following questions by filling in the appropriate names. 1. Cómo te llamas? 2. Cómo se llama tu profesor(a) de español? 3. Cómo se llama otra (another) persona en la clase? By modifying the direction lines to read Answer the following questions in complete sentences, it is possible to render this paragraph question in a short answer format. By using the answer contains setting rather than answer equals,

5 Tony Houston 661 the designer can program the answer key to check for whether the target form appears in the answer. Such a modification, however, is out of sync with the intent of this activity. This activity was clearly intended to check the learner s knowledge of names and was designed to be evaluated by the instructor. Instructors still have the option of overriding a score that does not reflect a thoughtful response, but, once the grading is automated, they will almost certainly opt not to do so. Although modifying such activities would generate more automated material, care should be taken not to subvert the author s intent when converting paragraph questions to short answer. Technical Challenge #2: Gratuitous Production It may not always be advisable to convert paragraph questions to short answer just for the purpose of generating more automated material. When the goal is also to eliminate gratuitous production, however, it may be worthwhile to convert some paragraph questions to multiple choice or matching. We found that the following paragraph question worked well as multiple choice. Lección 1 Durante la semana Actividad B La rutina del presidente Below are a number of activities that the President of the United States might do in a given week. Finish each with the phrase that indicates the frequency with which he performs each activity. MODELO: Se levanta temprano frecuentemente. Duerme ocho horas raras veces. 1. Consulta con el vicepresidente. 2. Hace ejercicio. 3. Le pide consejos a su esposa (wife). 4. Piensa en la situación económica. 5. Se duerme en la oficina ovalada. 6. Escucha música. 7. Habla por teléfono antes de (before) acostarse. In the modified activity, we provided the following expressions of frequency as alternatives. Because the question asked for the learner s opinion, full credit was awarded for any one of the following expressions of frequency: a. siempre b. frecuentemente c. a veces d. raras veces e. nunca

6 662 CALICO Journal, Vol. 24, No. 3 Although, again, care should be taken not to subvert the author s intent when adapting free expression questions to automated formats, reducing gratuitous production allows learners to convey the same information in a format that does not require instructor intervention to grade. In the case of Sabías que?, the modified activity above is similar to other opinion questions found throughout the text. Rendering a free expression question as a free expression question is clearly the most faithful representation of the original text. In the absence of guidelines for designers with respect to reasonable parameters for creative freedom, these decisions will fall to adopters. Technical Challenge #3: Questions that Depend on World Knowledge We converted the following activity, like the one above, from paragraph to multiple choice to automate the grading and to reduce gratuitous production. For this activity we found it necessary to modify the direction lines to eliminate the words with a written response. Rather than writing out their responses, students select either Sí or No. The activity below demonstrates the problem of questions that depend on learners real-world knowledge. Lección preliminar Más sobre las clases Actividad G En tu universidad? Based on what you know about your university, answer each question that you hear on the audio program with a written response. The audio script reads as follows: 1. Hay clases de química orgánica? 2. Hay cafetería? 3. Hay clases de economía? 4. Hay clases de japonés? 5. Hay clases de geografía? In the activity above, we adjusted the answer key to award full credit only for the correct answer but accepted the higher score out of two attempts so that there would be no consequences for an incorrect first attempt. The information, based on knowledge of the campus, will vary from one institution to another. Adopters will have to customize activities that depend on campus-specific knowledge to their own institutions. For purposes of scoring, adopters will have to consider whether they want to make students responsible for knowing or researching the information. Technical Challenge #4: Absolute Precision Demands Another limitation of Blackboard and WebCT is that short answer questions require absolute precision of the student. As Feustle (2001) explains, technologically savvy teachers have become accustomed to programs such as BASIC, PILOT,

7 Tony Houston 663 Hypercard, and ToolBook that deal better with free expression than course management tools such as WebCT and Blackboard. With these course management tools, the designer must anticipate multiple correct or partially correct responses in order to develop a product that is tolerant of minor errors. One of the tenets of communicative language teaching is that formal accuracy in the beginning stages should be neither required nor expected (Savignon, 1997, p. 29). Although the precision demands of WebCT and Blackboard are out of step with the tenets of communicative language teaching, the knowledgeable adopter can overcome this limitation in a number of ways. One partial solution is to reduce gratuitous production in short-answer questions as shown below. The original WebCT product rendered the following activity as a series of short answer questions although students task was to make a binary choice. We modified it using the multiple-choice question format. Lección preliminar Más sobre las clases ACTIVIDAD A Oscar el optimista o Pedro el pesimista? Below is a list of statements. Decide if each is made by either Oscar (O) the optimist or Pedro (P) the pessimist. O o P? 1. Mis clases son muy interesantes. 2. Mi profesor de computación no es inteligente. 3. Mi vida (life) es aburrida. 4. Mis amigos son buenos. 5. Mi familia no es amable. 6. Mis profesores no son dedicados. The above activity is a short-answer question that we converted to multiple choice. We converted the following activity from short answer to matching. In the modified version, students were instructed to match the name to the number of credit hours. Lección preliminar Más sobre las clases ACTIVIDAD D Horarios Paso 1. Listen as each person says his or her name and how many credit hours he or she is taking. Write down the information below. Besides reducing gratuitous production, another strategy for overcoming Web- CT s absolute precision demands is to favor the answer contains over the answer equals setting and to limit the text in the answer key to target forms. Using this technique, the designer/user can reduce the incidence of points taken off for extraneous errors and extra spaces in the student responses. By providing the full

8 664 CALICO Journal, Vol. 24, No. 3 text of the correct response in the general feedback box, the designer/user allows students to self-check any extraneous errors that the answer key permits. Figure 3 demonstrates this technique. The activity is from the customized version of Sabías que? (Lección dos: A quién le gusta?, ACTIVIDAD I Isabel y sus amigos). Figure 3 Short Answer Feedback Technical Challenge #5: Long Fill-in Passages Short answer questions in Blackboard and WebCT do not allow in-line placement of answer boxes within texts, although the technology to allow in-line placement exists (Mallard, developed at the University of Illinois). The Blackboard and Web- CT systems organize text answer boxes for short-answer questions vertically down the page. As a result, the questions often scroll off the screen before learners can enter all of the answers. One solution to this problem is to put the text on a separate web document linked to the quiz. It is necessary to code the link such that the document opens in a separate browser window. Technical Challenge #6: Affective Questions One feature of Sabías que? that Feustle (2001) observes with respect to its adaptation to WebCT is this text s penchant for questions for which either every answer is correct or there is no correct answer at all (p. 843). In the scholarship on structured input, such activities are called affective activities. Referential activities, in contrast, have one correct answer and the choice indicates having processed a form correctly. In structured input activities, grammatical form contains meaning and learners must attend to form in order to complete the task (Lee & VanPatten, 2003, p. 143). Farley (2005, p. 95) offers the following activity as an example of referential structured input.

9 Tony Houston 665 Activity A Referential SI Activity Items: Object marker a 1. A mi mama la besa mucho mi papá. a. My mom kisses my dad a lot. b. My dad kisses my mom a lot. 2. A mi papa no lo comprendo yo. a. I don t understand my dad. b. My dad doesn t understand me. This activity requires that learners pay attention to the formal features of the language in order to provide a correct interpretation. In affective activities, the correct answer depends on the learner. Farley (2005, p. 95) offers the following example: Activity B Referential SI Activity Items: Object marker a Cierto Falso A la generación X no la comprenden los viejos. Cierto Falso A los viejos no los respetan los jóvenes. As Feustle (2001) notes, questions for which there is no incorrect response make poor use of WebCT s scoring capabilities. Newer materials, such as the Sol y viento workbook (VanPatten, Leeser, Keeting, & Houston, 2005) are designed with the web in mind and include only questions with right or wrong answers. The inclusion of activities with affective questions, however, does not mean that Sabías que? is ill suited for adaptation to the web. When scoring an activity in which the correct response depends on learners, WebCT does exactly what the instructor would do when grading a paper submission to check to see whether the work has been completed and to trust that the learner has responded thoughtfully. In the original version of the WebCT activity above, each possible answer was allocated 100% of the total points possible for the questions, but learners may choose only one answer. The value of structured input activities is well established in scholarship. Although the research has established a direct relationship between language acquisition and structured referential input only, Farley (2005) recommends that affective structured input activities be used in conjunction with referential ones. He notes that affective structured input activities allow learners to apply the target forms in a way that is personally meaningful. Affective activities in the WebCT manual for Sabías que? are most often rendered as multiple-choice questions. Feustle (2001) uses the following activity as an example. It asks the learner to make a logical judgment. Lección nueve Las bebidas Actividad B Lógico o absurdo? Te parece lógico beber café a las once de la noche? Escucha las situaciones en el programa auditivo y indica si las decisiones que se toman son lógicas o absurdas ( en tu opinión!)

10 666 CALICO Journal, Vol. 24, No. 3 Although the activity above consists of a series of opinion questions, the audio script below demonstrates that there is always a more plausible choice. 1. Es agosto y hace mucho calor. Pedro está de vacaciones en Miami. Tiene mucha sed. Decide tomar un té de hierbas. 2. Hoy es viernes y son las seis de la tarde. Después de una semana larga y dura, Gloria y sus amigos van a tomar una cerveza y a comer nachos. 3. Es domingo y Andrés va a preparar un desayuno especial para su amigo. Va a servir huevos, panqueques y fruta. Para beber va a servir un vino blanco. 4. María tiene un examen de cálculo mañana a las nueve y necesita estudiar mucho. Pero está súper cansada y tiene poca energía. Decide tomar un café muy fuerte. 5. Elena es muy activa, muy atlética. Hoy jugó al tenis tres horas y también corrió dos millas. Tiene mucha sed, así que bebe agua fría. 1. It s August and it s very hot. Pedro is on vacation in Miami. He s very thirsty. He decides to drink an herbal tea. 2. Today is Friday and it s six in the afternoon. After a long, tough week, Gloria and her friends are going out to drink beer and eat nachos. 3. It s Sunday and Andrés is going to prepare a special breakfast for his friend. He is going to serve eggs, pancakes and fruit. To drink, he is going to serve a white wine. 4. María has a calculus exam tomorrow at nine and she needs to study a lot. But she is super tired and has little energy. She decides to have a very strong coffee. 5. Elena is very active, very athletic. Today she played tennis three hours and also ran two miles. She is very thirsty, so she drinks cold water. Although there are no incorrect choices in the above activity, we generated a feedback response for each incoherent choice to simulate an instructor s likely reaction. For example, if learners chose absurdo for the first item, the feedback response said Absurd? But herbal tea doesn t have caffeine! We also used the general feedback function to provide a written repetition of the audio script so that learners could review their answers. Figure 4 demonstrates the use of the feedback with this multiple-choice question. Figure 4 Multiple-choice Feedback

11 Tony Houston 667 While the above activity in its original form does not utilize WebCT s capacity to score right or wrong answers, the modified version does discriminate between coherent and incoherent responses. It was WebCT s interactive feedback capabilities that were underutilized in the original activity. The WebCT product treated these questions as affective ones with no incorrect answer, as does the print version of the manual. The question settings allocated full credit to coherent and incoherent responses alike. We treated the questions as referential. Because the instructions call for an opinion, we adjusted the settings to accept the higher score out of two attempts so that there would be no consequences for an incoherent response on the first attempt. To overlook the pedagogical value of affective activities merely because they do not utilize WebCT s capacity to score right or wrong answers would be to allow technology, not pedagogy, to drive our decisions. As demonstrated above, affective activities whose answers depend on logical judgments make good use of WebCT s functionality if feedback to incoherent responses is developed. Technical Challenge #7: Affective Check-all-that-apply Questions Although affective questions for which any one answer is correct need not necessarily be regarded as a problem, those exercises for which any number or none of the choices is correct do require modification in WebCT. Whenever there is an affective check-all-that-apply question, it should be adapted to WebCT as a forced-choice question. Online survey research suggests that forced-choice questions yield more authentic responses than check-all-that-apply questions. Order of presentation effects have been found that favor the items presented first (Dillman, 2000; Israel & Taylor, 2003; Sudman & Bradburn, 1982). To adapt a check-all-that-apply question to forced choice, there must be an applies/does not apply choice for each item or some other exhaustive range of mutually exclusive choices. The following activity demonstrates the problem. Lección 4 Mis relaciones con la familia Actividad B Me Paso 1 Select a relative of yours (padre, madre, hijo) or a set of relatives (padres, madres, hijos, abuelos). Indicate which of the following apply. Remember that me is an object pronoun, not a subject! nombre: relación: 1. Me quiere(n). 5. Me da(n) consejos (advise). 2. Me adora(n) (adore[s]). 6. Me conoce(n) más que nadie (more than anyone). 3. Me llama(n) con frecuencia. 7. Me. 4. Me quiere(n).

12 668 CALICO Journal, Vol. 24, No. 3 The above activity was problematic in the original WebCT product. The first six items were rendered as a check-all-that-apply question with multiple responses enabled in the settings. In WebCT, each alternative answer must be allocated a percentage of the total value of the question. The original version of the activity allowed learners to earn 0-600% depending on the number of responses selected. We modified the activity by generating six separate questions, each of which required a Sí or No response. We allocated full credit to either response, but learners could select only one response per question. The following activity represents the problem of a check-all-that-apply question with choices that are not mutually exclusive. The activity asks students to identify whether each of eight statements applies to university students or retired people. In a student s judgment, neither or both may also be appropriate answers. Lección 2 Actividades para el fin de semana Actividad H Los estudiantes universitarios frente a los jubilados What do you think is true for the two groups of people below? Mark each statement accordingly. 1. A estas personas no les gusta pasar el sábado en quehaceres domésticos (household chores) como, por ejemplo, limpiar la casa. 2. A estas personas les gusta quedarse en casa los viernes por la noche. 3. A estas personas les gusta pasar los fines de semana en la playa (beach). 4. A estas personas no les gusta el cine por la violencia. 5. A estas personas les gusta sacar vídeos los fines de semana. 6. A estas personas les gustan las barbacoas. 7. A estas personas les gusta recibir visitas (to receive visitors). 8. A estas personas no les gusta hacer ejercicio. LOS ESTUDIANTES UNIVERSITARIOS LOS JUBILADOS In paper form, the student can check either Los estudiantes universitarios university students, Los jubilados retirees, check both, or leave the item blank to indicate that neither is a logical choice. The original WebCT product allowed learners to earn 0-200% depending on the number of answers selected. We modified

13 Tony Houston 669 the question to include the words Choose the best response for each statement to the direction lines and changed the multiple answers setting to allow one answer. Alternatively, we could have expanded the choices to include neither and both, but this hardly seemed necessary given that there was always a case to be made for a more plausible choice. The above activity was adapted simply by directing students to choose the best response. In the following activity, it was necessary to expand the choices because they were not exhaustive. Lección 6 Épocas anteriores ACTIVIDAD B Contrastes How has your world changed? Which of the following were true for you as a child but aren t true now? Which were true both then and now? 1. Nunca me comía* las verduras (vegetables). 2. Tenía un amigo invisible. 3. Les tenía miedo a los perros grandes. (tener miedo = to be afraid, literally to have fear) 4. Me levantaba temprano los sábados por la mañana para ver la televisión. 5. Yo era el centro del mundo de mis padres. 6. No hacía muchos quehaceres (tasks) domésticos. 7. Mi familia me llamaba con un apodo (nickname). 8. Me gustaba hacer bromas (jokes). 9. Pasaba mucho tiempo solo/a. 10. Iba a la escuela en autobús. 11. Podía ver la televisión hasta muy tarde. 12. Me gustaba dormir con la luz prendida (the light on). 13. Visitaba a mis abuelos con frecuencia. 14. Me burlaba de mis hermanos. 15. Mis hermanos se burlaban de mí. *comerse = to eat up (not a true reflexive) cierto de niño/a y falso hoy cierto de niño/a y cierto hoy

14 670 CALICO Journal, Vol. 24, No. 3 In this activity, the answer depends entirely on learners experience. There is no judgment to be made about the most plausible response. It is also possible that learners would respond falso de niño y falso hoy false as a child and false today or falso de niño y cierto hoy false as a child and true today. By adding these two options to the modified exercise, we generated a set of mutually exclusive options that exhausted all possibilities so that one and only one answer is possible for each item. Technical Challenge #8: Referential Check-all-that-apply Questions Assigning percentages to responses is fairly straightforward when there is one correct response. When there are multiple correct responses to a question, each response must be allocated a percentage of the total. The following exercise is such an example. Lección 6 Épocas anteriores Actividad A Alternativas Paso 2 Match the verb phrases with a logical conclusion (a, b, c, d). More than one answer may be possible. 1. Escuché música a. un poquito anoche antes de estudiar 2. Vi las noticias b. siempre cuando estudiaba. 3. Escuchaba música c. siempre cuando podía. 4. Veía las noticias d. por lo general In the above activity, a is the correct response to Questions 1 and 2 because un poquito anoche antes de estudiar a little before studying is consistent with the use of the preterite. The value for a, then, is 100%. For Questions 3 and 4, the correct answers are b, c, and d because those responses are consistent with the use of the imperfect in Questions 3 and 4. The point values assigned to Questions 3 and 4 were 33.3%, 33.3% and 33.4% for b, c, and d respectively. The quiz settings allowed multiple responses so that learners could select all three. One problem that can arise when multiple responses are enabled for all questions is that learners could select all four responses each time and earn 100% on the assignment even though the response is contradictory. This problem can be avoided by selecting all or nothing scoring in the settings. DISCUSSION Not all users of materials in WebCT have the time and skills to implement the modifications provided here. Some instructors will depend entirely on web designers to produce products that match their pedagogical preferences. Others may decide that some of the modifications described here are worth the investment in

15 Tony Houston 671 time to implement. The intent of this paper is to demonstrate how these materials can be modified, whether the authors and web designers collaborate on the changes or leave them to adopters. Web designers, charged with rendering a faithful adaptation of a product cannot be expected to anticipate the preferences of individual adopters. Each user will have to decide which modifications represent an improvement that merits the time necessary to make them. Table 1 summarizes the recommendations for adapting retrofitted materials to the web. Table 1 Summary of Challenges and Solutions Technical challenge Proposed solution(s) Steps 1. Free expression questions 2. Gratuitous production 3. Questions that depend on world knowledge Copy paragraph questions to organizer page for off-line grading. Convert paragraph questions to short answer judiciously. Convert questions from paragraph or short answer to multiple-choice or matching. Allow multiple attempts. Award credit only for a factually correct response. Count highest score. Cut and paste activities into Word documents. Upload documents to WebCT. Create an organizer page and link documents (see Figure 1). Create new multiple-choice or matching questions as appropriate. Cut and paste text from old questions. Replace old questions with new ones. Adjust the settings (see Figure 2). 4. Absolute precision demands Eliminate gratuitous production in short answer questions. For long responses, limit the text of the answer key to target forms and use the feedback function to show the full correct response. 5. Long fill-in passages Link the text to the exercise as a webpage coded to open in new browser window. 6. Affective questions Award full credit for any response but allow only one response. If the question requires a logical judgment, allow multiple attempts and award credit only for coherent responses. Count the highest score. 7. Affective check-allthat-apply questions 8. Referential checkall-that-apply questions Convert to forced-choice. Provide exhaustive, mutually exclusive alternative responses. Use all or nothing scoring. See #2 above on gratuitous production. Reduce target responses and generate feedback (see Figure 3). Cut and paste text to new webpage. Link the webpage to the exercise. Adjust the settings (see Figure 2). Generate feedback (see Figure 4). Create a separate multiplechoice question for each item. Cut and paste the text into the new questions. Replace the old question with the new ones. Adjust the settings (see Figure 2).

16 672 CALICO Journal, Vol. 24, No. 3 CONCLUSIONS The effective utilization of web-based course management systems such as WebCT begins with decisions as basic as the appropriate choice of format for a particular type of question and the choice of settings that determine the scoring and number of attempts. Decisions about the settings will reflect the instructor s pedagogical values and the amount of time the instructor is willing to invest in maintenance of the page. By developing specifications for the web product that are distinct from those for the print versions of their product, authors and publishers could make significant improvements to their web-based materials. Instructors and students would benefit most directly from improvements to the web product. Instructors, administrators, and technical support staff would benefit from the time saved by having a product that requires less time to customize to their home institutions. Authors and publishers would benefit from greater control of the product that bears their name if the modifications are not left to adopting institutions. Texts that are retrofitted to the web have the advantage of being pedagogically, not technologically, driven. With adequate attention paid to the pedagogy of adapting such texts to the web, they can compete on their own merits with products that are designed from the outset for use on the web. After all, pedagogy, not technology, should drive our decisions. REFERENCES Dillman, D. A. (2000). Mail and Internet surveys: The tailored design method. New York: John Wiley and Sons. Farley, A. (2005). Structured input: Grammar instruction for the acquisition-oriented classroom. New York: McGraw-Hill. Feustle, J. A., Jr. (2001). Extending the reach of the classroom with web-based programs. Hispania, 84 (4), Feustle, J. A., Jr. (2004). Sabías que? Cuaderno de trabajo electrónico. New York: McGraw-Hill. Houston, T. (2001). Qué te parece? Cuaderno de trabajo electrónico. New York: McGraw- Hill. Israel, G. D., & Taylor, L. C. (1990). Can response order bias evaluations? Evaluation and Program Planning, 13 (3), Lee, J. F., & VanPatten, B. (2003). Making communicative language teaching happen (2nd ed.). New York: McGraw-Hill. Savignon, S. J. (1997). Communicative competence: Theory and classroom practice (2nd ed.). New York: McGraw-Hill. Sudman, S., & Bradburn, N. M. (1982). Asking questions. San Francisco, CA: Jossey- Bass. VanPatten, B., Lee, J. F., & Ballman, T. L. (2004). Sabías que? (4th ed.). New York: McGraw-Hill.

17 Tony Houston 673 VanPatten, B., Leeser, M., Keeting, G. D., & Houston, T. (2005). Manual de actividades que acompaña Sol y viento. New York: McGraw-Hill. AUTHOR S BIODATA Tony Houston is an Assistant Professor of Spanish at the University of Missouri- Rolla. His research specialization is second language acquisition. He has authored teaching materials in print form and on the web as well as articles on second language sentence processing, communication strategies, and outcomes assessment. AUTHOR S ADDRESS Tony Houston Assistant Professor of Spanish University of Missouri-Rolla Department of Arts, Languages and Philosophy 211 H-SS Rolla, MO Phone: Fax: l2spanish@gmail.com Web:

18 674 CALICO Journal, Vol. 24, No. 3