How To Teach An Online Course

Transcription

1 Psychology Learning and Teaching, 1(1), A Guide to Design and Testing in Online Psychology Courses NORMAN E. KINNEY 1 Southeast Missouri State University, USA A rapidly evolving reality in the teaching of psychology is the increasing prevalence of entirely web-based psychology courses. The author addresses basic design and testing issues of online courses and presents a frank discussion and evaluation of current technology problems and teaching strategies employed. Data and analysis of testing results from paired on-campus and online sections of an introductory psychology course are discussed, as well as their implications for broader issues of learning and performance. Developing and implementing a successful webbased course requires effective use of existing software and hardware solutions to achieve many traditional teaching goals, together with a willingness to adapt to the new demands, limitations and strengths of the online environment. The development of online classrooms is largely market-driven and it appears they are here to stay. A report released by the International Data Corporation estimates that 2.2 million college students in the US will be enrolled in distance education by 2002, up from nearly 710,000 in 1998 (Council for Higher Education Accreditation, 1999). In addition, four-year colleges and universities that offer distance education courses are expected to jump to 84% in 2002, an increase from 62% in The number of distance learning classes, including online web courses, is expected to grow rapidly to meet the demand. Many administrators believe that distance learning programs will attract more students and make more efficient use of faculty resources, even though there is little published research to support this position (Waschull, 2001). There are studies, however, indicating that many online courses and programs are poorly designed and pedagogically unsound (Schweizer, 1999). Many such courses amount to little more than lecture notes or published material cut and pasted into web pages. Students seeking online alternatives are often left to make relatively uninformed choices and all too frequently are stuck with a poor quality substitute for a traditional class. Nonetheless, the scheduling advantages for working students and those with families to support appears to be a powerful draw for many. The pressure to provide online courses at many institutions has created a need for instructors to consider this alternative format and to learn what is involved in the design of web courses and the pedagogical and practical issues which arise with the use of new technologies. Instructors are justifiably uneasy with the thought of significant changes in course design and especially testing. In fact, what is known regarding the effectiveness of online testing for measuring learning and performance compared to more traditional methods of testing? While most studies looking at distance learning versus traditional face-to-face classroom performance report similar findings, few of these appear in refereed journals (Hiltz, Coppola, Rotter, Turoff and Benbunan- Fich, 1999). Unfortunately, only a few of those specifically address questions of performance and learning in online courses per se compared to on-site courses. Hiltz et al. (1999) reported no differences in grade distributions when comparing traditional classroom and video plus online formats. Waschull (2001) found no significant differences in exam performance, whether students freely elected or were assigned to take an online course, when comparing online versus on-campus sections of the same course. That author, however, also rightly pointed out the dangers of generalising from small samples and the need for more research. The research reported below looks at exam performance in paired on-campus versus online sections of an introductory psychology course over a one-year period. STUDENT ISSUES AND COURSE DESIGN The increased freedom and independence afforded online students also brings new demands. Without lectures and the advantages of face-to-face communication, a number of students feel at a loss when thrust into the online environment. It is well known among online instructors, in fact, that student frustration and anxiety is higher in web-based classes. Therefore, a key task of the instructor is to provide a friendly and highly supportive presence online (White and Weight, 2000). In addition, the removal of the direct person-to-person contact of traditional classrooms requires the development of online communication skills and techniques and the establishment of a new type of learning community. Creating an active, studentcentred and interactive environment on a computer screen is a major challenge for instructors newly online. The software technology typically used presents new problems and opportunities and sometimes requires a significant shift in our thinking about important pedagogical matters like testing and the evaluation of performance and hence learning. 1 Correspondence concerning this article should be addressed to the author at: Southeast Missouri State University, Cape Girardeau, MO 63701, USA. norm@kinney.semo.edu 16

2 A GUIDE TO DESIGN AND TESTING The time required for development of an online course is generally greater than for traditional courses. The initial, up-front preparation time is increased due largely to the need for meticulous construction of detailed descriptions of course expectations, course activities, discussions, assignments, the use of software, course navigation, access and security issues and web site design. A typical figure is 40% increased work up front the first time a course is offered (Schweizer, 1999). While the institution may provide the software solutions and technical support needed, some time-consuming hands-on learning by the instructor is also often required. Since the primary causes of online attrition are student isolation, student procrastination, competing responsibilities and technical issues (White and Weight, 2000), these will need to be dealt with up-front during course design. These and other considerations discussed above increase the time required for development of an online course compared to traditional courses. What type of students are most suitable for online courses? The following list is based on the author s experience with online courses and is given to students so they may better understand what is expected and possibly reconsider early on whether they should be in an online course: The essential qualities of a successful online student are that they are self-reliant and self-motivated, have good selfdiscipline, have good basic computer skills and the ability to read carefully and to follow written instructions, be willing to spend as much time on this course as on any regular onsite course (since that is what will be required) and be self-organised and able to provide their own study structures such as detailed time-lines for studying and exam preparation, paper writing and completing assignments. Many university sites provide similar lists of desirable qualities and encourage prospective online students to take a quiz testing their suitability. A score is provided at the end, as well as an evaluative interpretation the school feels is appropriate for that score. For an example, see erview/studentprofile.html. When students first enrol for an online course, I ve found that it is best to require the establishment of e- mail communication immediately. quickly becomes a principal means of contact and is essential to personal communication between instructor and student. If students are allowed to voluntarily establish first contact, some will be phoning the instructor in the 2 nd or 3 rd week, bewildered and asking for instructions. Many students are not yet in the habit of communicating online and cling to the phone as their primary communication device. From my experience, the online classroom will also need a set of policies including a make-up policy, a courtesy and etiquette policy for discussions, attendance and participation policies and a policy governing academic honesty. To further discourage plagiarism, there are sites (for example, which, for a nominal fee, will search the internet (including sites known to provide pre-written papers for a fee) and highlight for the instructor cases of clear copying from internet sources. The students must submit their papers to a site ( in one of a variety of formats and a report is sent to the instructor from the site. All of the factors discussed above influence the construction of a user friendly and supportive online environment. BUILDING THE ONLINE ENVIRONMENT Learning environment Consider carefully the type of learning environment you want to construct in your online classroom. It is important to utilise interactive learning activities and not just those requiring passive rote learning. Online students will also need support in the areas of technical assistance, registration and obtaining textbooks, CDs and study guides. The class website should include clear instructions written in easily navigated web pages. The key goal here is to provide sufficient level of detail so that the student can proceed without the need for further substantial input from the instructor (Carr-Chellman and Duchastel, 2000). Students need to be taught first how to navigate and get around your course site, given clear instructions on how to access and use supportive software and given opportunities to take ungraded sample exams online. I ve discovered that a good guiding philosophy is to keep it simple and don t let the technology obstruct the learning. There are a myriad of details that need to be thought out ahead of time and presented in writing. All components of the course need to be provided to students in well-organised, linked web-pages at the course site. These would usually include such standard items as a course description, student expectations or outcomes, readings, exam dates, online assignments, course policies and instructions for software use and secure access. There are a few basic guidelines that should be observed while building your course web pages and that can be found in standard texts on web page construction. I ve found it advisable to use an easy-toread font (Arial or Times Roman) and pay close attention to the contrast between text and the background. The text should have high contrast against the background (such as black lettering on a light coloured background) and be large enough to be read with ease. A page that is full of non-text items (images, buttons, banners, marques, forms) will distract and confuse the user and can considerably increase the download time of your pages. For students with older machines and slower modems, this can be a further cause of frustration. Be sure the pages are easily navigable and avoid the use of frames, which older browsers can t handle. Again, keep it simple. In fact, the author has found it helpful to have an initial page that contains a link which reads simply Begin Here. 17

3 KINNEY Student-instructor interaction Instructors will want to maintain a friendly, supportive presence and work to foster a sense of belonging in the student (White and Weight, 2000). Help students to get to know you personally by, for example, providing a biography of yourself (your photo, history, education, family facts, special interests). Establish and maintain personal contacts with students and otherwise personalise your online classroom. Creating a warm and caring environment, not being afraid to express a sense of humour and talking to and not at students can considerably enhance the online experience and environment for both students and instructor. It is also essential to maintain an online visibility to reduce student isolation, one of the main reasons for online course attrition. This entails sending a variety of different types of messages, such as contentrelated messages (lecture or handout materials, commentary on the textbook, discussion questions), process-related messages (directions regarding assignments, using software, taking exams), protocol and etiquette guidelines and simply responding to individual queries and concerns. Communicate in a public forum as much as possible (students can t see sent to other individuals). This is easily done with group or online discussion software such as O Reilly s Webboard 1. Using software effectively Courseware is software designed to deliver educational or training courses over the internet. Web Course in a Box 2, for example, provides the student and instructor with a collaborative and interactive online environment in which creating and teaching a web course is fairly easy for the average instructor. The author uses a new suite of instructional software programs called Online Instructor Suite (OIS) written by campus programmers (manuals describing how each program in the suite works are available for free download 3. It includes Forum, a program which provides asynchronous forums that I use for online classroom discussions, course feedback and evaluations and as a messaging centre, UTest for online testing and scoring/grading, Grade A for online presentation of grades to students and Calendar for important dates, announcements, test times and other information. Anything instructors enter in Calendar will appear in each student s personal calendar page and they can then add or remove items using a friendly web interface. The OIS programs share a common database which eliminates the effort previously needed to maintain separate databases for each program. A welcome feature of OIS is one that allows instructors to construct their course site remotely and securely using a web browser. OIS will be made available to institutions outside of Missouri in the near future. The author also uses O Reilly s Webboard for 1 See 2 See 3 See separate assignments and because it has a chat function (not yet available in Forum). Webboard contains a chat function which allows for fairly rapid, real-time class discussions. It also has the capability of storing in a file all chat entries for later examination and evaluation. Each student s name and address appear with their messages, assisting in the process of class bonding and intercommunication. In addition, the author has found that many psychology courses lend themselves well to assignments on Webboard. Weekly assignments require students to post one message in response to that week s topic and a second message in response to another student s answer to the assignment. Topics like Describe in detail the drug problems of someone you know (names not necessary), the symptoms you observed, the effects on others and any resolution... trigger the posting of remarkable and thoughtprovoking stories and compassionate and empathetic responses. Virtually all students know someone with a drug problem and the discussion blends well with more than one of my psychology courses. Unless you ve run an online discussion, it is hard to realise how much many students will open up online, often to a degree uncommon in ordinary classrooms. Further, many of the students who would never speak up voluntarily in an onsite classroom will suddenly blossom online. Given the greater openness that occurs in the virtual world, many instructors feel the need to post a few words regarding online etiquette. I ask students, for example, to keep their comments respectful and professional in tone and to avoid sarcasm which tends to have a negative effect that somehow appears amplified online. Feedback I ve found it important to give regular, consistent and diplomatic feedback. All of the usual non-verbal communication signals of face-to-face interaction are not present, except perhaps for emoticons [for example, :) ], so frequent feedback is more crucial here than in an ordinary classroom. Since student motivation and success are often closely tied to feelings of self-efficacy, frequent and diplomatic feedback is important. This includes evaluative feedback using, for example, an online grading program like Grade A that allows students to access the details of their grades at any time and UTest which returns performance feedback to students on all test questions immediately after test completion. Sharing time-management skills with students can help to head off a frequent problem which occurs when students put off assignments and online meetings and then suddenly feel overwhelmed at the prospect of trying to catch up. Finally, the instructor needs to expect and demand a high level of self-discipline and maturity from students in the online classroom (Schweizer, 1999). TESTING Testing is an area of obvious concern and for online classes especially I ve found it requires patience, ingenuity and good planning. Some simply opt to 18

4 A GUIDE TO DESIGN AND TESTING require testing be onsite in a supervised location on or off campus. But for courses or programs that reach outside the local area, this becomes increasingly unfeasible the farther students reside from campus. Since an online exam is essentially open-book, cheating only becomes an issue if communication of answers between students is possible (as when unreasonably long time limits are set), or (for essay questions) if plagiarism occurs. The former problem can be largely resolved by time-limiting access to the exam and presenting to each student a random selection from a database of questions. I ve found this works well in my freshman-level online introductory psychology course. Questions which require processing of information, concepts, or theories or their application in a specific context seem more suitable than those just testing rote learning. Some schools require a proctored midterm or final, while others permit all online exams. Finally, others utilise take-home exams which need to be turned in at a specified time. From published reports, academic dishonesty is as much a problem (or worse if not handled properly) in online courses as in traditional ones (White and Weight, 2000). Not surprisingly, the biggest single complaint of students is the unreliability of the technology. There is often the false assumption that technology means better, faster and easier. But instructors run into technical and connectivity problems frequently, including the peculiar limitations of specific software. For example, the author s current online testing software UTest initially did not accept a number of keyboard characters in passwords. Also, our campus periodically experiences bandwidth overload and subsequent slowdowns and the campus server at one point was delivering consecutive messages out of sequence (inexplicably delaying some messages for many hours). In addition, service providers will often more quickly disconnect student users at peak usage times, causing premature termination of an ongoing online exam and submission of an incomplete answer set. In an earlier version of UTest, this problem was exacerbated by the html design of the program. Specifically, after downloading the entire test at the beginning as a web page, it then only required actions noticed by the service provider s server at the end of the test when the students submitted their answers. Such a long period of modem inactivity will cause many service providers to disconnect the user in far less time than the exam period. Initially, the author advised students to run their program in the background, set so that it checked their incoming mail every minute, to avoid being disconnected. Later, campus programmers built a Java script program which is now downloaded with the test. It frequently queries the UTest server, thus providing activity on the line and maintaining the line connection. In general, expect servers to go down and be prepared with contingency plans. In addition to resolving the technical problems of online testing which inevitably arise, I was also initially concerned with two fundamental questions regarding the use of time-limited online exams. The first was whether online students, who are required to take time-limited online exams, might be unfairly advantaged or disadvantaged with respect to students in my on-site sections of the same course. The second concern was whether the different testing formats in online versus regular onsite in-class testing might result in differential quality of learning prior to the exam. In an attempt to address these questions, a study was conducted comparing exam performance in online versus concurrently run on-site sections of the same introductory psychology course. The normal testing procedures utilised in my introductory course in these two locations were maintained during the study. Method for Performance Comparison During both the Spring and Fall semesters of 2000 at Southeast Missouri State University, an online and an on-campus version of an introductory psychology course were taught in parallel using the same textbook chapters and exams drawn from a common test bank. Students in the two online classes were given entirely online exams (UTest) during a limited specified time period. Students in the two on-campus classes were given equivalent exams in their normal classroom setting. All exams consisted of fifty multiple-choice questions with four answer choices each. Online students were required to complete the exam within 42 minutes, while on-campus classes were given 50 minutes in which to complete each exam. Half of the exam questions were factual in nature and half were applied. Exams were available online between 8 and 9 PM on either Sunday or Monday nights. All students were encouraged to study the text chapters, prepare extensive summary notes and learn and memorise the material well before exams. Online students were in addition advised to prepare for their exams as they would for a typical on-campus, closed book exam. These students were warned not to think of the online exams as open book, because in essence they are not, given that the short time allotted per question precludes looking up answers in the textbook. Obviously, online students would probably have the textbook and other materials with them during the 42- minute exam period. Performance Results Mean exam scores for the four classes, together with overall class means for all exams are provided in Table 1. Not shown are the weighted means for the combined campus classes (N = 55, mean = 73.27) and the combined online classes (N=44, mean = 75.36) across all exams. Analyses using the independent t- test for equality of means to compare same semester paired classes revealed a significant mean exam score difference only for Exam 4 for the two Fall classes, t(50) = -2.42, p< Discussion These data suggest that, with the possible exception of one exam given in the Fall semester, the online 19

5 KINNEY Table 1 Mean exam scores (standard error) for the four classes with the class mean for all exams combined shown in the bottom row. Class Exam Spring Campus Exam (2.25) Exam (1.75) Exam (1.66) Exam (2.38) Exam (2.26) Class means Spring Online (4.16) (3.46) (3.37) (3.47) (2.70) Fall Campus (2.50) (2.26) (2.50) (2.71) Fall Online (2.08) (2.70) (2.35) Key: exams not given due to family emergency; exam given with 40 questions instead of 50 due to instructor error. classes enjoyed no significant advantage or disadvantage by taking their exams online compared to paired control classes that took all exams in the standard classroom setting with no material aids. Further, the results indicate no significant difference in quality of learning between the online web sections and the on-site sections. It is also revealing that when, due to instructor error, the Fall web class was given 40 instead of 50 questions to complete in the 42 minute time period for exam 1, their mean score (86.54) was an unusual points higher than that of the Spring web class and points higher than the paired Fall campus class on that exam. It is worth noting that currently all online students at this institution must pass the standard admission requirements and are almost all concurrently taking on-campus courses. Demographic data is being gathered on all paired classes to help delineate any significant demographic differences between online and on-campus class members with regard to age, gender distribution, hours of employment or class year. These results are in agreement with findings reported by Waschull (2001) and Hiltz et al. (1999). However, with so many differences in the method of producing grades and grade distributions among instructors, there is a serious need for the establishment and use of standardised grading methods and measures of performance and learning in studies making this kind of comparison. CONCLUSIONS Flexibility is great, but many students also want a real education. Is this possible with an online classroom? I believe the answer is yes, if the course is demanding and well designed and the student is willing and capable of taking command of their own learning. This requires more maturity than many younger students possess and consequently does not seem appropriate for a sizeable portion of today s freshmen. For more mature and disciplined students and especially adults who are already in the workplace, however, the online classroom is a rapidly growing and viable alternative. Standards and guidelines for quality distance learning have now been set forth by the American Council on Education and the Western Interstate Commission for Higher Education. Online classrooms are here to stay and the developing technology should continually reduce current limitations, especially in the fundamental nature of human to human online communication and open up online classes to a large and significant population of future students. REFERENCES Carr-Chellman, A. and Duchastel, P. (2000). The ideal online course. British Journal of Educational Technology, 31(3), Council for Higher Education Accreditation. (1999). Distance learning in higher education. CHEA Update, 2, 1-5. Hiltz, R., Coppola, N., Rotter, N., Turoff, M. and Benbunan-Fich, R. (1999). Measuring the importance of collaborative learning for the effectiveness of ALN: A multi-measure, multi-method approach. Journal of Asynchronous Learning Networks, 4(2). (Available at: Schweizer, H. (1999). Designing and teaching an on-line course: Spinning your web classroom. Boston: Allyn and Bacon. Waschull, S.B. (2001). The online delivery of psychology courses: Attrition, performance and evaluation. Teaching of Psychology, 28(2), White, K.W. and Weight, B.H. (2000). The online teaching guide: A handbook of attitudes, strategies and techniques for the virtual classroom. Boston: Allyn and Bacon. Manuscript received on 31 January 2001 Revision accepted for publication on 14 June