Data Mining for Web Site Evaluation Janaki Santhiveeran a a

Transcription

1 This article was downloaded by: [University of Utrecht] On: 31 May 2010 Access details: Access Details: [subscription number ] Publisher Routledge Informa Ltd Registered in England and Wales Registered Number: Registered office: Mortimer House, Mortimer Street, London W1T 3JH, UK Journal of Teaching in Social Work Publication details, including instructions for authors and subscription information: Data Mining for Web Site Evaluation Janaki Santhiveeran a a Department of Social Work, California State University, Long Beach, CA, USA To cite this Article Santhiveeran, Janaki(2006) 'Data Mining for Web Site Evaluation', Journal of Teaching in Social Work, 26: 3, To link to this Article: DOI: /J067v26n03_12 URL: PLEASE SCROLL DOWN FOR ARTICLE Full terms and conditions of use: This article may be used for research, teaching and private study purposes. Any substantial or systematic reproduction, re-distribution, re-selling, loan or sub-licensing, systematic supply or distribution in any form to anyone is expressly forbidden. The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material.

2 Data Mining for Web Site Evaluation: An Exploration of Site Usage by Graduate Social Work Students Janaki Santhiveeran ABSTRACT. This paper evaluates the actual use of a course Website by graduate social work students. The study utilized data mining techniques to discover meaningful trends by using the data from server logs. The course Website was accessed 24,730 times by all 49 graduate students during a semester. The students utilized the course Website 23 hours a day, 7 days a week, day and night, evening hours, and weekends. The most popular area was the course documents area and the least popular was the student area. Study findings have implications for distance learning in higher education. doi: /j067v26n03_12 [Article copies available for a fee from The Haworth Document Delivery Service: HAWORTH. address: <[email protected]> Website: < HaworthPress.com> 2006 by The Haworth Press, Inc. All rights reserved.] KEYWORDS. Site evaluation, site usage, data mining, social work education, research methods, MSW students, Web logs, Web trends, Blackboard Janaki Santhiveeran, PhD, is Assistant Professor, Department of Social Work, California State University, Long Beach, CA. Address correspondence to: Janaki Santhiveeran, PhD, Assistant Professor, Department of Social Work, California State University, Long Beach, 1250 Bellflower Boulevard, Long Beach, CA ( [email protected]). An earlier version of this paper was presented at the Sixth Annual Technology Conference, Charleston, South Carolina, USA, August 10-13, Journal of Teaching in Social Work, Vol. 26(3/4) 2006 Available online at by The Haworth Press, Inc. All rights reserved. doi: /j067v26n03_12 181

3 182 JOURNAL OF TEACHING IN SOCIAL WORK The evaluation of instructional technologies is done to enhance teaching effectiveness and to offer a means of augmenting the learning experiences of students (Wilson, 1986). Evaluations specifically focusing on Web-based technology are gaining importance in the social work field (Knowles, 2000; Finn & Smith, 1997). These evaluations focus primarily on the perceived effectiveness and the perceived usefulness of the Internet in delivering course materials. These studies are based on the self-reports of the students. The evaluation of the actual use of Websites or Web components is sparse in social work (Gonzalez & Huff, 1998; Huff & Johnson, 1998). The understanding of user patterns offers objective measures on the usefulness and the ease of the use of Internet technologies and offers a perspective on time-demands. These reasons justify the importance of studying the use of the Web. The purpose of this study is to evaluate the actual use of a course Website. The study examines whether or not the presence of a course Website has met the goal of serving students 24 hours a day, 7 days a week. The study has the following objectives: (1) To measure the frequency of the use of the course Website developed for a research methods course, (2) to understand when the Website is being utilized (the most popular days and the most popular hours of the day), and (3) to understand which of the Web-components are used most frequently and which are used least frequently. Studying the usefulness is helpful in understanding the effectiveness of a course Website, which is a commonly used instructional tool. For the purpose of this study, the Web usage or actual use is calculated based on the frequency of use. The visitors or users are the registered students of the research methods course Website. The students mainly accessed the course Website in between the class sessions and sometimes during the face-to-face class sessions. This explorative study is the summative evaluation of their Web usage. The data for the study was retrieved at the end of the semester. The study utilized data mining techniques to discover meaningful patterns and trends by using the large amounts of data from server logs (SPSS Inc., 2002). Data mining goes beyond the frequency of visits, by presenting various means of tracking student activity levels at a course Website. The data mining technique offers a scientific understanding of the use of a course Website including how and when the students utilized the course Website. The study of Web usage has both potential benefits and limitations. For example, understanding user trends help instructors evaluate how many students used the Web components, when they used it, and what navigation pattern they used. An actual use offers a perspective on both

4 Janaki Santhiveeran 183 the usefulness and the ease of use of the Websites or the Web components utilized by instructors in their courses. On the other hand, the frequency of use does not help us understand the duration of use of Web components, which is an important measure for understanding time demands. Understanding Web usage does not provide a perspective of the intent of the users. Despite these limitations, the study is significant to social work educators, educators in higher education and administrators of Web courses. This paper presents a review of social work literature and other literature in higher education written on the use of Websites or Web-based technology in a course(s), the methodology of the current study, and the descriptive findings. The paper concludes with a discussion of implications to higher education in general, social work education and distance learning and recommendations for future research. LITERATURE REVIEW The long and rich logs of evaluations link student characteristics to both their satisfaction and their perception of course Websites or Web components (e.g., Faux & Black-Hughes, 2000; Knowles, 2000; Santhiveeran, 2002; Schoech, 2000). These studies confirmed that the class Websites promote education 24 hours, 7 days a week and the students spend an enormous amount of time outside their classrooms in continuing their learning in the Web-supplemented courses. Knowles (2000) found that forty percent of college students reported a heavy workload in the courses enhanced by Web supplements (Knowles, 2000). Evaluation of Web usage is essential to find the use of course related Websites and the Internet resources could lead to meaningful learning (Bainbridge, 1995; Sunal, Smith, & Sunal, 1998). There are two categories of evaluative literature that has been written on web usage. The first category is literature written on the perceived usefulness and the actual usefulness. The second category is comprised of studies written on the both actual usefulness of websites. Literature on the perceived usefulness is growing across disciplines (Davis, Smith, Rodrigue, & Pulvers, 1999; Rosen & Petty, 1997). For example, Davis et al. (1999) used self-report data on the use of the Internet by 349 undergraduate students. Davis et al. found that students self-reported spending on an average 4.66 hours/week (range = 0-55 hrs/week) browsing the Internet. Men at a public university reported spending significantly more

5 184 JOURNAL OF TEACHING IN SOCIAL WORK time online than women at a public university and men and women at a private university. There are only a few studies on Website evaluations that use the data retrieved from server log files or from the actual use of Web components such as and listserv (Breithaupt & Hoehle, 1996; Gonzalez & Huff, 1998; Huff & Johnson, 1998). Therefore, this paper is a valuable contribution to evaluation literature. In a rare study of website evaluation in social work, Gonzalez and Huff (1998) presented a case study of a Website by measuring requests, visits and visitors. Requests are the visits to each Web page, graphic, or object. Visits are comparable with the frequency of use measured in the current study. In a four month period, History Station website was visited 5,046 times which averages to 10.5 times a day. The majority of visitors came from the websites maintained by National Association of Social Workers (NASW) and Council on Social Work Education (CSWE). Since several factors including accessibility, necessity and simplicity enhance the use of websites and the Internet resources (Gifford, 1998; Gonzalez & Huff, 1998), it is important to note that the History Station Website is a public domain Website with no password protection. The present study offers an interpretation of the factors that influenced the use of the course Website discussed in this article. The earlier studies that took place on college campuses focused mainly on the actual use of the Web components such as electronic mail and list serves (Huff & Johnson, 1998; Gilbride, Breithaupt, & Hoehle, 1996). Huff and Johnson (1998) studied the frequency of communications in a social work course. They quantified transmissions from the students during a semester where they were enrolled in distance education courses. Thirty students attended class at a studio on campus, while 46 were distant students attending classes at distance sites. Of the total 369 transmissions during a semester, only 15.7% (n = 58) came from campus students. The highest number of transmissions received from a single student was 37. The most common message topics were grades and outside resources or information. Similarly, Gilbride, Breithaupt, and Hoehle (1996) studied the use of a listserv in rehabilitation education. Students posted a total of 862 messages during a semester. The analysis of messages showed that the students used listserv mainly to discuss course content. The students found listserv was useful in maintaining communication with their instructor and peers. Prior research studies presented evidence that student characteristics such as student status were related to the use of or the course

6 Janaki Santhiveeran 185 Websites (Huff & Johnson, 1998; Schoech, 2000). Huff and Johnson found a difference between campus and distant students with the frequency of messages sent to their instructors. Only 10 students from a distance education class sent the majority (71.5%) of the transmissions and more than half of the on-campus students never sent an . Schoech (2000) found a difference between part-time and fulltime students in their use of the Internet course. Schoech found that returning students tend to use the class website more extensively than traditional students due to their need to cope with multiple responsibilities. Use of Website logs are popular in other disciplines (e.g., Nicholas & Huntington, 2000; Wober & Gretzel, 2000). Some studies linked perceived usefulness with actual site usage. For example, Wober and Gretzel (2000) found that perceived ease of use and perceived usefulness were related to actual usefulness of a tourism Website. Apart from these studies, some social work literature has described how Websites and Internet resources are used in learning (Finn & Smith, 1997; Galambos & Neal, 1998; Santhiveeran, 1998). Although the course Websites are being used extensively in social work, evaluations focusing on the actual use of course Websites are lacking in the social work literature. Therefore, the present study adopted an innovative summative evaluation approach to quantify the Website activity levels, as a distinct measure to evaluate the usefulness of a course Website. Similar to Gonzalez and Huff (1998) and Wober and Gretzel (2000), the study retrieved data from Web logs. In addition, the study attempts to understand Web usage of graduate social work students by day of the week and hour of the day. ABOUT THE COURSE AND THE WEBSITE Blackboard software (Blackboard Inc., 2002) was used to design the course Website that is discussed in this paper. The class Website was password protected so that only 49 students and the course instructor had access. The university server administrator accessed the Website whenever assistance was requested. Several components of the course Website were secure and offered only limited access to the guests. The Blackboard software offers multiple opportunities for Web-supplements in a course (Blackboard Inc., 2002). The author utilized the following Web components. Announcements were posted on a regular basis. Required documents such as syllabus and assignments were posted in the beginning of the semester. Lecture outlines, readings,

7 186 JOURNAL OF TEACHING IN SOCIAL WORK group or independent exercises and weekly objectives were posted prior to each class session. Several online forums were created for small groups to interact on an ongoing basis. The ask me forum was created to enable students to get progressive feedback on their assignments. Students sent messages from the course website to their peers and their instructor. Chat rooms were used during the in-class group activities. The course Website was used only as a web supplement for two sections of a traditional graduate research methods course, which met regularly face-to-face three hours per week. The full-time students (section one) met at a high-tech computerized classroom on Fridays for the entire semester. The part-time students (section two) met in a high-tech computerized classroom for less than 50% of the class sessions on Saturdays and two students shared a computer terminal on the days the students met in a computerized smart classroom. The traditional classrooms in which the part-time students met for more than 50% of their class session did not have a computer terminal. Consequently, only full-time students had access to computer terminals during their class time on a regular basis. THE CURRENT STUDY This study is the summative evaluation of the course Website developed for graduate students. The aim of this evaluation is twofold: first, the study intends to evaluate whether the website is used 24/7 and second, the study aims to evaluate user trends of a course website. The aspects discussed in this investigation are primarily explorative in nature. Since secondary data were derived from server log files, personal characteristics of the students were not available to associate with student activity levels in a course website. The study utilized a summative evaluation approach to understand student activity levels, the measure of actual usefulness of a course Website. The guest visits and the instructor s use were not taken into consideration. Since the research methods course was the only course that had Web supplements in a graduate program in the Southwestern part of the USA, data for only 49 students were collected. Therefore, the study utilized a purposive sampling procedure with a sample size of 49 graduate social work students. Web usage is measured by hits or visits. A visit is defined as the number of visits or the number of Web pages accessed by a student. The frequency of visits is quantified by individual user visits to Web

8 Janaki Santhiveeran 187 components or areas, visits by the hour of the day, and visits by the day of the week. The activities performed by the students included accessing course content, reading messages, posting messages, and responding to the postings. The course Website did not record remote activities such as using MS Word for writing assignments before sending them to the instructor via digital drop box. In addition, the actual use does not include the use of materials in external Web resources or any programs outside the course website. For the purpose of user activity analysis, the Web components are divided into three areas including content areas, communication areas and student tools. These three areas are described as follows: Content areas include course documents such as lecture outlines, assignment pages, syllabus, weekly objectives, and announcements. Communication areas include asynchronous and synchronous communication activities involving online group forums, chat rooms, and discussion forums. Student tool areas include digital drop box, online grade check, student home pages, and calendar. The Blackboard software offered a Course Statistics option to retrieve the server s log files (Blackboard Inc., 2002). The course statistics feature focuses on one of the data mining techniques of analytical reporting. The analytical reporting feature generates descriptive statistics by quantifying Website statistics on user trends by the content area, days of the week, hours of the day and the individual user. Consequently, the study did not utilize any specific study instrument to collect data. For the summative site evaluation, student activity data was gathered only once for all 49 users at the end of the given semester. RESULTS Descriptive results are presented in this section by quantifying the actual use of the course website. These descriptive findings include the total number of accesses both by area and by individual users. Individual visits are further presented in detail by the hour of the day and by the day of the week.

9 188 JOURNAL OF TEACHING IN SOCIAL WORK Total Number of Accesses by Area. Figure 1 presents the total number of accesses by area or Web components of the course Website. The students accessed the content areas much more frequently than they accessed other areas. More than half (52.6%) of the accesses were recorded for accessing the content areas, followed by the communication areas (31.3%) and the student areas (13.5%). The students utilized the digital drop boxes less frequently. Based on the overall observation of the course Website, the author found that only a handful of students created their own Websites and a few students submitted their assignments through their digital drop box. Therefore, it was easy to interpret that the students accessed the student areas (13.5%) mostly to check their grades. Although the instructor posted grades only five times during the given semester, on an average a student checked his/her grades a little less than 60 times during a semester. User Accesses by Hour of the Day. Figure 2 presents the user activity analysis by the hour of the day. Except for 4 am, the students accessed the course Website 23 hours a day. The most popular daytime hour for the students was 9 a.m. followed by 1 p.m. and the most popular evening hour was 8 p.m. followed by 9 p.m. More than one fourth of the visits occurred during evening hours. Interestingly, the other one-fourth of visits occurred during night and early morning hours between 10 p.m. to 7 a.m. User Accesses by Day of the Week. Figure 3 provides the student visits by the day of the week. Students accessed the course Website 7 days a week. More than two-fifths of the accesses were on the days students had face-to-face classes. Therefore, the popular days are Fridays and Saturdays. The highest access was on Fridays (36.7%); on the day the FIGURE 1. Total Number of Accesses per Area. Total Visits = 24,730

10 Janaki Santhiveeran 189 FIGURE 2. User Accesses by Hour of the Day, Total Visits = 24,730 full-time students had an access to the computer terminals during their class time. The other 13.2% of the visits occurred on Thursdays, when the instructor had promised to upload the course materials and offer performance feedback for student postings in the asynchronized online forums. The students accessed more frequently on weekends (29.1%). In addition to the days they met in person, the students mostly accessed the class Website on Thursdays and Wednesdays. Total Accesses by the Students. The course Website logged the individual student visits (Figure 4). Interestingly, all 49 students utilized the course Website. The minimum number of use by a single student was 30 times followed by 153 times. The highest number of use by a single student was 1,264 times and the second highest was 950 times. Despite the outliers, fifty percent of the students accessed the course Website more than 454 times. Only one fourth of the students visited the course Website more than 632 times, which is an average of 12.9 times per day.

11 190 JOURNAL OF TEACHING IN SOCIAL WORK FIGURE 3. User Accesses by Day of the Week, Total Visits = 24,730 DISCUSSION Site evaluation is multidimensional. This study used analytical reporting techniques to understand the usefulness and how well the course Website has served the students 24/7. The study found that the course Website was accessed a total of 24,730 times by all 49 registered users of the course Website in a given semester, meaning that each student visited the Website approximately 4.5 times per day. The most popular Web component (area) was the content area, which held the course documents, and the least popular area was the student area. The students accessed the course Website 23 hours a day, 7 days a week. The popular hours were 9 a.m. and 9 p.m. The most popular day was Friday. An evaluation of the actual Web usage is more powerful and accurate than the self-report data. The findings are generalizable to the students of the current study. The students were given an option to drop this section and enroll in the other research methods courses, which had no Internet technology integration. However, none of the students dropped

12 Janaki Santhiveeran 191 FIGURE 4. Total Accesses by the Students, Total Visits = 24,730 the class after hearing about the depth of Internet technology integration. Therefore, the students who are enrolled in this course are self-selected and are aware of the nature of the web supplements introduced in the course. The availability of the Internet access affected the outcome, as the most popular day was Friday, when the students had access to computers during their class time. Schoech (2000) found that accessibility to computers is one of the important contributing factors for computer use in his study. Therefore, ready access of campus computers and computer terminals during the class time for the students on Fridays and Saturdays might have contributed partly to the highest usage on these days the students attended the classes. Other unknown factors such as motivation and Internet skills might have affected the outcome as well. Some of the study findings concur with prior research. Only onefourth of the students accessed the course Website extensively by visiting 12.9 times per day. This concurs with previous findings, which found that a group of graduate social work students tend to use the Internet technology extensively (Huff & Johnson, 1998). Huff and

13 192 JOURNAL OF TEACHING IN SOCIAL WORK Johnson found that the students primarily contacted the instructor about their grades. Similarly, the current study found that more than one-tenth of the students activities were to access their online grade book, which was a major feature in the student tools. Although the course Website is password-protected thus confining access to a registered students, the course Website traffic is several times more than is traffic to a social work Website in a public domain (Gonzalez & Huff, 1998). Since the users are defined and oriented properly about how to use the Web components, the user visits are high. Such training is not possible for the users of a public domain. Several factors including orientations, training, type of material, Web activities, faculty release time, and student characteristics and enthusiasm might have contributed to the highest visits logged in the course server. It is also important to note that more than half of the users were part-time students who had multiple responsibilities and attended classes primarily on weekends or evenings. The part-time students actively sought assistance via online forums. Similarly, prior research found that part-time students used the course Website actively (Schoech, 2000). The server administrator of the course Website also noted that the research methods course Website, discussed in this paper, had the highest visits in the given semester when compared to all other course Websites in the university server. Unlike Huff and Johnson (1998), all the students enrolled in the course utilized the course Website. Prior research found that course websites augment quality instruction (Schoech, 2000), however, the activity levels of the students show that the web supplements are heavily time consuming for the students. For the first time, analytical reporting techniques helped the instructor to understand the frequency of use, an important measure to understand the time demands and the usefulness of the course Website. Although the actual use of a course Website is limited in its scope, insights could be drawn from this understanding. Data mining offered (1) an objective means of evaluating the use of the course Website by the students, (2) a perspective on the time-demands placed by the Web supplements on the students and (3) a measure to validate in an objective manner whether the course Website has met the objective of serving students 24/7. The methodology followed in this study will offer an insight for faculty members who are struggling to understand the usefulness of their course Websites or Web components. The findings had a major effect on the author in making informed decisions about what to integrate in the following semesters. The instructor has minimized the

14 Janaki Santhiveeran 193 Web components and online interactions in other courses in order to minimize the workload for students. The Website evaluation using data mining has implications for social work programs that are seeking to develop, implement or evaluate distance learning. The study found that course documents are most frequently accessed by the students and are heavily used on the days students met with the instructor face to face. Therefore, the instructors who do not use Web supplements in traditional and ITV distance education courses could consider posting course materials such as syllabi, assignment guidelines and lecture outlines to enable students to access course materials in-between class sessions. Such dissemination of course materials will facilitate the quick dissemination of information for both traditional and distance education students and will reduce the costs associated with copying and mailing course materials to the students who are in the distance education programs. Although students were not given grades for the use of the Website, the course Website was used extensively by the students. This might have contributed to the outliers and disparity in the student uses. This also has implications for teaching and course planning both at the graduate and the undergraduate programs. Not offering grades for the use of Web components such as online participation might increase confidence among the students to learn the Internet technology by trial and error. The students used the Website day and night, 23 hours a day, during evening hours, non-university hours and weekends. The evidence of active use of the course Website has implications to planning online courses. It is important to note that the Website is designed to supplement a traditional graduate course, which met regularly three hours per week. At large, the study findings are expected to influence decisions on how and when to deliver Web-based course materials to meet the needs of virtual learners in both the graduate and undergraduate and the distance and on-campus courses. Instructors could design and implement synchronized Web-based activities during evening hours to meet the needs of both the full-time and part-time students. The study findings are limited to the visits to the course Website. However, the duration of time spent by the students is unknown. As a pilot project, the instructor was offered release time to implement several Web components for one section of a research methods course. Since the instructor was scheduled to teach two sections of a graduate research methods course, the instructor introduced all web components to both the sections. Consequently, the online activity levels increased considerably. By taking into consideration all these limitations, the

15 194 JOURNAL OF TEACHING IN SOCIAL WORK actual use is not transferable to other courses. Unlike prior research (Faux & Black-Hughes, 2000; Knowles, 2000), none of the online activities was graded; therefore, students used the Website voluntarily and felt comfortable in learning the required skills. Therefore, the students use could not be generalized to the courses in which students received part of the course grade for utilizing Web components. Since in both sections, both full-time and part-time students were combined, the differences between full-time and part-time students is unknown. The access levels were much more than usual, as Web supplements were introduced as a pilot project. The software did not offer means to track how many days in a term the course Website did not have any hits. Several of the students submitted portions of their assignments into the online forums to get feedback on an ongoing basis; the students accessed the site several times to read online messages and offer feedback to student postings. Therefore, the author suggests two measures: instructors need to set limits on the number, length, time, and types of online postings in higher education and second, instructors need to minimize Web components and online interactions to reduce the burden placed on students. Future site evaluations need to focus on the duration of time spent by both students and instructors in a course Website. Future research linking student characteristics and grades with the actual use could certainly offer a strong measure on the impact of the course Website on student outcomes. CONCLUSION Findings are presented about the use of a course Website, which was developed as a Web supplement for traditional face-to-face courses. The course Website was extensively used by graduate social work students, a strong measure to validate the usefulness of the website. On the other hand, the active use also shows the time demands placed by the course Website on the students and the ease of use. The actual use offers a perspective on the ease of navigating the Web components. Although the course website was made available 24/7, the study sample utilized the Website 23 hours a day, 7 days a week. The methodology offered a new perspective in studying the actual use, a reliable measure to evaluate the usefulness of the course Website.

16 Janaki Santhiveeran 195 REFERENCES Bainbridge, W. S. (1995). Sociology on the World Wide Web. Social Science Computer Review, 13, Blackboard Inc. (2002). Welcome to Blackboard. Retrieved Aug 27, 2002, from Davis, S. F., Smith, Brandy, G., Rodrigue, K., & Pulvers, K. (1999). An examination of Internet usage on two college campuses. College Student Journal, 33(2), Faux, T.L. & Black-Hughes, C. (2000). A comparison of using the Internet versus lectures to teach social work history. Research on Social Work Practice, 10(4), Finn, J., & Smith, M. (1997). The use of the World Wide Web by undergraduate social work education programs. The Journal of Baccalaureate Social Work, 3(1), Galambos, C. & Neal, C. (1998). Untangling the net: Using policy resources in the classroom. Conference program and proceedings: Information Technologies for Social Work Education and Practice, (pp ). Columbus: University of South Carolina College of Social Work. Gifford, E. D. (1998). Social Work on the Internet: An introduction. Social Work, 43(3), Gilbride, D., Breithaupt, B., & Hoehle, R. (1996). The use of the Internet to support both on- and off-campus learners in rehabilitation education. Rehabilitation Education, 10(1), Gonzalez, J. E., & Huff, D. (1998). Assessing the impact of instructional technology: The case of the Cyber-History of Social Work. Conference program and proceedings: Information Technologies for Social Work Education and Practice, (pp ). Columbus: University of South Carolina College of Social Work. Howard, P. E. N., Rainie, L., & Jones, S. (2001). Days and Nights on the Internet. American Behavioral Scientist, 45(3), Huff, M. T., & Johnson, M. M. (1998). Students use of and a Listserv in Distance Education Courses. Conference program and proceedings: Information Technologies for Social Work Education and Practice, (pp ). Columbus: University of South Carolina College of Social Work. Knowles, A. (2000). Implementing Web-based learning: Evaluation results from a mental health course. Conference program and proceedings: Information Technologies for Social Work Education and Practice (CD ROM). Columbus: University of South Carolina College of Social Work. Rosen, E. F., & Petty, L. C. (1997). Using Internet resources in a research methods course. Behavioral Research Methods, Instruments & Computers, 29(2), Santhiveeran, J. (1998). Social Work Online (SOLE): A dynamic Website in social work. Conference program and proceedings: Information Technologies for Social Work Education and Practice, (pp ). Columbus: University of South Carolina College of Social Work. Santhiveeran, J. (2002). Focus groups: An approach for summative evaluation of Webenhancement. Professional Development: The Journal of Continuing Education in Social Work, 5(2).

17 196 JOURNAL OF TEACHING IN SOCIAL WORK Schoech, D. (2000). Teaching over the Internet: Results of one doctoral course. Research on Social Work Practice, 10(4), SPSS Inc. (2002). SPSS. Retrieved Dec 7, 2002, from Stocks, J.T. & Freddolino, P.P. (2000). Enhancing computer-mediated teaching through interactivity: The second iteration of a world wide Web-based graduate social work course. Research on Social Work Practice, 10(4), Sunal, C. S., Smith, C., & Sunal, D. W. (1998). Using the Internet to create meaningful instruction. The Social Studies, 89, Wilson, R. C. (1986). Improving faculty teaching: Effective use of student evaluations and consultants. Journal of Higher Education, 57(2), doi: /j067v26n03_12