College student Web use, perceptions of information credibility, and verification behavior

Transcription

1 Computers & Education 41 (2003) College student Web use, perceptions of information credibility, and verification behavior Miriam J. Metzger a, *, Andrew J. Flanagin a, Lara Zwarun b a University of California, Santa Barbara, Department of Communication, Ellison Hall, Santa Barbara, CA 93106, USA b Department of Communication, University of Texas at Arlington, Box 19107, Arlington, TX 76019, USA Received 21 June 2002; accepted 22 April 2003 Abstract Concerns about the potentially dubious nature of online information and users ability to evaluate it appropriately prompted this research on college students use of Web-based information, their perceptions of information credibility, and their online verification behaviors. Two studies were conducted to address these issues. Results of the first study show that college students rely very heavily on the Web for both general and academic information, and that they expect this usage to increase over time. Results of the second study indicate that students find information to be more credible than do those from a more general adult population, across several media and considering many different types of information. Nonetheless, students verify the information they find online significantly less. Implications are discussed in light of current efforts of educators to improve Internet literacy. # 2003 Elsevier Ltd. All rights reserved. Keywords: Media in education; Multimedia/hypermedia systems; Pedagogical issues; Post-secondary education; Teaching/learning strategies 1. Introduction Studies indicate that as online information resources proliferate, college students are making fewer visits to the campus library to retrieve information, and that their use of the World Wide Web as an information resource is increasing overall (Lubans, 1998, 1999; Pew Research Center, * Corresponding author. Tel.: addresses: metzger@comm.ucsb.edu (M.J. Metzger), flanagin@comm.ucsb.edu (A.J. Flanagin), zwarun@ uta.edu (L. Zwarun) /03/$ - see front matter # 2003 Elsevier Ltd. All rights reserved. doi: /s (03)

2 272 M.J. Metzger et al. / Computers & Education 41 (2003) ; Shackleford, Thompson, & James, 1999). In fact, college students are increasingly taking advantage of the Internet and Web to complete course assignments (Browne, Freeman, & Williamson, 2000; Lubans, 1998, 1999; Pew Research Center, 2002; Shackleford et al., 1999). 1 The movement away from the library and toward cyberspace raises important concerns about the nature of information obtained by students and their ability and desire to evaluate online information effectively. Accordingly, groups such as the American Library Association, the National Institute for Literacy, and myriad librarians and instructors have called for schools and universities to teach Internet literacy skills, with an emphasis on evaluating the quality of Web-based information. While the number of students using the Internet grows annually (National Telecommunications and Information Administration, 1999), studies of the nature of students Web use have not kept pace. There is remarkably little research addressing how frequently students use the Internet and Web, their motivations for doing so, and what efforts, if any, they make to verify that the information they find online is credible. However, this is a crucial topic to monitor, given students use of the Web as a source of information and the potential for online information to be misleading or inaccurate. There are several characteristics of the Internet that make it susceptible to accuracy or credibility concerns, for example, its lack of centralized control and regulation, the ease of electronic sabotage and content alteration, and its blurring of traditional genre distinctions. These characteristics are examined in turn. Unlike most other mass media, the Web has few centralized information filters relative to the amount of information available (Flanagin & Metzger, 2000; Johnson & Kaye, 1998; Rieh & Belkin, 1998). Although sites that parallel their traditional media counterparts, such as the New York Times Online or MSNBC, do have formal information gatekeepers, the vast majority of Web sites do not. Instead, most sites have no explicit editorial review policies that serve as a means to analyze content and to verify factual information posted on the Web. Johnson and Kaye point out that this lack of editorial review in the online environment results in less social and professional pressure to ensure accuracy of Web information compared to more traditional sources of information. The fact that lawmakers have yet to establish regulatory policy concerning Web-based information further exacerbates this problem (Alexander & Tate, 1999). The result is both ambiguity for those who are trying to set standards for online material and limited social, political, or legal pressure for Web content providers to ensure accuracy. 1 Terminology to describe the Internet, Web, and related technologies has often been used inaccurately or inconsistently. The Internet refers to the physical infrastructure of interconnected computers, cables, and other devices that serves as the infrastructure for global communication. The Web refers to a system of computers ( servers ), utilizing graphical user interfaces and accessed via the Internet, that provides access to documents, multimedia files, and Web sites, that are connected by hyperlinks to other documents, multimedia files, and Web sites. Although the Internet does encompass the Web and other resources (such as chat rooms), the Web typically refers only to hypertext linked sites and their content. Nonetheless, research often groups the Internet and Web together, particularly when addressing credibility issues, because many users/respondents tend not to make the distinction noted above. Although in this article we strive to use the terms appropriately, at times we refer to the Internet and Web together, when the overlap is informative and relevant. In the data collection, we were careful to cue respondents to whether they should attend to their views about the Internet (infrastructure and access issues) or the Web (Web-based information content). This research, however, did not focus on other Internet resources such as Usenet or chat rooms.

3 Web-based information also differs from information delivered through more traditional channels in that it is more prone to alteration, which can be difficult to detect (Alexander & Tate, 1999; Flanagin & Metzger, 2000). As Mitchell (1992) points out, digital information is particularly susceptible to electrobricollage that renders traditional notions of originals and copies meaningless, while also making the location of the original source of information problematic. Web-based information may be altered unknowingly as well, as when technical problems occur when information is uploaded or downloaded (Alexander & Tate, 1999). The issue of altering electronic information without detection is coupled with the problem that many Web sites do not have established reputations that can aid users in assessing the sites veracity (except those with well-known offline counterparts). Without reputational cues, users may rely on more superficial information in evaluating Web sites. Additionally, regardless of authorship, professional-quality Web sites are easy to create and can appear credible, even when they are not (Alexander & Tate, 1999; Flanagin & Metzger, 2000; Johnson & Kaye, 1998; Rieh & Belkin, 1998). Lastly, the convergence of information genres on the Web, particularly the blending of advertising and informational content, may be problematic for people trying to establish the credibility of online information (Alexander & Tate, 1999; Flanagin & Metzger, 2000). Unlike traditional print publishing, it may be harder to discern which is which in the online environment because information is often presented seamlessly, without clear visual distinctions between advertising and information. Alexander and Tate (1999) also note that it is sometimes difficult to discern if the advertising and informational content on a Web site are produced by the same organization. Not knowing the source of information or its intent makes it difficult to know whether to trust it. In sum, there are several factors to suggest that those relying on Web-based information should scrutinize it carefully Implications for students M.J. Metzger et al. / Computers & Education 41 (2003) The Internet is used increasingly for educational purposes. Many campuses are requiring students to have computers to take advantage of Web-accessible classrooms, and many educators are incorporating the Web into their curricula (Gunaratne & Lee, 1996; Perry, Perry, & Hosack- Curlin, 1998; Pew Research Center, 2002; Shelton, Lane, & Waldhart, 1999; Sutherland & Stewart, 1999). Among the proposed advantages of incorporating advanced technologies into instruction are accomplishing new or existing tasks better and more efficiently, preparing students for the job market, enhancing productivity, and reaching students who are unable to attend classes on campus (Albright & Graf, 1992; Witmer, 1998). One of the most basic uses of the Internet by students is to search for sources and information to complete course assignments. The Internet s ability to provide students quick access to government documents, scholarly listservs, and databases located at geographically-removed institutions makes it a valuable information source for students (Benson, 1994; Browne et al., 2000; Lubans, 1998; Ryan, 1994). Yet, along with these sources, the Web also contains millions of other Web sites that are operated by individuals, businesses, advocacy groups, clubs, and so on, which may offer inaccurate or biased information. To deal with this, students using the Internet need help. University reference librarians have become expert Web users in order to give students the assistance they need in finding relevant and credible information, but students do not always take advantage of librarians expertise (Lubans, 1998). In fact, recently, college librarians reported a

4 274 M.J. Metzger et al. / Computers & Education 41 (2003) decrease in the number of reference desk visits by students as campus online services have proliferated (Pew Research Center, 2002). Furthermore, Lubans (1998) reports that librarians believe that student use of the Internet is at an alarmingly un-informed level (p. 2). Students instead rely on their own expertise or the advice of friends to navigate the Web: many report that they are good users of the Web, and that they are more likely to learn about the Web from surfing it by themselves or from classmates than from library staff (Lubans, 1998). So, although many students use the Internet for their schoolwork, they often do so without formal help or training. This situation has important implications for the quality of work that students are able to produce based on their Internet use. Students need accurate, timely information from which to learn, but if they are faced with information that is inaccurate or misleading, and they lack the ability to distinguish this information from more credible sources, their learning is compromised. They may mistake erroneous or biased information for fact, and they may perpetuate such error or bias in their own work. Moreover, students tend to be pragmatic when it comes to their assignments and are not necessarily concerned about the Web s potential for misleading information. For example, Lubans (1999) found that although a majority of students feel that the Internet helps them find a large number of sources, a minority believe that the Internet improves the overall quality of their work. This finding suggests while students appreciate the Internet s ability to provide them with an array of information resources, the quality of those resources is not necessarily important to them. The cavalier attitude that these students seem to have about the quality of their learning is inconsistent with the goals of a college education. Therefore, the issue of the credibility of Webbased information is extremely important when it is considered as a source of information for students in educational contexts. To assess students use of Web-based information sources, and to determine the extent to which they act as critical consumers of the information that they obtain, we performed two studies of college students use of online information resources. First, we conducted a survey of college students that examines the nature of their Web use. With this as a basis, we then compared students to nonstudents in terms of how credible they believe the Internet to be in relation to other media, and what strategies they use to assess the quality of online material. Together, these two studies provide valuable data on the relationship between a particular population of information seekers, college students, and a significant source of their information, the Web. 2. Study 1: college students use of the Web Research shows that people rely a great deal on the Internet for retrieving information, and that college students are a particularly avid user group of this technology (e.g., Flanagin & Metzger, 2001; Pew Research Center, 1999, 2002). Reasons for this are provided by diffusion of innovations theory (Rogers, 1995) and the technology acceptance model (TAM; Davis, 1989), both of which also suggest future trends of Web usage and reliance. Diffusion theory notes that several factors tend to increase the likelihood of innovation adoption, for example, the relative advantage of the innovation, high trialability and observability, and the innovation s compatibility with one s existing values, beliefs, and current needs. Furthermore, the TAM says that

5 M.J. Metzger et al. / Computers & Education 41 (2003) adoption of an information technology is a function of individuals perceptions that the technology is easy to use and is useful for them. Based on these perspectives, college students heavy usage of the Web can be understood as a function of their familiarity with the Internet and Web, their ample opportunities to witness its use and experiment with it, and the resultant comfort they gain with this tool. Indeed, today s college students have grown up with the Internet and thus may perceive it as very easy to use. Moreover, given students strong need for large amounts of timely information, they may think of the Internet as a particularly useful tool. In fact, recent data (Pew Research Center, 2002) indicate that the current generation of college students is quite comfortable with computer use and many began using computers during early childhood (20% began between the ages of 5 and 8). It is surprising, then, that despite students tendency to be active users of computers, and the regular need for gathering information inherent in their education (Rieh & Belkin, 1998), only a small number of studies have specifically examined the nature of students Web usage. The few studies that exist show that college students have embraced the Internet as a means of making information more readily available. A survey of university students in 2002 found that 85% reported having their own computer at college and 57% said they used the Internet for something other than at least once a day (Pew Research Center, 2002; see also Hester, 1999). Many students perceive the Web as a vast library with speedy retrieval and no closing hours (Blake, 2000, p. 4). Furthermore, Perry et al. (1998) found that significantly more younger students (ages years) than older students (age 26 or older) reported using the Internet to obtain university information, illustrating that students may be increasingly turning to the Internet as a source of information in the near future. However, most studies that assess how frequently students use the Internet do not specify exactly what students are using it to do. For example, Lubans s study (1999) reports that was the most common usage, followed by using library guides and databases, but he does not report whether those databases were used for academic or nonacademic searches, or how the information found there was used. In contrast, the Pew Center s (2002) survey of over 2000 college students examined academic uses of the Internet in some detail and found that students are using Internet primarily to communicate with their professors, search for information, contact other students, and carry on discussions with classmates. Students seem to like using the Internet for their studies: the Pew Center (2002) found that 79% of college students feel the Internet has had a positive impact on their academic experience. Interestingly, however, the study provides no data on specifically how the technology impacts students education. Also, only one study has examined the importance of Web-based information relative to information from more traditional sources, and this study was conducted using focus groups of 7 10th graders (Lubans, 1998, 1999). One-third of these students reported using the Web and print resources equally, and younger students demonstrated a bias toward the Web over print media (Lubans, 1999). Recognizing these gaps in the research literature, Study 1 was conducted in order to better assess college students usage and reliance on the Internet as a source of both general and academic information. As mentioned earlier, the TAM predicts that favorable perceptions of the ease of use and usefulness of a technology lead to its adoption and future use. If students perceive the Internet to be easier and more convenient than older channels, they are likely to use the Internet more than older channels; and if through this experience they find the Internet useful for

6 276 M.J. Metzger et al. / Computers & Education 41 (2003) their schoolwork, then use of the technology will likely increase in the future. Based on this, the following research questions were posed: RQ1: How often do college students use the Internet? RQ2: For what general purpose(s) do college students use Web-based information? RQ3: For what academic purpose(s) do college students use Web-based information? RQ4: Compared to other information sources, how often do college students use the Internet to complete course assignments and papers? RQ5: In what ways do college students perceive the Web to impact their schoolwork? RQ6: Do college students believe that their reliance on the Web as both an academic and a general information source will increase in the future? 2.1. Method Participants and procedure Data for this study came from 356 undergraduate students enrolled in introductory communication courses at a large, public university located on the west coast of the U.S. 2 The students ranged from freshmen to seniors, although about half (51%) indicated that they were in their sophomore year of study and most had not yet declared a major. Using students from a non-elite public university helped to ensure the generalizability of the findings to the larger population of U.S. college students attending 4-year institutions. The student participants reported moderate Internet expertise on average (M=3.89, S.D.=1.7, on a seven-point scale), which is similar to U.S. national estimates (Pew Research Center, 2002). All participants were asked to complete an anonymous questionnaire about their use of several information channels, including the Internet. The questionnaire took about 10 minutes to complete and respondents received extra course credit as an incentive for their participation General Internet usage Students rated the frequency that they used the Internet for any purpose. Response choices included never, less than once a week, about once a week, several times a week, and every day. Frequency of use for specific purposes was assessed on a five-point scale ranging from 1= never to 5= every day. Respondents indicated how often they used the Internet for general (not academic) research, and to get entertainment, news/current events, and business/ financial information. Examples were given to make each of these purposes clear to respondents. In addition, participants were asked in an open-ended item if there were any other general purposes for which they used the Internet. 2 A total of 515 students were originally surveyed, but due to the fact that many more females (n=337) were enrolled in the courses than males (n=178), a subsample of the female students was taken to equalize sex. The subsample was derived by randomly eliminating 159 females from the original sample. Comparisons of the females who were selected for inclusion in the dataset showed no differences from those who were not selected on any of the key variables in this study.

7 M.J. Metzger et al. / Computers & Education 41 (2003) Academic Web usage Due to the dearth of research on the topic and the exploratory nature of this study, an openended measure was used to determine how students use the Web for academic purposes. Responses were itemized and then tallied. In addition, participants were asked to indicate the frequency that they used a variety of information sources for completing their schoolwork on a 5- point scale, ranging from 1= never to 5= all the time. The information sources included the Internet, books, academic journals, magazines, and newspapers Benefits of using the Internet for academic purposes Respondents assessed the impact of online information in terms of how helpful they felt Web sites on the Internet were to them (excluding the campus library s Web site) in several aspects of their schoolwork. Specifically, students indicated the degree they felt that Web-based information increased the quality of their written work, the use of their time looking for information, the number of sources they typically find, and their grades on five-point scales anchored by 1= very unhelpful and 5= very helpful. In addition, an open-ended item asked students to indicate if there were any other ways they felt the Web benefited their schoolwork Future reliance on the Internet as an information resource Following Perry et al. (1998), two items measured students perceived future reliance on the Internet. Respondents were asked Two years from now, do you think your use of the Internet as a general information source will increase, stay the same, or decrease? and, later in the questionnaire, Two years from now, do you think your use of the Internet for your schoolwork will increase, stay the same, or decrease? 2.2. Results For Research Questions 1, 3, and 6, percentages were used to summarize data within response categories for ease of comparison to prior research. Chi square tests were used to examine differences between sex and class level because of the categorical nature of these variables. Research Questions 2, 4, and 5 were tested via repeated-measures ANOVA because, in each case, subjects mean responses over multiple items were being compared for statistical significance and, thus, observations were not independent. Repeated contrasts were used as the follow-up tests and Bonferroni adjustments were used whenever multiple statistical tests were made in order to hold experimentwise error at the 0.05 level. Unlike omnibus tests, the follow-up contrasts allowed specific differences in respondents answers to be located for a more nuanced understanding of the data. With regard to the first research question, results demonstrate that the students in this study, like most others, are heavy users of the Internet. A vast majority of the students reported using the Internet every day (51.0%) or several times a week (29.9%). 14.9% reported using the Internet about once a week. Interestingly, only a few students (4.2%) said that they used the Internet less than once a week and none said they never use the Internet. Cross tabulations indicated that there were no significant differences in either sex or class level (freshmen/sophomores versus juniors/seniors) on overall frequency of Web use.

8 278 M.J. Metzger et al. / Computers & Education 41 (2003) Beyond general usage of the Internet, Research Question 2 was interested in the students use of the Internet for specific purposes including entertainment (leisure time chat, playing games, etc.), nonacademic research (finding medical information, driving directions, travel tips, etc.), news and current events information, and conducting business transactions (purchasing products and services, online banking, etc.). Table 1 shows the mean usage scores for each of the specific purposes. A repeated-measures ANOVA showed a significant main effect for type or purpose of use [F(3, 864)=287.43, P<0.001, 2 =0.50], and follow up repeated contrasts indicated that the students use the Web for entertainment significantly more than they use it for nonacademic research, or for news and business information. 3 The ANOVA also showed a significant interaction between type of use and sex [F(3, 864)=16.80, P<0.001, 2 =0.06]. Follow-up contrasts with Bonferroni adjustments revealed that the females used the Web significantly more than did the males for entertainment, and that the males used the Web significantly more for news and business than the females did. Both sexes report using the Web equally for nonacademic research. Odell, Korgen, Schumacher, and DeLucchi (2000) found similar results for male and female students uses of the Internet for all but entertainment they found that males used the technology for entertainment more than females did. This discrepancy can be explained, however, by the fact that was not included as entertainment in their study, whereas it was in the present research. There were no significant differences for class level. The open-ended responses showed that the students used the Internet and Web largely for school purposes, banking and stock market information, , checking sports scores, and downloading music. Other, less-common uses were for finding jobs/internships, weather information, and pornography. These uses parallel those found by other researchers (see Odell et al., 2000; Pew Research Center, 2002). An open-ended question was also used to answer the third research question. By far the most frequently cited school-related use of the Web was to do research/get information (mentioned by 89% of the respondents). ing instructors and viewing course Web sites were the second and third most-often cited uses (mentioned by 51 and 32% of the respondents, respectively). Accessing library information and databases was another popular use (20%), as was checking and completing homework assignments (14%). Other, less common purposes were getting class notes, news information, and registering for classes/checking grades. These results are similar to those of Table 1 Frequency of use of the web for general purposes (N=356) Purpose M S.D. Entertainment Nonacademic research News 2.51a 1.16 Business 2.40a 1.04 Common letters indicate that means are not significantly different. The questions were measured on a five-point scale anchored at 1= never, 2= rarely, 3= occasionally, 4= often, and 5= all the time. 3 F-tests, degrees of freedom, and significance levels for these and all other contrasts are available from the first author.

9 M.J. Metzger et al. / Computers & Education 41 (2003) the Pew Center (2002) study, which found heavy use of the Internet by students for contacting classmates and professors, searching for course-related material, and obtaining grade information. To answer Research Question 4, the students indicated how often they used the Web for academic information in comparison to other popular sources of school-related information. A repeated-measures ANOVA indicated a significant main effect for information source [F(4, 1160)=136.01, P<0.001, 2 =0.32]. The students reported using books significantly more than the Internet and academic journals, which they reported using significantly more than newspapers and magazines. Although books were the most frequently used information source (students reported using them often ), the Internet was the second most frequently used source. These results are summarized in Table 2. In addition to the main effect for source of information, there was also a significant interaction between source and sex [F(4, 1160)=4.30, P<0.01] and between source and class level [F(4, 1160)=5.78, P<0.001], but very little variance was explained by these factors (only 2% each). Follow-up tests revealed that the females reported using books and journals more than the males, the males reported using newspapers and magazines more than the females, and both sexes reported using the Internet equally often for schoolwork. Finally, older students (juniors/seniors) reported using journals more and newspapers less than younger students (freshmen/sophomores) in this sample. This finding may reflect the fact that as students gain experience in college courses, they are socialized to the norm of using academic journals in university-level scholarship. Finally, older and younger students reported using the Internet equally. With regard to the fifth research question, the students indicated that the Internet was moderately to somewhat helpful for increasing the number of sources they found in researching course assignments and saving time searching for information. The students felt the Internet was only slightly useful in achieving higher grades and improving the quality of their written work. A repeated-measures ANOVA indicated a main effect for type of help [F(3, 852)=22.41, P<0.001, 2 =0.07] and follow-up contrasts showed significant differences for all means (see Table 3 for a list of means and standard deviations). There were no significant sex or class level differences. From the open-ended item that asked if there were any other ways that the Internet benefited students schoolwork, several respondents reported that the Internet helped them find a diversity of perspectives on topics they were researching. Table 2 Students use of various information sources for schoolwork (N=356) Information source M S.D. Books Internet 3.59a 0.98 Journals 3.46a 1.08 Newspapers 2.68b 0.99 Magazines 2.63b 1.01 Common letters indicate that means are not significantly different. The questions were measured on a five-point scale anchored at 1= never, 2= rarely, 3= occasionally, 4= often, and 5= all the time.

10 280 M.J. Metzger et al. / Computers & Education 41 (2003) The last research question asked about students use of the Web as an information source in the future. The students overwhelmingly reported that they felt their reliance on the Web in the future for general information would increase (80%), as opposed to stay the same (19%) or decrease (1%). These differences are significant [w 2 (2)=151.74, P<0.001], and this response pattern held for both sexes and across both class levels. Because the question assessing future reliance on the Web for academic purposes is largely not applicable to students who are about to graduate, only the data from the freshmen and sophomores were used for this analysis (n=160). These students overwhelmingly saw their use of the Web for academic purposes growing in the future (64%), as opposed to remaining at its current level (32%), or decreasing (4%). These differences were significant [w 2 (2)=84.76, P<0.001]. In all, the results indicate clearly that the students in this study are using the Internet and Web extensively for both general and academic purposes and will continue to do so into the future. 3. Study 2: credibility and verification of Web-based information The results of Study 1 demonstrate that college students rely extensively on the Internet for both general and academic uses. This, in turn, prompts questions about how students assess Webbased information. For instance, given their heavy usage patterns, to what degree do students feel the Web provides credible information as compared to more traditional information sources? Also, because of the need to get accurate, timely information, are students particularly vigilant in verifying the information they receive on the Web? Finally, how do students compare to other user groups in both their perceptions of credibility and their verification strategies? Study 2 considers these issues. Little prior research has focused on the issue of Internet credibility and, of that research, none has considered the perceptions of different user groups or of students in particular. Furthermore, studies of online information credibility have concentrated primarily on news or political information obtained by users, to the exclusion of other types of information (Johnson & Kaye, 1998; Mashek, 1997; Pew Research Center, 1999). Like the college students in Study 1, all Internet users are pursuing a wider variety of online information, increasingly seeking, for example, entertainment and financial information in addition to news (Pew Research Center, 1999). The many Table 3 Students perceptions of how the web helps their schoolwork (N=356) Aspect of schoolwork M S.D. Number of sources found Use of time searching for information Grades Quality of written work All means are significantly different. The questions were measured on a five-point scale anchored at 1= very unhelpful to 5= very helpful.

11 different kinds of information available, and the shift toward obtaining non-news information via the Internet, suggest that information type may be an important factor in the assessment of Internet information credibility. In addition, regardless of whether or not users perceive information delivered via the Internet to be credible, efforts to validate that information are important indicators of users behavior and should serve as guides for the formation of appropriate Internet policy. Validation is often achieved by referring to sources with credible reputations, seeking the advice of trusted others, and by relying on personal experience to determine the characteristics of trustworthy sources. However, with information obtained over the Internet, these same strategies are not always effective or available (Gilster, 1997; Ward, 1997). Accordingly, although Internet literacy began as the set of basic skills required to access the medium (involving such things as the ability to use a Web browser, for example), it has shifted away from this skills-based view and moved toward the ability to critically evaluate information from media sources. Contemporary notions of Internet literacy thus now include higher-level processing of information, which implies being selective about information, making informed judgments about content, and evaluating the impact of that information appropriately (Cortes, 1992). The questions remain, however, to what extent have such efforts been successful and to what degree are users exercising this more critical aspect of Internet literacy? Thus, informed by the results of Study 1 and the above rationale, we pose a series of research questions in Study 2 intended to address the issues of Internet credibility and verification strategies, comparing students to nonstudent Internet users. Our research questions are guided by the TAM and diffusion theory, which suggest that characteristics of different user groups (e.g., the innovation s compatibility with the values of college students, perceptions of the technology s usefulness, etc.) might suggest differences in usage behaviors across these groups. Moreover, because experience with Internet has been shown to affect both credibility perceptions and verification behavior (Flanagin & Metzger, 2000), these factors are considered in the following research questions as well. RQ1a: What are students perceptions of the relative credibility of information sources such as television, newspapers, Internet/Web, radio, and magazines? RQ1b: Do students and nonstudents differ in their perceptions of the relative credibility of these information sources? RQ2: Do students and nonstudents vary in their perceptions of the credibility of various types of information (i.e., news, reference, entertainment, and commercial) across different media? RQ3: To what extent do student and nonstudent users verify online information and what specific verification strategies do students, in particular, use? 3.1. Method M.J. Metzger et al. / Computers & Education 41 (2003) Participants and procedure Student and nonstudent respondents for this study were recruited using convenience sampling. The student sample came from communication courses at the same university used in Study 1 (N=436). 4 Nonstudent respondents were recruited using a snowball sampling technique whereby a team of eight research assistants was instructed to find at least five respondents varying

12 282 M.J. Metzger et al. / Computers & Education 41 (2003) in geographic locale who would complete the questionnaire, and who would themselves find five more participants to complete the survey. This technique resulted in 307 completed surveys from nonstudent respondents who resided in three different states, representing 116 unique zip codes. Respondents in the nonstudent sample were slightly younger, wealthier, and more educated than the average US adult population, which makes them fairly representative of typical Internet users (see National Telecommunications and Information Administration, 2000). The students differed significantly from nonstudents in age [M student=19.96, M nonstudent=33.68, t (309.62)= 18.15, P<0.001], years of education [M student=14.11, M nonstudent=15.28, t (391.18)= 8.58, P<0.001] and income [M student=6.51, M nonstudent=5.41, t (661.19)=4.01, P<0.001] 5 but, overall, their Internet usage, ability to access, and familiarity with Internet information were not significantly different. The sole difference between the samples in terms of Internet behavior was that students had slightly more experience using the Internet (M=4.49) than did nonstudents [M=4.07, t (564.24)=3.06, P<0.01; measured on a seven-point scale], a finding implied by the results of Study 1. Because respondent fatigue was a factor, two versions with varying question order were administered randomly. All respondents were ensured anonymity; undergraduate participants received extra course credit for participation in the study, whereas nonstudents participated voluntarily Internet credibility The measurement of media credibility has been the subject of much debate and studies show that the way in which it is operationalized can make a difference in credibility ratings (Gaziano & McGrath, 1986; Meyer, 1988; Newhagen & Nass, 1989; Rimmer & Weaver, 1987; Shaw, 1973; West, 1994). However, a recent literature review performed by Flanagin and Metzger (2000) identified believability, accuracy, trustworthiness, bias, and completeness of information as the most consistent dimensions of media credibility. Hence, participants were asked to rate information obtained via the Internet on these dimensions using seven-point scales ranging from 1= not at all to 7= extremely. Bias scores were reverse-coded so that higher scores on all dimensions indicated higher credibility perceptions Verification of Internet information Items used to assess the degree to which respondents verify the information they obtain via the Internet were derived from a number of on- and off-line sources that provide guidelines for evaluating the quality of online information. Five types of verification strategies emerged from the literature: accuracy, authority, objectivity, currency, and coverage (see Alexander & Tate, 1999; Brandt, 1996; Gilster, 1997; Harris, 1996; Jones, 1998; Smith, 1998). Accuracy refers to the 4 The original sample consisted of 723 students, but because of a gender imbalance similar to that of Study 1 in the college sample, a random subsample of the female students was taken. As with Study 1, tests showed no differences between the females who were selected randomly to be included in the study from those who were not selected. The final sample for Study 2 consisted of 436 students and 307 nonstudents (49.3% male and 50.7% female overall). 5 Student income is higher than nonstudent income because students typically reported their parents income, whereas nonstudents, who were relatively young (M=33.68) and not that long out of college themselves, reported their personal income. A mean annual income of 6.51 corresponds to between $50,000 and $70,000, and a mean income of 5.41 corresponds to between $40,000 and $60,000.

13 degree to which the Web site is free from errors, whether the information can be verified elsewhere offline, and the reliability of the information on the site. The authority of a Web site is indicated by who authored the site and whether contact information is provided for that author, whether the author s qualifications and affiliations are listed, and whether the Web site is recommended by a trusted source. Objectivity involves identifying the purpose of the site and whether the information provided is fact or opinion. Currency is how up-to-date the information is, and coverage is the comprehensiveness or depth of the information provided on the site. Based on this research, nine items were used to assess the degree to which respondents verified the information they retrieved from the Web on a five-point scale, where 1= never, 2= rarely, 3= occasionally, 4= often, and 5= all the time. The nine items were combined into a verification scale (Cronbach s alpha=0.92 across all information types; see Table 6 for scale items) Type and sources of information Because perceived credibility may vary depending upon the type of information sought and retrieved, the credibility of four types of information was assessed: news or current events information, entertainment information, reference (factual) information, and commercial or product information. Short descriptions and examples were provided to aid respondents understanding of what was meant for each type of information. 6 Based on past research, current usage, and the availability of all information types included in this study, five information sources were selected for this study: the Internet/Web, newspapers, television, magazines, and radio Results M.J. Metzger et al. / Computers & Education 41 (2003) Research Questions 1 and 2 were analyzed using repeated-measures ANOVA because, as in Study 1, observations were not independent. The comparison of students to nonstudents necessitated t-tests as follow up tests, and experimentwise error was controlled via Bonferroni adjustments. Research Question 3 was assessed by examining means on the verification items, and a t-test was employed to evaluate if there were differences in overall verification between the student and nonstudent samples. With regard to RQ1a, students in this study reported that they perceived the Internet to be moderately to somewhat credible (M=4.09, S.D.=0.82), although less credible than newspapers, television, and magazines. Compared to the nonstudent sample, which was the focus of RQ1b, the students found all information channels to be more credible than did the nonstudents, as 6 News and current events information were described as information you get about political events, public interest items, or current affairs. Respondents were told that this information is about what is going on in the world, either locally or elsewhere. Reference or factual information was information such as things you might want to look up or refer to and examples included directories (addresses, phone numbers), maps, dictionaries or encyclopedias, or facts you are interested in (such as sports trivia or quotations). Commercial or product information referred to information you might get about products you are considering purchasing or about companies you want to learn more about that can include descriptions, prices, or ratings of products you are thinking of buying or general information about an organization or company. Finally, participants were told to think of entertainment as information that appeals primarily to you for entertainment purposes, such as entertainment programming, games, hobbies, pornography, or gossip.

14 284 M.J. Metzger et al. / Computers & Education 41 (2003) indicated by a significant between-subjects (student status) effect in a repeated-measures ANOVA (F[1, 738]=24.97, P<0.001, 2 =0.03). T-Tests with Bonferroni corrections to hold experimentwise error at the 0.05 level indicated that, although the students differed significantly from the nonstudents in their perceptions of the credibility of all traditional media, they did not differ in their ratings of the credibility of the Internet specifically (see Table 4). RQ2 was tested using a repeated-measures ANOVA, which showed a significant main effect for information type (F[3, 2202]=160.55, P<0.001, 2 =0.18) and a significant between-subjects effect for student status (F[1, 734]=24.16, P<0.001, Z 2 =0.03). This indicates that both the students and nonstudents reported that the various types of information were differentially credible. Follow up t-tests with Bonferroni corrections show that the students reported finding all types of information to be more credible than did the nonstudents, with the exception of commercial information, which did not differ between the subsamples (see Table 5 for a summary of these results). With regard to Internet-based information specifically, both the students and Table 4 Students versus nonstudents relative credibility ratings Information source Students (n=436) Non students (n=307) M S.D. M S.D. t Newspapers * Television 4.17a b * Magazines 4.14a b * Internet 4.09a Radio 4.07a b * Within each column, matching letters indicate that the means are not significantly different. Credibility was measured on seven-point scales where 1 indicates not at all credible and 7 indicates very credible. Please see Study 2 Methods section for the items comprising the credibility scale. * P< Table 5 Student versus nonstudent credibility ratings of various information types Information type Students (n=436) Non students (n=307) M S.D. M S.D. t News a ** Reference a ** Entertainment * Commercial Within each column, matching letters indicate that the means are not significantly different. Credibility was measured on seven-point scales where 1 indicates not at all credible and 7 indicates very credible. Please see Study 2 Methods section for the items comprising the credibility scale. * P<0.01; **P<0.001.

15 M.J. Metzger et al. / Computers & Education 41 (2003) Table 6 Students verification strategies of online information (N=436) Strategy M S.D. Check to see if the information is current Check to see if the information is complete/comprehensive Consider whether the views represented are facts or opinions Seek out other sources to validate the information online 2.75a 0.93 Consider the author s goals/objectives for posting information 2.75a 0.91 Check to see who the author is Look for a stamp of approval or recommendation Check if contact information is provided for the author Verify the author s qualifications or credentials Common letters indicate that means are not significantly different. The questions were measured on a five-point scale anchored at 1= never, 2= rarely, 3= occasionally, 4= often, and 5= all the time. nonstudents found news and reference information to be more credible than either entertainment or commercial information. 7 The last research question sought to determine the extent to which students verify the information they find online, compared to nonstudents, and what specific strategies they use in doing so. There was a statistically significant difference between the students (M=2.49, S.D.=0.68) and nonstudents (M=2.69, S.D.=0.86) in terms of their overall verification strategies [t (475.82)= 3.18, P<0.01]. The nonstudents indicated that they verified online information more than the students did, although both groups reported that they verify online information only rarely to occasionally. Table 6 reports the means of all nine verification strategies for the student subsample and shows that the most frequent verification strategies used by students are checking to see if the Web site information is current, complete, and comprehensive, considering whether the views represented by the author are facts or opinions, seeking out other sources to validate online information, and considering the author s goals or objectives in posting information to the Web. Less-often used strategies include checking to see who the author is, looking for a stamp of approval or recommendation for the site, making sure contact information is provided by the author, and verifying the Web site author s qualifications or credentials. 7 A within-subjects repeated-measures ANOVA using repeated contrasts as follow-up tests showed that nonstudents rated each type of information on the Internet to be significantly different in terms of credibility [F (3, 1287)=63.11, P<0.001, 2 =0.13]. Nonstudents rated reference information on the Internet to be most credible (M=4.37) followed by news (M=4.27), entertainment (M=4.07) and commercial information (M=3.75). Students perceived three of the four types of information on the Internet to be differentially credible [F (3, 882)=26.09, P<0.001, 2 =0.08], with reference and news information online rated equal in credibility (M=4.33 and M=4.25, respectively) but significantly higher than either entertainment (M=4.08) or commercial information (M=3.70).

16 286 M.J. Metzger et al. / Computers & Education 41 (2003) Discussion and practical implications Concerns about the nature of online information and users ability to evaluate it effectively prompted this study of students use of Web-based information, their perceptions of information credibility across media, and their verification efforts. This evaluation is particularly crucial given the increasing use of Web-based information sources among college students and a parallel increase in misinformation online (Null, 2000). Furthermore, because the utility of the Internet for students, in particular, rests upon an appropriately critical view of the veracity of Web-based information, it is important to assess the extent to which students differ from a more general adult Internet user population. Results of this research show that the students in this study rely heavily on Web-based information for a wide variety of purposes, despite the fact that they do not find it particularly credible in relation to traditional information sources and do not verify it very diligently. The students also reported that they use the Internet in their research just as much as academic journals and more than either newspapers or magazines. Although their minimal verification efforts may not pose much of a problem for certain types of information (e.g., entertainment or commercial information), it may be an issue for others, such as news and reference information, which students report as the most credible types of information across all media channels investigated here. Overall, this suggests that many students may depend on the Internet to provide accurate information without taking the requisite steps to ensure the veracity of the information they obtain. Furthermore, similar to Lubans s (1999) findings, our data suggest that although many students view Web-based information as a convenience, it may not necessarily improve the quality of their work. For instance, when considering the benefits of using the Internet for academic purposes, the students in this study reported that it was most helpful for increasing the number of sources available. Next in importance was the Internet s ability to save time. Behind this was its importance in improving their grades and, finally, its usefulness for improving the quality of their work. In essence, this suggests that students from our sample are rather opportunistic: they favor the Internet s capacity to increase quantity, but not necessarily quality. This pragmatic perspective is consistent with previous research, in which a majority of students reported that the Internet improved their college experience but said that some of their most common academic uses of it were to find out grades, clarify assignments, or notify a professor of absences (Pew Research Center, 2002). In our sample, students generally found all sources of information to be more credible than the nonstudents did, rating all channels except the Internet as significantly more credible than did the nonstudents. This was true across several types of information as well. The students rated news, reference, and entertainment information to be more credible than did the nonstudents. Overall, this indicates that the students found both conduit and content to be more trustworthy than did the nonstudents. Interestingly, however, the students and nonstudents did not vary significantly with respect to how credible they found the Web. One possible explanation for this finding is that because of its relative newness, everyone has less experience with the Internet than they do with more traditional sources of information, such as newspapers or television. Consequently, it may be difficult for all users to apply their existing knowledge of this information source in judging its credibility, because that knowledge is still being formed. With respect to the other information sources, however, our data suggest that students appear to be more trusting than nonstudents.

17 M.J. Metzger et al. / Computers & Education 41 (2003) Another possible explanation for the similarity in students and nonstudents judgments of the credibility of the Internet stems from differences in these two groups. Because they have more experience with the Internet, having used it for a greater proportion of their lives, students may feel that they can trust the information they find there. To some extent, this may result in a false sense of security, but it might also be a reflection of their extensive experience with the medium. After all, this study did not ask students about particular Web sites, only about sources of information in a more general sense. It may be that as experienced Internet users, students are adept at selecting appropriate, credible Internet sources, and this is where their trust comes from. At the same time, nonstudents may find the Internet credible due to the higher rate with which they report verifying the information they find there. Thus, the similarity among the two groups may in fact be due to differences between them. On the other hand, the fact that for all other media, students are more trusting than nonstudents could be because of their age and experience levels: because they are younger than the nonstudents in this study, they may have had less opportunity to see that sometimes the information they get is not accurate or trustworthy. The lower levels of credibility that nonstudents report with respect to other information sources may be a reflection of a realism, or perhaps even a cynicism, that comes with age. On the other hand, it may be because students are simply less critical of the information they acquire, particularly if they are acquiring it as a means to an end (e.g., creating a reference list for a school research paper). This latter idea is supported by this study s findings on students verification strategies. Student participants reported that they verify online information only rarely to occasionally, and it is quite possible that these results are somewhat inflated due to the social desirability inherent in this measure. In general, our student sample indicated that they are more apt to verify online information if it requires subjective judgment (e.g., considering whether the views represented are facts or opinions), or if there is a consistent, easily recognizable way to check the information (e.g., checking the date stamp at the bottom of many Web pages). Surprisingly, however, the students in our sample were not necessarily likely to do only the easy things to verify online information (e.g., see who the author is or look for a stamp of approval). Instead, they reported higher levels of some of the more laborious verification behaviors, such as checking to see if the information is complete and comprehensive, and seeking out other sources to validate online information. This is good news for educators students do seem willing to expend some effort to verify Web-based information, when they do so at all, although as mentioned earlier, there is the possibility that these findings are at least in part due to a social desirability response bias. When considering the Internet specifically, and the findings on both credibility and verification, the data suggest that many students take a relatively uninformed view of online information credibility. That is, their assessment of the credibility of the Internet across information types does not appear to be a function of the degree to which they verify that information. The students, for instance, were no less likely than the nonstudents in our sample to find the Internet credible, in spite of the fact that they verify Web-based information significantly less. The nonstudents appear to be less prone to this type of unchecked attribution: they found most media to be less credible than the students did, and they reported verifying Web-based information more. That said, if it is true that college students Internet use today likely reflects what future online behavior may be like when the current cohort of students graduates and moves into the adult

18 288 M.J. Metzger et al. / Computers & Education 41 (2003) world and the workforce (Pew Research Center, 2002, p. 19), then the implications of these results are ominous. Overall, our findings suggest that there is cause to be concerned about students use of the Web as an information resource. Students rely heavily on the Internet, and most intend to continue to do so in the future. When these facts are coupled with the reality that students may verify information only minimally, there appears to be a strong case for efforts to foster and expand Internet literacy curricula. Currently, most of the literature on using the Internet that is assigned within university courses focuses on computer-mediated communication (Berge, 1994; McComb, 1994); our data speak to a need for a greater focus on critical evaluation skills Future research There are several avenues for future research on these issues. For example, the use of a random sample to ensure data accuracy and generalizability is needed in subsequent studies on this topic. However, although the results of the research reported here might have been affected by the use of convenience samples, there is evidence that our data are reliable. Specifically, our findings are similar to other research on Internet use that has used large samples of subjects from multiple universities, and that have used national probability samples of college students (see Odell et al., 2000; Pew Research Center, 2002). Although we know many subjects in our study use the Web for academic reasons, these students were not asked about the extent to which they were encouraged to do so by instructors. It is possible that some of the attitudes and behaviors we uncovered are the result of students being required to use the Internet for assignments, for example. If it is indeed the case that students are being encouraged or required to use the Internet for specific academic information gathering, then as educators, we would be wise to also encourage or require verification activities for that information. An example would be asking students to identify or explain who the Web-based sources they cite are; this would be a beneficial exercise given our finding that students identify the author of a Web site as a verification strategy only rarely to occasionally. Another type of data that could be gathered in future research is the particular content that people are accessing, such as specific television or radio stations listened to, newspapers or magazines read, or Web sites visited. Knowing the specific information sources people are considering when they answer questions about credibility and verification would be useful. For example, if someone s exclusive use of the Web is to visit the New York Times Online, this person may justifiably rate the Web to be just as credible as newspapers. On the other hand, someone who visits a greater variety of sites may find the medium less credible overall than other channels. However, despite this limitation of the present study, we are able to conclude that there is a problem reflected in the disparity between students belief in the credibility of the Internet and their low frequency of verifying Web-based information. By assessing the frequency with which students use different verification strategies, this study provides direction for educators who hope to encourage students to assess the value of the information they find on the Web. For example, students can be taught easy verification techniques such as noting if a Web site has a.com or.org domain name (Lubans, 1999). Or, since our data show that students report that they rarely verify Web site authors credentials, instructors whose students use online resources could request that students provide this information

19 M.J. Metzger et al. / Computers & Education 41 (2003) when citing a Web site in a research paper. An exercise such as this demonstrates for students the importance of being critical information consumers. Finally, from the perspective of Web site design, our data point to the need for Web site authors to date-stamp their homepages, increase the visibility of reputable site awards or seals of approval, and keep links current as ways to assist students in establishing the veracity of online information. Of course, Internet literacy curricula would not solely benefit students, nor would it only apply to use of the Web as a source of academic information. Obviously, the more credible Web sites are, the less likely people are to fall prey to commercial fraud, or to erroneous medical or news information, for example. However, due to public outcry, it is sometimes easy for consumers to know when these instances of deliberate misinformation have occurred online and to identify their repercussions. By contrast, students using poor quality materials in a perfunctory effort to lengthen reference lists is a more subtle offense, and one that busy educators are not likely to catch. Thus, there is significant potential for benefit in the quality of students university education if Internet literacy curricula are emphasized in the ways that our data suggest would be most beneficial to students. References Albright, M., & Graf, D. L. (1992). Instructional technology and the faculty member. In M. Albright, & D. L. Graf (Eds.), Teaching in the information age: the role of educational technology (pp. 7 15). San Francisco: Jossey-Bass. Alexander, J. E., & Tate, M. A. (1999). Web wisdom: how to evaluate and create information quality on the Web. Hillsdale, NJ: Lawrence Erlbaum. Benson, T. W. (1994). Electronic network resources for communication scholars. Communication Education, 43, 120. Berge, Z. L. (1994). Electronic discussion groups. Communication Education, 43, Blake, K. (2000). Using the World Wide Web to teach writing online. Journalism and Mass Communication Educator, 55, Brandt, D. S. (1996). Evaluating information on the Internet. Computers in Libraries, 16(5), Browne, M. N., Freeman, K. E., & Williamson, C. L. (2000). The importance of critical thinking for student use of the Internet. College Student Journal, 34(3), Cortes, C. E. (1992). Media literacy: an educational basic for the information age. Education and Urban Society, 24, Davis, F. D. (1989). Perceived usefulness, perceived ease of use and user acceptance of information technology. MIS Quarterly, 13(3), Flanagin, A. J., & Metzger, M. J. (2000). Perceptions of Internet information credibility. Journalism and Mass Communication Quarterly, 77, Flanagin, A. J., & Metzger, M. J. (2001). Internet use in the contemporary media environment. Human Communication Research, 27, Gaziano, C., & McGrath, K. (1986). Measuring the concept of credibility. Journalism Quarterly, 63, Gilster, Paul. (1997). Digital literacy. New York: John Wiley & Sons. Gunaratne, S. A., & Lee, B. S. (1996). Integration of Internet resources into curriculum and instruction. Journalism & Mass Communication Educator, 51, Harris, C. (1996). An Internet education: a guide to doing research on the Internet. Belmont, CA: Wadsworth. Hester, J. B. (1999). Using a web-based interactive test as a learning tool. Journalism & Mass Communication Educator, 54, Johnson, T. J., & Kaye, B. K. (1998). Cruising is believing? Comparing Internet and traditional sources on media credibility measures. Journalism & Mass Communication Quarterly, 75, Jones, D. (1998). Exploring the Internet: using critical thinking skills. New York: Neal-Schuman Publishers.

20 290 M.J. Metzger et al. / Computers & Education 41 (2003) Lubans, J. (1998, April). How first-year university students use and regard Internet resources. Available from Duke University Libraries Web site: (accessed may 2000). Lubans, J. (1999, May). Students and the Internet. Available from Duke University Libraries Web site: (accessed may 2000). Mashek, J. W. (1997). Lethargy 96: how the media covered a listless campaign. Arlington, VA: The Freedom Forum. McComb, M. (1994). Benefits of computer-mediated communication in college courses. Communication Education, 43, Meyer, P. (1988). Defining and measuring credibility of newspapers: developing an index. Journalism Quarterly, 65, ,588. Mitchell, W. J. (1992). The reconfigured eye: visual truth in the post-photographic era. Cambridge, MA: The MIT Press. National Telecommunications and Information Administration (2000). Falling through the Net: toward digital inclusion. A report on Americans access to technology tools. Available from US Department of Commerce Web site: [accessed January 2003]. National Telecommunications and Information Administration (1999). Falling through the Net: Defining the digital divide. Retrieved May, 2000 from US Department of Commerce Web site: Newhagen, J., & Nass, C. (1989). Differential criteria for evaluating credibility of newspapers and TV news. Journalism Quarterly, 66, Null, C. (2000). Web of lies: hoaxes, jokes, fakery, fraud: your online credibility faces an uphill battle. PC/Computing, 6. Odell, P., Korgen, K., Schumacher, P., & DeLucchi, M. (2000). Internet use among male and female college students. Cyberpsychology and Behavior, 3(5), Perry, T. T., Perry, L. A., & Hosack-Curlin, K. (1998). Internet use by university students: an interdisciplinary study on three campuses. Internet Research: Electronic Networking Applications and Policy, 8, Pew Research Center (15 September, 2002). The Internet goes to college: how students are living in the future with today s technology. Available: (accessed January 2002). Pew Research Center for the People and the Press (14 January 1999). The Internet news audience goes ordinary. Available: (accessed July 1999). Rieh, S. Y., & Belkin, N. J. (1998). Understanding judgment of information quality and cognitive authority in the WWW. Proceedings of the ASIS Annual Meeting, 35, Rimmer, T., & Weaver, D. (1987). Different questions, different answers? Media use and credibility. Journalism Quarterly, 64, 28 36,44. Rogers, E. M. (1995). Diffusion of innovations (4th ed.). New York: Free Press. Ryan, S. M. (1994). Uncle Sam online: government information on the Internet. Communication Education, 43, Shackleford, J., Thompson, D. S., & James, M. B. (1999). Teaching strategy and assignment design: assessing the quality and validity of information via the Web. Social Science Computer Review, 17(2), Shaw, E. F. (1973). Media credibility: taking measure of a measure. Journalism Quarterly, 50, Shelton, M. W., Lane, D. R., & Waldhart, E. S. (1999). A review and assessment of national educational trends in communication instruction. Communication Education, 48, Smith, A, G. (1998, September). Evaluation of information sources. Available: agsmith/evaln/ evaln.htm (accessed July 1999). Sutherland, P. J., & Stewart, R. K. (1999). How accredited programs use the World Wide Web. Journalism & Mass Communication Educator, 54, Ward, M. (1997). Surfing for the suckers. New Scientist, 156, 29. West, M. D. (1994). Validating a scale for the measurement of credibility: a covariance structure modeling approach. Journalism Quarterly, 71, Witmer, D. (1998). Introduction to computer-mediated communication: a master syllabus for teaching communication technology. Communication Education, 47,