1 WIKIed Biology: Using Web 2.0 tools to teach science inquiry, content, and critical thinking Paper presented at the annual conference of the Association for Science Teacher Education (ASTE), Charleston, SC, January 11, Jennifer K. Frisch, Associate Professor of Biology Education, Kennesaw State University Paula Jackson, Professor of Biology, Kennesaw State University Meg C. Murray, Professor of Information Systems, Kennesaw State University Introduction When I began graduate school, we still had to do research in the library. We had to go to periodical indexes and search for papers that were relevant to our research, and search the references of papers that were relevant for more papers that were relevant. If the library didn t have the journal you needed it might take months to get a copy from another library. Now, we get disgruntled if we have to wait a day or two before getting a pdf file of an interesting paper delivered directly to our inboxes. Research has changed so much in fifteen years. But the way we teach science in the university seems exactly the same. (-Author reflection notes) The American Association for the Advancement of Science (AAAS) has long championed teaching science through inquiry in K-12 education (Benchmarks for Science Literacy, 1993), and National Science Education Standards (1996) have mirrored the benchmarks with regard to making science learning more student-centered. More recently, AAAS in collaboration with the National Science Foundation (NSF) issued Vision and Change in Undergraduate Biology Education: A Call to Action (2011) which seeks to strongly encourage university biology instructors to make learning science at post-secondary levels more student-centered. To this end, AAAS makes the case that the methods university instructors overwhelmingly use to teach undergraduate biology, including focusing on breadth rather than depth and using teacher-centered instruction (e.g., lecture) as the primary means of disseminating information, will need to drastically change. A collaboration between the authors of this paper arose as a result of conversations about the way we teach, the limitations of our methods, and how we can do better. Paula, a professor in the biology and physics department, watched Mike Wesch s A Vision of Students Today YouTube video (http://www.youtube.com/watch?v=dgcj46vyr9o) and struck up a conversation with Meg, who has expertise in Web 2.0 technologies and interest in improving its use in education. Jennifer, a biology educator in the biology and physics department, expressed interest in the project and was brought in to help with course design and assessment. The collaboration
2 led to a three-year NSF Transforming Undergraduate Education in Science, Technology, and Engineering (TUES) grant (# ) to support a course we called WIKIed Biology for three fall semesters, in The course was designed to give students a chance to develop, research, and communicate results of their own scientific inquiries while also developing skills in using Web 2.0 technologies and working collaboratively in groups. When the project began, the term Web 2.0 was fairly unfamiliar to those of us in the sciences. Web 2.0 is a term used to describe a suite of technologies that allow for evolving use of the Internet as a dynamic, participatory and collaborative medium for finding, organizing, managing, and sharing sources of information. The essence of Web 2.0 tools is collaboration: participants create knowledge together by synthesizing information from many different forms of digital media. Giving students an opportunity to design, collaborate, and write using Web 2.0 tools can engage and empower students and facilitate creation of lifelong learners (e.g., Anson and Miller-Cochran, 2009; Barnes and Tynan, 2007; Renner, 2006). The use of Web 2.0 is increasing in higher education (Ajjan and Hartshorne, 2008), but there are some institutional difficulties with using them as a main method of knowledge transmission. As Serat and Rubio (2012) point out, traditionally, the Academy was the main builder and keeper of the knowledge (p. 294) and so Web 2.0 tools could be perceived as threats to that role. Using Web 2.0 tools within educational settings is not new, but little empirical research has been published on how students use these tools to increase their understanding of content and skill in inquiry or critical thinking (Gray and Sheard, 2010; Greenhow, Robelia, and Hughes, 2009; Serat and Rubio, 2012). One of the important components of the nature of science is the skill and practice of collaboration, and so we felt that getting biology majors to use collaboration as a learning tool to increase their understanding of science content and ability to write about research should be a natural fit (see also Purdy, 2010). In order to skillfully use these technologies in our teaching, we need to help our students make new connections and form relationships between disparate pieces of information, and ultimately to create something new that can be shared with others (Maloney, 2007). However, in order for this type of learning to be effective and result in qualitative gains in content understanding, it will be important for students to understand that they must contribute a new mood to their learning process (Serat &
3 Rubio, 2012, p. 306) and begin developing and practicing skills of self-regulation and motivation for their own learning, rather than accepting information passively. This paper will describe how the course design has changed to meet the needs of our students over the three years of the WIKIed Biology project. We will also examine the extent to which the evidence we have collected supports the learning outcomes we designed for the course, including improving students skills in the following areas: discriminating between scientifically valid information and information that is un-sourced or invalid; using the Essential Features of Science Inquiry (NRC, 2000, p. 25) to pose scientific questions, critically evaluate sources and explanations, and communicate their findings; increasing proficiency with emerging Web 2.0 technologies; and demonstrating a deeper understanding of a broad biological concept. Finally, we will reflect on the successes and challenges of this course design approach and future directions including a version for pre-service and in-service 5-12 grade science teachers. Methods Setting and Course Structure Kennesaw State University (KSU) is the third-largest university in Georgia, located just northwest of Atlanta. The WIKIED Biology course was taught as an elective Special Topics course for biology majors, which meant that it had a 4000-level course number. Each year, the course met for two one-hour-and-fifteen-minute sessions a week for 16 weeks. For each semester the course was offered, all three co-authors served as (unofficial) co-instructors of the course, though Jackson was the instructor of record. Year 1: the pilot The first year of the course took place in fall of 2010, with a total enrollment of 10 students, all biology majors. Because we intended to focus project content within the area of ecology, we used the Ecology course as a prerequisite for this course. Nine of the 10 students enrolled in the first year were seniors, and one was a junior. For the pilot year, we designed the course so that the technological tools (listed in Table 1) would be taught one at a time throughout the semester as students were developing their projects. The intent was to allow students to display their understanding of the tools in context. Students in the pilot year had to complete four projects, each with a different content emphasis and of increasing complexity and increasing student direction as the semester
4 progressed. A summary of course project assignments for each year is listed in Table 2. Table 1. Internet-based and Web 2.0 tools used in WIKIed Biology, by year(s) used Technology Tool Year(s) Purpose Used Virtual Training Suite Tutorial 1,2,3 Learn to critically evaluate web-based resources Del.icio.us CiteULike Mendeley PBWorks Google Sites Google Docs/Drive https://drive.google.com Google + https://plus.google.com 1,2 Social bookmarking: to help students learn about tagging as an organizational structure and share tags and bookmarks with group members 1,2 Bibliographic management site: used to collect, share, and tag appropriate online resources, especially peer-reviewed 3 Same as CiteULike, but with some different features (including more user friendly format and ability to organize papers on desktop as well as online) 1 Wiki-building site: provided as one option for students to communicate their research project findings 1,2,3 Website-building tool: provided as one option for students to use to communicate their research project findings 1,2,3 Allows students to share documents and allow each group member to make edits and additions in real time 3 Social networking site: works with Google Drive and Sites, allows students to communicate easily with a circle including group members and instructors
5 Table 2. Major assignments for WIKIed Biology by year Assignment Year(s) Used Purpose: Students who complete this activity will be able to Website Evaluation Rubric 1,2,3 Develop and apply a rubric that could be used to judge a website s credibility and validity Tagging Assignment 1,2,3 Design and apply a hierarchy of categories in the form of tags that could be applied by the group to websites, papers, and other resources Web 2.0 project: Definition of Ecology Web 2.0 project: Plants and their Environment Web 2.0 project: Environment and Scale (Global warming and coral reefs) Web 2.0 project: Capstone Project 1 Apply skills of website evaluation and social tagging using del.icio.us to convey understanding of a directed inquiry 1 Apply skills of resource evaluation and finding resources using CiteULike to convey understanding of an inquiry of their choice within a topic area 1 Apply skills in Web 2.0 technologies to convey understanding of different components of a broader inquiry based on a topic chosen by the class 1,2,3 Apply skills in Web 2.0 technology to develop an open-inquiry question and communicate findings Asking Questions Activity 2,3 Develop an appropriate scientific question for inquiry Website Evaluation Table 2,3 Show evidence of being able to critically evaluate websites and other sources, and qualitatively and quantitatively compare them Annotated Bibliography 3 Apply understanding of how to read peerreviewed journal articles and skill in using bibliographic management site(s) to write syntheses of five peer-reviewed papers related to their inquiry Peer Critiques (1) 2,3 Write constructive criticism evaluating other groups projects strengths and weaknesses, and communicate them professionally Group Evaluation (1,2) 3 Evaluate the contribution of each member in the group * Years in (parenthesis) denote that the assignment was collected and reviewed, but not used as a part of the grade for the course.
6 Year 2: refining the idea and deepening content The second iteration of the course took place in fall of 2011, with an enrollment of 21 students. Based on feedback from students in the first cohort, who felt that they would have liked to have had a similar course earlier in their career, we changed the prerequisite to be the second course in the introductory biology sequence. Final enrollment included one sophomore, five juniors, and 15 seniors. The median GPA for students in the course was between 2.51 and Based on student feedback, we frontloaded all of the technology tool practice and instruction into the first part of the course. One change in technological tool use from the first year was incorporating a more structured process of using Google Docs to share and edit documents among group members (Table 1). Some course assignment elements also changed in the second year (Table 2). The Asking Questions activity was added early in the course in order to help guide students in the process of formulating a solid scientific research question. Another major revision to course design was that over the course of the semester, students focused on one major content-based inquiry rather than several smaller ones. Additionally, we opened up the areas of inquiry to students, allowing them to pursue questions outside of the areas of the instructors expertise, and enlisted the help of other biology faculty members to help students with their project focus and content. Year 3: improvements and reflections The final round of the course took place in fall of 2012, with an enrollment of 22 students (4 juniors and 18 seniors). The median grade-point-average range self-reported by these students was between ; 12 students reported a GPA in this range of the 18 that responded to the question. More changes in the technological tools used for the course were made in year three (see Table 1). Based on feedback from the previous two semesters, we changed some of the technological tools used in the course, eliminating the social bookmarking tool (del.i.cious); using Mendeley for citation gathering/reference review instead of CiteULike; and using Turnitin.com to evaluate levels of writing and paraphrasing in student work. An addition to the course assignments for the final year (Table 2) included a workshop on reading and synthesizing peer-reviewed journal articles, and the annotated bibliography assignment to evidence understanding and assimilation of
7 those skills. We also introduced Google + as a way for groups to communicate with each other and the instructors (using circles ), as this social networking tool can be used to post messages and documents, and also provides support for video chatting. The hangout video chat feature could also be used with Google Drive so that students could communicate with each other while everyone was looking at a document that was being edited. Learning Outcomes: collection and analysis of data In order to evaluate the extent to which the students in our course were able to meet the learning outcomes described above, we used a variety of assessments including the Critical thinking Assessment Test (CAT) developed by Tennessee Tech University (http://www.tntech.edu/cat/home/), a pre- and post-course Student Assessment of Learning Goals (SALG) survey (http://salgsite.org), phenomenological content analysis of projects using open coding, and case studies. Table 3 lists the learning outcomes of the course, and ties each outcome to the assignment(s) designed to develop each outcome, and the assessment(s) used to determine the extent to which students were able to meet each outcome. Table 3. Evaluation plan for WIKIed Biology, including outcomes, assignments, and evaluation instruments Learning Outcome: Assignments Instrument(s) used to evaluate (year(s) used) Students will be able to Distinguish between scientifically valid information and information that is unsourced or invalid Use the Essential Features of Science Inquiry (NRC, 2000, p. 25) to pose scientific questions, evaluate sources and explanations, and communicate their findings Website validity tutorial Website validity rubric Website evaluation table Asking Questions Activity Annotated Bibliography Peer Critiques Web 2.0 Project(s) Critical thinking Assessment Test (CAT) developed by Tennessee Tech (2, 3*) Phenomenological content analysis of artifacts from final projects (1,2,3*) Instructor reflections (1,2,3*) Increase proficiency with Tagging Assignment Student Assessment of
8 emerging Web 2.0 technologies Annotated Bibliography Web 2.0 Project(s) Learning Gains (SALG) (2,3*) Instructor reflections (2,3*) Demonstrate a deeper understanding of a broad biological concept Web 2.0 Project(s) Case Studies (1,2,3*) * Data from year 3 has been collected but has not yet been fully analyzed; the course finished in December 2012 Outcome #1: Students will distinguish between scientifically valid information and information that is un-sourced or invalid One of the goals of the course was to help students navigate the enormous amount of information available on the Internet, and develop their own skills and tools to distinguish between scientifically valid information and information that is unsourced. When developing the course, we felt strongly that this component was foundational to the course because of the importance of this skill in today s information era. Additionally, our students have expressed the belief that they know a great deal about how to use technology, but in our courses we have informally seen evidence that this perception has not translated into their ability to discriminate valid from invalid sources of information disseminated through technological means (e.g., using Wikipedia as a cited reference in their papers). The CAT instrument was designed to test a broad range of skills that faculty across the country feel are important components of critical thinking and real world problem solving, (http://www.tntech.edu/cat/technical/) including evaluating and interpreting information, problem solving skills, creative thinking skills, and skills of effective communication. Many of the specific critical thinking skills (Table 4) assessed by the CAT (e.g., provide alternate explanations for a pattern of results that has many possible causes ) were not addressed in our course, and so we did not expect to find significant improvement in those skills, but we expected to see a gain in others that the course did emphasize (Frisch, Jackson, and Murray, in review). Although the CAT instrument was administered as a pre- and post-test in both years two and three of the project, only year two data are available thus far. Of the 21 students who completed the course in the second year, 19 took both the pre- and post-cat test.
9 Outcome #2: Students will use the Essential Features of Science Inquiry (NRC, 2000, p. 25) to pose scientific questions, evaluate sources and explanations, and communicate their findings In order to evaluate the extent to which students have shown evidence of further developing the skills of posing questions, evaluating sources, and communicating findings, final group projects were analyzed using HYPERresearch (v.3.0, ), a qualitative analysis software that can be used for open-coding of text, images, and video. A database of instructor notes, feedback on assignments, and student artifacts was maintained in order to strengthen dependability of data (Lincoln and Guba, 1985; Merriam, 2001). Student artifacts collected included screenshots of the projects that were taken using Grab (on Mac) and text-based assignments that were transferred into.rtf files for the first year. Artifacts from second-year final projects were collected primarily through screenshots using Snagit (v , 2012), a software that allows the user to take screenshots of full webpages, including scrolling down. The same process will be undertaken for year three final projects. The final project s text and images were entered into HYPERresearch and open-coded, resulting in several broad categories of project components related to this outcome: question, sources, and communication. Data were analyzed within and across cases using a phenomenological approach (Creswell 1998). Question meaning units that arose from data reduction included: topic focus, ill-defined question, and well-defined question. Source meaning units used across cases included: unsourced information or images, questionable sources, and scientifically valid sources. Communication meaning units included: good use of media, scientific examples, some depth of understanding, synthesis of multiple sources, unclear communication, and inadequate paraphrasing. An overall impression of the essence of the phenomenon encompassed by each project year was developed from these data, and that impression was used to guide the selection of one case study that could serve to exemplify this essence in order to drive the assessment of the fourth outcome (below), describing the extent to which students show evidence of deepening their content knowledge during each iteration of the project. Outcome #3: Students will increase proficiency with emerging Web 2.0 technologies Baseline (pre-course) and post-course Student Assessment of Learning Gains (SALG) surveys (http://salgsite.org) were completed by students enrolled in the final two years of the
10 course. Students could respond to this survey anonymously, although we could access a report of which students had responded once the survey window had closed. The baseline survey asked students to rate their present level of skills and understandings of concepts related to the course. The post-course survey asked students to self-report their perception of how much different aspects of the course had resulted in a gain in understanding or skills. Our surveys included questions about a broad array of understandings and skills, but for the purposes of this paper, we will focus on the technology-related items rated, including the extent to which the course resulted in a gain in students skill in: collaborating with others using Web 2.0; communicating scientifically using internet-based technology; using the internet to learn more about biology; using a social bookmarking site; and creating a tag and tagging hierarchy. We have also included data on students perception of how much the following tools helped them gain understanding during the course: del.icio.us, CiteULike, Google sites, Mendeley, and Google plus. Outcome #4: Students will demonstrate a deeper understanding of a broad biological concept In order to investigate the extent to which this outcome was met in sufficient depth, a case study design (Stake, 1995) was appropriate, where each capstone project could be considered a case. As described above (for Outcome #2), coding of data from final projects seeking to examine the extent to which students developed inquiry skills combined with instructor reflections and discussion allowed us to develop an overall impression of how each project year progressed, including the successes and challenges of each year. While examining the data and discussing it with each other, we chose one project that could serve to exemplify the overall essence of the phenomenon (in this case, each year of the project), and wrote a narrative case study describing how that project progressed over the course of the semester. We considered examining two project cases for each year one of the best and one of the worst to provide a rich source of comparison. However, after the authors discussed this idea, we realized that it would be more instructive to choose a case from each year that we felt showed evidence of improvement throughout the semester. The groups that did poorly tended to do poorly throughout the semester, and the groups that did exceptionally tended to produce excellent work throughout the semester as well. What we were interested in examining was those groups whose work seemed to change somehow over the semester, perhaps as a result of this non-traditional form of instruction.
11 For the purposes of this paper, each narrative case study will describe how the group s capstone project exemplified students progress based predominantly on student website artifacts, instructor comments, and instructor discussion. Since this paper and presentation are attempting to broadly describe the extent to which the students in our course met all of the learning outcomes, we felt it would be appropriate to keep the cases relatively brief. However, future work based on these data will seek to develop the rich, descriptive nature of each case in more detail using many more artifacts collected, including focus group interviews, peer critiques, and completed assignments. Pseudonyms for student participants (designated by * in text of case study) were chosen by a random name generator. For the first year, the essence could be described as learning technology with topic focus, and the project chosen to represent the essence was a project about salamanders. The second year s essence was defined as improved questions and group challenges and the project chosen to exemplify that year was related to breast cancer genes. For the final year, preliminary analysis has led us to describe the essence as improving communication, and we have chosen a project discussing genetic factors of Type II diabetes to illustrate the essence of that year. Results Critical thinking Assessment Test (CAT). CAT scores from the second year of the course did not show significant gains overall. However, the results of the pre- and post- CAT test (Table 4) showed evidence that after taking the course, our students showed increased ability to separate relevant from irrelevant information when solving a real-world problem (t(18) = 2.282, p = 0.035, effect size +0.53). Kevin Harris, the Associate Director of the CAT, indicated that our project was the first of over 150 institutions reports analyzed to show a significant increase on this particular skill.
12 Table 4. CAT (critical thinking) instrument pre- and post-test data for year 2 pretest mean; SD posttest mean, SD Paired t-test p-value Skill assessed by CAT question Summarize the pattern of results in a graph without making inappropriate inferences 0.63; ; Evaluate how strongly correlationaltype data supports a hypothesis 1.16, ; Provide alternative explanations for a pattern of results that has many possible causes 1.32; ; Identify additional information needed to evaluate a hypothesis. 0.95; ; Evaluate whether spurious information strongly supports a hypothesis. 0.68; ; Provide alternative explanations for spurious associations. 1.26; ; Identify additional information needed to evaluate a hypothesis. 0.37; ; Determine whether an invited inference is supported by specific information. 0.84; ; Provide relevant alternative interpretations for a specific set of results. 0.84; ; Separate relevant from irrelevant information when solving a real-world problem.* 2.84; ; Use and apply relevant information to evaluate a problem. 0.89; ; Use basic mathematical skills to help solve a real-world problem. 0.89; ; Identify suitable solutions for a realworld problem using relevant information. 1.05; ; Identify and explain the best solution for a real-world problem using relevant information. 2.47; ; Explain how changes in a real-world problem situation might affect the solution 0.63; ; CAT total * significantly different pre- post result
13 Content analysis of Features of Inquiry. A summary of inquiry-related codes uncovered in projects from all three years of the WIKIed Biology course can be found in Table 5. Because the final year of the course has only recently concluded, most of the data from that year have not yet been analyzed. Participants in the course show increased evidence of developing and attempting to answer well-defined scientific questions over the course of the project. In the first year, our course design was to increase student autonomy in choosing their own project questions over the course of the semester; however, most of the groups never quite achieved the goal of developing a web-based project based on a question. Instead, the focus of most projects in year one seemed to be based more on a broad topic; essentially, students used the web to write a review or topical paper rather than develop their own scientific question and synthesize web-based media and peer-reviewed literature into a new product. The instructors discussed this aspect of the course in depth before the second iteration, and an asking questions assignment (Table 2) was developed in order to scaffold the process of developing a well-defined question. Additionally, the course now allowed students to work on only one inquiry-related project over the course of the semester, rather than several projects of increasing complexity. Data from the second year seem to indicate that most (5/6) groups were able to develop and attempt to answer a well-defined question; the remaining group had a reasonable question but their final project included a lot of information that did not relate directly to the question. The third year had a similar approach to questioning as the second, and again, most (6/7) projects included a well-defined and reasonable well-answered question. One group had a question that underwent several variations over the course of the semester, and ultimately was not effectively answered by the information included in the project. In the first year of the course, student projects seem to show evidence of an increase in students ability to evaluate and use appropriate sources as the semester progressed. Scientifically valid sources cited increased as the projects increased in complexity, and examples of questionable sources included in the projects decreased over time, though not dramatically. Student groups seemed to do a better job of making sure that sources were cited in their documents as the course progressed in the first year, though many groups (4/5) still neglected to source their images on the final project. In the second year, three of six groups had at least one incident of neglecting to cite source material in their final projects; two of six groups used questionable sources within their reference list. Number of citations of scientifically valid
14 sources in final projects was relatively similar between years one and two. Note that we required each group to include at least five peer-reviewed papers as a part of their reference list. Communication of ideas within projects was examined first by looking at how well final projects used and integrated pictures and media, not just by including the media but by referring to the media in a meaningful way in the text. In the first year of the project, the number of examples of good use of media increased as the semester progressed, and in the second year the combined number of examples of good use of media across projects (55) far exceeded that of the first year s capstone total (9), which is a large increase even considering that there were five capstone projects in year one and six in year two. Other aspects of communication that we analyzed within projects included evidence for depth of understanding. Depth of understanding was not noted at all in the first two projects for year 1, and only noted in one of the third projects. However, the capstone project for year one included instances of coding for demonstrates depth of understanding in all but one project. In the second year, four out of six final projects included some evidence of depth of understanding. Synthesis of data from multiple sources was another code we examined in the final projects in order to evaluate communication. Again, this code was not in evidence in the first two projects during the first year, though most groups had evidence of synthesis in projects three and four (2/3 and 4/5 respectively). For the second year of the course, four out of six final projects showed evidence of ability to synthesize information from multiple sources. In the third year of the project, we sought to emphasize the need for students to synthesize information in their final projects, but analysis of those data has not yet begun. Two other factors were identified as aspects that would detract from a group s ability to communicate their findings, including problems with grammar/spelling and inadequate paraphrasing of source material. Both of these negative communication codes are more in evidence as the projects grew in complexity in year one, and three out of six groups evidenced one of these communication codes in year two. Based on analysis of these data, for year three we emphasized more critical review of text in websites using the campus Writing Center, and also used Turnitin.com to help students assess their own ability to paraphrase.
15 Table 5. Content analysis summary of inquiry-related skills evidenced in WIKIed projects by year. In year 1, students worked on different projects (project 1, 2, etc.) and were given a topic from which to develop a question at each stage, except for the capstone project. In years 2 and 3 students developed their own question and worked towards their capstone project throughout the semester. Titles of project topics (year 1) or capstone topics (year 2) are included. QUESTION Year 1 project 1 (What is ecology?) Year 1 project 2: Plants and their Environments Year 1 project 3: Coral Reefs and Global warming Year 1 project 4: capstone Year 2: C. diff Year 2: behavior-altering parasites Year 2: Kudzu Year 2: Breast Cancer genes Year 2: HIV Year 2: coral reefs Year 3: preliminary data Focuses on a TOPIC rather than a scientific question Focuses on a question, but question is not welldefined or question is largely unanswered X 4/5 X X 1/5 X 1/7 Focuses on, and attempts to synthesize an answer to, a well-defined X X X X X 6/7 question SOURCES and SOURCE VALIDITY Incidents of unsourced information found in project 2/4 1/4 1/3 4/5 a 0 0 yes 0 yes yes * Examples of questionable sources exist in project # of scientifically valid sources cited COMMUNICATION # of examples of good use of media # of times project uses specific, scientific examples Project shows some depth of understanding 3/4 4/4 1/3 2/5 yes yes * * * * 0 0 1/3 4/5 yes yes no yes yes no *
16 Project includes instances (#) of good syntheses of information from multiple sources Project includes instances where communication is unclear (grammar, spelling, etc.) 0 0 2/3 3/5 yes (4) yes (6) no yes (2) yes (1) no * 2/4 3/4 1/3 2/5 no no yes no yes yes * Project shows instances of inadequate 0/4 0/4 0/3 2/5 no no yes no no no * paraphrasing Shaded rows denote target codes for each essential feature of inquiry *Year three data collection ended in December 2012, and analysis is ongoing. a All of these unsourced incidents were images used without attribution Student Assessment of Learning Gains (SALG) Analysis. Students in years 2 and 3 of the project took a baseline and a post-course SALG survey. Baseline SALG data were aggregated for both years of the project (N = 52), and is summarized for applicable technology-based skills in Figure 1. discriminate between scienti<ically valid information and information that is not scienti<ic Presently, I can... integrate technology and biology tag and create tagging hierarchy use a social bookmarking site use the internet to learn more about biology 0% 20% 40% 60% 80% 100% Percentage responding N/A not at all just a little somewhat a lot a great deal Figure 1. Baseline SALG responses to selected questions related to perceived abilities in technological skills before completing the course (combined year 2 and year 3; N = 52) Before the course, most students felt fairly confident in their ability to use the internet to learn more about biology (xˉ = 5.2 where 1=not applicable, 2 = not at all, 3 = just a little, 4 =
17 somewhat, 5 = a lot, and 6 = a great deal), and most students felt they lacked understanding and skill in tagging and developing tagging hierarchies (xˉ = 3.2). The mean responses for other technology-based skills assessed here included: use a social bookmarking site (xˉ = 4.0); integrate technology and biology (xˉ =4.3); and discriminate between scientifically valid information and information that is not valid (xˉ = 4.6). Post-course SALG surveys served to collect student self-report data on how much certain aspects of the course resulted in gains in their learning. Data was disaggregated by year because some of the technological tools used changed in those two years (Table 1), and the responses to some of the broader skill-based questions seemed different enough to warrant separation of the analysis. In year two (Figure 2), students (N =15) reported the greatest gain in their skill in using a social bookmarking site (xˉ = 4.5 where 1= no gains, 2 = a little gain, 3 = moderate gain, 4 = good gain, and 5 = great gain). Students seemed to find CiteULike the most effective course tool in helping them achieve gains in learning (xˉ = 4.5). In year three (Figure 3), however, students (N = 20) seemed to feel the greatest gain in their skill was in the area of using internet resources to learn biology (xˉ = 4.4) and discriminating between scientifically valid information and information that is not based in science (xˉ = 4.3). Students also responded positively to the use of Google + as a communication tool (xˉ = 3.8), and less positively to the use of Mendeley as a reference management tool (xˉ = 3.2).
18 how much Google sites helped you learn how much CiteULike helped you learn how much Delicious helped you learn create a tag and use an appropriate tagging hierarchy use a social bookmarking site use the internet to learn more about biology how to discriminate between scienti<ically valid information and invalid information how to communicate scienti<ically using internet based technology no gains a little gain moderate gain good gain great gain how to collaborate with others using web 2.0 0% 20% 40% 60% 80% 100% Percent responding Figure 2. Student perceptions of their learning gains (SALG) in technology-based skills after the course in year 2 (N = 15).
19 how much Google + helped you learn how much Google sites helped you learn how much Mendeley helped you learn create a tag and use an appropriate tagging hierarchy use a social bookmarking site use the internet to learn more about biology how to discriminate between scienti<ically valid information and invalid information how to communicate scienti<ically using internet based technology how to collaborate with others using web 2.0 no gains a little gain moderate gain good gain great gain N/A Figure 3. Student perceptions of their learning gains in technology-based skills after the course in year 3 (N = 20). Case studies 0% 20% 40% 60% 80% 100% Percentage responding Year one: Learning technology with topic focus Students were allowed to choose their own groups for the final (capstone) project, and could choose to investigate any question they liked within the realm of biology for the final project. Because the number of students remaining in the course was low (down to 11) at the end of the semester, we also allowed students to work on their final projects individually if they chose to do so. Clayton s* final project was one of three final projects that was completed individually. Based on group dynamics and evaluations from previous projects, however, Clayton did not appear to have chosen to work alone due to group problems (for project three he was rated between 8-10 on a scale of 10 as a collaborative group member), but rather because he wanted to pursue a topic in which other students did not share his interest. Clayton chose the
20 topic Larval Salamanders for his final project, and told instructors that he had been interested in salamanders since he was a child, so was excited to have an opportunity to pursue his interests. Clayton s Larval Salamander project, like most of the capstone projects completed in the first year of the project, focused primarily on a TOPIC rather than a scientific question based on that project. Examination of the introductory page of Clayton s website titled larval salamanders (Figure 4) reveals that Clayton seemed to be attempting to ask something along the lines of What role does Eurycea wilderae play in the aquatic ecosystem of the Blue Ridge region of the United States? However, this question is not stated directly, and as the figure shows, his communication of the goal(s) of his project is not completely clear. In fact, there are several grammatical and spelling errors throughout Clayton s final project, although generally these do not obstruct the meaning of his project. Figure 4. Screenshot of introductory page of Clayton s capstone project (year one) on larval salamanders (arrows added for emphasis).