A Collaborative Approach to a Course on the Semantic Web

Transcription

1 A Collaborative Approach to a Course on the Semantic Web Heidi J. C. Ellis Department of Engineering and Science, Rensselaer (RPI) Hartford Hartford, CT heidic@rh.edu Abstract The Semantic Web is an area of current research focus in many disciplines including software engineering. However, the breadth of knowledge required to understand development on the Semantic Web combined with the rapidly changing state of the field requires that teaching approaches to the Semantic Web be flexible. This paper describes a collaborative approach used to teach a course on the Semantic Web. Techniques used in the course included positioning the instructor as co-learner, joint decision-making by instructor and class, and the use of a group project. Survey results on student satisfaction with the teaching approach are presented, as well as student and instructor opinion on the approach. Index Terms Adult learning, Collaborative learning, Semantic Web, Software engineering education. INTRODUCTION experience and utilizes techniques from adult learning and collaborative learning theory including instructor participation in the learning process, elimination of the hierarchical distinction between instructor and learner, creation of a strong sense of community, self-directed learning, and incorporation of the most recent developments in the field. A group project is utilized to allow students to employ the techniques and technologies covered in class. This paper begins by describing the course including student characteristics, the content and goals of the course, and course deliverables. The challenges in teaching such rapidly changing material are outlined and the instructional approach used to support a student-oriented, flexible learning experience in the presence of change is presented. Results of a student survey on satisfaction with a collaborative learning approach are presented and student and instructor opinion of the approach are discussed. SOFTWARE ENGINEERING II The Semantic Web is receiving increasing interest from both academia and industry for its potential for automating tasks and improving collaboration between applications [4]. However, the Semantic Web is a work in progress and the approaches to the development of Semantic Web applications are in a state of flux. In addition, some of the standards and technologies upon which the Semantic Web is based are wellestablished (e.g., XML), while other standards and technologies are still under development by the W3C and various other groups (e.g., SPARQL). The Semantic Web is an attractive area of emerging technology to study within an academic course, yet it presents several challenges from a teaching perspective including the range of background required by students, the continual emergence of new standards, and the rapid rate of change in tools and approaches. Therefore, the nature of the Semantic Web requires a flexible and collaborative teaching approach. There are an increasing number of courses being taught on the Semantic Web [2]. Many of these efforts focus on one or two aspects of the Semantic Web such as data representation and integration, or logic used on the Semantic Web. These courses frequently grow out of existing artificial intelligence or data management areas and typically focus on a specific aspect of the Semantic Web. Indeed, the Semantic Web itself has been viewed as enabling elearning [1,10,12]. This paper describes a graduate course on the Semantic Web with the goal of providing a breadth of coverage of the topic. The course is taught as a collaborative, joint-learning ECSE-6780 Software Engineering II (SE-II) is an advanced graduate course in software engineering and is a topics course where the instructor may select course material from a variety of topics. In the spring 2005 semester, the instructor chose to spend the semester exploring the topic of the Semantic Web. SE-II is a second course in software engineering and requires Software Engineering I as a prerequisite. The course meets one evening a week for three hours during a 15 week semester. The course is supported by WebCT to provide bulletin boards and chat rooms for student interaction. The student audience in the spring 2005 offering of SE-II comprised 14 adult working professionals. Student ages ranged between 27 to 40 and students were employed full-time typically in software or IT positions. Therefore, this student body was mature and relatively technically savvy. The stated goal of the SE-II course is for students to gain a broad understanding of the goals and perspectives of the Semantic Web including: Knowledge representation and semantics including RDF and OWL, Approaches to the design and construction of ontologies, Reasoning and logic and their use on the Semantic Web, Interaction between artificial intelligence, software engineering, databases, and information systems and how they contribute to the development of the Semantic Web, Current tools and techniques for developing Semantic Web applications, Current applications on the Semantic Web, and S3C-1

2 Future directions of the Semantic Web. The text used in the course was Antoniou and van Harmelen s "A Semantic Web Primer" [3] which was heavily supplemented with current articles and standards specifications for the various technologies investigated in the course. These supplemental readings were posted on the course web site ( The syllabus followed the text for the first half of the semester and included topics: Overview of Semantic Web Data modeling using RDF and RDFS Ontology management using OWL Reasoning and description logics Semantic Web applications Query languages for the Semantic Web Information extraction and integration Agents on the Semantic Web Semantic Web Services Trust and security The course deliverables included two quizzes worth 15% of student grade each, a group project worth 40% and a paper worth 30%. The intent of the project was to provide students with the opportunity to gain experience with the approaches and technologies covered in class. The project was developed by groups of four or five students. The paper was intended as a lessons learned reflection on experiences gained during project development where students could both describe their learning experience as well as provide insight into future directions for the project and the course as a whole. CHALLENGES As experienced instructors know, teaching a new course involves a significant amount of preparation in order for the instructor to master the material and present it in a fashion from which students can easily learn. Teaching a course on the Semantic Web where the content is still in a state of flux gives rise to a unique set of challenges that must be met for effective learning to take place. One primary challenge to teaching SE-II was the fact that the content was continually evolving and advancing. Since the course material was developing at the same time that the course was being taught, the instructor had to do just in time learning and preparation. It is difficult for instructors to maintain fluency in course material under these constantly changing conditions. One effective solution used to face this challenge was to construct a collaborative learning environment in the classroom where instructor and students learned the material together. A second major challenge was the breadth of background required of the students in order for them to master the material. The Semantic Web encompasses a wide range of topics from areas including artificial intelligence, knowledge representation, distributed computing, visualization, and software engineering. The majority of students coming into the course lacked the required breadth of background. The use of a group project leveraged the existing student knowledge and fostered a collaborative learning environment where students learned from each other. A third challenge resulted from the lack of academic texts for this newly emerging area. The selected text, while useful, could not provide the depth required for students to comprehend all of the course material. Finding readings at the proper level of detail that fit the learning objectives was an ongoing project throughout the semester. In addition, the difficulty in locating appropriate material meant that finding answers to even minor questions was time consuming. The innovative nature of the course material resulted in a fourth challenge. The tools that supported the approaches and technologies covered were frequently immature or incomplete. In addition, most of the tools were academic efforts and so came with a range of documentation and little or no support. THE APPROACH The approach used in SE-II to address the challenges outlined above was a modification of an approach used in previous (c. 2000) versions of the course [6] and was very collaborative in nature. The approach relied heavily on adult learning theory [5,7,8,9] in that it supported self-directed, application-oriented learning and used Whipple s characteristics of collaborative learning as a base [13]. This section discusses the techniques used in offering SE-II and identifies any modifications made to the technique based on understanding gained from the prior collaborative learning experiences described in [6]. The first technique used to support a collaborative learning process was to set proper student expectations at the beginning of the semester. It was critical that the students knew what to expect from the course since this was an atypical learning experience. Understanding gained from the previous use of collaborative learning caused the instructor to refine this technique to make the expected learning experience more explicit to students. The course description posted on the web site informed potential students that the version of the SE-II course covering the Semantic Web was a new offering and that the material was still under development. In addition, during the first class the instructor informed students that a collaborative learning approach would be used. A portion of the first class was spent describing a flexible approach and students were told that progress would be jointly assessed regularly and the course would be redirected as necessary. This information was reiterated periodically throughout the semester, another modification to the technique. A second approach to supporting collaborative learning was that the instructor identified herself as co-learner at the outset. Throughout the semester the instructor asked as many questions as the students and when material was being explored, the instructor let students talk for the majority of the time. The instructor also identified the areas that she didn t understand and let students teach the instructor what they knew. This technique was shown to be highly successful in the prior learning experience and was improved only slightly in that the instructor prepared a list of questions that she did not know the answers to before most class meetings. S3C-2

3 The immediate instillation of a strong sense of community among the class members (including the instructor) was a third effective technique used to create a cooperative learning environment. Since the text only covered part of the course material and the instructor could not be the subject matter expert, students needed to rely on their peers for direction in their learning. This collaborative attitude was fostered by the instructor asking students to solve other student s problems and collaboratively solving problems in class and on WebCT. In addition, the instructor would ask students to share their knowledge of the material with the class. For instance, in one case a student gave the instructor and the class a short tutorial in the use of a tool. This technique was little modified from the previous collaborative learning experience and was so successful that the class became an open forum where students in the class would ask questions directly of other students rather than asking the instructor. In addition, some classes were entirely spent in joint problem solving where one group would identify a problem in their project development and the class would jointly solve the problem. The challenge of student background was ameliorated by putting students into groups to complete the project and to write the paper. Since students entered the course with differing backgrounds, by constructing groups it was hoped that if one student lacked background in a particular area, this shortcoming would be made up by another student and that students would learn from each other. The group project was not employed during previous offerings of the course and the project further enhanced the sense of community within the class by fostering student interaction both within groups and among groups. Another strategy employed to support collaboration was to encourage students to be self-directed in their learning process. The technique of self-directed learning was employed in a somewhat lesser manner in the previous collaborative learning experience reported in [6]. In the previous report, the instructor provided greater definition of topics and guided students by providing explicit methods for mastering the material (e.g., tutorials). In the recent SE-II offering, the selfdirection was expanded by allowing students to identify concrete topics to be explored. In addition, students were allowed to evaluate and make decisions about learning style and resources. While the learning content was somewhat driven by the needs of the project development, the learning style was left up to the student. Some students preferred to read articles and tutorials while others preferred to implement small examples as a learning exercise. As students went through the design and implementation process, they discovered the areas of knowledge that they lacked and then directed learning towards acquiring that information. The Project The group project was a major instrument used by the instructor to foster a collaborative learning environment. In addition, the project afforded students a hands-on learning experience where they could directly apply the knowledge gained in the classroom to a real-world project. The learning was structured around the project where class meetings consisted of coverage of new material during the first half of the class and discussion of how the material was supported by or impacted the project during the second half of the class. The domain of the project was the SWENET Network Community for Software Engineering Education ( The SWENET site is intended to provide a repository of tested educational materials that can be adopted, modified, and enhanced by software engineering educators across the nation. The overarching goal of the project used in SE-II was to add a layer of semantics to SWENET to guide users in the selection and retrieval of appropriate software engineering course material. The project was split into two major portions. The first portion was the design and development of an ontology for representing data about software engineering education teaching modules. Learning ontologies typically involve an overarching ontology that incorporates three ontologies where each lower level ontology represents a different aspect of the data [12]. The pedagogy ontology includes data that represents the different aspects of learning such as exercises, lectures, and homework. The content ontology contains the domain data based on the Software Engineering Education Knowledge categories [11]. The structure ontology defines how the various pieces of the pedagogy and content ontologies are related. The overarching ontology defines high-level data pertaining to the domain. The Protégé ontology and knowledge-base editor ( was used to develop the ontology in the OWL ontology language. The second portion of the project was to develop a semantic search tool which would allow users to identify software engineering modules based on more than the knowledge area to which a module belonged. For instance, the users may want to locate all material related to real-time systems, a functionality not currently supported by the SWENET site. This development was supported by the Jena Java framework ( THE COLLABORATIVE LEARNING EXPERIENCE This section discusses the impact of the collaborative learning experience used in the SE-II class. Results of a survey of student opinion on the learning experience are described and instructor experiences and observations are reported. Student Opinion Survey In order to determine the effect of the collaborative learning approach, students were surveyed as to their satisfaction during the tenth week (out of 15) of the semester. The survey was anonymous and students were told that responses could not be traced to individual students and that survey results would in no way impact student grades. The survey consisted of 16 statements and used a five-point Likert scale to determine student strength of agreement with the statements. Table 1 contains a summary of the questions asked and an indication of the scale used for the answers. A 17 th question asked students for free-form feedback on their opinions on the course and their learning experience. S3C-3

4 The first half of the survey posed questions in the form where students indicated their level of agreement with statements about various aspects of the collaborative learning environment. The goal of these first eight questions was to indirectly assess the impact of the collaborative approach. A more direct approach was used in the second half of the survey where students were asked directly about their satisfaction with the level of collaboration used in the course as well as the general level of satisfaction with the course. TABLE 1 STUDENT OPINION SURVEY Scale: Strongly Agree to Strongly Disagree 1. I have a high level of interest in the course subject matter: 2. My background knowledge was sufficient for this course: 3. My learning has been enhanced by learning from my peers: 4. The selection of supporting material for the course such as articles and links to web sites was sufficient for my learning: 5. The stability and maturity of the tools used in the course had a positive impact on my learning: 6. The group project has had a positive impact on my learning: 7. I like the level of self-determination used in project functionality definition: 8. I am very satisfied with my learning experience in this course: Scale: Very Low to Very High 9. My satisfaction with the level of collaboration in my learning process is: 10. My satisfaction with the flexible approach used in the course is: 11. My satisfaction with the level of interaction with my peers is: 12. My satisfaction with the level of interaction with the instructor is: 13. My satisfaction with the course text is: 14. My satisfaction with the supplemental course readings is: 15. My satisfaction with my learning so far in the course is: 16. My overall satisfaction with the course so far is: Survey Results Thirteen of the 14 students in the SE-II class completed the survey resulting in a 92.9% response rate. The survey responses were transformed into numeric form with a range of 1 to 5 where the very low and strongly disagree categories mapped to a value of 1, the neutral category mapped to a value of 3, and the very high and strongly agree categories mapped to a value of 5. Results for survey questions one through 16 are shown in Figure 1. As can be seen in Figure 1, student satisfaction with their learning experience across the range of aspects surveyed was high, with an average response of 3.89 for all questions. While the data set is too small to support statistical evaluation, the relatively high satisfaction demonstrated by students appears to indicate a corresponding high level of satisfaction with their collaborative learning approach. The notable exception to the overall satisfaction is in the response to question 5 which asks about the impact of tools on student learning experience. As might be expected, it appears that the relative immaturity of Semantic Web tools had a negative impact on student learning. FIGURE 1 AVERAGE STUDENT SURVEY RESPONSES In order to more accurately assess student opinion of the cooperative learning approach, the average response to questions directly related to collaboration were examined. Figure 2 shows the results of the survey questions that focus on the students collaborative experience. Average responses to all of these questions were on the positive side of neutral, indicating that students felt positively about their collaborative experience. It is interesting to note that student response to question nine which asks directly about student satisfaction with the level of collaboration is somewhat lower than responses to questions which ask indirectly about student collaboration. However, the high level of satisfaction shown by the results to question six about the impact of the group project and to question 11 about the satisfaction with interaction with peers indicates that students felt that their learning was positively influenced by interacting with their peers. Question Number Average Response Question Number Average Response FIGURE 2 AVERAGE STUDENT SATISFACTION WITH COLLABORATIVE APPROACH In order to determine overall student satisfaction with the SE-II course, the average response to questions asking about student satisfaction in general were examined. Figure 3 shows the average student response to questions related to the overall learning experience. S3C-4

5 Question Number Average Response FIGURE 3 OVERALL AVERAGE STUDENT SATISFACTION One very interesting observation that can be made from examining Figure 3 is that while the students appear to be satisfied with their overall learning experience, they also appear to be more satisfied with the course itself rather than the learning experience. In other words, student satisfaction with learning was lower than overall satisfaction with the course. This response may be due to the fact that students have a high interest in the subject matter (question 1 in Figure 1) thereby causing their opinion of the course to be higher than their opinion of their learning experience. Student Comments In addition to the responses from the 16 questions that used the Likert scale, student free form comments supplied in response to survey question 17 provide support for the observation that students had a positive learning experience. Question 17 asked students What other opinions and/or observations about the course and your learning experience would you like to provide? Approximately half (7) of the students provided some form of free form feedback. It is interesting to note that many of the responses provided for question 17 commented on the degree of self-direction and flexibility in the SE-II course and less directly on the collaborative aspects of learning. High-level comments that provide insight into the learning experience in general included I think this has been a great learning experience. Another student indicated satisfaction with the course coverage: This is a good class to have for advanced degree students, because it exposes us to topics that do not come up (for most people) in the course of everyday work. The most direct support for the approach used in teaching SE-II was found in comments related to the project. One student explained I think the group project gives students an excellent way to practice what they've learned through the course text... A second student stated I am a strong believer in project based classes. Bringing in a real-world example to work on is a great motivator. I love the interactive teaching style. I only wish I could take more classes like this one." Students appeared to like the freedom afforded by the approach, and the degree of self-direction in particular. One student mentioned Overall I like the course very much. The level of self-determination is greater then most classes. A second student commented It was really enjoyable and was open enough to allow us to be creative. GREAT! This feedback appears to indicate that students were comfortable with a flexible approach to the course where direction changed from week to week. In addition to the positive feedback, there were a few negative comments. The high level of self-direction in the course was not embraced by all students as one student commented that The lack of structure is often frustrating. I realize that the course is new and there is a learning curve for students and instructor alike, but it is often less than obvious what the learning objectives are in terms of the project, paper and quizzes. Several other students commented that they would prefer to see more software engineering aspects incorporated into the course. This last observation may be directly tied to the fact that students expected a different sort of course based on the course title. Lastly, it appears that, at the time that the survey was administered two-thirds of the way through the course, student opinion on the collaborative learning experience was still being formed. One student commented The verdict is still out on if this will impact my learning experience in a positive or negative way. If I had to give an answer now - I would say positive. Instructor Observations Several observations can be made from the instructor s perspective on the effectiveness of the application of the collaborative learning approach employed in the SE-II class. First, the level of student learning appears to have been high. Student mastery of the course material was reflected in the A grades achieved by all students on the project and paper. More concretely, each student successfully created semantic annotation for a SWENET module based on the ontology developed, demonstrating a complete understanding of the ontology that was constructed. In addition, in the paper each student correctly identified a next step that would be an additional piece of work that would build off the learning in the course, indicating a thorough understanding of the material covered during the semester. The accurate setting of student expectations appears to have been only partially successful. Many students indicated a satisfaction with the course, but several ad hoc comments made to the instructor as well as free form feedback from survey question 17 indicate that some students would have preferred more software engineering material to be included in the course. While not a direct comment on the collaborative learning experience, this feedback may indicate that the instructor was not completely clear in conveying course expectations. Conversely, this feedback may simply indicate a desire on the students part for different topic coverage. The strategy of positioning the instructor as co-learner appears to have been quite successful. During coverage of new material in the first half of the class meetings, the instructor clearly identified questions and areas of incomplete understanding. During this time, questions were frequently initiated by students. In fact, students would answer each other s questions and solve each other s problems during S3C-5

6 class, even while the instructor was speaking. Students felt very free to both initiate discussions and to jump into the middle of discussions. In addition, much joint problem-solving occurred on WebCT with little or no intervention on the part of the instructor. A strong sense of community was also built early in the course. Within the first three weeks of class, students had formed themselves into groups and identified themselves with their topic (one group per lower-level ontology was formed) as evidenced by the fact students quickly developed a gentle teasing among group members. The sense of community was also evident in that groups selected a food of choice (e.g., trail mix) and group meetings advertised the presence of the food. While the impact of such collaborative interaction is difficult to measure, it could be expected that the high level of camaraderie among group members would enhance the exchange of knowledge. Given the changeable nature of the course material, the instructor observed that students showed a remarkable tolerance for the flexible approach used in the class. During the middle of the course, changes were made in deliverables and in the schedule four weeks in a row. Perhaps due to their work experience, students were able to adapt to these changes with ease. The adaptation was aided by the fact that decisions related to changes were made collaboratively with the input of all students. While the collaborative learning approaches used in SE-II had a positive outcome for both instructor and students, experiences gained have resulted in several observations on the potential problems inherent in the process. First, the collaborative approach requires instructors to be able to think on their feet and make decisions on the fly. Since the instructor is learning along side the students, the instructor must be able to quickly assess and analyze the information in order to be able to make reasonable decisions about the direction that learning should take. Another potential drawback is that the nature of the course material and the collaborative approach meant that course preparation could not be done long in advance. The lead time was around a week since the discussions and decisions made in class directed the readings and discussion for the next class. Lastly, it should be noted that while the overall results of the collaborative approach used in SE-II were positive, the approach may not be applicable to some student populations and courses. The relatively small size of the class allowed a sense of community to be easily established. It could be expected that the sense of community and corresponding high degree of interaction among students would be more difficult to establish in a class with a higher enrollment (e.g., above 25 students). It could also be expected that the fluid sort of collaborative learning and interaction used in SE-II would be difficult to maintain in a larger class. Another main key to the collaborative learning experience was the novelty of the course material. It may be more difficult to construct a collaborative learning environment for courses where the material is well known (e.g., Data Structures). In addition, the experience and maturity of the student population in SE-II allowed for greater shared learning. These working professional students were familiar with working in environments where requirements and deliverables changed frequently. An undergraduate student population may have difficulty with the relatively unstructured learning approach used in SE-II. CONCLUSION This paper has reported on a collaborative approach used to teach a graduate-level software engineering course on the Semantic Web. Approaches to collaborative learning employed included positioning the instructor as co-learner, joint decision-making by instructor and class, and the use of a group project. Survey results on student opinion of their learning process were, in general, positive and these approaches appear to have been successful. In addition, ad hoc student feedback indicates student s enjoyed the learning experience. REFERENCES [1] Allert, H., C. Richter, and W. Nejdl, Learning Objects on the Semantic Web Explicitly Modeling Instructional Theories and Paradigms, Proceedings of E-Learn 2002: World Conference on E-Learning in Corporate, Government, Healthcare, Higher Education (formerly the WebNet Conference), Montreal, Canada, Oct [2] Antoniou, G., J. Maluszynski, W. Nejdl, H. Allert, W. Siberski, and A. Wilk, Existing university courses on the Semantic Web, REWERSE, Reasoning on the web, Aug. 31, 2004, downloaded from on March 5, [3] Antoniou, Grigoris and Frank van Harmelen, "A Semantic Web Primer," The MIT Press, 2004, ISBN [4] Berners-Lee, T., J. Hendler, and O. Lasilla, The Semantic Web, Scientific American, 284, No 5, 2001, pp [5] Brookfield, S. D., Understanding and Facilitating Adult Learning, San Francisco, CA: Jossey-Bass, [6] Ellis, H.J.C., An Experience in Collaborative Learning: Observations of a Software Engineering Course, The 2000 Frontiers in Education Conference, Kansas City, MO, Oct [7] Hiemstra, R., Self-Directed Learning, In T. Husen & T. N. Postethwaite (Eds), The International Encyclopedia of Education, 2 nd ed. Oxford: Pergamon Press, [8] Knowles, M., Andragogy in Action: Applying Modern Principles of Adult Learning, Jossey-Bass, San Francisco, [9] Knowles, M., et. al, The Adult Learner: The Definitive Classic in Adult Education and Human Resource Development, Gulf Publishing, Houston, TX, [10] Nilsson, M., M. Palmer, A. Naeve, and S. Stockholm, Semantic Web Meta-data for e-learning-some Architectural Guidelines, Proceedings of the 11th World Wide Web Conference, [11] SE2004. Software Engineering Curriculum Guidelines for Undergraduate Degree Programs in Software Engineering. IEEE/CS and ACM Joint Task Force on Computing Curricula [12] Stojanovic, L., S. Staab and R. Studer. elearning in the Semantic Web, Web-Net World Conference on the WWW and the Internet. Orlando, Florida, USA, [13] Whipple, W. R. "Collaborative Learning: Recognizing It When We See It." AAHE Bulletin, Oct pp S3C-6