GRASSROOTS: How to get lecturers to go digital? Advanced Use of BlackBoard for Online Testing in Aircraft Structural Design Courses

Transcription

1 GRASSROOTS: How to get lecturers to go digital? Advanced Use of BlackBoard for Online Testing in Aircraft Structural Design Courses R. De Breuker 1,*, J.M.A.M. Hol 2, G.N. Saunders-Smits 3, M.E.D. van den Bogaard 4 1 Delft University of Technology, Faculty of Aerospace Engineering, NL-2629HS Delft, the Netherlands (r.debreuker@tudelft.nl) 2 Delft University of Technology, Faculty of Aerospace Engineering, NL-2629HS Delft, the Netherlands (j.m.a.m.hol@tudelft.nl) 3 Delft University of Technology, Faculty of Aerospace Engineering, NL-2629HS Delft, the Netherlands (g.n.saunders@tudelft.nl) 4 Delft University of Technology, Faculty of Technology, Policy and Management, NL-2628DX Delft, the Netherlands (m.e.d.vandenbogaard@tudelft.nl) * Corresponding Author Abstract Although digital learning applications have been available to lecturers for several years, lecturers are still hesitant to intensively use them due to restrictions in time and budget. To help lecturers overcome this hurdle TU Delft offers its lecturers small grants to implement digital learning in their courses through the GRASSROOTS project. One such project is based at the faculty of Aerospace Engineering and aims to use the test module in the Blackboard electronic learning environment in the course Aircraft Stress Analysis and Structural Design. This is a 2nd year BSc course. The test module is used for three reasons. First of all, the lectures cover an extensive amount of material and students need to be encouraged to keep up with the course contents. Secondly, at the end of the course the students need to be able to deal with large design problems. Necessary skills cannot properly be assessed through a traditional handwritten exam only. Third, the course traditionally enrols a large number of students (350+), therefore, automatic evaluation of the students answers is desirable. To encourage the students to practice their design skills, we put regular Quizzes online for the students, to test them on material dealt with in the recent lectures. This stimulates students to look at the course notes, which contributes to a better understanding of future lectures. The second challenge is dealt with by providing students with an extensive design exercise to be solved using numeric tools. After one - two weeks, students are examined on the design problem via BlackBoard. This is offered three times during the course, and is not compulsory. This paper describes the experiences and lessons learned whilst making advanced use of the BlackBoard online testing tools. The paper gives recommendations on how to prepare and deal with such tests for future users. Included in the lessons learned are reports on the opinion and responses of the students. Keywords: Grassroots, BlackBoard, Large group assessment

2 1 INTRODUCTION The course Aircraft Structural Analysis and Design, course code AE2-521N, is taught in the second year of the Bachelor s program of the faculty of Aerospace Engineering, Delft University of Technology, the Netherlands. The course builds on the first year s mechanics and materials courses and extends the students knowledge on structural analysis. The students need this knowledge to study the concept of structural design. The structural elements used in the course are truss members, beams elements and plates, which can be both statically determinate and indeterminate. First the students are introduced to the stress and displacement analysis of the structure. At a later moment these tools are used to design the structural element to satisfy certain stress and displacement requirements. In case of indeterminate structures, the concept of design iterations is taught as well. Apart from the three main structural elements, side issues are dealt with as well, such as composite cross-sections, stress concentrations and design for multiple versus combined loads. The course content of AE2-521N is diverse and extensive. The concept of design can be taught in a classroom, but the real understanding of the subject can only be obtained through exercise and exposure to challenging design problems. Therefore the obvious assessment method of a written exam is not sufficient to achieve the abovementioned learning objective. A more project-oriented method would be more appropriate. Individual long term projects during the lecture period provide the students with the opportunity to practice their skills frequently. However, given the typically large number of students attending this course ( per year), such an approach seems to be unfeasible. This paper describes a potential solution to the problem outlined above. The authors propose advanced use of automatic assessment and grading tools in BlackBoard. First we look at the historical back-ground of the course AE2-521N and describe how we intend to use certain innovations in delivering the course and assessment. We also describe the implementation of the project in BlackBoard. The paper is concluded with a summary of the project-oriented approach in BlackBoard and gives suggestions for lecturers who are interested in implementing similar technology in their classes. 2 PROBLEM STATEMENT The course of Aircraft Structural Analysis and Design in its current form was designed 4 years ago in 2004 [1]. The course was designed because of a gap in the Bachelor s program when it came to the design of aerospace structures. The course consists of two main parts, an analysis and a design part. The analysis part introduces new concepts for mainly displacement analysis, which students traditionally experience as challenging. The design part poses even more challenges, since this is the first time the students are exposed to this concept in the Bachelor s curriculum. First the general concept of design is explained, after which it is applied to stress design using the so-called fully stressed design criterion in particular. Displacement analysis is introduced as well, together with a combined design for stresses and displacements. The assessment method used for this course is a 3 hour written exam. Since the group of students easily exceeds the number of 300 each year, creating such a course suitable at the level of second year s students is very labour intensive. Therefore, the course was initially staffed with one professor, two assistant professors and a researcher. From an efficiency perspective, the number of active staff members needed to be reduced. For the students to become a natural at using the analysis tools, they need a lot of practice, and they also need to be exposed to real-life design problems. The problems are summarised below: Large number of students in the course. The students need to be exposed continuously to sets of analysis problems. The students need to be exposed to real-life design projects. The number of staff members needs to be reduced to an acceptable level.

3 3 COURSE AND ASSESSMENT REDESIGN In 2007, an initiative was started at the Delft University of Technology to stimulate lecturers to introduce information and communication technology (ICT) applications in the courses. This initiative is called the Grassroots project. The lecturers are encouraged to implement new ideas into or modify existing items of the course content aided by the use of ICT. To compensate for the time invested by the lecturer, a financial compensation is provided. The redesign of this course, making intensive use of Blackboard, has been selected as one of the Grassroots projects. Above we stated that the course consists of an analysis and a design part. One must keep in mind that both activities are located in different levels of Bloom s taxonomy [2]. The analysis part in on the knowledge level, for which a written assessment method is a suitable solution. The design part, however, is situated on the integration level, for which a project assessment approach might be more suitable, in combination with a written assessment as part of the 3 hour written exam. This form of assessment fits in well with the education goals and lecture format of the course. To achieve constructive alignment [3] we added the following additional tasks to the course: The students are exposed to short questions on the topics of the analysis part of the course in Blackboard, through online Quizzes. Students have to take about 6 Quizzes during the entire course. The Quizzes are not compulsory for the students; however, they can gain bonus points to add to the final grade of the course. These bonus points are only awarded if the written exam results in a grade of 5/10 or higher. In order to allow the students to gain experience with design problems, they are given three larger real-life design problems during the course, referred to as Mini Projects. The problem statement is formulated in words, including the accompanying drawings. The students get a few weeks to write a computer program which solves the problem at hand. After the preparation phase, the students have to answer an online test which they must solve using their custom made computer program. These MiniProjects count for 30% of the final course grade, independent of the result of the written exam. The large number of tests and students forced us to consider innovative ways to deliver this course. Written tests would lead to a large, disorderly pile of work to be graded by the teaching staff involved. BlackBoard has been used at Delft University of Technology for many years now and offers excellent possibilities to organise such electronic assessments. This teaching aid reduces the number of people required for managing the course to two: a lecturer and a course manager (for instance a PhD student or a teaching assistant). In the following section, the way the Quizzes and MiniProjects are handled in BlackBoard is described in more detail. 4 ADVANCED ELECTRONIC ASSESSMENT IN BLACKBOARD All students have personalised access to the Blackboard system, therefore they can complete assessments individually via the internet at any preferred time. Obviously, such a setup conceals a lot of pitfalls, in the sense that it is fraud sensitive. There is no means of checking that the individual sitting behind the computer is the individual who is supposed to take the test or the quiz. In this section the potential pitfalls with regards to academic frauds are described, followed by the measures taken to counteract those.

4 4.1 BlackBoard test pitfalls and solutions Students have to take the MiniProjects and Quizzes individually at home or on campus. It is virtually impossible to check whether the students are doing the tests themselves, or whether they are using the results of a fellow student who already completed the test. BlackBoard offers an excellent means to include randomness in the test setup. The question of the Quiz can be posed parametrically, and the answer can be implemented parametrically in closed-form as well. For each of the parameters, a range can be appended within which BlackBoard can randomly generate numbers, provided to the students when posing the question. BlackBoard immediately calculates the correct answer based on the random parameters and checks with the answer of the students. The same approach is used for the MiniProjects. An additional control mechanism is implemented in the following fashion. As mentioned earlier, the MiniProjects are long term projects for which the students get a generic problem statement of a real-life design problem, with a certain set of design parameters. They have to generate a computer program with which they are able to solve the design problem for a randomly generated set of input parameters. The latter is already explained in the previous paragraph. The students are all provided with the same problem statement, but when making the test, they have to upload their computer program as well, which can be checked by a random check. 4.2 BlackBoard implementation The tests are composed in the Assignment section of BlackBoard. For the Quizzes, each test consists of 3-4 analysis questions. Each question is created in the Pool manager, such that for each type of question, several specimens can be made. Together with the randomness in parameters, described in the previous section, the chance that two students get an identical test is almost zero. An example of a test is given in figure 1. This format applies to both Quizzes and MiniProjects. Figure 1: Example of a BlackBoard MiniProject For the MiniProject, the same applies as for the Quizzes in terms of generation of questions. On top of that, the students need to upload their computer program to BlackBoard. They should upload the program prior to starting the MiniProject test itself. In order to make sure that they do so, an interesting option is available in BlackBoard, called Adaptive release. This option ensures that the MiniProject test only becomes available when BlackBoard has received the program. The MiniProject test consists of 4 questions, of which 3 are analysis questions, and 1 is a design question.

5 For the Quizzes, the students get half an hour to complete the test, while in case of the MiniProjects, this term is extended to one full hour. They have to complete the test in one time to prohibit them from taking a look at the questions first, and then starting to prepare the answers. The computer program they developed should be such that they can immediately solve the problem with the given parameters. The grades of the Quizzes and MiniProjects are entered automatically into the grade book of BlackBoard. The only drawback is that, to the best of the author s knowledge, there is no option for a conditional awarding of grades of the Quizzes, which are only awarded of the written exam grade is a 5/10 or higher. 4.3 Pedagogical-didactical approach E-learning and the development of the Internet has provided a new entrance point for students to knowledge [4]. Whereas before the dawn of the knowledge society students used books and lecture notes to acquire knowledge, knowledge is now available at the click of a button and student expect to be taught the same way. Knowledge is also no longer perceived by students as being someone s individual accomplishment. Knowledge must be shared. This poses a whole new challenge for the educators of this new type of students. They must be addressed in a way that meets the new demands students put on their learning. Using a system where students are forced to actively engage with study material during the course rather than during the last week before the exam has traditionally always been a very successful way. In this course however the teaching staff has gone one step further by using the medium today s students feel most comfortable with which can be accessed at the time the students wants it. The knowledge is available anytime any place suiting this new attitude. Although today s students feel that sharing information is legal and essential they also perceive the sharing of assignments as legal. Students do not perceive this as fraud. Knowledge to them is a right not a privilege. By making each assignment different from the other, the educators can still assess an individual student s ability, whilst at the same time encouraging students to discuss the generic problem amongst them. 4.4 Transfer of experiences to other courses The experience gained by implementing the abovementioned approach should be transferred to other courses taught at the Delft University of Technology. Many courses, mainly in the Bachelor s phase of the curriculum, deal with limited resources in terms of available lecturers and course managers, and have a large number of students enrolled. Since the online testing implementation in BlackBoard is part of the Grassroots initiative at the Delft University of Technology, the experiences can be shared through this channel. All lecturers using a variant of electronic assessment in their course are joined in a discussion group which gathers on a regular basis. During such discussions, lecturers can learn from each other s experiences, and use those tips and tricks for their own project. Furthermore, all lecturers, when their project is fully implemented, need to write down their experiences on a special website [5] which is publicly accessible for all lecturers of the Delft University of Technology. The goal of this website is that, in case other lecturers would like to make use of electronic assessment, they can look for hints how to implement their ideas. They can also contact the responsible lecturer for a personal discussion on the topic. 5 RESULTS In total, 291 students participated in the written exam of the course AE2-521N in January % of the participants scored a 5/10 or higher on the written part of the exam, while 38% got a final mark of 6/10 or higher, including the MiniProject results. The reason for listing the students with a 6/10 or higher for the final mark is that this is the threshold at the Delft University of Technology to get a pass. The written exam

6 average was 3.9/10. Next, the correlation between the written exam results and the MiniProject and Quiz results are investigated. 97% of the students who participated in the written exam completed their Quizzes, out of which 79% got a final Quiz result of 6/10 or higher, which the lecturers marks as a sufficient result. The success rate of those students in the written exam getting a 5/10 or higher is 47%, which is significantly higher than the average of 29% for the total group of students. The exam average of the group with a sufficient result for the Quizzes is 4.2/10. The correlation between the written exam results and the Quiz results is 0.33, which is labelled as medium in the literature. 95% of the students, completing the written exam, did their MiniProjects. 67% of them got a sufficient mark. The success rate of those students in the written exam is 54%, which is higher than the Quiz results, and significantly higher than the average exam results. The average written exam grade of the students which have a sufficient MiniProject grade is 4.3/10. The correlation between the MiniProject results and the written exam results is 0.31, which is, again, medium. 6 CONCLUSIONS This paper investigated how students can be exposed to an environment allowing them to get actively involved with the course material presented in the Aircraft Stress Analysis and Structural Design course. Because of the large number of students in the course as well as the changing way student want to learn, an electronic environment is used, called BlackBoard. The students get the necessary exercise on the analysis part of the course by so-called Quizzes, which are short analysis questions posed via BlackBoard. The design part of the course is covered by introducing real-life design to the students, called MiniProjects. With the Quizzes, bonus points can be earned, while the MiniProjects account for additional grades to the written exam. Using BlackBoard as an assessment system for large groups has proven to be very valuable. It provides the opportunity for the students to get the required exercise to process the course material, and get acquainted with the principle of design. It also decreases staff workload. Furthermore it has been shown that there is a correlation between the results of the students who completed their Quizzes and MiniProjects satisfactory and their written exam results. Students, who did their Quizzes satisfactory, increased their success rate on the written exam with 38% with respect to the overall success rate of the written exam. If they completed their MiniProjects satisfactory, they increased their chance of success in the written exam by even 46%. Acknowledgement The authors would like to thank the Executive Board of the University and the Grassroots project management for providing funding for this project as well as all teaching staff involved in this course, in particular Dr. M.M. Abdalla and Prof. Dr. Z. Gürdal. References [1] Gillian N. Saunders -Smits, Zafer Gürdal, Jan Hol, Teaching Innovation for Aircraft Structural Analysis and Design Mathematica in an Engineering Education Environment, Proceedings of the 2005 American Society for Engineering Education Annual Conference & Exposition, Portland, Oregon, June 2005 [2] Bloom, Benjamin S. Taxonomy of Educational Objectives. Allyn and Bacon, Boston, MA. 1984, Pearson Education [3] Biggs, J. (1999). Teaching for quality learning at university. SRHE and Open University Press, Buckingham.

7 [4] Bourgeouis, E., Higher Education and Research for the ERA: current challenges and trends for the future - Final report of the STRATA-ETAN expert group, Office for official publications of the European Communities, Luxembourg, [5] May 2008