Kuwait University College of Engineering and Petroleum Office of Academic Assessment College of Engineering and Petroleum Online Course Assessment For Academic year 2007-2008 July, 2008 (Updated December, 2008)
Table of Contents Introduction... 3 Preparation and Implementation of the Process... 3 Results and Discussion... 4 Departmental Results... 11 Appendix A: Instructor Class Evaluation Form... 15 Appendix B: Instructions for the Courses Assessment... 18 2
Introduction: This report presents the results of the online course assessment at the College of Engineering and Petroleum Instructor conducted during Fall, Spring and Summer Semesters for the academic year 2007-2008. The online assessment form was developed, conducted and analyzed by the Office of Academic Assessment (OAA). Preparation and Implementation of the Process: The assessment was conducted for the academic year 2007-2008. The form was provided to the faculty through an online system. Faculty members were able to leave the system at any time before submitting the form without losing any of the data they entered; they also were able to view the previously submitted forms. After submitting the form, an instruction page appears. It contains guidelines on how to prepare a course assessment file to be submitted to the departmental assessment coordinator. For the Fall semester, a total response of 87 was recorded out of 168 faculty members who were teaching courses during the semester (52% response rate). The responses covered 168 different courses. The table below shows the total number of faculty members who taught courses in Fall 2007-2008 and the number of faculty members participated in the survey according to departments. Table 1: Response statistics Fall Semester [DEPARTMENT] Total Responses Unique Responder Active Instructors Response Rate Chemical 19 9 22 36% Civil 26 15 37 41% Computer 18 10 22 45% Electrical 35 19 29 62% Industrial & Management Systems 6 3 7 43% Mechanical 44 20 33 52% Petroleum 19 10 18 50% Total 168 87 168 52% For the Spring semester, a total response of 69 was recorded out of 169 faculty members who were teaching courses during the semester (41% response rate). The responses covered 134 different courses. The table below shows the total number of faculty members who taught courses in Spring 2007-2008 and the number of faculty members participated in the survey according to departments. 3
Table 2: Response statistics Spring Semester [DEPARTMENT] Total Responses Unique Responder Active Instructors Response Rate Chemical 24 12 22 32% Civil 15 8 39 21% Computer 18 9 21 43% Electrical 22 13 29 38% Industrial & Management Systems 2 1 7 14% Mechanical 40 18 33 30% Petroleum 13 8 18 28% Total 134 69 169 41% For the Summer semester, a total response of 32 was recorded. The responses covered 48 different courses. The table below shows the total number of faculty members who taught courses in Summer 2007-2008 and the number of faculty members participated in the survey according to departments. Table 3: Response statistics Summer Semester [DEPARTMENT] Total Responses Unique Responder Chemical 8 4 Civil 0 0 Computer 7 4 Electrical 11 8 Industrial & Management Systems 1 1 Mechanical 13 8 Petroleum 8 7 Total 48 32 4
Results and Discussion The main results of the online course assessment for the whole college are presented in Tables 4-5, and Figures 1-2. Table 4 and Figure 1 gives the average ratings and weighted averages for all courses in the college. In Table 5, student performance in departmental courses as well as core engineering courses is given in terms of weighted average scores. The relevance ratings given for each course are used as weights. Since the response rate is somewhat low, and therefore, some courses are not represented in a balanced way, the results may not be valid for some programs. In the survey instrument, the first eleven outcomes are those corresponding to ABET Criterion 3 (a-k) outcomes, and the rest of the outcomes are program specific. Therefore, the analysis and departmental comparison has been performed on the first eleven outcomes. More detailed results are presented in the departmental tables where both relevance ratings and student performance are given for all outcomes. The departments are encouraged to review the results and mapping tables carefully and to make necessary adjustments. As can be seen from Table 5, almost all outcomes except functioning effectively as a team and (78% - 3.9 out of 5) and using techniques, skills and modern tools for Engineering practice (78% - 3.9 out of 5) need some improvement. Our goal is to raise the ratings of all outcomes to above 75%. It is difficult to draw a conclusion for some of the departments because the responses were low. It was noticed that recognizing applying mathematics, science, and engineering scores low in chemical, computer and mechanical departments. It was interesting to note that petroleum scored above 70% in all outcomes. Industrial Management System Engineering scored high at using the techniques, skills and modern engineering tools outcome (90%). 5
Table4: Students Performance College O Outcome 5 4 3 2 1 0 Average Weighted Average a Apply mathematics, science, and engineering 28 134 119 19 1 49 3.6 3.5 8% 38% 34% 5% 0% 14% 72% 70% b Design and conduct experiments and analyze and interpret data 20 45 55 6 1 223 3.6 3.7 6% 13% 16% 2% 0% 64% 72% 74% c Design a system, a component or a process d Function as an effective team member 29 86 92 19 1 123 3.5 3.7 8% 25% 26% 5% 0% 35% 70% 74% 29 85 51 5 0 180 3.8 3.9 8% 24% 15% 1% 0% 51% 76% 78% e Identify, formulate, and solve engineering problems 30 111 103 21 0 85 3.6 3.6 9% 32% 29% 6% 0% 24% 72% 72% f Understand professional & ethical responsibilities g Communicate effectively h Understand the impact of engineering solutions i Recognize the need for life-long learning j Know the contemporary issues 12 44 83 9 2 200 3.4 3.5 3% 13% 24% 3% 1% 57% 68% 70% 25 96 94 16 2 117 3.5 3.6 7% 27% 27% 5% 1% 33% 70% 72% 12 72 89 12 0 165 3.5 3.6 3% 21% 25% 3% 0% 47% 70% 72% 9 68 68 11 1 193 3.5 3.6 3% 19% 19% 3% 0% 55% 70% 72% 13 42 67 15 1 212 3.4 3.5 4% 12% 19% 4% 0% 61% 68% 70% k Use the techniques, skills and modern engineering tools for engineering practice 58 125 75 10 3 79 3.8 3.9 17% 36% 21% 3% 1% 23% 76% 78% 6
Figure 1: Student performance College 7
Table5: Students Performance (weighted Averages) comparison among programs L Outcomes chemical civil computer electrical ims mechanical petroleum core Weighted Average a Apply mathematics, science, and engineering 66% 80% 66% 74% 72% 68% 74% 70% 70% b Design and conduct experiments and analyze and interpret data 70% 76% 64% 78% 78% 74% 78% 72% 74% c Design a system, a component or a process 70% 84% 84% 74% 80% 68% 78% 62% 74% d Function as an effective team member 80% 86% 84% 72% 82% 72% 82% 70% 78% e Identify, formulate, and solve engineering problems 68% 78% 84% 72% 78% 68% 78% 72% 72% f Understand professional & ethical responsibilities 68% 72% 78% 68% 64% 72% 72% 66% 70% g Communicate effectively 72% 74% 84% 74% 84% 66% 76% 70% 72% h Understand the impact of engineering solutions 68% 68% 72% 68% 76% 72% 76% 72% 72% i Recognize the need for life-long learning 70% 74% 84% 72% 70% 66% 72% 72% 72% j Know the contemporary issues 64% 72% 78% 66% 72% 72% 76% 68% 70% k Use the techniques, skills and modern engineering tools for engineering practice 76% 86% 76% 76% 90% 78% 74% 80% 78% 8
Figure 2: Students Performance (weighted Averages) comparison among programs Part 1 (Outcomes from 1 to 6) 9
Figure 2: Students Performance (weighted Averages) comparison among programs Part 2 (Outcomes from 7 to 11) 10
Departmental Results: Civil Engineering Program Fall Semester: R E L E V A N C E Course Number Course Name a b c d e f g h i j k l 0620-201 Introduction to Design 1 1 3 3 3 3 2 0620-201 Introduction to Design 2 1 3 3 2 2 3 3 2 2 2 3 0620-271 Structural Analysis I 3 2 3 1 2 1 1 0620-310 Fluid Mechanics 3 2 1 3 1 1 0620-311 Water Resources 2 3 2 2 3 2 1 1 1 2 0620-311 Water Resources 2 3 2 2 3 2 1 1 1 2 0620-350 Soil Mechanics 3 3 1 2 3 1 1 0620-366 Transportation Engineering 3 2 1 1 2 1 2 2 2 2 2 2 0620-373 Reinforced Concrete I 3 1 3 1 3 2 2 1 2 2 2 3 0620-401 Coastal Engineering Fundamentals 3 1 1 3 2 0620-411 Water and Wastewater Treatment 0620-411 Water and Wastewater Treatment 0620-412 Open Channel Hydraulics 2 2 2 2 1 2 0620-476 Computer Applications in Structural Engineering 2 3 3 3 1 3 1 3 1 3 0620-476 Computer Applications in Structural Engineering 2 3 3 3 1 3 1 3 1 3 0620-493 Construction Engineering Design 3 3 3 3 2 3 3 3 3 3 3 0620-493 Construction Engineering Design 3 3 3 3 2 3 3 3 3 3 3 0620-497 Structural Engineering Design 3 1 3 3 3 1 2 2 2 2 3 2 P E R F O R M A N C E Course Number Course Name a b c d e f g h i j k l 0620-201 Introduction to Design 4 3 4 4 3 4 3 0620-201 Introduction to Design 5 3 5 4 4 4 5 5 3 3 4 5 0620-271 Structural Analysis I 4 3 4 2 3 3 2 0620-310 Fluid Mechanics 3 3 4 4 4 4 0620-311 Water Resources 4 5 4 5 4 5 4 4 3 4 0620-311 Water Resources 4 5 4 5 4 5 4 4 3 4 0620-350 Soil Mechanics 3 4 3 4 3 3 5 0620-366 Transportation Engineering 5 4 3 3 4 3 3 3 3 3 3 3 0620-373 Reinforced Concrete I 4 2 4 2 4 4 3 2 3 3 3 4 0620-401 Coastal Engineering Fundamentals 3 3 3 3 3 0620-411 Water and Wastewater Treatment 0620-411 Water and Wastewater Treatment 0620-412 Open Channel Hydraulics 4 4 4 3 4 4 0620-476 Computer Applications in Structural Engineering 4 4 4 4 2 3 3 4 3 5 0620-476 Computer Applications in Structural Engineering 4 4 4 4 2 3 3 4 3 5 0620-493 Construction Engineering Design 4 4 5 5 4 4 4 3 4 5 5 0620-493 Construction Engineering Design 4 4 5 5 4 4 4 3 4 5 5 0620-497 Structural Engineering Design 5 3 5 5 5 3 3 4 4 3 5 4 Weighted Average 4 3.94.24.33.93.63.73.53.73.74.24.1 11
Remarks and Suggestions Course Number Course Name Remarks and Suggestions 0620-201 Introduction to Design 1. The course is long, need to split and concentarte on Erath retaining structures 2. Slope stability, dewatering are tought in graduate courses 3. The number of students (only 4) need to be increased by means of involving the faculty more in presenting their courses 0620-201 Introduction to Design There was four sections for this course and every instructor taught the course differently from the others. There should be coordination between the instructors. Moreover, the course outline is not well defined and the textbook does not fit with the objective of this course..i expressed my views to two colleagues taching the same course but no coordination made between sections. 0620-271 Structural Analysis I need more basic principles of statics and mechanics of materials 0620-310 Fluid Mechanics This was a girls' section and the background of the students in mathematics and basic sciences was exceptionally good. Most of the students had a GPA above 3.0 and their English was very good. Students submitted their homeworks on time. Their performance in the tests was above average. The students participated in the discussions. This was an 8:00 a.m class and the attendance was not very good. Some assignments required team work, and they effectively contributed. The overall performance of the class was excellent. 0620-311 Water Resources A mixed class of 24 females and 4 males. Females are far stronger than male students. The male students were added to the class due to lake of teaching faculty in that spring semester. 0620-311 Water Resources This is 28 male students of very good performance 0620-350 Soil Mechanics 0620-366 Transportation Engineering 0620-373 Reinforced Concrete I 0620-401 Coastal Engineering Fundamentals 0620-411 Water and Wastewater Students need to strengthen their skills in Treatment reading, writing, and speaking in English. 0620-411 Water and Wastewater Treatment 0620-412 Open Channel Hydraulics This is a mixed class with a small numder of students [ 6 ] which made it easy to teach and get to know the students better. 0620-476 Computer Applications in Structural Engineering 0620-476 Computer Applications in Structural Engineering 1. Students seems to already form teams 0620-493 Construction Engineering before the start of the capstone deisgn 2. Design Last phase usually not fully-completed due to last minute work 0620-493 Construction Engineering 1. Students seems to already form teams 12
Design before the start of the capstone deisgn 2. Last phase usually not fully-completed due to last minute work 0620-497 Structural Engineering Design The class size shall not be more than 12. But I experiance always large class size. This takes the quality out. The Civil Engineering Department council has approved students per class shall not more than 12, but we face as high as 20. I hope this can be resolved, so that we can deliver quality education. Thanks Spring Semester: R E L E V A N C E Course Number Course Name a b c d e f g h i j k l 0620-200 Civil Engineering Drawing 3 2 3 0620-201 Introduction to Design 2 2 3 3 2 3 2 1 2 2 1 0620-201 Introduction to Design 2 2 3 3 3 2 3 2 1 1 2 0620-310 Fluid Mechanics 3 1 3 0620-311 Water Resources 2 3 2 2 3 2 1 1 1 2 0620-371 Structural Analysis II 3 2 3 3 2 2 3 0620-371 Structural Analysis II 3 2 3 2 2 1 3 0620-414 Hydraulic Engineering 2 3 1 2 2 2 Computer Applications in Water 0620-425 Resources and Environmental Engineering 3 3 2 3 2 2 3 0620-451 Foundation Engineering 2 1 3 2 2 1 2 1 2 1 2 1 0620-471 Steel Design I 3 3 3 3 2 3 3 3 3 3 Computer Applications in Structural 0620-476 2 3 3 3 1 3 1 3 1 3 Engineering P E R F O R M A N C E Course Number Course Name a b c d e f g h i j k l 0620-200 Civil Engineering Drawing 5 4 4 0620-201 Introduction to Design 3 4 4 3 3 3 3 3 2 4 3 0620-201 Introduction to Design 5 5 5 5 5 5 5 3 5 5 4 0620-310 Fluid Mechanics 4 4 4 0620-311 Water Resources 4 5 4 5 4 5 4 4 3 4 0620-371 Structural Analysis II 3 3 3 3 3 3 4 0620-371 Structural Analysis II 4 3 4 3 3 3 4 0620-414 Hydraulic Engineering 4 4 2 5 4 5 Computer Applications in Water 0620-425 Resources and Environmental 4 5 5 4 4 4 5 Engineering 0620-451 Foundation Engineering 4 4 4 4 4 4 4 0620-471 Steel Design I 5 1 5 5 4 4 4 4 4 5 Computer Applications in Structural 0620-476 4 Engineering 4 4 4 2 3 3 4 3 5 Weighted Average 4 3.54.34.34.13.73.83.33.83.54.4 3 Remarks and Suggestions Course Number Course Name Remarks and Suggestions 0620-200 Civil Engineering Drawing 0620-201 Introduction to Design It was difficult to motivate the students on certain ocassions due to their limited knowledge. It is felt that such a design course would be more productive at a later stage. Their command of English seemed miserably poor, especially their 13
0620-201 Introduction to Design 0620-310 Fluid Mechanics 0620-311 Water Resources 0620-371 Structural Analysis II 0620-371 Structural Analysis II 0620-414 Hydraulic Engineering 0620-425 Computer Applications in Water Resources and Environmental Engineering 0620-451 Foundation Engineering 0620-471 Steel Design I 0620-476 Computer Applications in Structural Engineering writing skills. The students felt the project very interesting since they wanted to prove their understanding of the subject and they were encouraged to be presented in a professional way. This is a class of 18 male students of an above average standing. The lecture was scheduled from 8-9 am and it was hardly a day when all the students attended the lecture. There were numerous students who absented themselves innumerous times. Many were habitual late comers. This has affected the overall performance of the class. This is a very strong group of students of 4 girls and 6 boys. They scored very high marks in all Tests and Final Exam. The students were very interested in using the Haestad Softwares. The small number of class [ 10 students ] allow the students to have a hand on experience using and applying the various Softwares [ FlowMaster, WaterCAD and StormCAD ] It was a design course yet the number of students was very high. It is recommended that the number of students for such a class to be limited to 20 maximum This is a batch of excellent students-they are very mature and possess high GPA. Moreover they are very eager to learn specially knowing that they are on the verge of graduating. The students were given enough confidence to allow them to attain proficiency in design and professional issues for employment at an entry level. Students are very weak in english reporting. Most of them resust attempts to assist them in gaining some technical writing skills. This aspect should be followed up by any colleague who teaches this course in the future. 14
Appendix A Instructor Class Evaluation Form 15
Kuwait University College of Engineering & Petroleum Instructor Class Evaluation Form Course Number and Title: Instructor: Semester: Number of times that you taught this course at KU: EVALUATION GRADING METHOD SYSTEM TOTAL 100 % GRADE DISTRIBUTION A A B+ B B C+ C C D+ D F or FA Sum I W Weight (W) 4.0 3.6 3.3 3.0 2.6 2.3 2.0 1.6 1.3 1.0 0.0 No. of Students (N) ΣN = N*W Σ(W*N) = CLASS GPA = Σ (W* N) / Σ N = CLASS GPA without (F or FA) = 16
Relevance Performance Program Outcomes Not Relevant Somewhat Relevant Moderately Relevant High Relevant Very Weak Weak Satisfactory Very Good Excellent Explanation Activities and Practices Interpretation & Evidence 1. Apply mathematics, science, and engineering 2. Design and conduct experiments and analyze and interpret data 3. Design a system, a component or a process 4. Function as an effective team member 5. Identify, formulate, and solve engineering problems 6. Understand professional & ethical responsibilities 7. Communicate effectively 8. Understand the impact of engineering solutions 9. Recognize the need for lifelong learning 10. Know the contemporary issues 11. Use the techniques, skills and modern engineering tools for engineering practice 12. An ability to assume responsibility at the entry level in the areas of specialization that are important to Kuwait and the region Remarks and Suggestions: 17
Appendix B Instructions for the Courses Assessment 18
Kuwait University College of Engineering and petroleum Office of Academic Assessment Instructions for the Course Assessment Introduction All instructors at the college should carry out course assessment and submit a course assessment file to the departmental assessment coordinators at the end of the term. In the following some guidelines on how to prepare an assessment file are given: Objectives of Course Assessment To obtain direct measurements of student performance To assure that students are acquiring the required outcomes To assure that learning experiences are consistent with the outcomes To establish the required feedback loops The items to be included in the course assessment file A hardcopy of completed instructor class evaluation form (ICEF) submitted online. A copy of the list of final grades Course syllabus including the following information as a minimum Instructor contact details and office hours Textbook and references Tentative course outline Dates of mid-term and final exams Grading policy A list of course outcomes and their relationship with the college or program outcomes (Course description and classification) A copy of final exam and major term project(s) Summary data and analysis from various assessment tools (e.g., oral and written report evaluation, teamwork, self evaluations) Samples of student works supporting the ICEF (e.g., key assignments, homework, exams, project reports, essays etc) Any other supporting material demonstrating student achievement (e.g., sample class portfolios, video recordings, etc.) Instructor Class Evaluation Form The main assessment tool used for the course assessment is the Instructor Class Evaluation Form. This form reports the grade distribution as well as the assessment of Program outcomes served by the course. First, the instructors are asked to indicate the level of importance of each outcome as it relates to the course. Normally, this rating should have been already assigned by the Teaching Area Group using the following guidelines: H (highly relevant): Demonstrating this outcome is critical for the students to perform successfully; or the students may benefit significantly from this course toward the outcome (formal instruction, practice, assessment). M (Moderately relevant): Demonstrating this outcome has considerable impact on the overall performance of the student, or the students may benefit moderately from this course toward the outcome (informal instruction, practice, and assessment). 19
L (Somewhat relevant): Demonstrating this outcome has only minor impact on the overall performance of the student. However, there are opportunities to observe this outcome (practice and assessment). The instructors then evaluate student performance relative to what is normally expected from them at their level according to the following scale: Students performance was very weak Students performance was weak Students performance was satisfactory Students performance very good Students performance excellent The best method of evaluation of the student achievement is to assess individual students relative to the outcomes. Then, an average rating can be obtained for the whole class. The rating should be justified by referring to specific student works or assessment results. The instructors are also asked to provide feedback on the course content and outcomes, instructional and assessment methods. They also comment on the achievement of program outcomes and indicate any deficiencies observed. Assessment Methods The assessment methods include but not limited to the following: Performance Appraisals (e.g., written and oral presentations, teamwork, lab experiments, artwork etc) Surveys (Online tools, or custom designed forms seeking student perception of learning gains, or their opinions on certain aspects) Traditional assessment methods (Exams, homework, project, etc) Assessment tools The following is a list of available assessment instruments to be used in course assessment. Instructors are encouraged to use standard tools as much as possible to facilitate analysis. However, these tools can be modified to suit a specific course, or additional tools can be adopted. Written reports Oral presentations Lab reports Teamwork Term Project Final Exam The instructors encouraged to submit summary statistical data in addition to the copies of the completed forms. 20