EVALUATION OF PROGRAM OUTCOMES THROUGH THE FINAL YEAR PROJECT

Transcription

1 Journal of Engineering Science and Technology Special Issue on UKM Teaching and Learning Congress 2013, June (2015) School of Engineering, Taylor s University EVALUATION OF PROGRAM OUTCOMES THROUGH THE FINAL YEAR PROJECT MASLI IRWAN ROSLI 1, *, NURINA ANUAR 1, SITI ROZAIMAH SHEIKH ABDULLAH 1,2, ABU BAKAR MOHAMAD 1 1 Department of Chemical and Process Engineering, 2 Centre for Engineering Education Research, Faculty of Engineering and Built Environment, Universiti Kebangsaan Malaysia, UKM Bangi, Selangor Darul Ehsan, Malaysia *Corresponding Author: irwan@eng.ukm.my Abstract The Program Outcomes (PO) of the Department of Chemical and Process Engineering (JKKP) is designed to ensure that the students enrolled in the department attain the objectives set by the Program Educational Objectives (PEO). Thus, evaluation was conducted on student achievement vis-à-vis the PO program to assess the extent to which the PO was achieved and to identify opportunities for quality improvement. As the method adopted in this study, PO achievement was evaluated through the student accomplishment in one of the core courses and in a final course in JKKP, namely, the Final Year Project (FYP) for the academic year (AY) Students in both programs at JKKP - the Chemical Engineering Program (KK) and the Biochemical Engineering Program (KB) - were assessed for their performance in FYP. Their PO achievement was analyzed based on their acquired FYP performance. Both programs generally achieved excellent POs, and higher PO achievements were obtained mostly by students who had a higher cumulative grade point average (CGPA). Opportunities of improving the learning process based on this PO evaluation are proposed in this article to enhance the quality of learning in future AYs and to ensure that the PEO of JKKP is achieved. Keywords: Program outcomes; Final year project; Chemical engineering. 1. Introduction An outcomes-based education (OBE) has been implemented in the Faculty of Engineering and Built Environment (FKAB) of the Universiti Kebangsaan Malaysia Peer-review under responsibility of Universiti Kebangsaan Malaysia 92

2 Evaluation of Program Outcomes through Final Year Project 93 Abbreviations AY CGPA FKAB FYP JKKP KB KK OBE PEO PO Academic Year Cumulative Grade Point Average Faculty of Engineering and Built Environment Final Year Project Department of Chemical and Process Engineering Biochemical Engineering Program Chemical Engineering Program Outcomes-Based Education Program Educational Objectives Program Outcomes since the academic year (AY) to ensure that the process of teaching and learning is effective [1]. The Department of Chemical and Process Engineering (JKKP) has employed the OBE approaches, and the Program Outcomes (PO) has therefore been designed for and applied in both programs of JKKP: the Chemical Engineering Program (KK) and Biochemical Engineering Program (KB). PO refers to a knowledge or skill that students should have mastered upon completing their studies in these programs. The present study primarily aims to enable sustainable quality improvement that can be implemented in both the KK and KB programs by evaluating PO achievements. Abdullah et al. (2013) applied a planner of research activities as a means of continuous quality improvement on the implementation of the research project. Their findings showed positive effects based on the student feedback [2]. As such, the present study was conducted to identify students achievements based on the designed POs. The weaknesses identified could be used as an opportunity for improvement and to further reinforce the reputation of JKKP in producing quality engineers. Previous studies have evaluated PO achievement through several methods, such as conducting surveys among selected graduates [3, 4] and assessing particular POs through student presentations [5, 6]. Studies on PO achievement have been conducted using more efficient online questionnaires among alumni [7]. The evaluation of PO achievement has also been conducted in stages, such as in surveys for students of integrated projects [8]. Evaluation has also been conducted by assessing student performance in the final examination [9] and final year projects (FYP) [10]. Table 1 shows a list of the POs implemented in JKKP for the AY The POs listed in Table 1 are evaluated in the present study, except for POs 5, 6, and 10, which are irrelevant to the course outcomes of FYP in JKKP. 2. Methodology In this study, PO achievement was evaluated by acquiring the detailed marks obtained by the students in their FYP. The marks of their FYP were distributed based on the relevant PO of the FYP: POs 1, 2, 3, 4, 7, 8, 9, 11, and 12. The evaluation forms utilized in the FYP were composed and mapped out to facilitate the evaluation of PO achievement. All marks in the FYP progress presentation, viva, and thesis marks were acquired for the PO evaluation. For the AY 2012

3 94 M.I. Rosli et al. 2013, KK consists of 35 students, and KB of 22, thereby resulting in a total of 57 students who participated in the study. PO PO1 PO2 PO3 PO4 PO5 PO6 PO7 PO8 PO9 PO10 PO11 PO12 Table 1. List of program outcomes in JKKP. Description To apply knowledge of mathematics, science, chemical/biochemical engineering fundamentals and an engineering specialisation to the solution of complex engineering problems. To identify, formulate, research literature and analyse complex chemical/biochemical engineering problems reaching substantiated conclusions using first principles of mathematics, natural sciences and engineering sciences. To design solutions for complex chemical/biochemical engineering problems and design systems, components or processes that meet specified needs with appropriate consideration for public health and safety, cultural, societal, and environmental considerations. To conduct investigation into complex chemical/biochemical engineering problems using research based knowledge and research methods including design of experiments, analysis and interpretation of data, and synthesis of information to provide valid conclusions. To create, select and apply appropriate techniques, resources, and modern engineering and IT tools, including prediction and modelling, to complex chemical/biochemical engineering activities, with an understanding of the limitations. To apply reasoning informed by contextual knowledge to assess societal, health, safety, legal and cultural issues and the consequent responsibilities relevant to professional engineering practice. To understand the impact of professional engineering solutions in societal and environmental contexts and demonstrate knowledge of and need for sustainable development. To apply ethical principles and commit to professional ethics and responsibilities and norms of engineering practice. To communicate effectively on complex chemical/biochemical engineering activities with the engineering community and with society at large, such as being able to comprehend and write effective reports and design documentation, make effective presentations, and give and receive clear instructions. To function effectively as an individual, and as a member or leader in diverse teams and in multi-disciplinary settings To recognise the need for, and have the preparation and ability to engage in independent and life-long learning in the broadest context of technological change. Demonstrate knowledge and understanding of chemical/biochemical engineering and management principles and apply these to one s own work, as a member and leader in a team, to manage projects and in multidisciplinary environments.

4 Evaluation of Program Outcomes through Final Year Project 95 This study compares the average performance of the KK program and that of the KB program for each PO. This study also focuses on the relationship between the PO achievements and the general performance of the students based on their cumulative grade point average (CGPA). The CGPA performance of the students was divided into three groups: Group 1 with a CGPA equal to or more than 3.50, Group 2 with a CGPA of less than 3.50 and equal to or greater than 3.00, and Group 3 with a CGPA of less than The achieved POs were plotted and compared with the CGPA performance for the analysis of their comparison and relationship. 3. Result and Discussion Figure 1 compares the performance of the students achieved PO in KK and that in KB. The achieved POs for both programs are generally excellent given that the rating exceeds 75%, which is equivalent to a grade of A to A in the current FYP grading system. The PO achievements in KK are also slightly better than those in KB with a standard error of less than 1.7% (Fig. 1). However, the comparative performance of the POs in both programs in the previous AY shows that KB students have a slightly higher achievement than the KK students [10]. Thus, the performance of the students varies and depends on the particular batch. The continuous evaluation of PO achievement is necessary for a better understanding of the trends of PO achievement in the two programs. In this study, the average PO achievement for KK and KB is 84% with a standard deviation of less than 7.0 and 79% with a standard deviation of less than 7.7, respectively. The achieved PO for KK is approximately 83% 87%, whereas that for KB is between 76% 82%. The PO achievements are consistent with the results of the average CGPA of the students in both programs. The average CGPA of KK students is 3.45, whereas that of KB students is 3:37, which is slightly lower than that of the former. Thus, the PO achievement results are in line with the average CGPA of the students in KK and KB. The lowest PO achieved can be observed in PO 2 for both programs, and the highest in PO 12 for both programs. The prevalence of this same pattern in both programs demonstrates the need to improve the ability of students to accomplish PO 2, namely, to identify, formulate, research literature and analyze complex chemical/biochemical engineering problems to reach substantiated conclusions based on the basic principles of mathematics, natural sciences, and engineering sciences. In the FYP, PO 2 achievement is measured by evaluating the ability of students to conduct a literature review, particularly in analyzing previous studies related to research and providing comments and exchanging ideas on the literature review. An improvement on the explanations provided to students for the methods of conducting literature reviews can be proposed to the FYP coordinator. Such measure will better prepare the students before they conduct a literature review. For example, the coordinator can collaborate with the library in organizing a workshop on how to conduct and search information in the library and journal database. Such an activity will enhance the general performance of the students, particularly in achieving PO 2. Enhancements must also be implemented to increase the achievement of POs involved in FYP. Efforts should be undertaken to bridge the gap between the KK and KB programs. For example, the evaluation of PO can be improved by organizing and assessing a poster competition for the FYP. This activity may also help the distribution of the PO assessment, to which a

5 96 M.I. Rosli et al. percentage of marks is contributed by the poster competition. Furthermore, the involvement of people from related industries who can support the examination of the students performance may be beneficial in reducing the gap between the PO achievement of the KK program and that of the KB program. Fig. 1. Comparison of PO achievements between the program of Chemical Engineering and Biochemical Engineering. Figures 2 and 3 show the distribution of PO achievements based on the CGPA groups. Students were grouped based on their CGPA results. The first group consists of excellent students with a CGPA equal to or more than The second group consists of good students with a CGPA equal to or more than 3.00 and less than The third group consists of intermediate students with a CGPA of less than Figure 1 shows the data distribution in KK of the first group with a standard deviation of less than 7.1 and standard error of less than 1.8%, the second group with a standard deviation of less than 8.2 and standard error of less than 1.9%, and the third group with a standard deviation of less than 7.1 and standard error of less 5.0%. Figure 2 shows the distribution in KB of the first group with a standard deviation of less than 6.7 and standard error of less than 2.5%, the second group with a standard deviation of less than 7.3 and standard error of less than 2.0%, and the third group with a standard deviation of less than 0.0 and standard error of 0.0. The zero values of the standard deviations and errors in KB s third group result from the total number of students in the group, which is only one. Overall, no clear pattern can be inferred from the findings on the students of KK (Fig. 2). However, for the students of KK, achieving the best grade does not necessarily mean exerting the same best performance in PO achievement. For example, the highest achievement in POs 3 and 12 was obtained by students in the second group (good students) and not by those in the first group (excellent students). These results may be attributed to PO 3, which is generally related to the students ability to design experimental methods and identify tools and experimental equipment, whereas PO 12 is generally related to project planning

6 Evaluation of Program Outcomes through Final Year Project 97 and management. The same pattern is observed among students of KB (Fig. 3), indicating that the highest PO 12 achieved was obtained by the group of intermediate students and not by the excellent and good students. Although at this stage of the study the relationship between the achieved PO, particularly POs 3 and 12, and groups of good and intermediate students remains unclear, the present results call for a further study on students PO achievements in the coming AYs to clarify the relationship between PO achievement and CGPA performance. Fig. 2. PO achievements based on CGPA groups in Chemical Engineering. Fig. 3. PO achievements based on CGPA groups in Biochemical Engineering. In addition, a comparison of the PO achievements of the different groups of students in KK shows that PO achievements are approximately the same for both Journal of Engineering Science and Technology Special Issue 1 6/2015

7 98 M.I. Rosli et al. groups of excellent and good students. Meanwhile, the PO achievements of intermediate students are slightly lower than those of excellent and good students. Therefore, no significant inconsistency in PO achievement is noted across the groups of the students in KK. Meanwhile, among the students in KB, the PO achievements of excellent students are slightly higher than those of good students. However, significant differences are observed between the PO achievements the group of excellent and good students and those of intermediate students. Hence, some developments should be undertaken, particularly for the students in KB to minimize the gap of PO achievements across groups. Nonetheless, the PO achievements generally follow the CGPA performance of the students: the highest PO achievements are attained by the group of students who scores better CGPA. However, this observation requires further evaluations of PO in future AYs. 4. Conclusions This study found that the students of JKKP in both KK and KB programs achieved excellent POs, exceeding the 75% mark. KK students obtained better PO than KB students, which is consistent with their CGPA achievements. As expected, PO achievements were associated with the students CGPA performance, with the higher POs obtained by students with higher CGPA. However, POs 3 and 12 exhibited interesting patterns, in which higher PO achievements were acquired by students who did not excel in terms of CGPA. Nevertheless, such patterns may be too early and limited to generate conclusions. Hence, further studies that evaluate the POs for the next AYs are necessary to derive solid conclusions. The improvement of teaching and learning activities is vital to the enhancement of PO achievements, particularly PO 2, which is indicated as the lowest. The enhancement of teaching and learning may similarly reduce the inconsistency in the POs across the groups of excellent, good, and intermediate students. Acknowledgment The authors wish to express appreciation to the Universiti Kebangsaan Malaysia for supports and funding these research activities through PTS and PTS References 1. Nor, M.J.M.; Hamzah, N.; Basri, H.; and Badaruzzaman, W.H.W. (2006). Pembelajaran berasaskan hasil: Prinsip dan cabaran. Pascasidang Seminar Pengajaran dan Pembelajaran 2005, Othman, N.T.A.; Misnon, R.; Abdullah, S.R.S.; Kofli, N.T.; Kamarudin, S.K.; and Mohamad, A.B. (2011). Assessment of programme outcomes through exit survey of chemical/biochemical engineering students. Procedia Social and Behavioral Sciences, 18, Abdullah, S.R.S.; Anuar, N.; Rosli, M.I.; Kofli, N.T.; and Rahman, N.A. (2013). Improvement in the implementation of undergraduate research project. Procedia Social and Behavioral Sciences, 102,

8 Evaluation of Program Outcomes through Final Year Project Mutalib, A.A.; Maulud, K.N.A.; Yusuff, N.I.M.; Rahmat, R.A.A.; and Chik, Z. (2012). The achievement of program outcomes from the perspective of CSED, UKM final year students: Part II-Session 2006/2007. Procedia Social and Behavioral Sciences, 60, Ismail, M.; Yaakob, Z.; Markom, M.; Nordin, D.; Talib, M.Z.M.; Rosli, M.I.; and Jahim, J.M. (2012). Tahap kemahiran penggunaan perisian bermatlamatkan amalan kejuruteraan professional dan pembelajaran sepanjang hayat pelajar tahun dua Sesi 2011/2012 secara ukuran PO6 dan PO9 melalui pembentangan projek bersepadu. Prosiding Seminar Pendidikan Kejuruteraan & Alam Bina, Ismail, M.; Yaakob, Z.; Markom, M.; Nordin, D.; Talib, M.Z.M.; Rosli, M.I.; and Jahim, J.M. (2012). Tahap komunikasi dan kerjasaama berkumpulan pelajar tahun dua Sesi 2011/2012 secara ukuran PO7 dan PO8 melalui pembentangan projek bersepadu. Prosiding Seminar Pendidikan Kejuruteraan & Alam Bina, Khamis, N.K.; Ihsan, A.K.A.M.; Abdullah, S.; Ramli, N.F.L.; and Sajuri, Z. (2012). Soal selidik pencapaian objektif program dan hasil pembelajaran program alumni Jabatan Kejuruteraan Mekanik dan Bahan melalui kaedah atas talian. Prosiding Seminar Pendidikan Kejuruteraan & Alam Bina, Kamarudin, S.K.; Abdullah, S.R.S.; Kofli, N.T.; Rahman, N.A.; Tasirin, S.M.; Jahim, J.M.; and Rahman, R.A. (2012). Pengukuran hasil pembelajaran program untuk projek bersepadu melalui soal selidik pelajar Jabatan Kejuruteraan Kimia dan Proses Tahun III Sesi 2010/2011. Prosiding Seminar Pendidikan Kejuruteraan & Alam Bina, Arshad, I.; Razali, S.F.M.; and Mohamad, Z.S. (2011). Pengukuran hasil pembelajaran program melalui peperiksaan akhir bagi Program Kejuruteraan Awam dan Struktur. Prosiding Seminar Pendidikan Kejuruteraan & Alam Bina, Anuar, N.; Abdullah, S.R.S.; and Mohamad, A.B. (2012). The measurement of program outcomes through research project in the Department of Chemical and Process Engineering, UKM. Procedia Social and Behavioral Sciences, 60,