ASSESSMENT OF LEARNING OUTCOMES TO EDUCATIONAL OBJECTIVES OF MECHANICAL ENGINEERING DEPARTMENT/ BAGHDAD UNIVERSITY

ASSESSMENT OF LEARNING OUTCOMES TO EDUCATIONAL OBJECTIVES OF MECHANICAL ENGINEERING DEPARTMENT/ BAGHDAD UNIVERSITY SOROOR K. H. AL-KHAFAJI Mechanical Engineering Department College of Engineering of Baghdad University Baghdad, Iraq sor.k.hussain@gmail.com IMAN Q. AL-SAFFAR iman_alsaffar@yahoo.com ZUHAIR ISSA AHMED zouhairaldaoud2010@gmail.com ABSTRACT: The aim of this research is to assess the learning outcomes of the Mechanical Engineering Program/ College of Engineering at Baghdad University according to the measuring criteria of ABET. This is carried out so as to evaluate the undergraduate courses of the mechanical engineering with standards of ABET as a global accreditation that would be an important endeavor for the College of Engineering at Baghdad University (BU). Gap analysis has been done between the learning outcomes of ME program (BU) with the learning outcomes of other ME program which already has got the ABET, in order to identify which of the learning outcomes of ME program (BU) are not satisfied and need to develop. Therefore, suggestions have been proposed to improve the program outcomes with low levels. Keywords: Learning Outcomes, Objectives, ABET INTRODUCTION: Learning (program) outcomes specify what a learner is expected to know, understand, and able to do upon completion of a program. The educational program is supposed to have several outcomes, which are needed to clearly and reflect what students will learn from all courses of the program. Each major course topic may have one or more learning outcomes. Program educational objectives are broad statements that describe the career and professional accomplishments that the program is preparing the graduates to achieve. While the program outcomes are narrower statements that describe what students are expected to know and be able to do by the time of graduation. Assessment is one or more processes that identify, collect, and prepare data to evaluate the achievement of program outcomes and program educational objectives [1]. The Student Outcomes Assessment Plan (SOAP) of Mechanical Engineering (ME) program based on the requirements of the Accreditation Board for Engineering and Technology (ABET) is stipulated statement of the educational objectives of the program and how they ensure that the objectives are achieved. Achievement the mission of the program, student learning objectives were developed to enable the graduates of the ME program to be able to apply skills of engineering science, design and test systems, and other objectives that are compatible with ABET objectives of ME program [7]. Such kind of plan requires conducting surveys to evaluate the learning outcomes and to follow a continuous process to assess the achievement of these outcomes by the students taking in the program. Recently, institutions of higher education commenced to develop and asses their educational programs, including the programs of engineering colleges, therefore, the Department of Mechanical Engineering/ Baghdad University has assessed the ME program according to ABET criteria as an endeavor to get the international accreditation of engineering educational programs, where the first self assessment report accomplished during the academic year (2010-2011) has been adopted in this research. ABET-CRITERIA FOR ACCERDITING ENGINEERING PROGRAMS Every educational institution has its own vision, mission and educational objectives for which it was founded, ABET is focusing on the verification of specific criteria for any educational program for a Bachelor s degree in engineering. All programs seeking accreditation from the Engineering Accreditation Commission of ABET must demonstrate that they satisfy all of the General Criteria for Baccalaureate Level Programs, [2]. The eight criteria are stated as: Students, Program Educational Objectives, Students Outcomes, Continuous Improvement, Curriculum, Faculty, Facilities, and Support. Criteria 2 requires declaration of program educational objectives that are consistent with the mission of the institution, need to involve the program s various constituencies such as faculty, students, alumni, and employers. There must be a documented and effective process to facilitate the periodic review and revision of these objectives. In criterion 3, the program must have documented student outcomes that prepare graduates to attain the program educational objectives. Therefore, documentation procedures have been adopted in the department of Mechanical Engineering in a more organized and effective than had been before the 1052

ASSESSMENT OF LEARNING OUTCOMES TO EDUCATIONAL OBJECTIVES OF MECHANICAL. assessment year (2010-2011) to ensure that the graduates meet all the graduation requirements of ME program. LINKAGE OF THE EDUCATIONAL OBJECTIVES TO THE STUDENTS OUTCOMES The attainment of the outcomes by graduates is measured according to the achievement of the program educational objectives, where most of ME programs for bachelor s degree are somewhat share in the educational objectives of ABET, such as of these basic objectives: [2] Prepare graduates with knowledge, problem solving ability, and hands-on skills to enter careers in design, installation, manufacturing, testing, evaluation, and maintenance. Provide students with the necessary instruction and practical experience to work well in local and international team environments. Produce graduates that recognize the importance of being engaged in life-long learning, whether through self-study, continuing education programs or workshops. Produce graduates that have an understanding of ethical responsibility and service toward their peers, employers, and society. The learning outcomes of ME programs should be well defined and consistent with the minimum requirements of student outcomes specified in ABET, which often defines 11 program outcomes, and each of these outcomes is expected to achieve one or more of the program educational objectives. The linkage of ME program educational objectives to the program outcomes are shown in table 1, where the information of this table is given by depending on different assessment reports of ME programs from several world universities. SELF ASSESMENT OF MECHANICAL ENGINEERING (ME) PROGRAM OF BAGHDAD UINEVERSITY (BU) In self assessment report of ME program\ BU is that: [6] VISION The Department of ME endeavours to be one of the leading mechanical departments in Iraq and the region. PROGRAM EDUCATIONAL OBJECTIVES Since its establishment, the ME Department at the College of Engineering\ BU, worked hard and continuously on its noble mission in society service to achieve a number of educational objectives, the most important are: PEO1 Apply the latest technology and engineering tools to solve technical problems in applying mechanical engineering technology and related interdisciplinary fields. PEO2 The program aims to prepare students for successful engineering careers. PEO3 Provide students with practical design experience through partnerships and cooperation with industry. PEO4 Provide opportunities to qualified students through developed and improved teaching and research skills. PEO5 Graduated mechanical engineers to serve in industry, construction and other sectors of the mechanical engineering labour market. PEO6 Develop the abilities of management and technical skills for higher quality engineer. PEO7 Establish viable applied research that generates knowledge for local and foreign markets. MEASURING THE PROGRAM OUTCOMES IN ACHIEVING THE EDUCATIONAL OBJECTIVES By reviewing the general outcomes of engineering programs defined in ABET- Criterion-3, it is found that they are consistent with the educational objectives of the ME-program/BU, therefore the outcomes that adopted for the ME-program\BU are well defined and documented as follows: [6] (a) An ability to apply knowledge of mathematics, science, and engineering. (b) An ability to design and conduct experiments, as well as to analyze and interpret data. (c) An ability to design a system, component, or process to meet desired needs for mechanical engineering. (d) An ability to function on multi-disciplinary teams. (e) An ability to identify, formulates, and solves engineering problems. (f) An understanding of professional and ethical responsibility. (g) An ability to communicate effectively. (h) The broad education necessary to understand the impact of engineering solutions in a global and societal context. (i) Recognition the need for and an ability to engage in life-long learning. (j) Knowledge of contemporary issues. (k) An ability to use the techniques, skills, and modern engineering tools necessary for engineering practice. These outcomes are closely linked to the program educational objectives, the relationship of the program outcomes serving each objective is mapped as shown in table 2, which includes a list of program educational objectives reflecting their relationship with a-k outcomes of ME program\ BU in a term of frequent percentage. THE PROCESS OF DEVELOPING ME- PROGRAM ACCORDING TO EMPLOYER QUESTIONNAIRIES 1053

PRIMARY ASSESSMENT OF ME- PROGRAM OUTCOMES To support the educational program of ME- Department\BU, questionnaires to view employer s opinions about the program outcomes have been carried out with 17 different institutions in which the alumni of ME department are employed for the assessment process of the program outcomes. The results summary of these questionnaires is shown Figure 1. Table 3 shows the percentage of link between the questions and outcomes, and this linkage is represented in Figure 2. TOTAL RESULTS ANALYSIS OF EMPLOYERS QUESTIONNAIRES Table 4 lists the questions used to assess the program outcomes, Agree has given scores (5, 4) and Disagree given scores (3, 2, & 1) for all of the received employers responses. Theses scores have been analyzed to determine the Agree Rate (AR) for scores (1, 2) and the Disagree Rate (DR) for the questionnaire failed out scores (3, 2, & 1). AR and DR are defined in relations (1) and (2), respectively: [5] -------------------------------- (1) ------------------------------ (2) Where, TN A total number of Agree questionnaires, with scores (5, 4). TN total number of questionnaires TN D total number of Disagree questionnaires, with scores (3, 2, &1). It has been found that the percentage of agree rate (AR) approximately equals 81.8% out of total number of questionnaires (340), while the percentage of disagree rate (DR) is 18.2%. Figures 3, represents the total results of the questionnaires analysis. COMPARING THE OUCOMES OF ME PROGRAM\BU WITH A PROGRAM ACCREDIATED BY ABET In this paragraph, a comparison is undertaken between the program outcomes of (BU) with the ME program outcomes of Kettering University (KEU), which already been accredited by ABET. Information from the report titled Program Assessment Process/ Strategy, [4] have been adopted in this comparison. This comparison was made based on a weighted correlation for each of a-k outcomes related with specified educational objectives, which are defined in each program for both universities (BU, and KEU), as shown in table 5. The collected data of KEU report was taken by surveys with constituent groups of faculty, students, alumni, and industry. The weighting factors, range from 1 (low influence) to 4 (high influence) where 0 (no influence), are important to identify which outcomes need to be addressed when a deficiency is detected in relation with an objective. The same process of using weighting factors is used to assess the outcomes of ME program\bu related to its objectives. This comparison is made to identify which outcomes have good weights and which of them are still weak and need to be improved. Figure 4 shows the rating results based on weighted correlation. RESULTS DISCUSSIONS AND CONCLUSIONS RESULTS DISCUSSION a) Table 2 shows that each outcome of the ME program is defined to achieve at least two educational objectives of the program. b) From the results of table 3 which explained in figure 2 proved that the questions set out in the employer questionnaires had linked well with the program outcomes, where the highest linkage is reach to about 73% of total outcomes, while the lowest percentage of link is about 36%. Each question has been formulated to verify at least 4 of the expected outcomes. c) The results analysis of table 4 which are represented in figure 3 showed that the rate of agree is greater than disagree by an average of 63.6%. Those questions have been developed to know the perspective of employers about the program graduates. From table 4, the maximum rate of disagree is given to question 2 (Has, or the graduate, sufficient skills related to employment issues?). Therefore, the program of ME/ BU is supposed to focus on the promotion of professional abilities for the graduates in mechanical engineering, and to develop their technical skills and abilities to deal with modern engineering technologies. d) Table 5 shows the frequency rating of the program outcomes as related to its educational objectives for the ME Program\ BU and ME Program\ KEU where the last one was assessed according to ABET criteria. Then, in figure 4 the comparison between the two programs related to their rating of outcomes has been represented. Equation (3) has been used to find the difference as a percentage between the outcomes of the two programs, and the results of comparison are summarized in table 6. CONCLUSIONS - The outcomes of the ME program\bu are specified well to ensure the achievement of the educational objectives and that considered as one of basic standards for ABET. - The ME program of BU is focusing on the student outcomes for the graduate to have sufficient knowledge in basic science of engineering, mathematics, and good ability in design and analysis. 1054

ASSESSMENT OF LEARNING OUTCOMES TO EDUCATIONAL OBJECTIVES OF MECHANICAL. - Program outcomes are supposed to be directed towards the development of ability to communicate effectively by improving the skills of writing, speech in both Arabic and English, and developing general skills of communication especially in contracting and negotiation. REFERENCES [1] Accreditation Engineering Commission; (2010-2011) Criteria for Accrediting Engineering Program, Accreditation Board for Engineering and Technology (ABET), Baltimore, MD. [2] Accreditation Engineering Commission; (2012-2013) Criteria for Accrediting Engineering Program, Accreditation Board for Engineering and Technology (ABET), Baltimore, MD. [3] ABET Program Outcome and Assessment Report, Mechanical Engineering Department, University of Nevada, Reno, USA, 2005. [4] Program Assessment Process/ Strategy; www.kettering.edu/orgs/assess/me/me_assessment _Strategy.pdf, 2002. - There are weaknesses in the educational program for the graduate to understand the responsibility professionally and ethically. - Also, the ME program of BU should focus on giving general knowledge about the contemporary issues, and how to use the general knowledge in solving problems of surrounding environment. [5] Reilly R.; (2004) ABET, Self-Study Report for Mechanical Engineering Program, Department of Mechanical Engineering, New Mexico Institute of Mining and Technology (New Mexico Tech), Socorro, New Mexico. [6] Self Assessment Report of Mechanical Engineering Program (ME-program SAR), Mechanical Engineering Department, College of Engineering, Baghdad University, 2010, 2011. [7] Student Outcomes Assessment Plan, (SOAP); Mechanical Engineering Program; Department of Mechanical Engineering, College of Engineering, California State University, Fresno; April, 2008. Table 1 Linkage of ME Program educational objectives to the program outcomes [3], [5] Outcomes of ME Program ME Program Educational Objectives 1. Prepare graduates with knowledge, problem solving ability, and hands-on skills to enter careers in the design, installation, manufacturing, testing, evaluation, or maintenance. 2. Provide students with the necessary instruction and practical experience to work well in local and international team environments. 3. Produce graduates that recognize the importance of and engage in lifelong learning, whether through selfstudy, continuing education programs or workshops. 4. Produce graduates that have an understanding of ethical responsibility and service toward their peers, employers, and society. (a) Math, science, engr. fund. (b) Design & conduct experiments (c) Design (d) Work in Team (e) Problem solving (f) Professionalism & ethics (g) Communication (h) Broad education (i) Acquire new knowledge (j) Contemporary issues (k) Use modern tools\ techniques X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X 1055

Table 2 Linkage Program outcomes to the educational objectives Program ME Program Outcomes\ BU Educational Objectives (PEO\ BU) a b c d e f g h i j k PEO1 X X X X X X X X X X X PEO2 X X X X PEO3 X X X X X X X PEO4 X X PEO5 X X X X PEO6 X X X X X PEO7 X X X X Frequent % 43 71.4 57 57 28.6 28.6 14.3 57 43 57 71.4 University of Baghdad / College of Engineering / Mechanical Engineering Department Work Institutions Opinion Questionnaire About Graduates of BU / 2010 2011 Year of Graduation: College: Department: Specialization: Date of Appointment: Mail: Female: Ministry \ Institution That Employs Graduate: Department\section: Work Position: Governmental Individual Interest in the performance of graduates in their jobs in the institutions where they work in, is very useful to develop the quality of scientific programs and training courses for all disciplines in the universities. The feedback to the college and department will help to supplement the institutions in the public and private sectors with graduates those are highly qualified in the functional performance of their duties. Please provide us with information through the specimen to answer the following questions by mark ( ) in the place which reflects your point of view. Observation %, Highest score (5), Lowest score (1) Question 5 4 3 2 1 No. Stro ngly agree Agre e I don t know Disa gree 1 Has sufficient knowledge and information related to employment issues 3 11 1 2-2 Has sufficient skills related to employment issues 2 8 1 6-3 Possesses the skills of social communication with customers 3 11 1 2-4 Have written communication skills (writing the required reports are properly) 3 12 1 1-5 Possesses the skills of research and analysis in the affairs of the work 4 10 1 2-6 Possess critical thinking skills and the ability to solve problems 3 11-3 - 7 Possesses the skills of teamwork 2 12 2 1-8 Has the skills to work within the team 3 11 2 1-9 Possesses the skills of planning and organization for work 4 9 2 2-10 Has the ability of high productivity at work 2 10 2 3-11 Has the quality of work performance piece 6 9 1 1-12 Has the capacity to creativity, innovation and work development 4 8 2 3-13 Has the ability to comply with the various conditions of the work 4 10 3 - - 14 Has the ability to take responsibility 6 9-2 - 15 Possesses the skills of social interaction with colleagues 7 8 2 - - 16 Has the ability to accept guidance and ready for implementation 6 9 1 1-17 Has a sense of the importance of work performed by 7 9 1 - - 18 Has the ability to audit and review the work assigned to him 4 12 1 - - 19 Has the ability to deal with the problems and difficulties of work 6 7-4 - 20 Has the capacity to follow up on any up-dates in the field of work 4 11 1 1 - I don t agree Figure 1 Summary of employer questionnaires to assess the learning outcomes of ME program, (ME, SAR, 2010-2011) 1056

ASSESSMENT OF LEARNING OUTCOMES TO EDUCATIONAL OBJECTIVES OF MECHANICAL. Table 3 Linkage the questions of questionnaires to the ME program outcomes Outcomes a b c d e f g h i j k Questions Q1 X X X X X X X X Q2 X X X X X Q3 x x x x X Q4 x x x x X Q5 x x x x x x X Q6 x x x x X Q7 x x x X Q8 x x x x X Q9 x x x x x X Q10 x x x x x x X Q11 x x x x x x X Q12 x x x x x X Q13 x x x x X Q14 x x x x x X Q15 x x x x X Q16 x x x x x x x X Q17 x x x x x X Q18 x x x x x X Q19 x x x x x X Q20 x x x x x x x Figure 2 The Linkage percentage of Questionnaire s questions to the ME program outcomes 1057

Q.s 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 N A 14 10 14 15 14 14 14 14 13 12 13 12 14 15 15 15 16 16 13 15 TN A = 278 N D 3 7 3 2 3 3 3 3 4 5 4 5 3 2 2 2 1 1 4 2 TN D = 62 TN = 340 AR = 0.818 DR = 0.182 Table 4 The process to assessing the learning outcomes of ME program\bu Figure 3 Representations of the results analysis of employer questionnaires 1058

ASSESSMENT OF LEARNING OUTCOMES TO EDUCATIONAL OBJECTIVES OF MECHANICAL. Table 5 Weighted correlation of ME program outcomes for both of BU and KEU related to their educational objectives Program Educational Objectives PEO \ BU 1. Apply the latest technology and engineering tools to solve technical problems in the practice of mechanical engineering technology and related interdisciplinary fields. 2. The program will prepare students for successful engineering careers. 3. The program will provide students with practical design experience through partnerships with industry 4. Improve the abilities of management and technical will provide opportunities for qualified students to develop teaching and research skills. 5. Graduate mechanical engineers to serve in industry, construction and other sectors of the mechanical engineering labour market 6. Develop the abilities of management and technical skills to achieve higher quality for employment 7. Establishing viable applied research that generates knowledge for local and foreign markets. Math, science, engr. (a) fund. Design & conduct (b) expts. (c) Design Outcomes of ME Program \ BU (d) Teaming Problem solving (e) (f) Professionalism & ethics (g) Communication 3 2 2 1 4 2 0 1 1 0 3 4 1 2 3 3 4 3 2 1 1 2 1 3 4 1 3 3 2 4 3 1 1 2 2 1 2 4 2 3 2 2 3 2 2 1 1 1 1 2 4 1 3 3 3 4 2 2 1 1 3 2 2 4 2 3 4 4 3 3 2 1 1 1 1 2 4 2 4 3 3 2 3 2 1 1 1 1 1 2 2 Rating\ BU 3 3 2.7 3 2.8 2.1 0.8 1.1 1.6 1 2.1 3.7 1.6 (h) Broad education (i) Acquire new knowledge (j) Contemporary issues (k) Use modern tools\ techniques (l) Computers (m) Foreign culture/global context Program Outcomes (PEO s) Program Educational Objectives (PEO s) a b c d e f g h i j k l m 1. Are knowledgeable in the effective use of modern problem solving and design 3 3 3 2 4 1 1 1 1 0 4 3 4 methodologies 2. Understand the implications of design decisions in the engineering marketplace 1 2 3 2 2 3 3 3 2 2 2 2 1 3. Are effective engineers, i.e. ones who are able to formulate and analyze problems, think creatively, communicate effectively, synthesize 4 4 4 4 4 3 4 3 2 2 4 4 4 information, and work collaboratively 4. Have an appreciation and an enthusiasm for life-long learning. 1 1 2 2 1 1 1 2 4 3 2 1 1 5. Perform effectively on teams engaged in continuous improvement activities in 1 2 2 2 2 1 1 1 2 1 2 1 1 engineering and business processes. 6. Practice professionally and ethically in the field of Mechanical Engineering. 2 2 2 3 2 4 2 2 2 2 2 2 2 7. Are prepared for positions of leadership in business and industry. 1 1 2 3 3 3 4 3 3 3 2 2 1 Rating\ KEU 1.9 2.1 2.5 2.5 2.5 2.3 2.2 2.1 2.2 1.9 2.8 2.1 2 1059

Figure 4 Rating survey data of ME learning outcomes for BU and KEU Table 6 Results of program outcomes comparison between ME\KEU and ME\BU Criteria A b c d e f g H i j k l m Diff. % -58-43 -8-20 -12 9 64 48 27 47 25 100 100 1060