Drafting Program Educational Objectives for Undergraduate Engineering Degree Programs

Transcription

1 Drafting Program Educational Objectives for Undergraduate Engineering Degree Programs Ramakrishnan Sundaram Gannon University, Abstract - This paper outlines the process to draft Program Educational Objectives (PEOs) appropriate for undergraduate engineering degree programs at ABETaccredited institutions of higher education. In the ECE department at our University, the existing PEOs were deemed to have language that was very similar to that used in the ABET student outcomes. Therefore, it was imperative to distinguish the PEOs from the ABET student outcomes since the PEOs must quantify the expected attainments of graduates a few years after graduation. First, PEOs must reflect the Mission Statement of the institution and serve as a yardstick of student achievement three to five years following graduation. The objectives represent the expectations of the department from its graduates. Active participation by the faculty in defining the PEOs yield clear and concise objectives and promotes ownership of the goals of the Department and ABET process. However, not all faculty members are necessarily familiar with the assessment language and the process to evaluate the PEOs. In order to ensure a meaningful contribution from all faculty members involved in defining the PEOs, this paper presents a framework to define the PEOs that (1) adhere to the Mission of the University (2) achieve consistent and measurable expectations. Index Terms Program Educational Objectives, Accreditation, Assessment, Bloom taxonomy INTRODUCTION Program Educational Objectives (PEOs) for undergraduate engineering degree programs must clearly defme the vision of the program for its graduates three to five years after graduation. These objectives represent the expectations of the department from its graduates and must be crafted such that measurable objective evidence can be obtained through alumni surveys. In addition, there must be in place internal and external review procedures to periodically review and revise the language, assessment, and evaluation of the PE s. The focus of this paper is on defming the process to draft the PE s. The framework that we propose is first to provide an assessment language that is similar to the cognitive learning pattern of Bloom's Taxonomy for Knowledge, Comprehension, Application, Analysis, Evaluation and Synthesis during the different stages completed by our graduates in their career path. Faculty members should recognize that the students in the undergraduate degree program are viewed as learners who must acquire the intended skills over the duration of the baccalaureate program. These students are then expected to demonstrate the application of these skills three years following graduation. They are expected to asswne more responsibilities in management or to lead in their profession and/or society six years after graduation. Ten or more years after graduation, these students should become contributors with significant influence on policy and decision making in their chosen profession and/or society. The assessment language at these different stages of our graduates' career path is different. Hence, it is important to use the proper set of assessment language intended for the PEOs that focuses only on the immediate or first three years to measure the direct impact of our program on young careers, not the sixyear or the ten-year longer term objectives which would measure learning experiences and contributions beyond the direct impact of our program. Second, the framework requires faculty to extract key words from the Mission Statement of the institution which emphasize the broad and specific intent of undergraduate education along with a set of key words that the faculty see as attributes to gauge graduates in becoming the individuals the PEOs intended. Third, capture the distinction between educational objectives and student outcomes, and formulate each objective recognizing the need for measurability. The faculty of the Electrical and Computer Engineering (ECE) department at our institution met on different occasions for discussion and a final work session for almost six hours to complete the steps of the process in order to revise the existing set of PE s. The PEOs of our fully accredited undergraduate ECE program (accreditation period from 25 to 211) were deemed to have language that was very similar to that used in the ABET student outcomes. Therefore, it was imperative to distinguish the PEOs from the ABET student outcomes since the PEOs must quantify the expected attainments of graduates a few years after graduation. For instance, the ECE program at one pre-eminent institution [2] develops the PEOs according to three expected outcomes from graduates five years after graduation - professional expertise, innovativeness, and leadership. At another institution [3], continuous education program assessment is addressed by bringing together a working team of departments and education specialists using a Web-based assessment process to gather and evaluate the data. There are also institutions that stipulate /13/$ IEEE

2 the PEOs of the program without empowering the faculty to develop, own, and adopt a consistent set of measurable PEOs. Levell: PEO Assessment Section 2 provides a broad overview of the PEO assessment cycle at our institution with internal and external review processes. Section 3 describes the framework to formulate the PEOs. Section 4 discusses the PEO assessment and evaluation process, summarizes the mapping of these PEOs to the ABET student outcomes, and the preparation of the alumni survey. The conclusions and future issues appear in Section 5. SECTION 2: PEO ASSESSMENT CYCLE The following requirements are stipulated by ABET in its Self-Study Questionnaire [1] as part of Criterion 2. Program Educational Objectives. A. Mission Statement of the Institution B. Program Educational Objectives (PEOs) C. Consistency of the PEOs with the Mission Statement D. Program constituencies E. Process for revision of the PEOs In order to fulfill the requirement of consistency between the PEOs and the Mission Statement of the institution, key words and phrases are identified in the Mission statement that would directly relate to the language used to formulate the PEOs. For example, the following Mission Statement of our institution has words and phrases underlined to indicate their significance in the drafting of PEOs. This Institution is a Catholic, Diocesan, student-centered University which provides for the holistic development of undergraduate and graduate students in the Judeo Christian tradition. As such, it offers each student outstanding teaching and a value-centered education in both liberal arts and professional specializations in order to prepare students for leadership roles in their careers, society, and church. The University faculty and staff are committed to excellence and continuous improvement in teaching, learning, scholarship, research, and service. The University's environment is to be one of inclusiveness and cultural diversity. Figure 1 displays the overall Quality Assessment cycle with Level 1 representing the PEO Assessment highlighted. as the focus of this paper. The assessment cycle for Level 1 is once every three to five years to gauge the relevancy of the PEOs to the requirements of the program constituents because the PEOs are measured three to five years after graduation. Student outcomes assessment, identified as Level 2, is in a different cycle but provides the major input to address the attainment of the PEOs. The Level 1 PEO Assessment cycle is shown in Figure 2. FIGURE I QUALITY ASSESSMENT CYCLE Constituents (EAC) Faculty defines/refines r- plan/refines PEO PEO 1 Faculty make decisions for changes or implementation I- Assessment cycle: 3-5 years & performance criteria Committee analyzes evidence and generate action items FIGURE 2 LEVEL I PEO ASSESSMENT CYCLE Level 2: SO Assessment Level 3: CO Assessment Collectevidence forpeo The program constituents for the PEO assessment process comprise the faculty, staff, students, and the local industry represented by its members on the Engineering Advisory Council (EAC). The toolsets used in the PEO assessment process are as follows: Alumni survey (external) EAC review (external) Executive summary report for student outcomes (internal) SECTION 3: FRAMEWORK TO FORMULATE PEOs The first step was that faculty members as individuals had independent PEO ideas, leveraging our multi-cultural and multi-discipline resources. Then, the faculty members, objectively as a team, had long workout sessions to revise and commit to these new PEOs, thus promoting PEO ownership as a Department. At the beginning of the workout session, the faculty identified and agreed to the following keywords of the Mission Statement of our institution -

3 professional development, value-centered, leadership roles, cultural diversity, and lifelong learning. In addition, the faculty identified the categories in which objective evidence shall be obtained to measure each key word keeping in mind the proper set of assessment language. Faculty members then formed break-out groups of two to draft the PEO suitable for each key word. Finally, the faculty reconvened as one group to review and revise each objective as well as to arrange the objectives in the order of chronological impact and accomplishment before reaching agreement for adoption. The framework for the review and revision of the PEOs at the departmental level involving all the faculty members comprises the following broad stages. 1. Provide the assessment language 2. Use the key words and phrases extracted from the Mission Statement of the institution to identify attributes to gauge graduates 3. Capture the distinction between the educational objective and the student outcomes 4. Formulate each objective to be measurable embedded course assessment processes, to determine if the skill sets necessary for attainment of PEOs are being provided within each course and across the curriculum. The external review comprises the assessment of data gathered through the alumni survey to determine how well our graduates are performing in the workforce three to five years following their graduation. This measures the level of attainment of each PE. The PEO evaluation process determines the relevancy of each PEO to address and successfully meet the needs of the program constituents. The program constituents are faculty, students, alumni, and industry. The evaluation of the PEOs by both the EAC and the faculty members helps address the needs of program constituents. Action items generated at faculty and EAC review meetings are expected to lead to changes in the PEOs and have significant impact on the shaping of our educational practices Attainment The assessment language is patterned along the lines of the Cognitive domain of Bloom's taxonomy [4]. The Cognitive domain (or What should I know?) corresponds to the recall or recognition of specific facts, procedural patterns, and concepts that serve in the development of intellectual abilities and skills. There are six major categories from simple to complex - Knowledge, Comprehension, Application, Analysis, Synthesis, and Evaluation - to be mastered. Although these categories are covered within every course at all levels (first year to senior year), the focus of the pattern of learning at each level is as follows: ChangesloPEOs Changes to Educational Practices Pteparednessof Students'skillsets Level of Course First year Second year Third year Fourth year TABLE I FOCUS OF LEARNING Focus of learning Knowled e Comprehension Application & Analysis Evaluation & Synthesis SECTION 4: PEO ASSESSMENT & EV ALVA TION Figure 3 illustrates the components and the flow of the process of PEO assessment and evaluation. The two primary approaches to PEO assessment are (1) internal review, and (2) external review. The internal review consists of assessing the attainment of student outcomes, through senior exit surveys and FIGURE 3 PEO ASSESSMENT AND EVALUATION PROCESS The mapping of the student outcomes to the PEOs is intended to help us determine if our educational practices within each course and across the curriculum prepare our graduates to meet the intended PEOs. Table II illustrates the mapping. The PEOs used to complete this mapping are as follows: PEO 1. Sound preparation for adaptation in the exciting, rapidly-changing areas of technology and a passion for lifelong learning PEO 2. Ability to understand how engineering solutions affect our society and respond to ethical and public issues, including safety, social, and environmental concerns.

4 PEO 3. Ability to apply personal values to daily and professional life, develop the skills necessary for exercising infonned literary and aesthetic judgments, and appreciate diverse cultures and societies PEO 4. Ability to communicate effectively in both oral and written forms and work efficiently with multidisciplinary/multi-cultural teams which foster leadership qualities PEO 5. Foundation in problem formulation and problem solving skills to include the following: Sound preparation in general science and applied mathematics Strong preparation in electrical engineering and applications Strong preparation in computer and software systems development Effective use of computer-aided design and analysis tools Quality experience in engineering design PEO 2. Demonstrate teamwork and leadership qualities and/or the attainment of leadership roles in a global work environment PEO 3. Demonstrate technical competency in applying comprehensive engineering knowledge throughout their chosen profession PEO 4. Demonstrate passion for life-long learning through engaging in the rapidly changing and emerging areas of technology, and/or continued professional development This set of PEOs is also mapped to the ABET outcomes as in Table II. Table III summarizes this mapping. PEO# TABLE III SUMMARY OF REVISED MAPPING Student Outcomes 1 S6, S8, SO 1 TABLE II RELATIONSHIP OF STUDENT OUTCOMES TO PEOs 2 S4, S7 Student Outcomes ABEICrit"ia: SO 1: Ability to apply knowledge of mathematics, science, and - N rr,... engineering S2: Ability to design and condoct experiment, as well as to analyze and inteljlret data S3: Ability to design a system, component, or process to meet desired needs S4: Ability to function on multi-disciplinary teams S5: Ability to identify, formulate, and solve engineering problems S6: Understandin,. of professional and ethical responsibility S7: Ability to commwlicate effectire1y S8: Board education necessary to tulderstand the impact of engineering solutions in a global and societal context S9: Recognition of the need for, and an ability to engage in life-long learning SOl O: knowledge of contemporary issuts SO 11: Ability to use the techniques, skills, and mndero engineering tools necessary for engineering practice ECE Proer.m- Sp,cific Comp,t,nci", S12: Knowledge and ability to apply mathematics including probability, statistics, and discrete mathematics SOl3: Develop systems containing hardware and software components SO 14: Analyze and Design of complex electrical and electronic devices The language of the PEOs listed above does not make the clear distinction between PEOs and ABET outcomes. The implementation of the process described in this paper resulted in the following set of PEOs.,n 3 SOI-3, S5, S8, SOII-14 4 S1-3, S5, S7, S8-14 The alumni survey provides the data for assessment of the PEOs as part of the external review. The survey addresses the following question: How well are our graduates really doing in the workforce? The survey is administered through secure access on the Web [5] and has quantitative and qualitative sections in the following categories. General information Employment History Preparedness of Gannon Education Soft Skills and Broad Education Liberal Studies Intern Experience and Professional Societies Rate education based on the PEGs Table IV lists the questions asked in the quantitative section of the alumni survey for the category titled Preparedness of Gannon Education. PEO 1. Demonstrate professional ethics and personal values in daily and professional life that exercise infonned literary and aesthetic judgments by leveraging diverse cultures and societies

5 TABLE IV QUANTITATIVE SECTION - I Preparedness of Gannon Education Questions How well do you feel your undergraduate training helped you to develop the ability to apply technical knowledge in the solution of specific workrelated problems? 2 How well did your undergraduate training prepare you to design and conduct experiments? 3 How well did your undergraduate training prepare you to analyze and interpret research data? 4 How well did your undergraduate training prepare you to design a system, component or process to meet task needs? After graduation, how prepared were you to function effectively on a multi -disciplinary team? 6 How well did your undergraduate training prepare you to develop your leadership skills? 7 How well did your undergraduate training prepare you to identify, formulate and solve engineering problems? Compared to engineering friends and colleagues from other colleges and universities, how well do you feel the Gannon Engineering program prepared you to handle the technical aspects of your job? [I] REFERENCES "ABET Self-Study Questionnaire: Template for a Self-Study Report," Review Cycle. [2] [3] Yue, Dick K.P. & Masi, B. Beyond ABET: A new school-wide process for continuous improvement in engineering education Proceedings of the 23 American Society for Engineering Education Annual Conference & Exposition [4] Anderson, L. & Krathwohl, D. A. Taxonomy for Learning, Teaching and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives New York: Longman. 21. [5] Evaltooll': A Web-based Assessment tool ( The qualitative section for this category asks the graduate to comment on any special strengths or weaknesses they saw in the technical preparation they received at Gannon. Table V lists the questions asked in the quantitative section of the alwnni survey for the category titled Soft Skills and Broad Education. TABLE V QUANTITATIVE SECTION - II Soft Skills and Broad Education Questions Do you feel your Gannon education helped you to understand the ethical responsibilities of your profession? 2 How well did your Gannon education prepare you to give oral presentations? 3 How well did your Gannon education prepare you to write technical reports and other written communications? 4 Do you feel you have the ability to use the most modern engineering tools and techniques effectively? Do you feel your education at Gannon was broad enough to give you a perspective on how a particular engineering solution might impact society? SECTION 5: CONCLUSIONS & FUTURE PLANS This paper describes how the process to draft PEOs is and implemented in our ABET accredited ECE degree program. The key requirements to guarantee success of this process are as follows: Engagement and commitment of the entire engineering department to the discussion, ratification, and eventual adoption of the PEOs Active participation by the faculty in defining the PEOs to promote ownership of the goals of the Department and ABET process Adherence to the Mission of the University Distinguish PEOs from student outcomes Draft PEOs with measurable outcomes