1 ABET & NCAAA ACCREDITATION Prof. Ali M. Al-Bahi Director of the Academic Accreditation Unit Faculty of Engineering, King Abdulaziz University Modified and Presented by Dr. Faisal Iskanderani UBTVR for Q&D Sept 11, 2014
4 Introduction CEIT will be working towards National, and International Accreditation ( NCAAA and ABET) Time line: end of 2016 Today s workshop will deal with one component of accreditation, i.e., Students Learning Outcomes delay this
5 Introduction ABET and NCAAA accreditation is assurance that a college or university program meets the quality standards established by the profession for which it prepares its students. NCAAA and ABET have similar accreditation criteria We will work together in parallel to achieve both at the same time, using the same documentation
6 ABET Documents Criteria for Accrediting Engineering Programs Accreditation Policy and Procedure Manual Template for the Engineering Self Study Report Course binders or files NB: Documents should be downloaded for the year of the accreditation cycle from: September 11,
7 NCAAA Documents National Qualifications Framework, NCAAA Standards for the Accreditation and Quality Assurance for Institutions and Programs (11 Standards), Self Evaluation Scales for NCAAA Standards, and Forms and Templates for Institutions and Programs as follows:
8 NCAAA Forms and Templates Course Specification Course Report Field Experience Specification Field Experience Report Program Specification Annual Program Report Template for Report on Periodic Program Self-Study
9 Accreditation Time-Line (a 2-3 year process) Note: In addition to one year for preparations September 11,
10 ABET Criteria for Accrediting Eng. Programs 1. Students 2. Program Education objectives (PEOs). 3. Student Outcomes (SOs). We will call them Program Learning Outcomes 4. Continuous Improvement 5. Professional Component (curriculum). 6. Faculty. 7. Facilities. 8. Institutional Support. + Program Criteria. September 11,
11 ABET Criteria for Accrediting Eng. Programs 1. Students 2. Program Education objectives (PEOs). 3. Student Outcomes (SOs). We will call them Program Learning Outcomes 4. Continuous Improvement 5. Professional Component (curriculum). 6. Faculty. 7. Facilities. 8. Institutional Support. + Program Criteria. September 11,
12 NCAAA Standards for Accreditation and Quality Assurance 1. Mission and objectives (ABET C2) 2. Governance and administration (ABET C6 ) 3. Management of quality assurance and improvement (ABET C4) 4. Learning and teaching (ABET C3) 5. Student administration and support services (ABET C1) 6. Learning resources (Part of ABET C7) 7. Facilities and Equipment (Part of ABET C7) 8. Financial planning and management (ABET C8) 9. Employment processes (Part of ABET C6) 10. Research (Part of ABET C6) 11. Institutional relationships with the community (ABET C2)
13 NCAAA Standards for Accreditation and Quality Assurance 1. Mission and objectives (ABET C2) 2. Governance and administration (ABET C6 ) 3. Management of quality assurance and improvement (ABET C4) 4. Learning and teaching (ABET C3) 5. Student administration and support services (ABET C1) 6. Learning resources (Part of ABET C7) 7. Facilities and Equipment (Part of ABET C7) 8. Financial planning and management (ABET C8) 9. Employment processes (Part of ABET C6) 10. Research (Part of ABET C6) 11. Institutional relationships with the community (ABET C2)
14 NCAAA Domains of Learning Outcomes knowledge, the ability to recall, understand, and present information including: 1. knowledge of specific facts, 2. knowledge of concepts, principles and theories, and 3. knowledge of procedures. cognitive skills, the ability to apply understanding of concepts, principles, theories and procedures in critical thinking and creative problem solving, both when asked to do so and when faced with unanticipated new situations. September 11,
15 NCAAA Domains of Learning Outcomes Interpersonal skills and responsibility, the ability to: 1. take responsibility for their own learning and continuing personal and professional development, 2. work effectively in groups and exercise leadership when appropriate, 3. act responsibly in personal and professional relationships, and 4. act ethically and consistently with high moral standards in personal and public forums. Communication, information technology and numerical skills, including the ability to: 1. communicate effectively in oral and written form, 2. use information and communications technology, and 3. use basic mathematical and statistical techniques. Psychomotor skills involving manual dexterity (this domain applies only to some programs). September 11,
16 NCAAA Domains of Learning Versus ABET Outcomes Domain of Knowledge (Not existing in ABET): 1. Ability to recall, understand, and present knowledge of facts, concepts, principles and theories of Math and basic sciences. 2. Ability to recall, understand, and present knowledge of discipline-related practices and procedures. Domain of cognitive skills (corresponding to a, b, c, e, h, & j) 3. Ability to identify, formulate and solve real-life discipline related problems. 4. Ability to design, implement and evaluate a discipline related system or process to satisfy desired needs. 5. Ability to identify, analyze, and critically evaluate arguments and take informed criteria-based decisions.
17 NCAAA Domains of Learning Versus ABET Outcomes Domain of Interpersonal skills (corresponding to i, d, & f) 6. Ability to take responsibility for their own learning and continuing personal and professional development 7. Ability to work effectively in groups and exercise leadership when appropriate. 8. Ability to act responsibly in personal and professional relationships. 9. Ability to act ethically and consistently with high moral standards in personal and public forums.
18 NCAAA Domains of Learning Versus ABET Outcomes Domain of Communication, IT, and numerical skills (corresponding to a, g, & k) 10. Ability to communicate effectively in oral and written forms 11. Ability to use information and communications technology. 12. Ability to use basic mathematical and statistical techniques. Domain of psychomotor skills (not existing for engineering, IT, or managerial education)
19 Why Learning Outcomes? September 11,
20 Why Learning Outcomes? ABET Definition of Program Learning Outcomes Statements that describe what the students are expected to know and be able to do by the time of graduation. These relate to the knowledge, skills, and behaviors that the students acquire as they progress through the program. September 11,
21 Why learning outcomes? Course Learning Outcomes Each course should have a set of specific learning outcomes that lead to: More measurable student outcomes Better assessment Higher quality feedback Improved courses and programs Improved student learning and achievement September 11,
22 Infusion of Learning Outcomes into Courses Design Backward Intended Learning Outcomes of the Lesson Intended Learning Outcomes of the Unit Intended Learning Outcomes of the Course Program Outcomes September 11, 2014 Deliver Forward 21
23 Criterion 3 Program Learning Outcomes (a-k) a)an ability to apply knowledge of mathematics, science, and engineering, b)an ability to design and conduct experiments, as well as to analyze and interpret data, c) an ability to design a system, component, or process to meet desired needs within realistic constraints such as economic, environmental, social, political, ethical, health and safety, manufacturability, and sustainability, d)an ability to function on multi-disciplinary teams, e)an ability to identify, formulate, and solve engineering problems, September 11,
24 Criterion 3 Program Learning Outcomes (a-k) f) an understanding of professional and ethical responsibility, g) an ability to communicate effectively, h) the broad education necessary to understand the impact of engineering solutions in a global and societal context, i) a recognition of the need for, and an ability to engage in life-long learning, j) a knowledge of contemporary issues, and k) an ability to use the techniques, skills, and modern engineering tools necessary for engineering practice. September 11,
25 NCAAA and ABET Definitions for Learning and Teaching Program Educational Objectives: Broad statements that describe what graduates are expected to attain within a few years after graduation. Program educational objectives are based on the needs of the program s constituencies. Program Learning Outcomes: Describe what students are expected to know and be able to do by the time of graduation. These relate to the knowledge, skills, and behaviors that students acquire as they progress through the program. September 11,
26 NCAAA and ABET Definitions for Learning and Teaching Course Learning Outcomes: Describe what students are expected to know and be able to do by the end of the course delivery. These relate to the knowledge, skills, and behaviors that students acquire as they progress through the course. September 11,
27 NCAAA and ABET Definitions for Learning and Teaching (Cont d) Assessment : one or more processes that identify, collect, and prepare data to evaluate the attainment of student outcomes and program educational objectives. Effective assessment uses relevant direct, indirect, quantitative and qualitative measures as appropriate to the outcome or objective being measured. Appropriate sampling methods may be used as part of an assessment process. September 11,
28 NCAAA and ABET Definitions for Learning and Teaching (Cont d) Evaluation: one or more processes for interpreting the data and evidence accumulated through assessment processes. Evaluation determines the extent to which student outcomes and program educational objectives are being attained. Evaluation results in decisions and actions regarding program improvement September 11,
29 Mission, Objectives, Outcomes & Learning Experience Undergraduate Studies Upon Graduation Work environment Mission Learning Experience Curriculum & Other Activities Student Outcomes of the Program Program Objectives Mission Accomplished September 11,
30 Mission, Objectives, Outcomes & Learning Experience Undergraduate Studies Upon Graduation Work environment Mission Learning Experience Curriculum & Other Activities Program Learning Outcomes Program Objectives Mission Accomplished September 11,
31 Mission, Objectives, Outcomes & Learning Experience Undergraduate Studies Upon Graduation Work environment Mission Learning Experience Curriculum & Other Activities Program Learning Outcomes Program Objectives Mission Accomplished September 11,
32 Mission, Objectives, Outcomes & Learning Experience Undergraduate Studies Upon Graduation Work environment Mission Learning Experience Curriculum & Other Activities Program Learning Outcomes Program Objectives Mission Accomplished September 11,
33 Student Learning Experience September 11,
34 Student Learning Experience A path to achieve Program Learning Outcomes and foster the attainment of program educational objectives: 1. Curriculum Design (into courses and other learning activities) 2. Course Design 3. Teaching and Learning 4. Assessment (course-level and program-level) 5. Evaluation and continuous improvement
35 Teaching versus Learning Education has become outcome-based instead of being content-based. Educational activities became centered around learning instead of teaching. INPUTS OUTCOMES INPUTS Learning OUTCOMES
36 Teaching versus Learning Education has become outcome-based instead of being content-based. Educational activities became centered around learning instead of teaching. INPUTS OUTCOMES INPUTS Learning OUTCOMES
37 Session # 2 Domains of Learning Levels of Learning Writing Course Learning Outcomes September 11,
38 Domains of Learning September 11,
39 Domains of the Learning Outcomes The outcomes of any learning activity, or the change in the learner s behavior, may fall in one of 3 distinguished domains: Cognitive Domain that involves knowledge and the development of intellectual skills Affective Domain that includes the manner in which we deal with things emotionally, such as feelings, values, appreciation, enthusiasms, motivations, and attitudes. Psychomotor Domain that includes physical movement, coordination, and use of the motor-skill areas. Development of these skills requires practice and is measured in terms of speed, precision, distance, procedures, or techniques in execution. September 11,
40 Levels of Learning September 11,
41 Cognitive Domain (mental) Levels of Learning Synthesis (Creative Thinking) Evaluation (Critical Thinking) Analysis Application Comprehension Knowledge September 11,
42 Classical Terms Evaluation Synthesis Analysis Application Comprehension Knowledge Levels of Learning New Terms Creating Evaluating Analyzing Applying Understanding Remembering (Based on Pohl, 2000, Learning to Think, Thinking to Learn, p. 8) The names of six major categories were changed from noun to verb forms. September 11,
43 Affective Domain (Attitudes) Degrees of internalization This domain includes the manner in which we deal with things emotionally, such as feelings, values, appreciation, enthusiasms, motivations, and attitudes. 1. Receiving: Awareness. Willingness to hear. Selected Action Verbs : chooses, asks, replies, selects 2. Responding: Active participation. Attends to and demonstrates compliance, willingness or satisfaction in responding. Selected Action Verbs: complies, conforms, assists 3. Valuing: Is based on the internalization of a set of values, and are expressed in overt and identifiable behavior. Selected Action Verbs: Demonstrates, proposes, initiates, justifies 4. Organizing: Values are organized into priorities. The emphasis is on comparing, relating, and synthesizing values. Selected Action Verbs: arranges, integrates, synthesizes, defends 5. Internalizing (characterization): A value system that controls behavior. Behavior is pervasive, consistent, predictable, and characteristic of the values held. Selected Action Verbs : acts, proposes, discriminates.
44 Psychomotor Domain (levels of physical skills) The psychomotor domain includes physical movement, coordination, and use of the motor-skill areas. Development of these skills requires practice and is measured in terms of speed, precision, distance, procedures, or techniques in execution. 1. Set: Readiness to act Includes mental, physical and emotional sets. Selected Action Verbs : begins, displays, moves, proceeds 2. Guided Response: Early stages in learning a complex skill. Involves imitation and trial & error. Selected Action Verbs: copies, traces, follows, reproduces, responds 3. Mechanism: Intermediate skill in learning a complex skill. Includes habituation and proficiency. Selected Action Verbs: assembles, constructs, dismantles, displays 4. Complex Overt Response: Skillful performance that involves complex movement patterns. Proficient in speed and accuracy. Selected Action Verbs: Same as Mechanism, but will have adverbs that indicate performance is quicker, more accurate and automatic 5. Adaptation: Skills are well-developed and the individual can modify to fit special requirements. Selected Action Verbs: adapts, alters, revises 6. Origination: Creating new patterns to react to particular situations. Emphasis on creativity based on highly developed skills. Selected Action Verbs: Builds, combines, composes.
45 A Fluid Mechanics Example Level Explanation Sample Question Remembering Understanding Applying The student is expected to recite memorized information about the concept. The student is expected to explain the concept in his or her own words. The student is expected to apply the concept to a particular situation. What is a fluid? Why mercury is considered as a fluid? Calculate the position of the shock wave in the given nozzle Analyzing Evaluating Creating The student is expected to separate materials or concepts into component parts so that their organizational structure may be understood. The student is expected to make judgments about the value of ideas or materials. The student is expected to put parts together to form a whole, with emphasis on creating a new meaning or structure. Analyse the flow field inside the test section a high speed wind tunnel. Compare the given two nozzle designs that perform the same task, which one is better and why? Design a variable area nozzle to execute the following flight mission.
46 A Computer Programming Example Level Explanation Sample Question Remembering Understanding Applying Analyzing Evaluating Creating The student is expected to recite memorized information about the concept. The student is expected to explain the concept in his or her own words. The student is expected to apply the concept to a particular situation. The student is expected to separate materials or concepts into component parts so that their organizational structure may be understood. The student is expected to make judgments about the value of ideas or materials. What is a program? How is a program similar to a recipe? What is the output of the following program Identify the customer high level requirements based on the following customer interview. What additional questions you may need? Given two programs that perform the same task, which one is better and why? The student is expected to put parts together to Write a program to perform a form a whole, with emphasis on creating a new given task meaning or structure.
47 Writing Learning Outcomes September 11,
48 Writing Learning Outcomes While it is possible to write learning outcomes of all types for each of the three domains, the vast majority are written for the cognitive domain.
50 NCAAA Suggested Guidelines for Learning Outcome Verbs Learning Domains Knowledge Cognitive Skills September 11, 2014 Meaning List of basic knowledge that students should know and understand when they complete the course. List of the thinking and problem solving skills the course is intended to develop. The list should include the ability to: (1) Use analytic and predictive formulae as well as conceptual tools that students should use when asked to do so. (2) Identify and use the tools that are appropriate for new and unanticipated problems. Suggested Verbs list, name, record, define, label, outline, state, describe, recall, memorize, reproduce, recognize, record, tell and write. estimate, explain, summarize, write, compare, contrast, diagram, subdivide, differentiate, criticize, calculate, analyze, compose, develop, create, prepare, reconstruct, reorganize, summarize, explain, predict, justify, rate, evaluate, plan, design, measure, judge, justify, interpret and appraise. 49
51 NCAAA Suggested Guidelines for Learning Outcome Verbs Learning Domains Meaning Suggested Verbs Interpersonal Skills & Responsibility List of the outcomes related to students interpersonal skills, capacity for self directed learning, and personal and social responsibility. demonstrate, judge, choose, illustrate, modify, show, use, appraise, evaluate, justify, analyze, question, and write. Communication, Information Technology, Numerical List of outcomes related to students communication, IT and numerical skills. demonstrate, calculate, illustrate, interpret, research, question, operate, appraise, evaluate, assess, and criticize. September 11,
52 1 st Break (10 Minutes) September 11,
53 Session # 3 Elements of Course Design Step by Step Course Design & Assessment
54 Elements of Course Design
55 To ABET Program Committee Elements of Course Design Modified from Felder and Brent Student outcomes Levels of Learning Instructor's goals Lectures Institutional technologies Labs Course Goals & Course Intended Learning Outcomes Students Exams and Quizzes Homework Feedback from ABET Program Committee Project Instruction Assessment Plus & Delta Syllabus Student expectations Learning Methods: Problem-Based, Project-based, Active Cooperative Student Portfolio Process Check Surveys
56 STEP BY STEP COURSE DESIGN & Assessment
57 STEP BY STEP COURSE DESIGN & Assessment 1. Determine Program Core Courses 2. Apply Course Design Methodology for each Core Course: Define the goals of the course Define the Course Intended Learning Outcomes Construct the Course Mapping Matrix Construct the Course Articulation Matrix Prepare the course syllabus 3. Construct Mapping Matrix for all core courses 4. Apply Course Assessment Methodology for each Core Course 5. Evaluate Course Assessment results
58 Program Core Courses Program core courses: a set of courses designated to address some or all of the Program outcomes. Compulsory courses in the Program would be obvious candidates for the core. Compulsory courses given by other Programs, such as math & physics might be included as long as they consistently address outcomes. Elective courses or courses whose content varies from one offering to another can't be included.
59 Step #1: Define the goal or goals of the course Two to three lines : The course is intended to. Example (Data Base Management Systems): Example: The course is intended to: Develop the student s ability to understand, design, and implement Data Base Management tools.
60 Step #2: Define The Course Intended Learning Outcomes Intended Learning Outcomes: specific observable (measurable) actions that the students should be able to perform if they mastered the course. i.e. by the completion of this course, the students should be able to: The first word must be one action verb from the process descriptive verbs related to LOL (see the first part of this workshop)
61 Step #2: Define The Course Intended Learning Outcomes Look at the items or concepts in detailed course content. For each item or concept decide what are the actions or tasks you want the students to do in order to demonstrate their level of mastery. Also think of how you will measure these actions (i.e. your assessment tools) and write down a suitable task statement. Each task statement includes one key action verb [such as list, explain, calculate, estimate, derive, model, design, choose, and criticize] along with a definition of the task and possibly a specification of the conditions under which the task is to be performed.
62 Step #2: Define The Course Intended Learning Outcomes Do you want the students to define terms, explain them, apply them to known situations, to new situations, or use them to design a new artifact or process? Think of the levels of learning and their associated action verbs and associate a targeted level of learning to each course learning outcome. The best choice is to have a maximum of 9 to 12 course learning outcomes (one to 2 per course topic). The more they are detailed, the easier you could design their assessment, but you may find it difficult to have sufficient assessment tools.
63 Step #2: Define The Course Intended LO (EXAMPLE) Course Title: Introduction to Database Systems Current Catalog Description: Introduction to the database management systems, database administration, data processing, data modeling, database design, development, and implementation. Contrasts alternative modeling approaches. Includes implementation of current DBMS tools and SQL. Major Topics Covered in the Course: Data modeling Logical database design Physical database design (including performance issues) SQL and other database access methods Database Administration (including Data Warehousing and Administration)
64 Step #2: Define The Course Intended Learning Outcomes (Good Examples) Data modeling 1. Model data relevant to a database task, given written description, reports and other information from a systems user. 2. Model data using entity-relationship as well as object-oriented methods. Logical database design 3. Transform entity-relationship model into a logical design, following the relational approach. Physical database design (including performance issues) 4. Assess aspects of the physical design, and transform the logical design accordingly.
65 Step #2: Define The Course Intended Learning Outcomes (Bad Examples) Understand the fundamentals of data base design (not measurable) Students demonstrate that they can derive and properly apply the equations of. (redundant words- what is meant by properly apply? In which situation) Statements of non-observable actions (such as learn, know, understand, or appreciate ) might qualify as educational objectives BUT NOT Intended learning Outcomes Understanding cannot be directly observed; the student must do something observable to demonstrate his understanding.
66 Session # 3 Hands-on Exercise Forming work teams Define the course goals Define a set of Intended course learning outcomes Presenting
67 Forming Teams September 11,
68 Forming work Teams Now we will form teams of 3 members each. All members of each team belong to one program. Each team will select one core course that falls in a domain of their common interest. September 11,
69 Define the Course Goals September 11,
70 Step#1: Define the goal or goals of the course (10 minutes) Use the catalogue description of the course to define its goals from the instructors view point Take 5 minutes to write 2-3 lines : The course is intended to. Remember the following example (Data Base Management Systems): The course is intended to: Develop the student s ability to understand, Design, and implement Data Base Management tools. September 11,
71 Define a set of Intended course learning outcomes September 11,
72 Step#2 Define the Course Intended Learning Outcomes (30 minutes) take 20 minutes to prepare 4 to 5 intended course learning outcomes (1 to 2 CLOs per major topic). Notice that each CLO should start by an action verb Define the level of learning for each CLO. Map each CLO to one or more of ABET Program Learning Outcomes. Follow the following examples: Physical database design: Assess aspects of the physical design, and transform the logical design accordingly. (Evaluating, level #3, outcome c). SQL and other database access methods Model data relevant to a database task, given written description from a systems user. (Analyzing, level #2, outcome c). September 11,
73 Empty Course Articulation Matrix Program LO September 11,
74 Presenting Your Work Now one team will come to podium. Take 5 minutes to present September 11,
75 Presenting September 11,
76 2 nd Break (30 Minutes) September 11,
77 Session # 4 Learning Depth and Breadth Apply Course Design Methodology for each Core Course Step #3. Construct the Course Mapping Matrix Step #4. Construct the Course Articulation Matrix Documenting Course Design Data September 11,
78 Learning Depth and Breadth September 11,
79 Learning Depth and Breadth 1. The Depth of Learning is measured by level of learning in the cognitive domain 2. The breadth of learning is measured by the number of contact hours allocated to the learning activities targeting an outcome. 3. The 6 levels of learning are grouped into 3 levels of learning depth: Level #1: low level of learning: Knowledge and Comprehension or Remembering and Understanding (weight = 1). Level #2: Medium level of learning: Application and Analysis or Applying and Analyzing (weight = 2). Level #3: High level of learning: Synthesis and Evaluation or Evaluating and Creating (weight = 3). September 11,
80 Student s Exposure 1. The product of depth of learning of a given student outcome by the corresponding breadth in a course indicates the student s exposure to this outcome in that course as compared to other courses addressing the same student outcome. 2. Student outcomes that have high exposure in a given course are considered as the critical outcomes of that course and the course is considered as a key course for these outcomes. 3. Add a column to your articulation matrix containing the CLO coverage hours. 4. Exposure of an outcome = SUM of (the Product of the alignment coefficient of the outcome with each CLO by CLO coverage). September 11,
81 Apply Course Design Methodology for each Core Course September 11,
82 STEP BY STEP COURSE DESIGN & Assessment 1. Determine Program Core Courses 2. Apply Course Design Methodology for each Core Course: Define the goals of the course Define the Course Intended Learning Outcomes Construct the Course Mapping Matrix Construct the Course Articulation Matrix Prepare the course syllabus 3. Construct Program Mapping Matrix for all core courses 4. Apply Course Assessment Methodology for each Core Course 5. Evaluate Course Assessment results September 11,
83 Step #3: Construct the course mapping matrix Course Learning Outcomes Program Learning Outcomes IE 301 LOL a b c d e f g h i j k l m n CLO_1 CLO_2 CLO_ Maximum Attainable : low levels of learning in Bloom s Taxonomy (Remembering and understanding) 2: medium levels of learning in Bloom s Taxonomy (Applying and Analyzing) 2: high levels of learning in Bloom s Taxonomy (evaluating and creating) September 11,
84 Step #4: Construct the course mapping matrix (Continued) Some schools fill the matrix with an indicator to demonstrate to which degree (High, medium, low) this course learning outcome (CLO) contributes to the achievement of the corresponding student outcome (using letters such as H, M, and L). Other schools indicate the corresponding level of learning (LOL). A simplest method is to use a binary system (ONE if the Course LO addresses the Program LO and ZERO if it does not). September 11,
85 Useful approach: Course Articulation Matrix Program LO September 11,
86 Empty Course Articulation Matrix Program LO September 11,
87 Documenting Course Design Data September 11,
88 Documenting Course Design Data A course articulation matrix assigns 1-2 Intended Course Learning Outcome (CLOs) to each topic and competency (a total of 9 to 12 CLOs per course). Each CLO is associated with a certain number of hours of exposure (contact hours allocated to this CLO). In-class & out-of-class learning activities are indicated for each CLO. Classical assessment tools are indicated for each CLO. CLOs are mapped into Student Learning Outcomes (a-k). Key outcomes are identified as those having the highest exposure.
89 Session # 4 Hands-on Exercise
90 Construct the Course Mapping Matrix (20 minutes) September 11,
91 Empty Course Articulation Matrix Program LO September 11,
92 Session # 5 Course Assessment & Course Binder
93 Course Assessment
94 Why Assessment Helps students understand where they should focus their time & energies Motivates students to do their best Helps students understand their strengths and weaknesses through feedback Gives students documentation of what they have learned which can be used to apply for jobs, awards, programs of advanced study, etc.
95 What Is Assessment? ABET Definition One or more processes that identify, collect, and prepare data to evaluate the attainment of student outcomes and program educational objectives. Effective assessment uses relevant direct, indirect, quantitative and qualitative measures as appropriate to the outcome or objective being measured. Appropriate sampling methods may be used as part of an assessment process.
96 Embedding Assessment With Regular Reflection Begin Section 1 Course Section 1 Teaching & Learning Assessment of Section 1 Topics Course Section 2 Teaching & Learning Begin Section 2 Evaluation of Achievement of Section 1 CLOs and Related SLOs Using HI-CLASS Revisions of Course Plan Embedding Assessment with Regular Reflection Approach Normal teaching Approach
97 Direct Assessment Methods (Clear and Compelling Evidence) Capstone experiences (research projects, presentations, theses, oral defenses, exhibitions, or performances) scored using a rubric Other written work, performances, or presentations, scored using rubrics Portfolios of student work Scores on locally-designed multiple choice and/or essay tests (e.g. final examinations in key courses, qualifying exams, comprehensive exams) accompanied by test blueprints describing what the tests assess Score gains between entry and exit on published or local tests or writing samples Summaries/analyses of electronic discussion threads Student reflections on their values, attitudes and beliefs, if developing those are intended outcomes of the course
98 Indirect Assessment Methods (Signs that Students Are Probably Learning, But Exactly What or How Much They Are Learning is Less Clear) Course grades (see next slide) Assignment grades, if not accompanied by a rubric or scoring guide (See an Example) Student ratings of their knowledge and skills and reflections on what they have learned in the course or program Questions on end-of-course student evaluation forms that ask about the course rather than the instructor
99 Assessment and Grading They do have one common characteristic: both intend to identify what the students have learned. Grades alone do not always give direct evidence: they don t identify which specific student learning outcomes and at what levels students have learned. Some course grades include additional student behaviors that are not related to student learning outcomes (e.g. attendance and participation). For example, a grade of B in a mathematics course indicates a student learned an acceptable level of math, but not the specific math knowledge or skills that the student mastered well or what areas need improvement. A project or assignment grade would be a better link to the evidence of student learning, but a rubric for the project or assignment would provide the best direct evidence and the level at which the student learned. Grades are based on direct evidence of student learning such as the evaluations of tests, papers, and projects, but need to be clearly linked and aligned to learning outcomes and/or rubrics to suffice as direct evidence for assessment purposes. September 11,
100 Proposed Course Level Assessment Tools Each semester Indirect measures: Course Learning Outcomes (CLOs) questionnaire (exit and sometimes entry) Course in-class and out-of-class learning activities questionnaire (exit) Student Outcomes (SOs) questionnaire (exit) Direct measures: Course Exams, HWs, Design projects, Written reports, Oral presentations, Course portfolios, Lab Reports evaluated using rubrics or mapped into CLOs Assignments for key SOs evaluated using rubrics.
101 Course Level Indirect Assessment Course Learning Outcomes (CLOs) Questionnaire: Put your course learning outcomes in the form of a questionnaire to measure the students confidence level in mastering the course learning outcomes. Use the 1 to 5 Eckert scale to cover Excellent-Medium-to-Poor mastery levels. Add some questions to evaluate the importance of course prerequisites. Distribute the questionnaire twice during the semester: in the 2 nd week (entry, if you admit this approach) and/or in the 12 th week (exit) of the course. Answer the questionnaire yourself to indicate your estimation of the students average level of mastery. (see an example)
102 Course Level Indirect Assessment (continued) Course learning activities (or instructional tools) Questionnaire: Ask the students to evaluate the impact of different inclass and out-of-class learning activities on their level of mastery of course learning outcomes. Use the 1 to 5 Eckert scale to cover Excellent-Medium-to-Poor impact. Distribute the questionnaire in 12 th week. Answer the questionnaire yourself to indicate your estimation of the impact of the learning activities. (see an example)
103 Course Level Indirect Assessment (continued) Student Outcomes Questionnaire: ABET Committee should divide each outcome into outcome elements and to define outcome attributes (measures) or KPIs associated with these outcome elements. A questionnaire should be developed for each outcome and used by the instructor who pretends that his course addresses this outcome. The questionnaire is to be distributed during the 12 th week of the semester. (see an example of outcome attributes/indicators and outcome Questionnaire)
104 Course Level Direct Assessment Direct Assessment of: HI-CLASS or Similar Assessment of Course Learning Outcomes (CLOs) Direct Approach Direct Rubric-Based Assessment of Key Student Outcomes.
105 Construct the Articulation Matrix (15 minutes) Now let us fill the Learning Activities and Assessment tools into the Matrix
107 THANK YOU
108 ABET & NCAAA ACCREDITATION Prof. Ali M. Al-Bahi Director of the Academic Accreditation Unit Faculty of Engineering, King Abdulaziz University Modified and presented by Dr. Faisal Iskanderani UBTVR for Q&D Sept 11, 2014 September 11,
Outcome-Based Education Outcome based education A student-centred learning system focusing on measuring student performance. Does not specify any specific style of teaching just that the student be able
TAXONOMY OF EDUCATIONAL OBJECTIVES (Excerpts from Linn and Miller Measurement and Assessment in Teaching, 9 th ed) Table 1 Major categories in the cognitive domain of the taxonomy of educational objectives
IACBE Advancing Academic Quality in Business Education Worldwide Bloom s Taxonomy of Educational Objectives and Writing Intended Learning Outcomes Statements International Assembly for Collegiate Business
SLO Action Verbs Action verbs are abundant in the English language, but how to do we know which ones are right to include in our SLO statements? Benjamin Bloom, an American educational psychologist, created
What is a learning outcome? A learning outcome states what a student should be able to do or know as a result of having completed a particular course or program. Learning outcomes can include knowledge,
Resources for Writing Program Learning Outcomes Supplementary Materials for Writing and Revising Learning Outcomes Workshop Presented Jointly by TLA and Student Affairs LEVEL OF SPECIFICITY AND REACH Learning
Writing learning objectives This material was excerpted and adapted from the following web site: http://www.utexas.edu/academic/diia/assessment/iar/students/plan/objectives/ What is a learning objective?
HONORS INDEPENDENT STUDY GUIDELINES: According to the Lake Land College catalog course description, independent study is designed to permit the student to pursue a course of study not typically available
Learning Domains Bloom's Taxonomy The Three Types of Learning There is more than one type of learning. A committee of colleges, led by Benjamin Bloom, identified three domains of educational activities:
A SYSTEMATIC APPROACH FOR DEFINING AND ASSESSING PROGRAM EDUCATIONAL OBJECTIVES AND OUTCOMES Nikos J. Mourtos 1 Abstract The USA Accreditation Board for Engineering and Technology adopted recently a new
OBE IMPLEMENTATION GUIDEBOOK FACULTY OF ELECTRICAL & ELECTRONIC ENGINEERING First Edition 2010 Second Edition 2012 Third Edition 2014 Prepared by: OBE Committee, FKEE Contents FOREWORD... ii 1. Outcome
Learning Outcomes Workbook Student Services How to write Learning Outcomes Academic Affairs July 2012 Overview of the Learning Outcomes Workbook This workbook will serve as your introduction to the concept
Industrial Engineering Definition of Tuning Tuning is a faculty-led pilot project designed to define what students must know, understand, and be able to demonstrate after completing a degree in a specific
Educational Goals and Objectives A GUIDE TO DEVELOPING LEARNER BASED INSTRUCTION Educational Objectives for this Presentation At the end of this presentation you will be able to: Compare and contrast educational
SLOs, Bloom s Taxonomy, Cognitive, Psychomotor, and Affective Domains. Benjamin Bloom (1948) developed classifications of intellectual behavior and learning in order to identify and measure progressively
Writing Learning Outcomes What is a learning outcome? Learning outcomes are statements that indicate what learners will know or be able to do at the completion of a learning experience (program, conference,
COMMENTARY For Architectural Engineering Programs Draft of September 2010 Purpose This document has been prepared by the ASCE Committee on Curriculum and Accreditation (CC&A). The purpose of this document
Page 1 of 32 STANDARDS AND CRITERIA FOR ACCREDITATION OF POSTSECONDARY CONSTRUCTION EDUCATION DEGREE PROGRAMS TABLE OF CONTENTS 1. INTRODUCTION 2. GOVERNANCE AND ADMINISTRATION 3. CURRICULUM 4. FACULTY
Framework for the Doctoral Transformation: Application of a Taxonomy of Educational Objectives The doctoral learning process is life changing. It begins with the decision to seek the doctorate and becomes
ASU College of Education Course Syllabus ED 4972, ED 4973, ED 4974, ED 4975 or EDG 5660 Clinical Teaching Course: ED 4972, ED 4973, ED 4974, ED 4975 or EDG 5660 Credit: 9 Semester Credit Hours (Undergraduate),
ASSESSMENT OF STUDENT LEARNING OUTCOMES IN THE MAJOR A PRIMER FOR DEPARTMENT ASSESSMENT LIAISONS I. WHAT IS ASSESSMENT? Assessment is the systematic collection, review and use of information about educational
Bloom s Taxonomy, Educational Objectives, Outcomes, and our Friends from ABET - An Engineering Case Study- Information collected by Steve Barrett Overview Bloom s Taxonomy Some background information Relationship
Writing Measurable Learning Outcomes Sandi Osters, Director of Student Life Studies F. Simone Tiu, Assistant Director for Institutional Effectiveness 3 rd Annual Texas A&M Assessment Conference You got
USING THE PRINCIPLES OF ITIL ; SERVICE CATALOGUE Examination Syllabus V 1. October 009 ITIL is a Registered Trade Mark of the Office of Government Commerce in the United Kingdom and other countries APMG
Looking at the First Year Objectively Stephen C. Zerwas University of North Carolina Greensboro 1 Measuring Student Learning What do we want students learn? Do students have the opportunity to learn it?
CREATING LEARNING OUTCOMES What Are Student Learning Outcomes? Learning outcomes are statements of the knowledge, skills and abilities individual students should possess and can demonstrate upon completion
Boise State University Department of Construction Management 2013-2014 Quality Assessment Report and Action Plan November 25, 2014 Executive Summary The Construction Management program is accredited by
Getting Started A working definition of a performance-based objectives: A Learning objective is a statement that describes the specific skills or knowledge a student will be able to demonstrate as a result
How to Write Learning Outcomes This document is a work in progress. As faculty utilize these guidelines, they will make suggestions for improvement. Changes will follow. What are learning outcomes? Learning
Assessment Processes Department of Electrical and Computer Engineering Fall 2014 Introduction The assessment process in the Electrical and Computer Engineering (ECE) Department at Utah State University
ELECTRICAL ENGINEERING GOALS AND OBJECTIVES AND ASSESSMENT PLAN Mission Statement: The mission of the Electrical Engineering Department is to provide a quality electrical engineering education with extensive
Plans for Computer Engineering Programs Spring 2015 1 Introduction The CpE B.S and M.S. degree programs at California State University, Sacramento are joint programs supported by both the Computer Science
OREGON INSTITUTE OF TECHNOLOGY Mechanical Engineering Program Assessment 2007-08 October 16, 2008 INTRODUCTION The Mechanical Engineering Program within the Mechanical and Manufacturing Engineering and
National Commission for Academic Accreditation & Assessment Handbook for Quality Assurance and Accreditation in Saudi Arabia PART 1 THE SYSTEM FOR QUALITY ASSURANCE AND ACCREDITATION Ver. 2.0 THE SYSTEM
Weldon School of Biomedical Engineering Continuous Improvement Guide The intent of this document is to assist faculty, staff, students, and constituents of the Weldon School of Biomedical Engineering in
PROFESSIONAL QUALIFICATION SCHEME INTERMEDIATE QUALIFICATION SERVICE LIFECYCLE CONTINUAL SERVICE IMPROVEMENT CERTIFICATE SYLLABUS Page 2 of 18 Document owner The Official ITIL Accreditor Contents CONTINUAL
Preparing the Self-Study Report for Engineering Jeffrey W. Fergus Auburn University Member of ABET EAC Executive Committee 2015 ABET Symposium April 23-24, 2015 Disclaimer The information presented here
Mechanical Engineering Technology 2011-12 Assessment Report I. Introduction The Bachelor of Science program in Mechanical Engineering Technology is offered in three locations Klamath Falls, Portland Metro
Writing Student Learning Outcomes for an Academic Program Student learning outcomes (SLOs) are statements of the knowledge, skills, competencies, or attitudes that students are expected to gain through
Feedback for the 2012-2013 Annual Assessment Report Department of Speech Pathology and Audiology Amy Liu, Ph.D. Director, Office of Academic Program Assessment (OAPA) Professor of Sociology Shannon Datwyler,
Degree- Level Expectations and Course Learning Outcomes Introduction to DLEs Degree-Level Expectations (DLEs) are a threshold framework for the expression of the intellectual and creative development of
I - T E C H T E C H N I C A L I M P L E M E N T A T I O N G U I D E # 4 Writing Good Learning Objectives I-TECH s Technical Implementation Guides are a series of practical and instructional papers designed
BS Environmental Science (2013-2014) Program Information Point of Contact Brian M. Morgan (email@example.com) Support for University and College Missions Marshall University is a multi-campus public
IACBE Advancing Academic Quality in Business Education Worldwide Key Content Areas of an Outcomes Assessment Plan International Assembly for Collegiate Business Education 11374 Strang Line Road Lenexa,
Industrial Technology (BS/IT & MS/IT progs.) SECTION IV DRAFT (July 6, 2011) Section IV Assessment: Undergraduate and Graduate Student Learning Outcomes (SLOs) courses identified in the department s assessment
PROFESSIONAL QUALIFICATION SCHEME INTERMEDIATE QUALIFICATION SERVICE LIFECYCLE SERVICE DESIGN CERTIFICATE SYLLABUS Page 2 of 18 Contents SERVICE DESIGN CERTIFICATE 4 Target Candidate 4 Prerequisite Entry
PROFESSIONAL QUALIFICATION SCHEME INTERMEDIATE QUALIFICATION SERVICE CAPABILITY SERVICE OFFERINGS AND AGREEMENTS CERTIFICATE SYLLABUS Page 2 of 23 Document owner The Official ITIL Accreditor Contents SERVICE
COURSE COMPETENCY GUIDELINES GUIDELINES FOR DEVELOPING COMPETENCIES Format: 1. Begin with a present tense action verb. (Example: Convert picas to points and inches.) 2. Each action verb requires an object.
DEFINING, TEACHING AND ASSESSING LIFELONG LEARNING SKILLS Nikos J. Mourtos Abstract - Lifelong learning skills have always been important in any education and work setting. However, ABET EC recently put
Draft Policy on Graduate Education Preface/Introduction Over the past two decades, the number and types of graduate programs have increased dramatically. In particular, the development of clinical master
PROFESSIONAL QUALIFICATION SCHEME INTERMEDIATE QUALIFICATION SERVICE CAPABILITY PLANNING, PROTECTION AND OPTIMIZATION CERTIFICATE SYLLABUS The Swirl logo is a trade mark of the Cabinet Office ITIL is a
Drafting Program Educational Objectives for Undergraduate Engineering Degree Programs Ramakrishnan Sundaram Gannon University, firstname.lastname@example.org Abstract - This paper outlines the process to draft Program
Developing Data Workshops BDK 31-01 Best practices for engaging audiences and using the BDK resources Jackie Wirz, PhD Oregon Health & Science University BD2K Open Educational Resources Oregon Health &
This fact sheet provides the following: Background Asse ssment Fact Sheet: Performance Asse ssment using Competency Assessment Tools background information about the development and design of the competency
Bloom s Taxonomy Each of the three categories requires learners to use different sets of mental processing to achieve stated outcomes within a learning situation. Benjamin Bloom (1913-1999) was an educational
Instructional Design What Is It and Why Should I Care? A CEIT Workshop Domenic Screnci, Ed.D. Executive Director Educational Technology, Training and Outreach Information Services and Technology Boston
Mission Statement The Department of Electrical and Computer Engineering has the following mission statement, as presented on the department website (http://www.engr.uky.edu/ece/general/missionvision.html).
PsyD Psychology (2014 2015) Program Information Point of Contact Marianna Linz (email@example.com) Support for University and College Missions Marshall University is a multi campus public university providing
3rd International Conference on Electrical & Computer Engineering ICECE 2004, 28-30 December 2004, Dhaka, Bangladesh IMPROVING ENGINEERING EDUCATION Muhammad Harunur Rashid Professor and Director Department
University of the District of Columbia School of Engineering and Applied Sciences Continuous Improvement Process And ABET Outcomes Assessment Protocol BASED ON ABET 2008-2009 Criteria April 2008 School
Page 1 of 12 UW-Madison Plan for the Assessment of Student Learning JANUARY 2015 Page 2 of 12 This document describes a strategic and systematic process of improving the quality of degree and co-curricular
Document Control Information Document Details Document Name Purpose of Document Document Version Number 5.4 Document Status Document Owner Prepared By The ITIL Intermediate Qualification Service Design
2005 ABET Program Outcome and Assessment 3. Program Outcomes and Assessment Program Outcomes We define our program educational outcomes as statements that describe what students are expected to be able
Bachelor of Science in Information Technology FY 2015 Enrollment: 693; Graduate: 31 Program Educational Objectives Program Educational Objectives (PEOs) are broad statements that describe what graduates
GEORGIA STANDARDS FOR THE APPROVAL OF PROFESSIONAL EDUCATION UNITS AND EDUCATOR PREPARATION PROGRAMS (Effective 9/01/08) Kelly Henson Executive Secretary Table of Contents Standard 1: Candidate Knowledge,
Degree Program Student Learning Report (rev. 7/14) Fall 2013 Spring 2014 The Department of Applied Technology in the School of Business & Technology Business Information Technology, B.S. Effectively assessing
B2. PROGRAM EDUCATIONAL OBJECTIVES B2.1 Mission Statements The fundamental mission of UW-Platteville and the entire UW System is to serve the people of Wisconsin. This basic goal is expressed in detail
PROFESSIONAL QUALIFICATION SCHEME INTERMEDIATE QUALIFICATION SERVICE LIFECYCLE SERVICE OPERATION CERTIFICATE SYLLABUS Document owner The Official ITIL Accreditor Page 2 of 18 Contents SERVICE OPERATION
PROFESSIONAL QUALIFICATION SCHEME INTERMEDIATE QUALIFICATION SERVICE CAPABILITY OPERATIONAL SUPPORT AND ANALYSIS CERTIFICATE SYLLABUS Page 2 of 21 Document owner The Official ITIL Accreditor Contents OPERATIONAL
Undergraduate Psychology Major Learning Goals and Outcomes i Goal 1: Knowledge Base of Psychology Demonstrate familiarity with the major concepts, theoretical perspectives, empirical findings, and historical
Council for Standards in Human Service Education National Standards ASSOCIATE DEGREE IN HUMAN SERVICES http://www.cshse.org 2013 (2010, 1980, 2005, 2009) I. GENERAL PROGRAM CHARACTERISTICS A. Institutional
An introductory guide to AACSB AoL (Assurance of Learning) at SDA Bocconi M.Brusoni P.Egi March 2012 Index SDA Bocconi AACSB Accreditation Process Assurance of Learning - AoL Process SDA Bocconi Learning
ABET Criterion 3: Outcomes Met By Course Content This brief content assessment should be consistent with the updated Course Syllabet Course #: BEE 200: Title: The BEE experience (1 credit) Semester/Year:
Student Outcomes Assessment Plan (SOAP) I. Mission Statement Department of Political Science College of Social Science Undergraduate Bachelor s Degree in Political Science The Department of Political Science
Building Critical Thinking Skills in General Education and Career Programs Wayne County Community College District Presented By: Mary Mahoney Jo Ann Allen Nyquist College Definition of Critical Thinking
Unit Plan for Assessing and Improving Student Learning in Degree Programs Unit: Civil and Environmental Engineering (CEE) Unit Head Approval: Prof. R.H. Dodds Date: SECTION 1: PAST ASSESSMENT RESULTS Brief
SCHOOL OF ENGINEERING Baccalaureate Study in Engineering Goals and Assessment of Student Learning Outcomes Overall Description of the School of Engineering The School of Engineering offers bachelor s degree
Developing a Department-Wide Learning Assessment Program Timothy L. Skvarenina 1 Session T1A Abstract - During the last several years, the North Central Association of Colleges and Schools (NCA) shifted
The University of Tennessee, Memphis Writing Learning Objectives A Teaching Resource Document from the Office of the Vice Chancellor for Planning and Prepared by Raoul A. Arreola, Ph.D. Portions of this
Your consent to our cookies if you continue to use this website.