Sixth Year Maintenance of Accreditation Report

Transcription

1 COLLEGE OF BUSINESS AND ECONOMICS Sixth Year Maintenance of Accreditation Report VOLUME 2 Prepared for AACSB International October 2013

2 Volume 2: Assurance of Learning Design & Implementation High Impact Research & Engagement Center Prepared by Sandy Luong 2013 Faculty Advisor Dr. Glen Taylor

3 Table of Contents Page Part 1: Introduction 1.1 Fifth Year Review Team Issues to be Addressed Part 2: System Improvements Part 2 System Changes Overview New Committees and Positions Formed Roles & Responsibilities Systematically Defined Synchronize the Assessment Language Systematize Rubric Review, Modification & Implementation Systematize Closing the Loop New Rubrics Adopted & Plan for Future Customization Five-Year Assessment Plan by Annual Cycles Curriculum Mapping Website Development & Assessment Transparency Blackboard Pilot Part 3: Faculty Engagement: Building a Culture of Assessment Part 3 Faculty Engagement/Culture of Assessment Overview Collecting Better Data with a Purpose Grading vs. Assessment Faculty Involvement. 52 Part 4: Closing the Loop Initiatives Part 4 Closing the Loop Initiatives Overview Oral Communication Workshop for Learning Goal: Communication MBA Boot Camp & MBA Guest Talks for Learning Goal: Communication Capsim Simulation for Learning Goal: Functional Integration Pre-requisite/Core Course for Learning Goal: Ethics Part 5: Spring 2013 Assessment Data & Results Spring 2013 Assessment Write-Up BSBA Learning Goal BSBA Learning Goal BSBA Learning Goal Spring 2013 MBA Assessment Write-Up MBA Learning Goal Spring 2013 Assessment Minutes Part 6: Meeting Material from AoL Redesign Efforts Part 2 to 4 Meeting Minutes & PowerPoint Presentations P age

4 Part 1: Introduction 3 P age

5 1.1 Fifth Year Review Team Issues to be Addressed Following the 2012 Five-year AACSB Accreditation Review visit, the visiting team identified key areas needing improvement as explained in the Maintenance of Accreditation Review report. One area identified by the review team was AACSB Standard 15, Management of Curricula. In the Team Visit Report Maintenance of Accreditation Review, the AACSB Review Team stated the College of Business & Economics (CBE) Assurance of Learning system needs further maturation, with clear roles and responsibilities, an enhancement of the quality of the data we collect, more useful analysis of the data, and evidence that the data is being used effectively to close the loop. Three main areas were recognized by CBE as key areas of weaknesses identified by the Review Team. These areas were: (1) weaknesses in our Assurance of Learning system with unclear roles and responsibilities; (2) a low level of faculty engagement; and (3) no systematic effort to follow through on assessment and take closing the loop actions. More specifically, the following was excerpted from the Team Visit Report: Although the College has been involved with the Assurance of Learning (AOL) for some time, significant progress was not noted until Prior to then, indirect measures using a variety of surveys of students, alumni and employers were the principal method used. Rubrics were developed in 2010 to provide direct assessment of the learning goals in all of the programs. The first learning goals were directly assessed in 2011 and a report generated. A second round of direct assessment occurred in Although the results of the 2011 assessment were provided to the Curriculum Committee, the process for analysis of the data and the process to close the loop are not fully developed or implemented. At this time a final report from the 2012 assessment has not been completed. The dean did provide grant funding for Instructional Improvement in the summers of 2011 ($65,500) and 2012 ($96,000) to address the assessment results. However, there is a concern that the improvements may affect individual courses taught by individual faculty members rather than provide improvements across all sections of strategically identified courses. The College has a Director of Assessment who compiles the data and provides a report, but it is unclear as to the roles and responsibilities of the faculty, especially the Curriculum Committee, and the department chairs and the process to be followed to make changes and close the loop. The Assessment Committee formed in 2010 has not met for two years. A comprehensive document listing the learning goals for each program, a course mapping and schedule for implementation should be developed. There is a concern that the assessment at the undergraduate level might occur at a sophomore level---too early in the program since the students are not officially enrolled in the CBE until their junior year. In addition, since the majority of their undergraduate students are community college transfers, this entire segment would not be assessed. This report presents the changes CBE has undergone since the AACSB Team Visit Report to demonstrate that the major concerns raised by the Team have been addressed and improvement efforts can be sustained. 4 P age

6 Part 2: System Improvements 5 P age

7 Part 2: System Changes Overview In response to the AACSB review, we began by addressing weaknesses in our AoL system. An AoL system provides the overall framework that guides assessment processes and procedures. In the past, our AoL system emphasized extensive collection of data with insufficient regard to the quality of the data and insufficient follow-through. Faculty was not invested in assessment process and went through the motions of collecting data with low expectations that any improvements would occur. In redesigning the AoL system, we used a clean sheet approach and designed a new system from the ground up. We turned to faculty to drive the process every step of the way. Our goal was to change assessment from an administrative task enacted by decree to a faculty driven process aimed at continuously improving program effectiveness and learning outcomes for our students. To steer the redesign process, we seated a new Assurance of Learning (AoL) Task Force to facilitate the process (see 2.1 New Committees and Positions Formed). The AoL task force defined the roles and responsibilities associated with the assurance of learning system and publicized the roles and responsibilities in committee meetings for transparency throughout the college (see 2.2 Roles & Responsibilities Systematically Defined). Next, we addressed the confusion in CBE that arose from our inconsistent and often confusing use of assessment terminology. We were speaking in several different languages at the same time. Historically, CBE used language chosen to address our regional accrediting body, WASC. As other people were brought into the assessment process, other terms were used from different accrediting bodies or out of personal preference. As a result, assessment discussions were confusing for many involved. It was determined that there was a need to provide clarity (see 2.3 Synchronize the Assessment Language). Once we were all speaking the same language and everyone involved had clearly defined responsibilities, we addressed the weaknesses in our assessment process. We needed to refine our use of assessment tools and procedures as well as systematize the assurance of learning process as a whole (see 2.4 Systematize Rubric Review, Modification & Implementation; 2.5 Systematize Closing the Loop; 2.6 New Rubrics Adopted & Plan for Future Customization; 2.7 Five-Year Assessment Plan by Annual Cycles; 2.8 Curriculum Mapping). After making major changes to the processes and procedures, we needed to make these changes transparent to the faculty to build a shared understanding of the overall system (see 2.9 Website Development & Assessment Transparency). This resulted in the creation of a website to facilitate that transparency and to provide a single resource for documents and information related to assessment. The website provides a space where faculty and students go to learn about the data collected, what is being done to address program weaknesses, and how we are closing the loop to improve our educational effectiveness. CBE also participates in assurance of learning practices at the university level. As we went through our redesign process, CBE became recognized within CSUEB for its efforts to develop a strong system of assessment and was asked to work at the university level in a Blackboard pilot program. This consisted of integrating assurance of learning assessment tools directly into Blackboard for faculty and administrative use. This pilot is ongoing and CBE is seen as one of the leaders in the pilot program (see 2.10 Blackboard Pilot). 6 P age

8 2.1 New Committees & Positions Formed HIRE (High Impact Research and Engagement) Center Sandy Luong worked with Professor Glen Taylor as the faculty advisor. Ms. Luong played the lead role in designing and implementing the new AoL system. Professor Taylor served as the Director of the HIRE Center during the redesign process and provided feedback and advice to Ms. Luong to be sure that a faculty view of the process was always well represented. In this report we will refer to Sandy Luong and Glen Taylor working together as the HIRE Center. The HIRE Center recommended a series of changes to the AoL system, beginning with the creation and clarification of new roles and responsibilities explained below. Creation of an Assurance of Learning Task (AoL) Force An Assurance of Learning (AoL) Task Force was recommended by the HIRE Center and approved by the Dean to develop the overall plan and guide the assessment process. This task force includes the Dean, Associate Dean, Program Directors, Department Chairs, and the faculty director of the HIRE Center and HIRE Center staff. The AoL Task Force is responsible for review of the following: Curriculum Mapping Review & modification Learning Goals & Objectives Review & Modification Data Review Closing the Loop Brainstorming/Implementation Report Reviews Creation of Course Coordinator Role A new position called Course Coordinator was recommended by the HIRE Center. Course Coordinators were appointed by Department Chairs to work with instructional faculty in each course that is mapped to a learning objective. The Course Coordinators convene teaching faculty (including non-tenure track instructors) and are responsible for leading discussions on rubric reviews to measure the learning objective. Course Coordinators also lead discussions about determining assignments to use across sections of a course for assessment purposes. Department Chairs support Course Coordinators in this effort. When using new rubrics or rubrics that have recently been modified, Course Coordinators co-lead the rubric norming process with the help of the HIRE Center. Course Coordinators also help in curriculum mapping review and learning goals/objectives review when called upon. Spring 2013 saw the appointment of two course coordinators for MGMT 4500 and MGMT The appointments followed the procedure below: 1. HIRE Center requested Department Chair to appoint a Course Coordinator for assessed course. 2. HIRE Center met with Course Coordinators to brief on role and responsibilities. 7 P age

9 3. Course Coordinator met with faculty members teaching the chosen course for rubric and assignment review. Moving forward, assessment protocols will require Department Chairs to appoint a course coordinator for every course being assessed. This protocol allows for consistency in terms of who takes the lead for the affected quarter. The hope is that over time more faculty members will play the role of Course Coordinator and become increasingly involved with assessment. The meetings convened by Course Coordinators allow for direct involvement of all teaching faculty. They create an opportunity to discuss issues relevant to AoL with lecturers and tenure track faculty. During the Spring 2013 assessment cycle faculty found it beneficial to meet and discuss not only assessment but also teaching practices, allowing for inconsistencies across sections to be identified, and for best practices to be shared across sections of an assessed course. 8 P age

10 2.2 Roles & Responsibilities Systematically Defined Table 2.1 below displays the roles and responsibilities of key players as identified in a two-year assessment cycle, covering seven key assessment steps. Table 2.1 Roles & Responsibilities of Key Players in Typical Assessment Cycle HIRE Center Program Directors Department Chairs Course Coordinators (CCs) Faculty working with CCs Curriculum Committee Associate Dean Dean Fall Step 1: Review & Modify Assessment Process: Learning Goals, Objectives, Rubrics & Mapping Provide leadership and support on assurance of learning planning and implementation Participate in periodic review and mapping efforts of program goals 1. Appoint Faculty Course Coordinators as indicated in the 5 year plan initiated by the HIRE Center 2. Support CCs when necessary 3. Lead course mapping efforts 1. Lead development of new rubrics. 2. Lead periodic reivew of rubrics. 3. Lead periodic review of learning objectives that support program goals 4. Participate in mapping efforts 1. Collaborate with CCs on development of new rubrics. 2. Collaborate with CCs on periodic review of rubrics. 3. Collaborate with CCs on periodic review of goals and objectives 4. Participate in mapping efforts Review improvement priorities and approve major changes to goals and objectives Member of the Assurance of Learning Task Force Fall Step 2: Plan Assessment Measures Support faculty data collection efforts Support CCs in coordinating with faculty Identify an assignment for periodic measurement of outcomes in selected courses Participate in review of assessment measures Winter & Spring Step 3: Measure Outcomes Support faculty measurement efforts Measure outcomes Summer Quarter Step 4: Report Outcomes Generate and publish preliminary outcome reports Review preliminary outcome reports Review preliminary outcome reports Review preliminary outcome reports Fall Step 5: Close the Loop Track improvement implementation plans Develop program improvement plans Facilitate discussions among faculty for closing the loop efforts Plan closing the loop efforts Plan closing the loop efforts 1. Review improvement plans 2. Vote on recommendations 3. Present recommendations to Dean for approval Review recommendations from the Curriculum Committee Winter & Spring Step 6: Measure Outcomes Support faculty data collection efforts Implement closing the loop efforts Summer Step 7: End of Cycle Review Report and publish outcome results to be considered in the next cycle 9 P age

11 2.3 Synchronize the Assessment Language The assessment language CBE used prior to the Fifth Year Review was not consistent with AACSB terminology. Terms were used in different ways by faculty, committee members, and administrative personnel. There was also confusion between assessment terms for regional versus national accrediting bodies, such as WASC versus AACSB. This made communication on assessment extremely difficult throughout the college. We needed to standardize the college s use of terms for AACSB. Specifically, we needed to clarify the Learning Goals and Learning Objectives for AACSB before moving on to other assessment tasks such as curriculum mapping. The HIRE Center researched AACSB terminology for appropriate usage of terms. The HIRE Center also consulted Kathryn Martell s 2007 AACSB White Papers, as well as Kathryn Martell herself. Others were consulted, including CSU East Bay s AACSB Liaison, representatives from AACSB accredited schools with strong assessment programs, like Kennesaw State and Eastern Michigan University, and assessment experts at AACSB conferences and seminars. The HIRE Center proceeded to take what the college currently had as learning goals and objectives and restructured the statements in a way that was in line with the appropriate AACSB use of the terms. The three issues we addressed are diagramed below in Graphic 2.1: Graphic 2.1: CBE System Issues To better assist faculty, committee members, and administrative personnel communicate using one assessment language, the HIRE Center created an AACSB Assessment Terminology Pyramid. See Graphic 2.2 below. 10 P age

12 Graphic 2.2: Assessment Terminology Pyramid Once all involved personnel were speaking the same language, a review of what the college had for learning goals and learning objectives was then possible. As stated earlier, CBE s learning goals and objectives were first restructured by the HIRE Center to be in alignment with AACSB use of terminology. The restructured statements were then presented to the AoL Task Force for review. Afterwards, the statements were presented to the Curriculum Committee for review and approval in Spring During the academic year there was also a review and modification of the BSBA and MBA learning goals and objectives. A review of the BA Economics and MA Economics learning goals and objectives will be conducted in Tables 2.2 and 2.3 show the progression of the learning goals and objectives for programs BSBA and MBA as both programs underwent revision. Tables 2.4 and 2.5 show the final version of the approved statements. 11 P age

13 Table 2.2: The Evolution of CBE BSBA Program Goals/Objective Statements Existing BSBA Divided into Program and Student Objectives Program Objectives: 1. To prepare students to enter, sustain, and advance in careers in business. 2. To provide knowledge, skills, and ethical values to meet a wide range of employment 3. To instill an ability to work with a diverse, global workforce. 4. To develop a foundation to pursue higher level education in business or related fields. Student Learning Objectives: 1. Demonstrate foundation knowledge relevant to business management. 2. Use appropriate data and technology to make effective decisions. 3. Communicate with others and work collaboratively in a diverse, global environment. 4. Apply ethical reasoning when making business decisions. 5. Integrate functional knowledge and critical thinking skills to address opportunities and solve business problems. Proposed BSBA Learning Goal Statements Goal 1: Students who graduate will be knowledgeable in business management Goal 2: Students who graduate will be effective data driven decision makers. Goal 3: Students who graduate will be effective communicators in a diverse and global environment. Goal 4: Students who graduate will be ethical when making business decisions. Proposed Revised BSBA Learning Goals & Objectives 4 Goals; 8 Objectives (8 Rubrics) Goal 1: Students who graduate will be knowledgeable in business management Objective 1A: Students who graduate will recognize and recall foundation knowledge relevant to business management. Objective 1B: Students who graduate will integrate and think critically across functional areas of business. Goal 2: Students who graduate will be effective data driven decision makers. Objective 2A: Students who graduate will understand quantitative methods and tools while evaluating business problems. Objective 2B: Students who graduate will apply technology when analyzing business problems. Goal 3: Students who graduate will be effective communicators in a diverse and global environment. Objective 3A: Students who graduate will apply effective oral communication skills. Objective 3B: Students who graduate will apply effective written communication skills. Objective 3C: Students who graduate will apply effective team working skills. Goal 4: Students who graduate will be ethical when making business decisions. Objective 4: Students who graduate will understand ethical issues and analyze solutions for ethical problems. 12 P age

14 Table 2.3: The Evolution of CBE MBA Program Goals/Objective Statements Existing MBA Divided into Program and Student Objectives Proposed MBA Learning Goal Statements Proposed Revised MBA Learning Goals & Objectives 5 Goals; 8 Objectives (8 Rubrics) Program Objectives: 1. The M.B.A. will provide students with the knowledge and skills managers need to succeed. 2. The M.B.A. will provide students with the opportunity to select an area of specialization. Student Learning Objectives: 1. Make legal and ethical decisions in organizations and society. 2. Develop business strategies that respond to emerging opportunities and challenges in the global environment. 3. Apply leadership, teamwork, and communication skills in diverse managerial environments. 4. Analyze and integrate information from different functional areas of business in managerial decisions. 5. Apply quantitative, analytical, and technological tools for data driven decision making. Goal 1: Students who graduate will be law-abiding and ethical in making decisions in organizations and society. Goal 2: Students who graduate will be global in their perspective when developing business strategies. Goal 3: Students who graduate will have advanced communication skills in diverse managerial environments. Goal 4: Students who graduate will be analytical and integrative with information from different functional areas of business when making managerial decisions. Goal 5: Students who graduate will be effective data-driven decision makers. Goal 1: Students who graduate will be law-abiding and ethical in making decisions in organizations and society. Objective 1A: Students who graduate will recognize and analyze ethical issues in decision making. Goal 2: Students who graduate will be global in their perspective when developing business strategies. Objective 2A: Students who graduate will identify global business opportunities and analyze global business challenges. Goal 3: Students who graduate will be have advanced communication skills in diverse managerial environments. Objective 3A: Students who graduate will apply advanced written communication skills. Objective 3B: Students who graduate will apply advanced oral communication skills. Objective 3C: Students who graduate will demonstrate leadership and teamwork skills. Goal 4: Students who graduate will be analytical and integrative with information from different functional areas of business when making managerial decisions. Objective 4A: Students who graduate will integrate knowledge across disciplines to make managerial decisions and reach solutions to complex business problems. Goal 5: Students who graduate will be effective data-driven decision makers. Objective 5A: Students who graduate will perform quantitative analysis and apply advanced technological tools to solve complex business problems. Objective 5B: Students who graduate will understand data structures and generate a professionally written quantitative report. 13 P age

15 Table 2.4: Approved CBE BSBA Learning Goals/Objective Statements BSBA Learning Goals & Objectives 4 Goals; 8 Objectives (8 Rubrics) Goal 1: Students who graduate will be knowledgeable and integrative in their approach to business management. Objective 1A: Students who graduate will recognize and recall foundation knowledge relevant to business management. Objective 1B: Students who graduate will integrate and think critically across functional areas to solve business problems. Goal 2: Students who graduate will be effective data driven decision makers. Objective 2A: Students who graduate will understand and apply quantitative methods and tools in evaluating business problems. Objective 2B: Students who graduate will apply technology to analyze business problems. Goal 3: Students who graduate will be effective communicators in a diverse and global environment. Objective 3A: Students who graduate will apply effective oral communication skills. Objective 3B: Students who graduate will apply effective written communication skills. Objective 3C: Students who graduate will apply effective team working skills. Goal 4: Students who graduate will be ethical when making business decisions. Objective 4A: Students who graduate will understand ethical issues and derive solutions for ethical problems. Table 2.5: Approved CBE MBA Learning Goals/Objective Statements MBA Learning Goals & Objectives 5 Goals; 7 Objectives (7 Rubrics) Goal 1: Students who graduate will be able to make legal and ethical decisions in organizations and society. Objective 1A: Students who graduate will recognize and analyze legal and ethical issues in decision making. Goal 2: Students who graduate will be global in their perspective when developing business strategies. Objective 2A: Students who graduate will identify global business opportunities, analyze global business challenges, and develop business strategies. Goal 3: Students who graduate will have advanced communication and leadership skills in diverse managerial environments. Objective 3A: Students who graduate will apply advanced written communication skills. Objective 3B: Students who graduate will apply advanced oral communication skills. Objective 3C: Students who graduate will demonstrate leadership and teamwork skills. Goal 4: Students who graduate will be analytical and able to integrate information from different functional areas of business when making managerial decisions. Objective 4A: Students who graduate will analyze and integrate knowledge across disciplines to make managerial decisions to reach solutions to complex business problems. Goal 5: Students who graduate will be effective data-driven decision makers. Objective 5A: Students who graduate will perform quantitative analyses and apply advanced technological tools to solve complex business problems. 14 P age

16 2.4 Systematize Rubric Review, Modification & Implementation As part of the AoL system redesign efforts, we discovered that our existing rubrics were unclear and vague, producing data that was ineffective, inconsistent and irrelevant when it comes to assessing both learning outcomes in courses as well as overall program effectiveness. The unreliability of the data generated by using these rubrics is displayed in the two sets of data below representing two points of data collection using the same rubric measurement. See Table 2.6. The extreme contrast in the data is indicative of the lack of common understanding among faculty regarding what is being measured and how it is being measured. This is the result of both (1) the weakness in the measures themselves and (2) a lack of norming among faculty in applying the rubrics. Table 2.6: Example of Disparities Between Data Sets Collected By bringing together groups of faculty to discuss CBE s existing rubrics, the above misunderstandings and inconsistencies were confirmed. Faculty interpreted the meaning of rubric traits and the appropriate use of assessment criteria in very different ways. It was then noted that the new rubric review process needed to be systematized in a way that incorporates a norming process, in which faculty will come together and norm the rubric to ensure that all traits are consistently interpreted and measured across sections of a particular course to produce actionable data that can be used to determine the level of student proficiency for a particular programmatic learning goal/objective. 15 P age

17 Accurate data can only be collected if the tools used to collect the data are clearly and consistently applied by those using them. Thus, the norming process was incorporated into the systematic review process for rubrics, as previously stated. It is necessary to review the rubrics each year to continually assess these tools for their effectiveness, although in many instances faculty might decide that changes are not needed. Teaching faculty now have the opportunity to review and modify the rubric as appropriate to continually ensure that assessment tools are working well and are up-to-date with curriculum changes. Every Fall quarter, Course Coordinators will coordinate with faculty to review the current rubrics for modifications. Only rubrics for learning goals/objectives that are being assessed that academic year will be reviewed. Changes will be reported to the HIRE Center who will publish the changes on the Assessment website and report the changes to the appropriate committees, such as the AoL Task Force and the Curriculum Committee. Changes will also be incorporated in our annual review reports. This process of review is documented in the five-year assessment plan and is similar to the review process for the curriculum mapping and learning goals/objective statements. However, review of the curriculum mapping, learning goals, and learning objectives are reviewed every two years instead of annually like the rubrics. 16 P age

18 2.5 Systematize Closing the Loop The AACSB s White Paper (2007) states the two principal foundations of Assurance of Learning (AOL) standards for AACSB accreditation. They are (1) accountability and (2) continuous improvement. Closing the loop addresses specifically the issue of continuous improvement. Closing the loop comes at the end of the cycle of assessment and encompasses a review of the data collected, reporting findings and implementing improvement actions. See Graphic 2.3. Once equipped with the knowledge gathered through accurate data collections, it can then be determined what can be done to improve student learning. Graphic 2.3: Cycle of Assessment In the past, CBE s assessment process ended at Step 4; heavy data collection and data reporting. Findings from the data were not widely publicized and there was no discussion of how to improve learning based on the information that was gathered. By performing extensive assessment in all courses every quarter of the academic calendar, too much effort was spent on data collection and not enough effort was devoted to discussion of the findings and brainstorming of actions to improve student learning outcomes. 17 P age

19 As a result, it was clear that (1) allocating significant time for review of the data collected, (2) publicizing the data results, (3) holding faculty discussions to brainstorm closing the loop actions, and (4) identifying clear time frames where actions are to be implemented, all needed to be incorporated into the redesigned AoL system. Tables 2.7 and 2.8 show how the redesigned assessment plan includes closing the loop. The tables indicate when learning objectives will be assessed in the academic year and depicts an overview of when data collected will be reviewed, analyzed and publicized. We have also added a schedule for when faculty will discuss the data and what it tells us about our students, the areas of the curriculum that need to be improved and to recommend actions to be taken to continuously improve student learning outcomes. The 5-year cycle can be seen in Section 2.7 titled, Five-Year Assessment Plan by Annual Cycles. Overall, Tables 2.7 and 2.8 show the following: Each Learning Objective (LO) is being assessed at least twice in a 5-year period. Data will be assessed, analyzed, reviewed, publicized and receive an action-based decision as indicated according to a detailed assessment system that cycles ever 5 years. While certain LOs are being assessed, others will undergo data review and be given actionbased decisions in a manner that systematizes assessment. We have significantly simplified data reporting and report sharing in a way that is more timely and absorbable. When we reviewed best practices in this area and consulted with colleges known for their strong assessment plans, we learned that it is best to display the results in a way that is clear and concise for faculty to quickly understand the data gathered and apply it towards closing the loop. For the purposes of meeting presentations and quick faculty review, the key pieces of information are now arranged on a one-page template displayed in Graphic 2.4. However, the onepage summary is part of a much larger assessment write-up for those who want to see the data analyzed in detail. See Part 5: Spring Assessment Write-up for an example of an assessment report write-up following an assessed quarter. This section examines how closing the loop has been integrated into the Assurance of Learning system. For examples of closing the loop activities being conducted, planned, and currently in-progress, see Part 4: Closing the Loop Initiatives. 18 P age

20 Table 2.7: Continuous Assessment, Analysis, Publication and Action BSBA Program Program: BSBA Timeframe Activity Assess Analyze Publicize Action Year 5: Summer Year Report As Needed As Needed As Needed As Needed Year 5: Spring 2018 Track Blue LO 1AB, 3ABC & 4A Year 5: Winter 2018 Implement Blue LO 1AB, 3ABC & 4A Year 5: Fall 2017 Faculty/Committee Review & Dosciss Closing the Loop LO 1AB, 3ABC & 4A LO 1AB, 3ABC & 4A Year 4: Summer 2017 Annual Report LO 1AB, 3ABC & 4A LO 1AB, 3ABC & 4A Year 4: Spring 2017 Assesss Blue & Track Pink LO 1AB & 4A LO 1AB & 2AB Year 4: Winter 2017 Assesss Blue & Implement Pink LO 3ABC LO 1AB & 2AB Year 4: Fall 2016 Faculty/Committee Review & Dosciss Closing the Loop LO 1AB & 2AB LO 1AB & 2AB Year 3: Summer 2016 Annual Report LO 1AB & 2AB LO 1AB & 2AB Year 3: Spring 2016 Assesss Pink & Track Blue LO 1AB LO 1AB, 3ABC & 4A Year 3: Winter 2016 Assesss Pink & Implement Blue LO 2AB LO 1AB, 3ABC & 4A Year 3: Fall 2015 Faculty/Committee Review & Dosciss Closing the Loop LO 1AB, 3ABC & 4A LO 1AB, 3ABC & 4A Year 2: Summer 2015 Annual Report LO 1AB, 3ABC & 4A LO 1AB, 3ABC & 4A Year 2: Spring 2015 Assesss Blue & Track Pink LO 1AB & 4A LO 1AB & 2AB Year 2: Winter 2015 Assesss Blue & Implement Pink LO 3ABC LO 1AB & 2AB Year 2: Fall 2014 Faculty/Committee Review & Dosciss Closing the Loop LO 1AB & 2AB LO 1AB & 2AB Year 1: Summer 2014 Annual Report LO 1AB & 2AB LO 1AB & 2AB Year 1: Spring 2014 Assesss Pink & Track Orange LO 1AB LO 1AB, 3A & 4A Year 1: Winter 2014 Assesss Pink & Implement Orange LO 2AB LO 1AB, 3A & 4A Year 1: Fall 2013 Year 6: Summer 2013 Year 6: Spring 2013 Year 6: Fall/Winter 2013 Faculty/Committee Review & Dosciss Closing the Loop Implement Assessment 5-Year Cycle Conduct Pilot Assessment Under New Assessment Plan Redesign CBE Assessment Program & Develop New Plan LO 1AB, 3A & 4A LO 1AB, 3A & 4A LO 1AB, 3A & 4A LO 1AB, 3A & 4A LO 1AB, 3A & 4A 19 P age

21 Table 2.8: Continuous Assessment, Analysis, Publication and Action MBA Program Program: MBA Timeframe Activity Assess Analyze Publicize Action Year 5: Summer Year Report As Needed As Needed As Needed As Needed Year 5: Spring 2018 Track Blue LO 1A & 3ABC Year 5: Winter 2018 Implement Blue LO 1A & 3ABC Year 5: Fall 2017 Faculty/Committee Review & Dosciss Closing the Loop LO 1A & 3ABC LO 1A & 3ABC Year 4: Summer 2017 Annual Report LO 1A & 3ABC LO 1A & 3ABC Year 4: Spring 2017 Assesss Blue & Track Pink LO 1A LO 2A, 4A & 5A Year 4: Winter 2017 Assesss Blue & Implement Pink LO 3ABC LO 2A, 4A & 5A Year 4: Fall 2016 Faculty/Committee Review & Dosciss Closing the Loop LO 2A, 4A & 5A LO 2A, 4A & 5A Year 3: Summer 2016 Annual Report LO 2A, 4A & 5A LO 2A, 4A & 5A Year 3: Spring 2016 Assesss Pink & Track Blue LO 4A LO 1A & 3ABC Year 3: Winter 2016 Assesss Pink & Implement Blue LO 2A & 5A LO 1A & 3ABC Year 3: Fall 2015 Faculty/Committee Review & Dosciss Closing the Loop LO 1A & 3ABC LO 1A & 3ABC Year 2: Summer 2015 Annual Report LO 1A & 3ABC LO 1A & 3ABC Year 2: Spring 2015 Assesss Blue & Track Pink LO 1A LO 2A, 4A & 5A Year 2: Winter 2015 Assesss Blue & Implement Pink LO 3ABC LO 2A, 4A & 5A Year 2: Fall 2014 Faculty/Committee Review & Dosciss Closing the Loop LO 2A, 4A & 5A LO 2A, 4A & 5A Year 1: Summer 2014 Annual Report LO 2A, 4A & 5A LO 2A, 4A & 5A Year 1: Spring 2014 Assesss Pink & Track Orange LO 4A LO 1A & 3B Year 1: Winter 2014 Assesss Pink & Implement Orange LO 2A & 5A LO 1A & 3B Year 1: Fall 2013 Year 6: Summer 2013 Year 6: Spring 2013 Year 6: Fall/Winter 2013 Faculty/Committee Review & Dosciss Closing the Loop Implement Assessment 5-Year Cycle Conduct Pilot Assessment Under New Assessment Plan Redesign CBE Assessment Program & Develop New Plan LO 1A & 3B LO 1A & 3B LO 1A & 3B LO 1A & 3B LO 1A & 3B 20 P age

22 Graphic 2.4: Template of One-Page Assessment Summary Program Assessment Results (Template) Term Assessed Course Assessed Learning Goal Assessed Learning Goal Statement Learning Objective Statement 21 P age

23 2.6 New Rubrics Adopted & Plan for Future Customization It was clear from faculty responses that there were clear problems with the CBE rubrics that were being used to evaluate programmatic goals. Rubric language was vague and was often interpreted in different ways by different faculty members. Consistency was lacking in how the rubrics were to be used as a measurement tool for assessment. In addition, rubrics often assessed too many objectives at once. For example, one single rubric was used to assess Communication (both oral and written), Leadership, and Teamwork. As a result, every rubric needed to be reviewed for effectiveness and modification or replacement. Spring 2013 began the process of rubric review with two learning goals for the BSBA and MBA programs: (1) Oral Communication and (2) Ethics. Faculty teaching core courses that were mapped (under the pre-modified curriculum mapping) to either of the two learning goals met in groups. They were presented with the CBE version of the rubric and given an option to either modify the existing rubric or replace it entirely with a new one. The BSBA faculty group teaching the course mapped to Communication and the BSBA faculty group teaching the course mapped to Ethics decided to replace the existing rubric entirely. In both cases they replaced the existing rubrics with the AAC&U VALUE rubrics (See Tables 2.9 and 2.10). The faculty then customized the AAC&U rubric to better fit CBE s program in the future. As part of AAC&U s Liberal Education and America s Promise (LEAP) initiative, the VALUE project seeks to contribute to the national dialogue on assessment of college student learning. With support from the State Farm Companies Foundation and the Fund for the Improvement of Post-Secondary Education (FIPSE), the VALUE project builds on a philosophy of learning assessment that privileges authentic assessment of student work and shared understanding of student learning outcomes on campuses over reliance on standardized tests administered to samples of students outside of their required courses. The result of this philosophy has been the collaborative development of 15 rubrics by teams of faculty and academic professionals on campuses from across the country over the course of 2-3 years. More than 100 faculty and other experts worked on rubric development teams, creating and revising the 15 VALUE rubrics. Over75 campuses are currently using AAC&U metarubrics to assess student work. Like the BSBA faculty, the MBA faculty teaching courses mapped to oral communication and ethics also met in groups and were presented with the CBE version of the rubric and were given the option to either modify the existing rubric or replace it entirely. MBA faculty also determined the existing CBE rubrics were vague and ineffective. As a result, the faculty determined to replace the rubrics. The HIRE Center researched graduate level rubrics published online and presented MBA faculty with options. The intention was to replace the CBE rubrics with something that more appropriately reflects graduate level learning outcomes. HIRE Center researched rubrics used at other schools and presented them to faculty teaching the mapped course (MGMT 6800 in Spring 2013) as options. The faculty then selected from among these rubrics and customized them to better fit CBE s program in the future.* The faculty selected an Oral Communication rubric from San Diego State University s College of Business MBA program and an Ethics rubric from San Francisco State University s College of Business MBA program (See Tables 2.11 and 2.12). 22 P age

24 The comments and critiques following one administration of the baseline rubrics are shown below. Example of Rubric Comments & Critiques (Post-Spring 2013 Assessment): Oral Communication VALUE Rubric: SDSU Oral Communication Rubric: 1. Delivery Trait Overall weak area for CBE BSBA students; include in description vocal pace (appropriate pauses) 2. New Traits to Consider Adding to Rubric: (1) Consistency Is the presentation strong throughout or are parts of the presentation weak? (2) Originality How creative was the presenation? (3) Command of Material How well do the students perform under the Q&A section of the presentation? (4) Time Management How well does the student manage time? (5) Visual Aids How effective are the visual aids? Were the powerpoint presentations ineffective (i.e. too much text on slides, distracting designs, messy graphs, etc.) 1. Change Trait Name from Organization to Content/Structure to more appropriately resemble what is being measured. 2. New Traits to Consider Adding to Rubric: (1) Visual Aids - How professional are the visual aids? Were the powerpoint presentations ineffective (i.e. too much text on slides, distracting designs, messy graphs, etc.) (2) Command of Material How well do the students perform under the Q&A section of the presentation? (3) Consistency Is the presentation strong throughout or are parts of the presentation weak? (4) Time Management How well does the student manage time? *Faculty group discussions regarding rubrics and decisions to replace CBE s rubrics, with the plan to better customize in the future, were documented in meeting minutes. See Part 6: Meeting Materials. 23 P age

25 Table 2.9: AAC&U VALUE Rubric for Learning Objective: Oral Communication 24 P age

26 Table 2.10: AAC&U VALUE Rubric for Learning Objective: Ethics 25 P age

27 Table 2.11: San Diego State University MBA Rubric for Oral Communication 26 P age

28 Table 2.12: San Francisco State University MBA Rubric for Ethics 27 P age

29 2.7 Five-Year Assessment Plan by Annual Cycles Tables 2.13 and 2.14 on the following pages display the BSBA and MBA five-year plans. Each table shows the following information: Each learning objective originates from a learning goal Every objective is assessed at least twice in a five year period The timetable for reviewing and modifying curriculum maps When learning goals and learning objectives statements are to be reviewed When measurements and assignment tools are to be reviewed for effectiveness and modifications (rubrics & course assignment) When data is to be aggregated and reported When assessment results are to be presented When faculty will gather to discuss results and make action-based decisions to close the loop When closing the loop actions will be implemented and tracked 28 P age

30 Table 2.13: 5-Year Assessment Plan by Annual Cycles for BSBA Program Pilot Year 1 Year 2 Year 3 Year 4 Year 5 Learning Objective W2013 Sp 2013 Su2013 F2013 W2014 Sp2014 Su2014 F2014 W2015 Sp2015 Su2015 F2015 W2016 Sp2016 Su2016 F2016 W2017 Sp2017 Su2017 F2017 W2018 Sp2018 Su2019 Program: BSBA Learning Goal 1 Learning Goal 2 Learning Goal 3 Learning Goal 4 1A) Foundation Knowledge 1B) Functional Integration 2A) Quantitative Methods 2B) Use of Technology 3A) Oral Communication 3B) Written Communication 3C) Teamwork 4A) Ethics Faculty Convene MGMT 4650 Capsim (All LGs) MGMT 4650 Capsim (All LGs) MGMT 4650 MGMT 4500 Data Aggregation, Results Publicized Review of Mapping, Goals/Objectives, Rubrics, Data Results & Actions (see Roles & Responsibilities Table) Cours e Embedded COURSE 000 Cours e Embedded COUIRSE 000 MGMT 4650 Capsim (All LGs) MGMT 4650 Capsim (All LGs) Data Aggregation, Results Publicized Review of Data Collected; Discuss Results/Actions Cours e Embedded COURSE 000 Cours e Embedded COUIRSE 000 Cours e Embedded COURSE 000 MGMT 4650 Capsim (All LGs) MGMT 4650 Capsim (All LGs) Cours e Embedded COUIRSE 000 Data Aggregation, Results Publicized Review of Mapping, Goals/Objectives, Rubrics, Data Results & Actions (see Roles & Responsibilities Table) Cours e Embedded COURSE 000 Cours e Embedded COUIRSE 000 MGMT 4650 Capsim (All LGs) MGMT 4650 Capsim (All LGs) Data Aggregation, Results Publicized Review of Data Collected; Discuss Results/Actions Cours e Embedded COURSE 000 Cours e Embedded COUIRSE 000 Cours e Embedded COURSE 000 MGMT 4650 Capsim (All LGs) MGMT 4650 Capsim (All LGs) Cours e Embedded COUIRSE 000 Data Aggregation, Results Publicized As Needed As Needed As Needed As Needed As Needed As Needed As Needed As Needed As Needed As Needed As Needed As Needed MGMT 4650 Capsim (All LGs) MGMT 4650 Capsim (All LGs) As Needed As Needed As Needed As Needed As Needed As Needed Data Aggregation, Results Publicized *Specific courses are to be mapped during Fall 2013 Quarter following re-mapping efforts. Re-mapping efforts have been underway since Spring 2013 and have included over 39 faculty members throughout the process. CBE s original curriculum mapping, prior to the re-mapping efforts of 2013, was developed (1) with little faculty involvement and (2) had identified a high number of management courses to be assessed by comparison to the other departments, such as Economics, Marketing, Accounting, and Finance. Following the re-mapping efforts, CBE will be in a better position to determine which courses throughout the program and across departments will be best suited for assessment. The 5-year plan above allocates time in Fall quarter to continue re-mapping efforts, such that assessment can be done in Winter and Spring 2014, as indicated in the chart above. For a complete report of the re-mapping efforts and process see Part 2: System Improvements Section 2.8: Curriculum Mapping. 29 P age

31 Table 2.14: 5-Year Assessment Plan by Annual Cycles for MBA Program Pilot Year 1 Year 2 Year 3 Year 4 Year 5 Learning Objective W2013 Sp 2013 Su2013 F2013 W2014 Sp2014 Su2014 F2014 W2015 Sp2015 Su2015 F2015 W2016 Sp2016 Su2016 F2016 W2017 Sp2017 Su2017 F2017 W2018 Sp2018 Su2019 Program: MBA Learning Goal 1 Learning Goal 2 Learning Goal 3 Learning Goal 4 Learning Goal 5 1A) Legal & Ethical Decision- Making 2A) Global Perspectives 3A) Written Communication 3B) Oral Communication 3C) Teamwork 4A) Integrative Knowledge 5A) Data-Driven Decision Making Faculty Convene MGMT 6215 MGMT 6800 Data Aggregation, Results Publicized Review of Mapping, Goals/Objectives, Rubrics, Data Results & Actions (see Roles & Responsibilities Table) Course Embedded COURSE 000 Course Embedded COURSE 000 Course Embedded COURSE 000 Data Aggregation, Results Publicized Review of Data Collected; Discuss Results/Actions Course Embedded COURSE 000 Course Embedded COURSE 000 Course Embedded COURSE 000 Course Embedded COURSE 000 Data Aggregation, Results Publicized Review of Mapping, Goals/Objectives, Rubrics, Data Results & Actions (see Roles & Responsibilities Table) Course Embedded COURSE 000 Course Embedded COURSE 000 Course Embedded COURSE 000 Data Aggregation, Results Publicized Review of Data Collected; Discuss Results/Actions Course Embedded COURSE 000 Course Embedded COURSE 000 Course Embedded COURSE 000 Course Embedded COURSE 000 Data Aggregation, Results Publicized As Needed As Needed As Needed As Needed As Needed As Needed As Needed As Needed As Needed As Needed As Needed As Needed As Needed As Needed As Needed As Needed As Needed As Needed As Needed As Needed As Needed Data Aggregation, Results Publicized *Specific courses are to be mapped during Fall 2013 Quarter following re-mapping efforts. Re-mapping efforts have been underway since Spring 2013 and have included over 39 faculty members throughout the process. CBE s original curriculum mapping, prior to the re-mapping efforts of 2013, was developed (1) with little faculty involvement and (2) had identified a high number of management courses to be assessed by comparison to the other departments, such as Economics, Marketing, Accounting, and Finance. Following the re-mapping efforts, CBE will be in a better position to determine which courses throughout the program and across departments will be best suited for assessment. The 5-year plan above allocates time in Fall quarter to continue re-mapping efforts, such that assessment can be done in Winter and Spring 2014, as indicated in the chart above. For a complete report of the re-mapping efforts and process see Part 2: System Improvements Section 2.8: Curriculum Mapping. 30 P age

32 2.8 Curriculum Mapping We approach assurance of learning from the point of view of the effectiveness of whole programs. We are not primarily concerned with assessing courses except as they contribute to program goals. The purpose of any curriculum mapping exercise is to ensure that there is a strong alignment of curricula with program goals. The following was extracted from the AACSB White Paper No. 3, which was revised on 3 May 2013: Since the learning goals are established within an across the curriculum context, the next step is to ensure that the curriculum addresses the learning goals. Ideally this occurs in a number of courses. That is, there should be clear evidence that the work students are doing in one or more classes directly supports student achievement of the learning goals. The more places in a curriculum that support one or more learning goals, the greater the probability of student success. For example, goals that address oral presentation skills, critical thinking skills, problem-solving skill, etc. can be addressed in a variety of courses regardless of the content area. The alignment of learning goals and curricula is critical. If learning goals are adopted but are not addressed in the curricula, the outcomes assessment process will be worthless. Throughout the assessment review and redesign process we were aware that CBE s existing curriculum mapping was not well developed in terms of identifying how learning outcomes build upon one another across courses as students make progress through their programs. It also did not reflect the views of faculty. In other words, we knew from the beginning that curriculum mapping needed a thorough review. As a result, re-mapping exercises were undertaken in summer and fall quarters of In preparation, the HIRE Center researched multiple approaches to curriculum remapping, consulted with other business colleges, and discussed formats with CSU East Bay s AACSB liaison. Tables 2.15 through 2.18 were used in explaining the different mapping formats and the purpose of mapping to various faculty and administrators at CBE in a variety of held meetings throughout the remapping efforts. The PowerPoint presentation that accompanied these meetings with faculty and administrators can be found in Part 6: Meeting Material. Table 2.15 This diagram was used to show how mapping can be accompanied by different kinds of coding. CBE decided to use the coding: I(Introduced), R(Reinforced), and M(Mastery). 31 P age