Public Disclosure of Student Learning Form

Transcription

1 International Assembly for Collegiate Business Education Public Disclosure of Student Learning Form Institution: Academic Business Unit: University of Saint Mary Business, Accounting, and Sport Management Academic Year: International Assembly for Collegiate Business Education Strang Line Road Lenexa, Kansas USA

2 Section I: Student Learning Assessment Outcomes Assessment Results For Academic Year: Department of Business, Accounting, and Sport Management Student Learning Assessment for Bachelor of Science in Business Administration and Accounting General Program Intended Student Learning Outcomes (General Program ISLOs) 1. Students will demonstrate a basic understanding of the knowledge, skills, and attributes needed by managers and/or professionals in a business, economic, and political environment. 2. Students will demonstrate competence in business and accounting necessary to pursue general management, financial and /or accounting specialists opportunities. 3. Students will demonstrate business and accounting career-oriented skills most appropriate to to their selected concentration and/or major(s). 4. Students will demonstrate critical thinking and communication skills in the areas of ethics, diversity, change and leadership. Intended Student Learning Outcomes: Management ISLOs 1. Students will demonstrate the ability to analyze Business Organizations in terms of Organizational Development and traditional and contemporary Human Resources roles within the legal and regulatory environment and identify actions needed to obtain and sustain a competitive advantage. 2. Students will demonstrate the ability to integrate the Management and Strategic processes to use major analytical tools to analyze case studies, complete computerized simulations and to formulate, implement and evaluate problems, policies, procedures and to develop plans from a strategic executive level of management. Intended Student Learning Outcomes: Marketing ISLOs 1. Students will demonstrate knowledge and understanding of how to execute a basic S.W.O.T. analysis and conduct basic secondary research including demographic and psychographic techniques that define the target customer. 2. Students will demonstrate the ability to conduct a basic segmentation process that helps define who the customer is for a specific product or service and results in in the creation of strategy-based marketing plan using the 4Ps in a marketing mix designed especially for the product or service. Intended Student Learning Outcomes: Bachelor of Science in Accounting ISLOs 1. Students will effectively define the needs of various users of accounting data and demonstrate the ability to use such data effectively, as well as the ability to provide knowledgeable recommendations. 2. Students will demonstrate the ability to recognize circumstances providing for increased exposure to fraud and define preventative internal control measures. i

3 Assessment Instruments for Intended Student Learning Outcomes Direct Measures of Student Learning: Objectives (Targets/Criteria) for Direct Measures: 1. Direct Measure 1 Objective (Target/Criterion) for Direct Measure 1 Capstone exam given in our senior level Business Policy and Ethics class (Appendix A1 & A2) 75% our students will correctly answer 80% of the questions in each of the content areas of the Capstone Exam 2. Direct Measure 2 Objective (Target/Criterion) for Direct Measure 2 Senior integrated case analysis (Appendix B) 75% of our students score at least 80% on their integrated case study analysis 3. Direct Measure 3 Objective (Target/Criterion) for Direct Measure 3 Third Party CPC Testing Under Development Assessment Instruments for Intended Student Learning Outcomes Measures of Student Learning: Objectives (Targets/Criteria) for Measures: 1. Measure 1 Objective (Target/Criterion) for Measure 1 Undergraduate Student Satisfaction Survey (See Appendix C) 75% of students will be at least satisfied with the Business Department 2. Measure 2 Objective (Target/Criterion) for Measure 2 Employment Rates (Appendix D) 75% of graduates of the business program will obtain jobs in the fields of business or accounting, continue on to graduate school as full time students, or enlist/be commissioned in the armed forces, within six months of graduating ii

4 Assessment Results: Bachelor of Science in Buisness Administration and Accounting Summary of Results from Implementing Direct Measures of Student Learning: 1. Summary of Results for Direct Measure 1 In-house Capstone Test Report Four Year Average of Test Scores Major Overall MGT MKT ECON FIN ACC Sample All Students 84% 91% 86% 85% 74% 68% 40 Accounting Students 85% 88% 88% 82% 81% 78% 7 Management Students 87% 93% 88% 84% 78% 74% 12 Marketing Students 82% 89% 85% 85% 68% 66% 21 Students scoring 80% or better per section by major % Students scoring 80% or greater Overall MGT MKT ECON FIN ACC Sample All Majors 60% 90% 75% 78% 48% 38% # of students above 80% total students Accounting Students 71% 71% 86% 57% 71% 43% # of students above 80% ACC students Management Students 67% 100% 75% 83% 50% 50% # of students above 80% MGT students Marketing Students 52% 90% 71% 81% 38% 29% # of students above 80% MKT students As this is the first year that we are reporting on a rolling average, there is no direct comparison available for this reporting year. The results of this analysis, however, do reflect in a statistically viable manner how our graduates have performed on the In-House Capstone exam. Future tracking of this rolling average will result in a more reliable comparison year over year, as the results are less likely to be skewed by the small number of graduates that we have in any individual year. On average over a four year period, however: Management students score an average of 87% on the Overall Exam and 93% on the Management portion of the exam Marketing students score an average of 82% on the Overall Exam and 85% on the Marketing portion of the exam. Accounting students score an average of 85% on the Overall Exam, 81% on the finance portion of the Exam, and 78% on the Accounting portion of the Exam. iii

5 2. Summary of Results for Direct Measure 2 Results of Capstone Case Study (May of Graduation Year) s Capstone Case Study (Overall) 5 Year Analysis Spring 2010 (8 Students, 2 groups) Spring 2011 (9 Students, 3 groups) Spring 2012 (8 Students, 3 groups) Spring 2013 (5 students, 1 group) Spring 2014 (12 Students, 3 groups) Average 81.23% 82.00% 71.69% 81.31% 85.00% 88.75% Median 83.50% 82.00% 67.00% 83.50% 85.00% 88.75% Mode 90.00% 90.00% 82.50% 72.00% 85.00% 90.00% Standard Deviation 8.89% 8.55% 9.15% 8.48% 0.00% 1.34% Variance (StdDev) 0.79% 0.73% 0.84% 0.72% 0.00% 0.02% Population StdDev 8.78% 8.00% 8.55% 7.93% 0.00% 1.25% Variance (PopStdDev) 0.77% 0.64% 0.73% 0.63% 0.00% 0.02% # Students w/ Scores over 80% 25 of 42 4 out of 8 3 out of 9 5 out of 8 5 out of 5 8 out of 12 % of Students w/ Scores over 80% 69% 50.00% 33.33% 62.50% % 66.67% In an attempt to normalize our scores, we have transitioned to a rolling average for this measure. However, as the Capstone Case Study is currently a group project, this still creates statistical anomalies in the results of our assessment because the students in each group receive the same grade for their group project, reducing our sample size to 12 groups rather than 42 students. Additionally, we recognize that our measure is taken from an overall score rather than from specific areas of the grading rubric, which only generally addresses the Intended Student Learning Outcomes to which this measure is tied. 3. Summary of Results for Direct Measure 3: Objectives Under Development iv

6 Summary of Results from Implementing Measures of Student Learning: 1. Summary of Results for Measure s Satisfaction Survey Results % of Business Students Satisfied (excludes Sports Management) Rating Average Above Avg Excellent (>=5 out of 1-10 Scale, 10 being highest 10) (>=7 out of 10) (>9 out of 10) Average Scores University Satisfaction (Overall) % 90.91% 9.09% 7.36 Job Preparedness % 81.82% 36.36% 8.18 Value of Education % 81.82% 18.18% 8.00 Available Technology 72.73% 45.45% 0.00% 5.82 Business Dept Satisfaction (Detail w/ Technology) % 90.91% 9.09% 8.06 Business Dept Satisfaction (Detail w/o Technology) % % 9.09% 8.27 Business Dept Satisfaction (Single Question Overall) % 90.91% 18.18% 8.18 For the Reporting Year, 91% of our students were at least satisfied with the Business Department, as indicated by a score of at least 7 on a scale of The average satisfaction rating for the Business Department was Summary of Results for Measure 2 Of 54 s from the Department of Business, Accounting, and Sport Management over the past five years, 50 were contacted regarding their employment status six months after graduation: % Employed in Related Field % Grad School % Military Service % Employed Non-related Field 60% (30 of 50 Students) 22% (11 of 50 Students) 0% 16% (8 of 50 Students) For Academic Years 2009/ /2014, 88% (49 out of 50) of the graduates that we were able to contact were either working in a business related field, pursuing their graduate education on a full time basis, or had enlisted/been commissioned in the armed forces. v

7 Summary of Achievement of Intended Student Learning Outcomes: Intended Student Learning Outcomes Learning Assessment Measures General Program ISLOs Senior Capstone Exam Senior Integrated Case Study Third Party Comprehensi ve Exam Direct Measure 4 Undergradua te Student Satisfaction Survey Employment Rates Measure 3 Measure 4 1. Students will demonstrate a basic understanding of the knowledge, skills, and attributes needed by managers and/or professionals in a business, economic, and political environment. 2. Students will demonstrate competence in business and accounting necessary to pursue general management, financial and /or accounting specialists opportunities. 3. Students will demonstrate business and accounting career-oriented skills most appropriate to to their selected concentration and/or major(s). 4. Students will demonstrate critical thinking and communication skills in the areas of ethics, diversity, change and leadership. NOT NOT Assessment Measures Under Devleopment NOT NOT NOT N/A N/A NOT vi

8 Intended Student Learning Outcomes: Concentration in Management Senior Capstone Exam Senior Integrated Case Study Third Party Comprehensi ve Exam Direct Measure 4 Undergradua te Student Satisfaction Survey Employment Rates Measure 3 Measure 4 1. Students will demonstrate the ability to analyze Business Organizations in terms of Organizational Development and traditional and contemporary Human Resources roles within the legal and regulatory environment and identify actions needed to obtain and sustain a competitive advantage. 2. Students will demonstrate the ability to integrate the Management and Strategic processes to use major analytical tools to analyze case studies, complete computerize simulations and to formulate, implement and evaluate problems, policies, procedures and to develop plans from a strategic executive level of management. NOT Assessment Measures Under Devleopment NOT vii

9 Intended Student Learning Outcomes: Concentration in Marketing Senior Capstone Exam Senior Integrated Case Study Third Party Comprehensi ve Exam Direct Measure 4 Undergradua te Student Satisfaction Survey Employment Rates Measure 3 Measure 4 1. Students will demonstrate knowledge and understanding of how to execute a basic S.W.O.T. analysis and conduct basic secondary research including demographic and psychographic techniques that define the target customer. 2. Students will demonstrate the ability to conduct a basic segmentation process that helps define who the customer is for a specific product or service and results in in the creation of strategy-based marketing plan using the 4Ps in a marketing mix designed especially for the product or service. NOT Assessment Measures Under Devleopment NOT viii

10 Intended Student Learning Outcomes: Bachelor of Science in Accounting Senior Capstone Exam Senior Integrated Case Study Third Party Comprehensi ve Exam Direct Measure 4 Undergradua te Student Satisfaction Survey Employment Rates Measure 3 Measure 4 1. Students will effectively define the needs of various users of accounting data and demonstrate the ability to use such data effectively, as well as the ability to provide knowledgeable recommendations. 2. Students will demonstrate the ability to recognize circumstances providing for increased exposure to fraud and define preventative internal control measures. N/A NOT Assessment Measures Under Devleopment N/A NOT Proposed Courses of Action for Improvement in Learning Outcomes for which Targets Were Not Met: 1. Course of Action 1: It is apparent based on the updated format of the IACBE Annual Report (most recently updated on 9/30/2014) that our Assessment Plan will need further revisions to better match the information being requested in the Annual Report. While our Assessment Measures meet the SMART model (specific, measurable, attainable, relevant, and time-bound), it is clear that we need to be able to report our results in an even more specific manner. Due to the new format being released close to the reporting date, we are unable to completely revise our Assessment Plan and reporting methods for the current reporting year. However, we are already pursuing more efficient methods of obtaining data that is more specific to the new reporting requirements; this project is scheduled for completion in August of Course of Action 2 (Direct Measure 1 In-House Capstone Exam) After reviewing several years worth of results from our current In-House Capstone exam, it became apparent that the exam does not specifically measure the skills that our graduates are expected to display, as described by our Intended Student Learning Outcomes. Therefore, a new In- House Capstone Exam has been developed from the test banks of the various textbooks that we are using. This allows us to more closely identify our students knowledge of concepts from both the business core courses and concentrations courses, as well as their ability to apply those concepts to skill based tasks, using questions that have been tested for statistical validity. The new Capstone Exam will be piloted in the Spring 2015 Capstone course. We will continue to administer the original Capstone Exam as well until we have enough data to provide statistically relevant results from the updated Exam. ix

11 3. Course of Action 3 (Direct Measure 2 Senior Integrate Case Study) As noted in Course of Action 1, we are pursuing more efficient methods of obtaining more specific data to measure our Intended Student Learning Outcomes. As part of this process, we will be reviewing the current grading rubric for the Capstone Case Study. Our current Management professor is in the process of retiring, so action on this item will be deferred until a new Management professor has been hired (expected completion AY 2015/2016). 4. Course of Action 4 (Direct Measure 3 Third Party Testing) Our Third Party Testing initiative is ongoing, however, we did not receive the number of results that we anticipated from the testing cycle. With only 24 records for comparison, we do not feel that the analysis at this point is statistically relevant. We will continue our Third Party testing efforts and review of the testing results with the intention of determining and reporting the performance target for this measure once we have a large enough sample size to statistically validate the data. x

12 Department of Business, Accounting, and Sport Management Student Learning Assessment for Masters of Business Administation General Program Intended Student Learning Outcomes (General Program ISLOs) 1. Identify critical business issues facing organizations. 2. Analyze business issues from a cross-disciplinary perspective and form practical responses to these issues. 3. Assess the impact of global influences on business decisions. 4. Formulate ethical approaches to business decisions and understand how to apply these within a business context. 5. Communicate effectively in both a written and oral manner. Assessment Instruments for Intended Student Learning Outcomes Direct Measures of Student Learning: Objectives (Targets/Criteria) for Direct Measures: 1. Direct Measure 1 Objective (Target/Criterion) for Direct Measure 1 Case Analysis (see Appendix E1, E2, and E3 for examples of the textbook cases used in this class) 1a. Overall 90% of our students in the Business Policy and Ethics capstone class will score at least 80% on a series of textbook cases that are analyzed in this course. 1b. Overall 90% of our students in the Business Policy and Ethics capstone class will score at least 80% on a series of ethical discussion papers that are analyzed in this course. 1c. Overall 90% of our students in the Business Policy and Ethics capstone class will score at least 80% on a series of strategy discussion papers that are analyzed in this course. 2. Direct Measure 2 Objective (Target/Criterion) for Direct Measure 2 Students Exiting Final Paper (Appendix F) Overall 90% of our students successfully complete the exiting assessment final paper with a grade of at least 80%. 3. Direct Measure 3 Objective (Target/Criterion) for Direct Measure 3 Third Party MBA Testing (first results expected Fall 2012) (Appendix G: MBA Third Party Test Results) 3a. Overall 90% of our MBA students will meet or exceed the national average score for all topic areas in the Ivy Software MBA Comprehensive Examination. 3b. Overall 90% of our MBA students will meet or exceed the average score of other IACBE schools tested for all topic areas in the Ivy Software MBA Comprehensive Examination. xi

13 Assessment Instruments for Intended Student Learning Outcomes Direct Measures of Student Learning: Objectives (Targets/Criteria) for Direct Measures: 4. Direct Measure 4 Objective (Target/Criterion) for Direct Measure 4 Capstone Presentation Series (Appendix H) 4. Overall 90% of our students in the Business Policy and Ethics capstone class will score at least 80% on a series of presentations that are analyzed in this course. Assessment Instruments for Intended Student Learning Outcomes Measures of Student Learning: Objectives (Targets/Criteria) for Measures: 1. Measure 1 Objective (Target/Criterion) for Measure 1 Exit Survey Question 2 (Appendix I) On average, graduating students will at least "Agree" (as indicated by a score of 4 or above) with the following question about the results they have achieved by completing their MBA: The program helped me to develop problem solving skills in a way that I am able to recognize critical problems and address organizational issues (ISLO 1&2). 2. Measure 2 Objective (Target/Criterion) for Measure 2 Exit Survey Question 3 (Appendix I) On average, graduating students will at least "Agree" (as indicated by a score of 4 or above) with the following question about the results they have achieved by completing their MBA: The program improved my ability to evaluate and balance decisions characterized by competing demands or requirements emanating from the organizational, stakeholder, societal and ethical contexts (ISLO 2&4). 3. Measure 3 Objective (Target/Criterion) for Measure 3 Exit Survey Question 5 (Appendix I) On average, graduating students will at least "Agree" (as indicated by a score of 4 or above) with the following question about the results they have achieved by completing their MBA: The program improved my communication skills so that I was better equipped to convey information and knowledge in an effective and efficient manner(islo 5). xii

14 Assessment Instruments for Intended Student Learning Outcomes Measures of Student Learning: Objectives (Targets/Criteria) for Measures: 4. Measure 4 Objective (Target/Criterion) for Measure 4 Exit Survey Question 6 (Appendix I) On average, graduating students will at least "Agree" (as indicated by a score of 4 or above) with the following question about the results they have achieved by completing their MBA: The program helped me to make decisions from a strategic perspective (ISLO 1&2). 5. Measure 5 Objective (Target/Criterion) for Measure 5 Exit Survey Question 7 (Appendix I) On average, graduating students will at least "Agree" (as indicated by a score of 4 or above) with the following question about the results they have achieved by completing their MBA: The program helped me to make decisions that incorporated a global perspective (ISLO 3). 6. Measure 6 Objective (Target/Criterion) for Measure 6 Exit Survey Question 8 (Appendix I) On average, graduating students will at least "Agree" (as indicated by a score of 4 or above) with the following question about the results they have achieved by completing their MBA: The program helped me to make decisions by evaluating critical factors in the marketplace (ISLO 1). 7. Measure 7 Objective (Target/Criterion) for Measure 7 Exit Survey Question 13 (Appendix I) On average, graduating students will at least "Agree" (as indicated by a score of 4 or above) with the following question about the results they have achieved by completing their MBA: Class discussions enhanced learning from your peers (ISLO 5). 8. Measure 8 Objective (Target/Criterion) for Measure 8 Exit Survey Question 14 (Appendix I) On average, graduating students will at least "Agree" (as indicated by a score of 4 or above) with the following question about the results they have achieved by completing their MBA: Use of cases in the class helped me to tie theory to real-life business problems (ISLO 2). xiii

15 Assessment Instruments for Intended Student Learning Outcomes Measures of Student Learning: Objectives (Targets/Criteria) for Measures: 9. Measure 9 Objective (Target/Criterion) for Measure 9 Exit Survey Question 16 (Appendix I) On average, graduating students will at least "Agree" (as indicated by a score of 4 or above) with the following question about the results they have achieved by completing their MBA: The amount of presentations were appropriate for a master's level program (ISLO 5). xiv

16 Assessment Results: Master of Business Administration Summary of Results from Implementing Direct Measures of Student Learning: 1. Summary of Results for Direct Measure 1 Of 62 MBA students who completed the Capstone Course this reporting year: % Students with Textbook Case Study sequence grade of 80% or better % % Students with Ethics Discussions grade of 80% or better 94.64% % Students with Strategy Discussions grade of 80% or better % 2. Summary of Results for Direct Measure % of MBA students who completed the Capstone course this reporting year scored 80% or better on the Exiting Final Paper 3. Summary of Results for Direct Measure 3 Subject ID Group Score IACBE Average National Average % Students Scoring at or above IACBE Average % Students Scoring at or above National Average Finance % 52% Financial Accounting % 36% Management % 48% Managerial Accounting % 52% Marketing % 55% Strategy % 48% Overall Score % 48% 4. Summary of Results for Direct Measure % of MBA students who completed the Capstone course this reporting year scored 80% or better on the Capstone Presentation Series Summary of Results from Implementing Measures of Student Learning: % of Students Agree Avg Score 1. Summary of Results for Measure 1 Question 2 The program helped me to develop problem solving skills in a way that I am able to recognize critical problems and address organizational issues. 94% Summary of Results for Measure 2 Question 3 The program improved my ability to evaluate and balance decisions characterized by competing demands or requirements emanating from the organizational, stakeholder, societal and ethical contexts. 96% 4.38 xv

17 Summary of Results from Implementing Measures of Student Learning: % of Students Agree Avg Score 3. Summary of Results for Measure 3 Question 5 The program improved my communication skills so that I was better equipped to convey information and knowledge in an effective and efficient manner. 86% Summary of Results for Measure 4 Question 6 The program helped me to make decisions from a strategic perspective. 96% Summary of Results for Measure 5 Question 7 The program helped me to make decisions that incorporated a global perspective. 90% Summary of Results for Measure 6 Question 8. The program helped me to make decisions by evaluating critical factors in the marketplace. 90% Summary of Results for Measure 7 Question 13. Class discussions enhanced learning from your peers. 74% 4 8. Summary of Results for Measure 8 Question 14. Use of cases in the class helped me to tie theory to real-life business problems. 94% Summary of Results for Measure 9 Question 16. The amount of presentations were appropriate for a master's level program. 88% 4.18 xvi

18 Intended Student Learning Outcomes: Master of Business Administration 1. Identify critical business issues facing organizations. 2. Analyze business issues from a cross-disciplinary perspective and form practical responses to these issues. 3. Assess the impact of global influences on business decisions. 4. Formulate ethical approaches to business decisions and understand how to apply these within a business context. 5. Communicate effectively in both a written and oral manner. Case Analysis Exiting Final Paper Third Party MBA Testing Capstone Presentation Series Exit Survey Question 2 Exit Survey Question 3 Exit Survey Question 5 Exit Survey Question 6 NOT N/A N/A N/A NOT N/A N/A NOT N/A N/A N/A N/A N/A NOT N/A N/A N/A N/A N/A N/A N/A N/A xvi i

19 Intended Student Learning Outcomes: Master of Business Administration (cont) Exit Survey Question 7 Exit Survey Question 8 Exit Survey Question 13 Exit Survey Question 14 Exit Survey Question 16 Measure Measure Measure 1. Identify critical business issues facing organizations. 2. Analyze business issues from a cross-disciplinary perspective and form practical responses to these issues. 3. Assess the impact of global influences on business decisions. 4. Formulate ethical approaches to business decisions and understand how to apply these within a business context. 5. Communicate effectively in both a written and oral manner. N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A Proposed Courses of Action for Improvement in Learning Outcomes for which Targets Were Not Met: 1. Course of Action 1 Measuring Specific MBA Concentrations For our programs, we were not informed until 2014 that we were required to report separate outcomes for the MBA concentrations. All concentration courses at the MBA level are either in the process or scheduled for full analysis and revision (where needed) to develop appropriate objectives and measures. 2. Course of Action 2 MBA Third Party Testing After reviewing the first three years of third party testing results for the MBA program, it is clear that our performance objective of simply meeting or exceeding the national average is unrealistic. We are already working with the third party test administrator to understand what further information is available for the IACBE and national testing population and how we might better measure the success of our MBA program against like-sized schools and the national testing population overall. xvi ii