Degree/Certificate: Bachelor of Business Administration Degree Program Submitted by: Patricia Nemetz Mills Date: November 18, 2014 Part I Program SLO Assessment Report for 2013 14 Part I for the 2013 14 academic year: Assessment for business programs underwent substantial development during the 2013 14 academic year in order to comply with AACSB accreditation standards. An assessment plan for years 2013 18 has been filed with the CBPA Office of the Dean and with the university s AOL Director. It is available on request from the CBPA Assessment Coordinator. All future assessments during the time span 2013 18 will be conducted at the business program level, not at the major level. This report includes results for assessment of multiple student learning objectives. AACSB standards require student learning objectives to be assessed twice (including the closing the loop cycle) in a five year period. Closing of the Loop may occur during the same academic year as the initial assessment, but in a different quarter. Dates for each activity are indicated. 1. Student Learning Outcomes: The following student learning objectives were assessed during Winter Quarter 2014: Critical Thinking Students will use appropriate information and/or concepts and skills from the common body of business knowledge to bear upon the critical analysis of business issues and problems. 1. Using case studies, designed exercises, and/or real life examples, our students know how to use an analytical framework to apply the common body of knowledge to solve problems, resolve issues, or evaluate situations. 2. Our students will be able to reference appropriate information for use as supporting evidence and differentiate between fact, opinion, and extraneous data when producing reports, analyzing cases and issues, and solving problems. Ethical Awareness Students will develop an understanding of ethical issues that affect business operations, along with an awareness of various stakeholders affected by business activities. 1. Our students will know and understand relevant concepts and frameworks for making ethical decisions. 2. Using a case study, designed exercise, or investigatory report, our students will be able to apply an ethical framework to analyze an ethical dilemma or ethical violation. 3. Using a case study, designed exercise, or investigatory report, our students will be able to determine a variety of interests, differences, or conflicts arising among stakeholders affected by business activities. Written Communications Students will communicate effectively in writing. 1. Our students will write logical and clear reports and documents. 2. Our students will demonstrate knowledge and awareness of mechanical errors in their writing and learn to correct them. 3. Our students will know how to use application software to properly format documents and review their writing 1
2. Overall evaluation of progress on outcome: Indicate whether or not the SLO has been met, and if met, to what level. Critical Thinking SLO is met after changes resulting from ongoing assessments, referencing assessment results from the previous year to highlight revisions; X SLO is met, but with changes (collaboration) forthcoming; SLO met without change required Ethical Awareness SLO is met after changes resulting from ongoing assessments, referencing assessment results from the previous year to highlight revisions; X SLO is met, but with changes (collaboration) forthcoming; SLO met without change required Written Communication X SLO is met after changes resulting from ongoing assessments, referencing assessment results from the previous year to highlight revisions; SLO is met, but with changes (collaboration) forthcoming; SLO met without change required 3. Strategies and methods: Description of assessment method and choices, why they were used and how they were implemented. Critical Thinking a legal case was analyzed by students in several sections of ACCT 261. The collaboratively developed Critical Thinking rubric was used to evaluate the results. Ethical Awareness an ethical dilemma case was analyzed in MGMT 423 and various scenarios were analyzed in MGMT 326. The collaboratively developed Ethical Awareness rubric was used to evaluate the results. Written Communications Papers from several classes (MGMT 423 and MGMT 490) were collected and evaluated by a business faculty using the collaboratively developed Written Communications rubric. All rubrics referenced above are attached. 4. Observations gathered from data: Include findings and analyses based on the strategies and methods identified in item #3. a. Findings: Aggregate Summary Sheets are attached for all Assessments. The percentage of students meeting and/or exceeding the criteria on each rubric are reported on the respective summary sheets. Problem areas are identified in curriculum committees by reviewing and discussing possible reasons where percentages tend to be lower (a specific standard based on percentage of students 2
meeting the expectation has not yet been set, as this is a pilot year for evaluating the new assessment plan) b. Analysis of findings: Critical Thinking: i. Many students used a method compatible with critical thinking. The method employed for teaching was deemed as useful in aiding the critical thinking skills developed in students. In general, the results of the assessment were found to be adequate due to the high percentages of students meeting or exceeding expectations for each criterion. ii. Other instructors should be encouraged to use a similar method (IRAC, described below) as an instructional aid for critical thinking. Ethical Awareness: i. Many students followed a process for ethical analysis when making a decision. In general, the results of the assessment were found to be adequate due to the high percentages of students meeting or exceeding expectations for most criteria. Lower percentages for the Mission and Values and Universalism/Relativism criteria were probably related to the fact that the case did not address those issues as directly as the other issues, therefore, it would be pre mature to suggest that students lacked an adequate understanding of these issues. However, the percentage of students who applied a decision making or ethical framework (53%) is a bit low. ii. Instructors should be encouraged to share methods and techniques that enhance the use of an ethical framework or decision process when analyzing ethical dilemmas. Written Communications i. Few strengths were identified. ii. This assessment found the most critical problems of all the assessments completed W2014. iii. What program changes will be made based on the assessment results? a) Describe plans to improve student learning based on assessment findings (e.g., course content, course sequencing, curriculum revision, learning environment or student advising). Critical Thinking: Without a structure identified, some students may be at a loss for how to think about problems in a critical way. The IRAC method (Identification of Problem, Rule Application, Analysis, Conclusion) will be applied by different instructors across several sections of ACCT 261 (Business Law) in the future. The IRAC method of analyzing legal cases will be presented to students with an explanation of what is expected in each category. Practice assignments will be given for use with the IRAC method throughout the quarter. The IRAC method is the teaching improvement and is designed to pose an analytic framework for thinking about complex conceptual cases. An additional assignment will be given along with the Critical Thinking rubric toward the end of the quarter to determine an individual student's ability to apply IRAC method to a complex legal case. Ethical Awareness: Though many students followed a process, use of a decision framework was a shortcoming for some students. The following improvement was developed and will be applied Fall 2014: Decision framing worksheets will be presented to students along with short dilemma cases and/or exercises. Either working in groups or individually, students will be asked to use the practice worksheets to guide their thinking when analyzing each case. The practice worksheets are the teaching improvement and are designed to frame the decision for students using a systematic methodology. Worksheets samples are included for viewing on Canvas along with this Improvement log. Subsequent to using the worksheets 3
for practice, students will be asked to formally analyze a case individually and in written form. (Worksheets are attached) Written Communications To make improvements, a Writing Error Elimination Policy (WEEP) and standard format for writing formal papers was developed (document included as attachment). A Writing Assessment Board was set up for evaluating written assignments so the instructor doesn t have to take the heat for initiating the assessment for later re assessment. b) Provide a broad timeline of how and when identified changes will be addressed in the upcoming year. All listed improvement activities will be applied Fall 2014. Re assessment of the same SLOs will take place in Fall 2014 as well. iv. Description of revisions to the assessment process the results suggest are needed and an evaluation of the assessment plan/process itself. The plan and process appears to be adequately addressing the needs of the business program at this time. The process will continue to improve as more faculty understand the procedures and gain experience in making collaborative improvements. Attachments Included: Critical Thinking Rubric Ethical Awareness Rubric Written Communications Rubric Critical Thinking Summary Sheet Ethical Awareness Summary Sheet Written Communications Summary Sheet Ethical Awareness Worksheets (for Improvements) Writing Error Elimination Policy (for Improvements) 4
Critical Thinking Rubric Critical Thinking Learning Objective Students can use appropriate information and/or concepts and skills from the common body of business knowledge to bear upon the critical analysis of business issues and problems. College of Business and Public Administration Eastern Washington University Description of Situation and Problem Identification Student can clearly describe the situation and/or identify the problem Student can select which of the given information can be used as factual evidence to analyze the situation Student can differentiate between fact and opinion or can determine which of the given information is extraneous, based too much on feelings or opinion, or is too distorted to use without further exploration Information Exploration and Explanation Student can locate and/or reference concepts, theories, models, frameworks, formulae, rules, and/or standards for analyzing the situation Student considers alternate concepts, theories, models, frameworks, formulae, rules, and/or standards which may also be applied to analyze the situation Student can accurately explain, in his/her own words, relevant concepts, theories, models, frameworks, formulae, rules, and/or standards which may be applied to analyze the situation Analysis Student can apply the selected concepts, theories, models, frameworks, formula, rules, standards to analyze the specific situation Student defines meaningful alternatives if necessary Student evaluates alternatives and makes a recommendation or can clearly state a conclusion Exceeds Expectations (4) Meets Expectations (3) Marginally Meets Expectations (2) Does Not Meet Expectations (1) Limitations Student can clearly identify assumptions Student can clearly identify limitations of the applied concepts, theories, models, frameworks, formulae, rules, and/or standards Student can clearly identify limitations of the conclusion/recommendation Overall Evaluation: 5
Ethics Rubric Ethical Awareness Learning Objective --Students will develop an understanding of ethical issues that affect organizations along with an awareness of various stakeholders affected by the organization s activities. College of Business and Public Administration Eastern Washington University Exploration of Problem and Perspectives Exceeds Expectations (4) Meets Expectations (3) Marginally Meets Expectations (2) Does Not Meet Expectations (1) Student identifies the problem(s) or issue(s) Student identifies stakeholders and their interests or perspectives Student distinguishes among organization s stated or inferred mission, vision, and/or values Student Identifies Conflicts or Differences: a) Individual v. Organization v. Society b) Legal v. Ethical c) Short-Term v. Long-Term d) Ethically Universal v. Culturally Relative Ethical Decision-Making Student applies a decision-making process or ethical framework Student defines meaningful alternatives Student determines action or takes a considered position Student evaluates possible results or consequences Overall Evaluation: Instructions: After observing the exercise to be evaluated, place an X in the appropriate space for relevant criteria. If a criterion does not apply, leave the row blank. 6
Written Communications Rubric Written Communication Learning Objective -- Students are competent in written communications College of Business and Public Administration Eastern Washington University Content Category Focus Controls idea throughout the communication Understands purpose of the communication Completes all parts of the task Content Demonstrates knowledge of the topic Provides supporting evidence Avoids Plagiarism Organization Demonstrates an appropriate writing structure Groups information logically Creates appropriate transitions Uses appropriate format Composition Implements clear writing Uses concise composition Develops ideas adequately Language Use Demonstrates awareness of audience Displays professional tone Avoids use of slang Uses appropriate word choice Mechanics Reflects appropriate control of conventions Uses correct English grammar Free of spelling errors Utilizes appropriate referencing Use of Technology (If Applicable). If written document was prepared with application software, it Is formatted correctly, or as specified Is devoid of errors that could easily be found using software tools, like spell check or grammar check Looks professional Expectations Marginally Does Not Exceeds Meets Meets Meet (4) (3) (2) (1) Comments Exceeds expectations -- student demonstrates considerably more knowledge, skills, and abilities than what students in collegiate business programs are expected to know or do. Meets expectations -- student demonstrates sufficient knowledge, skills, and abilities or does what is expected of students in collegiate business programs Does not meet expectations -- students does not demonstrate sufficient knowledge, skills, and abilities, or misses some element of what is expected of students in collegiate business programs. 7
8
9
10
11
The Honda Auction Evaluate Mr. Conant s decision according to each of the following principles. Do you think his decision was considered ethical or unethical based on each of the following principles? Principles of Ethical Conduct Ethical Unethical 1 Categorical Imperative 2 Conventionalist Ethics 3 Disclosure Rule 4 Doctrine of the Mean 5 Ends-Means Ethics 6 Golden Rule 7 Practical Imperative 8 Intuition Ethics 9 Might-Equals-Right Ethics 10 Organization Ethics 11 Principle of Equal Freedom 12 Proportionality Ethics 13 Rights Ethics 14 Theory of Justice 15 Utilitarian Ethics 12
13
14
15
EWU s Writing Error Elimination Policy EWU s business programs are instituting a new policy to improve writing by holding students accountable for proofreading their papers, correcting their writing errors, and for learning how to format a paper (see page 2) for formal writing assignments. A writing assessment board has been created to monitor writing embedded in class requirements. This board will be reviewing one of your papers, multiple times if necessary, before your instructor grades the assignment for content required in the course. Instructors are required to follow these guidelines: 1. Fatal errors are listed below. A paper will be marked with errors indicated by the abbreviations, up until a total of 4 errors appear in the paper. If that number of errors is reached, the paper will be returned to the student for correction of ALL errors in the paper. The paper will not be graded for course content until the errors are corrected. It will be the student s responsibility to find all remaining errors in the paper and correct them. For example, if 4 errors are found in the first paragraph, the student has the work of correcting those errors and all remaining errors in the rest of the paper. 2. If a paper is returned to the student the first time, the paper automatically loses 10 points on a 100 point scale (or 0.5 points on a 4.0 grading scale). The maximum grade the student can attain is 90% (3.5 on a 4.0 grading scale). 3. The returned paper is then reviewed for writing again. If the entire paper has more than 2 writing errors, the paper will be returned to the student, and the maximum grade the student can earn on the paper for course content is 75% (2.0). If the paper has two errors, the student will lose an additional 5 points, so the maximum grade the student can earn for course content is 85% (3.0). 4. If errors are still present after the third submission, the paper will be returned to the student ungraded, and the student will receive a maximum of 65% (1.0) for the assignment. 5. Due dates will be determined by the instructor, and all papers submitted after the due date will not be eligible for re-evaluation. They will be graded 0.0 The following list includes errors to be avoided as part of this program: (1) MS = Misspelled word (2) R = Run-on/fragment/incorrect paragraphing (3) CO = Comma error (4) CP = Capitalization error (5) P = Punctuation error (6) A = Agreement error (7) MA = Missing or inappropriate article (a, the) (8) PR = Preposition error (9) WC = Word choice (10) SS = Sentence Structure (11) F = Formatting (12) O = Other grammatical error Proofreading Resources Students are encouraged to seek help before the initial submission of the assignment, if given permission by the instructor to do so. Students given such permission are encouraged to make use of the following resources: 1. Be sure to turn on spell-check and grammar-check in your document software. Correct the errors found. While this is helpful, it is not sufficient for catching all errors, and the 16
software may make a few incorrect changes. You must use your good judgment when using spell-check and grammar-check. 2. Read the proofreading advice at Purdue OWL: https://owl.english.purdue.edu/owl/resource/561/1/. Be sure to click on the Next arrow on the bottom to see specific advice for certain types of problems. 3. Use self-arranged peer reviews. See if someone in your class is willing to work with you as a team for checking the writing of each team member. 4. Use the CBPA adopted writing manual, HOW 13. 5. As a last resort, check with the Writers Center, http://access.ewu.edu/writers-center. Please understand that the Writers Center does not edit papers, but they may be willing to listen as you read your paper aloud and suggest places where errors exist. 6. Download free Ginger software at www.gingersoftware.com and integrate into your word processing software. The class instructor is not allowed to assist with editing of individual papers! Format If a specific format/organization is given by your instructor, you are to use that format. If a specific format for writing the paper is not given by the instructor, the following guidelines are to be used, even if specific questions are included as part of the assignment. Unformatted answers to the questions alone are not sufficient. The paper must be in a prose format, with proper paragraphs and organization, as follows: Standard Paper Format If None is Specified by Instructor Name, Date Due, Title of Paper (Clearly indicated) Introduction Should describe the purpose of the paper and a short summary or background of the information provided as the writing prompt. (The prompt is the assignment provided by the instructor, such as, a case, a set of questions, or a paper topic). It may also include a problem identification or situation description if warranted by the assignment. Normally the introduction is one short paragraph. Body of Paper -- This section of the paper should include analysis or provide supporting evidence for critically thinking about the subject matter. If the instructor has provided a rubric for grading the course content in the paper, it should be used as guidance or as an outline for developing your thoughts. If no rubric is provided, the student should develop the body of the paper by applying concepts learned in class to the specific assignment. Summary/Conclusion/Closure The final part of the paper should clearly indicate your concluding thoughts about the subject matter. It may include a brief summary statement of your analysis, a recommendation, a list of possible consequences, or your educated opinion of the subject matter. 17
If a different format is required by your instructor, you are to follow that format instead of using the above guidelines. 18
Writing Review Flowchart 1. Student submits paper Paper is forwarded to Writing Assessment Board If 4 or more errors If less than 4 errors Paper is graded for content by course instructor. Maximum possible grade is 100% (4.0) 2.Student revises paper and resubmits according to due date established by instructor. Late papers are not accepted. Paper is forwarded to Writing Assessment Board If more than 2 errors If 2 errors or less Paper is graded for content by course instructor. Maximum possible grade is 90% (3.5) for less than 2 errors or 85% (3.0) for 2 errors. 3.Student revises and resubmits paper. Late papers are not accepted Paper is forwarded to Writing Assessment Board If 2 errors or less Paper is graded for content by course instructor. Maximum possible grade is 75% (2.0) If more than 2 errors 19 Paper is graded for content by course instructor. Maximum possible grade is 65% (1.0)
NEW: PART II FOLLOW UP FROM THE 2012 13 PROGRAM ASSESSMENT REPORT Business programs have substantially revised their assessment procedures to be in alignment with AACSB accreditation requirements. Prior assessments were completed at the course level. Current assessments are completed at the program level. This portion of the assessment report closes out the assessments completed at the course level in 2012 13. 1. Student Learning Outcome(s) assessed for 2012 13 Goal #1: Ethics (assessed in MGMT 326 & MGMT 423) Students will develop an understanding of ethical issues that influence business operations along with an awareness of various stakeholders affected by business activities. Goal #2: Multiculturalism and Global Awareness (assessed in MGMT 423 & MKTG 310) Students will develop an awareness and understanding of the cultural issues that impact business operations in a global society. Goal #3: Teamwork and Collaboration (assessed in MGMT 326 & MGMT 490) Students will understand and use team building and collaborative behaviors to accomplish group tasks. Goal #4: Understanding Financial Statements (assessed in ACCT 251 & FINC 335) Students will understand and utilize financial tools and analytic techniques (e.g., financial statements analysis, budgeting, and valuation) to make and justify important financial decisions. Goal #5: Data Analysis Skills (assessed in DSCI 346 & OPSM 330) Students will identify and perform appropriate quantitative analyses when given a particular business problem. 2. Strategies implemented, Summary of Results, and Changes during 2013 14 to improve student learning, based on findings of the 2012 13 assessment activities. A summary of results and changes is included on the following pages. 20
UG Business Program Assessment Closing of the Loop Summary of Observations, Improvements, and Actions Taken 2012-2013 Qtr/Year of: Pilot Qtr/Year of: Re-Test Learning Objective Assessed Met Std 1 st Time? Observation/Improvement/Action Advised or Taken Champion(s) (List Committee and/or Instructor Applying Improvement) Effective? (2 nd Data Set Shows Improvement) F2012 W2013 Ethics No Re-evaluate lectures to determine how ethics/stakeholder concepts can be clearly communicated in MGMT 326. Additionally, small-group exercises that improve class interaction when this topic is covered may be used to enhance topic engagement. While cognitive coverage of ethics may be adequate for MGMT 326, MGMT 423 must emphasize ethical reasoning. Be sure to include ethics readings and exercises in MGMT 423. (Review syllabi and inform faculty of importance of including exercises in ethics reasoning). Provide more focus on text concepts and spend less time on discussions that go beyond the book. This should help students to master the basics. They may not be ready to engage in more analytical discussions. Provide an opportunity for students to test their knowledge so that they can see what they don t know. Hopefully, this will encourage them to either ask questions or reread the material. Emphasize the importance of reading the material. MGMT faculty Yes In some cases, additional improvement in scores is needed. Continue to monitor this learning objective. Assessment methodology is changed in W2014. Ethics Reasoning Rubric has been developed for use across classes 21
In addition, add more real world business practices such as sample Corporate Social Responsibility Reports to course content. These samples could help students to make connection between topics and actual business practices. Consequently, they would be more capable in applying topics. An additional pedagogical tool is to use a fishbowl technique to enhance student approaches to ethical reasoning Next, go beyond cognitive understanding; teaching only cognitive concepts about ethics is insufficient to many business situations; we need to teach ethical reasoning. Using only textbook cases is too limited since typical textbooks do not question the ethics of the ubiquitous business concept of maximizing shareholder return no matter the impact on other stakeholders. All of our classes that address this goal should include student practice in ethical reasoning and should employ a much wider application to many current business principles and practices. Sp2013 Develop Ethics Rubric for assessment of ethical reasoning. Revise the assessment approach so that reasoning is evaluated instead of simple cognitive knowledge. W2013 Sp2013 Teamwork No Overall, three particular issues stood out as priorities for improvement. First, students displayed extremely weak understanding of those qualities that distinguish effective team process. In other words, they don t understand the issues that affect team effectiveness. This makes it difficult for them to intervene constructively. Second, students do not Assessment Coordinator UG Program Committee/Instructors making extensive use of teams Rubric to be put into use W2014 Yes, though not as much as needed (14%- 23%). Additional Review Needed 22
understand how team goals can be made more meaningful for team member. And, finally, they do not appear to understand the observed approaches shared by successful teams. These three issues all relate to the broad idea that our students do not really understand the distinction between group work and teamwork, much less the difference between a group and a team. This is where improvement must begin. Instructors using teams should provide lectures with a focus on a) the distinction between a group ad a team, and b) the characteristics of effective team process. We need to really look at our use of groups in courses. We need to evaluate whether we are truly creating circumstances where teamwork is required, or whether group work is being used to simplify the instructor s grading work. Group work is clearly confusing the students about the true concept of teamwork and undermining the performance of our college on this objective. As opposed to the last assessment, students performed much better with respect to understanding how to make team goals more meaningful to individuals, and with respect to understanding those qualities that are typically exhibited by high performing teams. However, students still struggle with distinguishing teams from working groups. Additionally, the results indicate a poor understanding of the phases through which teams develop, and how teams influence individual behaviors. 23
In addition to lectures on task initiatives, professors should teach some tools and techniques of soliciting cooperation and expected-result-compliance from some malingering team members who have exhibited some behaviors of being hitch-hikers and free-loaders, including use of social controls and consultations with professors. W2013 Sp2013 Teamwork Yes Students need to be instructed on proper use of peer evaluations so that response bias in favor of high ratings is minimized. Peer evaluations must be confidential and the assessment should include professors scorings as well. Students should possible be penalized if they rate all team members high F2013 Teamwork Yes Team concepts need to be taught throughout the class instead of in a single class. Class attendance requirements when teams are used W2013 Sp2013 Financial Statements Yes is suggested Developed an interactive web page that will take the students through the steps of finding the yield to maturity of a bond that makes semiannual interest payments. The page also includes videos on how to calculate YTM using a Tl84 calculator, A TIBAIIPlus calculator, and a spreadsheet. The web page is available at http://finance.ewu.edu/fincc335/lectures/ross Westerfield Jordan/yeild to maturity.html. These tools should also be useful for traditional sections of 335. W2013 Multiculturalism Yes In marketing classes, though some concepts are well-understood, improvements can be made by placing more emphasis on word definitions and using more examples of Faculty making use of teams Faculty making use of teams Finance Faculty Marketing Instructors Yes Marginally better Re-test Pending; No improvement noted so far N/A 24