Assessment of Learning Structure for MBA Cluster Programs

Transcription

1 Assessment of Learning Structure for MBA Cluster Programs LCOB Mission Statement Overarching Goal for MBA Cluster Programs The Mission of the LCOB is to be a leading state institution for the education of business students, and a contributor to the region s economic development. The College is committed to an overall balance among teaching, scholarly activity, and services. The LCOB is dedicated to graduating individuals who possess the communication, problem solving and decision making skills, managerial knowledge, and leadership skills necessary to meet the Tri State area s needs for the demands of the global marketplace. Definition of MBA Cluster: MBA Cluster consists of thee programs: Regular MBA (USA), Executive MBA (USA), and Bhavan-Marshall MBA (India) OVERARCHING GOAL: To facilitate the attainment of managerial skills and knowledge required to achieve the strategic goals of organizations in a multicultural and international environment. MEASURABLE LEARNING OBJECTIVES Measurable Learning Objectives for MBA Cluster Programs Mechanism to Assess Each Learning Objective 1. Written Communications: The student will be able to communicate effectively in writing within a business environment. 2. Oral Communications: The student will be able to communicate orally in an effective and professional manner. 3. Problem-Solving & Decision Making: The student will be able to identify a problem, develop alternative solutions, and choose a course of action. 4. Managerial Knowledge: The student will demonstrate a broad managerial knowledge required to understand and function in the environment within which organizations operate. 5. Leadership Skills: The student will be able to develop leadership skills to support career growth and prepare for executive responsibilities and challenges. ASSESSMENT CYCLES Faculty Review the Data; Implement Strategies to Improve Learning Cycle 1: Written Communications (every other year starting 2008) Cycle 2: Oral Communications (every other year starting 2008) Cycle 3: Problem-Solving & Decision Making (every other year starting 2009) Cycle 4: Managerial Knowledge (every semester starting 2009) Cycle 5: Leadership Development (every other year starting 2009) 1

2 Table 1 - Assessment Cycles for Measurable Learning Objectives for MBA Cluster Programs Measurable Learning Objectives Written Communication Oral Communication Problem Solving/Decision Making Managerial Knowledge Leadership Skills (I) Development Phase (II) Measurement Phase (III) Continuous Improvement Closing the Loop Develop Learning Objectives Fall 2006 Fall 2008 EMBA 2008 Fall 2008 Fall 2008 Develop Measure Summer 2007 Spring 2009 Spring 2009 Spring 2009 Spring 2009 Gather Data Designated classes Spring 2009 and Fall 2009 Designated classes Spring 2009, and Fall 2009 Designated classes Spring 2009 and Fall 2009 Designated classes Spring 2009 and Fall 2009 Designated classes Spring 2009 and Fall 2009 Review Data External Expert External Expert Fall 2009 and Fall 2009 and Fall 2009 and Fall 2009 and Action Plan MBA Faculty MBA Faculty MBA Faculty MBA Faculty MBA Faculty Implement Changes Graduate Comm., MBA Faculty Summer 2010 and Fall 2010 Graduate Comm., Summer 2010 and Fall 2010 Graduate Comm., Summer 2010 and Fall 2010 Graduate Comm., Summer 2010 and Fall 2010 Graduate Comm., Summer 2010 and Fall 2010 Re gather Data Fall 2010 Fall 2010 Fall 2010 Fall 2010 Fall

3 Assurance of Learning Process Diagram Stage I LCOB Mission Program Overarching Goal Stage II Learning Objectives Approved by Faculty Rubric for Each Learning Objective: Developed and Approved by Faculty Stateg III Collection of AOL Data from MBA Cluster Courses Development and Implementaiton of Improvement Strategies Collection of AOL Data after the Program Improvement 3

4 The following diagram illustrates what is an AOL Loop (Cycle). Assessment Loop Learning Objective Assessment after Implementation Assessment Via Rubric Implementation of Improvement Strategies Analysis of Assessment Data 4

5 The following table shows how each Learning Objective is assessed based on course-embedded activities, assignments, projects, etc. Table 2: The MBA Cluster: Alignment of Course-Embedded Activities with Learning Objectives for Collecting AOL Data Course/Learning Objectives Written Communications Oral Communications Problem Solving & Decision Making Managerial Knowledge Leadership Skills MGT601: Quant. Methods X X ACC613: Profit Planning & Controls X X X FIN620: Finance Management X X ECN630: Managerial Economics X X MGT672: Organizational Behavior X X X X MGT674: Production/Operations X X MGT MIS678: Management Info. Systems X X MKT682: Adv. Marketing Management X X X X X LE691: Government/Business relations X MGT699: Business Policy X X X X X 5

6 The MBA Cluster: Rubric for Written Communications Circle one: MBA EMBA IMBA Semester Year Course#/Title Evaluator Student# Criteria Unacceptable (1) Needs Improvement (2) Proficient (3) Advanced (4) Content Neither purpose nor thesis clearly present. Information seems random. Analysis is vague or not evident. Reader is confused or misinformed. Organization Writing lacks logical organization. Reader cannot identify a line of reasoning. Purpose Tone Word Choice Grammar and mechanics (sentence structure, punctuation, spelling) Purpose is generally unclear and is not stated. Tone is neither professional nor appropriate to genre, audience, and purpose. Many words are used inappropriately, confusing the reader. Inappropriate cliches and colloquial diction. Sentence structure and grammar errors so numerous confused reader stops reading. Information supports thesis at times. Analysis is basic or general. Reader gains few insights. Writing is not arranged logically. Frequently, ideas fail to make sense together and are not linked. By thinking hard, reader can most often figure out the line of reasoning. Purpose is not clearly stated, nor is it easy to find an implied purpose. Tone is neither consistently professional nor appropriate to genre, audience, and purpose. Word choice is adequate, but range is limited and some words are used inappropriately. Confusing sentences. Numerous errors distract reader. Information provides firm support for thesis and displays evidence of a basic analysis of a sufficiently limited topic. Reader gains some insights. Ideas are arranged to support the writer s purpose, but order is sometimes questionable. Ideas are usually clearly linked to each other. For the most part, reader can follow line of reasoning. Writing has a stated purpose, but may occasionally digress. Tone is generally professional, but may have a few lapses in fit or appropriateness to genre and audience. Word choice is generally precise, effective, and appropriate to genre and purpose. Functional sentences. Errors present but do not distract from meaning. Even, balanced information effectively supports a central purpose or thesis and displays a thoughtful in-depth analysis of a topic. Reader gains insights. Ideas are arranged logically to support the purpose. Ideas flow smoothly from one to another and are clearly linked to each other. Reader can follow reasoning. Writer s purpose is readily apparent to the reader and is stated clearly. Tone is consistently professional and appropriate for the audience and the purpose. Fits genre and expected audience for the genre. Word choice is consistently precise, effective, and appropriate to genre and purpose. Sentences vary in length, structure; flow well; demonstrate transitions. Writing almost error free. Use of Evidence is not cited to Although occasional evidence Evidence to support claims is Compelling evidence is given 6

7 Research Data Presentation (Charts, Appendixes, Tables, Graphs Visual Presentation of Data support claims. No citation present. No data or totally irrelevant data. Visual data presentation fulfills none of the previous 3 things. is provided, writer relies too heavily on unsubstantiated statement. The reader is confused about the source of ideas. Many errors in citation format. Data only occasionally integrated into purpose. Extraneous data present. Visual data presentation fulfils 1 out of previous 3 things. generally present. Attribution is often clearly given. Some inconsistencies in format of citation. Some data presented and logically integrated into main purpose. Visual data presentation fulfills 2 out of previous 3 things. to support claims. Attribution is clear and fairly presented. Consistently correct and appropriate format of citation. Sufficient data presented and all data logically integrated into the main purpose of the paper. All visual data presentation does three things: 1.compels attention 2. is concise and 3. is understandable. Rubric Score Classification: Unacceptable: 9-18 Needs Improvement: Proficient: Advanced: 36 7

8 The MBA Cluster: Rubric for Oral Communications Circle one: MBA EMBA IMBA Semester Year Course#/Title Evaluator Student# Performance Criteria Below Expectations 1 Meets Expectations 2 Exceed Expectations 3 Score Organization Inadequate or illogical flow. Ill-defined or no discernable purpose. Support info was unclear or incorrect. Lacked basic understanding of topic, audience and flow. Some incidences where lack of logical flow and incomplete understanding of topic was evident. Purpose was sometimes difficult to discern. Support info was not easily understood. Logical flow. Purpose and support info was easily understood. Demonstrated thorough understanding of topic, audience and flow. Content Accomplished assignment only partially or indirectly. No supporting data or data presented was not relevant or accurate. Accomplished assignment in a capable manner. Major topics covered but supporting data lacks specificity, accuracy, or relevance. Accomplished assignment directly and completely in an exceptional manner. All major topics covered and supported by relevant data. Delivery Exhibited extremely low level of enthusiasm & confidence. Was not able to effectively answer questions. Uneven levels of enthusiasm & confidence exhibited. Some questions were answered more effectively than others. Exhibited high level of enthusiasm & confidence. Responded fully and accurately to questions. Generated audience interest and interaction Projection Non-Verbal Technology Projection consistently detracted from presentation. (Speech was too slow/fast, could not be heard, used fillers such as uhm, incorrect pronunciation) Non-verbals consistently detracted from presentation (read from notes, monitor or screen; made no eye contact; no/excessive movement, stood behind podium or in front of screen) Visuals were unclear, unattractive, or unprofessional. Did not support presentation and contained several spelling and/or grammatical errors. Pace, volume, & enunciation were acceptable but did not enhance the presentation Eye contact, gestures, and movement occasionally distracted from the presentation Visuals used distracting slide design (template, font, clip art) There was at least one spelling or grammatical error. Pace, volume, & enunciation enhanced the presentation Eye contact, gestures, and movement were used effectively Visuals were clear and professional Visuals reinforced the presentation There were no spelling or grammatical errors. Appearance Unprofessional attire and demeanor. Seated presenters exhibited distracting behavior during presentation. Acceptable professional attire and demeanor Highly professional attire and demeanor enhanced credibility of the speaker. Check one: Below Expectations = 7-13; Meets Expectations = ; Exceeds Expectations = 21 8

9 The MBA Cluster: Rubric to Assess Problem Solving/Decision Making Circle one: MBA EMBA IMBA Semester Year Course#/Title Evaluator Student# 1:Unacceptable 2: Needs Improvement 3: Acceptable 4: Exemplary Score Problem Identification Interpretation Does not clearly identify the problem; Identifies an inappropriate problem or represents the issue inaccurately Fails to question data; Ignores bias; Misses major content areas; Detects no inconsistencies; Chooses biased sources. Rudimentary problem identification; Identifies main problem but does omits relevant issues; Does not identify the relationship between different aspects or issues within the problem. Identifies some questions; Notes some bias; Recognizes basic content; States some inconsistencies; Selects sources adequately. Clearly identifies main problem and includes some of the subsidiary issues; Some discussion or relationships between subsidiary issues Asks insightful questions; Detects bias; Categorizes content; Identifies inconsistencies; Recognizes context. Clearly identifies the main problem and subsidiary, embedded or implicit aspects of the problem; Clearly addresses the relationship among subsidiary issues; Identifies not only the basics of the issue but also recognizes subtle nuances of the issue Analyzes insightful questions; Refutes bias; Critics content; Examines inconsistencies; Values information Identification of Alternatives Analysis & Evaluation Presentation Does not identify any alternative to solve the problem; Has no understanding of the value of alternatives and their relationship to the identified problem. Fails to draw conclusions; Sees no arguments; Overlooks differences; Repeats data; Omits research Omits argument; Misrepresents issues; Excludes data; Draws faulty conclusions; Shows intellectual dishonesty Identifies alternatives that reflect a limited understanding of the situation. Identifies some conclusions; Sees some arguments; Identifies some differences; Paraphrases data; Assumes information valid. Misconstrues arguments; Generalizes issues; Cites sources; Presents few options; Overlooks some information Identifies alternatives that reflect a basic understanding of the situation and the problem. Formulates conclusions; Recognizes arguments; Notices differences; Evaluates data; Seeks out information Argues clearly; Identifies issues; Attributes sources Suggests solutions; Incorporates information. Identifies alternatives that reflect an in-depth understanding of the situation and the problem. Examines conclusions; Uses reasonable judgment; Discriminates rationally; Synthesizes data; Vies information critically Argues succinctly; Discusses issues thoroughly; Shows intellectual honesty; Justifies decisions; Assimilates information. Total Points: Scoring: Check one: Unacceptable: 5-9 ; Needs Improvement: ; Acceptable: ; Exemplary: 20 9

10 The MBA Cluster: Rubric to Assess Managerial Knowledge MGT601 ACC613 FIN620 ECN630 MGT672 MGT674 MIS678 MKT682 LE691 MGT699 Key for MBA Core Curriculum Quantitative Methods Profit Planning & Controls Financial Management Managerial Economics Organizational Behavior Production/Operations Management Management Information Systems Advanced Marketing Management Business & Government Relations Business Policy & Strategy Direct Measures: Course-embedded assessment from each MBA course using individually-assigned assignments, papers, etc. 10

11 Circle one: MBA EMBA IMBA The MBA Cluster: Rubric to Assess Managerial Knowledge Semester Year Course#/Title Evaluator Student# MANAGERIAL KNOWLEDGE MGT601 ACC613 FIN620 Unacceptable 1 Grade Points of 69 and below quantitative methods; not able to apply any of them in business Accounting; not able to apply any of them in business Finance; not able to apply any of them in business Acceptable 2 Grade Points between 70 to 79 tools of quantitative methods; some ability to apply them in business tools of Accounting; some ability to apply them in business tools of Finance; some ability to apply them in business Proficient 3 Grade Points between 80 to 89 and tools of quantitative methods; adequate ability to apply them in business and tools of Accounting; adequate ability to apply them in business and tools of Finance; adequate ability to apply them in business Exemplary 4 Grade Points 90 and above and tools of quantitative methods; skillfully able to apply them in business and tools of Accounting; skillfully able to apply them in business and tools of Finance; skillfully able to apply them in business Score ECN630 MGT672 Managerial Economics; not able to apply any of them in business Organizational Behavior; not able to apply any of them in business tools of Managerial Economics; some ability to apply them in business tools of Organizational Behavior; some ability to apply them in business and tools of Managerial Economics; adequate ability to apply them in business and tools of Organizational Behavior; adequate ability to apply them in business and tools of Managerial Economics; skillfully able to apply them in business and tools of Organizational Behavior; skillfully able to apply them in business 11

12 MGT674 MIS678 MKT682 Production and Operations Management; not able to apply any of them in business Management Information Systems; not able to apply any of them in business Marketing Management; not able to apply any of them in business tools of Production and Operations Management; some ability to apply them in business tools of Management Information Systems; some ability to apply them in business tools of Marketing Management; some ability to apply them in business and tools of Production and Operations Management; adequate ability to apply them in business and tools of Management Information Systems; adequate ability to apply them in business and tools of Marketing Management; adequate ability to apply them in business and tools of Production and Operations Management; skillfully able to apply them in business and tools of Management Information Systems; skillfully able to apply them in business and tools of Marketing Management; skillfully able to apply them in business LE691 MGT699 Legal Environment for Business; not able to apply any of them in business Little or no understand of and ability to integrate and apply key concepts, principles, and tools from various business functions to solve business problems. tools of Legal Environment for Business; some ability to apply them in business Some understanding of ability to integrate and apply and tools from various business functions to solve business problems. and tools of Legal Environment for Business; adequate ability to apply them in business ability to integrate and apply and tools from various business functions to solve business problems. and tools of Legal Environment for Business; skillfully able to apply them in business ability to integrate and apply and tools from various business functions to solve business problems. 12

13 The MBA Cluster: Rubric to Assess Leadership Skills Assessment Method: Peer Evaluation The following are activities that leaders do. Please rate each of your team members on a scale below. Please DO NOT evaluate yourself. Please read the following carefully and complete the form in private. We guarantee that your assessment will be kept confidential. Only the authorized faculty/administrator will have access to it with your name removed. DO NOT fill out this form in the presence of any student in this or any other class. Thank you for your cooperation. Scale to assess leadership skills of your team members: Score of 4: Exemplary is the level of performance expected of an effective leader. More than expected leadership performance. Score of 3: Proficient is the level of performance expected of initial (beginner) MBA Candidates; Satisfactory or average leadership performance. Score of 2: Needs Improvement indicates limited evidence of meeting a leadership standard; leadership performance is Inconsistent or inadequate. Score of 1: Unacceptable: There is little or no evidence of any leadership skills; did not meet a leadership standard. Total Scale Unacceptable = 9-17 Needs Improvement = Proficient = Exemplary = 36 13

14 The MBA Cluster: Rubric to Assess Leadership Skills Circle one: MBA EMBA IMBA Leadership Skills Team Member #1 Print Full Name Team Member #2 Print Full Name Team Member #3 Print Full Name Team Member #4 Print Full Name 1.Worked to build the entire team on the basis of trust, respect, and fairness. 2.Clarified the purpose of the team, fostered a collective sense of the mission of the team, and talked up trust. 3.Put suggestions made by the group into operation 4.Was friendly and approachable, looked out for the personal welfare of group members, and did little things to make it pleasant to be a member of the group. 5.Took the lead in scheduling work to be done, assigning group members to particular tasks, and deciding what should be done and how it should be done. 6.Listened attentively to other team members, provided advice, treated others with respect, developed the strengths of others, and taught and coached other team members. 7.Was willing to make changes in how he/she worked in order to help the entire team. 8.Understood the impact of his/her decisions on the decisions of other team members and vice versa. 9.Was optimistic about the future of the team, articulated a vision of where the team was going, envisioned new possibilities for the team, and set high standards for the team. Total Score on Nine Leadership Behaviors for Each Member 14

15 Lewis College of Business The MBA Cluster: Assurance of Learning Completion of AOL Loop for Leadership Skills in Summer 2009 to Collection of AOL Data on Leadership Skills MGT699: Summer 2009 Collection of AOL Data on Leadership Skills ACC613: Review and Analysis of AOL Data on Leadership Skills Implemention of Improvement Strategies Fall 2009 Development of Improvement Strategies by faculty Improvement Strategies: The MBA Cluster faculty developed and approved the Improvement Strategies for Leadership Skills in the Executive MBA Program. The basis for the improvement strategies was the results from AOL data analysis, student feedback, faculty input, and benchmarking. The approved strategies are as follows: 1. Extensive coverage on Leadership in MGT672: Organizational Behavior 2. Incorporation of a required non-credit workshop on leadership skills and team-building conducted by ROTC Officers/Staff at Marshall University. The workshop was scheduled prior to the start of the Class of 2011, the current cohort. 3. Corporate leaders serving as guest speakers in MGT601: Quantitative Methods and ACC613: Profit Planning and Control. Results of Improvement Strategies: The following table displays the summary results from Assessment#1 (prior to the Implementation) and Assessment#2 (after the Implementation). 15

16 Leadership Skills in the EMBA Program: Summary Results of the Loop Rubric Categories/Assessment Loop Assessment #1 (Prior to Implementation): AOL Data from MGT699: Class of 2009: 28 Students Assessment#2 (After Implementation) AOL Data from ACC613: Class of 2011: 31 Students Exemplar 7 (25%) 4 (13%) Proficient 19 (67%) 27 (87%) Needs Improvement 1 (4%) 0 Unacceptable 1 (4%) 0 As can be noted above, the improvement strategies have significantly improved the percentages in the category of Proficient, with a slight drop in the Exemplar category. 16