Follow-Up Program Review. Bachelor of Business Administration in Management Information Systems. College of Business. March 2014 MARSHALL UNIVERSITY

Transcription

1 Follow-Up Program Review Bachelor of Business Administration in Management Information Systems College of Business March 2014 MARSHALL UNIVERSITY

2 The Marshall University Board of Governors requested that the Bachelor of Business Administration Degree in Management Information Systems provide it with an updated assessment report for academic year during its review of programs in March Attached is the program s assessment report for academic year , as well as results for the fall of 2013.

3 Part 1 Relationships of Program s Mission to Missions of the College of Business and Marshall University

4 BBA Mangement Information Systems Program Information Point of Contact M. Shane Tomblin Support for University and College Missions Marshall University is a multi-campus public university providing innovative undergraduate and graduate education that contributes to the development of society and the individual. The University actively facilitates learning through the preservation, discovery, synthesis, and dissemination of knowledge. University Mission Programs must state their Program Mission and specify how this mission supports both the University's Mission as well as any Mission that the College the Program is housed under happens to have. College Mission The Mission of the College of Business (COB) is to be a leading state institution for the education of business students, and a contributor to the regions economic development. The College is committed to an overall balance among teaching, scholarly activity, and service. The COB is dedicated to graduating individuals who possess the communication, critical thinking, and problem solving skills necessary to meet the Tri-State areas needs for the demands of the global marketplace. Program Mission The mission of the Management Information Systems program is to graduate individuals possessing the knowledge, skills, and abilities necessary to be leaders and decision makers in the area of business and organizational technology. Support for College Mission As a part of the College of Business, the MIS program provides both the general business and discipline-specific knowledge as a part of a quality education for its students. MIS educators remain current in their discipline through application of widely accepted and innovative

5 teaching methods in graduate and undergraduate courses, conduct of research, and service to the College, University, and community. Support for University Mission The MIS program offers up-to-date graduate and undergraduate education as a part of a variety of programs both within the College and in cooperation with University programs outside the College. MIS Faculty are actively engaged in research with affects instruction and curriculum development.

6 Part II Program s Assessment Plan 1. Program Learning Outcomes 2. Courses in which program learning outcomes are assessed (at two assessment points). 3. Assessments used to collect data. 4. Benchmarks set for each assessment point.

7 Learning Outcomes Effective Written Communication: Compose clear, concise, reasoned individual and/or group paper which prove the extent to which learned concepts, theories, and practices are usefully applied and used. Assessment Point 1 Embedded Assessment Expected Benchmark Level MIS 310/340 Individual writing assignment/group project Capstone Assessment Point 2 Embedded Assessment Expected Benchmark Level MIS 475/MGT 460 Group project/written case Capstone Effective Oral Communication: Verbally present educated thoughts and opinions in clear, concise and reasoned manner. Assessment Point 1 Embedded Assessment Expected Benchmark Level MIS 310/340 Individual/group presentation Capstone Assessment Point 2 Embedded Assessment Expected Benchmark Level MIS 475/MGT 460 Group project/case presentation Capstone

8 Teamwork/Collaboration: Work effectively as a member of a project team/group toward the identification, selection, and production of technology-related business systems. Assessment Point 1 Embedded Assessment Expected Benchmark Level MIS 310/340 Group project Capstone Assessment Point 2 Embedded Assessment Expected Benchmark Level MIS 475 Group project Capstone Systems Thinking: To apply and use part-whole relationships in the analysis, discussion, and representation of artifacts, infrastructure, and technology. Assessment Point 1 Embedded Assessment Expected Benchmark Level MIS 300/310/333 Programming project/systems analysis project/infrastructure design project Milestone Assessment Point 2 Embedded Assessment Expected Benchmark Level MIS 475 Project or case analysis Capstone

9 Diagrammatic Representation and Reasoning: Use and develop diagrams commonly found in the representation of systems, infrastructure, architecture, database, and programming. Assessment Point 1 Embedded Assessment Expected Benchmark Level MIS 310/333/340 Individual assignment Milestone Assessment Point 2 Embedded Assessment Expected Benchmark Level MIS 475 Individual assignments/group project Capstone

10 Part III Rubrics

11 Effective Written Communication: Compose clear, concise, reasoned individual and/or group paper which prove the extent to which learned concepts, theories, and practices are usefully applied and used. Traits Performance Levels Introductory Milestone Capstone Advanced Organization Writing lacks logical organization. Reader cannot identify a line of reasoning. Purpose Tone Grammar and mechanics (sentence structure, punctuation, spelling) Purpose is generally unclear and it is not easy to find an implied purpose. Tone is not appropriate to audience and purpose. Confusing sentences. Numerous errors distract reader. Writing is not always arranged logically. Ideas sometimes fail to make sense together and are not always linked. By thinking hard, reader can figure out the line of reasoning. The writer s purpose is clear, but digressions distract the reader and detract from the meaning. Tone is not consistently appropriate to audience or purpose. Sentences are for the most part functional. Errors, at times, make it hard to follow the meaning. Ideas are arranged to support the writer s purpose, but order can at times be questioned. Ideas are usually clearly linked to each other. Reader can generally follow line of reasoning. Writing has a clear purpose, but may occasionally digress. Tone is generally consistent, but may have a few lapses in fit or appropriateness to audience or purpose. Sentences are functional. Errors are present but do not distract from the meaning. Ideas are arranged logically to support the purpose. Ideas flow smoothly from one to another and are clearly linked to each other. Reader can follow the line of reasoning. Writer s purpose is readily apparent to the selected audience and is clear. Tone is consistently professional and appropriate for the selected audience and the purpose. Sentences are wellconstructed. They vary in length and structure, and flow well. Writing is almost error-free, and reads well.

12 Effective Oral Communication: Verbally present educated thoughts and opinions in clear, concise and reasoned manner. Traits Performance Levels Introductory Milestone Capstone Advanced Clear Topic Coverage The speaker presents a topic and a focus that are inappropriate for the purpose, time constraints, or intended audience. The speaker presents a topic or a focus that is inappropriate for the purpose, time constraints, or intended audience. The speaker presents a topic and a focus that are appropriate for the purpose, time constraints, and audience. The speaker presents a topic and a focus that are exceptionally appropriate for the purpose, time constraints, and Organization The speaker fails to use an introduction or conclusion and fails to provide a reasonably clear and logical progression within and between ideas. The structure of the presentation interferes at times with the clarity and logic of the ideas being presented. By paying close attention, the listener can follow the presentation. The speaker uses an appropriate introduction and conclusion and provides a reasonably clear and logical progression within and between ideas. audience. The speaker uses an exceptional introduction and conclusion and provides an exceptionally clear and logical progression within and between ideas. Audience Appropriateness Voice Quality The speaker uses unclear or inappropriate language that would result in substantial miscommunication with the intended audience. Presenter mumbles or speaks too quietly for all audience members to hear. Delivery is forced with little or no vocal variety. The speaker uses some inappropriate language that would distract the intended audience. Delivery contains some instances where speaker could not be heard or understood. Voice quality is erratic. The speaker uses language that is reasonably clear and appropriate for the intended audience. Delivery is mostly clear and natural. Speaker can be heard and understood. With few exceptions, voice quality is used to emphasize points. The speaker uses language that is exceptionally clear and appropriate for the intended audience. Delivery is clear, natural, conversational, and enhanced by good vocal variety. Vocal quality is used effectively to emphasize points and reinforce structure.

13 Grammar/Word Choice Eye Contact and Body Language Communication Aids Personal Appearance Listeners are distracted by grammatical errors and inappropriate word choice. Sentences do not flow well. Distracting body language and lack of eye contact prevent effective communications with audience. Communication aids distract or are used inappropriately, are difficult to read, and contain unimportant information. Inappropriate for audience and/or topic. Grammar errors, incomplete sentences, run-on sentences, and poor word choice occasionally distract the listener. Body language and lack of eye contact reflect reluctance to interact with audience. Communication aids have numerous errors that interfere with effectiveness of presentation. Questionable for audience and/or topic. Some grammatical errors; some incomplete sentences or run-on sentences; some poor word choice. Body language and eye contact reflect some discomfort interacting with audience. Communication aids contribute overall but have errors, are difficult to read, or include too many or too little information. Appropriate for audience and/or topic. Insignificant grammatical errors and appropriate word choice. Sentences flow together well. Body language and eye contact indicate that the speaker is at ease interacting with audience. Communication aids enhance presentation. Aids appear professional and important information is included. Highly professional appearance.

14 Teamwork/Collaboration: Work effectively as a member of a project team/group toward the identification, selection, and production of technology-related business systems. Traits Performance Levels Introductory Milestone Capstone Advanced Team/ Collaboration Tools Preparation/Task Performance Communication/ Information Sharing Climate/Cohesion Little or no use of relevant team/collaboration tools. Unable to execute team processes using relevant tools. Unprepared for meeting/collaboration. Little or no effort expended on assigned responsibilities. Lack of communication. Information not shared with designated or relevant team members. Little or no effort to identify relevant targets of communication/informati on sharing. Frequently misses meetings. Disrespectful of member contributions. No assistance provided to other members. Negative attitude about team goal/task. Beginner-level understanding and use of team/collaboration tools. Difficulty in executing team processes using relevant tools. Incomplete preparation for meeting/collaboration. Assigned responsibilities incomplete. Beginning ability to communicate and share information with designated or relevant team members. Needs assistance in identifying targets of communication/informa tion sharing. Attends meetings. Respectful of member contributions. Provides requested assistance to other team members. Positive attitude toward team goal/task. Appropriate choice, understanding and use of team/collaboration tools. Able to execute team processes using relevant tools. Complete preparation for meeting/collaboration. Assigned responsibilities complete. Efficient and complete communication and information sharing with designated or relevant team members. Able to identify targets of communication and information sharing. Attends and contributes to all meetings. Respectful and encouraging of other member contributions. Actively provides assistance, even when not requested. Positive to enthusiastic attitude toward team goal/task. Superior choice, use, and understanding of team/collaboration tools. Able to execute team processes in innovative and productive ways using team/collaboration tools. Superior preparation for meeting/collaboration. Assigned tasks show superior ability. Superior communication and information sharing. Team member regarded as point of contact for communication/information sharing. Superior ability to identify targets of communication/information sharing. Attends meetings and anticipates needed contributions. Proactively encourages contributions of other members. Seeks opportunities to provide assistance to other team members.

15 Systems Thinking: To apply and use part-whole relationships in the analysis, discussion, and representation of artifacts, infrastructure, and technology. Traits Performance Levels Introductory Milestone Capstone Advanced Systems Concepts Difficulty in matching and applying systems concepts in analysis, discussion, and representation. Frequent by inconsistent application of systems concepts to analysis, discussion, and representation. Efficient application of systems concepts to analysis, discussion, and representation. Efficient and consistent application of systems concepts to analysis, discussion, and representation. Functional Decomposition Transfer Little or no ability to decompose target into part-subpart hierarchy. Lack of understanding of sub-part relationship within super-part. Little or no ability to assemble sub-parts into coherent whole. Unable to recognize similarity of target system to previous systems. Unable to relate target system to existing systems. Unable apply systems relationships and learning from previous systems. Inability to explain system objects and relationships to others. Difficulty in decomposing target into sub-part hierarchy. Beginning understanding of subpart relationships within super-part. Beginning ability to assemble subparts into coherent whole. Inconsistent recognition of target system similarity to previous systems. Incomplete relation of target system to existing systems. Incomplete application of systems relationships and learning from previous systems. Inconsistency in explaining system objects and relationships to others. Able to decompose target into part/sub-part hierarchy. Understanding of subparts within super-parts. Efficient assembly of sub-parts into coherent whole. Recognition of target system similarity to previous systems. Complete relation of target system to existing systems. Complete application of systems relationships and learning from previous systems. Consistent explanation of system objects and relationships to others. Efficient and comprehensive decomposition of target into part/sub-part hierarchy. Thorough understanding of relationships of part/subpart relationships. Superior ability to assemble subparts into functioning whole. Ready and complete recognition of target system similarity to previous systems. Ready and complete relation of target system to existing systems. Superior application of systems relationships and learning from previous systems. Superior explanation of system objects and relationships to others.

16 Diagrammatic Representation and Reasoning: Use and develop diagrams commonly found in the representation of systems, infrastructure, architecture, database, and programming. Traits Performance Levels Tools Composition/Mechanics Semantics/Syntax Mapping/Recognition Unable to identify and utilize relevant and appropriate tools for the construction and representation of diagrams. Ill-defined and inadequately depicted entities. No conveyance of relationships between entities. Diagram fails to convey meanings and relationships. Improper use of shapes, lines, annotations, and indicators. Diagram not usable by other viewers/users. User is unable to recognize and explain nature of entities and relationships. Inability to associate diagram constructs with actual system. Beginning but inconsistent ability to identify and utilize relevant and appropriate tools for the construction and representation of diagrams. Defined and depicted entities. Conveyance of relationships between entities. Confused meanings and missing or confused relationships. Inconsistent use of shapes, lines, annotations, and indicators. Diagram not easily understood by others without guidance. Inconsistent ability to recognize and explain nature of entities and relationships. Difficulty in associating diagram constructs with actual system. Consistent ability to identify and utilize relevant and appropriate tools for the construction and representation of diagrams. Adequately defined and depicted entities. Adequate conveyance of relationships between entities. Adequate conveyance of meanings and relationships. Consistent use of shapes, lines, annotations, and indicators. Diagram conveys meanings and relationships with assistance. Consistent ability to recognize and explain nature of entities and relationships. Consistent ability to associate diagram constructs with actual system. Consistent and knowledgeable in the identification and utilization of relevant and appropriate tools for the construction and representation of diagrams. Well-defined and welldepicted entities. Superior conveyance of relationships between entities. Clear, concise, and thorough conveyance of meanings and relationships. Exemplary use of shapes, lines, annotations, and indicators. Diagram conveys meaning even to non-technical viewers/users. Easy recognition and explanation of nature of entities and relationships. Ease in associating diagram constructs with actual system.

17 Part IV Assessment Results, Analysis and Planned Actions: Academic Year

18 Assessment Results, Analysis, and Planned Action Effective Written Communication: Compose clear, concise, reasoned individual and/or group paper which prove the extent to which learned concepts, theories, and practices are usefully applied and used. Assessment Point 1 Trait Below Intro Milestone Capstone Advanced Total Organization 0 (0.00%) 0 (0.00%) 9 (75.00%) 3 (25.00%) 0 (0.00%) 12 (100.00%) Purpose 0 (0.00%) 0 (0.00%) 2 (16.67%) 10 (83.33%) 0 (0.00%) 12 (100.00%) Tone 0 (0.00%) 0 (0.00%) 0 (0.00%) 12 (100.00%) 0 (0.00%) 12 (100.00%) Grammar and mechanics (sentence structure, punctuation, spelling) 0 (0.00%) 0 (0.00%) 2 (16.67%) 10 (83.33%) 0 (0.00%) 12 (100.00%) Analysis & Planned Action: The small sample size is explained by the small size of the class. All students were used for this class in the sample. Indication that students are not at the desired level of achievement, which should be 75-80% at Capstone level on Organization. Performance is better in the other areas, but the fit between the Communication Rubric and the assignment assessed raises questions. Assessment Point 2 Trait Below Intro Milestone Capstone Advanced Total Organization 0 (0.00%) 0 (0.00%) 4 (33.33%) 3 (25.00%) 5 (41.67%) 12 (100.00%) Purpose 0 (0.00%) 0 (0.00%) 1 (8.33%) 0 (0.00%) 11 (91.67%) 12 (100.00%) Tone 0 (0.00%) 0 (0.00%) 0 (0.00%) 2 (16.67%) 10 (83.33%) 12 (100.00%) Grammar and mechanics (sentence structure, punctuation, spelling) 0 (0.00%) 1 (8.33%) 2 (16.67%) 7 (58.33%) 2 (16.67%) 12 (100.00%) Analysis & Planned Action:

19 Extra Commentary or Narrative: Assessment Point 2 will happen in the Spring 2013 semester in the MIS Capstone course (MIS 475). We are currently using the COB-wide Written Communication rubric developed for the purposes of AACSB. While this is a good beginning, the style of writing for this course is more technical and targeted than general business written communication, and may need to be tailored to fit this type of project. Discussion by the entire MIS faculty will be necessary. (UPDATE: Spring Much of the work done in MIS consists of group projects, which makes individual assessment difficult, but not impossible. Our courses need to be structured for better individual assessment. This could consist of smaller individual assignments as a prelude to larger group efforts. More discussion needs to be done to determine if the General BBA AOL Rubric for this area needs to be tailored to better fit the MIS major.)

20 Effective Oral Communication: Verbally present educated thoughts and opinions in clear, concise and reasoned manner. Assessment Point 1 Trait Below Intro Milestone Capstone Advanced Total Clear Topic Coverage 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) Organization 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) Audience Appropriateness 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) Voice Quality 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) Grammar/Word Choice 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) Eye Contact & Body Language 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) Communication Aids 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) Personal Appearance 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) Analysis & Planned Action: Assessment Point 2 Trait Below Intro Milestone Capstone Advanced Total Clear Topic Coverage 0 (0.00%) 0 (0.00%) 0 (0.00%) 11 (78.57%) 3 (21.43%) 14 (100.00%) Organization 0 (0.00%) 0 (0.00%) 0 (0.00%) 11 (78.57%) 3 (21.43%) 14 (100.00%) Audience Appropriateness 0 (0.00%) 0 (0.00%) 0 (0.00%) 7 (50.00%) 7 (50.00%) 14 (100.00%) Voice Quality 0 (0.00%) 0 (0.00%) 1 (7.14%) 5 (35.71%) 8 (57.14%) 14 (100.00%) Grammar/Word Choice 0 (0.00%) 0 (0.00%) 0 (0.00%) 0 (0.00%) 14 (100.00%) 14 (100.00%) Eye Contact & Body Language 0 (0.00%) 0 (0.00%) 2 (14.29%) 5 (35.71%) 7 (50.00%) 14 (100.00%) Communication Aids 0 (0.00%) 0 (0.00%) 0 (0.00%) 4 (28.57%) 10 (71.43%) 14 (100.00%) Personal Appearance 0 (0.00%) 0 (0.00%) 2 (14.29%) 8 (57.14%) 4 (28.57%) 14 (100.00%)

21 Personal Appearance 0 (0.00%) 0 (0.00%) 2 (14.29%) 8 (57.14%) 4 (28.57%) 14 (100.00%) Analysis & Planned Action: Oral Communication was not assessed at an initial point. However, this will not be a problem in the future. Depending on which class is chosen for the initial point (i.e., MIS 340 or 310), targeted assignments will be able to address this issue. As with Written Communication, further discussion is required to determine if the current BBA AOL Rubric needs to be tailored for the needs of the MIS major. Extra Commentary or Narrative:

22 Teamwork/Collaboration: Work effectively as a member of a project team/group toward the identification, selection, and production of technology-related business systems. Assessment Point 1 Trait Below Intro Milestone Capstone Advanced Total Team/Collaboration Tools 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) Preparation/Task Performance 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) Communication/Information Sharing 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) Climate/cohesion 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) Analysis & Planned Action: Assessment Point 2 Trait Below Intro Milestone Capstone Advanced Total Team/Collaboration Tools 0 (0.00%) 0 (0.00%) 1 (6.67%) 9 (60.00%) 5 (33.33%) 15 (100.00%) Preparation/Task Performance 0 (0.00%) 0 (0.00%) 0 (0.00%) 6 (40.00%) 9 (60.00%) 15 (100.00%) Communication/Information Sharing 0 (0.00%) 0 (0.00%) 1 (6.67%) 4 (26.67%) 10 (66.67%) 15 (100.00%) Climate/cohesion 0 (0.00%) 0 (0.00%) 1 (6.67%) 5 (33.33%) 9 (60.00%) 15 (100.00%) Analysis & Planned Action: Teamwork/Collaboration was not assessed at an initial point. As much of the coursework in MIS consists of group effort, an initial point can easily be determined for this. Extra Commentary or Narrative: Results here show that students scored fairly high. Effort will need to be applied to determine if the Rubric needs to be modified or if the rigor of the targeted assignment should be increased. Teamwork is an important part of the MIS major and will need to be monitored as a skill area. The process which provides input to the rubric should be examined for adequacy. Currently, students rate each other, but no rating is done by the instructor. Instructor rating will be more difficult, but may be necessary. More discussion is needed.

23 Systems Thinking: To apply and use part-whole relationships in the analysis, discussion, and representation of artifacts, infrastructure, and technology. Assessment Point 1 Trait Below Intro Milestone Capstone Advanced Total Systems concepts 0 (0.00%) 0 (0.00%) 5 (41.67%) 7 (58.33%) 0 (0.00%) 12 (100.00%) Functional decomposition 0 (0.00%) 0 (0.00%) 2 (16.67%) 10 (83.33%) 0 (0.00%) 12 (100.00%) Transfer 0 (0.00%) 0 (0.00%) 5 (41.67%) 7 (58.33%) 0 (0.00%) 12 (100.00%) Analysis & Planned Action: The target of 75-80% at the Capstone level was achieved only in the area of Functional Decomposition. the other two areas are well below target. Systems thinking is not formalized within the course. Efforts will have to be made to either formalize this within the body of material, or to provide better grounding in this in a prior class. Assessment Point 2 Trait Below Intro Milestone Capstone Advanced Total Systems concepts 0 (0.00%) 0 (0.00%) 0 (0.00%) 0 (0.00%) 12 (100.00%) 12 (100.00%) Functional decomposition 0 (0.00%) 0 (0.00%) 0 (0.00%) 0 (0.00%) 12 (100.00%) 12 (100.00%) Transfer 0 (0.00%) 0 (0.00%) 0 (0.00%) 0 (0.00%) 12 (100.00%) 12 (100.00%) Analysis & Planned Action: Although these results seem to show some significant improvement, it may be that the assignments are not at the same level of rigor and may not be immediately comparable. Extra Commentary or Narrative: The Systems Thinking Rubric was developed especially for the MIS program and is not a part of the general AACSB AOL effort. The nature of the MIS program is naturally underpinned by systems and systems thinking. Formalizing this, however, has been somewhat difficult. At this current stage more discussion of the comprehensiveness of the rubric needs to be undertaken. The Transfer trait tacitly assumes that there is previous work on the part of the student to which to compare the current project work. That is a problem. This suggests something about the types and timings of assignments within the course or any similar course in which this Rubric would be used.

24 The link to the goal of Diagrammatic Representation and Reasoning must also be analyzed as systems thinking is somewhat inherent in that area. The MIS Faculty will continue to assess the effectiveness of this Rubric through the Spring term to modify it for further use.

25 Diagrammatic Representation and Reasoning: Use and develop diagrams commonly found in the representation of systems, infrastructure, architecture, database, and programming. Assessment Point 1 Trait Below Intro Milestone Capstone Advanced Total Tools 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) Composition/Mechanics 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) Semantics/syntax 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) Mapping/recognition 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) 0 (NaN%) Analysis & Planned Action: Assessment Point 2 Trait Below Intro Milestone Capstone Advanced Total Tools 0 (0.00%) 0 (0.00%) 0 (0.00%) 0 (0.00%) 12 (100.00%) 12 (100.00%) Composition/Mechanics 0 (0.00%) 0 (0.00%) 0 (0.00%) 1 (8.33%) 11 (91.67%) 12 (100.00%) Semantics/syntax 0 (0.00%) 0 (0.00%) 0 (0.00%) 0 (0.00%) 12 (100.00%) 12 (100.00%) Mapping/recognition 0 (0.00%) 0 (0.00%) 0 (0.00%) 1 (8.33%) 11 (91.67%) 12 (100.00%) Analysis & Planned Action: Although data was entered in this area for Point 2, no Point 1 data was collected because this rubric did not exist at that time. Thus, results here are hard to interpret and are confounded by the general feeling that this rubric may need to be folded into the Systems Thinking rubric. Extra Commentary or Narrative: If this rubric is to be combined with Systems Thinking, then any assignments used for the combined rubric will have to be carefully crafted. More investigation and discussion is needed.

26 Summary and Additional Commentary The Assessment effort of the MIS program has been somewhat slow to get up and running, primarily due to a missing plan since before the time of Lumina. Missing is not entirely accurate, but accurate enough. With the advent of Lumina, we have have been able to focus our effort, especially with the Lumina-imposed structure, and really see in a clearer fashion the natural learning outcomes that are not reflected in the COB-wide AACSB AOL goals. We have developed and will continue to refine additional learning outcomes that are particular to MIS and develop assignments and assessment instruments that allow us to properly assess those goals. It is obvious that we have some work to do, although we are off to a decent beginning. Most of our students are either at Milestone or Capstone levels of achievement. However, the fit of the assignments/projects with the rubrics (and vice versa) needs to be further addressed. (UPDATE: Spring More discussion is needed on our Rubrics. We will likely fold Systems Thinking and Diagrammatic Representation/Reasoning into one rubric. Oral and Written Communication rubrics were adopted from the general BBA AOL effort without modification. This may not be the best course of action. Will these need to be modified to better fit the particulars of MIS? The Teamwork rubric is felt to be sufficient for now, but the assessment process needs some refinement. The MIS program needs to evaluate the assignments used at Point 1 and Point 2 to make sure that improvements indicated are valid. We may also add an additional area of "General/Overall MIS Knowledge". Our Capstone class is experimenting with a national assessment exam, which we are considering making a permanent part of the MIS curriculum.)

27 Part V Assessment Results, Analysis and Planned Actions: Fall Semester 2013

28 As part of its continuing assessment program in conjunction with the Lumina Initiative the MIS program collected additional Assessment Point 1 data in MIS 340 (Database Management Systems) from the fall 2013 semester. Results indicate that the majority of students (approximately 83%+ in most cases) place at the Capstone or Advanced levels, with the majority of students falling at the Capstone level. See the tables below: Written Communication Introductory Milestone Capstone Advanced Organization 0 (0%) 2 (33%) 3 (50%) 1 (17%) Purpose 0 (0%) 1 (17%) 4 (67%) 1 (17%) Tone 0 (0%) 1 (17%) 4 (67%) 1 (17%) Grammar/Mechanics 0 (0%) 2 (33%) 3 (50%) 1 (17%) Oral Communication Introductory Milestone Capstone Advanced Clear Topic Coverage 0 (0%) 0 (0%) 4 (67%) 2 (33%) Organization 0 (0%) 2 (33%) 3 (50%) 1 (17%) Audience 0 (0%) 0 (0%) 4 (67%) 2 (33%) Appropriateness Voice Quality 0 (0%) 1 (17%) 3 (40%) 2 (33%) Grammar/Word Choice 0 (0%) 1 (17%) 4 (67%) 1 (17%) Eye Contact and Body 0 (0%) 0 (0%) 4 (67%) 2 (33%) Language Communication Aids 0 (0%) 0 (0%) 3 (50%) 3 (50%) Personal Appearance 0 (0%) 0 (0%) 5 (83%) 1 (17%)

29 Teamwork/Collaboration Introductory Milestone Capstone Advanced Team/Collaboration Tools 0 (0%) 1 (17%) 5 (83%) 0 (0%) Preparation/Task 0 (0%) 1 (17%) 4 (67%) 0 (0%) Performance Communication/Information 0 (0%) 0 (0%) 6 (100%) 0 (0%) Sharing Climate/Cohesion 0 (0%) 0 (0%) 6 (100%) 0 (0%) Systems Thinking Introductory Milestone Capstone Advanced Systems Concepts 0 (0%) 1 (17%) 2 (33%) 3 (50%) Functional 0 (0%) 1 (17%) 3 (50%) 2 (33%) Decomposition Transfer 0 (0%) 1 (17%) 3 (50%) 2 (33%) Diagrammatic Representation/Reasoning Introductory Milestone Capstone Advanced Tools 0 1 (17%) 3 (50%) 2 (33%) Composition/Mechanics 0 1 (17%) 4 (67%) 1 (17%) Semantics/Syntax 0 1 (17%) 4 (67%) 1 (17%) Mapping/Recognition 0 1 (17%) 4 (67%) 1 (17%) Care should be taken when interpreting these results. Data were collected from a semester project, which was completed by students working in groups. Thus, data points and resulting percentages show group-level outcomes for each rubric goal.

30 The concern is, of course, that it is hard to attribute performance solely to individual efforts; although it should be kept in mind that much work in this discipline is performed as a part of a team. Further work needs to be done programmatically to determine intraclass assessment instruments that examine individual effort as a complement to group work. Assuming a modicum of data reliability with respect to the MIS-specific rubrics, namely Systems Thinking, Diagrammatic Representation and Reasoning, and Teamwork/Collaboration, students are performing fairly well. However, with respect to the rubrics which were adopted from the general COG Bachelor s level assessment effort, students appear to have some minor problems which appear to be targets for improvement. Keeping in mind that these are Point 1 results, effort will have to be focused on improvements in the traits of Organization and Grammar/Mechanics within Written Communication. There also appears to be a need for additional effort to improve the trait of Organization within Oral Communication. This may result from students taking the MIS 340 class early in their degree program, before they have had time to take courses which aim to improve this aspect. Some correlation needs to be established to verify this assertion. Finally, the MIS Program has decided to continue with the five rubrics which were created and/or adopted during the phases of the Lumina Project. Additionally, we have adopted the use of a national MIS content exam to test MIS-specific knowledge attainment in our program capstone course, MIS 475: Strategic Information Systems.

31 Office of Assessment & Program Review August 5, 2013 Dr. Shane Tomblin, Program Assessment Coordinator Management Information Systems College of Business Dear Shane: The University Assessment Committee reviewers and I have completed our evaluations of the BBA in Management Information System s assessment of student learning for academic year , as submitted in the Open Pathways Project report last updated in May This letter will provide general comments and suggestions for improvement. Please refer to the attached assessment rubric for additional comments from reviewers. Please note that the reviewers comments are based on the reports you submitted in February 2013, so may not be appropriate for your final report. Your program s learning outcomes are well articulated and address higher orders of learning/thinking. Your assessment plan shows that complementary assessment measures are integrated throughout the curriculum and your rubrics are well developed. I appreciate your thoughtful analysis of results, which you are using to guide modifications to your assessment plan and pedagogical practices. During the academic year , programs will continue to report assessment results and plan actions using the online reporting form used last year. These reports will be due at the end of the academic year. If you have questions or concerns, please let me know. Sincerely, Mary E. Reynolds Mary E. Reynolds, Associate Vice President Assessment and Quality Initiatives C: Dr. Haiyang Chen, Dean, COB Dr. Margie McInerney, Interim Division Head, Management, Marketing, MIS One John Marshall Drive Huntington, West Virginia Tel 304/ Fax 304/ A State University of West Virginia An Affirmative Action/Equal Opportunity Employer