SOUTHEASTERN OKLAHOMA STATE In\fIVERSITY

SOUTHEASTERN OKLAHOMA STATE In\fIVERSITY Department OfManagement and Marketing Program Outcome Assessment Report General Business Program AY 08-09 September 11, 09 Dt:e;t:(---- Department ofmanagement and Marketing sey School ofbusiness 1

Table of Contents JMSB Mission Statement Department Mission Statement Program: General Business AACSB Accreditation and Its Impact on the JMSB Assessment Process Table 1. Learning Goals, Outcomes, Assessment Methods and Results External Assessment Results Figure 1. Overall Average Score for the JMSB vs. MFT Norm Figure 2. General Business Major Scores Analysis - Strengths/Weaknesses Program Modifications Web-based /IETV Additional Assessment Figure. MFT Scores vs. Number of Online Courses Faculty Contribution Page 5 6 6 7 7 8 8 9 9 2

JMSB Mission Statement The mission of the John Massey School of Business is to be the premier regional business program in southeast Oklahoma and north Texas. We will prepare our business and aerospace students to meet the challenges in a competitive and ever changing economy by providing an excellent and continuously improving learning environment that emphasizes both academic and applied knowledge and research. We will further assist in the economic development of the region by providing effective application of knowledge with small and midsize businesses and Native American Tribes. Department of Management and Marketing Mission Statement The primary mission of the Department of Management and Marketing is to provide baccalaureate programs in Management, Marketing and General Business designed to provide an environment of academic excellence in undergraduate business education, and experiences both academic and applied that prepare students to operate in a diverse and global environment. Through these programs, students will develop an interest in lifelong learning. The General Business Program The General Business program is designed to prepare students for employment in a wide range of administrative positions in private business, government, or not-for-profit businesses. It is an interdepartmental major intended for students who wish to remain generalists rather than specialists in the business field. The curriculum promotes a general understanding of the business environment while providing a strong foundation in the common body ofbusiness knowledge. It includes required upper-division course work in accounting, finance, management and marketing, beyond the business core. AACSB Accreditation and Its Impact on the JMSB Assessment Process Last academic year, the JMSB earned AACSB accreditation. As part ofthe evaluation, AACSB sent a site visit team in October 08. Following the visit, the three major recommendations from the team were to: Update the strategic plan Redesign the Assurance of Learning (AoL) process to increase utility and efficiency Revamp the MBA Admission and advising processes As a result, the JMSB AoL Committee began redesigning the process. The JMSB AoL process had been in place since 05; however, the site visit team suggested the process was too cumbersome and offered limited value for the effort required. Specifically, their recommendations were to (l) train faculty in AoL concepts and (2) redesign both the undergraduate and graduate current AoL processes to increase usefulness, while reducing faculty efforts in collecting data.

For (1), Dean David Martin from LaSalle University was hired in November 08 to review the JMSB AoL process and deliver a seminar on how to improve the existing process. Additionally, AoL committee members attended the AACSB Assessment Conference held in Dallas in December 08. For (2), the AoL committee is currently redesigning the undergraduate AoL process as follows: Reduce the ten current goals to a more manageable five. Develop goal-specific rubrics for course-embedded assessment, to allow goal statistics to be generated independent of faculty and course. Develop an assessment plan that requires less data collection than the current process, but requires faculty to address any deficiencies within a two-year planning cycle. Table 1 shows the five new goals, objectives, learning outcomes, assessment method and results. Note that several goals are 'under development' or no data exists. The AoL committee plans to present its work to the Dean and faculty, as well as begin collecting course-specific data during the F09 semester. Once finalized, each program (management, marketing and general business) in the department will develop additional specific goals, objectives and traits to measure. Consequently, no course-specific data was collected for AY0809. The only assessment data collected was the MFT in Business scores given to graduating seniors in the capstone course, MNGT 46 Business Policy. The MFT in Business is an objective, end-of-program test for the Business disciplines. The content ofthe Major Field Test in Business reflects the basic knowledge and understanding gained in the core undergraduate curriculum. The test is a two-hour, multiple-choice examination designed to assess mastery of concepts and principles as well as knowledge expected of students at the conclusion ofa major in specific subject areas. It goes beyond measurement of factual knowledge, however, to evaluate students' abilities to analyze and solve problems, understand relationships, and interpret material. Students are also required to interpret graphs, diagrams, and charts related to the field ofbusiness. The test covers assessment indicators in nine areas: Accounting, Economics, Management, Quantitative Business Analysis, Finance, Marketing, Legal and Social Environment, Information Systems, and International Issues. 4

TabIlL e. earnmg Goa s, outcomes, AssessmentMeth 0 d sandresuits GOAlS Goal 1. C01nmunication School of business gradualeswill be effective conununicators with the ability to prepare and deliver oral and written presentations using appropriate technologies. OBJECTIVES LEARNING OUTCOMES Cleu Logic & Organization ASSESSMENT METHODS RESULTS Effective use oflanguagelstyle,..students will create well Correct Spelling and Gramrrar written documents on a business topic Clear Purpose Course Embedded Assesstrenl (begin AY091O) Begin AY0910 Excellent Developtrent ofideas Correct Citation and Documentation Effective Organization b.studenls will deliver an Knowledge ofcontent effective oral presentation Persuasive Conclusion Course Embedded ASSeSStrenl (begin AY091O) Begin AY0910 on a business topic Excellent delivery Effective use ofbody language c.students will usc Effective us ofmedia I. Begin appropriate technologies to ). Course Embedded Assessment (begin AY091 0) AY0910 enhance the effectiveness of 2. MIT Information Systems Subscore >= n01ul 2. See Figures 1 their written and oral trean (n =8) Effective Use of Technology presentations. Derme the problem Research and gather Infonnation I. Begin Generate plans to sol ve problem, integrating I. Couse Embedded Assesstrent (begin AY0910) a Use problem solving AY0910 relevant factors in decision making. 2. MFT Quantitati ve Methods Subscore >~ Dorm trethod 2. See Figures 1 trean (n ~ 8) Select and implemenl best solution Goals 2.J',til!lem solving - School Of Connnunicate the results [business gr~uales can apply critic8i Usc algebra to formulate/solve problems 1. Begin thinking skills to prob1!'01 solving. correctly J. Couse Embedded Assessment (begin AY09IO) b. Use quantitative AY0910 reasoning to solve problems Use statistics methods to formulate/solve 2. MFT Quantitative Methods Subscore = norm 2. See Figures 1 problems correctly trean (n ~ 8) c. Use technology such as specialized computer Use MS Ollice tools Sllch as Excel and Couse Embedded Assessment (begin AY091 0) Begin AY0910 ooftware to solve problems Access to solve problems correctly Goals. Ethics School ofbusiness graduates can identily and assess ethicalissues in b usiness. under development. "" ~, Goal 4; Leadership, Teamwork and Collllboration - SchocH ofbusiness graduates possess th~leadeyllhip skills nece' 'ai)i to aive dir~ctionand guidance,llt)d.toc!elegate wode tasks under development in a manner which proves to.be eff~ctiveand which motivatescothers to do their best in both a manager/subordinate and a team sening, Recognition oftlte critical importanceof Wlderstanding the concepts inherent in diversity issues. GoalS. Divers~. S'ehoolof Students will demonstrate an business graduates willlidderstand the Application and detronstration ofindividual appreciation ofhuman and understanding ofdiversity issues. I. Couse Embedded Assesstrent (begin AY0910) diversity ofthe local an4global cultural differences through 2. MFf Legal and Social Environment Subscore >= See Figures 1- coijjillunities, including cultural, Analyze, evaluate and assess the impact of a deeper understanding of norm trean (n =8) social, political and economic differences in individuals and groups. the world they inhabit. difrerences. Evaluates, assesses, and rates the impact of college experience regarding diversity and its influence on personal and/or professional lire 5

External Assessment Results The Major Field Test Comparative Data (http://www.ets.org/media/tests/mft/pdf/mft%pdfs%07/business4cmf.pdf) was used for comparison with the results achieved locally (see Appendix A for tables). The norm group represents over 8,2 senior students from 564 institutions who had taken the test between August 06 and June 08. Figure 1 shows the overall average score of the JMSB students compared to the norm since the fall 02 semester. The minimum and maximum score is 1 and 0, respectively. Assessment indicators for the Business test cannot be compared to testing years prior to 02 due to changes in the Business test that were introduced in 02. For the JMSB overall score (the diamond-shaped points), a 95% confidence interval is provided around the mean. Since the confidence interval contains the ETS overall average score (the square-shaped points), no significant difference exists between the JMSB average score and the overall average ofthe norm group. Although, no significance exists, the JMSB average score did drop from AY08, and this was the first time that scores have dropped since 05. Figure 1. Overall Average Score for the JMSB vs. MFT Norm _ ETS Overall Mean Score -""'JMSB Overall Mean Score In addition to the overall score, Figure 2 shows the average score for the nine assessment indicators, specific to General Business majors. The minimum and maximum score is and 100, respectively. Similar to Figure 1, the General Business majors' average scores are represented with the diamond-shaped points and are bounded by a 95% confidence interval. Across the board, the subscores dramatically improved; however, one ofthe eight General 6

Business students who took the exam scored hi~her than any JMSB student since the exam has been given at SE. The student scored in the 99 t percentile. Figure 2. General Business Major Scores...~~~~ -~~~~ _ ETS Comparative Mean _ETS Comparative Mean - 1. Accounting --2. Economics 80.----,...---,...-,...-_,...--, 50 80.-----..,...,-_,...-..,...,-..., 50 _ETS Comparative Mean --4. Quantitative Business Analysis 80 1..,...,..,...,"7'7""""""""""""7'7"""""""':""'i'i'iT"'''1 ~.,.,...,' 50 80...,...,,,,,,,... 50 _ ETS Comparative Mean --7. Legal and Social Environment 80.--..,...,..,...,..,...,..,...,..,...,..,...,-..,...,..,...,..., 50 80 -r-==""""""""""""""" 50 +-"----_+_ Analysis - StrengthslWeaknesses A potential strength is the increased 'International Issues' subscore, which climbed to be significantly greater than the norm average. The General Business curriculum was modified two years ago, requiring an international course to be taken. Potentially, this modification may be linked to the increase. 7

Program Modifications The brief calculus course, MATH 214, was removed from the core because the course was not a prerequisite for any other course. In 05, MATH 214 was removed as a prerequisite for any BBA course. Although required for earning a BBA, MATH 214 was not attached to any subsequent course. Some students delayed (are delaying) until their senior year. However, since 05, standardized test scores (ETS Major Field Test in Business) have improved or remain unchanged both in finance and in quantitative methods. Business calculus is not suggested by AACSB International. Several AACSB schools do not require the course. During our site visit in October, one of our reviewer's had recently removed business calculus as a required course from his business school. Business calculus is taught infrequently at feeder schools even iftechnically on the books. Consequently, students must take when arriving at SE, which places these junior-level students in 00-level courses. Grayson County CC requires a pre-calculus course prior to their brief calculus. The consequence is that to earn the A. S. at GCC within a 2 year period, the 2+2 agreement requires one semester of 19 hours. Students can meet their general education math requirements through MATH 151 College Algebra. For those students sufficient in college algebra, we will advise them to take a more advanced math course (such as brief calculus). We would also advise those planning to pursue an MBA to take brief calculus. Web-based IIETV Additional Assessment Within the JMSB, regression was used to measure the MFT overall score against the number of core online courses taken. The total number ofcore courses is 14. The data is for the last four years. Based on the data, Figure shows that no relationship exists, and that MFT scores are not impacted by delivery method. As course embedded assessment resumes within the JMSB, additional course-specific assessment will occur. 8

Figure. lvift Scores vs. Number of Online Courses Source: JMSB Seniors for the Last 4 Years with at least One Online Course n= 16 0 180 F1 ~ 1 ~ 1 6 100 o 80 f M o II o 1 2 4 y = -0.108x + 15.6 R 2 = 0.000...... 5 6 7 8 # Online Courses Faculty Contribution Dr. Jim Speers provided the online analysis. The AoL Committee (Drs. Speers, Campbell, Hrncir, Topuz) are revamping embedded course assessment. The department faculty reviewed and edited the report. 9

Appendix A - ETS Major Field Tests Comparative Data Tables 10

@M- ajor r/e'l ~-'d Tests Listening. Learning. Leading. Major Field Test in Business (form: 4CMF) Individual Students Total Score Distribution Data includes Seniors from Domestic Institutions -- August 06 to June 08 Total Score % at or below Range (1-0) 179-0!lS 178 177 176 175 174 17 172 171 1 169 168 167 166 I I 90 85 80 165 164 16 162 161 1 75 159 158 157 156 155 154 65 55 15 152 45 151 150 149 148 5 147 146 145 144 25 14 142 141 1 19 18 17 16 15 14 1 12 1?n _ 11 15 10 5 Number of Examinees 82 Mean 152.9 Median 152 Standard Deviation 14.0 Total Scores are reported as scaled scores. % at or below based on percent below the lower limit of the score interval. Copyright 08 by Educational Testing Service. All lights reserved. ETS and the ETS logo are registered trademar1<s of Educational Testing Service

@M ~. ajor rie" Tests 'd Listening. Learning. Leading. Major Field Test in Business1 (form: 4CMF) Institutional Assessment Indicator Mean Score Distributions Data includes Seniors from Domestic Institutions -- August 06 to June 08 Mean Percent Correct Assessment Indicators 2 % at or below Mean Percent Correct I AI1 AI2 AI AI4 AI5 AI6 AI7 AIS AI9-100 9-100 69 95 95 95 69 68 90 68 67 95 67 66 90 95 90 85 90 66 65 95 65 64 8fi 85 80 95 64 6 95 III 90 80 6 62 62 61 75 85 65 75 61 fifi 90 59 6t> t>u 65 59 58 Rr 'fi 90 58 57 80 5 55 0 5 57 56 fir?fi fifi 90 56 55 It> Il~ 9l 41 ;,Il t>l 55 54 Rn lfi 4fi 85 85 54 5 65 75 5 0 10 5 52 ~[ ~fi 52 51 ;,It> ;,It> 5 50?[?fi 50 49 4 49 15 48 15 48 47 It> 47 46?fi 10 10 46 45 ;,Il 10 5 45 44 15 25 44 4 ;,IU 't> 2;> 5 10 4 1 42 1fi 5 5 42 41 5 15 10 41 5 0 10 9 1 9 1 8 5 5 1 5 8 7 1 1 7 0-6 1 1 1 0-6 ~l 51 Number of Institution! 564 564 564 564 564 564 564 564 564 Mean 49.8 47.4 54.7 46.2 54.9 52.0 46.1 58.0 54.0 Median 50 47 55 46 55 52 46 58 54 Standard Deviation 6.9 7.0 7.5 5.9 8.5 6.8 6.1 5.2 8.0 'Assessment Indicators for this test cannot be compared to testing years prior to 06 due to changes in the Business Test that were introduced in 06. 2Assessment Indicator 1: Accounting Assessment Indicator 2: Economics Assessment Indicator : Management Assessment Indicator 4: Quantitative Business Analysis Assessment Indicator 5: Finance Assessment Indicator 6: Marketing Assessment Indicator 7: Legal and Social Environment Assessment Indicator 8: Information Systems Assessment Indicator 9: International Issues Assessment Indicators are reported as percent correct. % at or below based on percent below the lower limit of the score interval. Copyright 08 by Educational Testing Service. All rights reserved. ETS and the ETS logo are registered trademarks of Educational Testing Service.