1 Past AOL reports and reviews are available at

Transcription

1 1 ASSURANCE OF LEARNING REPORT DEGREE PROGRAM: Master of Science in Information Systems (MSIS) REPORT AUTHOR(S): Amy B. Woszczynski, PhD SUBMISSION DATE: January 29, Following up on the previously submitted AOL report 1 and the feedback your program received from the AOL Council, what AOL related activities have faculty in your program conducted in the interim? The MSIS program underwent significant curriculum refinement in The curriculum revision was a result of analysis of trends in industry, input from the CSIS Advisory Board, and discussion of the revisions recommended by the MSIS 2006 Curriculum ( which is the standard for graduate degree programs in information systems. The revised curriculum (Appendix 1) was approved in Fall Since new students require a minimum of one year to complete the program, many of our students will be operating under a transition model, taking some classes from the prior curriculum, and other classes from the revised curriculum. We expect the first students to graduate under the revised requirements beginning not earlier than Summer We plan to use our previous assessment methods as a guide to measure learning outcomes with the revised curriculum. Specifically, we will use two methods to assess the achievement of learning outcomes: 1) surveying graduating students on their perceptions, and 2) using representative student project work in the introductory and capstone courses, to measure student learning. To measure student perceptions, we will deliver an online survey using SurveyMonkey. Students will be asked their perception of accomplishment of each of the learning outcomes, on a scale from 1=Strongly Agree, to 5=Strongly Disagree. Students will also have the opportunity to complete open ended questions to provide valuable qualitative feedback for program improvement. The survey we used prior to curriculum revision is included in Appendix 2. Our previous degree philosophy was based on the importance of strong technical skills and an ability to effectively manage technology in the organization. We updated our curriculum to meet the changing world environment, focusing on the development of skills needed to become a successful IT leader in a global economy. We revised the survey to reflect our change in degree philosophy, with an emphasis on global issues of concern to IT leaders. The revised survey is shown in Appendix 3. Descriptive statistics and qualitative comments will be reported to the MSIS working group to identify potential areas of improvement. The MSIS working group is a volunteer faculty service opportunity. Faculty members who regularly teach in the MSIS program are asked to join. This group recommends curriculum changes, assesses and reports on achievement of program learning outcomes, reviews the admissions process, and recommends program strategies to pursue in the future. To secure respondents, students petitioning to graduate will be asked to volunteer to complete the survey prior to setting an appointment for review of the degree program with the MSIS Director. We plan to begin implementation of this portion of AOL with students who petition to graduate for Summer 2010 and beyond. In prior years, we have received a response rate of 90% or higher, yielding rich data for analysis and interpretation. 1 Past AOL reports and reviews are available at

2 2 The second method we will use to assess learning is through a systematic and objective measure of student work products as evidence of mastery of learning outcomes. We will use a 2 pronged approach to measure learning: 1) a comparison of final research projects in IS 8005, Informatics, with IS 8800, IT Leadership, to assess student learning throughout the program; and 2) an evaluation of final research projects in IS 8700, IS Policy & Strategy. IS 8005, Informatics, is the first course that all MSIS students take. This course provides essential skills in formulating and communicating ideas to technical and non technical audiences using multiple communication formats. Students in IS 8005 also gain substantial research skills, which are built upon in subsequent courses. IS 8800, IT Leadership, is one of two capstone courses in the program. Students take the capstone courses at or near the end of their program of study, at which time we would expect that they will meet most learning outcomes, and that they will obtain a maturity in their writing style and in their understanding of IS issues. The research projects in IS 8005 and IS 8800 both address issues of concern to IT leaders. While students will select different topic areas for these courses, we would expect to see substantial improvement in communication skills and IS knowledge and maturity over the duration of the program. As such, we will assess these products as pre tests (IS 8005 research paper) and post tests (IS 8800 research paper). We would expect substantial improvement to be evident from the research papers completed in IS 8005, when compared to those completed in IS We will ask instructors from both courses to retain representative samples of student research papers to be retained by the Program Director. The MSIS working group will evaluate the work products using a longitudinal analysis, to assess learning over time. Specifically, each member of the MSIS working group will be asked to review paired research papers from IS 8005 and IS 8800 to assess learning. The members of the working group will independently evaluate how well the author improved through an analysis of the following questions, from 1=Shows exceptional improvement, to 3=Shows expected improvement, to 5=Shows no improvement: Question 20. I possess the skills and knowledge needed to be an effective IT leader in a global organization. Question 27. I can create effective written communication. Question 28. I create effective presentations. Question 29. I can develop presentations appropriate for specific audiences. Question 30. I understand how to communicate effectively with non technical stakeholders. Scores will be averaged and discussed among the MSIS working group, and reported to the AOL Committee, with recommended strategies devised to improve the percentage of students who exhibit expected levels of improvement (or higher) in meeting the stated learning outcomes. Our goal is that 80% or more of the research project comparisons will show expected levels of improvement or better between the paired research papers, with an average score of 3 or better. IS 8700, IS Policy & Strategy, is the other capstone course for the program. The professor for IS 8700 will be asked to select a representative sample of student work to retain for review by the MSIS working group. The final project for this course integrates critical reasoning skills obtained and refined by the students during their course of study toward the MSIS degree. Members of the working group will independently evaluate each project on how well the author shows mastery of the stated learning outcome, from 1=Exceeds expectations, to 3=Meets expectations, to 5=Fails to meet expectations: Question 3. I can discuss the need to design information security into a technology project. Question 6. I can explain legal issues related to information technology.

3 3 Question 7. I can discuss ethical issues related to information technology. Question 8. I understand strategies to protect IS intellectual property in the global environment. Question 17. I can evaluate and recommend global sourcing strategies. Question 20. I possess the skills and knowledge needed to be an effective IT leader in a global organization. Question 31. I can discuss the relationship between strategic, tactical and operational planning. Question 32. I can explain the importance of IS policy in organizations. Question 33. I can develop an effective organizational IS policy. Question 34. I can describe the interaction between IT departments and other business units. Scores will be averaged and discussed among the MSIS working group, and reported to the AOL Committee, with recommended strategies devised to improve the percentage of students who meet or exceed the stated learning outcomes. Our goal is that 80% or more of the research projects evaluated should meet the stated learning outcomes, with an average score of 3 or better. Based on the quantitative and qualitative data obtained, the MSIS working group will make recommendations to better meet AOL guidelines in future semesters. The MSIS working group will complete this activity in the Fall and Spring semesters, with the Program Director compiling and summarizing survey data prior to meetings during the academic year. Systematic deficiencies in one or more learning outcomes will be addressed through an action plan recommended by the MSIS working group and approved by the Department Curriculum Committee. 2. As a result of these activities, what did your program s faculty discover about the quality of teaching and learning within the program and/or its AOL processes? It was helpful to consider and plan for assessment during the curriculum redesign process. We believe that this simultaneous development of curriculum with an embedded assessment plan led to an excellent program of study. We plan to continuously evaluate the program over time, with a goal of continuous improvement. 3. As a result of these discoveries, what changes did your program s faculty implement to enhance the quality of teaching and learning in the program and/or its AOL processes? While we have not assessed the new curriculum yet, we have a plan of action in place. We plan to continue to carefully evaluate proposed courses and plan for assessment in advance of introducing new curriculum. By learning what our students know and where they may have gaps in knowledge, we have the opportunity to continually improve our program over time. 4. As a follow up to the discoveries from the AOL process and the changes that were implemented, what AOL related activities are your program s faculty planning to conduct in the upcoming year? We plan to begin collecting assessment data in Spring 2010, specifically beginning with students who apply to graduate for Summer 2010 and later. These students will be asked to complete an online survey prior to completion of the petition to graduate. Students will have the opportunity to provide qualitative and quantitative feedback as input to the AOL process. Student responses will be anonymous and confidential, with reporting of data in summary format only. We will also gather student work samples to assess achievement of learning outcomes, as described previously. 5. Update the Grid of Global Learning Outcomes and Assessment Methods for KSU Degree Programs. Please note if the grid needs updating with a No or Yes: Yes See attached Excel file.

4 4 Appendix 1. MSIS Curriculum (effective Fall 2009) Credit Hours CORE REQUIREMENTS 27 IS 8005 Informatics (should be taken in 1 st semester) 3 IS 8100 Advanced IT Project Management 3 IS 8200 Legal & Ethical Issues in IS 3 IS 8300 Disaster Recovery/Business Continuity Planning 3 IS 8400 Enterprise Process Models 3 IS 8500 Emerging Technologies 3 IS 8600 Global IS Management 3 IS 8700 IS Policy & Strategy (capstone) 3 IS 8800 IT Leadership (capstone) 3 ELECTIVES 9 Students must take three courses (9 semester hours) of elective credit. Courses may be selected from any approved non premium priced graduate courses at KSU, or from electives offered in the MSIS program. Since prior permission is required to take courses outside of the department, students should contact the program director before they register for non MSIS courses. MSIS DEPARTMENTAL ELECTIVES IS 8900, Special Topics in IS 3 IS 8910, Special Projects in IS 3 IS 8916, Co op in IS* 1 3 IS 8918, Internship in IS* 1 3 IS 8920, Customer Relationship Management 3 IS 8930, Information Security Administration 3 * Only 3 semester hours of elective credit for IS 8916 or IS 8918, will be counted toward the MSIS degree.

5 5 Appendix 2. MSIS Survey of Graduating Students (relevant for curriculum prior to Fall 2009) Distributed to students online in SurveyMonkey, upon their request to petition to graduate. Questions deleted from the prior assessment tool marked with strikethrough; additions indicated in revised survey (Appendix 3). 1. I have the ability to analyze real world business problems and develop information systems solutions to address them. 2. I can explain the theoretical foundations of information systems. 3. I can describe computer hardware networking. 4. I can design and implement computer programs. 5. I can design and develop databases. 6. I can use Internet technologies to support information technology projects. 7. I understand the practical applications of information systems. 8. I feel comfortable analyzing information systems using appropriate methods, techniques and tools. 9. I can describe how to use the Systems Development Life Cycle to solve a business problem using information technology. 10. I can discuss the need to design information security into a technology project. 11. I feel comfortable using current methodologies and techniques to model the conceptual design of an information system. 12. I can explain concepts of electronic business (e business) systems. 13. I can describe concepts of data communications as they relate to information technology. 14. I can articulate concepts of systems administration as they relate to information technology. 15. I can discuss concepts of information security. 16. I can explain legal issues related to information technology. 17. I can discuss ethical issues related to information technology. 18. I can articulate the responsibilities of the IS practitioner, including professional codes of conduct. 19. I understand the importance of protecting individual privacy in organizational systems. 20. I can explain the foundations of project management. 21. I understand the importance of team dynamics when completing group projects. 22. I can demonstrate team leadership. 23. I can discuss the use of modern technologies and techniques to manage project resources. 24. I can describe the need for collaboration and individual effort in completing projects. 25. I can explain the importance of diversity in team projects. 26. I communicate effectively in oral presentations. 27. I can create effective written communication. 28. I create effective presentations. 29. I can develop presentations appropriate for specific audiences. 30. I understand how to communicate effectively with non technical stakeholders. 31. I can discuss the relationship between strategic, tactical and operational planning. 32. I can explain the importance of IS policy in organizations. 33. I can develop an effective organizational IS policy. 34. I can describe the interaction between IT departments and other business units. We struck through those questions that focused primarily on technical skills, such as programming, database development, and hardware. We then added appropriate questions to address the global nature of our program, with the resulting survey on the next page.

6 6 Appendix 3. MSIS Survey of Graduating Students Revised Fall 2009 First planned use is for Spring Additions noted in red. 1. I have the ability to analyze real world business problems and develop information systems solutions to address them. 2. I can explain the theoretical foundations of information systems. 3. I can discuss the need to design information security into a technology project. 4. I feel comfortable using current methodologies and techniques to model the conceptual design of an information system. 5. I can discuss concepts of information security. 6. I can explain legal issues related to information technology. 7. I can discuss ethical issues related to information technology. 8. I understand strategies to protect IS intellectual property in the global environment. 9. I can articulate the responsibilities of the IS practitioner, including professional codes of conduct. 10. I understand the importance of protecting individual privacy in organizational systems. 11. I can explain ERP systems and how they relate to the enterprise architecture. 12. I know how to leverage emerging technologies to enhance the organization s effectiveness. 13. I can explain the foundations of project management. 14. I understand how to manage global IS project portfolios. 15. I can lead multicultural IS project teams. 16. I know how to overcome challenges associated with virtual or geographically dispersed project teams. 17. I can evaluate and recommend global sourcing strategies. 18. I can describe how to use global IT infrastructure for organizational effectiveness. 19. I understand global systems development and implementation issues. 20. I possess the skills and knowledge needed to be an effective IT leader in a global organization. 21. I understand the importance of team dynamics when completing group projects. 22. I can demonstrate team leadership. 23. I can discuss the use of modern technologies and techniques to manage project resources. 24. I can describe the need for collaboration and individual effort in completing projects. 25. I can explain the importance of diversity in team projects. 26. I communicate effectively in oral presentations. 27. I can create effective written communication. 28. I create effective presentations. 29. I can develop presentations appropriate for specific audiences. 30. I understand how to communicate effectively with non technical stakeholders. 31. I can discuss the relationship between strategic, tactical and operational planning. 32. I can explain the importance of IS policy in organizations. 33. I can develop an effective organizational IS policy. 34. I can describe the interaction between IT departments and other business units.