Florida Gulf Coast University. Report of the Online Distance Learning Task Force

Transcription

1 Florida Gulf Coast University Report of the Online Distance Learning Task Force April 1, 2013

2 Table of Contents Executive Summary 2 Introduction 3 Charge 4 Findings 17 Recommendations 20 Summary 21 Appendix 1: Sloan Instrument 22 Appendix 2: Responses to Sloan Instrument Items 29 Page 1

3 Executive Summary FGCU delivers nearly 600 course sections each year online (400 at the undergraduate level). In addition it is possible to earn a number of degrees and certificates online including upperdivision course work for the B.S. in Criminal Justice, Health Sciences, and Legal Studies. Complete master s degrees in Public Administration, Special Education, Business Administration, Forensic Studies, Criminal Justice, Health Sciences, and Educational Technology (through the Master s in Curriculum and Instruction) are also available online. Collectively online enrollment accounts for approximately 15% of FGCU s total enrollment annually. A joint task force comprising faculty, staff, and students worked during the fall and spring of to assess the current state of institutional support services for online instruction using a recognized standard instrument reflecting best practices. The task force scored 70 items covering a wide variety of categories. With regard to the vast majority of items, the Task Force found the University was providing an acceptable level of service. Notwithstanding, it identified a number of areas that need greater attention. These include such things as online student authentication, standards for course design, guidelines and enhanced opportunities for faculty engagement/professional development, enhanced availability of student support services for online students, and additional assessment of student success to promote further student retention. The Task Force offers the following seven recommendations to enhance the online teachinglearning experience for students and faculty at FGCU: 1. Student identity authentication strategies/technologies, including such things as online proctoring, should be identified and implemented that will ensure a student completing an online course is indeed the same student who enrolled in the course. 2. The Teaching, Learning, and Assessment Initiative (TLAI) and the Faculty Senate s curriculum teams should explore standards for online course design and delivery to enhance student retention and course quality beyond the current record of student success. Greater use of the Quality Matters rubric in the design of online courses should be encouraged. TLAI and faculty should develop a student perception of instruction (SPoI) especially suited to address online learning. 3. Web, e-learning, and Publications (WEP) should expand its outreach to faculty with regard to disseminating information on current and emerging technologies to improve online learning. 4. Faculty professional development opportunities associated with online student instruction and assessment should be greatly increased. A certification process for faculty teaching online utilizing the Quality Matters system should be established. 5. Access to online student services support should be extended at critical times. 6. The Enrollment Retention Management Committee of the Planning and Budget Council should examine student retention in online courses and programs, share data with relevant departments and program faculty, and recommend strategies for improvement. 7. A strategic plan for online learning should be developed. Page 2

4 The Task Force recognizes that existing resources dedicated to online learning are not adequate for the implementation of the majority of these recommendations. Its members strongly advocate that a distance learning fee be implemented to provide resources to implement these recommendations. Introduction This report and its recommendations represent the work of many different individuals and groups within the university over the last two years. It grew out of the efforts of the Sloan Consortium to develop an instrument for determining the efficacy of support mechanisms for online learning related to institutional support, technology support, course development and instructional design, course structure, teaching and learning, social and student engagement, faculty and student support, and evaluation and success. The current work was part of a charge provided by the Planning and Budget Council to the Information Resources Committee in 2011, and was included in the actions to be undertaken in as part of the university s strategic plan related to the institution s mission of access employing instructional technology. The purpose was to assess the status of support to online learning using the Sloan instrument, identify weaknesses as a result of the assessment, and to make recommendations to address the perceived weaknesses, and, by so doing, enhance the online learning experience for students and faculty. The study was broken into two phases. During the academic year the first phase of the project was begun: collecting responses and supporting documentation to the 70 items contained in the Sloan instrument to provide a factual context for scoring the items. After initial work by David Jaeger, Director of WEP and Paul Snyder, Associate Provost, a number of other individuals and groups were asked to provide input to the response document. This included e-learning, Library Services, Student Affairs, the Assessment Council, the Writing Center, the Center for Academic Achievement, and Deans Council. In late spring 2012, Faculty Senate leadership was consulted about phase 2 of the project: to involve faculty and students in the actual scoring of the items contained in the instrument based on their own experience and the responses to the items prepared in phase 1. While the Sloan Consortium instrument was intended to assess individual online degree programs, it was decided that an approach looking across all online courses and programs collectively rather than individually would better capture the breadth of online support that currently exists at FGCU, allow for more diverse perspectives to be recorded, and acknowledge that there are many more discrete online courses offered at FGCU than either online degree programs or students pursuing a degree entirely online. In fall 2012, with input from the Faculty Senate, the following individuals were appointed by Provost Toll to serve on the Online Scorecard Task Force: Ms. Sheila Bolduc-Simpson Dr. Carolynne Gischel College of Arts and Sciences College of Education Page 3

5 Dr. Mary Krome Dr. Megan McShane Dr. Halcyon St. Hill Dr. Hulya (Julie) Yazici Dr. Mary Ann Zager Ms. Patricia Cunningham Ms. Marylinda Flores Ms. Holley Holland Mr. David Jaeger Dr. Paul Snyder Lutgert College of Business College of Arts and Sciences College of Health Professions and Social Work Lutgert College of Business College of Arts and Sciences Graduate student Undergraduate student Graduate student Director, Web, e-learning, and Publications Associate Provost and Associate Vice President The Task Force was charged with three tasks: Charge 1) Fill in the gaps on the draft response to the items on the scorecard (see attached) 2) Complete the scorecard 3) Review the results and make recommendations for improvements based on the results The group addressed its work using a Delphi approach (i.e., relying on experience and knowledge of a small number of individuals rather than a quantitative design) due to the size of the group. Only the faculty members of the Task Force scored the items since the students did not feel they had enough personal information concerning the topics covered in the 70 items. When scoring the items (task 2) each individual scored the items separately and submitted their scores along with associated item score comments for compilation. Possible responses to the items included not observed =0, insufficient =1, moderate use=2, and meets criterion completely =3. Because these were discrete categories rather than continuously distributed scores, when evaluating the responses the mode was used as the measure of central tendency and as a basis for identifying weaknesses. The Task Force spent four meetings reviewing the 70-item response document, seeking clarification in certain instances, challenging statements and supporting documentation in other instances, and making extensive revisions to the document before achieving consensus that scoring utilizing the document was appropriate. The results of that scoring are found in the following table (it should be noted that these are subjective judgments and the use of basic descriptive statistics is not meant to imply any sense of statistical precision, but rather serve as a tool to identify commonly held perceptions among the scorers of strengths and weaknesses): Page 4

6 Scoring Key: 0 = Not Observed 1 = Insufficient 2 = Moderate Use 3 = Meets Criterion Completely Quality Scorecard for the Administration of Online Education Programs Points possible Per Category Committee Mean Institutional Support The institution has a governance structure to enable effective and comprehensive decision making related to online education. 2. Policies and guidelines are in place to authenticate that students enrolled in online courses and receiving college credit are indeed those completing the course work. 3. Policy for intellectual property of course materials exists. (If policy exists, 1 pt.; If policy specifically addresses online course materials, 2 pts.; If policy specifically addresses online course materials and is publically visible online, 3 pts.) 4. The institution has defined the strategic value of online learning to its enterprise and to its stakeholders. Committee Mode Points possible Per Category Committee Mean Committee Mode 1. A documented technology plan that includes Technology Support Page

7 electronic security measures (e.g., password protection, encryption, secure online or proctored exams, etc.) is in place and operational to ensure quality standards, adherence to FERPA, and the integrity and validity of information. *** 2. The technology delivery systems are highly reliable and operable with measurable standards being utilized such as system downtime tracking or task benchmarking. *** 3. A centralized system provides support for building and maintaining the online education infrastructure. *** 4. The course delivery technology is considered a mission critical enterprise system and supported as such. 5. The course delivery technology is considered a mission critical enterprise system and supported as such. 6. Faculty, staff, and students are supported in the development and use of new technologies and skills Page 6

8 Points possible Per Category Committee Mean Committee Mode Course Development and Instructional Design 1. Guidelines regarding minimum standards are used for course development, design, and delivery of online instruction (such as course syllabus elements, course materials, assessment strategies, faculty feedback). *** 3. Instructional materials, course syllabus, and learning outcomes are reviewed periodically to ensure they meet program standards. *** 4. Courses are designed so that students develop the necessary knowledge and skills to meet learning objectives at the course and program level. These may include engagement via analysis, synthesis and evaluation. *** 5. Learning objectives describe outcomes that are measurable Page 7

9 6. Selected assessments measure the course learning objectives and are appropriate for an online learning environment Student-centered instruction is considered during the coursedevelopment process There is consistency in course development for student retention and quality Course design promotes both faculty and student engagement Current and emerging technologies are evaluated and recommended for online teaching and learning. 11. Instructional design is provided for creation of effective pedagogy for both synchronous and asynchronous class sessions Page 8

10 12. Curriculum development is a core responsibility for faculty (i.e., faculty should be involved in either the development or the decision making for the online curriculum choices) Points possible Per Category Committee Mean Committee Mode Course Structure The online course site or other web site includes a syllabus outlining the course objectives, learning outcomes, evaluation methods, textbook information, and other related course information, making course requirements transparent at time of registration. *** 2. The institution ensures that all online education students, regardless of where they are located, have access to library/learning resources adequate to support the courses they are taking (SACS statement). *** 3. Expectations for student assignment completion, grade policy, and faculty response are clearly provided in the course syllabus. *** Page 9

11 4. Links or explanations of technical support are available in the course (i.e., each course provides suggested solutions to potential technical issues and/or links for technical assistance). 5. Instructional materials are easily accessible and easy to use for the student The course adequately addresses the needs of students with disabilities via alternative instructional strategies and/or referral to special institutional resources. 7. Opportunities and tools are provided to encourage student-student collaboration (i.e., web conferencing, instant messaging, etc.) if appropriate. 8. Documents attached to modules are in a format that is easily accessed with multiple operating systems and productivity software (PDF, for example). Points possible Per Category Committee Mean Committee Mode Teaching and Learning 1. Student-to-Student interaction and Page

12 Faculty-to-Student interaction are essential characteristics and are facilitated through a variety of ways. *** 2. Feedback on student assignments and questions is constructive and provided in a timely manner. *** 3. Students learn appropriate methods for effective research, including assessment of the validity of resources and the ability to master resources in an online environment. *** 4. Students are provided access to library professionals and resources that help them to deal with the overwhelming amount of online resources. 5. Instructors use specific strategies to create a presence in the course. Points possible Per Category Committee Mean Committee Mode Social and Student Engagement Students should be provided a way to interact with other students in an online community. Points possible Committee Mean Committee Mode Per Category Faculty Support Technical assistance specifically for online course development and Page 11

13 online teaching is provided for faculty. *** 2. Instructors are prepared to teach online education courses and the institution ensures faculty receives training, assistance, and support at all times during the development and delivery of courses. *** 3. Faculty receives training and materials related to Fair Use, plagiarism, and other relevant legal and ethical concepts. *** 4. Faculty is provided on-going professional development related to online teaching and learning. 5. Clear standards are established for faculty engagement and expectations around online teaching. 6. Faculty workshops are provided to make them aware of emerging technologies and the selection and use of these tools. Points possible Per Category Committee Mean Committee Mode Student Support Before starting an online program, students are advised about the program to determine if they possess the selfmotivation and commitment to Page

14 learn online. *** 2. Before starting an online program, students are advised about the program to determine if they have access to the minimal technology required by the course design. *** 3. Students receive (or have access to) information about programs, including admission requirements, tuition and fees, books and supplies, technical and proctoring requirements, and student support services prior to admission and course registration. *** 4. Students are provided with access to training and information they will need to secure required materials through electronic databases, interlibrary loans, government archives, new services and other sources. *** 5. Throughout the duration of the course/program, students have access to appropriate technical assistance and technical support staff. *** 6. Student support personnel are available to address student questions, problems, bug reporting, and Page

15 complaints. *** 7. Students have access to effective academic, personal, and career counseling. 8. Minimum technology requirements for skills and equipment are established and made available to students. 9. Student support services are provided for outside the classroom such as academic advising, financial assistance, peer support, etc. 10. Policy and process is in place to support ADA requirements. 11. Students are provided easy access to required course materials in print and/or digital format, such as ISBN numbers for textbooks, book suppliers, and delivery modes. 12. Program demonstrates a student-centered focus rather than trying to fit service to the online education student in on-campus student services. 13. Efforts are made to engage students with the program and institution. 14. Students are instructed in the appropriate ways of communicating with faculty and students. 15. The institution provides guidance to both students Page

16 and faculty in the use of all forms of technologies used for course delivery. 16. Tutoring is available as a learning resource. 17. Students are instructed in the appropriate ways of enlisting help from the program. Points possible Per Category Committee Mean Committee Mode Evaluation and Assessment 1. The program is assessed through an evaluation process that applies specific established standards. *** 2. A variety of data (academic and administrative information) are used to regularly and frequently evaluate program effectiveness and to guide changes toward continual improvement. *** 3. Intended learning outcomes at the course and program level are reviewed regularly to ensure clarity, utility, and appropriateness. *** 4. A process is in place for the assessment of support services for faculty and students. 6. Recruitment and retention are examined and reviewed. 7. Program demonstrates compliance and review of accessibility Page 15

17 standards (Section 508, etc.) 8. Course evaluations are examined in relation to faculty performance evaluations. 9. Faculty performance is regularly assessed. 10. Alignment of learning outcomes from course to course exists. 11. Course evaluations collect student feedback on quality of content and effectiveness of instruction. Perfect Score= Page 16

18 Findings Institutional Support Strength With regard to the items falling under the heading of Institutional Support, the group felt the university has an appropriate governance structure to support decision-making concerning online learning. The selection of Canvas as a replacement of the Learning Management System Angel was cited as an example. The group also acknowledged there was a policy in place concerning intellectual property and that the university strategic plan, which aligns with the institution s mission, recognizes the strategic value of online learning to the university and to the stakeholders it serves. Weakness The group acknowledged the existence of a basic security method for ensuring a student who enrolled in an online class was indeed the same student who completed the work for the class (i.e., secure login and password, proctored exams, and judicial code sanctioning violation of terms of use) but felt more elaborate identification techniques needed to be employed. In some instances, faculty required online students to submit to proctored exams. A variety of possibilities exist and the Task Force recommends that these be investigated. Technology Support Strength The Task Force found existing technology support to be robust, reliable, and appropriate and noted the strengths of the current Learning Management System Angel and the enhanced features to be provided by Canvas which will replace it in the coming academic year. Course Development and Instructional Design Strength The Task Force acknowledged the strengths of the existing course development and instructional design with regard to guidelines for syllabi, definition of student learning outcomes, use of instructional materials, and the central role of faculty in curriculum development. Weakness Acknowledging that student outcomes in online and face to face classes were very similar, the Task Force believes more can be done to promote student success and retention. The Task Force believes faculty and students alike would benefit from the use of standards for online course development and delivery and encourages the use of the Quality Matters rubric in the development of online classes. It was also felt that additional outreach to inform faculty of current technologies and emerging technologies that could enhance course development would also be beneficial. Page 17

19 Course Structure Strength Various elements of online course structure were evaluated by the Task Force: access to course materials, access to library resources, access to syllabi, accommodations for disabled students, opportunities for student collaboration, expectations for student assignments, access to technical issue resolution, etc. were all found to be adequate and appropriate. Teaching and Learning Strength The group felt that the Learning Management System provides mechanisms for student to student and faculty to student interaction. This has been borne out by audits conducted by the WEP office. The Library provides opportunities for students to gain online research skills and provides access to librarians remotely to assist online students with their needs. SAI results suggest that the majority of students are satisfied with the timeliness of feedback they receive in online courses. Social and Student Engagement Strength The Learning Management System provides a mechanism for students to interact within a given online class. Online students also are afforded opportunities to participate in registered student organizations. However it was expressed that greater efforts should be made to create an online community presence for online students through such social networking media as Facebook and Twitter. Faculty Support Strength The Task Force recognized that many elements of faculty support for online instruction are in place and appropriate, including: assistance for course development, training for faculty who teach online, information on plagiarism and fair use of instructional materials, on-going professional development, and workshops providing information on emerging technologies. Weakness Task force members could not identify the existence of clear standards for faculty engagement and expectations around online teaching. They believe that additional opportunities for faculty professional development related to online teaching and learning would be greatly beneficial. Use of the Quality Matters system to develop standards for faculty instruction and engagement is strongly encouraged. Additionally the Task Force supports the establishment of a voluntary certification process for faculty who wish to pursue it. Page 18

20 Student Support Strength The vast majority of items related to student support services were found to be adequate and appropriate by the Task Force. Among these were student access to program information, admissions requirements, tuition and fee information, access to technical support, access to training materials to pursue online courses, access to course materials, efforts to engage students with the institution, access to tutoring, information on how to seek assistance, etc. Weakness While the Task Force was clearly positive about the variety of resources available to support students pursuing online learning, the Task Force believes that access to certain support functions is not always available to them when needed (i.e., outside of normal business hours). The Task Force believes, that access to financial aid, registrar assistance, advisors and counselors should be expanded. Recognizing that it is not pragmatic to provide these services on a 24/7 basis, the Task Force believes critical support services should be provided on an extended basis at key points in the academic calendar to better meet student needs. Evaluation and Assessment Strength The Task Force noted that online courses and programs are subject to the same assessment protocols as face to face courses and programs. This includes assessment of faculty performance in the online setting. Weakness Notwithstanding, the Task Force determined that assessment methods could be more specifically tailored for online instruction to include an assessment of materials used, support services, student success, faculty responsiveness, etc. This information should be collected more systematically and provided to the relevant departments/faculty for action. The Task Force also suggests that faculty coordinated by the TLAI develop a suitable student perception of instruction instrument specifically geared to key features of online to provide additional feedback to faculty to enhance online instructional delivery. Page 19

21 Recommendations Recommendations were developed by the Task Force after reviewing the findings above: 1. Student identity authentication strategies/technologies, including such things as online proctoring, should be identified and implemented that will ensure a student completing an online course is indeed the same student who enrolled in the course. 2. The Teaching, Learning, and Assessment Initiative (TLAI) and the Faculty Senate s curriculum teams should explore standards for online course design and delivery to enhance student retention and course quality beyond the current record of student success. Greater use of the Quality Matters rubric in the design of online courses should be encouraged. TLAI and faculty should develop a student perception of instruction (SPoI) especially suited to address online learning. 3. Web, e-learning, and Publications (WEP) should expand its outreach to faculty with regard to disseminating information on current and emerging technologies to improve online learning. 4. Faculty professional development opportunities associated with online student instruction and assessment should be greatly increased. A certification process for faculty teaching online utilizing the Quality Matters system should be established. 5. Access to online student services support should be extended at critical times. 6. The Enrollment Retention Management Committee of the Planning and Budget Council should examine student retention in online courses and programs, share data with relevant departments and program faculty, and recommend strategies for improvement. 7. A strategic plan for online learning should be developed. Recommendation seven is intended to provide a more coherent set of strategies for the further development of online learning at FGCU that will ensure its growth is commensurate with enhanced quality and continues to be well aligned with FGCU s mission of access to higher education. Page 20

22 Summary FGCU has a long-standing commitment to online learning that dates back to its creation just over two decades ago. Today, 600 course sections account for 15% of FGCU s annual enrollment. The current study was undertaken to determine the status of support for online teaching and learning at FGCU. The report documents many of the strengths that have been achieved during the last 16 years. Yet there is a need to invest more resources into online learning to ensure it remains contemporary, competitive, and of high quality. Additional resources are clearly required to implement these recommendations and realize online teaching and learning s full potential. The Task Force strongly advocates for the introduction of a Distance Learning fee to provide the necessary resources. The Task Force also believes a strategic vision for online or e-learning at FGCU be developed along with a plan for its achievement. Page 21

23 Appendix 1: Sloan Instrument Page 22

24 QUALITY SCORECARD FOR THE ADMINISTRATION OF ONLINE EDUCATION PROGRAMS Not Observed Insufficient Moderate Use Meets Criterion Completely Score Points Possible Per Category INSTITUTIONAL SUPPORT The institution has a governance structure to enable effective and comprehensive decision making related to online education. Policies and guidelines are in place to authenticate that students enrolled in online courses and receiving college credit are indeed those completing the course work. Policy for intellectual property of course materials exists. (If policy exists, 1 pt.; If policy specifically addresses online course materials, 2 pts.; If policy specifically addresses online course materials and is publically visible online, 3 pts.) The institution has defined the strategic value of online learning to its enterprise and to its stakeholders. TECHNOLOGY SUPPORT A documented technology plan that includes electronic security measures (e.g., password protection, encryption, secure online or proctored exams, etc.) is in place and operational to ensure quality standards, adherence to FERPA, and the integrity and validity of information. *** The technology delivery systems are highly reliable and operable with measurable standards being utilized such as system downtime tracking or task benchmarking. *** A centralized system provides support for building and maintaining the online education infrastructure. *** The course delivery technology is considered a mission critical enterprise system and supported as such. The institution maintains system backup for data availability. Faculty, staff, and students are supported in the development and use of new technologies and skills. Page 23

25 COURSE DEVELOPMENT AND INSTRUCTIONAL DESIGN Guidelines regarding minimum standards are used for course development, design, and delivery of online instruction (such as course syllabus elements, course materials, assessment strategies, faculty feedback). *** Technology is used as a tool to achieve learning outcomes in delivering course content. *** Instructional materials, course syllabus, and learning outcomes are reviewed periodically to ensure they meet program standards. *** Courses are designed so that students develop the necessary knowledge and skills to meet learning objectives at the course and program level. These may include engagement via analysis, synthesis and evaluation. *** Learning objectives describe outcomes that are measurable. Selected assessments measure the course learning objectives and are appropriate for an online learning environment. Student-centered instruction is considered during the course-development process. There is consistency in course development for student retention and quality. Course design promotes both faculty and student engagement. Current and emerging technologies are evaluated and recommended for online teaching and learning. Instructional design is provided for creation of effective pedagogy for both synchronous and asynchronous class sessions. Curriculum development is a core responsibility for faculty (i.e., faculty should be involved in either the development or the decision making for the online curriculum choices). COURSE STRUCTURE 24 Page 24

26 The online course site or other web site includes a syllabus outlining the course objectives, learning outcomes, evaluation methods, textbook information, and other related course information, making course requirements transparent at time of registration. *** The institution ensures that all online education students, regardless of where they are located, have access to library/learning resources adequate to support the courses they are taking (SACS statement). *** Expectations for student assignment completion, grade policy, and faculty response are clearly provided in the course syllabus. *** Links or explanations of technical support are available in the course (i.e., each course provides suggested solutions to potential technical issues and/or links for technical assistance). Instructional materials are easily accessible and easy to use for the student. The course adequately addresses the needs of students with disabilities via alternative instructional strategies and/or referral to special institutional resources. Opportunities/tools are provided to encourage student-student collaboration (i.e., web conferencing, instant messaging, etc.) if appropriate. Documents attached to modules are in a format that is easily accessed with multiple operating systems and productivity software (PDF, for example). TEACHING AND LEARNING Student-to-Student interaction and Faculty-to- Student interaction are essential characteristics and are facilitated through a variety of ways. *** Feedback on student assignments and questions is constructive and provided in a timely manner. *** Students learn appropriate methods for effective research, including assessment of the validity of resources and the ability to master resources in an online environment. *** Page 25

27 4 5 Students are provided access to library professionals and resources that help them to deal with the overwhelming amount of online resources. Instructors use specific strategies to create a presence in the course. SOCIAL AND STUDENT ENGAGEMENT 3 1 Students should be provided a way to interact with other students in an online community. FACULTY SUPPORT Technical assistance specifically for online course development and online teaching is provided for faculty. *** Instructors are prepared to teach online education courses and the institution ensures faculty receives training, assistance, and support at all times during the development and delivery of courses. *** Faculty receives training and materials related to Fair Use, plagiarism, and other relevant legal and ethical concepts. *** Faculty is provided on-going professional development related to online teaching and learning. Clear standards are established for faculty engagement and expectations around online teaching. Faculty workshops are provided to make them aware of emerging technologies and the selection and use of these tools. STUDENT SUPPORT Before starting an online program, students are advised about the program to determine if they possess the self-motivation and commitment to learn online. *** Before starting an online program, students are advised about the program to determine if they have access to the minimal technology required by the course design. *** Page 26

28 Students receive (or have access to) information about programs, including admission requirements, tuition and fees, books and supplies, technical and proctoring requirements, and student support services prior to admission and course registration. *** Students are provided with access to training and information they will need to secure required materials through electronic databases, interlibrary loans, government archives, new services and other sources. *** Throughout the duration of the course/program, students have access to appropriate technical assistance and technical support staff. *** Student support personnel are available to address student questions, problems, bug reporting, and complaints. *** Students have access to effective academic, personal, and career counseling. Minimum technology requirements for skills and equipment are established and made available to students. Student support services are provided for outside the classroom such as academic advising, financial assistance, peer support, etc. Policy and process is in place to support ADA requirements. Students are provided easy access to required course materials in print and/or digital format, such as ISBN numbers for textbooks, book suppliers, and delivery modes Program demonstrates a student-centered focus rather than trying to fit service to the online education student in on-campus student services. Efforts are made to engage students with the program and institution. Students are instructed in the appropriate ways of communicating with faculty and students. The institution provides guidance to both students and faculty in the use of all forms of technologies used for course delivery. 16 Tutoring is available as a learning resource. 17 Students are instructed in the appropriate ways of enlisting help from the program. Page 27

29 EVALUATION AND ASSESSMENT The program is assessed through an evaluation process that applies specific established standards. *** A variety of data (academic and administrative information) are used to regularly and frequently evaluate program effectiveness and to guide changes toward continual improvement. *** Intended learning outcomes at the course and program level are reviewed regularly to ensure clarity, utility, and appropriateness. *** A process is in place for the assessment of support services for faculty and students. 5 Course and program retention is assessed Recruitment and retention are examined and reviewed. Program demonstrates compliance and review of accessibility standards (Section 508, etc.) Course evaluations are examined in relation to faculty performance evaluations. 9 Faculty performance is regularly assessed. 10 Alignment of learning outcomes from course to course exists. 11 Course evaluations collect student feedback on quality of content and effectiveness of instruction. Perfect Score = *** Adapted from the Institute for Higher Education Policy's report Quality on the Line: Benchmarks for Success in Internet-based Distance Education (2000). Page 28

30 Appendix 2: Responses to Sloan Instrument Items Page 29

31 Quality Scorecard for the Administration of Online Education Programs: FGCU Summary Response This draft reflects each item of the Quality Scorecard instrument; this document will be used to assist the current group with its assessment using the Quality Scorecard. Institutional Support 1. The institution has put in place a governance structure to enable effective and comprehensive decision making related to distance learning. a. Decisions related to distance learning follow established policies and practices with ultimate authority resting with the Provost. zation_6_19_07.pdf pdf Curricula development is the responsibility of the faculty and the FGCU Faculty Senate. The faculty senate policies and practices must be followed; specific to this, syllabus guidelines at: are used and curricula oversight occurs through the Senate General Education Council, Undergraduate Curriculum Team, Graduate Curriculum Team and Program Review Team as appropriate. b. Technical support for distance learning is a shared responsibility among the offices of Curriculum and Instruction (Web, E-learning), Computing Services, and Network Services. Student support services are provided by the Library, Student Affairs, the Registrar s office, and Administrative Services and Finance. 2. Policies are in place to authenticate that students enrolled in online courses and receiving college credit are indeed those completing the course work. a. Access to Learning Management Systems (LMS) is equipped with a secure login and a pass code. Policies regarding the sharing of such login and pass code is made clear in writing in the Technology Acceptable Use Policy at: pdf b. hts_92011.pdf c Policy for intellectual property for course material exists. a. Copyright ownerships of course materials exist. This policy uses general terms such as "computer assisted instructional course work", "programmed instructional materials" and "which are used to assist or enhance instruction" to address online course material. b. See policy located at: 05.pdf 4. The institution has defined the strategic value of distance learning to its enterprise and to its relevant parts. a. In the Strategic Plan found at: Goal 1, Strategy 1 specifically mentions Page 30

32 Technology Support making existing programs more accessible and Benchmark 1.3 specifically mentions "Master of Education in Special Education will be offered entirely online". b. In the same document, Goal 1, Strategy 4 states "4.2: Enhanced tools and response in support of E-Learning/Distance Learning. Conduct review of best practices in the administration of D-L". c. The Guiding Principles at states "The University employs information technology in creative, experimental, and practical ways for delivery of instruction, for administrative and information management, and for student access and support. It promotes and provides distance and time free learning." 1. A documented technology plan that includes electronic security measures (e.g., password protection, encryption, secure online or proctored exams, etc.) is in place and operational to ensure quality standards, adherence to FERPA, and the integrity and validity of information. a. b. FGCU provides secure login to course material and makes available publically the acceptable use policy at: pdf and monitors innovations in authentication technologies. 2. The technology delivery systems are highly reliable and operable with measurable standards being utilized such as system downtime tracking or task benchmarking. a. Uptime and usage reports are monitored. Usage growth is tracked per semester (file is maintained by the ANGEL system administrator). Updates, patches, etc. are done between semesters or during breaks. Backups are made at regular intervals. 3. A centralized system provides support for building and maintaining the distance education infrastructure. a. The technologies used in online courses have become available to all courses and therefore the technical infrastructure has strong support. b. Although various aspects of the technical infrastructure are supported by a number of departments, the Information Resources Committee (IRC) provides a mechanism of centralizing technology discussions and coordination efforts. The IRC updates annually service and support benchmarking data, and makes recommendations on enhancing or purchasing systems. 4. The course delivery technology is considered a mission critical enterprise system and supported as such. a. Adequate budget is in place to maintain current technological infrastructure. Benchmarks are kept and monitored for the purpose of improving and upgrading technology. An annual Student Use of Technology survey is conducted to monitor changes in student technology preferences. b. A Load Balancer is in place and monitored to avoid a single point of failure for the Learning Management System (LMS) and other technologies. There is a commitment to provide adequate support to faculty and students and hours are adjusted as needed. Page 31

33 24/7 support has been identified as a need but the magnitude of the need has yet to be ascertained. 5. The institution maintains system backup for data availability. a. System backup is in place and supported by current budget. b. A disaster recovery plan is maintained by Academic and Event Technology Services and Computing Services. 6. Faculty, staff, and students are supported in the development and use of new technologies and skills. a. Professional Development opportunities are provided by Web, e-learning and Publication Services (WEP), Teaching Learning, and Assessment Initiative (TLAI), Library, and Business Technology Services (formerly CS). TLAI and WEP hold a showcase every year to demonstrate best practices and encourage the use of new learning technology. b. Support and tutorials are maintained by WEP, Library, CS, and Academic and Event Technology Service (AETS) and are publically available to students from: Course Development and Instructional Design 1. Guidelines regarding minimum standards are used for course development, design, and delivery of online instruction a. The Quality Matters (QM) rubric is available to evaluate the quality of course. b. An online course template was developed based on the revised QM rubric. This course template is available to faculty but faculty is not required to use it. Instructional Designers are available to work with faculty to assist in the design of their online courses. The e-learning staff teaches best practices on faculty presence and response times during course consultations, design workshops and course revision projects. c. Faculty is given minimum standards for design and delivery of courses as indicated in the following syllabus guidelines: is followed. 2. Technology is used as a tool to achieve learning outcomes in delivering course content. a. The online Learning Management System (LMS) is in place and available for any class. Adobe Connect and other web conferencing tools are also available. b. WEP, TLAI, and the Library provide guidance on the use of new technologies through presentations, demonstrations and training material made available online. c. New technologies are evaluated for accessibility, affordability, usability and addressing specific instructional/course delivery needs. 3. Instructional materials, course syllabus, and learning outcomes are reviewed periodically to ensure they meet program standards. a. A modified version of the Quality Matters rubric is used to systematically review and refine instructional material. The suggested online course template encourages student feedback at the end of a module or theme as a way of reviewing course material usefulness. Page 32

34 QM_Rubriuc_ pdf b. Faculty is expected to continuously review instructional materials to ensure they meet program standards. c. An analysis of the curriculum and associated outcomes occurs in the program review process. d. Annual evaluations and promotion reviews of faculty often include a review of instructional materials, course syllabus, and learning outcomes. 4. Courses are designed so that students develop the necessary knowledge and skills to meet learning objectives at the course and program level. These may include engagement via analysis, synthesis, and evaluation. a. The objectives of each course are described in the syllabus along with the means for assessing them. is followed. b. All FGCU programs assess critical thinking, communication and content knowledge. HUM 2510 is an example of an online course that assesses critical thinking and communication skills. 5. Learning objectives describe outcomes that are measurable. a. Programs are required to develop assessment plans for the measurement and attainment of learning objectives. All programs are required to address critical thinking, written communication and discipline-specific content knowledge and skills. b. The Program Review Team reviews learning objectives as part of the program review process. 6. Selected assessments measure the course learning objectives and are appropriate for an online learning environment. a. Faculty assesses the same course learning objectives whether the course is delivered online as a hybrid or face to face, and utilizes similar assessment strategies. Outcomes for students taking either online courses or online programs and student feedback indicate these strategies are appropriate. Hybrid is defined as a course that has face to face sessions and provides components of the course online. Syllabus and portfolio evaluations may occur as an example of the mechanisms used to determine whether or not course learning objectives are appropriate for an online environment 7. Student-centered instruction is considered during the course development process. a. Examples of challenge questions that trigger individual responses and motivate interaction among learners can be seen within the online courses. Using the QM Rubric aids in checking a course for engaging activities. QM standards: i. 5.2 Learning activities provide opportunities for interaction that support active learning. ii. 5.4 The requirements for student interaction are clearly articulated. iii. 6.2 Course tools and media support student engagement and guide the student to become an active learner. 8. There is consistency in course development for student retention and quality. Page 33