Custom Research Brief May 4, 2009

Transcription

1 S T U D E N T A F F A I R S L E A D E R S H I P C O U N C I L Online Academic Support Programs Custom Research Brief May 4, 2009 TABLE OF CONTENTS RESEARCH ASSOCIATE Nina Capriotti RESEARCH DIRECTOR Jennifer Yarrish I. Research Methodology & Parameters II. III. IV. Executive Overview Academic Support Services Programs Online Platforms for Academic Support Programming V. Assessments for Online Academic Support Services 2009 THE ADVISORY BOARD COMPANY WASHINGTON, D.C.

2 I. RESEARCH METHODOLOGY & PARAMETERS Project Challenge: A private university located in the Northeast approached the Council with the following questions: What are the academic support services that are offered in an interactive (real time or not) format online? What are the platforms used by institutions for online tutoring and other services? How is the service being assessed? Sources: During the course of the research, the Council searched the following resources to deepen our understanding of online academic support in higher education, including: Education Resources Information Center (ERIC) The Chronicle of Higher Education Research Parameters: Based on the member institution s request, the Council focused its research on universities that currently offer academic student services in an online format. Over the course of research, the Council spoke with directors of learning and academic support centers with online programs as well as technology department administrators overseeing university elearning programs. The term synchronous used in this brief refers to online services that permit dynamic, real-time, and instant interactions. Correspondingly, asynchronous refers to those online services that convey electronic communication in a static, one-directional manner by The Advisory Board Company 2

3 II. EXECUTIVE OVERVIEW A Guide to Universities Profiled in this Brief Institution Location Classification Enrollment (Total/Undergraduate FTE), Fall 2007 University A Midwest Research University, very high research activity 40,000/33,000 University B Mideast Large Master's College/University 7,000/6,000 University C Southeast Research University, high research activity 7,000/6,000 University D Midwest Large Master's College/University 5,000/3,000 University E Midwest Research University, very high research activity 40,000/26,000 University F Northeast Research University, very high research activity 9,000/5,000 University G Southwest Master s University, larger programs 29,000/25,000 University H Southeast Research University, very high research activity 30,000/23,000 SOURCE: National Center for Education Statistics and U.S. News & World Report 2009 by The Advisory Board Company 3

4 II. EXECUTIVE OVERVIEW (CONT.) Key Observations Universities select internally developed or externally purchased platforms to provide online academic support depending on available resources, the level of in-house expertise, and student population needs. Although externally developed platforms are costly and uncustomizable, contacts state that their value extends past online academic support. Specifically, faculty often use these platforms for research collaboration with other professors. Internally-developed platforms take time to create and demand in-house expertise but are customizable for specific purposes and are easily integrated into university-wide systems. Typically, students display an initial hesitation to access online academic services; however, once they have used the services, they are likely to return. Contacts posit that the new and unfamiliar technology, lack of awareness of services offerings, and awkward interactions created by online communication all hinder students use. Most universities recruit peer tutors on a voluntary basis, and interest in volunteering has been high. For many universities, approximately two-thirds of all student tutors volunteer. Peer tutors are particularly drawn towards online tutoring because they can work from their own computers at home. There are two measures for providing feedback on online academic support services: online student surveys, and login data and statistics. These measures profile student users as well as peak usage times. Most contacts have not been able to complete comprehensive assessments because many online academic support programs are still in beginning stages. While surveys are used to evaluate the effectiveness of online academic support programs, most contacts express difficulty in achieving healthy response rates. These surveys, administered online through or a pop-up window, are not mandatory, and thus are often neglected by students by The Advisory Board Company 4

5 III. ACADEMIC SUPPORT SERVICES PROGRAMS While a few universities offer online study groups and workshops, the most common academic support service offered by universities in an interactive online format is tutoring. Universities offer this service synchronously and asynchronously using various programs. The content area ranges from mathematics and science to writing and composition, although the latter topics are more prevalent. Currently, none of the institutions offer online study skills or mentoring in an online format because general utilization of online academic services has not been robust; thus, most institutions have not yet expanded these services beyond initial content area offerings. Academic Support Services Programs University B Online Service Interactive Content Area Drop-In or Appointment/ Peak Usage Intake Process Eligibility Cap Usage for Services Tutoring services Yes, Synchronous: Students interact with a peer tutors through live chat and whiteboard window during 45 minute sessions. Composition and writing Freshman chemistry and biology: Only offered for one semester and terminated because of low student use. Drop-In: Services are offered Monday through Thursday from 9pmmidnight. Contacts decided on this timeframe because it directly follows in-person tutoring (8am-9pm). Peak Usage: The most popular times are between 9-10pm. Web Site: A drop-down item on the Web site directs students to upload their assignment and click on the tutoring application. If the service is available, a whiteboard window immediately opens on the student s screen. If students send assignments when the office is closed, a setting informs them to visit later, and the Web site does not store the document. If applicable, students see the queue and calculate how long their wait may be (the window rings when their turn approaches). First-Year Students: Currently, first-year students in any composition course may utilize the service. Contacts note that next semester services will be open to composition courses at any level. Service is limited to twice weekly: Students can only use the service two times per week, which ensures greater availability for more students. There is no fee for the service by The Advisory Board Company 5

6 III. ACADEMIC SUPPORT SERVICES PROGRAMS Academic Support Services Programs University C Online Service Interactive Content Area Drop-In or Appointment/ Peak Usage Intake Process Eligibility Cap Usage for Services Tutoring services Yes, Synchronous: During a real time 20 minute session, peer tutors administer advice through text chat, a whiteboard application, and an audio component. Developmental Math: Services range from developmental math and non-credit math courses to the first level of calculus. Drop-In: Services are provided by one peer tutor from 7-10pm Monday through Thursday. Peak Usage: Contacts explain that more students access the program later in the evening, towards 10pm. As a result, the Center may extend operating hours to 11pm. Web Site: Students who need tutoring access the service online, which leads them to the lobby area. In the lobby, users view a window that lists incoming users as they arrive, welcomes users, explains expectations, and asks that they be prepared prior to the session. As tutors become available, they enter the lobby area and escort the new student to a private break-out room for tutoring. All Students: Any student with questions relevant to the content area can utilize the math tutoring program. Contacts note that basic math courses are a requirement for most concentrations, so many students could potentially utilize the service. No limits to usage: The center requests that sessions do not last longer than 20 minutes; however, students can re-visit the center an unlimited number of times. Contacts note that only a few students utilize the program, but those who do use it heavily by The Advisory Board Company 6

7 III. ACADEMIC SUPPORT SERVICES PROGRAMS Academic Support Services Programs Online Services Tutoring services Online, Interactive Learning Workshops Interactive Yes, Synchronous: Students join peer tutors in online Elluminate V-rooms 1, which facilitate one-onone interactions through real time text chat, whiteboard application use, audio, and video. Yes, Synchronous: During onehour workshops, the leader teaches a brief lesson about a particular topic using PowerPoint. Students then apply these teachings to preplanned practice exercises, which are returned with feedback from the leader. Content Area Composition and writing A mathematics tutoring service is currently being piloted. General study skills (e.g. active reading) Mathematics Composition and writing University D Drop-In or Appointment/ Peak Usage Appointment-only: Students must contact the office via phone or to arrange a tutoring appointment. Peak Usage: None specified. Registration Required: Individuals register online and RSVP to the workshop. Intake Process Phone or Once a student makes a request via or phone, the tutor sets up an appointment and sends an with the link to the V-room. Web Site: After students register online for the workshop, they are sent an with the appropriate V-room link. Eligibility All students: While all students qualify, freshman and students enrolled in writing composition courses most often utilize the online service. All students are entitled to participate: Contacts explain that the goal of online workshops is to draw participation from the entire campus, which inevitably encourages an e-learning environment. Cap Usage for Services Service is unlimited and free. There is no fee for this service. 1 As one of Elluminate Inc. s online learning products, V-rooms are elearning interfaces that provide users with the opportunity to meet privately and interact over the internet by The Advisory Board Company 7

8 III. ACADEMIC SUPPORT SERVICES PROGRAMS Academic Support Services Programs Online Service Tutoring services University E Interactive Content Area Yes, Asynchronous: During a 60-minute time period, tutors review submitted student writing assignments. Tutors make comments throughout the papers and include an extensive letter at the front of the assignment, which delivers detailed suggestions and feedback. Students can access their returned submissions online, usually within 72 hours. Composition and writing Drop-Off: Peer tutors sit in the center from 7-11pm on all weeknights to review student submissions. Drop-In or Peak Usage: In fall 2008, almost all of the available hours of this service Appointment/ were fully utilized; the director added additional hours to meet demand Peak Usage for the semester. There are currently 150 submissions for spring 2009, and the semester has not yet ended. Utilization increases in the fall due to the increased number of writing classes offered at that time of year. Intake Process Eligibility Cap Usage for Services Web Site: 1) Students submit an online form with both multiple-choice and fill-in-the-blank information, including grade level, number of times they have utilized the service, what type of help is needed, the assignment information, and the specific challenge they are hoping to resolve. Students also attach their assignments to the form before submitting this information to the Web page. 2) The submission goes into a depository, which is monitored by the office dispatcher. 3) The student dispatcher manages all of the incoming forms and delegates assignments to the various tutors by placing them in tutor portfolios (which are similar to inboxes). All Students: Any student with a university identifier and password can use this service. This includes students that are off-campus or abroad (as long as submissions are in English). There are no limits to using this service. Contacts mention that branch campuses can access the service; to date, no branch campus students have done so. In the event that branch utilization becomes widespread, the university would require supplemental funding from the branch campuses by The Advisory Board Company 8

9 III. ACADEMIC SUPPORT SERVICES PROGRAMS Academic Support Services Programs Online Service Interactive Tutoring services through refresher courses No, Asynchronous: Plato, a web-based program, provides supplemental instruction that complements university coursework and helps students review critical skills. Content Area Mathematics Writing History Chemistry Reading Geography University G Drop-In or Appointment/ Peak Usage Intake Process Eligibility Cap Usage for Services 24-Hour Access: After students obtain a user name and password to the program they can access the course online at any time. Peak Usage: Contacts have not recently completed an assessment of this program. Office Sign-Up: Potential users fill out a form with their student information at the office and receive a personalized username and password in return. Through a link on the office s Web site, users can use this information to log in and begin their session. All Students: Contacts explain that most users come for mathematics and chemistry assistance. There are no limits or costs for students using this service. Homework Delivery at the University F Through the learning services office at University F, students access a homework delivery program, called WebWork, for additional coursework practice. This public, online program presents users with individualized problem sets relating to mathematics, economics, physics, and other disciplines with quantifiable subject matter. Because WebWork is posted on the World Wide Web, anyone can access the program at any time for free. Contacts note that the university has not completed control studies or evaluations of the program by The Advisory Board Company 9

10 III. ACADEMIC SUPPORT SERVICES PROGRAMS University H Online Service Interactive Content Area Drop-In or Appointment/ Peak Usage Academic Support Services Programs Tutoring services are not available to the entire student population but are currently offered to distance learners upon request. The institution hopes to expand tutoring services to all students but plans to offer the services to all distance learners first. The institution further hopes to offer online faculty advising by the fall semester. Currently, the elearning student office implements this program, but the student learning center oversees the service. Study Groups, called cohort class groups, are presently offered to distance learning classes. Yes, Synchronous: Using a developing project The Virtual Student Center, students can interact in real-time through chat features and desktop (and consequentially application) sharing. All Areas: This program allows students to connect with current peer tutors; thus, services cover any content area as long as a tutor with such expertise is already in place at the institution. In the future, this virtual center will serve as a centralized place to access all types of student services. Appointment-only: Sessions are coordinated by the elearning student office and scheduled with the learning center. Peak Usage: Because The Virtual Student Center has not been fully launched, the tutoring services are not yet available to all students. The services have only been used by one distance learning student upon request. Specific Requests: At this time, only distance learning professors using the related platform for online teaching can offer study group sessions. After the project fully launches, the service will open to all courses. Appointment-only: Students in distance learning classes must work with the elearning student coordinator to create study sessions. Contact elearning Student Coordinator: Individualized: Students must Intake This coordinator serves as a liaison for online work with professors to set up Process student tutoring, filtering requests. study sessions. Eligibility Cap Usage for Services Distant Learners: The virtual center requires university username and password to enter. Within the next year, all students will have access to the program. Students enrolled in online courses: At this time, only students taking online courses can utilize the service. Limits have not been set because the program has not been fully launched by The Advisory Board Company 10

11 2009 by The Advisory Board Company IV. ONLINEPLATFORMS FOR ACADEMIC SUPPORT PROGRAMMING Universities offering online academic support develop programming internally or externally depending on in-house technology expertise, monetary and staff resources, and existing platforms already used by the university. Below are comparisons of synchronous internally-developed platforms, synchronous externally-developed platform, and asynchronous internally-developed platforms used by universities to offer online academic support. Institution Program Overview Costs University F University H Platform: WebWork is a Javabased web application framework developed by OpenSymphony Age of Program: 14 years Pros: Instant feedback for users, 24/7 student access, individualized problem sets, and constant development by international audiences Cons: None cited Platform: Centra platform was used to create the university's Virtual Student Center, which hosts a variety of online static and interactive services such as online tutoring. Age of Program: 1 semester Pros: Highly credible, multifunctional, thoroughly tested, and frequently updated Cons: None cited Synchronous Platforms Developed Internally As an opensource community, the program is free and can be accessed domestically and internationally. Because the university already had the license, the platform has no additional cost for the elearning student office. Reasons for Program Choice Presents students with exercises that directly complement course material. Developed by international professionals and experts. Other Platforms Attempted: None specified. Provides a private forum for services (the program is only accessible with university username and password). Has the capacity to offer all student services online. Contacts forecast that by the fall semester, the virtual center will have many new components, including a career services center, virtual professor offices, departmental material, and test-taking anxiety information. Other Platforms Attempted: Silicon Chalk, Breeze Presenter and other platforms proved less advanced. Staffing Program: No additional staff necessary. Technology Maintenance: Mathematicians at the university helped implement and develop the program. Program: An elearning student coordinator position was created to expand online services offered to students. Technology Maintenance: Internal specialists maintain and develop the program. Student Response Utilization Rate: The university has not completed control or usage studies. Utilization Rate: The program has not been fully launched; thus, services are currently offered only when necessitated (i.e., distance learners requiring additional learning assistance). 11

12 IV. ONLINEPLATFORMS FOR ACADEMIC SUPPORT PROGRAMMING (CONT.) 2009 by The Advisory Board Company Institution Program Overview Costs University B University C Platform: Worldwide Whiteboard; licensed membership with Link System Age of Program: 2 years Pros: Helpful and available technical support; built-in usage and transcript tracking; multiple uses beyond tutoring Cons: None specified Platform: Wimba platform working through Angel system Age of Program: 1 st semester (pilot stage) Pros: Affordable and easy to set-up because the university already had the user license; compatible with the university-wide Angel system Cons: Converters are necessary to use the equation editor feature, but this can be avoided with creative solutions such as rotating text boxes Synchronous Platforms Developed Externally When piloted with limited access (restricted to 20 online users at any given time), the program cost student services and the academic support program $3,500 per year. Now, an alumni donor funds the $10,000 annual license. Contacts do not know the cost of using the WIMBA platform since the university pays for the license. Reasons for Program Choice Permits real-time tutoring. Costs less than other options. Other Platforms Attempted: Contacts tried Elluminate, which was extremely costly and did not supply the services outlined in the limited license agreement. Utilizes in-house tutors. Provides a waiting room, which gives sessions privacy and allows users to see a queue. Serves as a private service only for the university community. Incurs no cost for the Center since the university purchases a license. Other Platforms Attempted: The university used Elluminate, which satisfied the above criteria except for cost, and Ask Online and Tutortrac, both which met too few standards. Staffing Program: No additional staff has been added to support online services. Technology Maintenance: Link System provides technical support, which proves extremely useful. Link System support is so receptive when you have a problem. Program: Because the tutoring coordinator assumes all tutoring program responsibilities, no additional staff was needed. Technology Maintenance: In addition to overseeing program operations, the tutoring coordinator handles all technology components of the program. Student Response Utilization Rate: The biggest obstacle has been getting students to use the service. It s almost as if we re invading the students space by offering these services online; they don t know who is on the other side of the technology. Utilization Rate: Contacts express satisfaction with the platform and hope to increase student utilization. Currently, only a few students utilize the program, but those who do access it often. Accessibility: The platform is fairly accessible, with adaptations for both visual and hearing impaired. All tutors are given headsets to facilitate the use of these features. 12

13 IV. ONLINEPLATFORMS FOR ACADEMIC SUPPORT PROGRAMMING (CONT.) 2009 by The Advisory Board Company Institution Program Overview Costs University D Platform: Online V-Rooms and Elluminate platform license Age of Program: V-rooms have been used for 2 years; the license has been in use for 1 year Pros: Easy to learn; highly accessible for all types of users; closely resembles face-to-face interactions; and multi-functional. Elluminate so closely replicates face-to-face tutoring appointments. That s why we like this software so much because you can use audio and even video with the license to give you that inperson feel. Cons: Students using outdated software cannot support all of the technology s capabilities; unsecure wireless connections interrupt program functions. Synchronous Platforms Developed Externally Contacts do not know the cost of the platform because the university s technology learning office pays for the licensing. Reasons for Program Choice Replicates face-to-face communication. Came highly recommended by the university s technology learning office. Other Platforms Attempted: Prior to Elluminate, the university offered online tutoring services through Blackboard, but contacts desired more than Blackboard s asynchronous communication. Staffing Program: No additional staff has been necessary to accommodate these services. Technology Maintenance: The university s technology learning office oversees the platform s operations. Student Response Utilization Rate: Students access online tutoring much less than tutoring in-person, but students who have used the service are very likely to return for additional online services. The institution hopes to continue promoting the service and expanding offerings to garner interest. Accessibility: Contacts describe Elluminate as accessible and compatible with adaptive computer equipment. Asynchronous Platforms Developed Externally Institution Program Overview Costs Reasons for Program Choice Staffing Student Response University G Platform: The online program is developed and maintained by Plato Learning Environment, Inc. Age: Approximately 4 years Pros: Content areas vary greatly and users can access service at any time. Cons: Program is not synchronous or interactive and lacks detailed feedback for student users. The cost of a yearly license is $900. This includes access to all subjects. Contacts note that the current program was selected by previous staff for unknown reasons. Other Methods Attempted: Contacts have not been able to explore other options, but soon plan to experiment in an effort to offer interactive and synchronous online services. Program: No additional staff has been added to support online services. Technology Maintenance: Maintenance is covered by the Plato Learning Environment, Inc. Utilization Rate: Contacts note that the university s in-person tutoring services, which have greatly expanded in recent years, are much more popular than the online services. Further, the learning center encourages only students needing refreshment on particular subjects to use this service because of its non-interactive format, lack of extensive feedback, and noncustomized content. 13

14 IV. ONLINEPLATFORMS FOR ACADEMIC SUPPORT PROGRAMMING (CONT.) 2009 by The Advisory Board Company Institution Program Overview Costs University E Platform: The program that created the Online Writing Lab (OWL) interface is an in-house, web-based program available to the university community. The Dreamweaver application was used to create the fields in the interface before it was imported into program. Additionally, tutors provide feedback using the Professional Edition Adobe Software. Age of Program: Approximately 2 years Pros: Benefits of the overall program include interface control, customization, and internal database information collection at no cost. Further, Adobe Professional allows tutors to deliver non-invasive and clear feedback for students. Cons: System is not synchronous and lacks an automated return system (i.e. as tutors complete their reviews, students must return to the Web site to retrieve their completed submissions). Asynchronous Platforms Developed Internally Since the interface was developed inhouse, the program is offered at no cost to the center. Reasons for Program Choice Recommendation by an in-house programming specialist working in the instructional support office. Urgency around offering tutoring services in an online format. Other Methods Attempted: Contacts experimented with an system to collect student submissions, but the system faced coding issues and the inbox limited the number of submissions. Staffing Program: No additional staff has been added to support online services. Technology Maintenance: The IT specialist, who created the program, also oversees it. In addition, the director troubleshoots, completes administrative work, and prepares online records and forms. Student Response Utilization Rate: Contacts describe the service as widely popular amongst students and heavily utilized. Also, while participation in the online side of tutoring is voluntary, twothirds of all tutors opt to offer these services. Spotlight: University A s Creation of a Synchronous, Interactive Online Writing Lab (OWL) Contacts describe the Online Writing Lab (OWL) at University A as an internet-based writing instruction program, sponsored and maintained by the Department of English. With ongoing development and expansion, the Web site now hosts PowerPoint presentations, podcasts, flash videos and other media forms presenting critical writing and composition guidance. Although not yet synchronous or interactive, the OWL offers abundant information and serves as the leader and model writing lab across the nation. Currently, University A is internally developing an online, interactive program for writing and composition tutoring. Contacts explain that the university chose to develop this platform internally because the university wants to meet the specific needs of the student population using the platform and to make this platform accessible only to the university community. Like all OWL material placed online, this service is designed according to empirical research conducted by researchers in the Department of English. 14

15 V. ASSESSMENTS FOR ONLINE ACADEMIC SUPPORT SERVICES Overall, assessments show that online academic support services are useful, yet under-utilized. This new form of academic support is new and unfamiliar to students, and students consequentially hesitate to use it. While a few contacts identify the difficulty of marketing online services to students as the reason for under-utilization, others believe that utilization is low because students use technology to communicate with each other not with authority. As contacts from University B note, It s almost as if we re invading students space. They seem uncomfortable using the online services because they don t know who is on the other side of things. Aside from somewhat low utilization, contacts note that the programs are effective. Even though learning outcomes have not been measured by any of the institutions, many contacts state that students who use the service find it helpful and return for continued use. To learn more about the success of online academic programming, universities utilize survey and tracking assessments. The following describes the two measures used by universities to evaluate online academic support services. Measure #1: Student Survey The primary way universities evaluate online academic support is through a student survey, administered after the completion of a session. These surveys assess students satisfaction with the structure and format of the relevant service rather than assessing learning outcomes, gathering qualitative information through multiple-choice and open-ended questions. To access the survey, student users either receive a link to the online survey through their university address or meet a pop-up screen upon completion of their session. Common questions in the survey include: Was this service helpful/useful? Why did you log off? Was the technology easy to use? Did this help with your assignment? University D uses a university-wide survey to evaluate its online academic support programs. The program coordinator also created a blog to share information about the university s Elluminate workshops. At University E, students are required to fill out a survey before their session and requested to fill out an additional evaluation survey at the end of the session. The initial survey helps to inform the tutor about the student s needs and also provides information such as the student s class year, the number of times the student has used the service, and the primary courses troubling the student. University B uses two different surveys, one that is given to all students and the other that is administered additionally to first-time users. This additional survey helps to monitor if the services are spreading to new users. Contacts explain that while the online survey can provide useful feedback, student response rates are continually low. Surveys are not mandatory at any university by T he Advisory Board Company 15

16 V. ASSESSMENTS FOR ONLINE ACADEMIC SUPPORT SERVICES Measure #2: Built-in Assessment/Tracking The second measure used to evaluate online academic support programs is built-in assessment tracking. This feature allows the university to collect usage information, such as who is using the service, what time the service is used, and for how long the service is accessed by each user. The Wimba platform has a built-in feature that the University C uses for assessments. Specifically, platform administrators can observe who is online and how long the users have been online. The information will be used at the end of the semester to run assessments about the success of the tutoring program. University B uses Worldwide Whiteboard to track the online tutoring interactions. This includes information such as when users log in but also includes a recording of the entire transcript of the conversation (both chat conversations and whiteboard edits). These logs are used for program assessments and as teaching points during tutor trainings to demonstrate effective tutoring interactions. Contacts note that Linksystem also has data points but they have not yet used the information to evaluate programming. University E s database program stores various types of information in the system, including who uses the services, when and what days it is used, what courses are accessing the service the most, and similar details by T he Advisory Board Company 16

17 PROFESSIONAL SERVICES NOTE The Advisory Board has worked to ensure the accuracy of the information it provides to its members. This project relies on data obtained from many sources, however, and The Advisory Board cannot guarantee the accuracy of the information or its analysis in all cases. Further, The Advisory Board is not engaged in rendering clinical, legal, accounting, or other professional services. Its projects should not be construed as professional advice on any particular set of facts or circumstances. Members are advised to consult with their staff and senior management, or other appropriate professionals, prior to implementing any changes based on this project. Neither The Advisory Board Company nor its programs are responsible for any claims or losses that may arise fro m any errors or o missions in their projects,whether caused by the Advisory Board Company or its sources The Advisory Board Company, 2445 M Street, N.W., Washington, DC Any reproduction or retransmission, in whole or in part, is a violat ion of federal law and is strictly prohibited without the consent of the Advisory Board Company. This prohibition extends to sharing this publication with clients and/or affiliate companies. All rights reserved.