A Replicable Model of Online Mentoring: An Ongoing Study Rana Khan Director, Biotechnology Program University of Maryland University College Arhonda Gogos External Evaluator Introduction The Professional Science Master s Biotechnology program (http://www.umuc.edu/programs/psm.shtml) at the University of Maryland University College (UMUC) is offered fully online and has students who are geographically dispersed, with some even living abroad. As one of 11 degree granting institutions in the University System of Maryland, UMUC has a reputation of providing quality programs to the working professionals. One of the primary goals of the Biotechnology program is to develop a strong and sustained relationship with the industry since the majority of the graduates are either employed or will seek jobs in the private sector. Our mission is to assist students think about and develop their career goals, as they progress through the degree program, as well as identify ways to achieve them. To achieve the aim of fostering a closer relationship with the biotechnology industry and also guide the students towards setting realistic career goals, we have developed a novel, web-based mentoring program (psmmentoring.umuc.edu) that can easily be replicated and customized to fit the needs of other programs or departments at any institution. Empirical studies indicate that mentoring programs enhance student learning and have a positive impact on the personal and professional development of students (Levinson, 1978). A more recent study at Stanford University (Bettinger et al., 2011) indicates that coaching of undergraduate students leads to 13% higher completion rate and 10-15% higher retention rate. Biotechnology Professional Mentoring Program Web-Based Mentoring Model To address the needs of our students and in-line with the mission of the biotechnology program, we have developed a novel online mentoring program that pairs students with mentors from the biotechnology industry, with the goal of improving student retention and students career prospects. In the model developed by UMUC, students within the first 18 credits of the master's program are eligible to apply for the mentoring program. The ones selected are paired with a mentor, a volunteer from the biotechnology industry. The students seek advice from the mentors as they progress with their degree. Each pair is assisted by a mentor assistant, an alumnus of the biotechnology program. This model has the following key features that set it apart from other mentoring programs: It is offered at graduate level and is embedded in the degree program. It utilizes Web-based technologies that enable easy access and participation, provide flexibility and easier management of resources. It provides industry guidance to students in exploring and developing realistic career goals It employs Mentoring Assistants, graduates of the degree program, to facilitate the interaction of the mentors with the mentees. It is potentially sustainable through the participation of program graduates as mentors. Copyright 2011 The Board of Regents of the University of Wisconsin System 1
Specific objectives addressed through the mentoring program include: Assisting students in exploring and developing realistic career goals. Preparing students with an awareness and skill set for successful careers. Providing an opportunity for the industry to have a direct say in shaping future employees. Developing online components to enhance the mentoring interaction and data collection. Disseminating the model to other programs and institutions. The pilot of the mentoring program was launched in the Fall of 2009 with 19 pairs of mentors/mentees. Since then, the participation has more than doubled with 41 participating pairs in Fall 2010 and two students graduating in the Fall of 2010. Program Participation The number of students and mentors who completed each semester since the start of the program is shown in Figure 1. Blue, orange, and yellow indicate the participants who joined the program in fall 2009, spring 2010, and fall 2010 respectively. 50 40 30 20 10 0 Students per semester 18 16 14 19 15 9 Fall 09 Spring 10 Fall 10 joined in Fall 10 joined in Spring 10 joined in Fall 09 50 40 30 20 10 0 Mentors per semester 12 15 11 19 16 18 Fall 09 Spring 10 Fall 10 joined in Fall 10 joined in Spring 10 joined in Fall 09 Figure 1. Number of Students (top) and Mentors (bottom) Who Completed Each Semester. Notably, some of the students and mentors left the program in their second or third semester. Most common reason provided was time limitations due to personal or professional responsibilities. Fall 2010 End-of-Semester Questionnaires Feedback from the participants is regularly reviewed and suggestions for improvement are considered and implemented before the start of the next round of fresh applicants. Twenty-nine of 41 students and 22 of 41 mentors responded to the assessment questionnaires at the end of the fall 2010 semester. Table 1 shows a sample of their responses. For similar statements between mentors and students (e.g. setting goals, or taking advantage of networking opportunities), the students most common responses were more favorable than the mentors most common responses. Copyright 2011 The Board of Regents of the University of Wisconsin System 2
Table 1. Sample of Student (A) and Mentor (B) Responses to the Fall 2010 End-of-Semester Questionnaire. A. Sample of statements to students: My mentor is helping me set shortterm goals and work through them. My mentor is helping me develop a long-term plan for career development or career change. My mentor facilitates networking opportunities in my field of interest. B. Sample of statements to mentors: With my assistance, my mentee set short-term goals and is working through them. With my assistance, my mentee developed a long-term plan for career development or change. My mentee has begun taking advantage of networking opportunities. SA A SlA SlD D SD 34.5% 37.9% 17.2% 6.9% 0.0% 3.4% 41.4% 31.0% 13.8% 6.9% 3.4% 3.4% 44.8% 20.7% 20.7% 6.9% 3.4% 3.4% SA A SlA SlD D SD 4.5% 54.5% 31.8% 9.1% 0.0% 0.0% 9.1% 27.3% 40.9% 22.7% 0.0% 0.0% 14.3% 23.8% 47.6% 4.8% 9.5% 0.0% Note. 29 of 41 students (70.7%) and 22 of 41 mentors (53.7%) responded. SA: Strongly Agree; A: Agree; SlA: Slightly Agree; SlD: Slightly Disagree; D: Disagree; SD: Strongly Agree. The participants were also asked to rate the program on a 5-level scale (Poor to Excellent). The program ratings have been consistent through the three semesters of the program implementation. % mentors (n=22) % mentees (n=29) Poor Fair Good Very Good Excellent 0 20 40 60 80 100 Figure 2. Program Rating by Mentees and Mentors at the End of Fall 2010 (shown as % of participants for each rating level). Twenty-nine of 41 students (70.7%) and 22 of 41 mentors (53.7%) responded. Copyright 2011 The Board of Regents of the University of Wisconsin System 3
As illustrated in Fig 2 the rating of the program by the Fall 2010 participants who responded to our questionnaire mostly ranges from good to excellent. Most of the mentors rated the program as good and most of the mentees as very good. This difference is significant, according to a Mann-Whitney test: U=467, p=.005, r=.39. (Due to the small size of the mentoring program and the use of non-parametric tests, we consider all values of p<0.01 to be significant. The Mann-Whitney test and the Wilcoxon signed ranks test were performed on the VassarStats website of Dr. Richard Lowry, Vassar College (http://faculty.vassar.edu/lowry/vassarstats.html). The effect size r is calculated as z/ N, where Z is the z- score and N is the total number of observations, in this case the total number of participants and non participants in each comparison (Rosenthal, 1991). In summary, the overall experience of mentors and mentees appears to be positive and meaningful. Results from the Fall 2010 Student Data Table 2. Wilcoxon Signed-Ranks Tests Comparing the Academic Performance of Mentoring Program Participants With Non-Participants. Wilcoxon Signed-ranks tests n Median (mentees) Median (comparison group) Z p r Number of classes completed by the mentees vs. the comparison group by the end of Fall 2010. GPA of the mentees vs. the comparison group at the end of Fall 2010. For the mentees who took at least 1 class before joining, number of classes completed after joining vs. their comparison group. For the mentees who had GPAs before joining, GPA for semesters after joining, vs. their comparison group. 41 6 5 2.6.009.41 41 3.75 3.5 3.14.0017.49 33 2 1 2.88.004.5 24 4 3 2.78.005.57 The data collected so far (Table 2) suggests that the participating students successfully completed all their classes. In summary, the Fall 2010 mentees academic performance appears to be significantly better than the academic performance of the comparison group, based on student GPA and number of courses completed. This difference is still significant when limiting the sample to students who took classes or had a GPA before joining the program and comparing their performance after they joined the program. These results indicate a possible effect of student participation in the mentoring program and will be reevaluated in subsequent semesters. Replication of the Model The mentoring model that we have developed is unique in its simplicity and structure. The forms to be completed by the participants are minimal and all material is Web-based. There is more responsibility on the students to post minutes from their monthly meetings with the mentors. All participants are required to complete the assessment forms at the end of each semester. The interaction mode is flexible and up to Copyright 2011 The Board of Regents of the University of Wisconsin System 4
the pairs, they can use the mentoring platform, video chat, Skype, email or a telephone call. The Master of Arts in Teaching (MAT) program at UMUC decided to replicate the mentoring relationship to increase the retention of teacher candidates and to integrate knowledge and use of technology tools by teachers and MAT teacher candidates for instruction and programming. Both programs share the common use of Webbased technologies including the use of a platform that the mentor and the mentee can both access, the use of web conferencing technologies, and similar application and assessment surveys. We envision that the outcomes of this program will be similar to those of the biotechnology program, and will lead to better academic performance of the students and higher retention rate. References Bettinger, E. P., & Baker, R. (2011). The effects of student coaching: An evaluation of a randomized experiment in student mentoring. NBER Working Paper No. 16881. Levinson, D. J. (1978). The seasons of a man s life. New York, NY: Knopf. Rosenthal, R. (1991). Effect sizes: Pearson's correlation, its display via the BESD, and alternative indices. American Psychologist, 46(10), 1086-1087. About the Presenters Rana Khan is a professor in the Biotechnology Program at the University of Maryland University College. Her research interests include developing strategies to enhance the synergy between industry and academia and to increase graduate degree attainment among minorities. Recipient of several awards in recognition of her dedication and commitment to the cause of education and two grants from the Department of Education, she received a Ph.D. in Cell and Molecular Biology from the University of Maryland at College Park. Address: University of Maryland University College 3501 University Blvd East, Adelphi, MD 20783 Email: rkhan@ umuc.edu URL: http://www.umuc.edu/programs/grad/biotech_studies/index.shtml Phone: 240-684-2430 Fax: 240-684-2402 Arhonda Gogos is the External Evaluator of projects implemented at the Biotechnology Program, University of Maryland University College, and supported by the Department of Education, Fund for the Improvement of Postsecondary Education. As a Research Associate at AAAS, Project 2061, she worked on the clarification of science learning goals, identification of student misconceptions, and development and evaluation of science assessment items. She also has extensive research experience in the Biotechnology industry, with expertise in methods for structure-based drug design. She received a Ph.D. in Biophysics from the Johns Hopkins University. Email: arhonda.gogos@gmail.com Copyright 2011 The Board of Regents of the University of Wisconsin System 5