The Strategic Priorities Consultation Survey

Transcription

1 Strategic Priorities Consultation Survey #2 Results 2013 Institutional Planning & Analysis Prepared by: Stephanie Klassen, Kristen Hamilton Date: November 15, 2013 Prepared for: Strategic Priorities Advisory Committee

2 Executive Summary This report contains a detailed analysis of the results to the survey titled Thompson Rivers University Strategic Priorities Consultation Survey #2 and follows the Strategic Priorities Consultation Survey #1 Results report. This report includes answers from all respondents who completed the survey in the eight day period from Monday October 28, 2013 to Monday November 4, During this time, 1,110 completed responses were received. Respondent Profile Of those that responded to the survey, 46% reported being a student, as TRU alumni, as a faculty member, 9% as a staff member, 7% as an administrative member and 3% as a community member. Twenty four percent (24%) of respondents indicated that they have been affiliated with TRU for more than 10 years, while indicated that they have been affiliated with TRU for less than 1 year. Strategic Priority Rankings Respondents were asked to rank seven priorities, from most important (first) to least important (last). The results were as follows: First: Second: Third: Fourth: Flexible course and program delivery and Innovative and accessible academic programming Learning and teaching innovation and Student engagement Partnerships with educational institutions, communities, and the private sector and Professional development Life long learning The seven priority choices were not evenly ranked into seven categories. Rather, there were two priorities tied for first, two for second, two for third, and one ranked in fourth place. Respondent Comments Respondents were asked to comment regarding their priority rankings and on the survey itself. The 491 resulting comments were categorized into the following major themes: Concern about Priorities Qualifying or Describing Choice Emphasizing Choice Positive Comment Concern about Process Concern about Survey Page 1 of 16

3 Table of Contents Executive Summary Overall Strategic Priority Ranking... 3 Chart 1.1 Overall Strategic Priority Ranking Strategic Priority Ranking Across Respondent Groups... 4 Table 1.2 Priority Ranking Across Respondent Groups Strategic Priorities Qualitative Analysis... 5 Table 2.1 Qualitative Themes Respondent Profile... 7 Chart 3.1 Current Affiliations with TRU... 7 Chart 3.2 Length of Time Affiliated with TRU... 7 Chart 3.3 Length of Time Affiliated with TRU by Respondent Group... 8 Appendix A: Frequencies and Distributions of Rankings... 9 Table A.1 Frequency of Selected Ranking by Priority... 9 Table A.2 Percentage of Selected Ranking by Priority... 9 Appendix B: Distributions of Rankings by Respondent Group Table B.1 Percentage of Selected Ranking by Priority, by Respondent Group Appendix C: Qualitative Theme Descriptions Appendix D: Methodology Survey Methods Quantitative Analysis Methods Qualitative Analysis Methods Diagram D.1 Frequency of Selected Ranking by Priority References Page 2 of 16

4 1.0 Overall Strategic Priority Ranking The seven most frequently selected priorities from Thompson Rivers University Strategic Priorities Consultation Survey #1 were included in Survey #2. Respondents were asked to rank the seven strategic priorities in the order of importance, from 1 being most important to 7 being least important. Statistical analysis showed that the priorities were not evenly ranked into seven categories. As Chart 1.1 indicates, Flexible course and program delivery and Innovative and accessible academic programming were tied for first place; Learning and teaching innovation and Student engagement were tied for second place, Partnerships with educational institutions, communities, and the private sector and Professional development were tied for third place, and Life long learning came in fourth place. The detailed selection count and percent distribution of each priority are available in Appendix A. Chart 1.1 Overall Strategic Priority Ranking Priority Group Priority 1 st Group 1. Flexible course and program delivery 2. Innovative and accessible academic programming 2 nd Group 3. Learning and teaching innovation 4. Student engagement 3 rd Group 5. Partnerships with educational institutions, communities, and the private sector 6. Professional development 4 th Group 7. Life long learning Page 3 of 16

5 1.2 Strategic Priority Ranking Across Respondent Groups Table 1.2 illustrates the four priority groupings (as in Chart 1.0) with the respondent group detail. Table 1.2 shows general consistency in ranking across respondent groups, with some exceptions. For example, all respondents ranked Flexible course and program delivery in the highest priority group except Faculty, who ranked it in the mid range. As well, all groups ranked Professional development in the lowest priority group except Students, who ranked it in the mid range. All groups ranked Life long learning in the low range. The detailed percent distributions of each priority (by group) are available in Appendix B. Table 1.2 Priority Ranking Across Respondent Groups Priority Group 1st Priority High Mid Low Flexible course and program delivery Admin Alumni Staff Student Faculty Innovative and accessible academic programming Admin Alumni Staff Faculty Student 2nd Learning and teaching innovations Faculty Staff Admin Alumni Student Student engagement Admin Faculty Staff Alumni Student 3rd Partnerships with educational institutions, communities, and the private sector Admin Alumni Faculty Staff Student Professional development Student Admin Alumni Faculty Staff 4th Life-long learning Admin Alumni Faculty Staff Student Note. High = 1 st Priority Group, Mid = 2 nd & 3 rd Priority Group, Low = 4 th Priority Group Community and Other were excluded from the statistical analysis due to small group size. Page 4 of 16

6 2.0 Strategic Priorities Comments Analysis Respondents were asked two open ended questions: Do you have any comments to add regarding your ranking of the seven potential strategic priorities? and Do you have any comments about the survey itself? Themes were developed from both questions simultaneously, as respondents did not differentiate their responses based on the two questions. The distribution of comments across themes is shown in Table 2.1. Descriptions of the qualitative themes are available in Appendix C. Page 5 of 16

7 Table 2.1 Qualitative Themes Qualitative Themes Concern about Priorities Missing Priority 185 Terminology and/or Definitions 55 Difficult to Rank 27 Comments # % Total % Qualifying or Describing Choice Innovative and Accessible Academic Programming 13 Flexible Course & Program Delivery 12 Partnership 12 Student Engagement 10 Other 21 Total 68 Emphasizing Choice Flexible Course & Program Delivery 20 Learning and Teaching Innovation 12 Innovative and Accessible Academic Programming 11 Student Engagement 10 Other 15 Total 68 Positive Comment Support for Process 14 Other 19 Total 33 7% Concern about Process Questioning process 16 Restrictive/ Prescriptive 14 Total 30 6% Concern about Survey Restrictive/ Prescriptive 12 Survey design/ Methods 10 Other 3 Total 25 5% Total comments % Note. Other indicates additional categories with less than 10 comments each Page 6 of 16

8 3.0 Respondent Profile The second survey yielded 1,110 responses (1,700 people responded to the first survey). Chart 3.1 displays the breakdown of respondents current affiliation with TRU, the largest group of which was students (46%). Chart 3.1 Current Affiliations with TRU Respondents were asked how long they had been affiliated with TRU at the time of the survey. Chart 3.2 indicates the response distribution, the largest group of which was more than 10 years (24% of respondents). Chart 3.2 Length of Time Affiliated with TRU less than 1 year 2% does not apply 24% more than 10 years I have been affiliated with TRU for... does not apply more than 10 years 6-10 years 3-5 years 1-2 years less than 1 year 1-2 years years 23% 3-5 years Page 7 of 16

9 Chart 3.3 displays the combination of current affiliation and length of time of affiliation. The largest group of respondents was students who have been affiliated with TRU for less than one year ( of all respondents). Table 3.4 displays these the length of time as a percentage distribution within the respondent groups. Chart 3.3 Length of Time Affiliated with TRU by Respondent Group Table 3.4 Length of Time Affiliated with TRU Distributions within Respondent Groups Page 8 of 16

10 Appendix A: Frequencies and Distributions of Rankings Table A.1 Frequency of Selected Ranking by Priority Table A.2 Percentage of Selected Ranking by Priority Page 9 of 16

11 Appendix B: Distributions of Rankings by Respondent Group Table B.1 Percentage of Selected Ranking by Priority, by Respondent Group Flexible course and program delivery Innovative and accessible academic programming Learning and teaching innovation Life-long learning Partnerships... Professional development Student engagement Flexible course and program delivery Innovative and accessible academic programming Learning and teaching innovation Life-long learning Partnerships... Professional development Student engagement Flexible course and program delivery Innovative and accessible academic programming Learning and teaching innovation Life-long learning Partnerships... Professional development Student engagement Flexible course and program delivery Innovative and accessible academic programming Learning and teaching innovation Life-long learning Partnerships... Professional development Student engagement Flexible course and program delivery Innovative and accessible academic programming Learning and teaching innovation Life-long learning Partnerships... Professional development Student engagement Flexible course and program delivery Innovative and accessible academic programming Learning and teaching innovation Life-long learning Partnerships... Professional development Student engagement Flexible course and program delivery Innovative and accessible academic programming Learning and teaching innovation Life-long learning Partnerships... Professional development Student engagement Ranking Selected on Survey 1st 2nd 3rd 4th 5th 6th 7th 23% 21% 4% 23% 4% 7% 3% 9% 4% 2 31% 23% 4% 5% 9% 23% 29% 27% 7% 3% 29% 9% 7% 20% 7% 39% 6% 20% 7% 6% % 9% 6% 26% 35% 9% 1 4% 32% 1 20% 4% 9% 33% 33% 22% 26% 4% 4% 1 20% 22% 20% 7% 1 6% 32% 3% 6% 23% 22% 9% 20% 23% 3% 1 33% % 6% 3% 1 22% 21% 9% 21% 6% 26% 7% 33% 9% 9% 7% 7% 1 21% 6% 27% 23% 3% 26% 26% 26% 7% 3% 26% 27% 20% 3% 42% 42% 5% 9% 2 22% 21% 7% 27% 21% Page 10 of 16

12 Appendix C: Qualitative Theme Descriptions Concern about Priorities Respondents expressed concerns about the priorities, which were further categorized into the following sub themes: Missing Priorities: Priorities respondents said were missing from the survey, including: o Course and program availability: requests for additional courses and programs o Research o Academic quality & reputation: academic reputation, quality of education, quality of professors o Affordable education: tuition rates, parking, books, daycare o Employee issues: turnover, understaffing, morale, personal and professional development, top heavy administration o Career Education: career preparation during studies, connection to labour market at graduation o Priorities not Reflected: respondents priorities are not included as options Terminology and/or Definitions: lack of definition of priorities, vague or broad topics, overlapping topics Difficult to Rank: Priorities are too connected or overlapping to rank; they are all important; difficult to distinguish between them Qualifying or Describing Choice Respondents explained the conditions or assumptions upon which they chose the following priorities: Innovative and Accessible Academic Programming: technology, distance learning, studentcenteredness Flexible Course and Program Delivery: distance education; flexible enrollment; working, adult and parent students Partnership: concern about inclusion of the private sector Student Engagement: student centeredness; connected to or will follow from other priorities Emphasizing Choice Respondents emphasized the importance of the following priorities: Flexible Course and Program Delivery Learning and Teaching Innovation Innovative and Accessible Academic Programming Student Engagement Positive Comments Respondents made positive comments regarding TRU or the consultation process Support for Process: support for consultation and inclusion of students Page 11 of 16

13 Concern about Process Respondents had concerns about the consultation process: Questioning Process: concern about who is included in the process, usefulness, and impact of results Restrictive/ Prescriptive: concern about the pre chosen priorities for survey 1 Concern about Survey Respondents had concerns about the survey itself: Restrictive/ Prescriptive: character limit of the comment boxes on the survey, too few options for priorities Survey Design/ Methods: concern that methods will produce false positives, technical issues Page 12 of 16

14 Appendix D: Methodology Survey Methods The survey was available (in English) online to approximately 40,505 known potential participants (including TRU administrators, alumni, faculty, staff, and students) as well as to an unknown number of community members. Various marketing techniques were used to encourage responses from all stakeholder groups. The survey was open from October 28 to November 4, Quantitative Analysis Methods Data Cleaning The data was examined for duplicate submissions. These were determined by IP address, time of submission, and content. A total of four duplicate entries were removed. Other selections for I am currently a respondent groups were recoded, where possible, based on the Other comments. If a responded indicated more than one affiliation group, the first group indicated was coded as the primary affiliation group (for example, I am a staff member and a student was coded as Staff member ). Does not apply and Other selections for length of time I have been affiliated with TRU for were combined for analysis purposes. Overall Priority Rankings Individual scores were assigned for each priority, according to each respondent s ranking. A first order ranking was scored the highest, and a last order ranking (seventh) was scored the lowest. These scores were aggregated overall and by respondent group to determine the overall priority rankings. Therefore, the priority that was ranked higher most often was calculated as the top priority. Priority Groups In order to determine whether there was any statistical significance to the descriptive 1 to 7 ordered ranking of the priorities, Analysis of Variance was applied to the aggregate priority scores, which were calculated as described in Overall Priority Rankings (the null hypothesis was that all the aggregate priority scores were equal). Post hoc analysis (Tukey s B and Bonferroni) was used to identify the homogenous subgroups, which are presented as Priority Groups. (Ploutz Snyder, 2005) supports the use of the Bonferroni correction to reduce inflated Type I error. Due to a significant Levine statistic (test for homogeneity of variance), the analysis results were confirmed with a Games Howell post hoc analysis (which does not assume homogeneity of variance) (Games, Keselman, & Rogan, 1981). Priority Ranking across Respondent Groups In order to determine whether there was a difference in the way that each group ranked the priorities, the analysis described above ( Priority Groups ) was applied to the scores of each respondent group with the exception of Community and Other, which did not have enough responses to warrant an inferential test. The post hoc yielded varying numbers of homogenous priority subgroups; the Students group Page 13 of 16

15 resulted in five subgroups of priorities, while the staff group resulting in two subgroups of priorities. To allow for comparison of homogenous subgroups across respondent groups, a descriptive indicator was applied. For example, the highest priority subgroup was assigned to high for every respondent group, and the lowest subgroup (regardless of overall number of groups) was assigned to low. Middle priority subgroups (ranging from one to three in number) were all assigned to mid. The Staff respondent group, as a result of only having two homogenous subgroups, was not assigned a mid indicator for any priority. Qualitative Analysis Methods Directed qualitative content analysis was used by to examine themes within the responses provided to questions two and three of the second Strategic Priorities survey. A total of 491 analysis units (based on topic) were generated from 480 comments. Themes were developed from both questions simultaneously, as respondents did not differentiate their responses based on the two questions. The theming was inductive and subject to the analysts interpretation. The theming process included multiple close readings and interpretations of the raw data (4 iterations) as well as coding consistency checks (independent coding and comparison among three research analysts). (Thomas, 2006) describes this method of analysis using the following diagram: Diagram D.1 Frequency of Selected Ranking by Priority Once the overall themes were established, descriptions were generated with a final close reading of all the raw text that was included in within each theme. Page 14 of 16

16 References Games, P. A., Keselman, H. J., & Rogan, J. C. (1981). Simultaneous pairwise multiple comparison procedures for means when sample sizes are unequal. Psychological Bulletin, 90(3), Ploutz Snyder, R. J. (2005). Analysis of variance applications in institutional research. In M. A. Coughlin (Ed.), Applications of Intermediate/Advanced Statistics in Institutional Research (pp ). Association for Institutional Research. Thomas, D. R. (2006). A General Inductive Approach for Analyzing Qualitative Evaluation Data. American Journal of Evaluation, 27(2), Page 15 of 16