UW-Platteville /Calendaring Steering Group

Transcription

1 UW-Platteville /Calendaring Steering Group /Calendaring Evaluation Final Report February 2014

2 Contents Executive Summary... 1 Introduction... 3 Steering Group Composition and Charge... 3 Overview of /Calendaring Options Considered... 3 Campus Communication... 3 Website... 4 Open Sessions... 4 Vendor Presentations... 4 Senate Visits... 4 Evaluation Process... 5 Campus-wide survey... 5 Scoring Criteria... 5 Focus Groups... 6 Methodology... 6 Results... 6 Costs... 7 Evaluation Results... 7 Criteria Scoring /Calendaring Products Strengths and Weaknesses... 8 Recommendations... 8 Members of the /Calendaring Steering Group References School References Reference Materials Web Reference Materials Appendix... 13

3 1 Executive Summary On August 9, 2013 the /Calendaring steering group met for the first time to begin the process of producing a recommendation for campus and calendaring services. Rob Cramer, Vice Chancellor for Administrative Services, encouraged the committee to be thoughtful in the recommendation process and to engage the entire campus community. 1. The objectives for the steering group were: a. Review of the current market and the contracts already negotiated and available to the University for and calendaring. b. Determination of important criteria and requirements based on campus feedback. c. Development of scoring criteria based on University needs for and calendaring and a fit/gap analysis of available solutions. d. The gathering of campus feedback on final potential solutions through surveys, listening sessions and vendor demonstrations. e. A recommended solution to the TOPC Committee by February The /calendaring solutions considered were: a. Google Apps for Education b. Microsoft Office 365 c. Zimbra hosted on-premises (Status Quo) d. Zimbra hosted off-site (e.g., Merit.edu) 3. The evaluation process consisted of the following general steps. a. Campus survey to determine important requirements b. Campus visits by vendors including open-session presentations and working meetings with the steering and technical groups c. Development of evaluation criteria, scoring of the criteria, and rankings 4. Focus groups allowing the campus community to test drive the products and provide feedback 5. The recommendations of the steering group are: a. Zimbra hosted off-site is recommended as the best overall solution for campus. b. Microsoft Office 365 is recommended if cost is an overriding factor. 6. It is difficult to directly compare costs because of the different features offered. For example, the Zimbra hosting contract would provide a 99.9% financially-backed guarantee, while Microsoft does not guarantee uptime for their basic version. On the other hand, Microsoft Office 365 for education includes 5 free Office licenses for each of our students. The estimated costs for Zimbra hosted off-site and Microsoft are: a. Zimbra hosted off-site: $101,300 annually (3-year contract); transition costs covered by annual fee. b. Microsoft 365 for Education: Approx. $100,000 initial transition costs, no annual charges for the basic version.

4 2 7. The above recommendations were made based on the scoring criteria and an analysis of the strengths and weaknesses of the products. The following were the most critical strengths and weaknesses identified by the steering group. a. Zimbra strengths: i. The campus is familiar with it and satisfied ii. Merit.edu (likely hosting provider) offers unlimited storage iii. More control we own the data iv. Easier transition b. Zimbra weaknesses i. Higher cost in the long run ii. Mobile device support iii. Company viability iv. More difficult to allow alumni to keep their addresses c. Microsoft strengths i. Lower long-term cost ii. Mobile device support iii. Office 365 and Sharepoint iv. Most UW schools are using it v. Students get 5 free licenses to Microsoft Office vi. Easier to allow alumni to keep their addresses vii. Desktop integration with Microsoft Windows viii. Student exposure to the current leader in corporate d. Microsoft weaknesses i. Limited calendar functionality in the web version ii. Some data will not transfer/difficult transition iii. Loss of control/longer to get support iv. Advanced features only available in outlook client v. Upgrade schedules not as flexible

5 Introduction On August 9, 2013 the /Calendaring steering group met for the first time to begin the process of producing a recommendation for campus and calendaring services. Rob Cramer, Vice Chancellor for Administrative Services, encouraged the committee to be thoughtful in the recommendation process and to engage the entire campus community. Steering Group Composition and Charge The /Calendaring steering group convened for the purpose of reviewing the current /calendaring services, establishing requirements, definitions and establishment of scoring criteria using gap analysis of available solutions, stakeholder feedback and vendor demonstrations. The /Calendaring steering group was drawn from the following areas: Distance Learning Center, Alumni Services, Faculty, Administrative Services, Provost s office, Student Senate, Information Technology Services and Academic Affairs. The /Calendaring technical steering group was made up of three representatives from Information Technology Services and one representative from the Distance Learning Center. The objectives of the steering group were: Review of the current market and the contracts already negotiated and available to the University for and calendaring. Determination of important criteria and requirements based on campus feedback. Development of scoring criteria based on University needs for and calendaring and a fit/gap analysis of available solutions. The gathering of campus feedback on final potential solutions through surveys, listening sessions and vendor demonstrations. A recommended solution to the TOPC 1 Committee by February Overview of E mail/calendaring Options Considered Three different vendors and four different solutions were evaluated, including our current contract with Zimbra. Considered solutions were Google Apps for Education, Microsoft Office 365, Zimbra on-premises and Zimbra hosted off-site. The /Calendaring technical steering group provided key data to support the decision matrix. Comparative analyses were made of the strengths and weaknesses including cost, integration with Microsoft Windows, data ownership, ability to migrate to mobile devices and quality of support. Other considerations included training, storage and the impact on the other campus systems and software. 3 Campus Communication To ensure a transparent evaluation process, multiple methods were used to communicate the progress of the committee to faculty, staff, and students. 1 The TOPC Committee has since been dissolved. The recommendation will be made to the new IT prioritization committee.

6 Website A project website was set up to provide updates to the campus. An explanation of the process and rationale was provided, along with announcements about progress. Resources were included on the website and minutes from all the meetings were posted. In addition, a project address was provided for questions or comments about the process. Open Sessions During the Week of October 28, 2013, open sessions were held to update the campus on the progress of the steering group. Three separate presentations were scheduled at different times during the day (morning, afternoon, and evening) in order to best accommodate those wishing to attend. The presentations covered the following: 4 The reasons for the /calendaring evaluation The results from the campus survey (further details on the survey are covered later in the report) Upcoming timeline in the process including vendor visits Though the open sessions were sparsely attended (each session had less than 10 attendees), the steering group felt that the sessions were needed in order to engage the campus in a transparent decision-making process. Vendor Presentations Each of the vendors under consideration (Google, Microsoft, and Zimbra) was invited to make presentations to a general campus audience, to the steering group, and to the technical group. Zimbra representatives made their presentations via videoconference to the steering and technical groups on December 10, A videoconference presentation for a general audience was held on December 12, Two Microsoft representatives visited campus on December 11 to make in-person presentations to all groups. The Google representative met via videoconference with the steering group on December 13. A general session for campus was not held due to delays in finding a responsive Google representative. As was the case for the open sessions, the vendor sessions for the campus were sparsely attended. The steering group felt it was important to allow campus involvement, but attendance at the events was not as high as was hoped. The steering and technical groups felt that the meetings with the vendor representatives were productive and very worthwhile. Senate Visits Throughout the process, Faculty Senate, Academic Staff Senate, Classified Employee Senate, and Student Senate were briefed on the progress of the steering group. Initial presentations to the senates were made in early September, 2013, to inform them of the steering group and our

7 5 charge. In late October, further visits were made to inform them of the survey results and upcoming steps in the evaluation process. In late January, 2014, and early February, all four senates were visited to inform them of the final recommendation and rationale in making the recommendation. Evaluation Process Campus wide survey On September 24, 2013 an & Calendar Survey was released. All current and retired faculty and staff, and current students (registered in fall 2013) were contacted via and asked to respond to the web-based survey. Non-Respondents received two reminders to take the survey (sent on September 26, 2013 and October 3, 2013). Survey creation and administration was achieved using Qualtrics online software. Significant findings from the survey were: 97% of faculty/staff and 86% of student respondents regularly use their UW-Platteville issued address (i.e., "uwplatt.edu"). 48% of faculty/staff and 64% of student respondents felt it was important to keep their uwplatt.edu address after they leave campus. 90% of faculty/staff and 79% of student respondents use the Zimbra web-based client every time or often. 28% of faculty/staff and 35% of student respondents use the client contained in their mobile device (ipad, etc.) to access their uwplatt.edu address. While there were exceptions, faculty, staff and students expressed general satisfaction with Zimbra and calendaring functionality. Scoring Criteria Criteria were developed to score and rank the products from each of the vendors. The criteria were organized in eight different categories: Minimum Requirements Products not meeting these requirements would not be considered. Functional Requirements (e.g., The product allows users to share mailbox folders, to create an auto-reply message, etc.) Non-functional Requirements (e.g., Uptime, migration tools, etc.) Financial Cost of converting to new system and annual fees. Security/Privacy Business Continuity (e.g., continuous service through upgrades, outage recovery, etc.) Regulatory Compliance Focus Group Testing

8 6 Focus Groups Methodology To provide the campus community an opportunity for hands-on comparison of the three products, focus group testing was offered. The technical team partnered with Deb Meyer, Communications and Training Coordinator from Information Technology Services, to organize and conduct the event. Seven exercises were created to showcase and calendaring features found to be of high or moderate importance to respondents of the survey conducted earlier in the fall (See Survey Results in the Appendix). Test accounts were then acquired for each of the three products. Google Apps for Education accounts were provided by UW-La Crosse; Office 365 accounts were acquired through a Microsoft trial program; and, Zimbra accounts were configured in our onpremises Zimbra version 8 test environment. Each account was pre-configured to allow participants to complete the same set of tasks in each product. Participants were given packets to guide them through the focus group testing. Each packet consisted of four pages: one color coded exercise worksheet per product and a final ranking sheet. Packets were assembled such that the order that products were evaluated was randomized in an effort to eliminate bias for the participant's first or last experience. Each exercise worksheet provided three areas for the participant to record his/her experience with the product: check boxes to track completion of each task, comments for each task, and an overall usability ranking for the product. The overall usability from each exercise worksheet was scored on a scale of zero to 10 with zero meaning "not user-friendly at all" and ten being "extremely easy to use". On the final ranking sheet, the participant was asked to pick his/her first choice and last choice for each feature showcased in the exercises. This essentially ranked the products for each feature evaluated in the decision matrix (See Scored Criteria in the Appendix). Results In total, 70 people completed the focus group testing: 64 during the drop-in event on Thursday, December 5th from 8:00 AM-7:30 PM and 6 more that completed the evaluation on their own. Overall usability of each product was calculated by averaging the scores indicated on each participant's exercise worksheet for that product. If an overall usability score was not provided, the sheet simply did not count towards the total calculated average. Zimbra ranked the highest in overall usability at Microsoft Office 365 and Google Apps for Education averaged 5.95 and 5.88, respectively. Values for the Focus Group section of the Decision Matrix, however, were calculated from the results of the final ranking sheet. For each feature, the number of times each product was selected as first choice and last choice was counted and then used to establish a ranking. The intention was that the product selected as the first choice by the most participants would be awarded two points for the corresponding line in the Decision Matrix with second choice receiving one

9 point and third, zero points. It was found that a clear first choice existed for each of the features, but clear second and third choices did not. Rather than award one point to the second choice and zero to the third choice, it was decided to award 1.5 points and 1 point to second and third, respectively. Total scores for the final ranking showed Zimbra with 32 points, as the first choice for all features, then, Microsoft Office 365 at 22.5 and Google Apps for Education with Of the total potential score, however, Office 365 and Google Apps were separated by less than 6.5%. 7 Costs Google and Microsoft offer their basic services (Google Apps for Education and Microsoft Office 365) for no cost. Hosting Zimbra on campus or using a cloud-based hosting solution will result in costs of approximately $100,000 per year. Moving /calendaring to Google or Microsoft would result in approximately $100,000 in one-time transitional costs. Transitional costs would be included in the yearly cost for the cloud-based Zimbra hosting solution. It should be noted that direct cost comparisons are difficult. For example, Microsoft offers students five free licenses to their Office suite of productivity software. This benefit is hard to quantify because of uncertainty in student demand for Office. In addition, a cloud-based hosting solution of Zimbra would come with a 99.9% uptime guarantee, with financial penalties paid by the hosting company for noncompliance. Google and Microsoft offer no such uptime guarantees for their service. Microsoft offers an enhanced version of Microsoft Office 365 that does include an uptime guarantee (among other additional features), but the cost for this would be approximately $470,000 per year. 2 Evaluation Results Criteria Scoring The criteria were scored using two methodologies: 1. Meeting or exceeding the criteria for each criterion, the vendor was given a 0 if the product did not meet the criterion, a 1 if the vendor met the criterion, and a 2 if the vendor significantly exceeded the criterion. This methodology was used to score results for Minimum Requirements, Functional Requirements, Non-Functional Requirements, Security/Privacy and Business Continuity. 2. Ranking each product was given a ranked score based on comparison to the other products. This methodology was used to score results for the Financial and Focus Groups categories. 2 This is based on an estimated 13,000 student accounts and 1500 faculty/staff accounts using the cost data available at

10 A summary of the scoring results are shown in Table 1. The category scores have already been multiplied by respective weightings in the table. The total score is the sum of the category scores. The normalized score sets the top-ranked option (Zimbra hosted off-site) at 100% and then assigns percentages to the other options based on the ratio of the total scores. The complete scores for all criteria are included in the Appendix B. Note that the Regulatory Compliance category was not included in the scoring. Any product not meeting regulatory requirements would not have been considered, and there was no other way to differentiate between the products so this category would not have provided meaningful information for the final score. Table 1 - Evaluation Criteria Results Summary Category (Weight) Google Microsoft Zimbra (on-premesis) Zimbra (hosted) Minimum Requirements (20%) Functional Requirements (20%) Non-Functional Requirements (10%) Financial (10%) Security/Privacy (15%) Business Continuity (10%) Focus Group Testing (15%) Total Score Normalized Score 97% 90% 95% 100% Rank E mail/calendaring Products Strengths and Weaknesses The steering group felt that the scores for each option from the evaluation criteria were so close (all were within the 90% - 100% normalized range) that any of the products would be a satisfactory solution for the campus. To aide in the recommendation process, the steering group created a list of strengths and weaknesses for each product. Strengths and weaknesses for each option are summarized in Table 2. 8 Recommendations After narrowing the search to Google Apps for Education, Microsoft Office 365, Zimbra (onpremises) and Zimbra (hosted). Zimbra (hosted) was selected because it offered the best match for UW-Platteville s functional, technical, and legal requirements. The committee s second choice was Microsoft Office 365, which is recommended if cost is an overriding factor.

11 9 Table 2 - Strengths and Weaknesses for each option Google Apps for Education Strengths Weaknesses Full suite of Google Apps/Google Drive with The interface and perceptions about privacy are enhanced license polarizing. Familiar to many students and staff Hosted off-campus; loss of control Many options for calendaring sharing (Do not own the data) Provides app that can be downloaded for mobile Data stored outside the U.S. devices Longer to get support 30 GB storage for and files Cannot control when upgrades occur Graduates can continue to use their UW- No backups Platteville address Can t share folders all or nothing Many school districts in Wisconsin are using it Limited service in select countries (e.g., China) Accounts are free Some data (e.g., files attached to appointments, messages over 25 MB,.exe attachments) will not transfer Changing feature set/functionality with little warning (e.g., compose ) Microsoft Office 365 Most UW schools are using it which will allow Upgrade schedules UW-Platteville to collaborate with other campuses (Cannot control when upgrades are done) Can t brand the Students get 5 free licenses for office (Does not have the university logo on interface Office 365 and Sharepoint like Zimbra has now) (Intranet sites for your teams and classes) Loss of control/longer to get support Mobile Interface (Face uncertainties over migration, compliance, (Have app that can be downloaded) infrastructure and security give up 50 GB of storage plus 25 GB of Sky- backup and recovery control to Microsoft) Drive file storage Calendar functionality in the web version Graduates can continue to use their UW- (need Outlook to get the full benefits) Platteville address Advance features (e.g., sharing) only available Desktop integration with Windows systems when using the Outlook client Accounts are free for basic service Some data (e.g., contacts, calendars,.exe attachments) EMS Collaborative programs are using Sharepoint will not transfer with UW-Colleges Microsoft is the dominant corporate provider (for now) Zimbra Campus is familiar and satisfied Does not have currently have a mobile devices Unlimited Storage (hosted) app More control data is on campus (onpremises) Must pay for accounts unlike Microsoft and Google Response time (on-premises) Company viability Interface can be branded and customization (Bought out in 2010 by Vmware and in 2013 by (on-premises and hosted) Telligent) More problematic to provide graduates continued use of their address MS Office Integration

12 10 Members of the E mail/calendaring Steering Group Kim Maier Distance Learning Center Julie Matuszak Alumni Services Matt Roberts Professor, Dept. of Civil & Environmental Engineering Joe Sigwarth President, Student Senate Liz Schaal Office of the Provost Tonya Stappert Information Technology Services Sally Swindall Administrative Services Joanne Wilson Academic Affairs References The following sources of information were consulted during the evaluation process and in putting together this report. School References UW-Oshkosh (Teleconference re: Google), UW-LaCrosse (Shared documents, telephone re: Google), UW-Madison (Shared documents, F2F, telephone re: Microsoft), UW-Milwaukee (Shared documents, F2F re: Zimbra, Microsoft, Google) Reference Materials Outsourcing Information Technology Services, Education Advisory Board, December 2012 Outsourcing Student , Education Advisory Board, April Report, Core Data Service, Educause, 2011 University of Maryland Change Report, Steering Committee, May 2011 North Carolina State University Evaluation of Student Services: Options for the Future, Office of Information Technology, March 2009 UW Madison and Calendaring Business Case, University Review Steering Committee, June 2012 UW Milwaukee Evaulation Report, UW Milwaukee Information Technology Services UW Milwaukee Calendar Requirements, UW Milwaukee Information Technology Services, 2012

13 UW LaCrosse Archiving , UW LaCrosse Information Technology Services Staff Member , January 2014 UW LaCrosse Purchasing, Deanne Otto, UW LaCrosse Purchasing Department, September 2013 UW System CIO Meeting Notes, Sue Traxler, CIO, UW Platteville, Fall 2013 Web Reference Materials - Google and Microsoft: The Battle Over College Storage space is a key concern for students. Google and Microsoft market share figures. Cost savings at Notre Dame: $1.5 M national Campus Computing Project (CCP) survey: 42% of schools migrated or were about to migrate to an outsourced student service. B7A5001B21BA1DA0# dayrssfeed

14 _

15 Appendix 13

16 & CALENDAR SURVEY ANALYSIS AND REPORT Survey Background: On September 24, 2013 an & Calendar Survey was fielded to all faculty and staff using an -web administration All respondents were contacted via (using uwplatt.edu addresses) and responded to the survey on a web-based instrument by clicking a link embedded in the or pasting a URL (also contained in the ). Non-Respondents received two reminders to take the survey (sent on September 26, 2013 and October 3, 2013) All current & retired faculty and staff, and current students (registered in fall 2013) were contacted. Survey creation and administration was achieved using Qualtrics online software (at Response rates by faculty/staff and student designation are as follows: Table 1: Response Rates by Faculty/Staff vs. Students Segmentation Segment Count Response Rate Faculty/Staff % Students 1,570 18% Total 2,225 23% Note: Students contain both undergraduate and graduate students Analysis: The analysis profiles Faculty/Staff and Students separately The survey instrument contained skip patterns which customized questions based on a respondent s answers to key questions. All respondents who indicated using rated the importance of various and calendar system features. Only those respondents who indicated using the Zimbra web-based client answered questions on satisfaction with and calendar features as implemented by Zimbra UW-Platteville Office of Institutional Research Page 1

17 & CALENDAR SURVEY ANALYSIS AND REPORT The heart of the analysis results in tables (separate ones for faculty/staff and students) showing importance and satisfaction with and calendar system features, side-by-side, and categorized as high, moderate, and low. High Importance (or Satisfaction) is defined as the percentage statistically exceeding 60% The percentage is the combination of the top two Likert Scale options of a given scale. For example, using the Importance Scale, the percentage is the percent of respondents who selected either Highly Important or Important out of the following possible options: Highly Important, Important, Somewhat Important, Only Slightly Important, and Not at all Important. For the Satisfaction Scale the percentage is the percent of respondents who selected either Extremely Satisfied or Very Satisfied our of the following possible options: Extremely Satisfied, Very Satisfied, Moderately Satisfied, Slightly Satisfied, and Not at all Satisfied. Moderate Importance (or Satisfaction) is defined as the percentage statistically equivalent to 60% Low Importance (or Satisfaction) is defined as the percentage statistically below 60% In determining the High, Moderate, and Low designations, a 99% confidence interval (CI) was computed around the estimate. If the lower CI exceeded 60%, then the finding was categorized as High. If the CI contained 60%, then the finding was categorized as Moderate. Finally, if the upper CI was below 60%, the finding was categorized as Low. For example, if the percentage on importance for a given feature was 68% (our point estimate), and the 99% CI for the 68% point estimate ranged from 63% (the lower CI) to 73% (the upper CI), then the result of 68% would be categorized as High. UW-Platteville Office of Institutional Research Page 2

18 & CALENDAR SURVEY ANALYSIS AND REPORT Basic Demographics by Faculty/Staff vs. Students Segmentation Table 2: Respondents by Faculty/Staff vs. Students Segmentation Segment Count Percent Faculty/Staff % Students 1,570 71% Table 3: Respondent Gender by Faculty/Staff vs. Students Segmentation Facstaff Student Respondent Respondent % in Sampling Respondent % in Sampling Gender % % Count Frame Count Frame Female % 52% % 36% Male % 48% % 64% Unknown 1 0% 0% - 0% 0% Total % 100% 1, % 100% Table 4: Average Respondent Age by Faculty/Staff vs. Students Segmentation Segment Average Age Faculty/Staff 47 Students 23 UW-Platteville Office of Institutional Research Page 3

19 & CALENDAR SURVEY ANALYSIS AND REPORT General and Calendar Usage by Faculty/Staff vs. Students Segmentation Table 5: Overall Usage by Faculty/Staff vs. Students Segmentation Please indicate which statement below is Most True regarding your usage: Faculty/Staff Base Size % of % Students Base Size % of % I regularly use my UW-Platteville issued address (i.e., "uwplatt.edu") % 1,286 86% Although I regularly use , I never use my UW-Platteville address at all (including forwarding it to another account); instead, I exclusively use another 15 2% % account (e.g., AOL, Yahoo, Gmail) I very rarely use at all; I prefer to communicate using Facebook, text messages, etc. 5 1% 33 2% Table 6: Importance of Keeping the uwplatt.edu address by Faculty/Staff vs. Students Segmentation How important is it for you to keep your uwplatt.edu address after you leave campus? Faculty/Staff Students Count % Count % Extremely Important, Very Important, or Important % % Neither Important nor Unimportant, Somewhat Important, and Not at all Important % % Total % 1, % Table 7: Alternative Ways of Accessing the uwplatt.edu address by Faculty/Staff vs. Students Segmentation Please rate how frequently you utilize the following alternative ways of accessing your uwplatt.edu address Count Selecting "Every Time" or "Often" % "Every Time" or "Often" Total Respondents Answering Question Count Selecting "Every Time" or "Often" % "Every Time" or "Often" Total Respondents Answering Question I use another client (e.g., Thunderbird, Eudora, etc.) for my uwplatt.edu address 31 5% % 1,223 I use the client contained in my mobile device (ipad, etc.) to access my uwplatt.edu address % % 1,235 My uwplatt.edu is forwarded to another account (i.e., System) 21 4% % 1,217 Note: Likert Frequency Scale for these questions is: Every Time, Often, Occasionally, Rarely, Never Table 8: Frequency of Using Zimbra to Access the uwplatt.edu address by Faculty/Staff vs. Students Segmentation How often do you use the Zimbra web-based client to Facstaff Student access your uwplatt.edu address? N % N % Every Time or Often % % Occasionally or Rarely 43 7% % Never 17 3% 52 4% Total % 1, % Facstaff Student UW-Platteville Office of Institutional Research Page 4

20 & CALENDAR SURVEY ANALYSIS AND REPORT Importance and Satisfaction-with-Zimbra Ratings on , Calendar, and Add-On Features Table 9: Student Importance and Satisfaction (with Zimbra) Ratings for Features Features Base Size for % % Category of % Render Opinion Base Size for % % Category of Ease and features of replying 1,320 90% High 86% 1,037 66% High Attaching documents to 1,285 92% High 83% % High Ease and features of composing messages 1,319 90% High 86% 1,044 63% Mod Inbox Management functionality (e.g., flagging, setting up folders) 1,288 75% High 78% % Mod The amount of storage space for 1,254 74% High 76% % Mod Versatile search function (e.g., searching by keyword, date, sender) Accessibility from other devices (cell phone, ipad, etc.) Undo functionality (e.g., undo delete, undo send , etc.) 1,282 68% High 72% % Low 1,263 78% High 69% % Low 1,258 71% High 66% % Low Versatile and easy to use Address Book 1,331 59% Mod 73% % Low Ability to easily generate mail lists (e.g., to make, import, and export contact lists) filtering (e.g., automatically organizing incoming according to userset criteria) 1,318 59% Mod 69% % Low 1,286 62% Mod 67% % Low Ability to add signatures 1,255 46% Low 57% % Mod Look (colors, shapes, layout, and typefaces) and feel (buttons, boxes, and menus) of the client display (e.g., Inbox) Importance Satisfaction with Zimbra 1,322 50% Low 82% % Low Exporting of (e.g. copying to another account, archiving, etc.) 1,315 49% Low 62% % Low Creating templates 1,266 37% Low 53% % Low sharing (e.g., allowing a designee to manage your inbox) 1,247 32% Low 45% % Low Ability to auto-respond/auto-reply 1,249 43% Low 50% % Low encryption 1,248 42% Low 46% % Low Table Notes: 1. Only respondents who indicated using the Zimbra web-based client rated satisfaction with Zimbra 2. The Base Size for the shows how many respondents answered the full, five-point scale for either importance or satisfaction. 3. % shows the percentage of respondents selecting the top two boxes (choices) of a given Likert Scale (importance or satisfaction) out of the Base Size for the 4. Category of is the result of the test around the benchmark percentage of 60% (see page 2 of this report for further explanation) 5. Percent Render Opinion is the percentage of respondents who chose to actually rate their satisfaction with the feature as implemented by Zimbra. It is the percent choosing either: Extremely Satisfied, Very Satisfied, Moderately Satisfied, Slightly Satisfied, or Not at all Satisfied. Respondents could also leave the question unanswered or select No Opinion. 6. Full Likert Scale for Importance is: Highly Important, Important, Somewhat Important, Only Slightly Important, and Not at all Important. 7. Full Likert Scale for Satisfaction is: Extremely Satisfied, Very Satisfied, Moderately Satisfied, Slightly Satisfied, or Not at all Satisfied UW-Platteville Office of Institutional Research Page 5

21 & CALENDAR SURVEY ANALYSIS AND REPORT Table 10: Faculty/Staff Importance and Satisfaction (with Zimbra) Ratings for Features Importance Satisfaction with Zimbra Features Ease and features of composing messages Base Size for % % Category of % Render Opinion Base Size for % % Category of % High 93% % High Ease and features of replying % High 93% % High Attaching documents to % High 93% % High Inbox Management functionality (e.g., flagging, setting up folders) % High 91% % Mod The amount of storage space for % High 86% % Mod Versatile and easy to use Address Book % High 88% % Low Versatile search function (e.g., searching by keyword, date, sender) % High 90% % Low Ability to auto-respond/auto-reply % Mod 69% % Mod Ability to add signatures % Mod 74% % Mod Look (colors, shapes, layout, and typefaces) and feel (buttons, boxes, and menus) of the client display (e.g., Inbox) % Mod 90% % Low Ability to easily generate mail lists (e.g., to make, import, and export contact lists) filtering (e.g., automatically organizing incoming according to userset criteria) Accessibility from other devices (cell phone, ipad, etc.) Undo functionality (e.g., undo delete, undo send , etc.) % Mod 76% % Low % Mod 66% % Low % Mod 66% % Low % Mod 66% % Low encryption % Low 40% % Mod Exporting of (e.g. copying to another account, archiving, etc.) % Low 60% % Low Creating templates % Low 46% % Low sharing (e.g., allowing a designee to % Low 33% % Low manage your inbox) Table Notes: 1. Only respondents who indicated using the Zimbra web-based client rated satisfaction with Zimbra 2. The Base Size for the shows how many respondents answered the full, five-point scale for either importance or satisfaction. 3. % shows the percentage of respondents selecting the top two boxes (choices) of a given Likert Scale (importance or satisfaction) out of the Base Size for the 4. Category of is the result of the test around the benchmark percentage of 60% (see page 2 of this report for further explanation) 5. Percent Render Opinion is the percentage of respondents who chose to actually rate their satisfaction with the feature as implemented by Zimbra. It is the percent choosing either: Extremely Satisfied, Very Satisfied, Moderately Satisfied, Slightly Satisfied, or Not at all Satisfied. Respondents could also leave the question unanswered or select No Opinion. 6. Full Likert Scale for Importance is: Highly Important, Important, Somewhat Important, Only Slightly Important, and Not at all Important. 7. Full Likert Scale for Satisfaction is: Extremely Satisfied, Very Satisfied, Moderately Satisfied, Slightly Satisfied, or Not at all Satisfied UW-Platteville Office of Institutional Research Page 6

22 & CALENDAR SURVEY ANALYSIS AND REPORT Table 11: Student Importance and Satisfaction (with Zimbra) Ratings for Calendar Features Importance Satisfaction with Zimbra Calendar Features Loading your class schedule into your calendar Accessibility from other devices (cell phone, ipad, etc.) Base Size for % % Category of % Render Opinion Base Size for % % Category of 1,161 65% High 53% % Low 1,119 66% High 49% % Low Ability to setup reoccurring events 1,161 61% Mod 50% % Mod Scheduling meetings with other people 1,167 61% Mod 54% % Low Ability to search appointments 1,118 61% Mod 45% % Low Event reminders 1,122 62% Mod 50% % Low Create calendar event from an message Viewing multiple calendars including external calendars (e.g., Google calendar) Sharing your campus calendar with external calendar systems Integrated resource-calendar integration (reserving room or equipment for meeting as part of an event) Attaching documents to a scheduled event 1,121 56% Mod 45% % Low 1,165 54% Low 48% % Low 1,168 52% Low 48% % Low 1,161 48% Low 44% % Low 1,116 52% Low 42% % Low Import/Export of calendar data 1,116 54% Low 45% % Low Calendar delegation (e.g., allowing a 1,115 35% Low 36% % Low delegate to manage your calendar) Table Notes: 1. Only respondents who indicated using the Zimbra web-based client rated satisfaction with Zimbra 2. The Base Size for the shows how many respondents answered the full, five-point scale for either importance or satisfaction. 3. % shows the percentage of respondents selecting the top two boxes (choices) of a given Likert Scale (importance or satisfaction) out of the Base Size for the 4. Category of is the result of the test around the benchmark percentage of 60% (see page 2 of this report for further explanation) 5. Percent Render Opinion is the percentage of respondents who chose to actually rate their satisfaction with the feature as implemented by Zimbra. It is the percent choosing either: Extremely Satisfied, Very Satisfied, Moderately Satisfied, Slightly Satisfied, or Not at all Satisfied. Respondents could also leave the question unanswered or select No Opinion. 6. Full Likert Scale for Importance is: Highly Important, Important, Somewhat Important, Only Slightly Important, and Not at all Important. 7. Full Likert Scale for Satisfaction is: Extremely Satisfied, Very Satisfied, Moderately Satisfied, Slightly Satisfied, or Not at all Satisfied UW-Platteville Office of Institutional Research Page 7

23 & CALENDAR SURVEY ANALYSIS AND REPORT Table 12: Faculty/Staff Importance and Satisfaction (with Zimbra) Ratings for Calendar Features Calendar Features Base Size for % % Category of % Render Opinion Base Size for % % Category of Scheduling meetings with other people" % High 84% % Mod Ability to setup reoccurring events" % High 81% % Mod Event reminders" % High 78% % Mod Attaching documents to a scheduled event" % Mod 49% % Mod Viewing multiple calendars including external calendars (e.g., Google % Mod 66% % Low calendar)" Ability to search appointments" % Mod 58% % Low Accessibility from other devices (cell phone, ipad, etc.)" % Mod 58% % Low Loading your class schedule into your calendar" % Low 38% % Mod Calendar delegation (e.g., allowing a delegate to manage your calendar)" % Low 41% % Mod Sharing your campus calendar with external calendar systems" % Low 54% % Low Integrated resource-calendar integration (reserving room or equipment for meeting as part of an event)" Importance Satisfaction with Zimbra % Low 55% % Low Import/Export of calendar data" % Low 43% % Low Create calendar event from an % Low 49% % Low message" Table Notes: 1. Only respondents who indicated using the Zimbra web-based client rated satisfaction with Zimbra 2. The Base Size for the shows how many respondents answered the full, five-point scale for either importance or satisfaction. 3. % shows the percentage of respondents selecting the top two boxes (choices) of a given Likert Scale (importance or satisfaction) out of the Base Size for the 4. Category of is the result of the test around the benchmark percentage of 60% (see page 2 of this report for further explanation) 5. Percent Render Opinion is the percentage of respondents who chose to actually rate their satisfaction with the feature as implemented by Zimbra. It is the percent choosing either: Extremely Satisfied, Very Satisfied, Moderately Satisfied, Slightly Satisfied, or Not at all Satisfied. Respondents could also leave the question unanswered or select No Opinion. 6. Full Likert Scale for Importance is: Highly Important, Important, Somewhat Important, Only Slightly Important, and Not at all Important. 7. Full Likert Scale for Satisfaction is: Extremely Satisfied, Very Satisfied, Moderately Satisfied, Slightly Satisfied, or Not at all Satisfied UW-Platteville Office of Institutional Research Page 8

24 & CALENDAR SURVEY ANALYSIS AND REPORT Table 13: Student Importance Ratings for Potential /Calendar System Add-On Features Importance Calendar Features Category Base Size for % % of Create, share, store, and publish online documents and spreadsheets 1,155 60% Mod (e.g., Office 365) Group Chat 1,160 49% Low Chat 1,160 45% Low Personal Web Space 1,159 40% Low SMS Messaging (texting) 1,146 40% Low Video Chat 1,160 35% Low PC-to-PC voice calls 1,160 28% Low Voice mail to 1,148 28% Low Table 14: Faculty/Staff Importance Ratings for Potential /Calendar System Add-On Features Importance Calendar Features Base Size for % % Category of Create, share, store, and publish online documents and spreadsheets % Low (e.g., Office 365) Voice mail to % Low SMS Messaging (texting) % Low Group Chat % Low Chat % Low Personal Web Space % Low Video Chat % Low PC-to-PC voice calls % Low Table Notes: 1. The Base Size for the shows how many respondents answered the full, five-point scale for importance. 2. % shows the percentage of respondents selecting the top two boxes (choices) of the importance Likert Scale out of the Base Size for the 3. Category of is the result of the test around the benchmark percentage of 60% (see page 2 of this report for further explanation) 4. Full Likert Scale for Importance is: Highly Important, Important, Somewhat Important, Only Slightly Important, and Not at all Important. UW-Platteville Office of Institutional Research Page 9

25 /Calendaring Evaluation Criteria Analysis Pge 1 of 8 0 = Does not meet criterion; 1 = Meets criterion; 2 = significantly exceeds criterion Google Microsoft Zimbra Zimbra Minimum Requirements Apps for Edu. Office 365 (on-premesis) (hosted) Common Web Interface: Provide access to , calendar, and address books through a web browser interface Individual Mailbox: Provide a minimum quota of 4.0 GB. Service must allow management of message size. Service must not restrict the number of messages other than the capacity of the quota itself. Service must allow for quota increase up to 16.0 GB Individual Calendars and Address Books: Provide at least one calendar and address book per user. Prefer support of multiple calendars per user. Support sharing and publication of calendar and calendar events with user configurable access controls Shared Mailboxes, Calendars, and Address Books: Provide the creation and management of shared mailboxes, calendars, and address books with multiple owners and access control lists Documentation: Provide complete documentation, service manuals, and appropriate system engineering bulletins for all components. Documentation must be received a minimum of one month prior to installation, in an electronic format and appropriate for loading onto University-owned servers Training: Include training for three Service Administrators to take place after execution of agreement. Online documentation for end users with set-up and troubleshooting information must also be available Train the Trainer Training: Provide training for end-user trainers Functional Requirements Minimum Requirements Weighted Score Weight 20% User Role: User Shared Mailboxes, Calendars, and Address Books: Allow all users who concurrently share a mailbox to view actions taken on messages Global Address Book: Allow users to access a single global address book, bound to a data source acceptable to the University (current students, faculty, and staff) Permissions: Allow users to delegate appropriate rights to internal and external individuals and/or groups to mailboxes, calendars and address books Searching: Allow users to search messages, calendars, and address books, either all at once or by selecting specific criteria such as folders or labels Categorization / Tagging: Allow users to apply multiple customizable categories or tags to messages, address book entries, and calendar events Message Display and Organization: Allow users to organize messages by size, folders, labels, and colors. Allow users to sort messages by various criteria (sender, subject, date, etc.). Allow users to drag and drop s into folders Threaded Message View: Allow users to view all messages in a thread/conversation, including the initial received or sent, any received replies, and any userforwarded copies. Threaded view should be accurate to the thread, and should not include similar but unrelated s Message Formats: Allow users to select a default format (text/plain or text/html), and override the default on a per message basis Message Templates: Allow users to create message templates. Allow users to create templates from an existing message Large Message Handling: Allow users to display the full contents of a message regardless of the number of lines. When forwarding or replying to messages, must be able to retain the full contents of the message regardless of initial size Forwarding: Allow users to forward from the service to multiple other e- mail services Auto-reply: Allow users to create a custom auto-reply message, and specify start and stop dates and times Message Filtering: Allow users to create and manage message filters based on custom criteria and actions

26 /Calendaring Evaluation Criteria Analysis Pge 2 of 8 Send/Read Receipts: Allow users to manage send, read, and delivery receipt functionality Undo Deletions: Allow users to undo message deletions Undo Sends: Allow users to undo message sends Remote Calendars: Allow users to subscribe to remote calendars, such as an institutional or third party calendar of events Event Scheduling: Allow users to invite internal and external users, groups and resources to one-time and recurring events. Users should be able to view people and resource availability via a free/busy search of people/resources. System should suggest an available time for multiple people/resources. System should notify user if they do not have appropriate permissions to schedule the invitee Calendar Attachments and Links: Allow senders and receivers to attach files and links (URLs) to calendar invitations and events. Possibility to add personalize meeting notes a plus Event Reminders: Allow users to customize event reminder notifications. System should have the ability for an , audio, or a visual notification of events Event Display Customization: Allow users to customize event display by labels and colors Multiple Calendar Views: Allow users to view calendar by daily, weekly, work week, monthly, and custom views, and allow scrolling through calendars by day, week, and month. Allow the ability to view multiple calendars in one window Import/Export Calendar Data : Allow users to import and export calendar data in a standard calendar format Import/Export Mailboxes and Messages : Allow users to import and export mailboxes, messages, conversations, and metadata in a standard format for archiving User Role: Service Administrator Account Management: Allow service administrators the ability to create and delete accounts and mailboxes with an Application Programming Interface. Allow service administrators the ability to disable compromised accounts immediately Message Blocking: Allow service administrators the ability to create custom message blocking based on sender, subject message, and other basic criteria Interface Customization: Allow service administrators the ability to customize user web interface to permit, at a minimum, the use of University specific logos and color schemes System Management: Allow service administrators secure, encrypted access via command line and industry standard web technology Privacy Management: Allow service administrators to exclude user information from publication to address books and set up users and groups that are protected from each other Namespace Management: Allow service administrators the ability to manage name, accounts, and forwarding across systems Reporting: Allow service administrators the ability to generate reports on mailbox count and storage utilization over time, inbound and outbound traffic (both message count and data volume), and information on internal and outgoing message delivery. Provide the ability to generate reports on authentication, authorization, or application session creation events appropriate for service access method (i.e. client/server, web or mobile device) Auditing: Provide the ability to view event logs on authentication, authorization, or application session creation events appropriate for service access method (i.e. client/server, web or mobile device) for the past 30 days. The University reserves the right to review and audit all usage and access logs upon request. Quick, easy access to logs and information associated with the service (e.g. access, usage) by administrators should be available without need for subpoena, search warrant, or other legal measure