PEER BENCHMARKING. A Powerful Tool for IT Portfolio Planning. Noah Wittman, Educational Technology Services, UC Berkeley UCCSC - 04 August 2014

Transcription

1 PEER BENCHMARKING A Powerful Tool for IT Portfolio Planning Noah Wittman, Educational Technology Services, UC Berkeley UCCSC - 04 August 2014

2 Presentation Overview In Fall 2013, UC Berkeley adopted a benchmarking framework from NYU to evaluate a portfolio of 23 IT services to support research and teaching. We found this approach to be an incredibly effective tool for portfolio management and strategic planning. This presentation will (a) summarize the project and (b) provide tips and templates for you to conduct your own peer benchmarking project.

3 Research Services Research Data Management Data Analysis (Quantitative & Qualitative) Survey Research Support Data Visualization & GIS Linked Open Data & Semantic Web Preservation & Archival Services Shared Research Computing (HPC) Museum, Archives & Special Collections Research Applications Teaching & Learning Services Online Courses Learning Management Systems & Support Instructional Content Creation Learning Spaces eportfolio Support Technology Enhanced Teaching & Learning Support Course & Program Evaluation Related Services Collaboration & Communication Web Publishing Software Licensing & Distribution Video & Web Conferencing Scholarly Networking Portals, Dashboards & Aggregrators

4

5 UC Berkeley Project Objective Ensure UC Berkeley maintains the highest quality services to support research and teaching by: Benchmarking Berkeley technology services with peer institutions Developing a set of recommendations around future resource realignment and investments Fostering collaboration and a shared understanding across domains and service areas

6 Process Define Research Share Identify Peer Institutions Define Service Areas Gather Benchmarking Data from University Websites Record Data in Worksheets Narrative Summary of Research Findings Group Presentation & Discussion Assign Researchers Follow-up Phone & Interviews One Page Executive Summary

7 Process Step Date Identify Peer Institutions July 2013 Assign Area Experts July 2013 Web Data Gathering (Comparison Worksheets) July - Aug 2013 Preliminary Analysis (2-Page Summaries) + 10 min Presentation + 10 min Discussion/Feedback Aug - Oct 2013 Phone Interviews + Follow-up Research Aug - Oct 2013 Final Analysis + Draft Recommendations + 10 min Presentation + 10 min Discussion/Feedback Oct - Nov 2014 Meet with Key Stakeholders to Refine Recommendations Dec Feb 2015

8 Project Team Role People Tasks Effort Executive Sponsors VC Undergraduate Education, VC Research, CIO Feedback on Scope & Framing (Peers, Services, etc.); Feedback and Guidance on Service Portfolio & Strategy 20 hours Functional Sponsors ACIO Academic Engagement, Director Research Information Technologies, Director of IST-API, Associate University Librarian Define Project Scope (Services, Peers), Feedback to Project Team, Engage Executive Sponsors, Develop Service Portfolio & Strategy 200 hours Project Manager FTE Project Manager (4 months) Coordinate Meetings & Assignments, Draft Templates, Track Progress 400 hours Researchers 24 Researchers from across campus (Educational Technology Services, Research Information Technologies, Library, Berkeley Resource Center for Online Education; Applications, Platforms & Integrations) Complete Research, Share Findings 30 Hours x 2 Researchers x 23 Service Areas = 1380 hours Total ~ 2000 hours

9 Research Services Research Data Management Data Analysis (Quantitative & Qualitative) Survey Research Support Data Visualization & GIS Linked Open Data & Semantic Web Preservation & Archival Services Shared Research Computing (HPC) Museum, Archives & Special Collections Research Applications Teaching & Learning Services Online Courses Learning Management Systems & Support Instructional Content Creation Learning Spaces eportfolio Support Technology Enhanced Teaching & Learning Support Course & Program Evaluation Related Services Collaboration & Communication Web Publishing Software Licensing & Distribution Video & Web Conferencing Scholarly Networking Portals, Dashboards & Aggregrators

10 Peer Institutions Berkeley MIT Columbia NYU Stanford UCLA Harvard Michigan UCSD Virginia

11

12 DRAFT

13

14 UC Berkeley Rankings Service Area Ranking Service Area Ranking Portals, Dashboards & Aggregators 2 Data Visualization & GIS 3 Technology Enhanced Teaching and Learning Support 3 Research Applications 3 Collaboration Tools 3 Software Licensing and Distribution 3 Video & Web Conferencing 3 Scholarly Networking Services 4 Instructional Content Creation 3 eportfolio 4 Online Courses 3 Support for Course and Program Evaluation 4 Learning Management System and Support 3 Research Data Management 4 Web Publishing 3 Linked Open Data & Semantic Web 4 Learning Spaces 3 Preservation and archival services 4 Data Analysis (Quantitative and Qualitative) 3 Shared Research Computing (aka HPC) 4 Survey Research Support 3

15 Key Themes Portfolio and Service Management - We can improve many of our services by making them easier to find, easier to access, easier to use, and coordinate across services. Identity/Groups Management - We need a way to create access for different groups of people and utilize those groups across services Service/Project Governance - Create appropriate bodies to provide input and guidance on priorities.

16 Templates Template: Data Collection Worksheet Template: Research Summary Report Template: Findings & Recommendations Project Website:

17 Upcoming Educause Workshop Seminar 08A - Plan for the Future of Research and Teaching Technologies with Peer Benchmarking 29 September - Educause Orlando, Florida

18 Contact Noah Wittman Manager, Teaching & Learning Services Educational Technology Services University of California, Berkeley