PEER BENCHMARKING A Powerful Tool for IT Portfolio Planning Noah Wittman, Educational Technology Services, UC Berkeley UCCSC - 04 August 2014
Presentation Overview In Fall 2013, UC Berkeley adopted a benchmarking framework from NYU to evaluate a portfolio of 23 IT services to support research and teaching. We found this approach to be an incredibly effective tool for portfolio management and strategic planning. This presentation will (a) summarize the project and (b) provide tips and templates for you to conduct your own peer benchmarking project.
Research Services Research Data Management Data Analysis (Quantitative & Qualitative) Survey Research Support Data Visualization & GIS Linked Open Data & Semantic Web Preservation & Archival Services Shared Research Computing (HPC) Museum, Archives & Special Collections Research Applications Teaching & Learning Services Online Courses Learning Management Systems & Support Instructional Content Creation Learning Spaces eportfolio Support Technology Enhanced Teaching & Learning Support Course & Program Evaluation Related Services Collaboration & Communication Web Publishing Software Licensing & Distribution Video & Web Conferencing Scholarly Networking Portals, Dashboards & Aggregrators
UC Berkeley Project Objective Ensure UC Berkeley maintains the highest quality services to support research and teaching by: Benchmarking Berkeley technology services with peer institutions Developing a set of recommendations around future resource realignment and investments Fostering collaboration and a shared understanding across domains and service areas
Process Define Research Share Identify Peer Institutions Define Service Areas Gather Benchmarking Data from University Websites Record Data in Worksheets Narrative Summary of Research Findings Group Presentation & Discussion Assign Researchers Follow-up Phone & Email Interviews One Page Executive Summary
Process Step Date Identify Peer Institutions July 2013 Assign Area Experts July 2013 Web Data Gathering (Comparison Worksheets) July - Aug 2013 Preliminary Analysis (2-Page Summaries) + 10 min Presentation + 10 min Discussion/Feedback Aug - Oct 2013 Phone Interviews + Follow-up Research Aug - Oct 2013 Final Analysis + Draft Recommendations + 10 min Presentation + 10 min Discussion/Feedback Oct - Nov 2014 Meet with Key Stakeholders to Refine Recommendations Dec 2014 - Feb 2015
Project Team Role People Tasks Effort Executive Sponsors VC Undergraduate Education, VC Research, CIO Feedback on Scope & Framing (Peers, Services, etc.); Feedback and Guidance on Service Portfolio & Strategy 20 hours Functional Sponsors ACIO Academic Engagement, Director Research Information Technologies, Director of IST-API, Associate University Librarian Define Project Scope (Services, Peers), Feedback to Project Team, Engage Executive Sponsors, Develop Service Portfolio & Strategy 200 hours Project Manager 1 0.5 FTE Project Manager (4 months) Coordinate Meetings & Assignments, Draft Templates, Track Progress 400 hours Researchers 24 Researchers from across campus (Educational Technology Services, Research Information Technologies, Library, Berkeley Resource Center for Online Education; Applications, Platforms & Integrations) Complete Research, Share Findings 30 Hours x 2 Researchers x 23 Service Areas = 1380 hours Total ~ 2000 hours
Research Services Research Data Management Data Analysis (Quantitative & Qualitative) Survey Research Support Data Visualization & GIS Linked Open Data & Semantic Web Preservation & Archival Services Shared Research Computing (HPC) Museum, Archives & Special Collections Research Applications Teaching & Learning Services Online Courses Learning Management Systems & Support Instructional Content Creation Learning Spaces eportfolio Support Technology Enhanced Teaching & Learning Support Course & Program Evaluation Related Services Collaboration & Communication Web Publishing Software Licensing & Distribution Video & Web Conferencing Scholarly Networking Portals, Dashboards & Aggregrators
Peer Institutions Berkeley MIT Columbia NYU Stanford UCLA Harvard Michigan UCSD Virginia
DRAFT
UC Berkeley Rankings Service Area Ranking Service Area Ranking Portals, Dashboards & Aggregators 2 Data Visualization & GIS 3 Technology Enhanced Teaching and Learning Support 3 Research Applications 3 Collaboration Tools 3 Software Licensing and Distribution 3 Video & Web Conferencing 3 Scholarly Networking Services 4 Instructional Content Creation 3 eportfolio 4 Online Courses 3 Support for Course and Program Evaluation 4 Learning Management System and Support 3 Research Data Management 4 Web Publishing 3 Linked Open Data & Semantic Web 4 Learning Spaces 3 Preservation and archival services 4 Data Analysis (Quantitative and Qualitative) 3 Shared Research Computing (aka HPC) 4 Survey Research Support 3
Key Themes Portfolio and Service Management - We can improve many of our services by making them easier to find, easier to access, easier to use, and coordinate across services. Identity/Groups Management - We need a way to create access for different groups of people and utilize those groups across services Service/Project Governance - Create appropriate bodies to provide input and guidance on priorities.
Templates Template: Data Collection Worksheet Template: Research Summary Report Template: Findings & Recommendations Project Website: http://ets.berkeley.edu/rae
Upcoming Educause Workshop Seminar 08A - Plan for the Future of Research and Teaching Technologies with Peer Benchmarking 29 September - Educause 2014 - Orlando, Florida
Contact Noah Wittman Manager, Teaching & Learning Services Educational Technology Services University of California, Berkeley wittman@berkeley.edu