Managing the IT Project Portfolio:

EDUCAUSE Center for Applied Research Research Bulletin Volume 2009, Issue February 10, 2009 Managing the IT Project Portfolio: A Project Ranking Methodology Randall Alberts, Georgia State University 4772 Walnut Street, Suite 206 Boulder, Colorado 8001 www.educause.edu/ecar/

Overview In 2005, the Information Systems and Technology (IS&T) division at Georgia State University (Georgia State) instituted a Project Management Office (PMO). The first step for the PMO was to define the parameters of an IS&T project. The second step was to identify the projects currently under way, along with the new projects being requested by the university. After the initial shock of examining the long list of projects, IS&T management asked the PMO to establish a system to rank projects so that resources could be appropriately allocated. The purpose of this research bulletin is to share the methodology used by Georgia State s IS&T department to define and rank IT projects. With three years of experience under its belt, IS&T hopes that the insights and lessons learned might guide other institutions interested in implementing a project portfolio management system. The bulletin also serves as a guide for identifying projects and ranking them for appropriate resource allocation. Highlights of Project Portfolio Management The goal of project portfolio management is to identify and prioritize an organization s projects so that appropriate resources can be applied in a timely manner. To accomplish the goal, systems must be established to provide ongoing project status reports to enable shifting resources to complete projects that support the organization s top priorities. Equally important, these status reports serve to identify projects that do not serve the organization s mission so that they can be stopped before critical resources are spent. Like many things, managing projects as part of an IT portfolio often shakes up the culture of an IT organization, especially when business as usual is disrupted. Senior organizational leaders must be committed to weathering the storms of change in order for project portfolio management to succeed. Resistance will often come from those areas or departments that have pet projects that they like to work on. Personnel in these areas might be concerned that their projects will be scrutinized more closely than in the past, and they might be concerned that resources may be cut or the project cancelled entirely. Projects that tie up resources that could be applied to higher-priority projects might also be at risk. A portfolio management approach that assesses resource allocation from a perspective of serving the overall good of the college or university might well disrupt long-standing practices in the IT department. Strong leadership is required to overcome resistance, facilitate new processes, and make portfolio management work. What Is a Project? The Project Management Institute, a global authority on project management, suggests this definition for a project: A project is a temporary endeavor undertaken to create a unique product, service, or result. 1 2

For Georgia State, this inclusive definition covers much of the work being undertaken by IS&T, from a two-hour project to build a learning object through a multiyear project to deploy an enterprise-wide system. Applying this definition to short projects is problematic because stepping through the project-defined processes can take longer than working on the project outcome. In order to avoid some of the project management overhead associated with small projects, Georgia State defined a project as having to meet one or more of the following criteria: The work to be accomplished requires more than 80 hours of total work effort (for example, one person working two weeks, or 10 people working one day). The work requires effort from more than one department internal or external to IS&T. This includes the database administrators having to make a request from the systems group to upload a patch to the database servers. Even if this were a small project, the inclusion of this as a project allows visibility by the other staff so that resources can be allocated accordingly. Any part of the project requires institutional funds to be expended, exclusive of salaries. The project has a large university impact. This includes updates and patches to major systems on campus that would require a scheduled outage for a majority of the campus users. Such projects require communication plans to keep the customers informed. Groups internal or external to IS&T wanting work accomplished that meets any of the criteria above must complete a project charter, usually with the help and input of IS&T. The charter defines the customer s needs, the project scope, and the name of the project sponsor. Signed charters are then taken forward for discussion. Establishing a Project Prioritization Model At the 2005 EDUCAUSE Annual Conference, Ernest J. Nielsen of Brigham oung University helped highlight the need to prioritize projects by presenting a project prioritization model that gives each project a numerical value. 2 The model mapped strategies and objectives (for example, optimize the use of resources; improve reliability and integrity; increase effectiveness and accessibility; seamless and interoperable technology; and so forth) to proposed criteria (for example, does it reduce unnecessary/likely system redundancy, hard costs, and cycle time; does it respond to the immediacy of need and the dimensions of integrity; does it enable or improve the ability of users/providers to do what they need to do; and so forth). The strategies were then scored on a scale of 1 to 5, and each score was anchored to a scale related to the strategy. Using this model, a numeric score for the project was assigned. Inspired by this model, in the fall of 2005 IS&T s senior management, led by the university s CIO, met to determine the criteria to be used to categorize projects. Each of these meetings focused on a set of variables that could be measured for each project and the appropriate weight for each. Once the initial scoring sheet was determined, the

directors and the PMO worked to score each of the existing projects to see how they would align under the new scoring system. Some initial adjustments were made, but the system was a step forward in ranking projects. The next step was to determine how many projects could be completed in a calendar year and which projects would receive resources. To accomplish this, the organization came up with a three-tiered system for projects. High-priority projects are classified as the top-10 ranked projects. In IS&T terms, all available resources would be applied to these projects to meet the defined project timeline. This included pulling resources from lower-priority projects in order to meet established targets. Medium-priority projects are the 15 projects ranked 11 25, just below the High tier. On these projects IS&T works to meet the defined deadlines; however, timelines might slip depending on the resource needs of the higher-priority projects. Low-priority projects are those projects ranked 26 or below. These projects might get some work cycles applied if the resources are not needed on higherpriority projects or on operational and maintenance tasks. Some low-priority projects get completed because they require specialized skills not needed by higher-priority projects. Ranking the projects helped the IT organization clarify its priorities and resource requirements. Workgroups were able to determine what they should be working on, and the exercise also helped focus the dialogue with customers regarding priorities and timelines. One major lesson learned early was that determining rankings based solely on the perceived expectations of customers was a bad idea. It soon became evident that the process of assigning project rankings must include people external as well as internal to IS&T. Working with Senior Executives on Project Ranking About the same time as IS&T started defining and ranking projects, the CIO established an Information Technology Steering Group (ITSG). The ITSG is co-chaired by the provost and CIO and consists of the university vice presidents, associate provosts, and the chair of the University Senate subcommittee on Information Systems and Technology. The ITSG tries to meet twice a month to discuss the major IT initiatives for the university. Soon after IS&T started ranking projects, it became apparent that the ranking of the projects by the directors did not reflect the broader perspectives of the members of the ITSG committee. IS&T was too involved with the projects to rank them objectively on behalf of the customers, so ranking needed to be removed from IS&T control. This also brought to light some political challenges, as department heads would often advocate for their smaller projects in order to meet a very specific need. 4

The ITSG started by reviewing the current ranking methodology put forth by IS&T. The group made some changes to the spreadsheet so that the rankings would correspond more closely to the business needs of the university instead of the technical needs or abilities of the IT department. This spreadsheet modeling process was a good learning experience for IS&T and provided insight into the work that the business people considered important. Once a new spreadsheet was developed, the ITSG then worked to reclassify all of the existing projects based on the new model. This half-day exercise included the ITSG members along with the PMO and IS&T directors. During this process, IS&T worked to make sure that the ITSG understood the scope and high-level details about the projects so that an informed decision could be made. Once everyone understood the projects, the ITSG assigned a numeric ranking to each one. Once ranked, each project was placed in priority order. Lines were drawn below the high-priority projects and the medium-priority projects so that everyone could understand where the projects fell in regard to IT s commitment for completion. The structure as it exists today was implemented in spring 2006. Since that time, very few adjustments have been made to the ranking process. When a new project comes forward from the customer or IS&T, a project charter is developed and it is taken to the ITSG for review. The ITSG reviews the project charter and completes a new spreadsheet for each project. The results of the spreadsheet are given to the PMO, which updates the project classifications and makes sure that a report is distributed to the appropriate parties. As projects progress, high-level change orders are passed along to the ITSG requesting that projects be reevaluated. This can result in projects moving up or down the list or being placed on hold as required. Logic of the Project Prioritization Model The project ranking model is based on weighted responses to high-priority questions for Georgia State, which are typical for IT projects in higher education. For example: Which institutional strategic goals does the project support? Is the project required to sustain the operations of the institution, to reduce risk, or to reduce cost? How long is the project time line, and to what degree is it flexible? What is the urgency of this project for the institution as a whole or for a college or department? To what degree will the project result in a reduction or an increase in staff resources? Who will benefit from this project, both within and outside of Georgia State? To what degree will this project improve the effectiveness of the user or provider of the service? 5

Does the project replace a current system that is nearing its end of life, or does it enhance a usable system? What is the implementation cost for the project? What are the operational costs for the project in terms of return on investment? Ranking Criteria While the criteria used to rank projects have developed over time, below are the criteria currently in use by Georgia State for ranking IT projects. Other institutions that review the processes Georgia State put into place will likely want to evaluate the criteria and scales and modify them to meet their own institutional missions. The ranking model is arranged in seven columns. Strategy This is the high-level criterion used to identify the scoring row. It is used to connect the project to the strategic mission or goal of the university. Weight: This field is the multiplier for the scoring row. Any factor can be entered into this field with the most common being 0.5, 1.0, 1.5, or 2.0. Criterion: This column is used to help ask an important question for the scorer. This question should help the scorer decide on the best descriptor for the scoring row. Scoring: The model is built in an Excel spreadsheet so that the scorer simply needs to put a next to the condition that best represents the answer to a given question. Based on where this is placed, a score for the row is calculated. Condition: Conditions are the possible responses for the criterion column. The scorer puts a in the scoring column next to the best answer from the Condition column. Each condition will have a corresponding score. Some conditions can carry a negative number to emphasize an unwanted characteristic. Sub-score: The sub-score determines the points available based on the selection of the condition. Total score: The total score is equal to the sub-score multiplied by the weight. For example, if a condition is chosen worth points and it has a weight of 1.5, then the total score will be 4.5 for that scoring line. Priority calculations are delivered in two ways: Calculated priority: This is the sum of the total scores for each question. The maximum number depends on the number of scoring rows and the highest possible score for each. Calculated priority on a 100-point scale: At Georgia State, the maximum that one can score on our scoring sheet is 60. This field defines the calculated priority on a 100-point scale. 6

The Model: Strategies, Weights, Criteria, Conditions, and Scores Below is an example of each of the categories that Georgia State uses in ranking projects, with responses included for a hypothetical project. Each row represents a different dimension to the total score. The number in parentheses in the strategy cell is the maximum number of possible points for the row. Strategic goal (7.5) 1.5 What strategic goal does this project support? Multiple strategic goals (+5) Single goal high priority (+) Single goal without high priority (+1) No strategic goal supported (0) 4.5 The first row reflects how the project relates to the strategic goals of Georgia State. In order to show the importance of some criteria over others, a higher weighting is given to the row. The row of Strategic Goal carries a weight of 1.5. Projects that support multiple goals get the maximum score; less strategic projects are weighted accordingly. Required to sustain university operations (+5) Required (7.5) 1.5 Is this project required? Required to reduce risk (+) Required to reduce institutional cost (+1) 4.5 Not required (0) The second row measures the extent to which the project is required for compliance, risk management, or cost reduction. A patch or upgrade to major university systems based on government regulations falls under the category of university operations. Members of the ITSG consider federal regulations and upgrade requirements in determining whether a project is required. Required in less than 12 months (+5) Flexibility (7.5) 1.5 Is the project timeline flexible? Required in 12 18 months (+) Required in 19 24 months (+1) 4.5 No deadline (0) 7

Flexibility measures when the project needs to be implemented. This row in no way guarantees that a project will be implemented in the timeframe requested, but instead is a weighting factor. If a project is needed in under 12 months but meets no other criteria, then the project could be ranked lower and not meet the project timeline. It will be up to the customer and project team to determine when the project can be implemented. Need for the system (7.5) 1.5 Is this an urgent need for the university? Urgent (+5) Pressing need (+4) 4 Urgent for college or department (+) Pressing need for college or department (+1) 6 In an effort to make sure that the ITSG considers whether a project benefits the university as a whole or a specific component of the institution, this criteria ranks both urgency and the numbers of customers who will benefit from the project. Staff or system reduction (5) 1 What is the effect on staff or systems reduction? Addresses staff or system redundancy (+5) Reduce head count/systems (+) Additional staff/ systems needed for support (-2) The criterion called staff or system reduction addresses the impact of the project on resources that will be needed to support it. If the project will ultimately free up resources, the score is positive; if it will consume additional resources, the score is negative. Customer base (5) 1 What users will be impacted? Beyond Georgia State (+5) Georgia State (+4) Large subset of Georgia State (+) Subset of Georgia State (+2) Small subset of Georgia State (+1) Projects that impact the largest customer base are ranked highest on this criterion. 8

Increase effectiveness (5) 1 Does it improve the ability of a user/ provider to perform a task? Improvement for user and provider (+5) Improvement for user or provider (+) No improvement for user or provider (-1) Improving effectiveness through IT is an ITSG goal. Projects that are designed to meet this goal receive high rankings. Current state (5) 1 What is the state of the current system? Completely inadequate/ end of life (+5) Functioning, but close to end of life (+) Functioning, but could be enhanced (+1) In evaluating the projects, the state of the current system or process is taken into account. Those systems that are out-of-date or approaching end of life rank higher than functioning systems. Cost (5) 1 What will the project cost to implement? Less than $25,000 (+5) $25,001 50,000 (+4) $50,001 150,000 (+) $150,001 500,000 (+2) More than $500,000 (+1) Implementation cost is one factor that is considered for project ranking. Low-cost projects receive a higher score than high-cost projects. Operational cost (5) 1 Is there a positive ROI? Will pay for itself and generate cash (+5) Implemented to avoid cash expenditure (+) No effect (0) Post-implementation operational cost is another factor considered for project ranking. Projects that generate new resources or pay for themselves receive a higher score. 9

What It Means to Higher Education Portfolio management has been a driving force in business over the past decade. Only recently has higher education determined the need to develop a portfolio view and manage resources accordingly. While developing a project portfolio discipline will take time, there are steps that can be taken to move an organization along that path. The first is to define what a project is, and the second is to develop a model for ranking projects against institutional goals. In both good and difficult economic times, it is important that institutional resources be allocated to the right IT projects. Many IT projects will clearly make things easier, or better, or more efficient, or more compliant, but it is the institution s role to determine which of these measures is most important. Likewise, many IT projects will make things much better for a large number of people; others may be perceived as improving student and faculty recruitment and retention (or supporting other high-level strategic goals); and still others may provide critical functionality for a small, revenue-generating arm of the institution. Which takes precedence when the same resources are required for each? It is important that decisions about the IT project portfolio be made collaboratively among a broad spectrum of institutional executives, IT leaders, and project stakeholders. Reaching consensus on the process and criteria for ranking projects is as important to the ultimate success of the project as is the execution of the project itself. Like good market research, this consensus-building process goes a long way toward ultimate acceptance and buy-in for a new IT system or service. How does our institution define a project? Key Questions to Ask What process does our institution use to request a new project? Who at our institution is responsible for prioritization of projects? How are resources assigned to projects, particularly when there is a resource conflict between two or more projects? Where to Learn More Alberts, Randall, and Joe Amador. Leading Engaging Group-Oriented Projects. Preconference seminar presented at the EDUCAUSE Annual Conference, Orlando, Florida, October 27 1, 2008. http://connect.educause.edu/library/abstract/leadingengaginggrouporien/47510. Alberts, Randall, and Karen Oates. Measuring IT Staff Time at Georgia State University (Research Bulletin, Issue 4). Boulder, CO: EDUCAUSE Center for Applied Research, 2008, available from http://www.educause.edu/ecar. 10

Project Management Institute. A Guide to the Project Management Body of Knowledge Third Edition. Newton Square, PA: PMI, 2004. Whitten, Neal. Let s Talk!: More No-Nonsense Advice for Project Success. Management Concepts, Inc., 2007. Endnotes 1. Project Management Institute, A Guide to the Project Management Body of Knowledge Third Edition (Newton Square, PA: PMI, 2004), 5. 2. Ernest J. Nielsen, Getting the Right People on the Right Projects at the Right Time (paper presented at the EDUCAUSE Annual Conference, Orlando, Florida, October 18 21, 2005), http://connect.educause.edu/library/abstract/gettingtherightpeopleonth/8668. About the Author Randall Alberts (ralberts@gsu.edu) is the Senior Project Manager and head of the Project Management Office in Georgia State University s Information Systems and Technology division. Copyright Copyright 2009 EDUCAUSE and Randall Alberts. All rights reserved. This ECAR research bulletin is proprietary and intended for use only by subscribers. Reproduction, or distribution of ECAR research bulletins to those not formally affiliated with the subscribing organization, is strictly prohibited unless prior permission is granted by EDUCAUSE and the author. Citation for This Work Alberts, Randall. Managing the IT Project Portfolio: A Project Ranking Methodology (Research Bulletin, Issue ). Boulder, CO: EDUCAUSE Center for Applied Research, 2009, available from http://www.educause.edu/ 11