IT Portfolio Management and Accountability in Higher Education

EDUCAUSE Center for Applied Research Research Bulletin Volume 2006, Issue 14 July 4, 2006 IT Portfolio Management and Accountability in Higher Education Walter Weir, University of Nebraska 4772 Walnut Street, Suite 206 Boulder, Colorado 80301-2538 www.educause.edu/ecar/

Overview An increased demand for accountability in higher education necessitates performancebased reporting across the board. It can be challenging to reduce the complex world of information technology (IT) to tangible measures like return on investment and alignment to mission. And yet, today it is important in higher education to represent these IT assets using traditional financial market terms. Those of us in IT organizations have limited resources and a plethora of requests for our time and resources. How do we objectively determine which projects or systems we should move forward, and which ones must wait? How can we involve others in the decision-making process, and how do we logically explain the result? Are there tools that can help us reach the best decision? This research bulletin explores such questions in the context of IT portfolio management. Portfolio management is a technique that is gaining favor and usage by more and more business institutions such as Sears and Hershey, as well as by local and federal governments, as a way to systematically choose technology investments, including applications, infrastructure, projects, and vendors. Portfolio management is a set of business processes that, when consistently applied, enables organizations to plan and control technology investments. Figure 1 illustrates the processes involved in the planning and control of IT investments. Figure 1. IT Portfolio Management Planning and Control PRIORITIZE AND SELECT INVESTMENTS PROPOSE INITIATIVES Planning ADJUST COURSE CLARIFY OBJECTIVES Control REVIEW PORTFOLIO TRACK PERFORMANCE As CIO for the University of Nebraska a large, multicampus institution I spend the majority of my time listening, communicating, and educating others on the importance of IT and how it impacts the overall institution. Today, IT is broadly recognized as an important institutional asset that must be aligned with enterprise strategies and objectives. 2

Over the past several years, the University of Nebraska has developed an IT Management Information System using portfolio and project management tools and concepts. The system enables us to obtain better information about ourselves and our environment. On the input side, we need to know what assets we have available for investment. We must understand and quantify the work we perform, the time it requires, and the costs associated with accomplishing it. The results of our work and our customers satisfaction with our work must also be measured. Finally, we must communicate this information clearly with others including the customers we serve, the administrative units we partner with, academic and administrative leadership, and, perhaps most importantly, our own IT organization. Highlights of IT Portfolio Management Using portfolio management techniques, we can move from the subjective and sometimes emotional debates about whose project we do first to engage in more objective and informed discussions about how to use our limited resources to best serve the university. Investment Institutions wrestle with important, high-impact questions relating to IT investments. What is the investment our institution should make in IT? How should our limited time and money be distributed among competing initiatives, such as administrative systems, distance education, and academic computing? How do we balance risk and maximize the rate of return? When should we shift our investment of resources among university initiatives? What are the ramifications of our decisions? Can we be confident our actions are the right ones? Communication A critical element of success is our ability to communicate within the IT organization, throughout the institution, and with our vendors. To be truly successful everyone must be on the same page, guided by the same benchmarks and objectives. We must use common, nontechnical language to communicate the status of our key initiatives. Accountability The relative health and the critical success factors of each IT initiative must be constantly monitored so that the appropriate course of action can be determined: continue, modify, or terminate. An early-warning system should allow potential problems to be easily identified and corrected. The resulting decisions must therefore be based on tangible, reliable data, not on guesswork or intuition. Awareness We operate in an environment that has limited resources and an apparently insatiable need for our assistance. Because there are always too many things to do and not enough people or funds to do them, we must constantly focus our efforts on what is 3

most important. In order to judge what work is the most important, we must clearly understand the following: What work is actually being requested? How do customers request our services? How is work assigned in the organization? How do we standardize the flow of work requests? What work should be selected to move forward, given the current workload? How do we know the priority of work? Does everyone understand priority in the same way? Do we really understand what we are supposed to do? Do we have adequate specifications? What are the customer expectations? Why is the work needed? When should it be complete? Is the project aligned with strategic initiatives? What is the value to the institution? Are the benefits greater than the costs? How does this project impact other efforts? How does it compare to other efforts? Is there duplication of effort? Is the right staff assigned to the work? Does the staff have the skills and training needed to accomplish the project or task? Is the effort staffed and funded correctly? Is the project on track? If not, are we correcting problems in a timely fashion? Did the result meet or exceed customer expectations upon completion? Additionally, we need to determine whether we have severe, long-term resource problems. If so, we need the logical basis to build a successful case for additional staff and funds. We also need to recognize high-performance teams and staff objectively so that everyone knows the criteria for recognizing good as well as poor performance. 4

Most importantly, we need to be increasingly responsive to the critical IT needs of the university, given growing business demands, tightening budgets, and compressed schedules. Focus on IT Concerns Implementing a portfolio management environment involves a concentrated effort and dedication to be truly successful. It requires collecting the most fundamental data about work, including project names, start and end dates, the names of people performing the work, and how many hours each person charged to the effort. All of this must be associated with individual payroll records, and operating costs must be applied as overhead. In addition, performance measurements must be selected, defined, and collected. This can include whatever is important to the organization, such as schedule health, compliance with standards, or clearly defined requirements. Information must be gathered from various data-collection environments. This often includes a time management system, financial management system, human resource and payroll environments, and project and portfolio management systems. A data warehouse is helpful to clean and integrate these data. Each step in this process is designed to gather and enhance the data into usable information for decision making and presentation. Integrating various data sources in an automated fashion makes possible reporting of many aspects of operations impractical to manually derive. Any journey is more successful with a plan. Configuring and successfully using a portfolio management system is no exception. Described below is a roadmap for initially implementing and further customizing a portfolio management environment. The sequence of this work is critical: the more planning that is accomplished up front, the more successful the effort will be. Planning the Implementation During this phase, it is important to determine the intended audience, what decisions they will make, and what types of information will be helpful in the decision-making process. A number of approaches can be used to determine this, including holding brainstorming and facilitated sessions, reviewing existing reports, considering material already provided in status meetings, or identifying questions asked of the organization that are difficult to answer. Regardless of the method used, the goal is to get a good handle on understanding the needs of the intended audience with as much specificity as possible. What decisions are they going to be asked to make? How can the data be arranged in a manner that makes the pertinent issues visible? These requirements are critical and will focus effort during the remaining phases of the project. 5

Design the Portfolio Management Configuration When designing the configuration, it is necessary to sequence to the various objects involved. Knowing what needs to be accomplished first and following an ordered plan will help create a successful project. Design a Data Collection System Once the requirements are defined, the required information must be sourced. Sourcing the data necessitates reviewing inventories of existing data and evaluating which data sources are best. Below are questions to help guide those determinations. Question: Is the data readily available? If so, what interfaces are required to capture it? What edits should be written to ensure the data contains appropriate values? At what point should scheduled processes retrieve information? Question: If the data is not available, how can it be collected? What edits should be written to ensure the data contains appropriate values? At what point should scheduled processes retrieve the new data? Once all source data is retrieved, it is often integrated within a data-warehouse environment to produce meaningful metrics. The location of this process and its automation and edit results must all be clearly thought through and designed. At the University of Nebraska, the data warehouse is used to integrate SAP payroll and financial data with time-entry system activity records. Exported from this environment are records of individual work history that have been assigned payroll and overhead costs. Team and project information is also embedded into detail records. Project leaders, team leads, and staff can run their own reports regarding project activity on an Intranet Web site. The data is also imported into the portfolio management system, assigned to appropriate items, and summarized into the relevant portfolios. ProSight Configuration The University of Nebraska uses ProSight, an out-of-the-box portfolio management tool that can be configured to meet each organization s implementation requirements. The requirements defined in the earlier stage of the implementation are used to plan the ProSight configuration. Value Lists and Categories As the phrase suggests, value lists are specific sets of data to be allowed, which must be defined prior to a category that uses them. A category is the most basic element of data in ProSight. It is analogous to a data element and must be defined for each specific piece of data collected or reported on within the environment (see Figure 2). Categories 6

can be calculated like spreadsheet cells using small pieces of JavaScript. In addition, the value for categories can be automatically imported on a regular basis, which is what the University of Nebraska does with project-costing and work-activity data. Figure 2. Category Definition Data Management Screens Data management screens (Figure 3) provide a simple means for both data capture and data review and provide mechanisms for creating common forms, documenting processes, and developing business cases directly in the software. E-mail capabilities help support necessary workflow processes. Figure 3. Data Management Screen 7

Workbooks and Dashboards Workbooks (Figure 4) are most useful for project managers and individual team members to manage project and support-area information. Using workbooks, individuals can annotate the impact of a project for specific investments. Information can be updated on a scheduled or an ad hoc basis. Figure 4. Workbook Dashboards (Figure 5) are a collection of graphs that provide a view of performance criteria trends for an investment. Historical trends provide insight into the life of an initiative. Color-coded bands quickly highlight when an investment has moved out of the acceptable range of performance. Dashboards are flexible, allowing for easy addition of pie, bar, and line charts. Figure 5. Dashboard 8

Scorecards Scorecards (Figure 6) are very useful at providing a spreadsheet-like presentation of items quickly. This type of reporting is useful for building an early-warning system, indicating when a specific category falls outside the predetermined acceptable level of performance. Scorecards provide an interactive means to manage investment performance through the use of categories that measure various attributes such as business value, risk, budget, service level agreements, and more. Figure 6. Scorecard Investor Maps Investor maps (Figure 7) are by far the most visual element of reporting within ProSight. A portrayal of the investment strategy for the University of Nebraska s IT investments can be displayed across an array of categorizations. The maps can be used to drill down into more specific information on investments and are easy to use to perform what-if scenario planning to determine the optimal portfolio mix. These tools make it easier to draw executive management into the decision-making process. Their key indicators can be displayed horizontally and vertically with IT investments populating the cells. Staff can enter updated data into the portfolio management system as projects proceed. The projects are represented by a bubble in a matrix. The horizontal axis measures the value from low to high. The vertical axis reflects ability to execute from low to high. Projects are represented as colored bubbles on this matrix. Large bubbles are associated with highly resourced projects; smaller bubbles are projects that involve fewer resources. Green bubbles are healthy projects; red bubbles represent projects that are in trouble. Executive management can immediately associate IT costs with the University of Nebraska Strategic Framework or in relationship to Running, Growing, and Transforming 9

activities by manipulating the investor map views. For example, one view can show dollars spent on presidential priorities versus dollars spent on presidential priorities that actually transform the business. In this way, investor maps can be used to start the conversation and provide higher-level perspective to collections of activities. Figure 7. Investor Map With investor maps, people who are not comfortable talking specifically about TCP/IP or DB2 environments are able to discuss technology in light of how the technology impacts strategic initiatives. Academic leadership can work with the business community to discuss how additional IT funding will improve the student experience, for example, or further the outreach function of the institution. What It Means to Higher Education People in the technology business do not always communicate well with senior executives. More often than not they innately drop into jargon that nontechnical people tune out. In addition, the belief senior executives hold about IT senior administrators is often, We don t know what you do, but you spend a lot of money. Fortunately, a number of tools and techniques can change this perception and make the IT department s business transparent to everyone. Portfolio management is one such approach. Work Activity Types One of the most disappointing lessons we learned was that there isn t much time available for new project activity. This discovery was made by categorizing all of our various efforts and dividing them into one of four project types and including absences, administrative, enhancement, and service activities. 10

Full-time employees typically work a 40-hour week, but that is far from the number of work hours available for project assignment. First, absences must be subtracted the time staff use for vacation or sick leave, doctor s appointments, or jury duty. We need to measure this time, as it is a consumer of potential resources. Next, administrative time must be applied. This time must be carefully managed and includes attending monthly departmental meetings, answering e-mail messages, engaging in training, researching new technology solutions, and so forth. While these activities are worthwhile, they are all time investments that must be planned. We have discovered that the largest consumer of time is what we call service activities: the amount of time we have to spend fixing things that should work is recorded in this area. We also distinguish time associated with helping people, such as help desk, documentation, and training functions. This time is also included in the service project type. After all of the absence, administrative, and service activity hours are removed, the organization has expended nearly 80 percent of the time available, which is consistent with industry measures. We call this time and expense keeping the lights on. What s left is only around 20 percent, which is the only truly actionable time available. This fact alone explained why we often felt that we were not advancing as quickly as desired. Decisions must be made how to invest this limited time toward enhancing or transforming the institution. Projects Early on, when we inventoried various project lists, we identified nearly 2,000 different projects a shocking number of activities. We found that some people tracked such minor activities as phone calls and e-mail responses. Conversely, others tracked multiyear efforts as single item. We learned that we needed to define something as fundamental as what a project is. Our very simple definition is that a project is something with specific start and end dates when you are finished, you will have something that you did not have before. It took some time to help everyone understand this, but once the understanding was in place, it cleared a path for how we talk about our work consistently. Once we had an inventory, we found a lot of duplication, so we whittled down the initial list. In that definitional phase, any projects that were not really projects, per our definition, were also pulled from the list. We then had a pretty good handle on what was going on within the organization. The next step was to evaluate the projects using the scorecard, whereby we apply a series of questions to each project. For example, for Project A, is it aligned with the strategic initiatives? Do we have the right staff? Do we have the necessary resources in terms of hardware? Do we have the skills? As we score the projects, we can also weight them. We can determine the number of attributes used and the process engaged for scoring. For example, staff skill sets might have more weight than resources available or alignment to mission. 11

Next, all attributes are scored using a team approach. The score can range from 1 to 5 or from 0 to 10, depending on preference. Once each attribute is scored, we determine an average score. If the attribute s score is above the average, it is a viable candidate for moving forward because we have the skills, resources, and support that make it likely to succeed. Other examples of possible attributes might be the number of people that will be affected by the change, or whether the change will be beneficial for only a short time. In these cases, allocating resources might be considered a poor investment. In this sifting process, we can identify even more low-value, low-probability-of-success projects. We examined our current status and where we need to go. We have this process down to a science now, and every month we have meetings with our teams to discuss the projects. There are a couple of ways to measure how we are doing. The first is through surveys, and the second is through causal (or concept) mapping, which measures the probable impact of a project. Causal mapping involves building conceptual models of the problem in the form of cause-and-effect diagrams. The goal of measurement is to verify whether what you are doing is providing the desired improvements. Key Questions to Ask In what ways does our organization control the flow of project decision making? Do project requests go directly to analysts and programmers, or do project requests flow through a central decision-making process? To what degree does our organization record all hours of work and associate staff work with project activity? Do we have a handle on our resource utilization patterns for absences, administrative, service, and enhancement work? How does our organization roll up work activity and relate the aggregated picture to the institution s strategic plan? Would it be advisable to engage an executive management team in a priority-setting session where they help guide our choices based on the institution s overarching goals? Are you able to provide a common, easily understood environment that anyone in the organization from the executive level to the individual team member can easily use to understand, review, and communicate about the technology investments being undertaken? Where to Learn More Datz, T. (2003, May 1). Portfolio management: How to do it right. CIO Magazine, Retrieved March 30, 2006, from http://www.cio.com/archive/050103/portfolio.html Ericson, J. (2005, December 4). Facing IT portfolio management. Line56. Retrieved March 30, 2006, from http://www.line56.com/articles/default.asp?articleid=5199&topicid=3 12

ProSight Portfolios, http://www.prosight.com/solutions/software/index.asp Songini, M. (2001, July 30). Start-up s tool keeps IT projects on track: ProSight's assessment applications help manage IT resource portfolios. ComputerWorld. Retrieved March 30, 2006, from http://www.computerworld.com/networkingtopics/networking/management/story/ 0,10801,62626,00.html About the Author Walter Weir (wweir@nebraska.edu) is Chief Information Officer at the University of Nebraska. Copyright 2006 EDUCAUSE and Walter Weir. All rights reserved. This ECAR research bulletin is proprietary and intended for use only by subscribers. Reproduction, or distribution of ECAR research bulletins to those not formally affiliated with the subscribing organization, is strictly prohibited unless prior permission is granted by EDUCAUSE and the author. 13