Critical Success Factors for Managing in a Virtual Environment

Transcription

1 Critical Success Factors for Managing in a Virtual Environment James H. Dobbins, Ph.D., MITRE Corp. and Richard G. Donnelly, Ph.D., George Washington University ABSTRACT The authors ongoing research in Critical Success Factors (CSF) began with a focus on Critical Success Factors (CSF) for Program Management of Department of Defense Programs. This work resulted in the development of a process whereby any manager can derive his or her own CSF for their particular position at any point in time. This represented an advance on the foregoing work in CSF, much of which attempted to develop a universal list of CSF from which managers could select appropriate items. The prior research had succeeded in developing pertinent lists, but without much commonality among the lists developed. The authors CSF process could be readily learned by managers, who could thereafter apply the process to any other managerial position to which they are assigned. Following the authors first publications in CSF for program management (beginning with Dr. Dobbins Ph.D. dissertation under the direction of Dr. Donnelly), the authors continued to do research in this discipline, working with a variety of managers to further investigate the applicability of the CSF process to their various positions. One key focus of our ongoing research was CSF applicable to management in a virtual environment. This focus was especially pertinent to technology innovation, an area of increasing importance, and an enabler to working virtually. The management of a virtual workforce remains poorly understood by scholars and is a skill that has to be developed among practitioners. The research supporting this paper was conducted by interviewing managers who were attempting to manage virtual organizations. Some organizations were within a single company division, and some were international operations being managed virtually. The results of the interviews of those managing virtually constitute the principal focus of this paper. Background Examination of the CSF literature over the past two decades shows that most of the prior studies had the goal of identifying lists of general CSFs, which managers could use universally. However, the totality of the literature clearly demonstrates that no usable universal list of CSFs has ever been identified by statistical analysis of survey results, the approach taken in most prior research. Also, many of the CSFs identified in prior research were not stated as activities, the form preferred for defining CSFs because it allows objective measurement of whether they are being addressed properly. Finally, all prior lists deliberately excluded contextually relevant CSFs. 1

2 In our initial research we found that there are significant deficiencies in the identification of CSFs by statistical analysis of survey results, when CSFs are tested in the laboratory of actual program management situations, in this case in program offices involved in acquisition of various technology-intensive systems. We also addressed a number of fundamental questions about CSFs raised by the observations from our pilot study and the literature findings. We achieved a research objective that is altogether different from that of most prior research we developed a generalized process by which program managers could identify CSFs rather than seeking to identify a superior list of general CSFs. The result was the CSF Process Model, an approach by which program managers could identify their own CSFs, determine the constraints applicable to each, and derive a set of measures for each CSF. The generalization is in the process, not in the list. This approach formed the basis for the dissertation (Dobbins, 2000) on which our continuing research is based. Among our most significant findings are: We found that it is critical to understand the applicable constraints in deriving both the CSF and the manner in which individual CSFs are to be measured. In addition, we found that changes in the constraints signal when changes to the set of CSFs are occurring. Also, we found that application of the CSF Process Model serves as a form of program risk assessment, providing a protocol through which a manager could determine the risk in accomplishing each identified CSF successfully, and therefore determining the risk of completing the program succesfully. As a practical matter, our findings demonstrate that by employing the CSF Process Model managers learn how to conceive, plan, and implement their programs in terms of the applicable CSFs. Having learned to think in terms of CSFs, managers can then repeatedly apply the approach to current programs to detect changes in CSFs as the program environment changes, and can apply the approach to subsequent program assignments. Peripheral benefits emerged as well: Senior management can use the process as a training tool to assist promotable junior managers to think like executives. Furthermore, a program organization can enhance the accomplishment of organization-wide goals and foster vertical communications if multiple levels of management all employ the CSFs process model, since managers are better able to engage in shared and cross-dependent activities and communication in this way. What Are Critical Success Factors? To establish a common understanding of the subject matter, and to understand the value of CSFs, one must be familiar with the fundamental definition of CSF. Rockart (1979) first introduced the critical success factor theory and defined CSF: A) CSFs are the limited number of areas in which results, if they are satisfactory, will ensure successful competitive performance for the organization. They are the few key areas where things must go right for the 2

3 business to flourish. If results in these areas are not adequate, the organization s efforts for the period will be less than desired. B) CSFs are areas of activity that should receive constant and careful attention from management. It is apparent from this definition that CSFs are activities critical to overall success but are not a statement of every routine thing that should be done for eventual success. Successful accomplishment of a set of CSFs creates an organizational environment conducive to enabling the associated projects to be successfully managed. However, successful management of any project always requires an appropriate project concept, detailed project plan, and effective execution of the plan to achieve time, budget, specification, and customer satisfaction goals. What Value Is There in Determining CSFs? The underlying theory of CSF analysis, as originally developed by Bullen and Rockart (1981), lays the foundation for a framework for determining those activities, critical to the success of the organization or project, which must be done well by top management. Armed with this information, the information systems departments can structure a knowledge management system focused on measurement of these critical activities. CSFs can also be used as the basis for strategic planning and for vertical and horizontal organizational communication. Once known, CSFs serve to focus the time of the project manager on those things of paramount importance and help to determine which things should be delegated. Some CSFs are intrinsic to the project itself, required by the nature and environment of the project. Others are extrinsic. Some may be outside the project itself but not necessarily related to external agents of influence. These may be activities unique to the manager and driven by his or her own personal needs. For example, a manager who has no experience whatsoever with software development or software acquisition may have a need to learn more about software acquisition management when taking over as the project manager of a software intensive program. Bullen and Rockart (1981, pp ) suggested that when interviewing managers, the objective is to help the manager make explicit that which is normally only implicit. They state that once the manager knows the organizational goals, it is equally important to determine, in a conscious explicit manner, what the basic structural variables are that will most affect his success or failure in the pursuit of these goals. These are the critical success factors (Bullen & Rockart, p. 13). Many of the more skillful program managers intuitively determine CSFs rather than rely on standard information from their own management information system (MIS) to manage projects. However, where the CSFs are not explicitly identified and recorded, they do not become a part of the project history and are not explicit elements of the management reporting process. Furthermore, the underlying constraints for the CSFs do not command attention, and the CSFs are seldom measured. A successor project 3

4 manager, given his or her own skill level and background, may be more or less capable of, or interested in, identifying CSFs or may focus on a different set of intuitively perceived CSFs, if indeed any at all. In the absence of an active and continuous process of identification of the project CSFs, all management activities and decisions are made without any documented record of those activities critical to project success having been incorporated into the project history. Once explicitly identified and made part of the project documentation, with the underlying constraints clearly and explicitly stated, the information gathered promotes project stability. Once a set of CSFs has been explicitly identified and communicated, the likelihood that the CSF will be ignored becomes minuscule. Some Prior Difficulties One of the more comprehensive early research studies conducted on CSFs was that done by Boynton and Zmud (1984). Boynton and Zmud conducted two case studies and then reported what they concluded were the strengths and weaknesses of the CSF-based method of supporting MIS planning and requirements analysis. They concluded that CSF analysis has been used successfully to identify key concerns of senior MIS management. They also indicated that, beyond the MIS arena, CSFs can be used in developing strategic plans and identifying critical implementation issues; in helping managers achieve high performance; and, in establishing guidelines for monitoring a corporation s activities. The authors also noted that CSF identification had been cited by others, in prior research, as having three principal weaknesses: (1) difficulty in use and therefore not appropriate unless analysts possess the capability to successfully apply the CSF identification process; (2) questionable validity because of potential analyst/manager bias introduced through the interview process; (3) questionable applicability as a requirements analysis methodology because the resulting information model may not accurately represent the deployment environment. A clear description of the CSF identification process was not given by the authors. Having noted the difficulties reported in prior research, an investigation was begun to determine whether there was a way to gain access to the advantages of CSFs while avoiding the problems being consistently reported. In a pilot study conducted by the authors three years prior to formulating the design for the research discussed herein, some of the same problems reported by others surfaced. The objectives of the pilot study were to determine if managers of highly complex defense projects are able to identify the CSFs applicable in their cases and to determine whether the reported CSFs were sufficiently general to apply to all or most projects of the same type. Managers of two groups were surveyed those managing large embedded systems and those managing automated information systems. The results of the pilot study were that some CSFs were mentioned by multiple managers, but that none of the CSFs were mentioned by all or a majority of managers, even when managers worked in a given domain. This emphasized the difficulty of identifying a set of general CSFs. This is discussed in greater detail in previously published studies (Dobbins & Donnelly, 2004a, 2004b). 4

5 The literature on CSF research reveals four key characteristics. (1) All of the past research performed with the objective of discovering general CSFs was done using a survey, as opposed to case methods, consistent with the objective of seeking general CSFs. (2) Past research tended to avoid consideration of CSFs that were primarily contextually relevant under the assumption that such CSFs would fail to be sufficiently general, thereby not serving the broad project management community. (3) The various research studies conducted to discover general CSFs tended to each come up with a different list of entries, even though there were some common or overlapping CSFs identified. As a practical matter, this left an open question as to which entries a project manager should use, and why. (4) Most of the CSFs identified were not stated in the form of activities, contrary to Bullen and Rockart s (1984) fundamental definition. Little or no attention was given to standards of measurement of the CSFs. The Rationale Behind the Design of the Current Research Overall, the process developed for the research described herein was designed to determine CSFs by utilizing the guidance of a skilled and impartial interviewer leading managers through the process steps. This is done by first identifying all elements of importance related to each of 10 process model criteria categories, then grouping all related items of information by topic, and then identifying the CSFs for each topic group in terms of an activity. This eliminated much of the abstractness and other difficulties previously encountered. This difficulty can be seen when general CSFs previously identified are examined. It would be difficult, if not impossible, to determine the constraints underlying CSFs identified as 1. top management support, 2. client consultation, 3. project mission, 4. troubleshooting, 5. well-defined schedule and plan, etc. It is also difficult to see how any of these CSF could be effectively measured. If one were to apply these proposed CSFs to any given project, one would have to identify a whole range of context specific activities needed to implement each, and identify a contextual set of measures for each of the CSFs supporting activities. The effort would be significant. CSFs are activities, not goals. Sum and Ang (1997) recognized this in a published material requirements planning (MRP) study, in which they stated: Past survey studies on CSFs in MRP implementation did not examine the contextual elements that make up each factor. This is not surprising because surveys are limited in exploring the contextual issues surrounding the CSFs. Previous studies definitions of CSFs were too broad and general to provide useful and meaningful guidelines for MRP implementation. For example, top management support has often been cited as a CSF, but what exactly constitutes top management support is not really known. Good performance of the CSFs requires that their elements (or constituents) be known so that top management can formulate appropriate policies and strategies to ensure that the elements are constantly and carefully being managed and monitored. Lack 5

6 of clear definitions of these CSFs may result in misdirected efforts and resources. Several conclusions were reached in considering these issues. Merely trying to identify CSFs was not sufficient. We must also know on what each CSF is based in order to test its current and continued validity. CSFs must be formulated as activities so they can be tracked over time and measured. It was also clear that the fundamental premise that CSFs must be relevant to the individual manager, was true and the research design had to take this factor into account. This, for all intents and purposes, eliminated survey research. A case study design was chosen to accomplish two objectives. First, instead of focusing on the identification of CSFs, the research focused on defining a comprehensive general process a manager could use to derive, not simply state, their CSFs. This process would have to guide the manager in the identification of the CSFs, determine the underlying constraints upon which each is based, and determine the measures by which the CSFs could be tracked and evaluated. The design also had to make sure that the CSFs were stated as activities and had to account for the fact that most managers have not been trained to think in terms of CSFs. To accomplish this, an interview based case study design was chosen that would lead a manager through the different process stages so that in doing so the manager would not only become educated in CSF thinking but would also understand how CSFs can be effectively used. The CSF Process Model To accomplish these objectives, a process-based model was designed and given the name CSF Process Model (CSFPM). Through an interview guided application of the process, the constraints underlying each CSF surface as the managers discuss issues of importance relative to each of the 10 criteria categories of the model. In developing and using the CSFPM to assist managers in identifying their CSFs, a particularly significant difference compared to survey research methods is that the managers interviewed are not asked to specifically identify, or list, their CSFs. Based on our preliminary research, we concluded that, until they have been educated in the CSF process, many managers are not sufficiently familiar with thinking in terms of CSFs to provide a credible list of CSFs for their projects. In the interview process, the manager is asked to identify all critical activities related to each of the 10 categories of inquiry in the general criteria model. Each category is defined for the manager and, where appropriate, examples are given. The manager understands that he or she may not necessarily have an activity for each of the 10 categories. They also consider not only what was done or is being done, but what that manager believes should be done or should have been done. The desire is not to constrict the thought process of the manager by the past or current activities actually performed on the program. The manager has to be given the freedom to think in a creative or innovative 6

7 way if that is beneficial. At the end of this data gathering process, the first interview is terminated. The interviewer then examines all the activities described under each of the 10 categories of inquiry and groups them according to subject or topic. Then the activities in each group are analyzed for internal consistency. If the activities are consistent, they are then evaluated for critical validity. This means that if the interviewer identifies an activity that on its face seems to be irrelevant to success, it is flagged to bring it to the attention of the manager during the follow-on interview to see if it should be modified or discarded. The residual set of activities is then analyzed and a candidate CSF proposed that exemplifies what that full set of common activities is advocating. Each proposed CSF is assigned a number. The grouped set of activities on which each CSF is based is the set of constraints for that particular CSF. Following the identification of all activities important in each criteria category, the interviewer examines each set of CSF-related constraints to determine a candidate set of measures and considers how the measurement data should be presented for effectiveness in knowledge management and communication. Each individual constraint is considered for measurement, and the measures may be either quantitative or qualitative. The collective set of constraint measures related to a given CSF becomes the set of measures by which the success in accomplishing that CSF is determined. This initial set of information is then provided to the manager in a scheduled follow-up interview. During this second interview, the interviewer inquires whether the data necessary to perform each of the stated measures is already available or, if not, if it will be available at the time it will be needed. Any changes the manager makes are incorporated and the interviewer then prepares the final report. This final report becomes the manager s product to use in his or her daily activity and to use as the foundation for a knowledge management process. By determining and recording all three types of information CSF identification, constraints, and measures and by making this part of the project office documentation, the managers will be able to incorporate the information into their executive knowledge management system, and use the information to determine when a change to a given CSF is occurring. The key to understanding the need for the change is recognizing when documented constraints supporting a given CSF are no longer valid. The new or changed CSF, and its related constraint information, can then be used as the foundation for revising the knowledge management information system content, the strategic plan, and possibly the organization structure, in any way necessary for the manager to have the best possible information and implementation strategy for managing the project. One of the advantages of having a skilled interviewer working with a manager during his or her initial application of the CSF Process Model is that the interviewer is not only helping them identify the model products (CSFs, constraints, measures), but is also educating them in the complete CSF analysis process, including learning to think in terms of CSF. Each step is explained in detail as the results are provided to the manager. 7

8 Having examined the CSFs in this way, we concluded that for Department of Defense project managers certain CSFs will likely be important and common to many of them. We also found that for any given manager, there will likely be identified some CSFs that are generally applicable to many managers, and some that are highly contextual and apply only to that project or that manager. Conclusions from the Basic Research Results obtained from application of the CSF Process Model show that it is both general and sufficiently comprehensive for each manager interviewed to be able to determine a set of contextual CSFs applicable to his or her needs. Because the model is general, the associated process by which the CSFs are determined is general, and thus this modelbased process can be incorporated into the skill base of the manager. However, the identified CSFs themselves will always be contextually specific and therefore directly useable as they are stated. Some, by their nature, will be generally applicable within a given domain application area. It is not necessary to specifically seek general CSFs. They will be identified as a natural outcome of the application of the model. The general CSFs identified are those applicable to the project. Given the definition of CSFs, that all are critical to project success, limiting a manager s consideration to only a predefined set of CSFs is to invite project failure in many of the chaotic environments of management today. Even more importantly, it also creates the impression that it is unimportant for the manager to learn how to think in terms of CSFs, leading to a diminished ability to effectively use CSFs or to recognize an inappropriate CSF. CSF for Managing in a Virtual Environment The initial dissertation research done by the authors was directed toward determining the Critical Success Factors for program managers of large Department of Defense programs, both information systems and weapon systems. Following publication of the dissertation, the authors conducted joint research expanding the scope of application of the process. We applied the process to a diverse set of managers, such as a Deputy Undersecretary of Defense, many of whom were not and had never been program or project managers. In each case, the process yielded the desired insights for the manager. After developing the CSF Process Model, and determining it can be successfully applied to managers other than defense program managers, even managers who were not program managers at all, we turned our attention to focused application of the CSF Process Model. We wanted to know if the model can be applied to one aspect of a manager s job, exclusive of the other aspects. Our first focus area for application of the model was the virtual environment. Given the government emphasis on telework, and the growing 8

9 technological capabilities that support working in a virtual environment, we were interested in determining the CSF for managing a virtual workforce. Our interest was to determine whether there are CSFs that specifically apply to those managing a virtual organization as opposed to those managing organizations which are largely geographically centralized. There were many questions which we believed had to be answered, such as: - How do employees respond to working away from the corporate body? - How do employees become inculturated and develop a corporate identity if they work in a separated location? - How do customers respond if they do not see the manager responsible for the people supporting them? - Does geographic location, and the indigenous culture of that location, need to be considered? - What impact does it have on the manager and the other employees when one part of the workforce is working virtually? In conducting the research on those managing virtual organizations, certain key elements of success surfaced which were not seen or were not prominent in the results of our prior research with those managing geographically centralized organizations. Some of the differences were related directly to distance from the central organization, and the effect that has on some employees, and some differences related to the cultural setting in which the remote employees were located. It is these differences which we will principally discuss, for they suggest elements that should be included in a knowledge management system for virtual organizations that may not be necessary for a geographically centralized organization. We will discuss a few cases in detail to expound on these points. Case 1: This manager is a project management consultant. For the first six years of his company s existence, he did not have an office. He had a networked staff of senior consultants geographically dispersed who satisfied client needs by traveling to the client site. He maintained contact with his consultants primarily via computer, using both a web site and . Since his consultants were all very experienced, he felt comfortable in giving them the freedom to operate in relative autonomy, provided they were willing to travel, file follow-up reports, and post lessons learned. The knowledge gained from each assignment was communicated to the rest of the consulting team through their communications network and could thus be added to the collective tool boxes of experience-based knowledge acquisition. In this way, experiential knowledge gained by one individual was multiplied among many. Clearly, it takes a staff with a reasonable level of maturity to take full advantage of this kind of knowledge acquisition and assimilation. Junior and inexperienced staff consultants could not be effectively used in such an autonomous operating environment. Case 2: Another manager was the CEO of a company based in Ireland. His company not only operates virtually, but makes products to assist those who desire to work virtually. They market three products, taking advantage of groupware, portal availability, and an underlying data base. His motivation for operating virtually was growth. He saw limited 9

10 opportunity for growth in the high tech arena in Ireland. He jokingly said, All the Irish want to do is sing, dance, and drink beer. He knew he had to expand into areas that were beds of high technology, or potential high technology marketplaces. His principal virtual office locations were Boston, Mexico City, and Sydney. He found three things were critical to his success in operating virtually. He had to establish excellent communications with his virtual staff; because the customers want to meet him in person, he had to spend a lot of time traveling; and he had account for the cultural differences at each of his major bases of operation. He found that he had to spend a lot of time traveling because the customers in each location wanted to meet face to face (F2F), and have meaningful dialogue ear to ear (E2E), with the person to whom they were giving their money. No matter how competent his virtual staff is, he still has to spend some personal time with the customer. His conclusion was that virtual is virtual for everyone but top management. Cultural differences are important. In Boston, the environment is high tech so his virtual work force was able to operate comfortably. He still had to have regular communications with them and make sure they identified themselves as a part of the company, not just an independent contractor. This means occasional trips to Ireland for the employees as well as other forms of staff communications. He found that in Mexico, it was very important to meet the customer in person because in Mexico the customer not only wants to meet you, they also want to have a meal with you. They are a culture that values personal contact, so operating virtually with his company is not something they desire on a continuous basis. They need the occasional personal visits. He said that in Mexico, They want to feed you and hug you. He also found that the people working in Australia took to virtual operations easily. He noted that there are only about twenty universities in Australia, so a large part of the Australian student body attends school virtually. This background of growing up in a virtual environment makes it easy for them to transition into a virtual working environment. Communication with his virtual staff is also important. is not sufficient. The people want to hear his voice. This is one way they feel connected to the company, so he spends considerable time on the phone each day with his people. Given that he has his home base in Ireland, and they are as far away as Australia, the time differences become an important aspect of his communications network and process. Therefore, when he travels, he spends time with his own people, not just the customers. Case 3: This manager was a naval officer who was the Program Manager of an advanced missile system. He had a local program office consisting of himself, and three support contractors. The rest of his program office was geographically dispersed across the United States. His biggest challenge was communication, which he approached in two primary ways. He set up a web site for the program, requiring special access, and used it in a unique way to communicate with his organization. Each person in the organization, 10

11 both program office personnel and contractors, had a unique log-on. The web site was configured so the opening page had the latest information about the program. If they did a test fire of the missile, within three hours the results of the test were posted on the opening page. Other current information on the program was also found there. This kept one of the contractors busy almost full time making sure current information was posted quickly and obsolete information was removed. When program office personnel logged on, the system could tell which items of information they had not yet accessed. When they went beyond the opening page it would bring up all the new items since their last log on. The Program Manager also had a monthly program meeting in Denver. Denver was centrally located and everyone was expected to attend. Case 4: This manager operated entirely within his corporation, a large defense contractor. He was responsible for a significant part of the data infrastructure, and his entire organization was virtually dispersed through the corporation. His company was his customer. His biggest challenge was also communication. He spent a lot of time on the road, and every Monday morning at a fixed time, he had a two hour conference call with his virtual workforce. He chose this approach after having tried individual phone calls. The individual calls, because of the size of his organization, were consuming too much of his time. Each call, which under ordinary circumstances might take five minutes, could easily take twenty or more minutes. The bulk of the time was spent bringing the people up to date on what was going on at the home base, something they felt they needed in order to maintain a sense of connectedness to the organization. The people wanted to know how Frank was doing, whether Suzie had her baby yet, and so on. After much soul searching, he chose the conference call process because they could all maintain contact not only with the manager but could become aware of what each of the others was doing. We also contacted Dr. Kimberly Klint, Vice-President of VirtualTEAMWORKS.com. Her company offers consulting services to people who want to begin working virtually, or who already do but need help. We asked her what she had found was the biggest first thing is you have to do to get it right. She said, The biggest mistakes corporations (new to virtual teamworking) make are: Thinking technology will solve problems instead of creating more; thinking people will effectively communicate via technology; and thinking that people and technology will never interact in any way. It is a social-design issue. She said the organization has to overcome specific barriers, which are organizational, geographical, temporal, lingual, cultural, ability, and access. We deduce from these cases that successful virtual operations requires several things: 1. Establishing effective communications is critical to success. This includes communication with both the virtual working staff and the customer. 2. Cultural differences must be part of the management plan. Customers and virtual team members operate where they are, not where you are. The indigenous culture is the one in which they function. Understanding the impact of that culture, possibly including the language, on how people work and interface with you is critically important. 3. Taking geography and time zones into account when setting up communications with the virtual work force is critical. Each of the managers we 11

12 interviewed took a different approach, but each recognized the criticality of immediate communication as well time phased communication, such as using Assuring the work force has the requisite skill in different technologies needed to operate in a virtual manner without sacrificing effectiveness or efficiency is essential. 5. Top management must plan for significant time spent in communicating, including travel, something often grossly underestimated. Top managers spend considerable time traveling and communicating, and this is just part of the price for establishing a virtual team. 6. Specific criteria must be established for hiring virtual staff. Personnel tasking is different for a virtual organization. People have to be able and willing to work productively without continuous in-place supervision. People whom managers usually refer to as high-maintenance employees would not be good candidates for virtual workers. You must be able to assign tasks and expect the work will be done effectively and efficiently. Initiative is an important trait in the virtual worker. 7. Management must budget with virtual operations in mind. Budgeting must take into account the need for additional management travel. The nature of the business may be such that the virtual workers operate out of their home and interface with the customer at the customer site. If so, that has to be accounted for in the budget, both for personal expenses of the employees that can be charged back to the business, as well as infrastructure that might have to be set up in their home. If periodic meetings with virtual workers at a central location are necessary, those travel expenses must be budgeted. References Boynton, A. C., & Zmud, R. W. (1984). An assessment of critical success factors. Sloan Management Review, 26 (Summer), Bullen, C. V., & Rockart, J. F. (1981, June). A primer on critical success factors (CISR WP No. 69). MIT Sloan School of Management. Dobbins, J. H., and Donnelly, R. G. (2004). Critical Success Factor Analysis for DoD Program Management, Proceedings, International Association of Management of Technology, 2004 International Conference, Washington, D.C., April. Dobbins, J. H., and Donnelly, R. G. (2004). Critical Success Factor Analysis for Technology Acquisition Program Management, Proceedings, Project Management Institute Research Conference, London, July. Dobbins, J. H. (2000). On a generalized CSF Process model for critical success factor identification and analysis for technology acquisition program management, Unpublished doctoral dissertation, George Washington University. Rockart, John F., Chief Executives Define Their Own Data Needs, Harvard Business Review, March-April 1979, Vol 57, No 2, pp Sum, C.-C., & Ang, J. S. K. (1997). Contextual elements of critical success factors in MRP. Production and Inventory Management Journal (Third Quarter),