The Contextualization of Project Management Practice and Best Practice

The Contextualization of Project Management Practice and Best Practice Claude Besner PhD, University of Quebec at Montreal Brian Hobbs PhD, University of Quebec at Montreal Abstract This research aims to provide an accurate, comprehensive, detailed and reliable portrait of project management as it is actually practices. It is based on the rigorous analysis of a large representative sample of responses from practitioners. The research investigates what practitioners do, how what they do varies between contexts and how variation in practice affects performance. Keywords: practice, context, performance. The Primary Objective Claude Besner and Brian Hobbs had been professors active in the Master s Programme in Project Management at the University of Quebec at Montreal s School of Management for several years prior to the initiation of this research programme. The research programme had several objectives, which are related 1) to the state of the literature in project management in general, 2) the state of project management research, and 3) the state of the knowledge upon which our teaching in project management is based. Many of the objectives are common to both researchers, but some are more salient for one or the other researcher. Claude Besner is responsible for the courses in the master s programme related to project planning and control. The majority the management practices and the tools and techniques that are specific to project management are used in the planning and controlling processes. Claude had been a project manager for several years prior to doing his PhD and becoming a professor. He knew that there was and still is a large gap between project management as it is presented in the literature and the reality of practice. Because the literature did not provide an accurate, comprehensive, detailed and reliable portrait of project management practice, the primary objective of this research program was to provide such a portrait. In 2002, Brian Hobbs finished a three-year term as a member of PMI s Members Advisory Group on Standards. He had become familiar with the processes used by PMI to produce and to revise its standards. He realized that the process was impervious to research results and that the research results that were available at the time were not particularly useful for the revision of standards that describe generally accepted practice. The expression generally accepted was used in the PMBoK Guide 2000 Edition, but was changed to generally recognized as good practice in the 2004 and later editions. What was missing in the literature was a comprehensible, detailed and reliable portrait of project management practice. Ten years ago, the literature in project management was very normative; it described how project management should be practiced. Most of the literature was opinion-based; authors expressed what they thought practitioners should do. There was little information based on reliable and accessible data. The literature was also quite fragmented; each publication typically addressed some aspects of project management practice, for example, risk management or scheduling, but did not provide a comprehensive coverage of the great variety of practices from all the different processes included in the field of project management practice. There has been a significant increase in the production of research in the past decade. However, each piece of research tends to cover a small portion of the entire field of project management practice. In addition, there has been a tendency for a greater focus on qualitative studies based on a limited number of cases.the few studies that have covered large portions of the field tend

to stay at a high level, for example the level of general processes such as planning or of knowledge areas and tend to have small samples, which are unable to provide comprehensive, detailed and reliable portraits of practice. Objective 1: to produce a comprehensible, detailed and reliable portrait of project management practice The Approach The goal was to reliably document what practitioners actually do when they manage projects. The study aimed to describe actual behaviour rather than rationalized interpretations of what they are doing and to do it with a large sample. The only economically viable option for data collection was a survey. The strategy that was used to access actual behaviour was to ask how extensively practitioners actually use the tools and techniques that are specific to project management. For example, rather than asking whether a respondent monitors progress, the study went down to a more detailed level and ask to what extent the respondent used the following tools and techniques: progress reports, stage-gate reviews, project scorecard/dashboard, monitoring critical success factors, trend reports and earned value. The extent of use of these six tools for monitoring progress was used as measure of the practitioners behaviour. Going to the level of the specific tool or technique, produces information that reliably describes actual behaviour for three important reasons. First, enquiring as to the extent to which a practitioner actually uses a tool or technique is a request to describe his/her actual behaviour. Second, this type of question reduces the problem of social desirability; monitoring progress is a socially desirable aspect of project management practice, while using each of these six techniques is less so. Third, the information is less subject to bias introduced by respondent s interpretation of he/she actions; responding to a question on the extent to which a practitioner monitors progress requires the person to interpret his/her behaviour and the intention he/she had when acting, while responding to questions regarding the extent of use of specific tools and techniques is more objective. As discussed below, the survey instrument contained a second question for each tool or technique enquiring at to the practitioner s belief concerning the potential effect of more or better use of the same tools; since this additional question was adjacent to the one concerning the actual level of use, its presence further reduced the desirability bias in measuring actual use. The field of project management practice is vast. Going to the level of specific tools and techniques, while being comprehensive meant that a large number of tools and techniques must be investigated; 108 tools and techniques were identified from the professional and scientific literature as well as extensive pretesting for concept precision and comprehension. Contextual Variation in Practice In about the year 2000, interest increased in how project management practice varied on different types of projects and in different contexts. This concern also inspired the research on project management practices, tools and techniques. In order to be able to investigate contextual variation questions on respondent demographics, project characteristics and the organisational context were included in the survey instrument. The large sample of experienced professionals responses gathered by this research allows the analysis of a wide array of contexts. Objective 2: to identify the extent and the nature of contextual variations in project management practice Patterns in Practice Tools and techniques are not used independently; they are used in groups or toolsets. The project management community recognises this by grouping practices into sets such as risk management, cost management, contact management, etc. The PMBoK Guide refers to knowledge areas. To the best of our knowledge, no attempt had been made previously to empirically validate groupings of practices or to identify alternate groupings. Exploratory principal

component analysis was used to identify groups of practices or toolsets. Nineteen such toolsets were identified Objective 3: to verify that tools and techniques are used in groups Objective 4: to identify groups of tools and techniques Common Practice and Best Practice The first four objectives refer to project management as it is actually practiced. This is different from the normative literature in general and standards in particular, which describe practice that is good, best or exemplary. In this research programme a clear distinction was maintained between descriptive and normative statements. The descriptions are based on the questions related to extent to use of specific tools and techniques. The normative statements in the literature are based either implicitly or explicitly on the idea that a practice is positively related to performance. In this research programme, the normative aspect was addressed in two ways. First, as previously mentioned, for each of the 108 tools and techniques a second question was posed In your opinion, more extensive or better use of this tool or technique would improve project performance. This question relates to practitioners belief in the unrealised potential of each tool or technique. Second, a construct labeled performing maturity was identified during the analysis of data that was used as a measure of performance. It emerged from the data as a strong principal component of the context variable set. The construct is composed of four variables: project management maturity, rate of project success in this organisation compared to other organisations in the same sector, organisational support for the use of project management tools and techniques and availability of competent personnel. This construct was used as the dependent variable in regressions to determine the practices for which variation in the extent of use is associated with variations in performance. These practices are referred to as best practices. Contextual differences were also identified. Objective 5: to identify which practices have a significant impact on performance Objective 6: to identify which practices have a significant impact on performance in different contexts Contextual Archetypes The analysis of variance revealed that the large number of contextual variables showed no obvious patterns or clusters; the correlations among the variables were often significant, but weak in terms of the variance explained. This type of data is particularly well suited for treatment by clustering algorithms. Cluster analysis was used to identify five groups of respondents with similar projects in similar organisational contexts, referred to as contextual archetypes. Each contextual archetype is a multidimensional construct. Differences in project management practice among the contextual archetypes were explored. Objective 7: to identify patterns in contextual variation or contextual archetypes Objective 8: to identify differences in project management practice among contextual archetypes Data Collection in Three Phases The web-based survey instrument was developed and pretested in 2003. The questionnaire was published. The initial version of the questionnaire did not contain the question related to performance i.e. the rate of project success in this organisation compared to other organisations in the same sector. Therefore, its data and the results of analysis were primarily descriptive. The sample of 963 usable responses from the first phase collected in 2004 supported the attainment of the first objective of providing a comprehensible, detailed and reliable portrait of project

management practice. It also supported the attainment of the second objective to identify the extent and the nature of contextual variations in project management practice. Following the analysis of the data from phase 1, the questionnaire was modified only slightly; a very small number of changes were made to the list of tools and techniques and the performance question was added in order to be able to explore best practices. In 2007, 1,029 new responses were collected and an additional 347 in 2009, for a total of 2,339 usable responses. The larger sample supported additional analysis, particularly the identification of toolsets. For the objectives 5 and 6 related to best practices and performance only the 1,296 responses from phases 2 and 3 could be used. Thorough analysis showed that no important differences exist in the measures of the extent of use of tools and techniques or the nature of contextual variations in project management practice between the phases. Research Results Achieved and their Relevance Objective 1: to produce a comprehensible, detailed and reliable portrait of project management practice This objective was accomplished with the data from phase 1 and enriched with the data from phases 2 and 3. The results are descriptive; they provide a portrait of project management as it is actually practices. The portraits of practice were reported for the 108 individual tools and techniques and for the nineteen groups of practices. Appendix 2 in the research monograph provides a one-page summary of the results for each of the 108 tools and techniques and each of the 19 toolsets. The extent of use of the different tools and techniques were compared with each other. The results do not provide normative guidance as to how project management should be practiced, except to the extent that a practitioner, an educator, an author or a researcher can compare practices in their specific context with those found in the large and diverse sample. For those that are already familiar with the field of project management providing information on the relative importance of different tools and techniques in terms of their extent of use can help determine priorities for the content of training and education activities, documents such as books, articles and standards, or the objects to include in research protocols. For those that are not familiar with the field, the relative extent of use can help in establishing priorities for the development of competencies. The relevance for standards is somewhat obvious because standards are based to a largely extent on prevalent practices. The case of the PMI s PMBoK Guide is of particular interest. Despite the presentation of the results at the PMI Research Conference, publication in the Project Management Journal and publication of the research monograph by PMI, these results have not influenced the 2008 or 2013 editions. However, with PMI s Researched Informed Standards Initiative this may change. Objective 2: to identify the extent and the nature of contextual variations in project management practice This objective was accomplished with the data from phase 1 and enriched with the data from phases 2 and 3. The results show that there are significant contextual variations, but project management practice is also quite generic; very similar patterns in extent of use of practices are found in all the different contexts investigated. The confirmation that practice is largely generic is important particularly for educators and for authors; particularly those that write standards. It also supports the mobility of persons from one context to another because they will find similarities in practice in different contexts. The research also identified significant contextual variations in practice. This is important because it facilitates the adaptation of practice to context. This can help individual practitioners and organisations adapt their practice to trends found in similar contexts. It can help authors and educators to adapt their material to particular audiences. It can help researchers identify the practicalities of projects in contexts they wish to investigate and to adapt their research questions

and protocols. Standards say that practices must be adapted to the context, but do not say what adaptations to carry out. If solid research-based information were available, general standards could address this issue; however, it seems unlikely that this will be done in the near future. The results include information on variation of practice for a large number of contextual variables. However, two are of particular interest because they show patterns of variation in which some practices are used more and some are used less; these are the type of deliverable and the nature of the project customer, internal to the organisation or external. The types of deliverables investigated were IT and telecommunications, software development, engineering and construction, and financial and business services. Grouping by type of deliverable produces more homogeneous groupings than by industry because an IT project in a construction company has more in common with other IT projects than with other projects in the construction company. Objective 3: to verify that tools and techniques are used in groups, and objective 4: to identify groups of tools and techniques These objectives were accomplished with the data from the entire sample. Exploratory principal component analysis was used to group 95 of the 108 individual practices into nineteen toolsets. This has important practical consequences for practitioners, educators and researchers. It also greatly facilitated the analysis of the data in this research programme. It greatly simplifies any treatment of practice because it provides a small number of empirically valid groups. It also has theoretical and methodological consequences in that it identifies an underlying structure within the myriad of practices. The groupings show significant face validity. Several are very similar to the well-known groupings, while others are conceptually original and different from the generally accepted groupings; for example, two types of use of project management software, basic and advance functionality. The former is very common; the other more advanced is used only very rarely. Note that methodologically this research examined functionalities rather than specific software products because this provides a better indication of what practitioners actually do with the software. Objective 5: to identify which practices have a significant impact on performance, and objective 6: to identify which practices have a significant impact on performance in different contexts As stated above, two sets of questions and analyses were used in the pursuit of these objectives; first, differences in the measure of practitioners belief in the unrealised potential of each tool or technique and each toolset, and second, the identification of best practices using regression analysis on performing maturity with toolset use as the independent variables. The simultaneous examination of the extent of use and the unrealised potential of each tool or technique revealed that extent of use and unrealized potential are not necessarily related. The analysis identified four extreme cases. 1) super tools are used extensively yet are seen as having significant unrealized potential; 2) at the other extreme, discredited tools: are rarely used and practitioners do not believe that they would improve performance 3) adequately used tools and techniques are used extensively, but the practitioners see no value in more extensive or better use; 4) underutilised tools and techniques are not used extensively, but practitioners believe that they have the potential to contribute to improved performance. An examination of the tools and techniques reveals that both the super tools and adequately used tools are prominent in the project management literature, as would be expected. An examination of the discredited tools also reveals that several tools in this group occupy a prominent place in the literature, particularly in textbooks and standards. There seems to be a lobby promoting these tools among those that write textbooks and standards, which seem to be a misalignment with what practitioners are saying about discredited tools. This is of concern because the content of standards is based in principle on a consensus among practitioners. This lobby could be challenged based on the research results produced here. The final group of underutilised tools is quite varied, but many require extensive organisational support to be implemented; organisational support was clearly associated with extent of use in all cases. The examination of practitioner

beliefs has revealed some interesting observations and challenges some aspects of the field of project management. The second less subjective way of examining the effects of practice on performance used regressions on performing maturity. In the overall sample, the regression model includes most of the practices each with a similar contribution to the explanation of performance. This indicates that there are not any silver bullets, i.e. single groups of practices that explain large portions of the variation in performance. This result supports the rich and varied content of standards. Many regressions were performed on subsamples from different contexts. Interestingly, the vast majority of the regression models contained fewer toolsets than with the overall sample, but explained similar proportions of the variations in performance. In addition, different toolsets were included in the regression models in different contexts. This indicates that best practices vary by context and that best practices when adapted to the context contribute more to the explanation of performance. This confirms that it is important to adopt practice to context. However, the adaptation is not a simple matter. The results produced here are a step in this direction, but much more needs to be done. Objective 7: to identify patterns in contextual variation or contextual archetypes and objective 8: to identify differences in project management practice among contextual archetypes A cluster analysis identified five contextual archetypes. The four archetypes from the private sector were differentiated primarily along two axes. One axis opposed high scores for performing maturity and greater use of project management practices on well-defined projects with low scores for performing maturity and less use of project management practices on ill-defined projects. The other opposed smaller internal projects in large organisations with larger external projects in smaller organisations. The public sector archetype was associated low scores for performing maturity and less use of project management practices on ill-defined projects and with smaller projects in larger organisations. A pattern of significant differences in the extent of use of specific toolsets was also identified. The identification of these contextual archetypes illustrates the complexity of the interaction effects of contextual variables and their effects on project management practice. This calls into question simplistic statements regarding the nature of the effects of a single contextual characteristic. In real situations that resemble one of the five archetypes these results provide a realistic and reliable point of comparison and can lead to significant improvements in understanding. In real situations that do not resemble one of the archetypes these results are less useful. Fortunately, a large number of real situations resemble one of the archetypes. The improved understanding can be a guide for practitioners and for researchers alike. Much of the theoretical development in the field of project management aims to provide conceptualizations of practice. Thus, the much more parsimonious description of practice based on clusters of practices has the potential to contribute to the development of theory. The fact that the clusters were derived inductively from practice implies that conceptual developments that employ these clusters will be anchored in the reality of practice, thus providing a link between conceptual developments and the empirical reality of practice. Overall the research programme has attained all its objectives. The current interest in contextual effects and in solid empirical results to inform practice and standards has created a context in which the results of this research can potentially have a significant impact. According to Google Scholar the journal articles from this research have been cited more than 250 times Originality of the research The primary objective of this research was to produce a comprehensible, detailed and reliable portrait of project management practice. This objective leads the research in two seemingly irreconcilable directions. On the one hand, the objective is to report on the real behavior of practitioners, which points to a qualitative approach. On the other hand, the objective of being comprehensive in such a vast and varied field and to produce results that are reliable and

generalizable points to the need to measure many variables and to have a very large sample. This seemingly impossible task was achieved by investigating to what extent practitioners actually use specific tools and techniques that are easily identifiable and well-know and thus reveal what they are actually doing. As this is the only research of this type and this scale in our field, many of the results are original. The monograph on which this summary is based is available for free download to members at http://www.pmi.org/learning/research-completedresearch/contextualization%20of%20project%20management%20practice%20and %20best%20practice.aspx or can be purchased at the PMI Marketplace.