Assessing Project Management Maturity
0 likes135 views
This study developed a project management maturity model and assessment methodology to evaluate the maturity level of project management processes in different organizations and industries. The methodology involves a 148-question survey that assesses maturity across 8 knowledge areas and 6 project phases. The survey was used to benchmark 38 companies across 4 industries. On average, companies scored a 3.26 on a 1 to 5 scale. Engineering/construction scored highest while information systems scored lowest. The study found that risk management and project execution scored lowest in maturity, while cost management and project planning scored highest. The methodology provides a way for organizations to identify opportunities to improve project management processes and assess potential returns from those improvements.
Assessing Project Management Maturity
- 1. 32 Project Management Journal March 2000 This paper summarizes the results of research con- ducted by investigators from the University of Cal- ifornia at Berkeley, with support and sponsorship from the Project Management Institute (PMI®) Educational Foundation and PMI’s Northern California Chapter. The project management maturity assessment for all companies averaged 3.26 on a relative scale of 1 (lowest) to 5 (highest). Overall, the engineering/con- struction industry had the highest score, and the infor- mation systems application area had the lowest. Com- pany scores varied considerably, so industrywide composite averages are somewhat misleading. Even in- dustries and companies with the highest scores have substantial opportunity to improve. Risk management and the project execution phase are areas of low matu- rity, and cost management and the project-planning phase are areas of high maturity. Many organizations are projectizing their opera- tions and processes to plan, manage, and complete projects more successfully. A driving reason for such projectizing is the growing pressure on managers to in- tegrate, plan, and control schedule-intensive and one- of-a-kind endeavors in order to improve overall organi- zational performance. However, it is fair to say that many organizations are uncertain, perplexed, and even misdirected about the status of current applications of project manage- ment. Moreover, the financial investment in project management tools, practices, and processes is often seen as quite difficult to justify. One of the reasons for this complication is that there are no universally accepted methodologies or well-defined processes for impartially measuring project management practices in any one organization or across different industries. As an example, A Guide to the Project Management Body of Knowledge (PMBOK® Guide), while an industry-accepted, authoritative source of project management information, lacks this level of detail. The upshot is that many companies are having major problems implementing or improving their project management processes. This is especially im- portant to managers who must justify in a rigorous manner the funding of implementing project manage- ment processes and practices in their organizations. Assessing Project Management Maturity C. William Ibbs, University of California, Department of Civil and Environmental Engineering, 213 McLaughlin Hall, Berkeley, California 94720 USA Young Hoon Kwak, The George Washington University, School of Business and Public Management, Department of Management Sci- ence, 2101 F. Street NW, Building AL, Washington, DC 20052 USA ■ Abstract The principal goal of this study was to determine the financial and organizational impacts of project management. This study began with the development of a project management maturity model and an analysis methodology to assess the maturity of project management processes. The project management maturity analysis methodology consists of 148 multiple-choice ques- tions that measure project management maturity, and cover 8 knowledge areas and 6 project phases. The maturity model and methodology were then applied by benchmarking 38 different companies and government agencies in 4 different industries. This assessment methodology provides solid and comparative studies on project management practices across industries and companies within an industry. It also provides a set of tools for organizations to use in identifying key areas of opportunity for improvement in project management. Keywords: project management maturity; maturity models; assessment; benchmarking ©2000 by the Project Management Institute — 2000, Vol. 31, No. 1, 32–43 — 8756–9728/00/$5.00 per article + $0.50 per page
- 2. Earlier research identified some of the benefits of project management practices and processes, albeit usually in qualitative and anecdotal fashion. Most of these studies are based on quick surveys of industrial professionals and academics. Such work tends to focus on the qualitative and narrative evaluation of prior experiences on project man- agement tools and practices, rather than be factually mea- sured (Al-Sedairy, 1994; Boznak, 1988; Bu-Bushait, 1989; Construction Industry Institute, 1990; Cleland, 1993; Deutsch, 1991; Gross & Price, 1990; Kwak, Clark, Grilo, Betts, & Ibbs, 1995; Ziomek & Meneghin, 1984). Other studies mainly focused on the productivity impact of using project-driven organizational structures that only deal with one aspect of project management practices (Donnelly & Kezsbom, 1993; Gobeli & Larson, 1986; Larson & Gobeli, 1989; Lundin & Soderholm, 1994; McCollum & Sherman, 1991; Might & Fischer, 1985). Dooley, Subra, and Anderson (1998) investi- gated the relevance of maturity and best practices to new product development. None of these prior studies examined an organiza- tion’s overall project management processes or prac- tices in a way that permitted evaluation of the business case of project management adaptation. This is a se- rious shortcoming and obstacle to the progress of the project management profession. To address the identified shortcoming, the authors embarked on research with the goal of determining the financial and organizational impacts of project manage- ment. In particular, the Berkeley research team with its industry advisory group concluded that the profession needs an analysis methodology that can be used reliably to assess quantitatively organizations’ current level of project management practices and processes. Ideally, this methodology would provide solid and comparative studies on project management practices across indus- tries and companies within an industry. It would also support any organization that wanted to evaluate the business case for adopting project management. A key objective of the investigation became to de- velop and provide managers with a procedure for mea- suring project management processes and the value of incorporating those processes in their business prac- tices. This analysis would assist managers in assessing which, if any, project management practices would be best for organizations. Also, it would assist companies in promoting improved project management practice. Research Methodology Research Steps. The following steps represent the ac- tion plan taken in this research: 1. Examine past and current studies related to project management and, as appropriate, build on that prior re- search through a literature review. 2. Develop a five-level project management process maturity model to position an organization’s current project management level. 3. Develop a comprehensive project management process maturity assessment methodology to evaluate an organization’s current project management level and actual project performance. 4. Select target organizations/industries to investigate. 5. Measure an organization’s project management level and actual project performance with a three-part project management process maturity assessment ques- tionnaire. 6. Analyze collected information to evaluate and benchmark an organization’s project management processes and practices level using specific criteria. 7. Identify organizational strengths and weaknesses in terms of project management practices and processes. 8. Search for any “success drivers” by testing for cor- relations between an organization’s project management level and actual project cost and schedule performance. 9. Develop a calculation procedure to estimate an organization’s order-of-magnitude project management return on investment (PM/ROI). That is, develop a tech- nique that can be used to estimate, in approximate way, project management’s financial return for companies. 10. Provide suggestions and recommendations for continuous project management improvement. One of the advantages of this particular process is that it can be repeated on a regular basis to assess and compare an organization’s project management March 2000 Project Management Journal 33 ■ About the Authors C. William Ibbs is professor and group leader of the Construction Management program at the University of California at Berkeley. His consulting firm provides a wide range of services in the project management field, including benchmarking and mea- suring corporate PM/ROI. Readers can obtain more information at www.ce.berkeley.edu/~ibbs. Young-Hoon Kwak, Ph.D., is currently an assistant professor for the Project Man- agement Program at The George Wash- ington University (GWU). He received his M.S. and Ph.D. in project management at the University of California at Berkeley. He also spent a year at Massachusetts Institute of Technology as a postdoctoral scholar and was a visiting professor at Florida International University be- fore joining GWU. His main area of interest includes project management maturity model, project manage- ment and control, construction management, and man- agement of technology.
- 3. practices and to continuously improve organizational project management effectiveness. A more thorough description and discussion of the five-level project management process maturity model and calculation procedure to estimate an organization’s PM/ROI (Re- search Steps 1, 8, and 9) are presented in a companion paper (Ibbs & Kwak, in press). Project Management Process Maturity Assess- ment Methodology. A project management process maturity assessment methodology was developed that uses statistical techniques to assess the maturity of project management processes and practices among these different industries. The purposes of the project management process maturity assessment methodology are to assess project management strengths and weak- nesses and to enable organizations to compare them- selves with similar organizations (a peer group). It also tests for correlations between an organization’s project management level and actual project performance. It al- lows us to derive and calculate an order-of-magnitude estimate of PM/ROI (Ibbs & Kwak, in press). Table 1 summarizes the characteristics of the project manage- ment process maturity assessment methodology. Before developing the project management process maturity assessment methodology, the authors reviewed and considered the Software Engineering Institute Ca- pability Maturity Model, which is quite popular among IS organizations. However, the authors felt that it was insufficient, because it is heavily slanted toward IS com- panies and it does not easily allow for measuring a ROI. Design of this project management process matu- rity assessment methodology made use of a compre- hensive and itemized questionnaire for data collection. Integrated Project Systems (IPS) of San Carlos, Cali- fornia, donated its benchmarking questionnaire for this task, and the University of California researchers amended it to meet the specific needs of this particular study. As a result, a three-section project management process maturity assessment questionnaire was devel- oped to measure any organization’s project manage- ment maturity level. This procedure is used to evaluate an organization’s current project management maturity status related to the five-level project management process maturity model. Three objectives are achieved using this questionnaire. ■ Organizations’ project management maturity can be benchmarked in various terms. Project management strengths, weaknesses, and improvements are identified by eight project management knowledge areas and six project management phases. ■ PM/ROI can be calculated on an order-of-magnitude basis using statistical relationships between project management maturity and project performance (Ibbs & Kwak, in press). ■ This questionnaire can help improve an organiza- tion’s project management processes and practices in very specific ways. The questionnaire is clustered into three major sec- tions: General Organizational Information (Section 1), Organizational Project Management Process Maturity Assessment (Section 2), and Actual Project Performance 34 Project Management Journal March 2000 Table 1. Characteristics of the Project Management Process Maturity Model Assessment Methodology Targeted Industries Variables/ Factors Model/ Procedure Research Methodology Contribution Project Management Process Maturity Assessment Methodology EC IMM IS HTM Eight project management knowledge areas Six project management processes Schedule and cost indexes (SI, CI) Percentage of project management spending Order-of- magnitude PM/ROI estimates Five-level project management process maturity model Project management maturity vs. CI&SI model PM/ROI calculation procedure Three-section project management process maturity assessment questionnaire General statistical analysis Correlation analysis Regression analysis Better understanding of the financial and organizational benefits of using project management tools and practices in organizations
- 4. Assessment (Section 3). Participating companies are coded to preserve confidentiality of specific responses. Organizational project management maturity levels are defined on a 1-to-5 point Likert scale, with 1 being the lowest level of project management maturity and 5 being the highest level. The scores for each deliverable are averaged to determine one score for each project management knowledge area and project management process. The scores are then combined and analyzed to calculate an overall project management maturity level. The scores are then logged into a database and com- pared against other organizations and industries. There are 148 multiple-choice questions in Section 2 of the questionnaire. The study team used A Guide to the Project Management Body of Knowledge Exposure Draft (Project Management Institute Standards Committee, 1994) as a primary reference. The study team also de- cided to add an additional project management process called the Project-Driven Organization Environment, which supports sustaining project-driven organizations. The numbers in the cells of Table 2 show how the 148 multiple-choice questions in Section 2 were distributed across the six project management phases and eight project management knowledge areas. The entire ques- tionnaire is too lengthy to publish. An example of one of the questions is shown in Table 3. To calculate the overall project management process maturity, the scores of all 148 questions were averaged. To calculate the specific maturity of the eight project management knowledge areas and the six project man- agement phases, the selected row or column of Table 2 was averaged. To calculate the project management ma- turity of an industry’s specific project management knowledge areas, each score of the selected areas within the same industry were averaged. Data Collection Technique. Data collection was one of the major challenges in this study. The main reason was that project management tools and practices were still rela- tively new to many organizations that were studied, and it was difficult to recruit organizations to participate in this research. Invitation letters were sent to potential partici- pants and announcements were made in project manage- ment-related magazines and meetings to promote the needs and importance of this study. Companies were then carefully chosen, since this study required a substantial amount of time and effort from participating organiza- tions. A pilot test was conducted with two selected organi- zations. The purpose of a pilot test was to confirm the va- lidity of the project management process maturity assessment methodology. The results of the pilot test con- firmed that the assessment methodology was appropriate and that the overall research methodology was valid. Par- ticular effort was made to collect objective, nonbiased, and representative project management processes and practices data. To ensure the confidentiality of the data, the authors signed and delivered to the companies a nondisclosure agreement that allows only “high-level data” in a forum such as this to be reported. Finally, all of the participating companies were anonymously coded. Also, all of the pro- prietary information, including cost and schedule of the participating organizations, was kept confidential and only used in the data-analysis process. Only the principal re- searchers knew the identity of the organizations. Targeted Industries. Four different industries and application areas were selected to collect project man- agement practices information. A total of 38 large in- ternational companies, including private and public sector organizations, participated in this study. The four industries were: engineering and construction (EC); in- formation management and movement (IMM), also known as telecommunications; information systems (IS), also known as software development; and hi-tech manufacturing (HTM). March 2000 Project Management Journal 35 Table 2. Distribution of the 148 Questions in the Organizational Project Management Process Maturity Assessment Questionnaire Initiating Project Management Processes and Knowledge Areas Planning Executing Controlling Closing Project-Driven Organization Environment Scope (30) Time (18) Cost (11) Quality (13) Human Resources (22) Communications (30) Risk (17) Procurement (7) Total (148) 6 1 2 1 2 5 1 1 19 7 2 2 2 4 3 1 1 22 8 12 2 3 5 12 7 1 50 3 1 1 3 2 7 6 1 24 3 1 1 3 2 2 1 1 14 3 1 3 1 7 1 1 2 19
- 5. Project management in the EC industry emphasizes development, engineering, procurement, and construc- tion processes for projects involving buildings and fa- cilities in the residential, commercial, and industrial sectors worldwide. The IMM industry is comprised of companies and individuals focused on technologies, primarily voice-data-video transmission and switching (e.g., analog, digital, terrestrial, satellite, microwave, fiber optic), services to the users of these deliverables, and network infrastructure providers. The IS industry or application area manages software development projects to deliver a quality product to the customer on time, within budget, and meeting perfor- mance specifications. The HTM industry is involved in the project planning, development, control, and execu- tion of hi-tech products, processes, and services. A hi- tech product may include computer hardware, semicon- ductors, and related equipment and accessories. Research Limitations. There were still some con- cerns about collecting information on project manage- ment tools and practices. The primary concern was the accuracy of information that organizations provided. Ideally, the project management level should be mea- sured by visiting each participating organization and conducting an in-depth interview with each organiza- tion’s project managers to determine current project management practices. However, this was impossible because of limited time and resources. Instead, the questionnaires were distributed to the organization’s project managers to complete. Generally, people tend to score higher when they self-critique themselves. Four steps were taken to collect more accurate data. First, there was a thorough prescreening process to choose participating companies. Companies were care- fully recruited and selected were those willing to share their current project management knowledge and per- formance data. Second, the authors stressed to each company’s re- spondent how important it was to be candid. A major point of persuasion was that it was in the company’s own best interest to respond honestly, because that was how it could best learn about its practices vis-a-vis other firms. Third, any unclear questions or concepts were dis- cussed with the researchers at the organization’s request. Effort had been made to easily understand all 148 ques- tions in the questionnaire. Also, a glossary section was provided to explain and clarify any unfamiliar concepts or terminology used in the questionnaire. Last, the Berkeley researchers reviewed the completed questionnaires with a number of the companies, typically with in-depth telephone interviews. Although these four actions cannot guarantee perfection, the research team be- lieves that reasonably accurate information was collected. Another issue was that some questions required several project managers’ knowledge to answer. For ex- ample, suggestions were made in the questionnaire to encourage asking several people in the organization to respond jointly, if possible. This enabled collection of collaborative and self-critiquing information, which the firms generally found to be a valuable exercise. Typ- ically, six-to-eight person-hours were required from each company to compile the requested information. Assessment Results Organizational Demographics. This study required considerably more time than originally planned— nearly two years. The main reasons for this were that many companies did not see the true value in this re- search, or they were reluctant to share the details of their project management practices. Some were simply embarrassed by their current project management prac- tices and processes. Others believed they that had something special, and they did not want to share with other companies. As mentioned, a total of 38 companies participated in this study: 15 EC companies, 10 IMM companies, 10 IS companies, and three HTM companies. Organizations were first classified by their size. Forty- eight percent of the organizations employed between 36 Project Management Journal March 2000 Table 3. Sample Question From Original Questionnaire Question #56: How is a Schedule’s Critical Path Identified? No critical path calculation done. Each subproject identifies critical tasks independently and sets work priorities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Critical path based on committed milestone dates. No CPM calculation performed, or CPM used on individual subprojects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Key critical tasks identified through nonquantifiable means, and used to drive the critical path calculation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Critical path calculated through integrated schedule, but only key milestone dates communicated back to subprojects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . All critical tasks identified and indicated in each individual subproject schedule. Critical path determined through integrated schedule . . . . . . . . . . . . . . . . . . . . . . 1 2 3 4 5
- 6. 1–100 people, and 52% of the companies had more than 100 employees. Organizations were then divided by years of experi- ence in project management practices. Overall, the an- swers ranged from 1 to 50 years, with averages of 15.4 years for EC, 10.7 years for HTM, 8 years for IS, and 6.8 years for IMM. The average for all companies was 10.7 years. Sixty-one percent of the organizations had fewer than 10 years of experience with project management. Clearly, companies have only recently begun to imple- ment project management, and it may still be too early to recognize all of its benefits. Annualized Cost of Project Management Ser- vices. Another key question that this research wanted to explore was the cost of project management services as a percentage of total project cost. These costs were computed as a percentage of a company or depart- ment’s project-based revenues or sales. A list of items that are classified as a project management-related ex- penditure was developed by the authors and provided to the company managers to help them more accurately and consistently estimate the average cost of project management services in the organization. There are in- stances, of course, where other items should be in- cluded. Admittedly, this question is difficult to answer precisely. Table 4 presents a checklist of project man- agement expenditures. The average cost of project management services was 6%. This figure is the sum of the costs listed in Table 4 divided by the final cost of the project. This result was actually higher than previously reported by earlier sources (Archibald & Villoria, 1967). Table 5 shows the result of 20 organizations that provided the informa- tion. Eighty percent of the companies responding to this question indicated that they spend less than 10% of total project cost for utilizing project management prac- tices. It was generally higher in EC companies: 9.3% versus 2% for all other types of companies. Assessing Overall Project Management Matu- rity. The overall average project management maturity of the companies and the industries ranged from a low of 3.06 for IS to a high of 3.36 for EC. Average overall project management maturity for all companies was 3.26. Since the rating scale ranged from 1 to 5, this means that there is still substantial opportunity for im- provement of project management practices in all four industries. Company EC2 scored the highest overall project management maturity (4.60), and IS1 scored the lowest (1.77). Table 6 and Figure 1 contain detailed information on the overall average project management maturity of all organizations. EC (3.36), HTM (3.34), and IMM (3.30) industries all have relatively high project management maturity compared to IS (3.06). However, the project management maturity of HTM should be deemphasized, because only three HTM companies participated in the study. Assessing the Project Management Knowledge Areas Maturity Levels. Also of interest are the details of how companies and industries compared. One way to examine the data is to investigate the project man- March 2000 Project Management Journal 37 Table 4. Project Management Expenditures Checklist Project Management Expenditures Checklist ❑ ❑ ❑ ❑ ❑ ❑ ❑ ❑ ❑ ❑ Professional, technical, and administrative personnel salaries Fringe benefits of project management personnel Training costs Travel and transportation costs Project management-related consulting services costs Hardware and software purchase and network installation, maintenance, and upgrade costs Moving, storage and rental costs Office, supply and equipment costs Utilities, telecommunications, mailing, and shipping costs Professional associates and society membership costs Table 5. Average Total Cost of Project Management Services Project Management Cost as a Percent of Total Project Cost Number of Organizations Percentage 0 to 1% 1.1 to 3% 3.1 to 6% 6.1 to 10% Over 10% Total 3 5 5 3 4 20 15% 25% 25% 15% 20% 100%
- 7. agement maturity levels according to each of the eight project management knowledge areas. Table 7 contains those details. In general, IS has the lowest project management maturity score, and EC and HTM have the highest. Each of the project management knowledge areas assessment results are briefly discussed. Project Scope Management. There are no significant differences in scope management maturity among the four industries and application areas. EC is the highest (3.62), and IS is the lowest (3.25). EC’s relatively high score indicates that this industry puts more emphasis on scope management than other industries and applica- tion areas. IS also has the most variability (standard de- viation = 0.82). According to the data, the IS industry is still behind in scope management practices. Project Time Management. Again, EC scored the highest, and IS scored the lowest. Conversely, EC’s time management maturity has the smallest standard devia- tion, and IS has the largest. A favorable time manage- ment index should help organizations deliver projects closer to the delivery dates promised to the client. Of course, a project’s scope is sometimes altered to allow its schedule to be met, so both scope and schedule management need to coordinated. Project Cost Management. The cost management area has the highest project management maturity levels, in which HTM received the highest score. The rise of new management practices, such as lean production and ac- tivity-based costing, may have contributed to this high maturity level. It seems that all four industries and ap- plication areas are very concerned about cost. Cost management maturity of IS and IMM is rela- tively low compared to EC and HTM. One possible in- terpretation of this result is that cost management may not be as important in these industries as in other project management knowledge areas. This may well be because an IS or IMM company makes money not from finishing within budget, but rather by completing the project quickly and selling it to potential customers. Construction contractors, on the other hand, generally earn a profit by finishing a project for less money than the budget. That means that cost management has sub- stantially different perspectives and emphases for dif- ferent industries. Project Quality Management. The quality manage- ment’s project management maturity level had the highest standard deviation (SD) of all project manage- ment knowledge areas, indicating a very high dispersion among these companies. EC and IS organizations con- sistently had low levels of quality management matu- rity. HTM companies, which scored the highest in quality maturity, also scored highest in SD. The SD value implies that IMM (SD = 1.10) is relatively consis- tent compared to HTM (SD = 2.00), which shows in- consistency. Application of quality assurance, quality control, and total quality management principles might increase the quality management maturity over time. Project Human Resource Management. On the whole, human resource (HR) management (and risk manage- ment) had the lowest SD of all eight knowledge areas. EC companies were extremely consistent in terms of their HR maturity. Industries should put more em- phasis on managing and dealing with project personnel and teams, because these people are intensively in- volved from the beginning to the end of the entire project (Taspinar, 1998). Project Communications Management. The communi- cations management maturity for the companies was very high, ranking second only to cost management among the eight project management knowledge areas. EC scored the highest communications maturity with tight consistency among the four industries. Since most EC projects are carried out by many subcontractors and suppliers, communication flow seems to be better es- tablished than other industries. Project Risk Management. Risk management’s project management maturity level was the lowest among all eight knowledge areas. Risk management was the only knowledge area where overall project management ma- turity rating was less than 3. Consequently, companies should put more effort into the risk management area. This area has a potential for substantial improvement. Project Procurement Management. Both EC and HTM scored the highest among the four industries. This is understandable since supplying and buying materials and services are such major parts of their businesses. On the other hand, IMM and IS scored low on procure- ment management. Assessing the Project Management Phases. This research also investigated the details of how companies, industries, and application areas compared with the ma- turity of six project management phases. This analysis 38 Project Management Journal March 2000 Table 6. Overall Average Project Management Maturity (Four Industries) EC IMM IS HTM All 38 Companies Project Management Maturity Standard Deviation 3.36 0.66 3.30 .077 3.06 0.88 3.34 0.87 3.26 0.74
- 8. reveals which of the project management phases are mature and which are less mature (see Table 8). Again, IS had the lowest project management matu- rity rating, and EC and HTM had the highest. Note that HTM only had three companies; therefore the result should be deemphasized. The details are summarized in the following sections. Initiating Phase. Initiating processes are those very early steps—usually conducted by a project sponsor— to determine the financial feasibility of a project and to start defining project scope. For this process category, HTM had the highest maturity index, and EC scored the lowest. Conversely, they had the lowest and highest SDs, respectively. One interpretation of this is that HTM and IS are very sensitive and proactive to market needs, whereas the competitive bid nature of EC encourages a more reactive response to client needs, especially during the initiating processes. Planning Phase. Planning processes scored the highest among the six project management processes March 2000 Project Management Journal 39 Figure 1. Overall Average Project Management Maturity (All 38 Organizations) Project Management Maturity 5.0 4.5 4.0 3.5 3.0 2.5 2.0 1.5 E C 1 E C 2 E C 3 E C 4 E C 5 E C 6 E C 7 E C 8 E C 9 E C 1 0 E C 1 1 E C 1 2 E C 1 3 E C 1 4 E C 1 5 I M M 1 I M M 2 I M M 3 I M M 4 I M M 5 I M M 6 I M M 7 I M M 8 I M M 9 I M M 1 0 I S 1 I S 2 I S 3 I S 4 I S 5 I S 6 I S 7 I S 8 I S 9 I S 1 0 H T M 1 H T M 2 H T M 3 4.26 4.60 3.33 2.26 2.55 4.02 3.69 3.16 2.87 2.77 3.68 3.74 2.93 3.66 2.89 3.07 4.00 3.66 2.49 3.95 2.18 3.88 4.33 2.39 3.01 1.77 2.10 3.73 2.45 3.36 4.00 3.08 2.59 4.57 2.91 2.51 4.26 3.26 Table 7. Eight Knowledge Areas of Project Management Maturity EC Project Management Knowledge Areas IMM IS HTM All 38 Companies Scope Time Cost Quality Human Resources Communications Risk Procurement Overall Project Management Knowledge Areas Maturity 3.52 3.55 3.74 2.91 3.18 3.53 2.93 3.33 3.34 3.45 3.41 3.22 3.22 3.20 3.53 2.87 3.01 3.24 3.25 3.03 3.20 2.88 2.93 3.21 2.75 2.91 3.02 3.37 3.50 3.97 3.26 3.18 3.48 2.76 3.33 3.36 3.42 3.37 3.48 3.06 3.12 3.44 2.85 3.14 3.24
- 9. and had one of the lowest SDs. Companies evidently are convinced that this is important to the overall suc- cess of the project and thus devote substantial attention to this step. The four industries were relatively consis- tent for this process. Executing Phase. HTM and EC had a comparatively strong degree of project management maturity. EC had a high executing maturity, because generally construction projects are very complicated and completing one re- quires thorough planning to allocate appropriate re- sources. On the contrary, IS has a relatively low executing maturity. This is frequently exhibited by the numerous software development projects with late releases. Controlling Phase. Again, EC had a high project man- agement maturity for this specific process, and IS had a low maturity score. The standard deviation of IS was also quite high, and EC’s was low. This result is not sur- prising, since controlling processes have been a forte of the EC industry for a number of years. Closing Phase. On balance, IMM was relatively strong in closing processes; however, IS needs improve- ment. IMM was strong because competition is high, and it is very concerned about time to market. The IS companies scored low. Again, numerous software de- velopment projects have late releases deviating from the authorized schedule. This explains low project man- agement maturity in the closing processes. Another in- terpretation is that only a few major software develop- ment companies dominate the whole IS application area. Consequently, they are not really concerned about project delays or high competition. Project-Driven Organization Environment Phase. At the outset of this study, the authors decided to assess the project management “infrastructure” of an organiza- tion—namely, processes that cover training, career devel- opment, and other similar functions that a company must perform to ensure that its project managers are ready to manage—which was called project-driven orga- nization environment process. The project-driven organization environment processes are, for all companies taken together, the least mature processes. And because they have the lowest SD of all the processes, the data suggests that all the companies in all the industries have substantial room to improve. Commentary on the Data Analysis. To comple- ment (and, in some cases, to substantiate) analysis of the assessing data, a series of lengthy, detailed inter- views were conducted with several representatives of the organizations that participated in the study. Gener- ally, those people interviewed were senior managers of project managers. Company EC-1 had strong process maturity ratings, according to the benchmarking applied tests. What was noteworthy about this firm was its indifference toward formal project management processes and especially toward the Project Management Professional (PMP) certification. In the words of the contact person, “We re- ally don’t need PMP testing because we’re above the norm.” This comment and the attitude of the company have major implications for the PMI® and the PMP cer- tification process. To some people, this rings true, and to others, it may imply an arrogance. There is always a danger in always looking inward and not looking at what developments are taking place outside of one’s own company (or university). Interdisciplinary organi- zations such as PMI are intended to help counter any myopic views. Company EC-2 is what is frequently called an owner organization. It does not construct projects with its own personnel, but works closely and regularly with engi- neering/construction companies to build its consumer- product manufacturing plants all over the world. It is a storied company with a well-regarded management phi- 40 Project Management Journal March 2000 Table 8. Six Project Management Phases of Maturity EC Project Management Knowledge Areas IMM IS HTM All 38 Companies Initiating Maturity Planning Maturity Executing Maturity Controlling Maturity Closing Maturity Project-Driven Organization Environment Maturity Overall Average Project Management Processes Maturity 3.25 3.61 3.31 3.55 3.28 3.14 3.36 3.34 3.49 3.27 3.31 3.43 2.99 3.31 3.57 3.43 2.90 2.98 2.90 2.73 3.09 3.60 3.55 3.32 3.25 3.05 3.25 3.34 3.39 3.53 3.19 3.31 3.20 3.00 3.28
- 10. losophy and a keen emphasis on time to market for its consumer products. Typically, a project manager is assigned to oversee the project after the business unit has completed its market studies. This presents a problem in the opinion of the person interviewed in this study. Much of the project scope has been formalized by this time, partic- ularly in terms of product definition and the manufac- turing technology. Formalizing the project scope is proper, but bringing the project manager into the project afterward creates problems. The parent com- pany to this project management organization has been very active in developing alliances and partnerships with suppliers, and many of the process technologies that will be used in the manufacturing process have al- ready been selected and locked in by the time the project manager is assigned. The authors concluded from this interview that one important best project management practice would be to designate and assign the project manager as early as possible to the project. This will allow the project man- ager to impart project management-related concerns to the project while it is still in the initiation phase. Company IMM-6 is an information management and movement company that has struggled to implement formalized project management procedures over the past four years. The manager leading this effort is still trying to finalize his charter (after eight months of revision), which, as currently written, is too customer-oriented in his opinion. It did not surprise the manager that his de- partment’s overall project management maturity score was significantly below average: 2.18 versus 3.26. Project management training is performed by an outside consultant. Another consultant was used until recently, but that company was too software develop- ment-oriented for this company’s needs, which are more inclined toward system integration. The project management training is more “employee-pull,” rather than “supervisor-push” (i.e., the employee, not the em- ployer, chooses which of eight modules to study and when). There are 60 people in this group, who are dis- persed throughout the United States and Canada, com- plicating communication and changing the basis for lessons learned between the group members. This company did not perform risk management very well when measured by the benchmarking analysis, so a large portion of the interview focused on this project management knowledge area. Essentially, this group’s approach to risk management is to make exten- sive use of technology partners and alliances (i.e., con- tract the risk to other parties). Disaster recovery groups are also used; however, these are normally formed after the problem has arisen, which is too late. Procurement management was rated as being weak for a variety of reasons, including the fact that often the project management group is brought into the project after the customer has essentially selected which prod- ucts/services will be supplied by outsider entities. This, in turn, often dictates or limits the range of procurement options for the project manager because of standard in- dustry practices in these different areas. Sometimes, the customer may even have already selected the technology partner, further limiting procurement options. The contact person for this company is a manager of project managers, and he has a goal of having his project managers billable at least 60% of their time to projects. It was especially difficult for this manager to calculate the cost of project management services be- cause very frequently personnel do not finish the project. The customer may cancel the project in mid- stream or may decide that it can be finished with her own personnel. Our conclusions from this interview were that: ■ The project manager should be brought into the project early, as was the conclusion with the previous company. In this instance, procurements may be han- dled more effectively. ■ Conducting risk management retroactively is far less effective than conducting it proactively. Establishing disaster recovery groups to solve problems is often nec- essary, but it is much more effective to conduct a risk assessment and develop contingency actions before the project begins. Company IMM-10 is distinguished by having a pro- gressive Project Management Center of Excellence (PMCoE), though it is seems to be understaffed (two full-time equivalent people to support 40-plus project managers). The distinguishing trait is its coaching pro- gram, where it has established a formal mentoring pro- gram for its program managers. The key facet is the ongoing oversight role that the PMCoE provides to the several dozen project managers when it supports. Oversight gives those project man- agers a clearer indication of their long-range career paths and offers them incentives. The PMCoE resource people each have more than 10 years with the company and a cumulative 30 years of project management ex- perience. The mentoring involves a proactive interac- tion with project managers, rather than reactive meet- ings at times of crisis. One of the important observations in this interview was the long- and short-term nature of a PMCoE. It should not only be a “hot-line” crisis resolution service (which is the case in many of the companies we re- viewed), but also should support the long-term career development activities of project managers. Company HTM-3 is a hi-tech manufacturing com- pany that has used formalized project management concepts for several years. This firm has a formal intro- ductory project management training course utilizing an outside training firm, but it is more employee-pull than supervisor-push. There is no advanced project March 2000 Project Management Journal 41
- 11. management training course for more experienced per- sonnel seeking a refresher course. Like Company IMM-6, this company’s project man- agement personnel are frequently pulled off projects before they are finished, making it difficult to measure the percentage of project revenues spent on project management services. There is extensive emphasis placed on each project’s budgeting and cost control, so its above-average rating on cost management is understandable. A strategic planning unit has been formed within this company’s project management group. Two of its key functions are to develop better screening criteria for assessing the likelihood of project success before the project passes the budget authorization stage, and to improve the lessons learned during the feedback cycle. If a project is canceled during this screening step, unused budget funds are returned to the customer. Customer service and quality (CSQ) are important to this company, and one third of corporate executive bonuses are based on CSQ measures. Most of the com- munication measures used in this company are based on the quantity of information passed and the number of messages sent, not the effectiveness or timeliness of that information. This company does not have a PMCoE. A point stressed by this respondent was the need to better define CSQ in a project management environ- ment. Quantitative measures such as a project cost index and schedule index are part, but certainly not all, of the issue. We need better metrics for customer satisfaction. Summary, Conclusions, and Recommendations This study is one in a series of papers reporting the re- sults of the authors’ investigation into the value of project management. In this particular paper, the au- thors show that there are some perceptions of project management that are not confirmed by the actual data—for instance, the actual average cost of project management. On the other hand, there are many sup- positions that have been substantiated (e.g., the IS in- dustry is less mature than other fields, such as construc- tion). To be academically credible, a professional field depends upon a set of first principles and a body of knowledge that is replicable. This study shows that we have a start in understanding and forming a theory of project management, yet there are still many gaps in our knowledge base. Though this research is only the first of many steps in coming to an understanding of the quantitative ben- efits of project management, the contributions of this research are: ■ The development of a systematic project management process maturity assessment methodology was a major contribution for this study. The methodology is specifi- cally suited to assess an organization’s project manage- ment level. It provides a means for identifying and mea- suring different project management levels by analyzing eight project management knowledge areas with six project management phases under a quantified scheme. ■ This study presents and analyzes the strengths and weaknesses of project management practices among various diverse companies and industries. It also identi- fies problem areas hindering a project manager’s use within the project management process. ■ Companies should benchmark their operations using factual, impartial techniques such as those con- tained in the project management process maturity as- sessment questionnaire. This gives them a legitimate and sustainable reference point from which to begin making process improvements. ■ This study is one of the few, if not the first attempt, to truly integrate project management knowledge areas and project management phases against actual project performance data. In doing so, this study is a step to- ward a factual and quantitative way to measure project management practices and performance. The study team is convinced that the quantitative project management process maturity assessment methodology can be applied to other industries and com- panies to further the understanding of project manage- ment in the future. It could also be reapplied at a later time to the companies that participated in the first phase of this study. This would allow them to determine the im- pacts of changes that have occurred since their first bench- marking. By collecting and sharing this information, project management organizations can benefit and con- tinuously improve their project management practices. Also, the project management process maturity as- sessment questionnaire should be continuously refined to reflect advances in our project management knowl- edge base. This assessment questionnaire could further determine and evaluate an organization’s project man- agement maturity more effectively. Finally, more research is needed to fully understand the project management practices and processes more thoroughly to achieve a true project-driven organiza- tion environment in the business world. Acknowledgments This research was sponsored by the PMI Educational Foundation and PMI’s Northern California Chapter. In particular, we thank Dan Ono, Jim McFarlin, Mike Mc- Cauley, Paul Nelson, Bill Ruggles, Ahmet Taspinar, Bob Thompson, and Cathy Tonne. We also appreciate the participation received from the 38 companies that pro- vided the data. The IPS assessment tool served as a basis for building the Berkeley assessment tool. 42 Project Management Journal March 2000
- 12. March 2000 Project Management Journal 43 References Al-Sedairy, Salman T. (1994). Project management prac- tices in public sector construction: Saudi Arabia. Project Man- agement Journal, 25 (4), 37–44. Archibald, Russell D., & Villoria, Richard L. (1967). Net- work based management systems (PERT/CPM). New York: John Wiley & Sons. Boznak, Rudolph G. (1988). Project management— Today’s solution for complex project engineering. IEEE. Bu-Bushait, K.A. (1989) The application of project man- agement techniques to construction and R&D projects. Project Management Journal, 20 (2), 17–22. Construction Industry Institute. (1990, April). Assessment of owner project management practices and performance. Special CII Publication. Cleland, David. (1993, September). The future of project management. PM Network, 7, 6–8. Deutsch, Michael S. (1991). An exploratory analysis re- lating the software project management process to project suc- cess. IEEE Transactions on Engineering Management, 38 (4). Donnelly, Richard G., & Kezsbom, Deborah S. (1993). Overcoming the responsibility-authority gap: An investigation of effective project team leadership for a new decade. 1993 AACE Transactions. Dooley, K., Subra, A., & Anderson, J. (1998, February). The impact of maturity and best practices in new product de- velopment. Quality Management Conference, Arizona State Uni- versity. Gobeli, David H., & Larson, Erik W. (1986). Matrix man- agement: More than a fad. Engineering Management Interna- tional, 4. Gross, Robert L., & Price, David. (1990). Common project management problems and how they can be avoided through the use of self-managing teams. 1990 IEEE International Engi- neering Management Conference. Ibbs, C.W., & Kwak, Young-Hoon. (1997). The benefits of project management—Financial and organizational rewards to cor- porations. Upper Darby, PA: Project Management Institute. Ibbs, C.W., & Kwak, Young-Hoon. (in press). Calculating project management’s return on investment. Project Manage- ment Journal. Kwak, Y.H., Clark, A., Grilo, A,, Betts, M., & Ibbs, C.W. (1995, January). Contemporary strategic planning tools and applications for construction managers. First International Con- ference on Construction Project Management, Singapore, pp. 25–40. Kwak, Young Hoon. (1997, May). A systematic approach to evaluate quantitative impacts of project management. Ph.D. Dis- sertation, Department of Civil Engineering, University of Cal- ifornia, Berkeley. Larson, Erik W., & Gobeli, David H. (1989). Significance of project management structure on development success. IEEE Transactions on Engineering Management, 36 (2). Lundin, Rolf A., & Soderholm, Anders. (1994, October). Conceptualization and learning—The black environment of PM. Proceedings of the 25th Annual Project Management Insti- tute’s 1994 Seminars & Symposium. McCollum, James K., & Sherman, J. Daniel. (1991). The effects of matrix organization size and number of project as- signments on performance. IEEE Transactions on Engineering Management, 38 (1). Might, Robert J., & Fischer, William A. (1985). The role of structural factors in determining project management success. IEEE Transactions on Engineering Management, EM-32 (2). Project Management Institute Standards Committee. (1994). A guide to the project management body of knowledge (PMBOK® Exposure Draft). Project Management Institute Standards Committee. (1996). A guide to the project management body of knowledge (PMBOK® Guide). Upper Darby, PA: Project Management Institute. Taspinar, Ahmet. (1998). Personal communication. Ziomek, N.L., & Meneghin, G.R. (1984, August). Training—A key element in implementing project manage- ment. Project Management Journal, pp. 76–83.