Week02 reading lechler_plan_and_changing
- 1. Research Policy 33 (2004) 1–15 Plans are nothing, changing plans is everything: the impact of changes on project success Dov Dvira, Thomas Lechlerb,∗ a School of Management, Ben Gurion University, Beer-Sheva, Israel b Wesley J. Howe School of Technology Management, Stevens Institute of Technology, Castle Point on Hudson, Hoboken, NJ 07030, USA Received 11 July 2002; received in revised form 9 January 2003; accepted 9 April 2003 Abstract Based on a sample of 448 projects, the interactions between three project planning variables, the quality of planning, goal changes, plan-changes and project success are analyzed. The most important results of this study are the interactions between the planning variables and their influences on project success. By using structural equation modeling, we gained insight into these complex indirect relationships. The results clearly show that the positive total effect of the quality of planning is almost completely overridden by the negative effect of goal changes. If we add the total effects of goal changes and plan-changes on project success, their combined effect is considerably stronger than that of the quality of planning. The study also identifies several contextual variables affecting the planning process. © 2003 Elsevier B.V. All rights reserved. Keywords: Project planning; Project success; Goal changes; Plan-changes 1. Introduction Ever since project management has become a for- mal discipline, the quality and importance of project planning has been considered a major cornerstone of every successful project. Although projects have ex- isted since the beginning of civilization, project man- agement, as a discipline, emerged in the 1950’s and 1960’s with the development of network techniques such as program evaluation and review technique (PERT) and critical path method (CPM). Since then project planning, focusing on scheduling and budget- ing has dominated project management research and ∗ Corresponding author. Tel.: +1-201-216-8174; fax: +1-201-216-5385. E-mail address: tlechler@stevens-tech.edu (T. Lechler). discussion. The establishment of the Project Manage- ment Institute (PMI) in 1969 has further strengthened this notion. Its guidelines, the project management body of knowledge (PMBoK) strongly advocates the importance of project planning (PMI, 2000). Numer- ous empirical studies of project management success factors suggested planning as one of the major con- tributors to project success (Murphy et al., 1974; Pinto and Slevin, 1987; Lechler, 1997). Yet, this orthodox thinking can be challenged. The strategic management literature provides a critical insight into the influence of strategic planning on the success or performance of a company. One mile- stone in this discourse is Mintzberg’s (1994) book The Rise and Fall of Strategic Planning. Even in the project management literature some doubts have been recently raised regarding the importance of formal 0048-7333/$ – see front matter © 2003 Elsevier B.V. All rights reserved. doi:10.1016/j.respol.2003.04.001
- 2. 2 D. Dvir, T. Lechler / Research Policy 33 (2004) 1–15 project planning (Bart, 1993; Andersen, 1996). So, is project planning that important? Our assumption was that good planning itself may not be a sufficient pre- dictor of success. This assumption follows previous doubts about project planning, such as Peters et al. (1988, p. 138), who wrote: Unfortunately, most innovation management prac- tice appears to be predicated on the implicit as- sumption that we can beat the sloppiness out of the process if only we’d get the plans tidier and the teams better organized. The role of experiments and skunkworks, the zeal of champions, the power gained from exploiting the innovative user as part- ner, is denigrated as an aid only fit for those who aren’t smart enough to plan wisely. This means planning is a necessary but not a suffi- cient condition for project success. Planning is not a one-time task. Eisenhower’s historical dictum: “Plans are nothing, planning is everything” points out the importance of the planning process itself. Most au- thors agree that projects are complex, time restricted, unique endeavors and special tasks that have not been done before. Consequently, it is very difficult or even impossible at the initial planning stage to know pre- cisely which activities have to be carried out in order to complete the project, and what their cost and dura- tion parameters are (Andersen, 1996). Adding to that the high uncertainty associated with projects, the tradi- tional emphasis on project planning in the industry as well as the unequivocal empirical results are even more surprising. Hence, we interpret Eisenhower’s dictum to mean that in managing projects, original project plans and project goals will have to be changed to address the dynamics caused by uncertainty, and to maximize project success. On the other hand, changes in plans can cause high transaction costs, which have a negative impact on project results. Changes in plans may be introduced for various reasons. They may come from a change required by the customer, from new and better ideas suggested by the project team, or even from the dic- tate of a new manager, who comes in at a later stage and wants to impose its own twist to the project. Quite often, projects undergo tremendous changes and when the project is finally completed it may no longer be relevant: too much “tweaking” can result in loss of the original project focus. The original question regarding project planning can thus be rephrased: “How do changes in either goals or plans impact project success?” This question is hardly addressed by previous research, and we believe that a careful empirical investigation is needed to better understand the effect of change on project management success. Referring again to Eisenhower, the central question of this article could be rephrased as: is it true that, plans are nothing, changing plans is everything? Our first objective was to study empirically the im- pact of project planning, project goal changes, and project plan-changes on project success, and to de- termine whether a high quality of project planning could compensate for the possible negative effects of changes. When referring to the quality of project plan- ning we refer to the quality of the initial project plans. The second goal was to understand how project con- textual variables affect goal changes and how such changes, in turn, affect project success. The next section discusses the theoretical and em- pirical literature on the limitations of project plan- ning and specifies hypotheses based on that review. In the third section, we derive and empirically test the conceptual framework. The results are reported in the fourth section. In the fifth section, the implica- tions of the results are discussed. The final section is an effort to discuss the practical implications of our results and to present some suggestions for further research. 2. Current research on project planning Planning is prevalent in project management and in the strategic management literature. The discussion of formal strategic planning and its impact on the corporate performance is somewhat parallel to the dis- cussion of project planning and its impact on project success. Therefore, the field of strategic planning seems to be an important source of knowledge com- plementing the discussion of project management. The literature on strategic planning generally addresses three different problem areas: the importance and im- pact of plans on performance, the planning process it- self, and contextual influences on the planning process (Armstrong, 1982). In the following discussion, we refer to these three issues within the context of project planning.
- 3. D. Dvir, T. Lechler / Research Policy 33 (2004) 1–15 3 3. The impact of project plans on project success The impact of strategic planning on corporate performance has been addressed in several studies (Rhyne, 1986; Ramanujam and Venkatraman, 1986). Only 10 out of 15 empirical studies have reported significant improvements resulting from formal plan- ning activities (Armstrong, 1982). In contrast, the results pertaining to the impact of planning on project success are much less ambiguous. A review of 44 empirical project management success factor studies (Lechler, 1997) identified 13 studies analyzing the effect of project planning on project success. All of the analyzed studies have demonstrated significant strong or medium positive effects on project success. However, there are other authors in the field of project management who claim that formal project planning is not necessarily helpful or even desirable (Bart, 1993; Andersen, 1996). Bart (1993) indicates that the traditional approach to planning and controlling of R&D projects tends to fail because of excessively restrictive formal control, which curtails creativity as a factor contributing to project success. Bart proposes to reduce formal planning and control to a minimum required level. Yet, even if this is done, there is no argument as to the contribution of complete and accu- rate capturing of end-user requirements to successful project completion (Chatzoglou and Macaulay, 1996). The main reason is that the output of the requirements analysis stage will most likely determine the output of the entire development process. Posten (1985a,b) has found that 55% of all defects in R&D projects occur during requirement analysis and specification whereas 43% of all defects are not found until after the testing stage. The almost unanimous agreement in the project management literature in contrast to the strategic man- agement literature could derive from methodological and conceptual factors. One possible argument could be the choice of the dependent variable. The strategic management literature indicates that the influence of planning differs over different performance measures (Armstrong, 1982; Ramanujam and Venkatraman, 1986). The criteria for measuring project success must reflect different views (Cooper and Kleinschmidt, 1987; Pinto and Slevin, 1988; Freeman and Beale, 1992). Nevertheless, the difficulty of measuring project success from several points of view have driven project managers to use simplistic formulae such as meeting or coming close to budget, attaining scheduled goals and achieving acceptable levels of performance. These measures are partial and some- times misleading (Baker et al., 1988). Although the multidimensional approach for assessing project suc- cess is a common understanding today, most of the project management literature does not differentiate between the impacts of success factors on the various success dimensions. Another methodological argument is that the suc- cess factor studies do not investigate the impact of success factors on the project performance over its life cycle. An exception is Pinto and Prescott (1990) who claim that critical success factors of project manage- ment fall into two distinct sub-groups: those related to initial project planning and those concerned with subsequent tactical operationalization. It was found that the relative importance of planning and tacti- cal factors varies across the project life cycle. When ‘internal’ success measures are used (meeting budget, schedule and performance goals), planning factors are initially perceived to be of high importance but are overtaken by tactical issues as the project progresses through its life cycle. When ‘external’ success mea- sures (perceived value of the project and client satis- faction) are employed, planning factors have dominant importance over tactics throughout the project’s life cycle. 4. The influence of the planning process on the quality of project plans Planning is a process with many different activities that cover a variety of issues, using numerous planning techniques and planning procedures such as analysis, design reviews, reports and interpersonal communi- cation. Bryson and Bromiley (1993) identified em- pirically two groups of planning activities having an impact on the project success: internal project commu- nication, with a positive influence on project success, and forced project goals, with a negative influence. The first result is also supported by Lechler (1997). The importance of the initiation phase stands out relative to other phases in the project life cycle (King and Cleland, 1988; Meyer and Utterback, 1995). Dvir et al. (1999) indicated in a study of 110 development
- 4. 4 D. Dvir, T. Lechler / Research Policy 33 (2004) 1–15 projects that the origination and initiation phase, has the greatest influence on project success. During that phase major decisions are made such as deciding the project’s objectives and planning the project’s execu- tion. They also found that although the preparation of formal design and planning documents has a strong positive effect on meeting time and budget objectives, it also contributes significantly to customer benefits deriving from the end−product. In a follow-up study, Dvir et al. (2003) suggested that no effort should be spared in the initial stage of a project to properly define the goals and the project’s deliverable requirements. This task cannot be achieved without customer or end-user involvement in the process. The use of planning tools is an integral part of the project planning process, especially the use of net- works. They are based on computational models orig- inating in large projects from the 1950’s onwards, and are used extensively predominantly in the aerospace, defense and construction industries (Kerzner, 1998). Project planning is mainly focused on detailed network scheduling approaches (Tatikonda and Rosenthal, 1999). Gutierrez and Kouvelis (1991) criticize the use of CPM as it systematically fails to predict the dura- tion of complex projects. Many truly excellent orga- nizations do not use the PERT approach to planning projects. For instance, one of Hewlett-Packard’s UK plants uses whiteboards and Post-it notes for project planning at the top level, with individual sub-project managers free to use computerized planning software at the task level (Maylor, 2001). Andersen and others propose replacing the standard planning approach with milestone planning (Andersen et al., 1995; Turner, 1993), where a milestone is de- fined as a result to be achieved. Since a milestone de- scribes what is to be done, but not the way it should be done, milestone planning promotes result-oriented thinking rather than activity-oriented thinking. The criticism of planning tools derives from prob- lems associated with the implementation process, which is prone to frequent changes of project goals and plans. The commonly used tools do not provide answers to managing changes. As the strong negative influences of frequent goal changes show (Murphy et al., 1974), project managers are not aware of the consequences of frequent changes and are not pro- vided with the information necessary to deal with changes efficiently. The efforts to develop alternative solutions, such as the use of Post-it notes or milestone approaches, instead of using detailed networks, also reflect the severity of frequent goal or plan-changes during project implementation. 5. Project goal changes versus plan-changes The discussion about project planning processes does not directly address the issue of goal changes or plan-changes. However, implicitly one can assert that project planning is an ongoing task and therefore it is subject to changes. Trade-offs are usually made be- tween the three traditional constraints: budget, sched- ule and scope, and are often made without taking into account the impact of these changes on project suc- cess, or specifically on customer satisfaction. In order to analyze the influence of changes on projects we have to distinguish first between two types, changes that have an impact on the project plan but do not have an impact on the project goals or meeting cus- tomer requirements: we call these plan-changes. And, changes that reflect a change in the project goals: we call them goal changes. Plan-changes are typically induced by the envi- ronment and prevent us from following the original project plan. Such changes can be a result of shortage in resources, delays, strikes, weather conditions, etc. Sometimes they are a result of poor planning requir- ing change in order to meet the requirements. The project manager has to make the necessary adjust- ments without changing the project scope and goals. Goal changes on the other hand, are typically a re- sult of a conscious decision by the stakeholders to change the goals of the project. They could be due to changes in requirements, lack of ability to meet exist- ing requirements within the available budget and time, or changes in circumstances that impact the necessity of the project end-product. Goal changes, when ap- proved, require a change in plans in order to meet the updated requirements. Both types of changes are unavoidable, but in the first case all what that the project manager can do is to find the most efficient way to deal with the situation while in the second case, the amount of change can be controlled by collaboration between the project team and the stakeholders.
- 5. D. Dvir, T. Lechler / Research Policy 33 (2004) 1–15 5 6. The contextual influences on planning process The main purpose of planning is to reduce uncer- tainty (Shenhar, 1993; Laufer et al., 1997) and hence the function of planning is dependent on the context in which it is undertaken. Several authors recognize the importance of contextual influences on strategic plan- ning. Formal planning systems can contribute highly to risky and important decisions (Sinha, 1990). Tech- nological uncertainty has been shown to have a nega- tive impact on the success of projects (Murphy et al., 1974; Rubenstein et al., 1976; Souder and Chakrabarti, 1978; Baker et al., 1988; Might, 1984; Ashley et al., 1986; Pinto, 1986; Larson and Gobeli, 1988) and the planning process itself (Grinyer et al., 1986). As the success factor studies show technical uncertainty is caused by external influences like a technical break- through of a competitor or technological risks inher- ent to the project task. Dawson and Dawson (1998) have shown that current planning techniques are inade- quate for projects involving uncertainty and risk. They suggest that projects should use probability distribu- tions to assess task durations and generalized activity networks with probabilities associated to each path. The findings of Turner and Cochrane (1993) suggest that contextual variables are one of the main causes for changes over the project lifecycle; among them, technological uncertainty is a typical variable which moderates the effect of project planning on project success by causing plan and goal changes (see also Balachandra and Friar, 1997). Bryson and Bromiley (1993) show that technolog- ical uncertainty has a significant negative impact on Goal Changes Efficiency Customer Satisfaction Context - -- - ++ ++ Quality of Planning Plan Changes + - - + - + ++ Fig. 1. Hypothesized relations between the planning variables and success. project success and that stability has a positive in- fluence. They identify eight different contextual vari- ables affecting the planning process of organizational change projects. Some of these variables, like tech- nological uncertainty and economic stability, also de- scribe the context of projects. The empirical literature has shown some evidence for the impact of contextual settings on the success of projects. Yet, their impact on planning is not an- alyzed, which is surprising because planning should reflect and anticipate contextual influences. From that discussion, we can conclude that the interactions of project planning with other process-related variables and their combined effects on project success, have not been studied in-depth. Therefore, only a model that describes the various interactions between plan- ning variables and several success dimensions can re- veal the ‘true’ effect of planning on project success. In conclusion, it seems that project plans and the planning process are an important part of project management, but the question remains whether their influence on project success is correctly assessed or overestimated. In essence, most of the literature sees the project plan primarily as a static and stable entity, and the question of how plan and goal changes affect project success is still relatively unanswered. 7. The conceptual framework of the study The definition and operationalization of the vari- ables and their interrelationships are discussed in this section. The hypothesized relationships between the model variables are graphically represented in Fig. 1.
- 6. 6 D. Dvir, T. Lechler / Research Policy 33 (2004) 1–15 7.1. Project success Pinto and Mantel (1990) identified three distinct aspects of project performance: the implementa- tion process; the perceived value of the project; and client satisfaction with the delivered project outcome. Shenhar et al. (1997) have used in their research three criteria for the assessment of project success: Meeting design goals; benefits to customers; and commercial success and future potential. Since each stakeholder assesses the project’s outcome from a different point of view, it is conceivable that the relative importance assigned to each dimension will vary with the stake- holder assessing the project success. Lipovetsky et al. (1997) who have used four dimensions for measuring project success have found that customer satisfaction is by far the most important criteria, almost twice as important as efficiency. The importance of the other two criteria, commercial success and future poten- tial was almost negligible. Therefore, in this study, we measure project success with the two success criteria: project efficiency and customer satisfaction. Several empirical studies show a strong correlation between project efficiency and customer satisfac- tion (Lipovetsky et al., 1997; Pinto, 1986). Thus we propose: Hypothesis 1. Project efficiency positively impacts customer satisfaction. 7.2. Planning variables The planning variables used in our study are: the quality of project planning (schedule, budget and scope), the frequency of plan-changes, and the extent of goal changes. Many empirical studies show the positive impact of project planning on project success (Murphy et al., 1974; Rothwell et al., 1974; Pinto, 1986 and many others). It is hypothesized therefore, that the quality of project planning positively affects project efficiency as well as customer satisfaction. Hypothesis 2a. Project success (both efficiency and customer satisfaction) is positively affected by the quality of project planning. This means implicitly that the definition of project goals as part of the project planning process will reduce the extent of project goal changes. We pro- pose also a reducing effect of the quality of project planning on the frequency of plan-changes although this effect might be less strong. Nevertheless, poor planning quality will cause many plan changes even if there are many other reasons for plan changes. Hypothesis 2b. High quality of project planning re- duces the extent of goal changes and the frequency of plan changes. Only a few empirical studies have analyzed the di- rect effects of goal changes (Murphy et al., 1974; Lechler, 1997) on project success. These studies show strong and significant negative effects of goal changes on project success. Thus, we can assume that the posi- tive effects of the quality of project planning on project success are counter balanced by negative effects of goal changes and plan changes. Hypothesis 3a. Goal changes have a strong negative effect on project success (both efficiency and customer satisfaction). The distinction between goal and plan changes al- lows for the assumption that the changes in plans will occur even when the project plan was carefully pre- pared and no goal changes are introduced. Therefore, it is not clear how plan changes will influence project success. We assume that a high frequency of plan changes will have a negative effect on project success. Hypothesis 3b. Plan changes have a strong negative effect on project success (both efficiency and customer satisfaction). Goal changes always lead to plan changes while the opposite is not true. Thus, we propose: Hypothesis 3c. Project goal changes lead to plan changes. 7.3. Contextual variables The third group of variables describes the context of planning. Technical risks are the one contextual variable analyzed and discussed most often in the lit- erature. Another major contextual influence on project
- 7. D. Dvir, T. Lechler / Research Policy 33 (2004) 1–15 7 success and the planning process are the available resources which negatively impact project success (Balachandra, 1984). Variation in the resource struc- ture during the project implementation is a significant factor for failure and is caused by insufficient man- power and too many parallel projects done at the same time. Yet, it is not clear how the quality of project planning is impacted. The importance of a project from the perspective of the company has a positive impact on project success (Pinto, 1986). In an exploratory approach several contextual vari- ables, describing the contextual influences on the resource structure, competitive internal and external environment, were assumed to influence the quality of project planning and stimulate goal changes and as a result, plan changes. We thus propose our exploratory hypothesis: Hypothesis 4. Contextual variables impact goal and plan changes and the quality of project planning. The core model is derived from our literature anal- ysis and theoretical ideas and therefore, the test of this part of the framework is confirmatory. On the other hand, the last hypothesis is an exploratory statement since only few studies analyzed contextual influences on project success. There are some indications that the context might influence the project planning and ex- ecution but how and which variable has an effect on project planning was not analyzed in detail yet. 8. Methodology 8.1. Research design and data collection Data collection was performed in Germany and resulted in a sample of 448 projects. A detailed questionnaire, which was designed to measure the impact of success factors of project management, was distributed to the members of the German Project Management Society (Gesellschaft für Projektman- agement, GPM)). Each respondent was asked to fill out two questionnaires gathering data on a pair of completed projects—a successful and a failed project. This concept of pair wise comparison was first intro- duced by Rothwell et al. (1974) and has the advantage of reducing the personal bias of the key informants. The questionnaire included 199 single items and some quantitative information about each project. Out of these, 67 items were directly taken from Pinto’s (1986) questionnaire, with permission of the author, and translated into German. The remaining items were developed with the help of several experienced project managers. Two versions of the questionnaire were pre-tested and modified after in-depth interviews and responses by a group of experienced project managers. The variables in the questionnaire that are rele- vant to this study are listed in Table 6: the extent and the frequency of goal-changes, the frequency of plan-changes, the quality of project planning, project efficiency and customer satisfaction and six contex- tual variables. Each item was assessed on a 7-point scale; from strongly agree to strongly disagree. Due to their exploratory nature five of the six contextual variables were measured by single items. Three items measure the sixth contextual variable, strategic impor- tance. All constructs that were measured with multiple items were tested with Cronbach’s Alpha for scale re- liability and with confirmatory factor analysis for uni- dimensionality. All scales achieve a Cronbach’s Alpha >0.8 and communalities of >0.6. All variables were also tested for normality. In preparation for the SEM estimations, a covariance matrix was calculated using the variables’ z-scores. The resulting covariance ma- trix is based on 448 responses. 8.2. Sample characteristics The data collection effort achieved an overall re- sponse rate of 43%, resulting in a sample size of 448 projects. The sample for the present investigation con- tains data on 257 successful and 191 unsuccessful projects (Table 1). About 50% of the respondents were project man- agers. In cases where the project managers could not be reached, team members were approached. The Table 1 Distribution of the respondents’ functions Function Frequency Frequency in percent Project manager 207 46.2 Team member (technical) 85 19.0 Team member (business) 36 8.0 Others 120 26.8 448 100.0
- 8. 8 D. Dvir, T. Lechler / Research Policy 33 (2004) 1–15 Table 2 Distribution of project types Type of project Frequency Frequency in percent Machine tool manufacturing 25 5.6 Plant construction 61 13.6 Building construction 30 6.7 Product development 116 25.9 Reorganization 52 11.6 Software 109 24.3 Others 55 12.3 448 100 group ‘others’ includes respondents who were exter- nal consultants or specialists with intimate knowl- edge of the project. This homogeneity of respondents participating actively in the project implementation enhances the validity and reliability of the measures. The sample is relatively balanced concerning the different types of projects (see Table 2). Around 26% of the projects in the sample are machine tool, plant construction or building construction projects. Prod- uct development projects account for another 26%, including projects for product modifications, product enhancements, and completely new products. Around 24% are software projects. The group ‘others’ includes mainly projects dealing with creating new technical concepts as well as technical feasibility studies. The sample provides a fairly representative cross-sectional distribution of projects carried out in the German industry. For further detailed information on the sample see Lechler (1997). 8.3. Data analysis The contextual variables of our model were identified using an exploratory correlation analysis (Table 3). Out of an initial list of twelve different con- textual variables six have been found to significantly affect the planning process: strategic importance of the project, level of experience of the project team, personnel constraints within the organization under- taking the project, parallel projects undertaken at the same time, occurrence of technological breakthrough which affects the project results, and the technological risk associated with the project. The interaction hypotheses were tested using a structural equation model. For the model estimation Table 3 Pearson’s correlations between six contextual variables quality of planning, goal changes and plan changes Contextual variables Planning variables Quality of planning Goal changes Plan changes Personnel constraints n.s. 0.226∗∗ 0.226∗∗ Parallel projects n.s. 0.202∗∗ 0.174∗∗ Occurrence of breakthrough n.s. 0.247∗∗ 0.127∗∗ Technological risk n.s. 0.175∗∗ 0.165∗∗ Importance 0.204∗∗ n.s. n.s. Level of experience 0.170∗∗ n.s. n.s. n.s.: not significant. ∗∗ P ≤ 0.01. linear structural relationships (LISREL) version 8.51 was used. LISREL is a statistical method that allows simultaneous analysis of hypothesized causal relation- ships for multiple variables (Jöreskog and Sörbom, 1993), e.g. the direct effects of goal changes on the two different project success variables and simultane- ously the indirect effects by taking into account the effects on the variable plan changes and its effects on the success variables. The evaluation of a structural equation model is quite complex since no single test offers sufficient evidence to accept or reject a model. Recognizing the problems associated with the evaluation of linear structural equation models (Bagozzi, 1980; Anderson and Gerbing, 1988; Bollen, 1989; Fritz, 1992; Soni et al., 1993; Baumgartner and Homburg, 1996), a comprehensive set of tests was employed to assess the goodness of fit. To accept the model, the following criteria have to be satisfied: a chi-square (P > 0.05), which tests the null hypothesis that the estimated variance–covariance matrix deviates from the sample variance–covariance matrix only because of sampling errors. The chi-square test is limited to the extent that it is dependent on the sample size. Browne and Cudeck (1993) showed that with an increase of the sample size any model could be rejected. Because of these weaknesses of the chi-square test, Jöreskog and Sörbom suggested the two global fit indices, goodness of fit index (GFI) and adjusted goodness of fit index (AGFI). To evaluate the fit of our structural equation models we used the AGFI, since its calculation is based on the GFI and because it accounts for the de- grees of freedom. Values below 0.90 indicate that the
- 9. D. Dvir, T. Lechler / Research Policy 33 (2004) 1–15 9 model should be rejected (Baumgartner and Homburg, 1996). The root mean square error of approximation (RMSEA) is a measurement of non-centrality and estimates how well the fitted model approximates the population covariance matrix per degree of freedom. Browne and Cudeck (1993) suggest that a RMSEA ≤ 0.05 indicates a close fit and that the model should be accepted. The comparative fit index (CFI) assesses the relative reduction in lack of fit as estimated by the chi-square of a target model versus a baseline model in which all of the observed variables are uncorrelated (Bentler, 1990). Models with a CFI below the 0.85 should be rejected (see Bentler and Bonett, 1980). 9. Results Table 3 provides the correlations between the six contextual variables having significant correla- tions with goal-changes, plan-changes, and quality of project planning. Only two variables are correlated with the quality of project planning (strategic impor- tance and level of experience) and the other four are correlated with goal changes and plan changes. In the next step of the data analysis, the interaction of the model variables were estimated simultaneously. We started the LISREL analysis with the confirma- tory part of the model only, e.g. the contextual vari- ables were not included in the estimation. Based on the results of the correlation analysis we introduced the contextual variables into the model. According to the pattern of correlations (Table 3) the paths from R 2 = .19 Goal Changes R 2 = .26 Efficiency R 2 = .55 Customer Satisfaction Imp. +.61 R 2 = .26 Plan Changes +.14 -.16 -.23 +.27 -.21 +.50 -.27 Expr. +.20 +.16 Pconst. +.11 +.16 PProj. Trisk. Break. +.15 +.14 +.18 n.s. Quality of Planning n.s. Fig. 2. Results of the structural equation model. Fit statistics: χ2 = 40.93, df = 25, P < 0.023, RMSEA = 0.038, AGFI = 0.94, CFI = 0.98. Parameter estimates are from the completely standardized solution and are significant at P < 0.05 or better. importance and experience variables to the quality of planning variable were added to the confirmatory model. Paths from the other four contextual variables were added to both goal-changes and plan-changes variables. The final model presented in Fig. 2 does not differ from the confirmatory model of step one, there- fore we do not present that structural equation model. Except for the chi-square index, all test criteria are met in assessing the model fit. As mentioned, the chi-square index depends on the sample size. The sam- ple used in this analysis exceeds 400 cases and that make it nearly impossible to achieve P-values above 0.05. Since all other tests achieve or exceed the re- quired fit criteria, the final structural equation model should be accepted. The results for the hypotheses tests are shown in Table 4. The high positive impact of efficiency on customer satisfaction fully supports our first hypothesis H1. As the significant path coefficients show project efficiency is directly affected by all three planning variables. Customer satisfaction is directly affected only by the quality of planning and goal-changes and not directly affected by plan-changes. The signs of the path coef- ficients indicate positive effects of quality of planning and negative effects of plan-changes and goal-changes on project success. Thus the hypotheses H2a and H3a are fully supported. On the other hand, hypothesis H3b is only partially supported since there is no significant path between plan-changes and customer satisfaction. Hypothesis H2b is also partially supported since the path between quality of planning and plan-changes is not significant but the path to goal-changes is
- 10. 10 D. Dvir, T. Lechler / Research Policy 33 (2004) 1–15 Table 4 Hypotheses testing results Hypothesis Result H1 Project efficiency positively impacts customer satisfaction Supported H2a Project success is positively affected by the quality of planning Supported H2b High quality of project planning reduces the level of goal and plan changes Partially supported H3a Goal changes have a strong negative effect on project success Supported H3b Plan changes have a strong negative effect on project success Partially supported H3c Project goal changes lead to plan changes Supported H4 Contextual variables impact goal and plan changes and the quality of project planning Supported considerably high and significant (−0.27). Hypoth- esis H3c describing the effect of goal-changes on plan-changes is fully supported by the high and significant path coefficient (0.50). The exploratory hypothesis H4, proposing effects of contextual vari- ables on the three planning variables is supported by the significant path coefficients in the LISREL model, which takes simultaneously into account the direct and indirect effects of all variables and variable groups on project success. 10. Discussion The main purpose of this study was to provide an in-depth investigation of the interactions between planning variables on two different dimensions of project success. A secondary research issue was to es- timate the impact of contextual variables on the plan- ning process, especially how goal and plan changes are affected by the project context. The most impor- tant results of this study are the interactions between the planning variables and their influences on project success. Only by investigating the three planning vari- ables separately using structural equation modeling instead of multivariate regression analysis can we gain insight into the complex indirect relationships among them and explore phenomena that would be otherwise Table 5 Total effects of the planning factors on success Planning variables Effects on efficiency Effects on customer satisfaction Direct Indirect Total Direct Indirect Total Quality of planning +0.27 +0.08 +0.35 +0.14 +0.25 +0.39 Goal changes −0.21 −0.10 −0.31 −0.15 −0.19 −0.34 Plan changes −0.23 – −0.23 – −0.14 −0.14 unobservable. The total effects (direct and indirect effects) of the planning variables on project success are summarized in Table 5. The results clearly show that the positive total effect of the variable quality of planning is almost completely overridden by the negative effect of goal changes. If we add the total effects of the two variables, goal changes and plan changes on project success, their combined effect is considerably stronger than that of the quality of planning. The significant but clearly differing influences of the planning variables on the project success variables in- dicate the importance of differentiating between these two success dimensions. While the quality of plan- ning positively affects both efficiency and customer satisfaction, changes are acting in the opposite direc- tion; namely, changes are compromising the project results. The magnitudes of these influences are of spe- cial interest. The quality of planning has the highest positive direct effect (+0.27) on efficiency, while goal changes have the highest negative direct effect (−0.16) on customer satisfaction. These results reflect the na- ture of traditional planning, which is focused mainly on project schedule and budget, while project goals are more focused on the project substance, which repre- sents the value for the customer. The differences in the total effects of the planning variables show that high quality planning cannot compensate for the negative
- 11. D. Dvir, T. Lechler / Research Policy 33 (2004) 1–15 11 Table 6 Operationalization of the constructs Construct Scale Measures Success Efficiency (Alpha:0.86) 1. The project had come in on schedule (Pinto). 2. The project had come in on budget (Pinto). Customer satisfaction (Alpha:0.81) 1. The clients were satisfied with the process by which this project was completed. 2. The clients are satisfied with the results of the project (Pinto). Planning Plan changes The project plans (schedule, personnel, budget) were often changed. Goal changes (Alpha:0.83) 1. Project goals were often changed. 2. At least one major project goal was changed considerably. Planning quality (Alpha:0.85) 1. The entire project task (scope) was structured in work packages. 2. Every work package was allocated with a specific time allowance. 3. We knew which activities contained slack time or slack resources. 4. All work packages had a predecessor and a successor work package (except the first and the last). 5. There was a detailed budget plan for the project. 6. The precise demand for key personnel (who, when) was specified in the project plan. Context Technical risks (Alpha:0.79) 1. The task was technically demanding. 2. The completion of the business goals included high risks. Importance (Alpha:0.78) 1. It was important that the results of the project could be used as soon as possible (Pinto). 2. The implementation of the project was important for the organization’s policy (Pinto). 3. The implementation of the project was important for the success of the organization (Pinto). Experience The projecting company had experience with the solution of similar problems. Manpower The project team did not experience any significant personnel losses or transfer during the project’s development. Parallel projects The completion of the project depended on other projects, undertaken at the same time. Break through The project was not subject to any recent technological breakthrough, which could have rendered it obsolete. All items are measured on 7-point rating scales, Cronbach’s Alpha in brakets. effects of changes. Although the total effect of the quality of planning (+0.39) is higher than the indi- vidual total effects of plan changes (−0.14) and goal changes (−0.34), their combined total effect is con- siderably larger (−0.48). The interactions between the planning variables are not straightforward and self-evident. While the quality of planning reduces the level of goal changes (−0.27), it does not affect plan changes at all. Changing project goals leads to changes of project plans as shown in the model by a strong direct effect (0.50). The percent- age of explained variance of the plan changes indi- cates that there are other influences causing changes in the project plan. These results indicate that high qual- ity project planning, although very important, cannot completely compensate for plan changes during the project life cycle. The third question this analysis addresses is how the context influences the project planning activi- ties. This part of the study is more exploratory. In contrast to our initial hypothesis, not all contextual variables have a significant impact on the planning and change variables. Regarding their influence, they fall into two groups. Strategic importance and the level of experience of the project team are affect- ing only the quality of planning, as the exploratory
- 12. 12 D. Dvir, T. Lechler / Research Policy 33 (2004) 1–15 correlations indicate (Table 3). Whereas the second group of contextual variables, personnel constraints, parallel projects, occurrence of breakthrough, and technological risk exclusively influence goal changes and plan changes. Out of the latter variable group only the manpower constraints variable directly af- fects plan changes. This result stands in contrast to the correlation analysis (Table 3) that shows signifi- cant correlations with four contextual variables. Only by simultaneously estimating the influences of the context variables on goal-changes and the impact of goal-changes on plan-changes can we build a more accurate model. The main source for plan changes, besides goal changes according to these results, is shortage in manpower to execute the project. Two counter intuitive results come out of our anal- ysis. First, the quality of planning has no significant impact on plan-changes and second, plan-changes have no significant direct effect on customer satisfac- tion. These two findings have important theoretical and practical implications. As of today, the project management literature discusses project planning as a single entity. From that perspective, it is only natural to assume that a higher quality of project planning will reduce the number of plan changes. However, our study does not support this view. The causes for changing project plans are obviously contextual-related or driven by goal changes. Addressing the second finding requires going back to the differences between goal-changes and plan-changes. While goal changes are mostly the re- sult of a common agreement between the customer and the project manager, there are many internal reasons for plan changes that do not interest the customer. The results also indicate that plan changes are basically efficiency-oriented. They reflect changes in a project plan but do not necessarily impact the end-product. That may be one explanation for the non-significant relation between plan-changes and customer satisfac- tion. These results suggest further investigations by using a more precise definition of the two types of changes and measuring the impact of these different aspects on the two criteria of project success. 10.1. Implications and outlook In contrast to the general understanding, this study sheds a different light on project planning. Projects are temporary, unique and ongoing tasks. It is unimag- inable that such tasks can be performed without any changes at all. The results of this study support this view but it seems that the essence of changes is even stronger than that of planning, and indeed, while plans are not nothing, “changing plans is everything.” By analyzing the interactions between the three planning variables, project success and contextual in- fluences this study contributes to theory building in three aspects. First, it explores the prominent negative impact of changes, especially goal changes, on project success. Second, it reveals the interaction structure between the planning variables themselves and their interaction with the two dimensions of project suc- cess. And third, it identifies several contextual vari- ables, which affect the quality of project planning and stimulate goal and plan changes. The most im- portant result is that the amount of changes during a project’s implementation clearly distinguishes be- tween successful and failed projects. These results point to several managerial implications. The quality of project planning affects the project success, but the major lesson is: while it is impossible to prevent project changes at all, they should be kept to a minimum. The analyzed interactions between the planning variables indicate a very high and positive connection between goal changes and plan changes; the number of plan changes is strongly affected by goal changes. It is therefore in the hands of the PM to control the negative effect of plan changes by care- fully screening out from all proposed goal changes only those which are really essential to the success- ful implementation of the project. Eventually project goal changes will negatively impact project success. Other measures to avoid the strongly negative effects of goal changes are to freeze the requirements and the design at earliest possible stage. Shenhar and Dvir (1996) however have shown that the appropri- ateness of design freeze points in a project depends upon its level of technological uncertainty. The higher the uncertainty, the later will be the point of design freeze. For prevention of changes in later project phases—project managers should invest in upfront activities to capture the ‘real’ requirements of the customer (Dvir, et. al., 2002). A true reflection of cus- tomer requirements at the initial phase of the project can significantly reduce the amount of changes in later phases.
- 13. D. Dvir, T. Lechler / Research Policy 33 (2004) 1–15 13 Besides the interactions between different planning variables, this study shows how contextual variables affect project planning. Such variables can be divided into two groups: those over which project managers have almost no control and those over which project managers have some level of control. The negative in- fluence of the second group of contextual influences can be mitigated by a high quality of project planning. The quality of project planning is to a large extent in the hands of the PM, and since the influence of plan- ning quality on goal changes is much stronger than the contextual variables, the PM can limit their im- pact. One contextual variable affecting both plan and goal changes is the manpower constraints variable. Therefore, in order to reduce the number of goal and plan changes at the same time it is the PM’s prime responsibility to assure that the required human re- sources for proper execution of the project are firmly secured. Although our basic model is mainly confirmatory, it has some limitations. One limitation is linked to the static treatment of the data. The measurements are all ex-post and therefore do not allow analysing the impact of changes over time. It is quite probable that the impact of goal and plan changes could vary over the life cycle of the project. Another limitation is the common method variance problem associated with such designs. This is related to the choice of the key informants (George and Torger, 1982), where the same person is asked about his/her activities and their outcomes. This approach is crit- icized by several authors who question the validity of the results (Allen et al., 1988; Cooper, 1979; Campbell, 1982; Bedeian, 1988). Crampton and Wagner’s (1994) meta-analysis does not support this criticism. Investigating 581 field studies and analyz- ing 42,934 correlations, they conclude that the risk of distorted results through the use of key informants’ self-ratings is relatively small. Another aspect in reducing the risk of a response bias caused is the exclusive selection of pairs of clearly ‘successful’ and ‘not successful’ projects. The comparison of ex- treme examples, which the respondents in this study are asked to draw, reduces the possibility of biased information. According to Duffy et al. (1998), the goal of such a study is to analyze higher order inter- actions between factors. They argue that it is unlikely that the respondents take into account the interactions between the factors and manipulate their answers in a specific pattern. Our study opens opportunities for further research. The low percentage of explained variance of the two change variables indicates an important research di- rection: the search for explaining variables. The causes of goal changes are not fully explored. A more accurate definition of change variables is required too. Such a definition may enable further in- vestigation into the interactions between the contex- tual variables and the change variables and how goal changes could be managed appropriately. Finally, we propose further research, focused on the life cycle of projects to deepen the understanding of the interac- tions between plan and goal changes as well as their impact on project success. Acknowledgements The Authors are thankful for the important and valu- able remarks of their colleague Prof. Aaron Shenhar who reviewed our first draft and helped to create a clear and more focused paper. References Allen, T., Katz, R., Grady, J.J., Slavin, N., 1988. Project Team Aging and Performance: The Roles of Project and Functional Managers. R & D Management 18 (4), 295–309. Andersen, E.S., 1996. Warning: activity planning is hazardous to your project’s health!. International Journal of Project Management 14 (2), 89–94. Andersen, E.S., Grude, K.V., Haug, T., 1995. The Goal Directed Project Management, second ed. Kogan Page, London. Anderson, J., Gerbing, D., 1988. Structural equation modeling in practice: a review and recommended two-step approach. Psychological Bulletin 3, 411–423. Armstrong, S.J., 1982. The value of formal planning for strategic decisions: review of empirical research. Strategic Management Journal 3, 197–211. Bagozzi, R., 1980. Causal Models in Marketing. Wiley, New York. Balachandra, R., 1984. Critical signals for making Go/NoGo decisions in new product development. Journal of Product Innovation Management 2, 92–100. Balachandra, R., Friar, J.H., 1997. Factors for success in R&D projects and new product innovation: a contextual framework. IEEE Transactions on Engineering Management 44 (3), 276– 287. Baker, N., Murphy, D., Fisher, D., 1988. Factors affecting project success. In: Cleland, D.I., King, W.R. (Eds.), Handbook of Project Management. Van Nostrand Reinhold, New York.
- 14. 14 D. Dvir, T. Lechler / Research Policy 33 (2004) 1–15 Bart, C.K., 1993. Controlling new product R&D projects. R&D Management 23, 187–197. Baumgartner, H., Homburg, C., 1996. Applications of structural equation modeling in marketing and consumer research: a review. International Journal of Research in Marketing 13, 139– 161. Bedeian, F.W., 1988. An empirical investigation of self-appraisal- based performance evaluation. Personnel Psychology 41, 141– 156. Bentler, P.M., 1990. Comparative fit indexes in structural models. Psychological Bulletin 107, 238–246. Bentler, P.M., Bonett, D.G., 1980. Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin 88, 588–606. Bollen, K., 1989. Structural Equations with Latent Variables. Wiley, New York. Browne, M.W., Cudeck, R., 1993. Alternative ways of assessing model fit. In: Bollen, K.A., Long, J.S. (Eds.), Testing Structural Equations Models. Sage, Newbury Park, pp. 136–162. Bryson, J.M., Bromiley, P., 1993. Critical factors affecting the planning and implementation of major projects. Strategic Management Journal 14, 319–337. Campbell, J., 1982. Some remarks from the outgoing editor. Journal of Applied Psychology 67, 691–700. Chatzoglou, P.D., Macaulay, L.A., 1996. A review of existing models for project planning and estimation and the need for a new approach. International Journal of Project Management 14 (3), 173–183. Cooper, R.G., 1979. The dimensions of industrial new product success and failure. Journal of Marketing 43, 93–103. Cooper, R.G., Kleinschmidt, E.J., 1987. New products: what separates winners and losers. Journal of New Product Manage- ment 4, 169–184. Crampton, J., Wagner, S.W., 1994. Percept-percept inflation in micro-organizational research: an investigation of prevalence and effect. Journal of Applied Psychology 69, 67–76. Dawson, R.J., Dawson, C.W., 1998. Practical proposals for managing uncertainty and risk in project planning. International Journal of Project Management 16 (5), 299–310. Duffy, M.K., Shaw, J.D., Ganster, D.C., 1998. Positive affectivity and negative outcomes: the role of tenure and job satisfaction. Journal of Applied Psychology 83 (6), 950–959. Dvir, D., Lipovetsky, S., Shenhar, A., Tishler, A., 1999. Common managerial factors affecting project success. Working paper, Tel Aviv University, School of Management, Tel Aviv. Dvir, D., Raz, T., Shenhar, A.J., 2003. An empirical analysis of the relationship between project planning and project success. International Journal of Project Management 21 (2), 89–95. Freeman, M., Beale, P., 1992. Measuring project success. Project Management Journal 23 (1), 8–17. Fritz, W., 1992. Marktorientierte Unternehmensführung und Unternehmenserfolg, Grundlagen und Ergebnisse einer Empiri- schen Untersuchung. Gabler, Frankfurt. George, J., Torger, R., 1982. The reliability and validity of key informant data from dyadic relationships in marketing channels. Journal of Marketing Research 11, 517–524. Grinyer, P., Al-Bazzaz, S., Yasai-Ardekani, M., 1986. Towards a contingency theory of corporate planning: findings in 48 UK companies. Strategic Management Journal 7, 3–28. Gutierrez, G.J., Kouvelis, P., 1991. Parkinson’s law and its implications for project management. Management Science 37 (8), 990–1001. Jöreskog, K., Sörbom, D., 1993. LISREL 8 User’s Reference and Guide. Scientific Software Inc., Mooresville. Kerzner, H., 1998. In Search of Excellence in Project Management: Successful Practices in High-Performing Organizations. Van Nostrand Reinhold, New York. King, W.R., Cleland, D.I., 1988. Life−cycle management. In: Cleland, D.J., King, W.R. (Eds.), Project Management Handbook. Van Nostrand Reinhold, New York. Laufer, A., Kusek, J., Cohenca-Zall, D., 1997. Taking the sting out of project surprises. Optimum 27 (3), 1–7. Larson, E., Gobeli, D., 1988. Organizing for product development projects. Journal of Product Innovation Management 5, 180– 190. Lechler, T.G., 1997. Erfolgsfaktoren des Projektmanagements. Lang, Frankfurt. Lipovetsky, S., Tishler, A., Dvir, D., Shenhar, A., 1997. The relative importance of defense projects success dimensions. R&D Management 27, 2. Maylor, H., 2001. Beyond the gantt chart: project management moving on. European Management Journal 19 (1), 92–100. Meyer, M., Utterback, J.M., 1995. Product development cycle time and commercial success. IEEE Transactions on Engineering Management 42 (4), 297−304. Might, R., 1984. An evaluation of the effectiveness of project control systems. IEEE Transactions On Engineering Manage- ment 3, 127–137. Mintzberg, H., 1994. The Rise and Fall of Strategic Planning: Re- conceiving Roles for Planning. The Free Press, New York, USA. Murphy, D., Baker, N., Fisher, D., 1974. Determinants of Project Success. Boston College, National Aeronautics and Space Administration, Boston. Peters, T., 1988. The mythology of innovation, or a skunkworks tale, part 2. In: Tushman, M.L., Moore, W.L. (Eds.), Readings in the Management of Innovation, second ed. Harper Business, New York, pp. 138–147. Pinto, J.K., 1986. Project implementation: a determination of its critical success factors, moderators and their relative importance across the project life cycle. Dissertation at the University of Pittsburgh, Pittsburgh. Pinto, J.K., Mantel Jr., S.J., 1990. The causes of project failure. IEEE Transactions on Engineering Management 37 (4), 269– 276. Pinto, J.K., Prescott, J.E., 1990. Planning and tactical factors in the project implementation process. Journal of Management studies 27 (3), 305–327. Pinto, J.K., Slevin, D.P., 1987. Critical factors in successful project implementation. IEEE Transactions Engineering Management 34 (1), 22–27. Pinto, J.K., Slevin, D.P., 1988. Project success: definitions and mea- surement techniques. Project Management Journal 19 (3), 67– 73.
- 15. D. Dvir, T. Lechler / Research Policy 33 (2004) 1–15 15 Posten, R.M., 1985a. Preventing software requirements specifi- cation errors with IEEE 830. IEEE Software 2 (1), 83–86. Posten, R.M., 1985b. Selecting software documentation standards. IEEE Software 2 (3), 90–91. Project Management Institute Inc., 2000. A Guide to the Project Management Body of Knowledge (PMBOK), Project Management Institute, Pennsylvania. Ramanujam, V., Venkatraman, N., 1986. Planning system charac- teristics and planning effectiveness. Strategic Management Journal 8, 453–468. Rothwell, R., Freeman, C., Horsley, A., Jervis, V., Robertson, A., Townsend, J., 1974. SAPPHO updated—Project SAPPHO phase II. Research Policy 3, 258–291. Rubenstein, R., Chakrabarti, A., O’Keefe, R., Souder, W., Young, H., 1976. Factors influencing innovation success at the project level. Research Management 5, 15–20. Rhyne, L.C., 1986. The relationship of strategic planning to finan- cial performance. Strategic Management Journal 7, 423–436. Shenhar, A.J., 1993. From low to high tech project management. R&D Management 23 (3), 199–214. Shenhar, A.J., Dvir, D., 1996. Toward a typological theory of project management. Research Policy 25, 607–632. Shenhar, A.J., Dvir, D., Levy, O., 1997. Mapping the dimensions of project success. Project Management Journal 28 (2), 5– 13. Sinha, D.K., 1990. The contribution of formal planning to decisions. Strategic Management Journal 11, 479–492. Soni, K., Lilien, G.L., Wilson, D.T., 1993. Industrial innovation and firm performance: a re-conceptualization and exploratory structural equation analysis. International Journal of Research in Marketing 10, 365–680. Souder, W., Chakrabarti, A., 1978. The R&D/marketing interface: results from an empirical study of innovation projects. IEEE Transactions On Engineering Management 4, 119–125. Tatikonda, M.V., Rosenthal, S., 1999. Successful Execution of Product Development Projects: The Effects of Project Management Formality, Autonomy and Resource Flexibility. The Academy of Management Conference, Chicago, IL. Turner, J.R., 1993. The Handbook of Project-based Management. McGraw-Hill, London. Turner, J.R., Cochrane, R.A., 1993. Goals and methods matrix: coping with ill defined goals and/or methods of achieving them. International Journal of Project Management 11 (2), 93– 106.