In spite of massive investments in project management best practices and the organizations to implement them, major oil and gas projects continue to experience painful cost overruns and schedule delays. Although there are many hypotheses as to why this is so, one root cause that has not been sufficiently explored is the built-in bias of owner and contractor organizations toward optimism and overconfidence. This article will explore the sources of this bias, and how techniques, such as using an outside view and improving the understanding of the drivers of risk exposure, can help correct it.
Investment Decisions Unknowingly Based on Optimistic Predictions of Cost and Time
The late management guru Peter Drucker said it best: Management by objectives works if you have the right objectives. Ninety percent of the time, you don’t. This describes the state of major capital projects in the exploration and production (E&P) sector today.
It is clear that oil and gas facilities projects are not starting off with the right objectives. Although E&P decision executives sanction and fund projects based on cost and schedule predictions they believe to be reasonable, this confidence is often misplaced.
Most decision makers assume their sanction-quality estimates of project cost and time have a 50% or greater chance of not overrunning. However, our firm’s analyses of a diverse population of projects indicate that the actual probability is much less, typically in the range of 5% to 25% (Fig. 1).
The tendency to optimism is confirmed by historical data. Speaking at the SPE Annual Technical Conference and Exhibition in San Antonio last year, Edward Merrow, president and chief executive officer of Independent Project Analysis, noted that only 22% of recent, large E&P projects demonstrated project performance success (Boschee 2012). In another study, PricewaterhouseCoopers found that only 2.5% of major projects met all their objectives of scope, cost, schedule, and business benefits. This is nothing new. Studies of oil and gas industry project performance in the 1990s (McVay and Dossary 2012) showed an average return on capital of 7%, even though hurdle rates (the minimum acceptable rate of return on a capital investment project) were typically 15% or more. The cause of this underperformance was attributed to poor project evaluation and selection due to chronic bias.
Clearly, major oil and gas projects are being sanctioned with optimistic objectives, and the inevitable cost overruns and delays can affect an organization’s financial strength.
There is some irony in this. For the past 2 decades, E&P organizations strengthened their project management competencies and improved work processes. Thousands of jobs have been created and tens of millions of dollars spent, all in the pursuit of predictable, cost-effective projects via front-end loading, internal cold eye (objective) reviews, and other best practices.
Why then are more projects than ever failing to meet their goals? The answer is surprisingly obvious. These best practices have failed to address the root cause of poor project performance: the built-in bias in the estimates of project cost and time. Ironically, the best practices, in which so many organizations have invested so much, are among the key drivers of this built-in bias.
Organizations Unintentionally Encourage Bias Toward Optimism and Overconfidence
An unintended consequence of improved project organizations has been to create a culture in which top-down and bottom-up mind-sets reinforce a bias toward optimism and overconfidence. Fig. 2 illustrates the drivers of bias.
Not only does the combined effect of these drivers cause capital investment plans and decisions to be based on unrealistic predictions of project results but, even worse, this bias also is typically unrecognized or ignored. Project managers who recognize these issues and present realistic estimates that consider all sources of risk are likely to be accused of “sandbagging.” Instead, they may prefer to present the optimistic estimate that will get the project approved, and hope that someone else will take the blame when the inevitable overruns occur.
Optimism and Overconfidence Destroy Project Value (and Careers)
Author and humorist Garrison Keillor describes the imaginary Lake Wobegon as where “all the women are strong, all the men are good-looking, and all the children are above average.” It is funny, since it captures the universal human tendency to be optimistic and overconfident.
Numerous studies over the years (Capen 1976) have demonstrated that, when asked to estimate something, professionals inevitably overstate the confidence level of their estimates. For E&P projects, this appears as optimistic estimates of cost, time, and profitability, compounded by overconfident estimates of the ranges of possible outcomes for these variables.
It is important to understand the difference between optimism and overconfidence, and the compounding effect of both, as illustrated in Fig. 3:
- Optimism results in an estimate of time or cost that is lower than the expected outcome when all risks are considered. In Fig. 3, the mean of the probability distribution on the left represents an optimistic projection of outcomes, because it is a lower value than the mean of the fully risk-adjusted distribution on the right.
- Overconfidence results in setting the range of possible outcomes too narrowly. In Fig. 3, the width of the probability distribution on the left represents a narrow range of potential outcomes, indicating high confidence that any deviation from the estimate will be small. The fully risk-adjusted distribution to the right is wider, reflecting the potential impact of all risks.
The combination of optimism (which understates the mean) and overconfidence (which understates the variance) can be devastating. If the board of directors and joint venture partners have sanctioned a project based on the cost estimate and probability of overrun described by the distribution on the left side of Fig. 3, it is easy to see the probability that the estimate will not be overrun, shown as the shaded area, is quite low. This is a good way to understand the root cause of the cost overruns illustrated in Fig. 1.
An Outside View Overcomes Built-In Bias
There is a way to overcome built-in bias. Bent Flyvbjerg, who has studied the causes of megaproject cost overruns, concludes that a root cause of bias is that decision makers rely entirely on an inside view of project costs and benefits (Flyvbjerg 2012). He recommends due diligence as a way to provide an outside view and reduce estimate bias. This view is echoed by Nobel laureate Daniel Kahneman in his recent international best-seller, Thinking, Fast and Slow.
Most projects are planned and estimated, and their risks assessed, using an inside view. The inside view is focused on the project itself, what is known and unknown, what risks and uncertainties it faces, and how it plans to overcome them. The engineers are apt to be familiar with its strengths and weaknesses, justifiably proud of their design and planning work, and confident that they can execute it according to plan. The industry’s emphasis on front-end loading, in which considerable time and effort is spent on engineering and planning the project, encourages this way of thinking. The result is an inside view with a bias toward optimism and overconfidence.
The best way to overcome it is with an outside view. In the project context, it requires a due diligence process that calibrates the predictions of cost and duration against the reality provided by the experience of other owners with similar projects.
An obvious approach is to compare the project at hand with one or more past, similar projects. The challenge is that the data set of truly comparable projects is apt to be small; this is particularly true of megaprojects. Moreover, meaningful comparisons between projects are difficult, since the data must be normalized to account for variations in factors such as:
- Time frame—global and regional economic trends and market conditions for project materials and services
- Location—metocean conditions, logistics, weather windows, water depth
- Subsurface—reservoir characteristics, flow rates, gas/oil ratio
- Technology—type of facility and process and drilling configuration
- Engineering—design standards, regulatory requirements, owner preferences
- Organization—capabilities of the owner/operator and joint venture partners
- Execution—labor availability, types of contracts, fabrication yard location and capabilities, offshore installation methods
The normalization process is further complicated by the fact that different owners and contractors are involved in each project and the critical data are not in the public domain. Even if it were possible to properly account for these variations, past project performance is not necessarily a meaningful predictor of future project outcomes.
Another way to facilitate due diligence is to focus on the key drivers of overall cost and time. Some examples are the availability, productivity, and cost of construction labor, the duration of the permitting process, the price of key commodities, and the potential effect of design changes associated with a given technology. These data are common to a broad class of projects, are more readily available, and are likely to be more relevant. However, it is not enough to make direct comparisons with past projects, as these parameters change with time. Given the unprecedented escalation in project costs that began in 2005 (costs have virtually doubled since then), and the increase in volatility that accompanied the financial crisis, it is clear that forward-looking risk analyses must remain part of any outside view.
Outside View Considers Strategic Risks
The authors’ independent risk assessments of major projects indicate that most cost and time overruns are not due to the types of risks that most project teams manage with risk registers and fund with contingency. We refer to these project-level risks as “tactical.” Project managers and teams can assess and control them. Most project teams, following best practices, do a good job of identifying and managing tactical risks. However, there is another type of risk, strategic risks, which project teams typically do not address. The broad perspective of the outside view improves the management of these risks.
Fig. 4 illustrates how the inside view focuses on tactical risks that project teams manage. These risks are covered by contingency. Since these risks are associated with engineering and planning definition, as the project moves through front-end loading and becomes better defined, the perceived level of risk is reduced, as is the level of contingency. The outside view can focus on the strategic risks.
Strategic risks include the following:
- Capability risks are associated with the ability of the owner and contractor organizations to provide the competencies needed to plan and execute the project. Megaprojects require more competencies than conventional projects, such as program management in which a large project is managed as an integrated group of subprojects. Capabilities in interface management are required, in which a large number of design, commercial, and organizational interfaces must be coordinated. Another capability is partner management, in which joint venture partners, with business models and project goals that are inevitably misaligned, must be aligned and work together effectively. Resource management capabilities plan for, acquire, and deploy a large number of engineering and construction resources.
- Commercial risks are associated with the agreements that drive project economics and cost/schedule objectives, such as feedstock, power, off-take agreements, as well as the contracts and subcontracts for design and construction.
- Geopolitical risks are associated with political changes in the region or location that affect major projects and may include regulatory, permitting, and taxation changes that affect project cost, time, and economics. The risks associated with nonfinancial stakeholders, such as nongovernmental agencies or local residents at the project site, are also factors.
- Economic and market risks are associated with prevailing global and regional economic conditions and the market trends for the materials, equipment, and services a major project requires. Megaprojects are particularly exposed to such risks because of the associated effect on demand.
The use of an outside view to determine strategic risk exposure is illustrated by the examples shown in Table 1, based on actual projects for which the authors provided an independent, outside view, risk assessment. The results are abridged for illustration purposes.
The Influence Curve Creates a False Sense of Confidence
The foundation of project management best practices rests on the influence curve, which states that an owner’s ability to influence project outcomes is highest at the early stages, and decreases rapidly thereafter.
Fig. 5 shows the influence curve in blue, representing the way many people interpret it: as an expression of the owner’s risk exposure. The inference is that the owner’s risk exposure is highest early in the project and then declines exponentially to sanction. If good front-end loading practices have been followed, decision executives are encouraged to assume that project definition risks have been mitigated, and the remaining execution risks are reasonably low. This often leads to a contracting strategy in which the execution risks are pushed to contractors.
What is missing is the concept of risk exposure. We define risk exposure as the product of probability, impact, and a factor representing the owner’s degree of control. The greater the owner’s control, the lower the risk exposure. True risk exposure is represented by the red line in Fig. 5. Early in the project, the owner has complete control of the work and little impact risk if the project should be deferred, so risk exposure is low. This situation changes dramatically after sanction. Now the owner is committed, large contracts are signed, control over the outcomes largely shifts toward the contractors, and the effect of major changes to the project plan is high. The owner’s true risk exposure rises rapidly and stays high until the project is complete.
Summary
The performance of major oil and gas projects provides an example of a condition described in a quote attributed to Albert Einstein: Insanity is doing the same thing over and over again and expecting different results. Even as best practices continue to be emphasized and reinforced, poor project performance persists.
We have suggested that the root cause is the built-in bias that virtually guarantees that estimates of cost and time will be optimistic and overconfident. We have supported the recommendations by others, such as Flyvbjerg and Kahneman, that an outside view be employed to correct for internal bias. Consideration of strategic risks is recommended, as is the use of risk exposure, to better understand the management of risks after sanction.
For Further Reading
Boschee, P. 2012. Panel Session Looks at Lessons Learned from Megaprojects. SPE Today, 10 October 2012.
Capen, E.C. 1976. The Difficulty of Assessing Uncertainty. J. Pet Tech 28 (8): 843–850.
Flyvbjerg, B. 2012. Quality Control and Due Diligence in Project Management: Getting Decisions Right by Taking the Outside View. International Journal of Project Management (in press; published online November 2012).
SPE 160189 The Value of Assessing Uncertainty by D.A. McVay and M.N. Dossary, Texas A&M University.
| Eric Briel is managing director at Westney Consulting Group. He can be reached at e_briel@westney.com. |
| Pete Luan is a principal at Westney Consulting Group and can be reached at p_luan@westney.com. |
| Richard (Dick) Westney is the founder of Westney Consulting Group. He can be reached at r_westney@westney.com. |