Why do projects perform badly?

A few weeks ago, the ‘i’ Weekend newspaper and The Financial Times contained an article ‘Why projects always take longer than expected’ written by Tim Harford, who also presents BBC Radio 4’s programme ‘More or Less’. His profile gives him influence and credibility – I certainly listen to some of his broadcasts with interest. In his article, he made a comparison between delays to the Covid-19 vaccine manufacturing programme and construction megaprojects by reference to the research by psychologists Daniel Kahneman and Amos Tversky in 1977 examining underestimation of task durations and Professor Bent Flyvbjerg’s research into the propensity of megaprojects to exceed their cost estimates.

Professor Flyvbjerg burst on to the scene around the turn of the millennium with the publication of several articles and gained a higher profile when the book ‘Megaprojects and Risk – An Anatomy of Ambition’ was published in 2003.  He has continued working in the same field, pursuing the same theory since. His theory – that persistent overspending cannot be explained by error; therefore, megaprojects are initiated based on deliberate understatement of their capital cost, duration and overstatement of their benefits – politely known as ‘strategic misrepresentation’, (sometimes less politely as lying).

Persistent overspending cannot be explained by error

This theory has been widely accepted. Flyvbjerg advocates the addition of uplifts to offset what he sees as a systematic understatement, known as ‘optimism bias’.  His work is very influential: he advises governments and acts as an expert at enquiries and, as evidenced by Tim Harford’s article, is frequently taken as a definitive explanation for megaproject overruns.  ‘Optimism bias’ neatly answers the question ‘Why do projects perform badly?’

Let me say, I agree that for some projects, there is ‘strategic misrepresentation’; in fact, I have some personal experience of it (as I suspect many people in the industry have). But is this the only reason for the poor performance of projects? I believe the answer is ‘no’, but as ever with complex questions, this is nuanced.

Is it realistic to compare psychologists research into underestimation of simple everyday tasks with the effort exerted by experienced contractors, consultants, and clients, both government and private, that draw on years of experience?

Second, there is debate about the validity of Flyvbjerg’s findings and the data he bases them on. He admits that a proportion of megaprojects are successful.  Hardford’s article states that Flyvbjerg asserts that 90% of megaprojects have overruns (but doesn’t say by how much). Others, such as Ed Merrow, in his book ‘Industrial Megaprojects Concepts, Strategies and Practices for Success’ report that around 66% of these projects overran their schedules and budgets by more than 25%.

Distinguish between projects

How, though, do we distinguish between projects that will be successful and those that aren’t? Presumably, if we could address that conundrum, there wouldn’t be a problem at all, unless, during their execution, management performance is unexpectedly poor – and from time-to-time it is.

At Kingsfield, we encounter badly managed projects, often caused by poor decision making, inexperience, or simply a lack of understanding of each party’s contractual obligations and ability to cope with the risks allocated to them.

If overruns are caused by poor management, shouldn’t the industry address the reasons for this rather than build them in to cost estimates, so the mismanagement is included in the budget?

Projects can also be badly affected by external events, as the current pandemic illustrates. There are other circumstances such as recessions in the world economy or random events like the fire at Kings Cross underground station in November 1987, which claimed 131 casualties including 31 lives, just as the Channel Tunnel project was commencing its construction phase, which had a severe impact on the public perception of the project’s fire safety with consequential knock-on into the design, leading to significant cost increases.

Projects are mismanaged, and they change, but is that entirely attributable to over-optimism or misrepresentation?

Professor Flyvbjerg’s solution to all overruns is to add uplifts to counter ‘optimism bias’ to the base estimate. The precise figure depends on the phase, sector and type of project but how is the precise figure to be calculated, if such uplift is necessary at all. How exactly do you calculate the ‘base estimate’? Does it include any contingency? How do you know if it does, and how much? What risks are covered and what are not?

Flyvbjerg used largely historically published data going back to 1910 to justify his findings and a number of recent and contemporary projects that provided first-hand rather than secondary (reported) sources of data. He admitted that in some cases, little detail existed beyond the headline figures. He also made a simple comparison between the very first published cost estimate with the actual final cost.

While this is frequently done, it does not help to understand if there was mismanagement or any unforeseeable external risk events. Nevertheless, he has robustly defended his position and asserts that statistical analysis proves the existence of ‘strategic misrepresentation’ by parties with vested interests in a project proceeding. From this assumption flows the conclusion that the only way to counter this is to systematically increase estimates to compensate for this behaviour.

One of Flyvbjerg’s critics – and there are several – is Peter Love, an Australian academic who has published several papers arguing against reliance on uplifts for ‘optimism bias’; one provocatively titled ‘Debunking Fake News in a Post-Truth Era: The Plausible Untruths of Cost Underestimation in Transport Infrastructure Projects.’ (This can be an emotive subject.) In another paper, Love and a team of researchers examined US$6.5 Billion of projects in Hong Kong, comparing their final cost with the estimates prepared prior to construction, rather than the initial estimate. They found that only 57% could be explained by ‘optimism bias’, but 43% did not, and of those, several underruns were reported.

Does this and similar research completely undermine the case for offsetting underestimation ‘optimism bias’ by uplifts to increase cost estimates?

Traditionally research into the problem of cost and schedule overruns identify reasons such as scope growth, the impact of changes, poor planning, poor data for estimating, insufficient ‘front end’ effort to define projects, and so on. Merrow focussed on improving the latter as his preferred solution to reducing the frequency of overruns. Despite efforts in this area, including systematic processes to address risk, improved integration of design, schedule, estimating, and document management few have delivered the benefits that were claimed for them.  None of them are ’silver bullets’.

Love calls these approaches the ‘project management paradigm’ for improving project performance, as opposed to the ‘governance paradigm’, which encompasses the use of ‘optimism bias’ uplifts and development of contracts to encourage collaborative working and the use of more equitable risk-sharing. These have had some success, but equally, there are failures too.

Sensibly, Love suggests that a combination of the two approaches is needed: improving project management whilst acknowledging that there are circumstances that require the use of judgement and experience to allow for uncertainties and risk to counteract excessive optimism, rather than relying solely on questionable analyses and dubious analysis of historical data.

The construction industry needs to counteract the perception that there exists a simple answer to complex problems. Forecasting and estimating future costs and performance is difficult. There are no ‘silver bullets’.

Paul Jobling – Associate Kingsfield Academy