The risk of confirmation and conformation bias in project management
In April 2007, a leading European multinational in semiconductor manufacturing (guess which one) engaged in a 3-year R&D project for an automotive customer. The deadlines were strict, as the product was scheduled to be introduced into the vehicles of model year 2012. Processes were in place (CMM level 3), sophisticated project management tools had been applied and the competency levels in the teams were high. Still, the project failed. It was terminated after 85 weeks, wasting over 20 million euros. I suspect the relationship with the customer was seriously damaged.
It took 53 weeks until the teams realized they were in dismal shape. This is very nicely described in an article by professor Kim van Oorschot and colleagues. It details how teams emphasized the positive news during progress reviews, how first they took away time from future buffers (stretch and slim) and when that didn’t feel comfortable anymore, they started to lower future estimates, justified by increased competencies and new insights (stretch and squeeze). This was all done to live up to the committed deadlines. We all want to be reliable employees, don’t we? And we know our performance is often judged based on our ability to live up to promises.
The article also addresses the question of why the teams didn’t get alarmed when it was clear they were slipping against the project plan. They could have realized much earlier the resource build-up wasn’t progressing according to plan - “the project team was chronically understaffed throughout the project.”
This is an example of a mistake that’s made often: the assumption that the plan is correct and when the execution deviates, the execution is to blame, not the plan and its assumptions. I once attended a progress review where an experienced project manager believed the project was halfway since 50 percent of the budget had been consumed!
We so much like our plans (even weak ones) to be correct that we continuously look for arguments to confirm them. This is known as the confirmation bias. It’s also a well-known risk in scientific research: many people try to prove a hypothesis by offering as many examples as possible to confirm it. However, the proper scientific attitude is to attempt to falsify it. A hypothesis is more worthy to believe when many serious and independent falsification attempts fail rather than when we have many examples confirming it.
In my opinion, it’s best in projects to organize early falsification of our initial assumptions. That’s why modern insights in project management advocate structuring the project in small, concrete deliverables, rather than in goals for which it’s hard to determine whether they’re completed.
This isn’t the case for projects with a traditional linear project management approach and stage segmentation, such as the project described at the start of this article. It explains why in these projects, the project lead is happy the first 90 percent of the time but has sleepless nights during the second 90 percent! Small deliverables allow for early falsification of our assumptions, both for the required performance as well as for the technical and commercial feasibility. It enables flexibility in scope and the date of market introduction.
This all seems so obvious, and you may wonder why it’s not widely applied yet. Many of us have an academic degree and have promised to adhere to academic principles, but we seem to have forgotten all as soon as we join the industry. You can still find many people advocating traditional project management methods focused on a committed scope, budget and time. Often, they seem surprised extensive risk management and buffers don’t help.
I once asked an experienced project manager why he still applied these traditional methods, and I was shocked by his response. He answered he knew all that stuff and even believed it would be better, but if he did what most people do or what his boss thought was the best method, he had the lowest chance of being fired. That’s what I call the conformation bias. I’ve seen similar examples in academia, with people being afraid to lose their status when they no longer support what they’ve said before or what others say.
I’d like to repeat the advice I once got from Pieter Geelen, one of the founders of Tomtom: challenge the conventional. I believe that’s the right mindset to grow and achieve extraordinary results. Search for reasons why you’re wrong! But it requires courage and isn’t always easy. It requires an environment where people dare to take a risk and are allowed to fail. Thus, we arrive at the interesting paradox: environments that allow for failure and quick adaptation are less likely to fail.
I really hope one day I’ll understand why all of the above is wrong.
