Survivorship bias in communication and business decisions
Survivorship bias shows how we can make poor decisions when we only look at the cases that reached the end. In communication and business, this happens when we only analyse clients won, proposals accepted or successful projects.
In this article
Perhaps Hitler would have thought of a plan B if he had known what Wald was going to discover later.
The son of a baker and the grandson of a rabbi, Abraham Wald studied mathematics in Vienna, where he worked as a researcher. With the German occupation and a new Nazi director, emigration became inevitable.
In New York, he joined a secret statistical research group that contributed to the war effort. The group’s contribution was made through “equations”.
The problem of survivorship bias
That day, they received a request from the military. They needed to armour aircraft against enemy fighters. They thought a mathematical equation could find the right point.
They were shown aircraft that had returned from missions, covered in bullet holes in some areas. The distribution of those holes was not uniform. There were many holes in the wings and almost none in the engine. The logic was, therefore, to add more armour to the area that had been hit, right?
Armour makes aircraft heavier. Heavy aircraft are harder to fly and use more fuel.
Too much armour is bad. Too little armour is also bad. How could they make the best decision??
Abraham Wald’s solution
Wald disagreed. The reinforcement used to arm the aircraft should be placed where there are no bullet holes.
“Where were the missing bullet holes? They were in the missing aircraft.”
The reason why aircraft returned with a few bullet holes in the engine was simple: those that had been badly hit in the engine did not return.
So they reinforced the armour around the engine. How did Wald see what others did not?
Because the others started from an assumption: that the aircraft that returned were a random sample of all those that had carried out missions. For a mathematician, the bullet hole problem is seen as a phenomenon called “survivorship bias” – a distorted view in the observation and analysis of the facts identified.
“How to apply survivorship bias to communication
When we want to develop a project, a brand or a service, it is common to study the profile we are attracting, the audience we have won over. But what about studying those who do not stay? What information can they give us?
How many ideas were put aside because they did not work straight away? What criteria did we use to conclude that they had not worked?
It is important to look at what goes well, to focus on the clients we attract and on the projects where we have been successful. And it is equally important to ask the questions that help us see our own survivorship bias.
Whenever I send a proposal, I ask for a reply. It may come some time later, but I need to know whether a decision has already been made and, if another option was chosen, what the main reason was. I manage to get one 90% of the time. Every reply helps me understand where I stand and where I need to reinforce the armour. This requires organisation, resilience and a lot of focus. It is easy to leave it forgotten at the bottom of everything else.
Even in the past, during recruitment processes, I always asked for feedback, to understand whether the choice had been financial, based on experience, training or something else. What I received was always useful to help me understand where I stood and guide the strategy I had.
It is common to look at what worked. In phases like the one we are living through, it is important to understand whether, with a different kind of “armour”, other things could have worked too.
What were you able to learn from what did not work? Are you focusing on the right people? Is the strategy aimed at you or at the audience you want to reach? It may seem clear to you, but is the message just as easy for others to understand?
Questions to identify survivorship bias
Before concluding that an idea does not work, it is worth asking: what data was left out? Which people did not reply? Which proposals did not move forward? Which clients did not return?
Survivorship bias is not only useful for analysing aircraft, numbers or statistics. It also helps us understand where communication may be failing and which signals remain invisible because they never reach us.
Now that many of the things that “worked” can no longer “work”, what are you looking at?
