The survival bias and what it reveals about your decisions in business and communication
Perhaps Hitler would have thought of a plan B if he had known what Wald was going to discover later.
The son of a baker and grandson of a rabbi, Abraham Wald studied mathematics in Vienna, where he was a researcher. With the German occupation and a new Nazi director, emigrating became inevitable.
In New York, he joined a secret statistical research group that contributed to the war effort. The group’s contribution was made in “equations”
The problem
That day they received a request from the military. They needed to armor their planes against the enemy’s fighters. They thought that a mathematical equation could find the right spot.
They showed planes returning from missions, covered in bullet holes in some areas. The distribution of these holes was not uniform. There were many holes in the wings and almost none in the engine. The logic was then to armor the affected area more, right?
Armor makes planes heavy. Heavy planes are harder to fly and use more fuel.
Too much armor is bad. Too little armor is also bad. How could they make the best decision?
The solution
Wald disagreed. The reinforcement to armor the plane had to be where there were no holes.
“Where were the missing bullet holes? They were in the missing planes.” The reason the planes came back with few bullet holes in the engine was simple: the ones that had been hit hard in the engine simply didn’t come back.
And they reinforced the armor on the engine. How did Wald see what the others didn’t?
Because the others made an assumption: the planes that came back were a random sample of all the planes that flew missions. For a mathematician, the bullet hole problem is seen as a phenomenon called “survivor bias” (a distorted view in the observation and analysis of identified facts).
“Survival bias”
When we want to develop a project, a brand, a service, it’s common to study the profile we’re attracting, the audience we’ve won over, but what about studying those who don’t stay? What information can you give us?
How many ideas have been shelved because they didn’t work straight away? What criteria did we use to conclude that they didn’t work?
It’s important to see what’s going well, to focus on the clients we’ve attracted, the projects where we’ve been successful. And it’s equally important to ask the questions that help us see our survival bias.
Every time I send out a proposal, I ask for a response. It may come some time later, but I need to know if they’ve already decided and, if they’ve decided otherwise, what the main reason was. 90% of the time I get it. All the answers help me to situate myself and know where I need to strengthen my armor. This requires organization, resilience and a lot of focus. It’s easy to forget about everything.
It’s common to look at what has worked. In phases like the one we’re going through, it’s important to see if other things could have worked with a different “armor”.
What have you learned from what didn’t work? Are you focusing on the right people? Is the strategy geared towards you or your target audience? It seems clear to you, but is the message just as easy to understand for others?
Now that many of the things that “worked” can no longer “work”, what are you looking at?
(Photos: Cameron Moll, Pixabay, Jens Mahnk)
