Chandler, Plato and Magic
Have you watched the popular sitcom Friends? In one of the show's episodes, Chandler needed to visit a tailor and his friend Joey suggested him a tailor. On visiting the tailor Joey suggested, Chandler was left scandalized as the tailor while taking measurements had proceeded to make inappropriate contact with Chandler’s groin. When Chandler informed Joey about this, Joey went on to state how this was normal as that is how tailors "measure the in-seam". It turned out that Joey and his father had never been to any other tailor in their entire life. To them, tailors were supposed to do that.
The Greek philosopher Plato presented us with an allegory in which prisoners were chained to the walls of a cave. A fire was burning behind these prisoners which led to shadows being cast on the wall they were facing. Since these men had not experienced anything in the outside world, they only knew about these shadows. To them, these shadows were real people. When one of the prisoners was freed from the cave he walked out into the world and eventually understood that the shadows on the wall were not real people. The man returned to the cave to tell the prisoners about the "real world". However, the prisoners in the cave shunned him away. The cave was their reality.
Our perception of reality often tends to differ from reality. We do not realize that the information that we have gathered may be incomplete. Both Joey and the cavemen had not experienced a world outside their personal experiences and so to them, their experiences were normal. That was their reality.
I’m going to randomly jump to one of my favorite hobbies - magic. One of the oldest tricks in magic is the famous cups and balls. While there are a zillion versions of the trick the basic premise remains the same. There are three cups and three balls. One by one the magician makes the balls disappear and reappear under the cups. Then he makes the ball transport from cup to cup until of course, the balls have transformed into bigger balls or lemons. So how are the balls jumping from cup to cup and then transforming into lemons?
One of the sleights used in the cups and balls routine is known as the French Drop. It involves holding a ball(or coin ) in your right hand and (apparently) moving it to your left hand. While your audience’s gaze (and attention) shifts from your right hand to your left hand, the ball never actually leaves your right hand. Sounds simple enough, right? It becomes even more magical when you add a magic wand to the equation.
While performing the French drop you never really see the ball change hands. You see one hand cover the other and then what you see is a closed fist and the coin has vanished. If you look at the other hand you see a magic wand which again directs your attention back to the closed fist. You combine all this information together and end up fooling yourself.
As a spectator, we tend to put out faith in what we think we see in front of us. Unfortunately, we forget that we don’t see everything. Therein lies the issue.
Think of Paul the Octopus. Paul was famous for predicting the result of football(soccer) matches in the 2010 world cup. Paul's keepers would present him with two boxes containing food. The boxes would be identical apart from the fact that they were decorated with the different team flags of the competitors in an upcoming football match. Whichever box Paul ate from first was considered his prediction for which team would win the match.
Assuming that there was no foul play in this method, Paul made eight predictions and all were correct. The odds of that occurring were 0.39%. Now that seems remarkably low and hence if we were to just look at that stat we may conclude that Paul had magical powers that allowed him to predict football games. However, we forget to incorporate what we don't see, or what is not published in the news.
In this case, we forget to incorporate all the other octopuses, monkeys, rabbits, and parrots who also tried to predict the matches but failed. If we assume that 1000 animals attempted to predict eight matches in a row, probabilistically 3.9 of them would have predicted all eight correctly. Now of course animals failing to predict football matches is not news and hence not talked about. However, Paul got lucky and became an international star.
We can go all the way back to World War I to get another example of the same problem. When steel helmets were introduced for soldiers, the number of head injuries suffered in battle increased. Were helmets bad? As you may have guessed, not really. Previously an injury to the head would have been fatal and hence very few people with head injuries survived to tell the tale. Once the head started getting protection the fatality rate reduced but the injury rate increased.
All these are examples of Survivorship Bias. The logical error of concentrating only on the winners and coming to conclusions by focusing on what is easily visible.
In the financial markets, we often make decisions on published fund data. Unfortunately, this published fund data does not factor in the funds which have gone bust. This skews the data and hence our decision-making.
The problem with data and statistics is that we often make decisions based on what we see in front of us. Magic ofcourse acts as a gentle reminder that what we don’t see also matters. (sometimes more than what we see)
For more on survivorship bias, you can refer to this article.