Motivated by Lior Pachter’s end of year blog post from 2014, I dusted off an old essay I wrote posted on my lab webpage (with a little editing)…
If I walk into a casino and find the person who has won the most money at Roulette that night, what does that tell me about the Roulette skills of that person? We all know that roulette is a game of pure chance. Given enough players, there will always be somebody who has been lucky and won a lot of money. One night’s winnings tell me nothing about the ability of that winner. In fact, we also know that if somebody wins night after night, that indicates something much more shady than Roulette skills.
Some games have less randomness, say poker, but nevertheless the results of any single night or a single tournament is not likely to signal the winner’s skills. Chance plays a major role in the outcomes of even seemingly high skill games. It is a well-established result that Wall Street fund managers’ year-to-year performance is uncorrelated and the overall pool of managers fails to do better than the overall market.
In one of his books, Malcolm Gladwell discusses examples of where it is difficult to assess somebody’s potential at some task, say a NFL quarterback, when they are measured for performance that is not related. According to the experts, the college football game is very different than the NFL game at the quarterback position. Thus, the best college quarterbacks seldom continue their success at the pro level. One might consider this just a problem of the correct measurement. So, it would seem a no-brainer that if we make the right measurement, e.g., previous success rate, then we should be able to predict true skills and future success probability. But, if a game has a high chance component–which is the case for most tasks in life, a person’s track record may not be so informative. The individual might have been lucky. And, given enough players there are always winners.
The problem is pronounced when considering extreme achievements in complex games. Any extreme achievement is a rare event and complex games makes the results more prone to chance fluctuations. We know coaching in NFL requires skills. But, does winning the Super Bowl foretell the ability of a coach to lead another team to the same rare achievement? The tricky thing is that there is always a Super Bowl winner; always somebody who made a killing in the stock market, etc. That is, there is always somebody in a large group that achieves extreme success; like the winners at Roulette.
The above thoughts came to me as I mused on the many extremely accomplished biomedical scientists I have met—those at the very top. What struck me the most was the degree to which many were narrow in view and single minded with a high level of self-conviction. These traits may come from the standard ego boost, to which all of us would be susceptible if we had similar levels of achievement. But, it also felt like there was more than that. There were perfectly nice, collegial, even modest, people who nevertheless had very dogmatic and narrow scientific views.
Biology is one of the youngest of sciences addressing highly complex and unpredictable phenomena. While important discoveries have been many, not too many of them could be described as having been predicted prior to the actual fortunate discovery. Nature is constantly surprising us (which is why the discipline is so delightful to some) and many of the previous knowledge is quickly overturned, either wholly or at least in details. And, one would be hard pressed to find Nobel Laureates who achieved high impact results more than once.
Given such variable history of progress, why is the field so conservative about accepting new possibilities? The answer I believe is that under such unpredictability, the successful strategy is to focus narrowly, ignore negative evidence, and then get lucky. Given ten thousand narrowly focused scientists and ten thousand broadly minded scientists, it is more likely that great success will come from the former rather than the latter. This is a basic result of portfolio theory for high variance games. Biology is a field for hedgehogs than foxes.
One. I once read a sports story that described the typical post-career trajectory of highly accomplished tennis players. Tennis is solitary sport with as many opportunities for errors as for winners during game play. An important trait for players at the top is the ability to immediately forget errors during play lest obsession on the mistake impedes the next point. The article suggested that such traits, while great for tennis, do not translate well into the next stage, leading many of them to repeatedly make mistakes in life’s decisions. Given a selection game, whether the game is science, sports, or finance, the traits that result in success can be surprising, unanticipated, and not translate well to seemingly related tasks.
Two. It is hard to believe that a Nobel laureate might not be the genius, deep thinking, and creative person we expect them to be. In some fields, that might be the case, in others a different trait might be selected. We have certainly stopped being surprised when we find that sport stars don’t turn out to be social role models. We should stop being surprised when the most successful fund manager from last year wasn’t the person with the deepest analysis of the market but rather a crazy risk taker. We also shouldn’t project Einstein into every highly accomplished scientist. A person’s record is important but just as important is the characteristics of the game in which they are successful; how much chance plays a role, and what traits are actually being selected.
Three. I do not mean for this to be a knock on successful biologists. I am confident that if biology were filled with deep creative thinkers, and only such people, it would make no progress. Given the complexity of the material, every observation would throw things into doubt, open new inquiries that abandon previous views, and generally cause endless meandering. It is highly likely that Biology requires an army of single-minded people whose work collectively results in great discoveries because a few chance upon important phenomena.
Biology from the point of view of portfolio management:
–What is good for the individual may not be good for the field.
–And, what is good about the field (depth, creativity, high-risk exploration) may not be the good (trait) of the individual.
Interesting post! I think biology definitely has a fair amount of randomness in who makes what are perceived as the “big discoveries”.
An additional related point: “big discoveries” from a single person are often not quite as big as we post-facto declare. I think many discoveries are already in the air, and were the one lone genius scientist not there to make it, someone else surely would have (e.g. CRISPR). This is not to say that those who make discoveries are not good scientists, but rather that there is a lot of talent, and that most discoveries are really the accumulation of many observations and experiments. We afterward craft a narrative of the hero, but it’s probably inaccurate most of the time. So the roulette wheel is more about credit than actual discovery.
At the same time, while this seems sort of unfair, I think overall socialistic nature of science more than makes up for it. While it is true that everyone has a low probability of making/getting credit for a “major discovery”, it is also true that this probability varies a lot from scientist to scientist. The rewards of science are certainly not flatly distributed, but I think they are far less skewed than this probability distribution.