Jeffrey Saltzman's Blog

Enhancing Organizational Performance

The Problem with Experts

with one comment

Say you had a group of 100 people who put themselves forth as experts in estimating the number of gumballs in jars. Divide that group of 100 into 2 groups of 50, and asked each person within the two groups to estimate the number of gumballs in a jar. With one group of 50 treat their estimates individually, 50 individual estimates of how many gumballs are in the jar. With the second group take the 50 estimates and average them together, so you have only 1 estimate. All together you now have 51 estimates of the number of gumballs. Out of these 51 estimates, which will be closer to the actual number of gumballs, the 1 averaged estimate or one of the 50 individual estimates? The answer is that most often one of the 50 individual estimates will be closest and the average estimate will often consistently be the second, third or fourth best estimate. Close but no cigar.  So now we look at the person who made the best individual estimate and crown that person “the” expert in gumball estimation. If we now run this experiment over and over, what you would find is that you are crowning a different “expert” each and every time. So there is no real expert, only an element of chance that causes one of the 50 to be crowned as a gumball estimation expert in that round of the experiment.

Estimating the number of gumballs in a jar is a complex problem. If you approached it methodically you would have to estimate the total volume of the jar and the total volume of each gumball. You would have to take into consideration the way the gumballs are placed in the jar. Are they tightly packed to the brim? Or is there airspace in the jar because of how they are sitting? If they are slightly irregularly shaped that will change the number of gumballs that can fit vs. if they are perfectly round. Estimating these kinds of variables visually, with our eyes, is not something humans are particularly suited for. Now we are smart enough that we can devise means that can determine precisely the number of gumballs in the jar, but visual inspection with our eyes, often from a distance, is not the way to go. Yet because of our inherent biases as humans (e.g. 95% of us are certain they are in the top 50% of drivers, 25% of us are certain they are in the top 1% in terms of leadership skill, and of course at Lake Woebegone all the children are above average), there are those among us who are pretty sure they can estimate the number of gumballs in the jars using that visual inspection approach.

Now, back to our one-time gumball estimation expert. What we tend to do is take the winner of one of our estimation rounds and put that person on TV as a talking head and ask that person their view points on various gumball estimation problems. Will the economy go up or down? Will Iran agree to a negotiated nuclear settlement? Which stocks should I pick to make the most money next year or will the bond market lose or make money? What will happen to inflation? Does that recent scandal eliminate any chance of electoral victory? What is the underlying cause of social unrest? What causes a person join a protest or engage in civil disobedience? We will be greeted as liberators or as oppressors when we march into Baghdad?

The underlying problem with those who answer these kind of questions, is that they attempt to identify a single or a small handful of underlying causes, (and assign them weights) based on their expertise, to complex problems. They may be using various assessments or may often be using measurement instruments, (remember the visual inspection for gumballs) that are simply not up to the task. And because of human limitations, they get past these short-comings by using rules-of-thumb or heuristics to take a question, that has a multitude of complex issues and boil it down to a simple answer. Answers that are more likely to be wrong than right.

For instance, the number of variables of what causes a Baltimore to erupt are immense and they are related to each other in extremely complex ways. The variables that affect human behavior are intricate and defy simple explanation. It is beyond rocket science. That is not to say that we should not work to understand and remedy issues. But the errand to put simple labels on socially complex issues is a fool’s errand, can result in gross mischaracterization and is easily debunked by other “experts” those who can point to other simple labels that they develop.

Now there are true experts out there. I will be the first one to run to a doctor or emergency room if I break a leg or have a heart attack. I will use a civil engineer to design and a qualified construction company to build my new bridge. I listen to climatologists about what is happening to our planet and attempt action. I listen to food experts when, based on the scientific method, it is known that GMOs have no harmful effects (every piece of food you eat has already been genetically modified from its original state). And I will have my children vaccinated against all the harmful diseases that used to be the scourge of our society.

There are huge differences between those using the scientific method to determine cause and effect, to improve the lives of people everywhere on this planet and so-called “gumball experts”. But part of the reason we have some of the issues we face as a society is because we don’t always distinguish between the two.

© 2015 by Jeffrey M. Saltzman. All rights reserved.

Written by Jeffrey M. Saltzman

May 3, 2015 at 7:56 am

One Response

Subscribe to comments with RSS.

  1. Reblogged this on Jeffrey Saltzman's Blog and commented:

    Now that the primary season is in full swing, this seem appropriate to repost.

    Jeffrey M. Saltzman

    February 2, 2016 at 6:42 am

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: