Jeffrey Saltzman's Blog

Enhancing Organizational Performance

Posts Tagged ‘risk

Unsupported by Evidence

with one comment

I was recently on a panel at a local meeting in NYC of Industrial Organizational Psychologists and after much discussion I made a statement. I said that as a field we have almost completely and utterly failed at bridging the gap between the science and research that we do, the evidence-based and experimental knowledge that we gain and those who are out there in the world writing about people at work or organizations in the lay press, or those in organizations, making day-to-day decisions about them which affect both the organizations and the people within them. After all most of the information about people at work is just “common sense” isn’t it? And I am a person and I work, so I guess that makes me an expert.

Unfortunately, much of that “common sense” is not supported by the facts and in some cases the facts support the opposite conclusion or the common sense is generated by those who have an agenda in which facts are simply inconvenient. Here are some of the more common statements that I keep running across that either have no basis in reality, are the opposite of the actual evidence, rest on very shaky expansions or extrapolations of a small kernel of observation, or are based on a small handful of people or organizations at one tail or the other of a distribution, but ignore the vast majority of those in the “fat part” or middle of the distribution.

  • Statement: People will find jobs once their unemployment checks run out, a social safety net is an incentive not to work.

o   Reality: The vast majority of people want to positively contribute to society, as it makes them feel valued. People want to feel valued, it is a universal. The data show that people would rather be overworked than underworked and the desire to work and contribute is not diminished in societies with strong social safety nets. Can you find people who fit the above statement? Sure, but not the vast majority.

  • Statement: The various generations want and expect different things from the work environment.

o   Reality: There is simply no evidence to support the notion that different generations want different things from work. Rather the differences often cited are driven by life stage and economic opportunity. In other words, give a person a mortgage and kids in college and job security becomes more important to them. A person right out of college with no responsibilities or financial obligations will act similarly regardless of which generation they come from. Because life stages take a rather long time to get though they give the appearance of being generationally driven.

  • Statement: People join companies and leave managers.

  Reality: Are there “bad” managers out there that have driven people out of an organization? Absolutely. But the majority of people join an organization and then leave when they don’t see a promising future for themselves within the organization. Sometimes that feeling is caused by a bad manager, and sometimes by the simply reality of a mismatch between a person’s career expectations and what the organization can offer. And sometimes it is simply a person’s life situation. The next time you are with a large group of people ask for a show of hands of how many of them left their last job because of a bad boss.

  • Statement: A good interviewer can determine if a person is a “fit” for an organization.

  Reality: We have known for a very long time that interviewers can actually diminish the ability to predict whether someone will succeed in an organization. An interviewer makes judgments that are often not based on job relevant characteristics.

  • Statement: Lie detector tests can determine if someone is lying and can be useful in making hiring decisions.

o   Reality: The evidence that lie detectors actually work and can determine if someone is lying is not there. And it is absolutely for certain that people with low affects can lie to lie detectors and get away with it. Lie detectors work on the notion that someone telling a lie will become more stressed and emotional and someone telling the truth will remain calm. The reality is that someone, even an innocent person, hooked to a lie detector and being asked about crimes will become stressed. (Generating false positives.) You might as well tie the person to a log and throw them in a river. If they float they are guilty and should be executed. If they sink and drown they are innocent, but unfortunately still dead.

  • Statement: Money doesn’t motivate people on the job.

o   Reality: Money is a great motivator (ask those on Wall Street). Money tends to show up on statistically generated lists of drivers of job satisfaction when people perceive themselves are being paid unfairly. When they perceive themselves as being paid fairly for the work they do, it tends to diminish in importance. People who claim money is not a motivator often seem to be people whose job it is to keep employments costs down.

  • Statement: It is good to regularly reorganization a company. It keeps people sharp; it keeps them on their toes.

  Reality: Organizations that regularly reorganize are consistently having people learning the ropes of new positions. In several studies it has been shown that better performance is achieved by people who have been in positions for longer periods of time then by people who are switched from job to job.

  • Statement: In business downturns, laying-off people is the best course of action.

o   Reality: If you can’t afford to pay people you need to get your costs down or you cease to exist. However, there is a good deal of evidence that shows that organizations that resist layoffs in down-cycles outperform as the economy recovers.

  • Statement: Women are more risk adverse than men, so if a job requires risk taking women are not a good fit.

o   Reality: It is pretty easy to find women who are more risk tolerant than many men. This is bias pure and simple and based on stereotypes.

Many of these statements are what Paul Krugman, the Nobel winning economist and NY Times columnist calls “Zombie Ideas”. Zombie ideas are statements that should have been killed by the evidence but refuse to die. From my perspective the field of Industrial Organizational Psychology, which is often concerned about publishing in scientific journals, (not that there is anything wrong with that), has a lot more work to do in getting our knowledge out into the mainstream and accepted.

© 2014 by Jeffrey M. Saltzman. All rights reserved.

Visit OV:

Written by Jeffrey M. Saltzman

June 13, 2014 at 6:53 am

Managing Risk vs. Uncertainty and why it Matters

leave a comment »

[tweetmeme source=”jeffreysaltzman”]

“As far as the laws of mathematics refer to reality, they are not certain; and as far as they are certain, they do not refer to reality.”

-Albert Einstein

The distinction between risk and uncertainty arose from the field of economics and is based on the work of Frank Knight. According to Knight, “risk” refers to a situation in which the probability of an outcome is known or can be roughly determined, while uncertainty refers to an event or an outcome whose probability is not or cannot be known. A common example used to illustrate the point is that games of chance are risky (because the odds of winning vs. losing can be calculated) and the outcome of a war with its multitude of changing environmental situations on the ground is uncertain. But it is not as simple as all that, it certainly can be very confusing, and a deeper understanding of risk vs. uncertainty can help people make better decisions.

People board airplanes routinely, strapping themselves into what is essentially a lounge chair (even if it is an uncomfortable one), inside of what is little more than a controlled missile, whose paper thin walls of plastic and metal guard you against external conditions that could not possibly sustain life, while jet engines furiously burn enormous quantities of highly explosive fuel within a few feet of your location, hurtling you and your lounge chair through the sky at hundreds of miles per hour, with thousands of pounds of airplane coming back to the earth with a typically ungraceful thump, on small rubber wheels (which always look flat to me), and then hoping that the plane will somehow slow down and avoid plunging into the water (I land at LaGuardia), or worse. Why in the world would they do that? Because it is not all that risky. Airplanes have very good track records and there are very few accidents. We can manage risks, uncertainty is more difficult.

But what about Wilber and Orville when they first attempted flight? They were not dealing with risk they were dealing with uncertainty, for there was little real understanding of whether man could build airplanes that could stay aloft for a period and then safety return the occupant to the ground. There was no track record to calculate risk upon, there were no computers to run simulations, there were no wind tunnels which could test airplane models. Yet facing this uncertainty they persevered.  Events can begin with uncertainty and then as track records about them build they can become simply risky. Though, some people who treat a situation as risky, when it is actually uncertain, can accumulate really awful track records of performance.

For instance, Gerd Gigerenzer of the Max Planck Institute analyzed the performance of 22 major international banks on predicting currency fluctuations.  These annual forecasts of currency values which occurred from December of 2001 to December of 2010 were used by the banks to guide their investment decisions. These annual forecasts were wrong, very wrong, for nine out of those ten years. Gred’s conclusion about the track record of the people who produced these annual guides to currency values was that “highly paid people produced worthless predictions.” He went on to explain that based on his analysis the risk modelers at the banks didn’t distinguish appropriately between risk and uncertainty. They treated the currency fluctuations as risky but in fact “it is uncertainty that rules in the real world, where risks can’t be known in advance because of a complex tangle of factors triggers new, extremely unlikely hazards.” What he meant was that many factors that could affect currency values, (e.g. oil shortages, war, weather, natural disasters, deepening recession) were not adequately accounted for in the prediction model and in fact could not be known as they were uncertain. The analysts at the banks though treated them like risks, however, with underlying probability distributions and they got it wrong.

When Steven Jobs, who famously stated that “people don’t know what they want until you show it to them”, produced his first computer he had no idea if people would be able to see its promise, what they could accomplish with it and whether they would buy it. Over the years with each new technology his team at Apple developed there was uncertainty, sometimes uncertainty with great consequence, as some of the products rolled out were “bet the company” kinds of decisions. Steven Jobs reveled in the world of uncertainty and showed that mastering the world of uncertainty can lead to enormous financial reward. But Apple’s ultimate financial success came as those uncertain new technologies became simply risky products. Was there any doubt in anyone’s mind if the iPhone 5 would be a success? The question was not “if”, the question was “how big”. “If” is uncertain, “how big” is risky.

There are hoards of managers out there who want to emulate Steve Jobs, or at least his success or even more precisely his financial success. They look at his management style, which at times bordered on abusive and wonder if that is the path towards their own success. Perhaps if you beat up your employees, driving them really hard, you and your company can also succeed and become like Apple. The scientific literature casts doubt on that approach (big time) as working for the majority of managers (from a risk management perspective). I do know of a few very successful CEO’s whose success could only be described as coming off the backs of their employees rather than through or with their employees. Yet Steven Jobs, with his style, and his ability to deal successfully with uncertainty, like the Wright Brothers, was able to build the most financially valued organization in the world.

In the retail world (and real estate in general) there is an expression, “location, location, location”, meaning that without this fundamental element in place, it simply does not matter if you have great merchandise. You will not be successful. And with Apple it is technology, technology, technology or perhaps product, product, product. The success of Apple should not be attributed to an abusive management style, but rather to Steve Job’s genius in developing technologies and products with an uncertain outcome and turning them into mere risks. Arguments about his management style could be viewed as a red herring – he was successful in spite of it, because of his overwhelming other abilities, not because of it.

If I need to hire 100 people for my sales force and I have developed a predictive analytics approach to helping me select the best 100 out of my applicant pool of 1000, I can determine the likelihood of expected performance outcomes across those 100 new hires by creating a probability distribution. The distribution of job performance across 100 sales people is something that can be known and so my hiring decision can be described as risky, not uncertain. But if I want to know and predict how a specific sales person will perform, that is more uncertain. I can create a probability score for that individual, but I cannot say with certainty what the performance outcome will be as external factors (e.g. a death in the family, a pregnancy, a spouse relocating, a divorce or marriage, a new educational degree, or a new opportunity) cannot all adequately be accounted for. One thing to keep in mind, as the quote by Einstein above describes, is that our models are representations of reality, accounting for only a portion of the variance, they are not reality.

Yesterday, a horrendous crime was committed in Newtown, CT where a 20 year old gunman shot and killed 28 people, 20 of them school children between the ages of six and seven. This was terribly disturbing and I had great trouble concentrating on anything else after I heard the news. As I saw the images of the parents finding out about their children I could feel my heart ache for them. Our own local school, about an hour away from Newton went to a heighten security status. I am concerned that there will be the usual hand-wringing about firearms and second amendment rights and then nothing will be done. This time is has to be different. Our children are dying.

From an uncertainty standpoint it would be very difficult to determine if any one individual is a risk of committing a gun related crime. As with the sales force example above a probability score can be created, but at the individual level what you are dealing with is closer to uncertainty than risk. Common-sense gun laws would suggest background screening and eliminating those with various mental illnesses and track records of violence or abuse from gun ownership.

But beyond that if you look at the methods that can be used to manage risk, and at the likelihood of gun violence, more guns simply provide more opportunity for guns to be used in gun violence. It is a simple relationship. Red herrings are constantly thrown up about arming people to stop the perpetrators of gun violence, as though if we simply have more bullets flying around that fewer people will get injured or killed. Very unlikely. The arithmetic simply does not add up. From a risk management standpoint fewer guns mean fewer gun crimes. Period. End of sentence. Trying to create the odd one-off scenarios whereby having the right person in the right place with the right weapon and the sensibilities to stop a crime in progress without creating further injury to other by-standers is just not logical.

A first step would be to ban the type of firearms that allow for mass-murder to happen within a few seconds without reloading.  Our ultimate goal from a risk management standpoint should be to reduce the number of guns available. Period. Given the difficulty of dealing with uncertainty, you cannot accurately reduce the number of guns available to only those who will commit gun violence, you will get it wrong. So the solution must be one that works with the probability distribution. Ultimately, we must reduce the overall number of guns that are floating around in our society. To paraphrase a quote about eating an elephant – how do you remove 300,000,000+ guns from our society? One at a time.

© 2012 by Jeffrey M. Saltzman. All rights reserved.

Visit OV:

Written by Jeffrey M. Saltzman

December 15, 2012 at 7:53 am

%d bloggers like this: