Posts Tagged ‘measurement’
Employee Engagement is often viewed as a magic bullet. All we have to do is increase our levels of employee engagement and all will be well. Is your engineering done poorly? That is because your engineering employees are not engaged enough. They would exceed your customer’s expectations if they were more engaged. Putting your stores in under-performing locations? That would not happen if your real estate people were more engaged. Are your customers unhappy with the quality of your products? If only you could make your sales people were more engaged. This kind of thinking is of course nonsense, but there is a deeper issue here.
Some, if not many organizations have bought into the notion that increasing employee engagement should be part of an organizations’ strategy. But that is like saying reducing an ill person’s fever should be the strategy to get them well, without addressing the underlying cause, like the tumor that is spreading rapidly in their pancreas. Maybe if we brought the fever under control that tumor would resolve itself? Not likely.
As we conduct employee surveys there are several distinct kinds of questions that are used to gage what is happening within an organization and how it is functioning. One question type is called an independent variable. These are items like “do you have the training you need to get your job done?” They are directly addressable if the response scores are low. Another question type is called a dependent variable, such as “I am proud to work for XYZ”. These kinds of questions are dependent on other things driving them high or low, such as, we were just caught up in a bribery scandal, so I am not so proud to work here. How would you address pride in that circumstance? While there may be other underlying issues, simplistically, you would address ethics in order to bring pride back to higher levels. There are other kinds of questions we use in surveys but discussing these two types will make my point.
Good strategy for an organization is strategy that is simply stated, easily understood and directly addressable. Good strategy could be thought of as independent variables. Is your engineering done poorly? Good strategy may be to upgrade or bring resources to your engineering group. Maybe you hire or maybe you acquire or maybe you outsource, but the hallmark of a good strategy is that you can directly address the improvement needed of the engineering function. The engineering employees will become engaged when they have what they need to do their jobs well, are treated in an equitable fashion, with respect etc.
A strategy that states, we will increase employee engagement as the strategy itself, is not directly addressable and does not give the management team any insight into specifically what needs to be done to accomplish that goal. Without insight into the direct strategic actions that must be taken you get warm and fuzzy words that are not directional and will be impossible to accomplish.
Having high levels of employee engagement is a good end result, but it is an end result of other strategic actions you take and is simply not strategic by itself.
© 2012 by Jeffrey M. Saltzman. All rights reserved.
Visit OV: http://www.orgvitality.com
”Which is more important, the sun or the moon?” A citizen of a small town not noted for its intellectual prowess asked. “Why the moon of course,” was the reply. “It shines at night when it is needed. The sun shines only during the day, when there is no need of it at all!” (Ausbel, N., A Treasury of Jewish Folklore, 1948)
“What to measure, what to measure…..” the executive muttered to himself as he walked down the hallway. Do you simply measure the obvious things, or are there less obvious metrics that need to be measured that can help the organization’s performance? And how many ways should you measure similar outcomes in an effort to triangulate and instill confidence in the measure? If you don’t understand what is really going on as the quote above implies, your measures may be entirely off-base. They could be based upon superstition-like folk wisdom or they may overlook critical, but subtle aspects of performance. Metrics can be hard measures, such as sales volume, widgets produced, defects per million etc. They can be soft measures such as customer satisfaction and repurchase intentions or employee confidence in the future of the organization or their own personal future in the organization. Metrics can be at the individual level, department level, organizational level etc. When you think of all the possibilities of organizational performance that you can measure, it is no wonder that some organizations seem to spend an awful lot of effort and time measuring, eating up significant resources. Occasionally they must ask themselves, “can we get by with fewer and simpler measures?”
Think for a moment of a car. Say you just filled the car’s tank with gas prior to heading out on an extensive road trip. You are interested in knowing when you should stop and refill the tank so that you will not become stranded by the side of the road without fuel. On your dash you have 3 gages, a fuel gage, an odometer and a clock. If you were to drive your car at a fixed speed, on a never changing road, achieving a consistent miles per gallon you could determine when to put gas in the car using any one of those 3 gages, they would be redundant. You could look at the fuel gage and see when the needle was inching toward zero. You could look at how many miles you had driven on the odometer, knowing that when you hit a certain number of miles it was time to stop for gas. Or you could look at the clock, knowing that with a steady consumption of fuel per minute, after a certain period of time gas was needed. That situation though rarely describes how a car is driven. You may be going up and down hills, starting and stopping at traffic lights, or driving at inconsistent speeds resulting in changing miles per gallon performance. Therefore each of those gages on your dashboard, while potentially highly correlated with each other in their predictions of when to stop for gas, will not be perfectly related. And with their uniqueness they will provide additional insight into maintaining the car’s or the driver’s performance. For instance the odometer can tell you that it might be time to replace the tires or change the oil and the clock can tell you how much more time you have to drive in the daylight or if rush hour awaits you in the city ahead that you will be driving through. Each measure of performance if chosen carefully should not only help triangulate on an outcome (the car needs gas) but should also provide additional insight into performance.
Those of us at OrgVitality have determined that if an organization were to focus on 6 aspects of their performance that they would have a pretty good dashboard of how the organization was functioning in the areas of softer and the often harder to measure metrics. Those aspects of performance include:
- Work Processes
- Product Offerings
- Service Orientation
- Customer Loyalty.
Further, we believe that each of these metrics should have a current performance measure and a future performance measure. For instance, leadership from a current performance orientation is about execution and leadership from a future orientation is about the vision of where the organization is heading, the ability to communicate and bring that vision to life, it is about the succession planning pipeline and the development of leadership talent.
If you would like to learn more about these metrics, we have set up an OrgVitality Self-Assessment that you can sample at http://demo.orgvitality.com/vsa. If you complete the survey you can compare the performance of your own organization against the norms from other organizations. Enjoy.
© 2011 by Jeffrey M. Saltzman. All rights reserved.
How important are words? Do words have the power to shape our thinking or are they nothing more than a reflection of what our minds are already processing, giving substance to existing abstractions floating in our heads? That is the essence of a debate that has gone on now for more than 100 years. Think for a minute of the words we use to describe numbers, one, two, ten, fifty. Are we naturally inclined to develop words to describe numbers? Do the words themselves, the words that we have made up to signify quantities give us the ability to think both abstractly and concretely about numbers, or is the ability to think numerically built into the structure of our brain? Said another way, is the ability to think logically about quantities an inherent ability, independent of language, and the words we have developed simply an expression of that ability or do the words shape our ability to think in a numerical sense?
There is a tribe from Brazil, the Pirahã, who have no word for the number one or any other exact quantity. This is apparently the first group ever studied that has no concept for the number one. A new study undertaken demonstrates that the Pirahã can still convey quantity somewhat but are essentially using words that mean few, some and more. Other researchers contend that the words they are using mean one, two and many. In either case in Pirahã society the need for being able to quantify things precisely and hence develop a language system that allows for that was not a cultural priority.
As strange as you might find it, that there is a society without words for specific numbers, remember that the first evidence for the use of the concept of zero is from the Sumerians in Mesopotamia about 5000 years ago. From there it traveled to the Indus Valley, and was used in Hindu society. In the Indus Valley it was picked up by Arab merchants and became very important in their trade and spread throughout Arab society. The Greeks only occasionally used the concept and Romans had no concept of zero (for those of you who remember Roman numerals try writing zero). On the other side of the globe the Maya independently invented their own version of zero. The concept of zero slowly migrated around the world and did not make it into European society until the late middle ages, as Europe was stubbornly holding on to the use of the Roman traditional counting system rather than adopting new methods. So while we take zero for granted today, it is a relatively new concept for western culture.
Do words shape our thinking? One urban legend states that Eskimos who live in snowy places, and hence deal with snow more regularly than most of us have developed many more words than exist in English to describe types of snow. That is apparently not true. First off Eskimos are not a unitary people and of the many groups that consist of Eskimos, many different languages exist. Second the language structure of these groups is different, allowing for combinations that do not exist in English, making comparison between the numbers of words that exist to describe snow very difficult. They may or may not have a few more words than in English to describe snow but it is certainly not hundreds as the urban legend claims.
There are words that have been consistently used to reinforce messages of hatred, words that need no repetition here. Those words tend to be used over and over to denigrate others within societies around the world. Does the constant use of words of hatred reinforce the pattern of biased and bigoted thinking within the minds of those who use them or are they simply an expression of what is already there? Clearly some believe that words of hatred create beliefs and behaviors of hate, as there are school children in various locations who are learning the vocabulary of hatred and to hate as part of their daily lessons. But what then happens to these children later on? Can they ever put the hatred aside once it becomes part of what they are, part of their essence? The future for the majority of children who grow up on hatred looks very bleak and greatly saddens me.
There is a raging debate going on about the vocabulary of rap music. Words that denigrate are built into the lyrics of certain performers. These words perpetuate negative stereotypes but are rationalized as somehow being ok since they are coming from within a community. I can’t agree with this at all. I strongly believe in first amendment rights but people should be aware of what they are doing and the implications of the choices they make. Words of hate will have hateful results – regardless of the source. Just as yelling “fire” in a crowded theater is not protected speech, yelling out hate filled words for mass distribution should not be protected speech as well.
Stringing together words of hatred into sentences can produce what some would call jokes. Jokes made at the expense of others, jokes that denigrate others for being different or being perceived as a threat to those giving word to those statements of hatred, hatred couched in supposedly humorous terms.
Each organization also has a vocabulary, words that they use in their day-to-day operations. (I am not talking about acronyms.) How important are the words that get used in our organizations? They can be no less important than the impact that words have in our everyday lives and in our shared histories. Developing unique organizational vocabularies that allows for both abstract reasoning and concrete discussions on the issues critical to the organization’s success may give an organization a competitive edge. Unique vocabularies, ways of expression may allow the organization to consider concepts and ways of working that competitors are unable to replicate. Words are important, they have power and they have impact and they should be used with care. People when speaking for themselves need to choose their words with care. People when speaking on behalf of organizations need to choose their words with care as well.
© 2010 by Jeffrey M. Saltzman. All rights reserved.
There are times when rigorous measurement of programs, processes or choices can greatly aid in the decision making process and surveys can be used in that evaluation. For instance, in organizations when the amount of resources that can be brought to bear are limited and difficult decisions need to be made about the deployment of those resources, a measurement process that gives insight into the expected benefits of action A vs. action B can be especially helpful. Consider the following situations:
- A school system wants to know if the investment it is making in advanced teacher training is improving educational attainment among its students;
- The GAO wants to know if efforts in information dissemination by government agencies can be documented as furthering government agency goals;
- A corporation wants to know if changes made in a reorganization are achieving the desired impact in terms of increasing organizational effectiveness and customer satisfaction;
- A zoo wants to know if it should proceed with increasing elementary school outreach by investing in an animal travel program, developing an on-site hands-on children’s zoo, or investing in bringing in more exotic animals and building new exhibition space in order to maximize attendance;
- A retailer wants to know if it should invest in more physical retail locations, beef up its website or send out additional catalogs with its limited budget dollars;
- A benefits departments wants to know how the company’s redesigned benefits package is being perceived by employees and how well the execution of the benefits program is being conducted by the new outsourced provider;
- A refiner wants to know if it should invest in a new Greenfield plant (a tangible asset) or if there would be greater ROI if it invested the equivalent dollars in additional training for all of its workers that would be expected to increase efficiency and throughput in its existing manufacturing assets (an intangible asset);
- A sales department wants to measure its sales funnel of potential deals, to determine if its marketing and sales efforts are having the payback it desires and to predict future sales volumes for the organization.
All of these decisions or potential decisions can be enhanced by an effective effort aimed at program evaluation. Clearly having effective measures is invaluable to those trying to manage these kinds of decisions. Where to invest, how much to invest, when to invest are critical decisions and managers would be well served to seek out systematic measures to enhance their judgment. The American Evaluation Association defines evaluation as “assessing the strengths and weaknesses of programs, policies, personnel, products, and organizations to improve their effectiveness.” While organizations vary in their efforts at program evaluation one interesting statistic comes from NASA-Goddard which has determined that for them between 7% and 9% of a total program’s budget should be spent on evaluation, and a recent Air Force initiative devoted 15% of the project budget to a rigorous evaluation of whether their new inventory control system was working as planned. Program evaluation though has been around long enough and has been practiced by a wide enough variety of people that not only have myths sprung up about program evaluation but other documents that attempt to debunk the myths of program evaluation are also in existence.
Traditionally when organizations have thought of program, policy or initiative evaluation what springs to mind is a list somewhat like the following as areas to be evaluated:
- Perception of Benefits –medical, life, dental, disability, wellness, stock options, matching contribution to retirement, vacation, maternity policy
- Views of Pay & Bonuses
- Health Safety and Environment (HSE) Emphasis and Policy,
- Effectiveness of Diversity Programs,
- Work Life Balance Initiatives,
- Psychological Recognition Efforts,
- Development & Training Opportunities,
- Advancement Systems,
- Physical Conditions,
- 10. Job Security.
One of my goals is to broaden out that traditional definition to include a broader array of various types of programs, policies and initiatives that an organization may undertake and would benefit from a rigorous evaluation methodology.
Many program evaluations have been sneered at as wasteful efforts due to poor evaluation methodologies being applied, but another factor that creates the typical cynicism is that whether a program gets continued funding may be somewhat contingent on issues outside of the effectiveness or the evaluation of the program. Common problems facing program evaluation consist of unclear program goals (so how do you know if you have accomplished the goal?), difficulty in collecting data, a lack of cooperation among those responsible for implementation (those whose plates are already full) and the politics of and vested interests surrounding various programs. Yet with a methodical approach insights can in fact be gained without too much pain into whether the program is having the hoped for impact.
Program evaluation can take place within a specific window in time or it can be an ongoing activity. For instance, an organization might implement a phased-in approach to a new program, or a new product/service etc. and decide that after 6 months the program will be evaluated to determine its impact, and whether funding should be continued at the current level, increased or diminished. If there is a positive finding to the evaluation effort, the program would be rolled out to the rest of the organization. Other organizations may not use a phased in approach and may simply have two or more programs being performed simultaneously in the organization and then conclude possibly, that it cannot afford all of them, so which ones are benefiting the organization more? An evaluation might be done once, at a single point in time to help to make that decision. Other organizations might decide that long-duration type programs, ongoing efforts, need to be constantly or periodically monitored to ensure that they are still delivering the desired for outcomes or that processes associated with them are not deteriorating in their effectiveness.
Surveys can be used for both single points in time and on-going long duration reviews as well as to measure program processes, content, and outcomes and there are appropriate uses to all in program evaluation. Some of the questions that program evaluation efforts can be used to answer include:
- Is the program being carried out well, and how can it be improved?
- How can you measure the impact of various programs on organizational processes?
- Are the people targeted by the program understanding, retaining and actually utilizing the content being provided?
- Is the content covering all critical areas?
- Will the program have an impact on organization effectiveness, increasing the organizations ability to perform?
- Which programs are the most cost effective, providing the biggest bang for the buck?
- How can I measure the return on investment (ROI) of various programs?
© 2010 by Jeffrey M. Saltzman. All rights reserved.
Visit OV: www.orgvitality.com
Are you proud of the company you work for? What makes you proud? Are you proud deep down inside internally or are you bursting at the seams proud wanting to tell everyone you meet about the great company you work for? What are the potential benefits to the organization if in fact the employees of an organization are extremely proud to work there? Is it more than bragging rights? Does it carry any weight; have any impact on how the organization actually performs?
When you look at organizational survey results inevitably the senior management group shows the highest levels of pride in the organization. As you move through the ranks the most common finding is that pride declines, but not always. Some organizations are able to maintain high levels of pride throughout the ranks. Why do the senior managers typically exhibit the highest levels of pride? Let’s examine what is often different about them, as a group, compared to others within the organization.
- They have more control over their own future and generally have more latitude in decision making; therefore, they are in some respects giving a self-rating and hence cognitive dissonance sets in. If you are the one calling the shots, how can you not be proud of what you have created or accomplished? If you are not, what does that say about your own perceptions about yourself and your own capabilities? The human mind facilitates the path of least resistance;
- They generally have a clearer understanding regarding what the organization is about and what it hopes to accomplish down the road. And what the organization is about is generally very congruent with their own personal beliefs and values, since they are setting the agenda; What the organization is about then, is of course of great importance, further generating feelings of pride;
- They generally can get what they need to accomplish their work, (at least more so than others) and can often leverage their efforts by enlisting others;
- They often feel more positive about their future within the organization;
- And, they are made to feel very valuable, critical to organizational functioning. Pay and fringe benefits are much more lucrative at the senior ranks and senior managers have people following their lead and doing what they say, an ego stroking that can go to almost anyone’s head;
- Interestingly, it is not unusual for senior managers to score more poorly in the area of receiving performance feedback. Senior managers are often not moved into position because they are the best “people person” for the job. Most of the time other characteristics and capabilities takes precedence. And while some are good people persons, others are not, and find it very difficult to relate to or give constructive feedback in a helpful manner to their direct subordinates.
There is a story of an august group of scientists who scored fairly poorly on recognition received from their boss, the head of the scientific research institution in their survey results. Upon feeding back this result to the boss, the boss related that he was uncomfortable giving feedback to these brilliant people who worked for him. “Who am I to give feedback to people who have done such great things and are so accomplished? Many have them have won very significant international recognition rewards.” The boss was having feelings of inadequacy. And yet when you went and talked to the scientists they would say things like, ‘look what I have accomplished and my boss can’t even walk down the hall and say “nice job”’. People are people, (both the boss and the scientists) and they are all looking for many of the same things, recognition of their accomplishments being one of those things.
While this is not necessarily an all inclusive list, organizations that have managed to sustain pride through their ranks are:
- Viewed as having effective leadership;
- Have innovative (sometimes paradigm shifting), high quality products (dominating or with a significant share of their industry);
- Have clear messages about who they are and the roles of individual employees in supporting who they are; often seen as serving a “higher” purpose;
- Give people what they need to get their jobs done (in the broadest sense);
- Give people a sense of future, that if they stick around with the organization good things will happen for them;
- Make people feel valued and treating them with respect and dignity.
Have you ever noticed that when you visit the tourist shops in Washington, D.C. you can find t-shirts, mugs etc, with the names of various agencies emblazoned upon them? You see tourists parading around in hats and collecting all sorts of paraphernalia that say FBI, CIA, Marines, Air Force One, ATF, NSA, and Army, among others on them. But I have yet to see one hat or one t-shirt with the initials IRS emblazoned upon it, or how about one with the words Senate or House of Representatives across the chest? Not likely. Yet the IRS the Senate and the House are some of the oldest institutions of the US Government, without which we would not be able to function as a country. What gives? If an IRS employee were to meet some new people at a dinner party do you think they hesitate before answering the question regarding where they work? Do you think their first reply might be to answer that they work for the government, rather than specifically saying the IRS?
An IRS employee is just as proud as any employee working in the private sector for a Fortune company and when you investigate some of the unique drivers of pride for the organization you would find both similarity and differences to other organizations. (One website where Federal government employee survey results can be reviewed is http://bestplacestowork.org/BPTW/about/). One driver of pride in the IRS is quite noble – employees that are proud to be in service to their country. They are serving a higher purpose. Without them they feel the country would not function – and you know what, they are right. What about workers at medical institutions, what are drivers of pride there? They are in service to mankind. They are helping the ill recover. They may be using techniques that only a few can use or they may have a reputation that this is the place to go if you really need good care, both of which would increase employee pride in the institution. Teachers are proud that they are preparing the next generation and helping to shape and develop young minds – serving a higher purpose. Sanitation workers are proud that they keep our communities clean and livable. (If you have ever been in NYC during a sanitation strike you would quickly agree with them). Police and firefighters are proud that they are protecting the public.
People in all different types of occupations want to be proud of the places they work. They want to take pride in what they do specifically and what the organization accomplishes. People can be proud of what they do, even though you or I may not find a certain job particularly interesting or it may be something what we would have a hard time relating feeling proud about. People are they same, they all want to feel pride, and yet they are all different and can feel proud of quite varied tasks, requiring quite different abilities and skill sets.
Instilling a sense of pride in your employees begins with instilling a sense of meaningfulness, a sense purpose, a sense of what they are doing as being important – and you know what, it is important, or they should not be part of your organization. Many organizations take advantage of this notion and use their organizational mission to help in recruiting new employees, sometimes employees who otherwise would not consider working at the organization. The Marines in their recruiting slogan “The few, the proud, the marines” are playing directly to that notion of people wanting to be part of something that generates pride. Join us and do something you can be proud of. And even going beyond that, is the notion that only a few, a select few get in (but that will be a different blog on exclusivity).
What the drivers of pride are in various organizations is not static. It can change over time. For instance today people can be proud if they belong to an organization that is operating in a “green” fashion. And in fact being green has become so trendy that many organizations are promoting their “greenness” publicly as part of their selling proposition to customers, even those that are really not green, not even a light shade of green. Eventually I believe that people will see through the sham and that marketing ploy will backfire.
When an organization does not function in such as manner that it instills a sense of pride in the employees it can be very damaging. Turnover is likely to rise (also dependent on external economic issues) as will absenteeism, the level of quality declines, errors will increase, customer service and hence satisfaction will suffer and the organization as a whole will start a downward spiral. Organizational performance in general will decline.
Think of two questions that have been fairly regularly used to measure pride. Both are straight forward. The first “I am proud to work for XYZ” and the second is “I am proud to tell people that I work for XYZ”. One is measuring an internal component of pride in working for their company the other is measuring an external component, being able to speak publicly of pride in working for one’s company. Which would you answer more positively? Are you deep down inside proud? Or are you bursting at the seams proud, just can’t wait to tell others about it? Which do you think is better for your organization?
© 2010 by Jeffrey M. Saltzman. All rights reserved.
Visit OV: www.orgvitality.com
Quite a few years ago I visited a number of car dealers as I debated about which new car to buy. When I pulled into one dealer’s lot for the first time I saw a large sign hanging in the large plate glass window in front. “Ample parking in rear” it read. Given the difficulties in finding parking at times in the metro area, having a sign telling you where to go to find some I initially thought of as being very helpful. After the usual painful negotiations, with my trade-in being worth next to nothing and some kid at the dealer telling me I was being foolish if I did not take out a large loan to finance the car, I ended up buying one. (Note to dealers, having your finance person tell your customers that they are being foolish is not a good tactic, and they may begin to question whether they are really being foolish in dealing with you at all.)
A few weeks after I had possession of my new car, I received a survey in the mail from the manufacturer wanting to ask me about my car buying experience. The very first question on the survey was “Was there ample parking when you visited the dealership”? I was taken aback slightly. Was it a coincidence that the dealer was making a big deal of his “ample parking” and that it happened to be the first question in a survey evaluating my car buying experience?
A few more weeks later I visited the dealer again for service to my new car. As I approached the service counter, where the service manager was to complete the paperwork on what kind of service I needed, I noticed a note tapped down to the counter. The note said “shortly after being serviced here, you will receive a survey in the mail asking about your service experience. If you can’t fill out the survey this way (and a completed survey was attached to the note showing the customer how they should fill out the survey) please let us know now”. Ok, now I wondered “are they proactively trying to head off any service issues, or are they trying to cook the books on how customers respond to the survey”?
The cynic in me said that the pattern of behavior I was seeing was an indication of a dealer who wanted to have good survey results and was trying to lead his customer down a path to insure that it was so. My guess was that he would achieve a performance rating that he could then use to his advantage in his marketing campaigns if he achieved certain scores.
There are many organizations out there that strive very hard to have exceptional survey results. When they are achieved celebrations ensue, especially when the results are tied to performance or incentive plans. I would argue though, that there should be an equal celebration when results come back that are just average. Why? If the results on a survey come back and are outstanding, you can’t learn as much regarding additional improvement as when average results are achieved. In highly performing organizations when average results are achieved it means you are asking the tough questions, questions that if properly constructed can point the way towards additional improvements. Asking feel good questions that achieve high results only accomplish what the name implies, they make you feel good, but they don’t give as much insight into improving performance.
Additionally, when you see a more average response, what you also usually see is more variance to the questions within the organization as your begin to drill down. This variance in responses means that some parts of the organization were able to achieve better scores that other parts of the organization. And that means that the organization now has an opportunity to learn from itself if it so chooses. The variance within the organization presents a potential learning and improvement opportunity by which to take organizational performance to new heights. If the organization achieves a very high score on a survey item, as I drill down deep into the organization I am much more likely to find similarly high scores, a uniformity of responses. This sameness in responses constricts the organization’s ability to compare differing responses and see which differences in actions or behaviors led to those differing response patterns. Uniformity means that the organization has less potential to learn and improve from itself.
Of course if an organization is uniformly low on an item – scoring poorly, that can be a sign of fundamental issues that cut across the board and the processes represented by that item may have to be reinvented on an organization-wide basis.
The underlying question though is why is an organization doing a survey in the first place? Is the desire to do a survey part of a genuine effort to improve performance? If so average responses should be celebrated for the opportunity they represent. If the desire to do a survey is “monitoring” or measuring managers on their scores then there is a risk that the managers will find ways to achieve exceptionally high scores so they can say we are doing well. Somewhat self-delusional behavior – but remember organizations get what they reward – if they are rewarding self-delusional behavior they should not be surprised at what they accomplish. Instead of rewarding the absolute scores themselves, achieving a high benchmark on a survey item that may limit the amount of learning that can occur, organizations should consider if it can be more productive to reward the learning behaviors, the continuous trek towards improvement within the organization that it desires to have demonstrated. In other words reward the behavior not the number.
A new twist on benchmarking; organizations may want to benchmark those survey items that allow them to achieve somewhat average responses. In other words which are the best survey items that give them mediocre scores – and enhance an organization’s ability to improve?
I think that an organization that asks the tough questions in the spirit of actually wanting information the help drive performance, which celebrates average responses for the opportunity they represent, that rewards improvement behaviors rather then absolute scores, will outperform. And isn’t that what it is all about anyway?
© 2010 by Jeffrey M. Saltzman. All rights reserved.
Visit OV: www.orgvitality.com
There is an old folk tale that describes a young man, mentioned as somewhat of a fool, who upon getting married was given a dowry by his father-in-law and told that he should become a merchant, in order to provide for his new wife. Upon being swindled in his first transaction, he comes to the realization that if he could read he would stand a chance at perhaps being somewhat more successful as a merchant, and less likely to get swindled in his next transaction. He heads off to the nearest large town in order to take reading lessons, lessons which he was told would take between 6 month and 1 year to make him proficient in reading. Upon entering town on the cobblestone streets, he comes across a spectacle shop in an alley and with the door to the small ancient looking shop open he overhears the proprietor asking a customer, as he adjusts her new glasses, “Can you read now”? To which the customer replies, “Yes, that is much better, I can read now”. The newly-wed and newly minted merchant of course must enter the shop, and ask the proprietor for glasses for he too is having trouble reading. How wonderful, he thought, rather than spend 6 months to 1 year working hard at learning how to read all he would have to do to become proficient at reading is put on a pair of glasses! The proprietor after trying pair after pair of glasses comes to the realization that the reason this new customer can’t read has nothing to do with his eyesight. Upon stating that conclusion to his customer, his customer complains that the glasses worked for that other organization …ops… I meant person, so why wouldn’t they work for him as well?
Correct diagnosis of a situation is critical to successful problem solving implementation. What works for one organization may not work for another and the magic silver bullet fixes that organizations often seize upon may offer no better organizational reading ability than where they are today. While the tools of choice may vary depending upon your specialty, comprehensive, customized organizational surveys for me have been the method of choice, along with focus groups and interviews to obtain the information necessary for a correct diagnosis on issues surrounding organizational culture and its ability to impede or enhance performance. Sometimes even if you correctly diagnose the situation though and know what has to be done to implement change, provide service to a customer, help a fellow employee or maybe even save a life, established procedures, authorizations, bureaucracy, silo thinking, inertia and organizational entropy get in the way of a successful resolution. Persistence or sometimes just flouting the system sometimes does pay off – especially when you know you are right.
I know a surgeon who was describing to me a particularly tough operation he had the other week. He was removing part of the colon of a person with colon cancer. There is a vein that runs through the pelvis which can get cut during this procedure and upon the severing of this vein, just as a band under tension retracts upon being cut, this vein will sometimes retract into the porous surface of the pelvis, with blood upwelling from multiple small openings in the boney surface. This is a life threatening situation, and patients have been known to bleed to death on the table when this happens. He spent a great deal of time trying to stem the flow of the blood. Clamping doesn’t work as there is nothing to clamp. The blood is just seeping up through the bone. He tried a type of surgical epoxy glue, covering the surface of a portion of the pelvis, he tried cauterization, staples, the blood kept oozing. He then thought of a technique that he had seen as a resident. He asked the OR staff if there were any stainless steel surgical thumbtacks available. The staff said no, that they were not standard equipment kept for surgical procedures. He sent people to scour the bulletin boards of the hospital to see if they held any stainless steel thumbtacks. They did and the OR staff began collecting them. As they were running them through the sterilizer the surgeon had to get approval to use the thumbtacks in the operating room. Not standard he was told, not according to procedure. You could fill out a form and seek approval, I guess, but there were long odds against the patient living that long while forms were approved by the powers that be. A partner in the surgical practice was called upon to see a hospital administrator and explain the necessary reasons for violating established protocol. Meanwhile for an hour and a half, the patient lay on the table and had utilized 6 units of blood as the bleeding continued. Approval was given to use the thumbtacks and the surgeon push about 10 of them into the pelvis, in the area surrounding the upwelling blood. About 10 minutes latter the bleeding stopped. The thumbtacks will be a part of this person for the rest of their life, which thanks to a surgeon willing to go outside the box, will hopefully be a long one, but I bet the patient woke up with a very sore back.
This surgeon has a track record of being able to go outside the box as evidenced by the OR staff giving him a tee shirt with the words “What would MacGuyver do?” emblazoned on the chest, in reference to the TV show popular a number of years ago. He also sometimes sports another tee shirt that supports the use of duct tape for a variety of uses, but that is another story.
Correct diagnosis of issues, whether it is the need for glasses, or a more fundamental lack of reading ability, or reasons that prevent employees from getting their jobs done easily while providing exceptional customer service, or producing high quality products are important to uncover. Making assumptions on the causes, especially assumptions based upon what has worked or not worked elsewhere are often misguided. Additionally though organizations need flexibility, to be innovative and to get things done in ways that are not always according to protocol – and hopefully done before the patient or organization dies. Folk tales were originally designed to teach us lessons (in addition to scaring little children) and many of their messages are still relevant today. And lessons can also be learned from an environment were most people do not expect or think that innovation is happening in real time – the operating theatre. But innovation is needed in everything thing we do, it is how humans progress and cope with changing environmental circumstances. Lack of innovation or lack of innovation on the correct issues is to seal your fate into organizational oblivion.
There is an old story that goes something like this about a very rich man, lets say he lived in Manhattan, who had everything he wanted out of life except for one thing – he wanted to live forever. He struggled mightily with trying to figure out how he could accomplish that goal. Finally he came to the realization that he needed the assistance of others if he was to accomplish that goal. He thought about consulting some I/O Psychologists he knew, or going to see his doctor, but he finally settled on the smartest people that he knew, the New York council of elders. He went to the council of elders and he said, “I am a very rich man and I want to live forever. If you can tell me how I can accomplish that I will give the city ½ of my wealth.”
Well to the council of elders in the perpetually cash strapped city this was an offer that they just could not refuse. So they decided that they should figure out a way for this rich fellow to live forever. After much deliberation, the demographer in the group said, “I have been studying my charts and tables, and I have found that as far back as records have been kept, that no rich man has ever died in the South Bronx. So if the rich man were to move to the South Bronx he could live forever.” Well the elders were astonished. They decided to study the charts for themselves and they too came to the conclusion that no rich man according to their records has ever died in the impoverished South Bronx.
They went to the rich man and said, “We have found a way in which a rich man can live forever”. They explained to him that he needed to move to the South Bronx and asked for their money. Well, the rich man not being a fool said to them, “I will give you half the money now and half when I have proof that I will live forever in the South Bronx”. The elders protested and said that they had found the answer that the rich man was looking for, but there was little they could do as the rich man was very shrewd.
The rich man moved his family and all his possessions to the South Bronx and in due course he passed away, the city never got the second half of his wealth.
Morale of the Story: You can conclude anything you want based on your point of view, statistical misinterpretation and depending on how you are measuring it or not measuring it as the case may be, or you can’t live forever in the South Bronx, only in Hoboken.
When we measure organizational performance, while not as obvious as in the above story, sometimes conclusions are based on what is not being measured as much as what is being measured or the misinterpretation of statistical results and that can clearly lead to erroneous outcomes.
© 2010 by Jeffrey M. Saltzman. All rights reserved.
Visit OV: www.orgvitality.com
We are surrounded by surrogate measures, they exist everywhere in life. A surrogate measure is when a particular measurable or observable condition is assumed to be meaningful about an underlying state or condition. They are usually used when it is difficult, impossible or unknown how to measure the underlying condition directly.
For example last summer we had the first tornado in Westchester County in 30 years. There was much debate regarding whether there was a tornado, and if so where was it on the Fujita scale. A tornado expert from NOAA showed up and looked at the damage, determining whether cars had been picked up, whether roofs had been torn off and whether trees had a twisting pattern around their breaks, he then pronounced that we had a strong F1 or a weak F2 tornado. All of these signs of damage are surrogate ways of measuring whether there was a tornado and what its wind speed was.
When a car is involved in a serious accident, the police are very busy afterward taking measurements of skid marks and other signs of speed, failure to yield and other indicators of what happened. All of these measures are surrogate measures of what actually happened since there are no printouts yet available from the cars themselves regarding “black box” measures, as is available in airplanes after an accident, logging the actual conditions that led to the accident. These measures are made in order to make assumptions about those conditions.
Surrogate measures abound in the medical world with doctors examining you for all kinds of signs of underlying medical conditions that are not being directly observed. These initial signs may lead toward more involved tests, which may be more evolved or advanced surrogate measures to help further define your ills. Unfortunately, sometimes the actual extent of the illness can not be determined until the surgeon cuts into you and takes a look or a biopsy of the tissue itself is done. The fact that autopsies are still done to the extent that they are indicates the limits of surrogate measures in medical diagnosis.
A subset of surrogate measures that has always intrigued me is called unobtrusive measures. When surrogate measures are used to make assumptions about an underlying condition in such a way that the people affected may not even realize that they are being measured, the measurements are called unobtrusive. An example would be looking at the wear and tear of floors in museums to determine the popularity of exhibits. The assumption being made that those exhibits surrounded by excessive wear and tear are more popular then those holding up well (as opposed to the possibility of poor quality flooring in front of those exhibits).
A whole set of potentially bias related surrogate measures are used when we observe people, their appearance, their dress, hair length, skin color, age, gender etc. and use these observations to make assumptions regarding their beliefs, expected behaviors or capabilities. These types of surrogate measures which can be quite poorly correlated to the reality of beliefs and behaviors will often lead to inaccurate conclusions and come about because of a tendency of people to quickly categorize others – often inappropriately.
Organizations and Organizational Psychologists use a whole host of surrogate measures in determining or making assumptions about underlying conditions in organizations and for use in employee selection. Surveys of organization attributes are measuring those attributes using a series of surrogate measures. If we ask someone if they have the training they need to do their jobs, we are surmising that if they answer affirmatively they will actually be capable of better job performance than someone responding in the negative. Some of the questions used in organizational surveys are better surrogate measures than others and it is the role of the survey expert to write questionnaires that use the best measures possible – those surrogates that will be better indicators of actual or potential organizational performance. Indices tied to constructs (e.g. loyalty, engagement) are simply average of surrogates or construct surrogates.
Let’s take a step back for a minute and ask ourselves why are we doing organizational surveys in the first place? In my opinion, the whole purpose of conducting organizational surveys is to help improve organizational performance, the effectiveness and efficiency of the organization. Many make the assumption, as do I that having an environment that is perceived positively by the work force and has certain attributes can help organizations perform. The survey itself then is a set of surrogate measures asking people about the characteristics of the organizational environment.
A well-done organizational survey uses the best possible surrogate measures to make assumptions about the actual conditions within the organizational environment – those conditions that have been demonstrated to be linked to important organizational outcomes. Surrogate measures within many fields have evolved over time, with more predictive surrogates which are better measures of underlying conditions replacing less informative ones. It is the role, the responsibility, of organizational psychologists and other survey experts to push the field and continue to search for better surrogate measures to help those that entrust us with consulting to their organizations.
© 2010 by Jeffrey M. Saltzman. All rights reserved.
Visit OV: http://www.orgvitality.com