Managers use measurements every day to guide their analyses, decisions, and planning. But even the simplest measurements can mislead. Indeed, measurement is much more difficult than most managers appreciate. Managers must protect themselves by understanding weaknesses in measurements and taking these weaknesses into account as they use them.
Consider the following scenario involving one of the most basic units of measure: time. I have a clock in my office that synchronizes itself at 1:00 am every day using a signal sent out by the National Institute of Standards and Technology (NIST) in Fort Collins, Colorado, which has an advertised precision in the fraction-of-a-second range. It should be trustworthy, right? Not so — it can be wildly inaccurate. For example, on Thursday, October 29, at 4:44 PM (as verified three different ways), the clock read, “Friday, November 2, 4:52 PM.” The last time November 2 fell on a Friday was in 2012, and the next time is in 2018; essentially, the clock was off by about three years and eight minutes!
Of course, we could argue that this is just a flaw in the product. But in my personal life, I can also cite bad measurement stories involving my bathroom scale, smartphone, and GPS. While hardly the stuff of cocktail party banter, when people think about it for a moment, many recall similar instances when supposedly simple measurements were way off.
My point here is not to trash my devices or the work of NIST. Rather, it is to illustrate that, even under the best of circumstances, measurement is incredibly difficult. And the things of interest to managers, such as the size of a market, the effects of an advertising campaign, and the true costs associated with a poor quality product, are far more complex than time, weight, and distance. Because of this I urge managers to adopt a healthy skepticism of all measurements until they understand them deeply.
Here are a few steps to dig into your measurements and identify if they can be trusted:
1. Clarify what you want to know. Too many managers give this step short shrift. Suppose you need to want to know how long a process consisting of three steps (A, B, and C) takes. It is easy for people to interpret this: the time it takes to complete A plus the time it takes to complete B plus the time it takes to complete C. But here’s an alternative: the time it takes to complete A plus the time it takes to complete B plus the time it takes to complete C, plus the queue time between each step. Both have valid uses, but owing to the queue time components, the two can be quite different. So you must be clear about you really want.
2. Understand how actual measurements line up with what you want to know.;