When Metrics StinkWhen Metrics Stink

Did you see the recent Dilbert comic strips featuring the Highly Productive but Useless Guy? The bug-eyed generator of junk delivers a white paper to Dilbert. It's a statistical analysis of the correlation between disk storage and employee absenteeism. "I don't know how to do statistics," he tells Dilbert, "but it doesn't matter, because I didn't have data." The pointy-haired boss tells HPUG that his "P/U ratio" (productivity to usefulness) is skyrocketing. The boss's solution is for HPUG to do less work. It's funny on so many levels, even the childish. (Say "P.U." instead of "P to U.")

This is satire at its best; on its face it's absurd, but it also cuts frighteningly close to the bone. If you give a manager a measurement that represents a ratio, and tell that person to ignore it as long as the number remains below a certain threshold, there are two conditions that will keep any alarms being raised: A relatively large denominator or a small numerator. Doing very little work keeps the P/U ratio very low, keeping that metric in the green and registering a smashing success! Assessing performance from a simplified view, such as a dashboard, balanced scorecard, or summary report, requires great reliance on the application developers as well as the business analysts who create its specifications.

So far, in general, businesses have done a very poor job of creating reliable simplified representations of performance. The Hackett Group just reported that, although almost 2/3 of typical companies have some type of Balanced Scorecard program in place or development, less than 20% have mature Balanced Scorecard implementations that are generating business value. Companies polled by Hackett were reporting an average of 132 measures to management each month, about nine times the effectual number of measures. Hackett also found that too many scorecard applications are overweighted with historical financial information instead of forward-looking measures such as external financial and operating performance. John McMahan of Hackett said when the study was published that it's "no wonder many financial executives look on the concept as an expensive, bloated, and useless substitute for the traditional paper reports."

If you're in one of the many companies struggling to get performance management right, you'll want to read this issue's feature articles. Mark Smith presents a "cycle of improvement" framework for whipping employee performance management applications into line. Craig Schiff helps you come up with sensible metrics. Follow the advice of these expert contributors, and your performance management project has a far smaller chance of becoming a joke.