Thanks to 20th century management theory, the world is filled with metrics and measures that are questionable. Many are useless. For example, a commonly used but effectively useless metric for knowledge creation workers (like those in IT, marketing, and other creative careers): utilization.

Utilization is a metric that was meant to measure the percentage of a day a machine on the factory floor was being used. The greater the utilization of the machine, the lower the cost passed on to the manufactured product. Utilization for people? Nonsense. People work in focused as well as unfocused states. Both states look the same, but both have radically different performance characteristics.

If you want to measure how long someone sat in front of a keyboard and typed stuff, go with utilization. But if you really want to know how much productive work is going on, you’d be better off measuring interruptions (see our fantastic training system, The Leadership Edge, for much more information about productivity and metrics).

Dr. Eliyahu Goldratt, author of Critical Chain, The Goal, and Necessary But Not Sufficient put it like this:

“Tell me how you will measure me, and then I will tell you how I will behave. If you measure me in an illogical way, don’t complain about illogical behavior.”

Metrics can do a lot of damage to an employee and a team. In this blog post, I want to provide a simple framework for creating metrics to measure both teams and employees.

Step 1: The Right Goal First

Goals come first. Metrics without goals are just numbers. Improper goals are even worse than none at all. For example, here’s a story about another questionable metric: velocity.

Velocity, when used in an Agile organization, is a metric that measures the amount of work done by a team during a Sprint. The effectiveness of velocity is minimal. It can be successfully used to approximate how much work fits into a Sprint so the team doesn’t waste time planning work it can’t do. Velocity can be used by Product Owners reporting updates to stakeholders by projecting how much work the team will likely complete over the next “n” Sprints.

However, the most popular mis-use of velocity is to create the following “goal:”

“Improve velocity by 10% over the next 6 Sprints.”

This goal is useless. What is the team supposed to in order to achieve the goal? work longer hours? Work faster? Change the way they estimate? No matter what’s happening, this goal won’t work. Productivity improvement is a by-product of doing the right things right; it’s not something we can generally directly influence.

What might be the right goal? Here are a few, all of which will result in improved productivity:

1. By the end of 2Q21, reduce in-Sprint interruption occurrence by 15%.
2. By the end of 2Q21, reduce in-Sprint interruption duration by 50%
3. By the end of September, reduce average PBI size to no more than 3 story points.
4. By the end of September, reduce average daily work-in-progress to 3.

All of these goals, if understood by the team and plans are put in place to achieve them will result in improved productivity.

Step 2: Plans and Metrics

Let’s use the first goal, “by the end of 2Q21, reduce in-Sprint interruption occurrence by 15%,” as an example. If we’ve set up the right goal, the plans and the metrics we need should be fairly straight-forward.

The goal of reducing in-Sprint interruption occurrences makes it pretty clear that the developers need to come up with strategies for deferring interruptions during the day, eliminating some (by suggesting another way to get an answer or getting someone in front of the developer to intercept the interruption) and batching the rest (so that most of the interruptions can be dealt with in a single block of time).

The metrics we need to achieve success are fairly easy to identify:

1. the number of interruptions experienced by developers (or, perhaps, the average number per day) during the Sprint.
2. the velocity of the team

If the developer’s plan to reduce interruptions works, we should see the number of interruptions go down while the velocity of the team increases.

Step 3: Plans and Decisions

Once you have a good goal and the proper metrics, the next step is to outline the actions and decisions to take depending on what the metrics do. You’ll find actions and decisions are easy to plan if the goal and the supporting metrics make sense. If the metrics change the way we expect, at some point we will achieve our goal; we can raise the bar or change the goal entirely.

However, if the metrics don’t change the way we expect, what should we do? When should we start doing it? Consider, what should the team do in each of these instances?

1. Velocity goes up (good) BUT Interruption Occurrence also goes up (bad).
2. Velocity goes down (bad) BUT Interruption Occurrence goes down (good).
3. Velocity goes down (bad) AND Interruption Occurrence goes up (bad).

The team should be evaluating their goals and metrics during every Sprint Retrospective. Did we count interruptions incorrectly? Were there unusually large or small backlog items that might have skewed our velocity? Did something unusual happen during the Sprint? What actions should the team take?

Conclusion

Too many Scrum teams suffer from metrics that are useless and often do nothing more than distract the team from the important work of creating value. While velocity is a commonly abused metric, there are many more poor metrics you should be on the lookout for.

Identifying a good metric is easy:

• Is there a clear, realistic goal the team can achieve?
• Are there clear and obvious actions that derive from the goal?
• Are there clear and obvious metrics that derive from the goal?
• Are there clear plans that derive from the goal and metrics?
• Can decisions be reached by examining the metrics?

If any of these questions can not be answered “YES,” rethink your team’s goals and metrics.

Metrics that exist on their own become a source of dysfunction. Clear goals, on the other hand, can be achieved.