In my years working with teams, nothing I have ever encountered can be more demotivating, more demoralizing, or more USELESS than poorly considered metrics.

There is a fundamental misunderstanding in the world today that metrics are an end in themselves. In other words, we create the metrics we believe make the most sense and then start measuring teams with them. As a result, we end up with ridiculously ineffective metrics. For example, measuring the amount of work “committed” during sprint planning to the amount of work that was actually finished by the end of the sprint – this metric, in my experience, LOWERS productivity and DECREASES outcomes – definitely not the desired outcome!

I did a web search on “what metrics” and “Scrum team” and got an endless list of nonsense blog posts, the most ridiculous of which is entitled “should management review the scrum metrics of scrum teams?” I’ll actually answer this question later in this blog post. But let’s start with this – there’s no such thing as “scrum metrics.”

A lot of people ask, “What metrics should I use with my Scrum team?” However, that’s not the right question.

In this blog post, I’m going to introduce you to a basic structure that anyone can use with any team (not just Scrum teams) that will answer all of your questions about metrics for now and forever. All you have to do is follow the steps I’ll outline for you.

Step 1: What are your organizational goals?

Take a minute, grab a piece of paper or open a Word document. Write down your organizational goals. If you don’t know them, talk with your manager. If your manager doesn’t know what they are, forget metrics for now, and focus on organizational goals. Come back to metrics later.

When we talk about goals, we’re talking about something that has a clear goal date and a clear achievement. These are often called SMART goals. SMART goals have five attributes:

• Specific – the goal is clear and specific, “achieve a x% improvement in sales” not “sell more.”
• Measurable – the goal to be attained is clear, “achieve a 15% improvement” as opposed to “get better at this.”
• Achievable – the goal is challenging, but achievable (goals that are too easy don’t inspire, goals that are too hard don’t motivate)
• Relevant – the goal has makes sense and the team can relate to it.
• Time-bound – the goal has a target date, “achieve a 15% improvement by June 30,” not “as soon as possible.”

When you have your organizational goals written down, proceed to step 2.

Step 2: Connect your organizational goals to your team

Sit down with your team and your manager and talk through the organizational goals. Discuss what the team could do to help achieve the organizational goals – connect the organizational goal to a team goal. Organizational goals are often bigger and involve more than what your team would provide; your team can have only an indirect impact on most organizational goals. When we “connect” the organizational goal to the team, we ask what specifically the team can do that would help the organization achieve its goals.

When you create the connection between organizational goals and your team, consider more than simply the first order aspects of the organizational goal. First order aspects are things that your team can do that will directly support the organizational goal. For example, if the organizational goal is to “improve Product A sales by releasing release 2 on time” the first order aspects are the things that your team needs to do to finish release 2 on time (most importantly, building the desired functionality). But there are second order aspects as well:

• Outcome goals – how can the team produce more value in a single sprint?
• Quality goals – assuming quality needs to be improved, what might your team need to do to improve quality? Finishing your work on time but with lots of defects won’t support the organizational goal in the long term.
• Waste reduction goals – what could your team do that would reduce operational costs while improving outcomes? Missing your release dates could be caused by too much waste, no matter how hard your team works.
• Learning goals – in what learning could your team invest that would support accomplishing development objectives on time? The more your team learns, the more likely they will find ways to deliver better functionality.
• Feedback goals – from whom could your team get more feedback to ensure that they built the right features right the first time? Better feedback can create better outcomes.

Discuss all of these possibilities with management and your team (preferably together). The outcome of the discussion should be a list of goals (10-20) that support the organizational goals, but represent achievements that the team can direct impact. These goals connect the team’s work to the larger organizational goals.

Make sure you write SMART goals. For example, “by March 15, the team will have at least one person who can write and update stored procedures.” And don’t buy into the old excuse that you can’t write a goal that has a specific date or measure. Yes, you can. If you don’t think you can, you don’t have the goal properly defined yet. Go back and work on it some more.

By this point, every goal on your list is something on which your team can have a direct impact. Proceed to step 3.

Step 3: Order the Goals by Impact

Now, working with your team and management, order the goals on your list by impact and date. In other words, the goals with the greatest impact on organizational goals should be on top of the list in rough date order while goals that are still worthwhile but will have less of an impact should be ordered further down the list. Keep in mind that first order impact goals will generally come first. For example,

1. “By June 30, complete the high value features for Product A Release 2” (first order impact)
2. “By March 30, reduce new defects found after integration testing to 3 per sprint” (second order impact, quality goal)
3. “By April 30, maintain an average of at least two stakeholders in two consecutive Sprint Review events” (second order impact, feedback goal)
4. “By April 15, have at least one person on the team who can use the new analysis tool” (second order impact, learning goal)
5. “By April 20, improve Sprint Planning so that the team has no more than 1 “information hold” during the Sprint” (second order impact, waste reduction goal)
6. and so on…

At this point, you have a list of connected goals, ordered by the degree of impact on organizational goals. You’re ready for step 4.

Step 4: Start With the Highest Impact Goals

Step 4 begins at your team’s next Sprint Planning event. At the very beginning of the event, the team is supposed to work out the “WHY” for the Sprint. This is where your goal list comes in. Start talking through the goals with the team and get a decision on which goals will be addressed in the sprint being planned. Don’t plan too many goals into your sprint. In fact, since many goals take more time than a single sprint to achieve, the team will probably have to break the goals they want to include in the sprint into smaller pieces (like Story Slicing).

A great example of this is the first order impact goal: “By June 30, complete the high value features for Product A Release 2.” Your team isn’t likely to finish all of the necessary work in a single sprint. What could they focus on? What feature or group of features could be done this sprint?

Likewise, you might need to step into a goal: “By March 30, reduce new defects found after integration testing to 3 per sprint.” Perhaps the team’s current new defect count is 15. I don’t think going from 15 to 3 in one sprint is achievable. How about 12 instead?

Note: we use the term “information hold” in the following paragraphs. An “information hold” is what my teams often call an instance where a Product Backlog Item is blocked more than one day for information the team does not currently have.

Similarly, the goal “By April 20, improve Sprint Planning so that the team has no more than one information hold during the Sprint” might not be achievable in one sprint. If the team is currently at 10 information holds, perhaps the team should shoot for 8 in the sprint they are planning?

Remember, you don’t want more than 3 goals in a single sprint. Just like the adage about too many “top” priorities (“if everything is a top priority, nothing is”), too many goals can have the same problem. So, at this point, your team might have this list:

By the end of the sprint,

1. complete user account definition as required in release 2 (first order impact goal reduce to a group of functions – the ability to properly define a user account)
2. reduce new defects found after integration testing to 12 per sprint.
3. reduce information holds during the sprint to 8 through more in-depth sprint planning.

This, by the way, is called a SPRINT GOAL.

You are ready for Step 5.

Step 5: Determine metrics

Ah! There’s that word again: metrics. Except now we’re ready for them.

Look at the sprint goal from earlier. The necessary metrics are now straightforward.

• Complete user account definition – measured by completing all of the product backlog items in the sprint that help complete the user account definition.
• Reduce new defects – measured as the defects found in integration testing; the goal for this sprint is twelve.
• Reduce information holds – measures as the number of times during the sprint that a product backlog item is blocked for more than a day; the goal for this sprint is eight.

This is how the metrics become obvious once the goals are understood. Also, this is the proper use of metrics — not created for their own sake, but as a way of measuring the achievement (or maintaining) of a goal.

During sprint planning, your team should discuss 1) which product backlog items will achieve the user account definition, 2) what they need to do differently to reduce defects in integration testing, and 3) what they need to do to reduce information holds during the sprint.

Step 6: Inspect and Adapt

During the sprint retrospective, your team can review goal attainment, discuss what they could have done better, and even decide which goals to include in the next sprint.

Some goals will carry forward – for example, the team wants to reduce information holds to one; they reduced it to eight in the previous sprint. So, they might decide to repeat the same goal in the next sprint with a lower measure (five, this time, instead of eight).

Oh, By the Way…

Yes, the goals and metrics being used by a team SHOULD be reviewed by management. After all, a manager has as much (sometimes, more) at stake as the team in achieving goals. When the metrics are properly constructed (as I’ve described in this post), any good manager should be able to see what she and her employees can do to help the team improve.

If you’re a manager or leader and aren’t sure how to handle a discussion with their teams about improving on their goals, check out The Leadership Edge.

What Next?

Once a goal is achieved, the team might still want to measure for it (for example, continuing to measure information holds during the sprint in case it increases again), but they needn’t keep repeating the goal once it’s achieved.

New goals will be added on a regular basis. Just make sure that the goal is properly ordered within your goal list. Old or outdated goals get removed.

What About Velocity (Productivity) Metrics?

Measuring velocity is one of the most worthless things that companies do today. Velocity is not something that a team can directly impact – teams can’t go faster just because we ask them to. Velocity is affected indirectly as best (that is, the team does something that causes something else to happen that causes velocity to improve). Velocity can be useful for answering questions like “how much work can we load this sprint?” or “where in the product backlog where we probably be in five sprints?” It has been, in fact, recommended, that teams understand their current velocity for planning and prediction purposes.

However, when management measures velocity as a goal in itself, it does nothing but place unwarranted and unhelpful stress on the team. My recommendation: get rid of all productivity metrics. Instead, if your team wants to improve their outcomes, have them discuss things they can actually DO that will, indirectly, improve productivity.

Instead of useless goals like: “improve productivity by 10% over the next three sprints,” try “reduce information holds by 50% over the next three sprints” or “reduce daily work-in-progress,” things that will actually reduce waste, resulting in an improvement in productivity.

When faced with a useless goal like “improve productivity by 10%,” teams often resort to either packing more into the Sprint than they know they can do or padding their estimates so that it appears like they are doing more.

Agree Upon Goals, Let the Metrics Emerge

Metrics are an end result of a goal-based discussion. They aren’t an end in themselves. When management wants to talk about metrics, you make the discussion about goals. For example, imagine someone says to a Scrum team:

“I need you to improve productivity by 5% within two months.”

The best response is, “can you tell us what it is you are trying to achieve with the extra 5%?” In other words, a 5% increase in productivity is rather arbitrary. What do they really want?

Perhaps they tell you that they need “Release 2.1” to be done on-time. If so, they’ve really missed the mark. Getting a project done on time is more about building the right things, not building everything faster. Instead of a productivity goal, focus on a value-based goal.

Perhaps they tell you that they believe your team is dragging their feet (sometimes called “sand-bagging,” wasting time, etc.). Your next move is to ask, “why do you feel that way?” Then you can address reality instead of a perception.

Productivity as an end in itself is rarely a worthwhile goal. The best goals do one of three things:

1. improve capability,
2. improve value, or
3. reduce waste.

If your goals aren’t doing one of those things, I’d recommend rethinking them.

Once you’ve got the goals figured out, the metrics emerge from the goals and MAKE SENSE!

Your Thoughts?

Tell us what you think about this blog post, tell us about your team’s goals, and definitely share the most ridiculous metrics you’ve encountered.

If you appreciate the value of this blog post, we recommend you take a look at Artisan Agility’s new training system, “The Leadership Edge,” where we discuss team goals and hundreds more useful tips and skills you can use for your teams and your entire organization.