Measure What Truly Matters (With Heart Intact)

During my bachelor's degree I came across a quote — "what gets measured gets accomplished" — that emphasizes how focusing on the right metric is just as important as choosing what to focus on.

Humans have feedback mechanisms we inherited as a gift of evolution. That's one reason why setting a goal and measuring progress against it is one of the best ways to achieve what you want. Creating a feedback system that inherently motivates people to do great work is a responsibility of leadership. Choosing a goal also means choosing the metrics it's evaluated against. The SMART framework for goal-setting is a good place to start.

When it comes to software, measuring the performance of a delivery team is harder. Our end goal is always a moving target, the context teams operate in differs, the unit of measurement is defined by the team, and quantifying one team member's contribution is nearly impossible. Because of this, we can't measure software delivery performance the way a manufacturing or product organization does. In software we're offering more of a service than a product, so most of the good metrics are qualitative and subjective. That may be why software teams often rely on gut instinct or neglect quantitative measurement entirely. Qualitative measurement has its advantages too — but when it comes to objectively measuring performance improvements, quantitative measurement is the only option.

Measuring people over the software delivery process

When there are no objective measurements for performance, decision-makers tend to measure people as a proxy for delivery performance. This is especially dangerous when those decision-makers are separated from the delivery process — because your best performers aren't always in your line of sight. They're busy doing work rather than showing off. When your measure of performance is how energetic an individual is in a meeting (the visible part), you may overlook your highest-performing introvert who rarely speaks yet delivers 10x more than the benchmark.

This is one reason I believe engineering managers must work closely with engineers — ideally alongside them on the ground — to get a more accurate representation of reality. It's one of the best ways to avoid the trap of measuring people as a proxy for a team's performance.

Measure to improve, not to evaluate

In a fear-driven culture, people use measurement as a tool for evaluating individuals. In a culture designed for engineering excellence, measurement becomes the starting point of the improvement cycle — a natural part of the team's feedback mechanism. Note that I'm talking about software delivery performance, not how to evaluate people. The primary focus of measuring delivery performance in a true DevOps culture is to improve engineering processes so the team can "deliver quality software faster" — not to evaluate people for firing decisions.

Play stupid games, win stupid prizes

It's true that what gets measured gets accomplished — so when you measure the wrong things, you achieve the wrong outcomes. Here are common wrong measurements (and their prizes) in software engineering, in my opinion (open for discussion).

Over-reliance on estimates rather than delivery. When a team is evaluated based on the estimate it gave versus the time something actually took, the team spends hours producing estimates and documentation purely for evaluation. I believe real quality metrics emerge through the natural course of the development process. (In auditing we say audit evidence is more reliable when it arises from the natural course of business.) Estimates are opinions about uncertain future workload, and enthusiastic team members tend to give optimistic estimates. The stupid prize: huge amounts of time spent estimating instead of building, and — over time — increasingly pessimistic estimates so people can look like overachievers. I'm not suggesting you get rid of deadlines. A good deadline pushes people to achieve more when it's directly tied to a business outcome that creates real value, rather than a number born in someone's imagination.

Measuring hours worked, lines of code, code coverage, or feature count as a proxy for contribution. Some of these are obviously flawed (in my opinion), yet people still use them — I sometimes use them to evaluate my own work too. 😅 The reason is that they're easy to measure without ambiguity. Yes, you can count lines of code, but does that reflect how well the program solves the problem, or how maintainable it is? Any engineer knows the answer. These measurements not only fail to capture delivery performance — they promote bad design that degrades quality over time. They're small proxies for something else (code coverage is a proxy for whether someone is doing good unit testing); the problem is becoming obsessed with the proxy instead of the real outcome.

A solution

As I said, measuring software delivery performance is hard. There are frameworks and tools to help engineering managers measure their improvements — one is the DORA metrics, built by researching great software teams across industries. They measure delivery improvement along two dimensions: speed and stability. In my opinion it's a great tool for measuring the impact of changes you make to your delivery process. (More on DORA in Speed and Stability Are Not Opposites.)

Final thoughts

Having the right metrics is key to success — but metrics should never be the end goal. They give the best approximation of the result you're trying to measure, so the best way to use them is to improve processes and guide decisions toward engineering excellence. I'm a big believer that a good feedback mechanism — built on the right metrics, honesty, and care — will always motivate people to do great work.