Output Metrics vs Outcome Metrics

Engineering teams have access to more delivery data than ever. Jira can measure cycle time and velocity. GitHub can track pull requests and code changes. Deployment tools can show how often software reaches production and how reliably those releases perform.

These metrics are useful, but they only tell us how efficiently a team produces software. They do not tell us whether that software made the product better.

In this episode of The Product Engineering Podcast, James explores the difference between output and outcome metrics, and why engineering leaders need both.

The episode begins with the McNamara fallacy: the tendency to focus on what is easy to measure while overlooking information that may be more important. Engineering organisations can fall into the same trap when they treat delivery activity as the primary measure of success.

James explains why output metrics work well for predictable work such as migrations and platform upgrades, where the intended destination is already understood. Product development is different. Teams are often working with uncertain assumptions about customers, behaviour and business impact. Efficient delivery cannot compensate for building the wrong thing.

The episode also covers:

• The difference between output and outcome metrics
• Why delivery metrics are easier to measure and attribute
• When metrics such as cycle time, deployment frequency and story points are useful
• Why product engineering cannot be managed like a manufacturing line
• How feature factories emerge when teams are rewarded mainly for shipping
• The growing importance of outcome accountability as AI makes implementation faster
• Why teams should decide how they will measure success before delivery begins
• The difference between leading and lagging indicators
• How outcome measurements can guide iteration and product decisions

James shares an example involving two engineering teams. One completed a predictable React migration, while the other delivered a navigation redesign that users disliked. Both teams delivered their planned work successfully, but only one produced the intended result.

The example demonstrates why delivery success and product success are not the same thing.

The question engineering teams should ultimately be able to answer is simple:

Did the product get better?

Learn more about The Product Engineering Podcast at doproductengineering.com.