Why Story Points Don’t Measure Productivity

Every few weeks, engineering leaders sit in the same review meeting. Someone pulls up a dashboard. One team delivered 82 story points. Another delivered 57. A third team has been steadily increasing velocity for six sprints. The conversation almost writes itself.

“Looks like Team A is outperforming everyone else.”

It sounds reasonable. It isn’t. The uncomfortable truth is that one of the most widely used engineering metrics tells you almost nothing about productivity.

That’s not because the story points are broken. They’re doing exactly what they were designed to do. The problem is that we’ve given them a job they were never hired for.

Story points help teams estimate work. They don’t tell you how productive a team is, how efficiently software is being delivered, or whether engineering investment is paying off. Yet many organizations continue to use them to answer all three questions.

It would be like judging a construction company by the number of blueprints it produces instead of the buildings it completes. The blueprint matters. It just isn’t the outcome.

As engineering organizations embrace AI-assisted development, platform engineering, and continuous delivery, this distinction matters more than ever. Teams are building software differently than they did even three years ago. The way we measure engineering performance hasn’t kept pace.

It’s time to separate planning metrics from productivity metrics.

Story points solved one problem. We created another.

It’s easy to forget why story points became popular in the first place. Before Agile teams adopted them, software projects were estimated in hours.

That sounded logical until reality intervened.

A developer would confidently estimate six hours for a feature. Halfway through implementation, they’d discover an undocumented dependency, a legacy service that nobody wanted to touch, or a third-party API behaving differently from its documentation.

The estimate wasn’t careless. The software was simply more complicated than anyone expected.

Story points were introduced to avoid pretending that software development was predictable to the hour.

Instead of asking, “Will this take eight hours or twelve?”, teams asked a different question. “Is this feature more complex than the one we built last sprint?”

That shift worked remarkably well. Teams stopped arguing over hours and started thinking in terms of relative effort and uncertainty.

Somewhere along the way, however, organizations quietly changed the rules.

• Velocity became a KPI.
• Sprint points appeared in executive scorecards.
• Engineering productivity started being discussed in terms of completed story points.

Nobody officially declared that story points measured productivity.

It just happened.

The same work can produce completely different numbers

Imagine two product teams building the same feature. Both are adding multi-factor authentication to a customer portal. The first team estimates the work at three story points. The second estimates it at eight. One quarter later, both teams delivered the feature. Customers are using it. Security has improved.

Which team was more productive? There isn’t enough information to answer that question. The first team may have spent years building a modern authentication platform. For them, adding another login method is straightforward. The second team could be working with a fifteen-year-old monolithic application where every authentication change touches half a dozen tightly coupled systems.

The feature is the same. The customer outcome is the same. The estimate isn’t.

Yet many organizations compare those numbers as though they represent units of engineering output. They don’t. A story point isn’t a standard measurement like a kilogram or a kilometer. It has meaning only within the team that created it.

Comparing story points across teams is like comparing temperatures where one person uses Celsius and another invents their own scale. The numbers may look precise. They’re not comparable.

Bigger numbers don’t always mean better engineering

Here’s another scenario that’s surprisingly common.

A team spends an entire sprint removing technical debt. They simplify an overly complex service, eliminate duplicate code, improve automated testing, and upgrade an outdated framework that has been slowing releases for months.

Customers don’t notice any visible changes. There aren’t many story points completed. On paper, the sprint looks average. Six weeks later, deployment frequency doubles because releases are no longer blocked by fragile code. Production incidents drop because the application is easier to maintain. New features reach customers faster because developers aren’t constantly working around old architectural problems. The sprint that appeared “unproductive” created the conditions for every future sprint to become more productive.

Story points couldn’t capture that. They weren’t supposed to. The same thing happens when engineers improve CI/CD pipelines, automate testing, optimize cloud infrastructure, or reduce build times.

None of these efforts produce impressive velocity charts. Every one of them improves engineering productivity.

When the metric becomes the goal

There’s an old principle in economics that applies remarkably well to software engineering.

Once people start optimizing for a metric, the metric no longer reflects reality.

Consider a leadership team that announces an ambitious objective. “We’d like every engineering team to improve sprint velocity by 20% this year.” Nobody asks developers to inflate estimates. Nobody tells Scrum Masters to assign higher point values. But estimation gradually changes.

Stories that once felt like three-point tasks become five-point tasks because the team has become more cautious. A feature that previously received five points now gets eight because engineers have discovered hidden complexity in similar work.

Nothing dishonest has happened. The team’s understanding of effort has simply evolved. But the dashboard now tells a different story. Velocity increases. Leadership concludes that productivity has improved. The software is being delivered at exactly the same pace.

This isn’t manipulation. It’s human nature.

Every measurement system influences behavior. The question is whether that behavior aligns with what the business actually wants.

If engineers are rewarded for increasing velocity, they’ll naturally focus on velocity. If they’re rewarded for delivering reliable software with shorter lead times and fewer production failures, they’ll focus on those outcomes instead.

The metric shapes the conversation. Eventually, it shapes the culture.

Software development doesn’t look like it did five years ago

Perhaps the strongest argument against using story points as a productivity metric has nothing to do with Agile. It has to do with how engineering itself has changed.

Take something as ordinary as writing an API today. A few years ago, a developer would start with documentation, create boilerplate code, write validation logic, implement tests, review coding standards, fix linting issues, and manually document the endpoint.

Today, much of that groundwork is handled in minutes. AI coding assistants generate scaffolding. Unit tests are suggested automatically. Documentation can be created alongside the implementation. Static analysis catches issues before code review even begins.

The engineer’s role hasn’t become less important. If anything, it has become more demanding. Less time is spent typing code. More time is spent making architectural decisions, validating AI-generated output, understanding trade-offs, reviewing security implications and ensuring that what gets deployed is actually fit for production.

That’s where the real value lies. Yet many engineering dashboards still behave as though productivity is measured by the amount of work teams estimated at the beginning of a sprint.

They’re measuring planning. The industry has moved on.

Why Story Points Don’t Measure Productivity

If story points don’t tell us whether engineering is becoming more productive, what does? There’s no single number that answers the question. And that’s probably a good thing.

Software engineering isn’t a production line where every unit of work is identical. One sprint might involve building a new customer feature. The next could be spent fixing security vulnerabilities. Another might focus entirely on upgrading infrastructure before it reaches end of support.

Looking at any one metric in isolation rarely tells the full story. The organizations getting this right have stopped searching for a magic number. Instead, they’re asking a better question:

“Is our engineering system getting better over time?”

That shift changes everything.

Start measuring flow, not just output

Imagine a customer reports a bug on Monday. The fix is written by Tuesday morning. It reaches production on Friday evening. At first glance, it feels like a four-day turnaround.

But what actually happened? The developer spent three hours fixing the issue.

The code review waited almost a day because the reviewers were busy. The CI pipeline took another hour. Security approval added eight more hours. The release window didn’t open until Friday.

The bottleneck wasn’t engineering. It was the process surrounding engineering.

Story points don’t expose that.

• Cycle time does.
• Lead time does.
• Flow efficiency does.

These metrics don’t ask how much work a team estimated. They show how work moves through the delivery system and where it gets stuck.

That’s why many engineering leaders are paying less attention to sprint velocity and far more attention to how quickly good ideas become working software.

DORA changed the conversation for a reason

When Google’s DORA research became widely known, many organizations expected another list of engineering KPIs.

Instead, they got something much simpler.

The framework focused on four measures:

• Deployment frequency
• Lead time for changes
• Change failure rate
• Mean time to restore service

None of them mention story points. None of them count commits. None of them measure how many hours developers worked. They focus on delivery.

More importantly, they balance speed with stability.

Shipping code every hour isn’t impressive if production breaks every afternoon. Likewise, achieving near-perfect reliability isn’t enough if releases happen once every three months.

High-performing engineering teams improve both at the same time. That’s much harder than increasing sprint velocity. It’s also much more valuable.

Productivity isn’t just about writing code

One of the biggest misconceptions in software engineering is that developers spend most of their day developing software.

Ask almost any experienced engineer how their week went and you’ll hear something different.

“I spent half the day waiting for a build.”

“We couldn’t test because the environment wasn’t available.”

“Code review took two days.”

“We lost a day because a dependency changed.”

Very little of that appears on traditional engineering dashboards.

Yet these small interruptions quietly consume hundreds of engineering hours every month.

Imagine employing two hundred developers. If each developer loses just thirty minutes every day waiting for builds, approvals or unstable environments, that’s roughly one hundred engineering hours disappearing every single day.

Over a year, you’ve lost the equivalent of several full-time engineers without hiring or losing a single person. That’s not a talent problem. It’s a systems problem. The most productive engineering organizations don’t simply ask developers to work faster. They remove the friction slowing them down.

AI has made the old metrics even less useful

Generative AI has sparked a fresh debate about engineering productivity. Many organizations expected AI coding assistants to double developer output overnight. Some expected dramatic reductions in delivery timelines.

Neither happened. At least not in the way leadership expected.

Consider two engineers implementing the same feature. The first writes every line manually. The second uses AI to generate test cases, boilerplate code, documentation, and repetitive validation logic. Both deliver the features. The second engineer finishes earlier.

What changed? Not the story points. Not the customer outcome. The productivity gain came from reducing repetitive work, allowing the engineer to spend more time solving the difficult problems that AI can’t solve on its own.

That’s exactly why measuring AI adoption through story points misses the point. The benefit isn’t that developers complete more estimates. It’s that they spend less time on low-value activities.

The organizations seeing the greatest return from AI aren’t asking whether engineers write more code. They’re asking whether engineers spend more time on engineering. Those are very different questions.

Engineering leaders need answers, not dashboards

Most organizations already collect an enormous amount of engineering data.

• Git repositories.
• CI/CD pipelines.
• Issue trackers.
• Testing platforms.
• Production monitoring.
• Cloud infrastructure.

The problem isn’t a lack of metrics. It’s that the metrics live in different places and rarely explain one another.

A dashboard might show deployment frequency dropping by 25%.

Useful information.

But why? Was it caused by longer code reviews? A surge in production incidents? Build failures? Security approvals? Infrastructure instability?

Without context, engineering leaders are left making educated guesses.

That’s where engineering intelligence changes the discussion. Instead of presenting disconnected metrics, it connects signals across the software delivery lifecycle to explain what is happening inside the engineering system.

Not just what changed but why has it changed.

That’s a much more useful conversation for a VP of Engineering than simply reporting that velocity fell by eight points last sprint.

This is where Matrix fits

At Ness, we often see organizations investing heavily in modern engineering practices while still measuring success with legacy metrics. They’re adopting platform engineering, cloud-native architectures and AI-assisted development.

But leadership dashboards still revolve around sprint velocity.

That’s a missed opportunity.

Modern engineering generates rich operational data across planning, coding, testing, deployment, and production. The challenge isn’t collecting more information. It’s turning that information into decisions.

Matrix was built with that challenge in mind. Instead of treating engineering metrics as isolated numbers, Matrix brings together signals from across the software delivery lifecycle to reveal patterns that individual dashboards can’t.

It helps engineering leaders answer practical questions like:

• Why are releases slowing down?
• Which delivery stages create the biggest bottlenecks?
• Where is developer time being lost?
• Is AI actually improving engineering throughput?
• Which improvements will have the greatest impact on delivery performance?

Those insights help leadership focus on fixing the system rather than pushing teams to increase arbitrary numbers.

That’s the difference between reporting engineering activity and understanding engineering performance.

The conversation needs to change

Story points aren’t going away. Nor should they. They’re still an effective planning tool inside Agile team. The mistake is expecting them to answer questions they were never designed to answer.

Engineering productivity has always been more complex than counting estimates. Today it’s influenced by developer experience, delivery flow, automation, architecture, platform maturity, AI adoption, and operational reliability.

None of those fit neatly into a sprint velocity chart. Perhaps that’s uncomfortable. Leaders naturally prefer a single number. But software engineering has never been that simple.

The organizations pulling ahead aren’t the ones completing the most story points. They’re the ones shipping reliable software faster, reducing engineering friction, learning from delivery data, and continuously improving how engineering works.

That’s what productivity looks like in modern software engineering. And it’s far more interesting than the number assigned during sprint planning.

Why partner with Ness?

Engineering organizations don’t need another dashboard. They need better questions and better answers.

Ness combines decades of software engineering expertise with an Intelligent Engineering approach that helps organizations understand how work flows across the entire software delivery lifecycle. With Matrix, engineering leaders gain visibility into delivery bottlenecks, developer experience, software quality and operational performance—turning engineering data into actionable insight rather than retrospective reporting.

As AI reshapes software development, the organizations that succeed won’t be the ones measuring more metrics. They’ll be the ones measuring the right ones—and using those insights to build better software, faster.

Because engineering productivity isn’t about how much work was estimated. It’s about how effectively ideas become value for the business.