Agile Zone is brought to you in partnership with:

I'm Chief Scientist at proAgile Ltd. You could pretty much say that software engineering methodologies are my bag. I specialize in agile transformations at enterprise scale, and tweet and blog quite actively about this. I'm also the curator at Ian is a DZone MVB and is not an employee of DZone and has posted 38 posts at DZone. You can read more from them at their website. View Full User Profile

Method Wars: On the Commoditization of Estimates

  • submit to reddit
In theory there is no difference between theory and practice, but in practice there is.
-  Yogi Berra

Some background

A couple of weeks ago, Dean Leffingwell broke his silence on Ken Schwaber’s recent blast about the Scaled Agile Framework. For those of you who haven’t been following this latest skirmish in the method wars, Ken – the co-founder of Scrum - tore into SAFe and dismissed Dean’s creation as a dangerous and essentially unagile spin on RUP. There’s politics involved in all of this of course, but there’s also a technical matter at its heart. Scrum is a conceptually simple framework, very clean, and remarkably non-prescriptive in the advice that it gives regarding its implementation. Yet as a good model, it should in theory scale well, and remain entirely applicable in multi-team environments at an enterprise level. SAFe has a different philosophy behind it: the real world is a messy place, and we cannot be naïve about this; we must extend and tailor model-theoretic approaches to suit the complex and often dirty conditions we have to deal with in practice.

I know I’m generalizing, but that’s essentially the context behind this dispute. Can Scrum theory, as far as it goes, really scale in a practical way? SAFe advocates have argued otherwise. On the other hand, is the theory behind SAFe too compromised to be viable as a practical agile framework? Some hard-core Scrum advocates are inclined to think so.

This might seem an awkward place to draw battle-lines, because we would expect agile theory and practice to meet. Yet as Yogi Berra rightly pointed out, in theory there is no difference between theory and practice, but in practice there is.

Into the Fray: The Normalization of Estimates

I’ve already documented my own thoughts about SAFe and some of the issues I found in applying it. One of the experiences I touched on was the thorny matter of comparing story points across teams:

Normalizing story points between teams robbed them of control of their own process. Their metrics were no longer their own for the purpose of inspection and adaptation. Instead they were used by management, ostensibly to facilitate planning, but also with a view to comparing teams. There was a real danger of "point-productivity" becoming more important than the actual delivery of potentially releasable increments. SAFe's contention that "with adjustments for economics of location, (US, Europe, India, China, etc.), work can be estimated and prioritized based on economics by converting story points to cost" is hard to consider with a straight face.

Dean was gracious enough to reply:

Velocities are NOT normalized across teams. Estimating is. If you don’t normalize estimating, then there is no meaningful economics; you can’t figure ROI if you don’t know that the “I” is. If you want to scale agile, and there is no meaningful way to bid work across teams, and within a program, you will be blocked before you even try. (Executive view: “That agile thing; not ready for prime time. They use gummy bears per sprint as their main metric, and they can’t estimate work in any meaningful way. They just don’t get it. I know, let’s just have the PMs do a work break down structure; that we understand. And of course, we’ll need good specs to do that, so let’s stop and write the SRS now”.) Then Agile loses.

Here we get to the nub of the issue. Dean is accepting the fact that managers have certain prejudices, and that Agile will “lose” if we don’t make certain compromises in order to accommodate them. It’s arguably a very pragmatic position to take. Scrum, on the other hand, takes a stance that is more in line with essential agile principles. You can only shift so far, then you are not being agile at all. Normalizing estimates is unacceptable, because in an agile world the value does not lie in, and should never be mistaken for, any forecasts that might be made. Agile practice is founded in empiricism, where the proof of success can only come from delivery.

Here’s what Johanna Rothman told me about using – or rather misusing – estimates for this purpose:

Placating management doesn’t change the conversation. We have to help management understand why value is part of the conversation, along with incremental funding. And that means technical teams have to do their part, and deliver value on a regular basis.

The Commoditization of Estimates

What Ken Schwaber said to me indicated that he was on the same page.

The only reason for estimating is for a team to get its arms around how much they should try to take on.

Of course he’s right, in so far as that represents a canonical take on what estimates should be used for in an agile way of working. Where Ken astounded me was in making the following very perceptive remark.

I’ve found that estimates cannot be commoditized.

That’s a wonderful interpretation, because it describes what is happening so succinctly. In SAFe, estimates are treated as being a tradable product of sorts, in so far as work can be bid across teams. That’s commoditization and it is indeed unagile, because the value of what you are doing has to lie in actual delivery. That is the only proof of success. The only acceptable commodity is a working product.

Even so, I felt that Dean was on to something. I have met the sort of board-room horrors he describes and is trying to “placate”, as Johanna put it. I had to give Ken the following reply:

If you make your estimates transparent, you can’t stop some damned fool from wanting to commoditize them.

I’ll even state that as a law of sorts, and unfortunately I’m not being entirely facetious. I think that the possible consequences of commoditization are genuinely frightening. History shows there’s no real limit to the complexity of exchange traded instruments, and there are even fewer constraints around over-the-counter deals. In a few years’ time we could see new exotics on the derivatives market, at least partly based on the numbers held up in planning poker sessions at hot-shot companies. Once estimates become commoditized - and some damn fool will eventually try - exactly that sort of nonsense becomes possible. It could make subprime look like the epitome of financial prudence.


What Dean Leffingwell seems to have done with SAFe is to acknowledge that commoditization is part of the toxic, background radiation of enterprise transitioning at scale. This management dysfunction clearly isn’t being challenged out of principle in the way that Ken or Johanna have set about doing. Instead, SAFe treats the commoditization of estimates to be axiomatic, a “given thing” that is germane to the transformation problem. The essential precept, perhaps, is to choose your battles wisely. SAFe’s advice seems to be: accept that commoditization is going to happen, address other issues that are more under your control, and make the best of this bad job.

Where does that leave us then? Well, when it comes to the matter of theory versus practice, I don’t know if Dean Leffingwell is right for his willingness to make such pragmatic compromises, or if Ken Schwaber is right for believing that certain core principles must hold in order to have a scalable agile framework that is worthy of the name.

Still, if I have to make a call…

Yogi Berra is right.

Published at DZone with permission of Ian Mitchell, author and DZone MVB.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)


Ian Mitchell replied on Wed, 2013/09/11 - 5:34am

Dean Leffingwell has provided me with the following comments:

"Hi Ian,
While there is room for more than one opinion on this topic, SAFe, is based in large part, on lean and principles of product development flow. Both emphasize the economic value of work, and ask that all involved understand the economics of work and the entire value stream. In the world of the large agile enterprise, opportunities and challenges abound, all constrained by resources and economics. Estimating work that may or may not be done (based on cost) is a requirement for success. It isn’t placating management. It’s living in the real world, and simultaneously using SAFe to help bringing the benefits of agile development to every team member, who otherwise is stuck in waterfall.
Giving no credibility to the need for estimating is one of the reasons basic agile didn’t make it across the chasm in the first place. Crossing that chasm requires additional maturity of thought around the economics of the business."

Dean therefore positions the commoditization of estimates as part of value stream economics. In so doing, he clearly aligns SAFe with Lean principles. This is interesting for two reasons:

  1. It implies that this "method war" could resolve to an old tension many of us are familiar with. That's the challenge of expressing value and flow within an Agile context that is predicated on emergence and uncertainty. SAFe arguably provides an economic framework for this purpose, and as such estimates have to be commoditized so useful models can be built.
  2. I'm sure that aligning SAFe with Lean would strike many as ironic. I know that the very idea of estimation - even within the narrow timebox of a Sprint - causes many Lean advocates to recoil! It's widely argued that only the "actuals" count in Lean economics.

Personally, I'd feel more comfortable with SAFe if it was positioned as a reflex of scientific management, and not as an expression of either Lean or Agile development sensu stricto. This doesn't make SAFe any less valuable or worthy. On the contrary, it helps accredit it as a transitional step in an agile transformation program.

I know that many organizations can be, and are, left better off by the application of SAFe than they were beforehand. As Johanna Rothman has put it, "SAFe is definitely more effective than waterfall". That is perhaps the damning of it by faint praise, rather than a ringing endorsement. Even so, it gives SAFe a place at the table...below the salt perhaps, but at least we have a diner who can pay his bill.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.