Share →
Andrew Clear (Mug Shot)

This is the first of a probably series of a generally unedited “stream of consciousness” posts I plan on writing while reading Donald Reinertsen’s The Principles of Product Development Flow: Second Generation Lean Product Development. I haven’t been this affected by a book in a while, and I wanted to share my excitement with the world and test some of the ideas it is provoking. As such, this is intended to be an invitation to discussion, and I hope you will add your comments and disagreements below. If you wish to engage me in a detailed conversation, please feel free to contact me.

The Asymmetric Payoff Amount

In the section on economics there is something that struck me as so profoundly true I had to stop for a while. Okay, it isn’t the first thing that has hit me like that in this book, but it was the first that triggered an extended series of thoughts. He is discussing the asymmetry between the investment cost and the payoff amount for development projects. He talks about a conference that he was at recently where a speaker pointed out the 96% failure rate in new product development and that poor management is to blame. We tend to spout the same sort of statistics and give the same reasons. Here’s what Reinertsen has to say:

“In fact, economics would suggest that the 96 percent failure rate tells us more about the asymmetry of payoffs from product development than it does about our management competence.”

This struck me as profoundly and unequivocally true. If our failure rate is so abysmal, we would have ceased investment in software projects ages and ages ago. Why would we have continued to create IT departments at a prodigious rate?

What happens in 10 years? In 20 years? I immediately began to think about early manufacturing and such. In the early times, inefficiencies were ok. It took 50 years before it was really necessary to be efficient. There is a parallel there.  The returns on software projects in the past could support incredibly high failure rates because the return on the few successes was so amazingly high.  The economics could be ignored because it was all going to come out in the wash. Companies that are successful right now with crap processes are okay with high failure rates because of the asymmetric returns.

This can’t continue. Just like the western model of car manufacturing was crushed by Toyota after the market became saturated with cars, our market is becoming saturated with software. No longer are new software projects replacing some archaic pen and paper version of functionality.  As that happens, the asymmetric returns are going to rapidly diminish. No longer will your 1 success return 1000x on your investment, masking your 4 previous failures.

I think this means a few things become super important in the new software development landscape. First, extraneous features need to die a quick death. Modern video games have realized this, releasing a core product then only adding new functionality when enough people scream for it. Case and point, Diablo 3 was released without any player versus player capabilities (a core component of the genre) and still sold record copies. If the game had tanked (which was a distinct possibility with expectations so high) Blizzard could have walked away and saved the cash.

Second, decisions can no longer be made in an economic sink-hole. Software development got away with this for a long time, but margins will(are? No empirical data at hand but I think I can safely assume this to be the case) shrink, and shrink rapidly. Think convincing people to buy Microsoft Office 2013 when they currently use Microsoft Office 2003. Now compare that with the jump from DOS to Windows 3.5.

Print Friendly
  • Looking forward to part 2!

  • I’m not sure the reduction in ROI has reached a point where we need to consider it. When talking with management teams, I often ask: “Why are you trying to restrain cost overruns or additional costs for software development at all (on the margins)? If you’re ROI is threatened by a 10 – 15 percent cost overrun, you have invested in the wrong project. ROI for dev projects should be measured in thousands of percentage points. If not, use out of the box software instead.”

    If you’re right,I may soon not be able to make that argument. As the barrier to entry to software development has dropped, you may be right that the potential returns have also fallen. It makes sense. I’d like more data!

  • Another set of thoughts around this:

    1) I think you’re right that the 94% failure rate has been fine in the past, because of the very, very high returns to software, AND

    2) I think your gut is right that the rate of return for corporate, both internal and customer facing software has declined (but I believe it is still very high), BUT

    3) I assert (without data) that the rate of return for customer facing code by startups with only a software business model (in other words, NOT Nike writing software to help sell shoes) has gone up dramatically, potentially by orders of magnitude.

    Basically, scale means that a good photo filter application deployed on the iPhone can be sold for a billion dollars after only a year in operation. The potential returns are nearly unimaginable for a popular application.

    4) I also assert (again without data) that, although there are far more software (web, mostly) startups now, we have a much higher failure rate (as an industry), but that the potential returns mean that even if 99.9% of software ideas fail the .1% that succeed pay for the failures many times over. And the .001% that blow the top off dramatically incentivize people to take the chance on success.

    Here’s the disturbing implications. If I’m right, then the very high ROI in the consumer, software-only model, may be displacing effort in the industry-supporting software development efforts. In this case, we have industry software competing for the scarce dev resources needed to deliver ever lower ROI, while the consumer side is offering ever higher (if more elusive) ROI.

    So, Cheryl, myself, you and Microsoft may have the WRONG message. At Karthik’s prompting we’ve understood that Modern Apps require Modern ALM. And that’s right. Yet, t

    A common pitch to enterprises building software for internal use or to sell their other products, is something along the lines of the following:

    “Look at these massive, global, megatrends in software development (all in the consumer space.) You need to make sure your “systems of record (Line of Business)” apps and your “systems of engagement (Consumer outreach or internal customer)” apps need to change to be more like the commercial systems.”

    But maybe we’ve got the causal relationship wrong. The REASON the consumer apps behave the way they do is that they have EXTREMELY high rates of return and extremely high rates of failure. But we’re blind to all the failures since the overwhelming net effect is a massive positive ROI. (Plus that’s what gets reported in the news.)

    For internal projects, we don’t have the same extreme ROI or extreme failure rates, so why should we try the same techniques used by the consumer firms. We’re asking them for a “monkey see, monkey do”, but we’re ignoring the underlying causes of the ALM moves in the consumer world.

    Now, I happen to think the consumer world has the right idea in the vast majority of cases, and that enterprises would do well to bring those ideas internally. However, are we simply using cargo cult science ( to persuade people to adopt these practices in a realm which has completely different economics?

    If so, enterprises should be skeptical. There may be very, very good reasons for them to adopt these new methods, but we’ll have to prove them out with an entirely different economic model, and from first principles.

  • Andrew Clear

    Quick reply, more to come. Cheryl and I just had a great conversation around the LOB applications. One thing we noted was that LOB apps tend to have almost no competition. This implies that terrible economic decisions in this sphere are easily masked. In fact, the whole “IT is a cost center” idea may be directly related to this fact. Trim the ROI margins, remove the competition, and never apply the right economics and you have a recipe for continued poor performance.