Subscribe via RSS Feed

The Wetware Crisis: the Thermocline of Truth

April 15, 2008 27 Comments

[Updated 09/12/13 — fixed some links and added a few.]

[Copyright 2008 by Bruce F. Webster. All rights reserved. Adapted from Surviving Complexity (forthcoming).]

ThermoclineA thermocline is a distinct temperature barrier between a surface layer of warmer water and the colder, deeper water underneath. It can exist in both lakes and oceans. A thermocline can prevent dissolved oxygen from getting to the lower layer and vital nutrients from getting to the upper layer.

In many large or even medium-sized IT projects, there exists a thermocline of truth, a line drawn across the organizational chart that represents a barrier to accurate information regarding the project’s progress. Those below this level tend to know how well the project is actually going; those above it tend to have a more optimistic (if unrealistic) view.

Several major (and mutually reinforcing) factors tend to create this thermocline. First, the IT software development profession largely lacks — or fails to put into place — automated, objective and repeatable metrics that can measure progress and predict project completion with any reasonable degree of accuracy. Instead, we tend to rely on seat-of-the-pants (or, less politely, out-of-one’s-butt) estimations by IT engineers or managers that a given subsystem or application is “80% done”. This, in turn, leads to the old saw that the first 90% of a software project takes 90% of the time, and the last 10% of a software projects takes the other 90% of the time. I’ll discuss the metrics issue at greater length in another chapter; suffice it to say that the actual state of completion of a major system is often truly unknown until an effort is made to put it into a production environment.

Second, IT engineers by nature tend to be optimists, as reflected in the common acronym SMOP: “simple matter of programming.” Even when an IT engineer doesn’t have a given subsystem completed, he tends to carry with him the notion that he whip everything into shape with a few extra late nights and weekends of effort, even though he may actually face weeks (or more) of work. (NOTE: my use of male pronouns is deliberate; it is almost always male IT engineers who have this unreasonable optimism. Female IT engineers in my experience are generally far more conservative and realistic, almost to a fault, which is why I prefer them. I just wish they weren’t so hard to find.)

Third, managers (including IT managers) like to look good and usually don’t like to give bad news, because their continued promotion depends upon things going well under their management. So even when they have problems to report, they tend to understate the problem, figuring they can somehow shuffle the work among their direct reports so as to get things back on track.

Fourth, upper management tends to reward good news and punish bad news, regardless of the actual truth content. Honesty in reporting problems or lack of progress is seldom rewarded; usually it is discouraged, subtly or at times quite bluntly. Often, said managers believe that true executive behavior comprises brow-beating and threatening lower managers in order to “motivate” them to solve whatever problems they might have.

As the project delivery deadline draws near, the thermocline of truth starts moving up the levels of management because it is becoming harder and harder to deny or hide just where the project stands. Even with that, the thermocline may not reach the top level of management until weeks or even just days before the project is scheduled to ship or go into production. This leads to the classic pattern of having a major schedule slip — or even outright project failure — happen just before the ship/production date.

Sometimes, even then management may not be willing to hear or acknowledge where things really are but instead insist on a “quick fix” to get things done. Or management will order the project to be shipped or put into production, at which point all parties discover (a) that the actual business drivers and requirements never successfully made it down through the thermocline to those building the system, (b) that there are serious (and perhaps fatal) quality issues with the delivered systems, and thus (c) that the delivered project doesn’t do what top management really requires.

[INSERTED – 04/30/08]

Since Jerry Weinberg (see comments) and others have disputed that the thermocline is “distinct”, let me insert two real-world examples that I have personal knowledge of from over a decade ago. Both examples involve Fortune 100 corporations that were undergoing Y2K remediation across the entire enterprise. In the first case, the corporate Y2K coordinator had a weekly meeting with the heads of ~20 divisions and departments within the corporation in which those senior executives would report on their division/department’s Y2K remediation status with a green/yellow/red code. Four weeks before Y2K remediation was scheduled to be completed, virtually all the division/departments were reporting green, with a few yellows. Just one week later — three weeks before remediation was to be completely — almost all the department/division heads suddenly reported their status to be yellow or red. The Y2K coordinator (who told me about the meeting right afterwards) looked around the room and asked, “So, what do you know today that you didn’t know a week ago?” No one had much of an answer.

A year later, I was asked by a major corporation to come in and review their Y2K remediation because almost exactly the same thing had happened: almost all the departments/division had been reporting each week that they were on schedule to complete their Y2K remediation until roughly two weeks before the remediation was supposed to be completed — and then suddenly about 70% of the departments/divisions said they weren’t going to be done on time. The mass shift from “on schedule” to “not on schedule” took place in exactly one week and happened just a few weeks before the deadline. I came in, interviewed some 40 people (under strict confidentiality, in spite of pressure from top management to reveal who said what), and wrote up an honest assessment of where things stood, with a plan for getting things done. The corporation then asked me to come in and implement that plan, so I ended up commuting over 2000 miles/week (back and forth) for 2-3 months to do just that.

I have seen the same pattern repeatedly in IT systems failure lawsuits I have worked on, particularly when I’ve had large numbers of internal e-mails and memos to review. At times, I can identify right where the thermocline is and how it creeps up the management chain as the deadline draws near. In such cases, it usually doesn’t reach the top of the management chain (which, in the case of these lawsuits, means the developer notifying the customer) until shortly (<1 month) before the reported deadline. In fact, this syndrome goes hand-in-hand with the IT system failure lawsuit pattern I call “The Never-Ending Story“.

[06/16/08]: In fact, here’s a real-world IT project review memo, written several years ago, that described a “thermocline of truth” with a very distinct and discrete boundary.

In short, Jerry’s arguments notwithstanding, I’ve seen the thermocline of truth, I’ve seen it be very distinct, and I’ve seen it work its way up the management chain — just as I’ve described. I’m not writing this to be clever or glib; I’m writing it because it really happens.


Successful large-scale IT projects require active efforts to pierce the thermocline, to break it up, and to keep it from reforming. That, in turn, requires the honesty and courage at the lower levels of the project not just to tell the truth as to where things really stand, but to get up on the table and wave your arms until someone pays attention. It also requires the upper reaches of management to reward honesty, particularly when it involves bad news. That may sound obvious, but trust me — in many, many organizations that have IT departments, honesty is neither desired nor rewarded.

I know that first hand. I can think of one project — being developed by one firm (the one that retained me) for another company (the customer) — where I was in on a consulting basis as a chief architect. In the final planning meeting before submitting the bid to the customer, the project manager set forth an incredibly aggressive and unachievable schedule to be given to the customer. I objected forcefully in the meeting — after all, we didn’t even have an architecture yet, much less a design, yet the project manager already had a fixed completion date — and later that afternoon, I wrote up a memo listing thirteen (13) major risks I saw to the project. While some of the engineers on the project cheered the memo, management told me in so many words to shut up and architect.

However, less that two months later, I wrote a new memo — based on the old one — and pointed out that 12 of the 13 risks I had pointed out had actually come to pass. Shortly after that, the project manager had to go back to the customer with a new delivery schedule that was twice as long as the original one. A month or two after that, my role as an architect came to an end. I had a final lunch with the two head honchos in upper management, and to their credit, they asked for my final assessment. I told them that many of the bumps and potholes were just part of the software development process — but that they should never have given that blatantly unrealistic schedule to the customer. As I told them, “When you do something like that, in the end you look either dishonest or incompetent or both. And there’s no upside to that.”

A few months after I left, I got word that the schedule had slipped to three times the original length, and not long after that, I got word that the customer had canceled the project altogether. As I said: just no upside.

[UPDATE: 09/12/13]: Here’s a $1 billion failed USAF project that appears to have largely foundered on the thermocline of truth.

[For a discussion of where the thermocline analogy originally came to me, see this post at And Still I Persist.]

Be Sociable, Share!

About the Author:

Webster is Principal and Founder at at Bruce F. Webster & Associates LLC. He works with organizations to help them with troubled or failed information technology (IT) projects. He has also worked in several dozen legal cases as a consultant and as a testifying expert, both in the United States and Japan. He can be reached at 720.895.1405 or at

Leave a Reply