Second-class software quality in major IT projects

Many years ago, software-engineering guru Gerald Weinberg famously said that if builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization.

I’d like to say that efforts at software quality have improved since he first said that, and in many organizations they have. On the other hand, within the past 15+ years I have reviewed far too many troubled or failed software projects in which software quality was treated more as an afterthought than as an essential aspect of IT project management. You would think, given the dollar amounts at stake in these projects – ranging anywhere from a few million dollars to over a billion dollars – that every effort would be made to ensure the quality of the software. You would be wrong.

My own observation, after nearly 40 years in IT, is that far too often upper management thinks of software quality assurance (SQA or QA) as that annoying (and hopefully brief) phase of testing between the end of coding and actual deployment of the application. As a result, the QA group — assuming there is one — is usually given too little time and money to carry out their tasks.

Beyond that, upper management and, quite frankly, IT management tend to define QA as being simply “testing”. As such, they often view QA as just bringing in some warm bodies to run some test scripts once the coding is all finished. There is actually far more to QA than just testing, and there is far more to testing than just running test scripts.

Back in 1975, Fred Brooks stated that a major software project should plan on spending 1/3 of its time in analysis, 1/6 of its time in coding, and ½ of its time in testing – percentages that would give most managers a heart attack. But Brooks notes that whether or not you plan to spend ½ of our time in testing, in the end you will anyway. I’ve seen that borne out in a number of major IT projects — projects that went months or years late because the organization was unwilling to plan and pay for proper QA up front.

If the organizational challenges to QA were simply lack of time and money, then they could be readily solved by simply allocating more time and money. However, there is a second, more intractable issue. Put simply, an IT engineer who works in QA has less professional status than one who works in actual coding. The old IT joke is that you conjugate the verb “to program” thusly: “I architect; you code; he or she tests.”

In many, many organizations, the unspoken or explicit technical hierarchy (with commensurate prestige, influence, and salary) is something like this:

• Enterprise architect
• Chief software architect
• Some other kind of architect
• Senior developer
• Developer
• Tester (I prefer the term “QA engineer”)

And in most organizations, “testing” really is at the bottom of the totem pole. Often, a senior, highly experienced QA engineer will have less credibility and status within the IT organization than an entry-level, still-wet-behind-the-ears programmer fresh out of college — which is precisely why so many IT projects are late or never ship at all.

There are, I believe, several reasons for this. First and foremost is that QA, as stated above, is often defined simply as testing, and many organizations hire non-technical personnel to do software testing. They figure it doesn’t take a college degree – at least, not one in computer science – to sit in front of a computer and step through a test script.

On the other hand, you can train a lay person to conduct medical surgery, but I suspect few of us would want such a person to operate on us. An IT engineer doing testing is far more likely to know what’s going wrong and will usually be more likely to be able to recreate the defect (a key point in testing).

Second, just doing “testing” — especially if it’s just running a test script — is ultimately boring for most software engineers, particularly the talented ones. They want to be creating or implementing programs, not figuring out where someone else goofed up. Again, the problem is the rather limited view of testing — running user-oriented scripts against a written requirement — taken by many organizations. Many other types of testing exist, all of which can be valuable and quite a few of which are vital. Most of these other tests do require software engineering skills and real expertise.

Third, many organizations have no career path in quality engineering. Instead, they have a pool of “testers” and a head of QA. At best, they may have some mid-level QA managers. But there really is little opportunity for professional advancement — except to move over into software development.

Fourth, as noted above, QA tends to be an afterthought in terms of budget and other resources. It is often the first area within IT to have its funding cut and the last to have its funding increased.

As a result of all this, a job in QA is usually seen as being an insecure, dead-end position. And the result – as Weinberg noted – is software that’s anything but reliable.

[Adapted from an article originally written for the online version of Baseline.]

About the Author: bfwebster

Webster is Principal and Founder at at Bruce F. Webster & Associates, as well as an Adjunct Professor for the BYU Computer Science Department. He works with organizations to help them with troubled or failed information technology (IT) projects. He has also worked in several dozen legal cases as a consultant and as a testifying expert, both in the United States and Japan. He can be reached at 303.502.4141 or at bwebster@bfwa.com.