Some years ago, I was called in to lead a team of three other people in reviewing a major project at a Fortune 50 corporation. This project, which I’ll call QUBE, was a major end-to-end re-engineering of that firm’s mission-critical systems, intended to replace all the existing legacy systems. The QUBE project was supposed to have taken just two (2) years and cost under $200 million; when I was called in, it had been going on for four (4) years and had reportedly consumed around $500 million at that point. My initial charge was to determine what minimal quality assurance (QA) efforts would be needed to get this system in to production. We spent three months full-time on site, reviewing documents and (where it existed) source code, and conducting extensive interviews with personnel on both the IT and business sides of the organization.
There’s a lot I could (and may yet) write about what that review uncovered, but what I want to touch on here is the following observation, quoted verbatim (but redacted) from the executive summary of our final report (which I wrote):
The root cause [of project difficulties] lies in QUBE’s organization, which is divided up into numerous vertical and horizontal silos within and among the major streams (…) and cross-stream efforts (…), leading to a complex balkanization of testing efforts. The resulting barriers to communication, coordination and cooperation will almost certainly undermine any effort to impose an improved CORE test strategy (including the recommendations in this report) unless an appropriate test/QA structure is laid on top of QUBE.
While the focus of my review was on QA, my concurrent finding was that the architecture of QUBE was likewise balkanized. In fact, the image that came to my mind again and again was that of the old Windows 3-D pipes screensaver, shown above. There was no conceptual unity, no natural divisions of functionality — it was a lot of intersecting pipes and silos, but even the silos were often divided up among different groups.
Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization’s communication structure.
Put more simply, a system’s architecture is a reflection of the organization that built it. Brooks, after citing (and naming) Conway’s Law, goes on to explain:
It is a consequence of the fact that two software modules A and B cannot interface correctly with each other unless the designer and implementer of A communicates with the designer and implementer of B. Thus the interface structure of a software system necessarily will show a congruence with the social structure of the organization that produced it.
The byzantine architecture of QUBE was a direct reflection of how responsibilities had been divided up (or seized upon, for political reasons) by various individuals, groups, departments, and divisions within the organization.
The logical inference from Conway’s Law is that once your have determined the architecture of the system to be development, you need to organize the development team to reflect that architecture. And since (as Conway notes) the first architecture is seldom the best, the development organization may need to change as the architecture is refactored and modified. On the other hand, if the overall project gets chopped up and parceled out based on politics, turf wars, and external bidding, the system you build — if you can get it working — will reflect that.
Food for thought.
About the Author: bfwebsterWebster is Principal and Founder at at Bruce F. Webster & Associates, as well as an Adjunct Professor for the BYU Computer Science Department. He works with organizations to help them with troubled or failed information technology (IT) projects. He has also worked in several dozen legal cases as a consultant and as a testifying expert, both in the United States and Japan. He can be reached at 303.502.4141 or at firstname.lastname@example.org.
Sites That Link to this Post
- Why US government IT fails so hard, so often « WORDVIRUS | October 10, 2013
- Why US government IT fails so hard, so often | Gadgets and Technology Blog | Gadgetship.com | All About Gadgets | Softwares | Mobiles | Tablets | Smartphones | October 12, 2013
- Obamacare and the Long Bomb : And Still I Persist… | October 26, 2013