Controlling IT Costs: Using a Maintenance Architect
Software rots over time.
Of course, it doesn’t literally decompose, but it often becomes fragile, harder to support and more likely to break when something else in the enterprise’s IT environment changes—another application, the hosting platform and operating system, a third-party product with which it communicates, a database schema.
When a defect is fixed, or new functionality is added, it is usually done with the least possible analysis and effort and without much consideration for the secondary implications of the changes. That, in turn, makes the next defect or request for functionality even more difficult to handle, and so on.
Compounding the problem is the unpleasant truth that software engineers typically view maintenance programming as an unpleasant and unrewarding task. As such, the work is often left to two groups. The first comprises entry-level programmers, who have to “earn” their way out of maintenance and onto new projects. The second comprises less-than-stellar IT engineers, who have — consciously or not — become the resident experts on one or a few legacy systems. By so doing, they have worked themselves into a niche that their peers don’t want, but that still provides value to the enterprise. In some cases, members of the first group become members of the second group without ever leaving software maintenance tasks. The general effect is often a slow but steady decline in the overall quality of the software being maintained.
This effect is compounded by the ever-growing number of software systems being maintained, as well as the new ones being created or installed. The inter-dependencies among these systems, direct or indirect, can be myriad; a fix or change in one system (or its hosting environment) can have a ripple effect throughout several other systems. Fixing those can in turn generate new ripples, and so on. In those cases, the resulting patches are often even more of a kludge than the original fix. In a phrase, the software rots.
Collectively, these are the reasons why so many large organizations spend a significant portion of their IT budget — usually well over half and in some cases approaching 80-90% — on IT maintenance costs. But there is, I believe, a solution: appoint a maintenance architect.
Fred Brooks set forth the compelling need for a chief software architect more than 30 years ago, arguing for the essential quality of conceptual integrity within a given software system. The role I’m proposing is more of a conceptual clearinghouse for all systems currently in production.
The maintenance architect would have the responsibility of learning — and documenting, if necessary — the essential architecture of all production systems, with a special focus on their inter-dependencies and interfaces. He or she would have to review all proposed changes to those systems or their operating environments (hardware, operating system and so on), including retirement or replacement of a given system. This architect would then issue an “environmental impact statement,” listing all the significant potential or actual repercussions and costs of the proposed bug fix or feature addition. This impact statement does not need to be lengthy or complicated, though it may well be. It would be circulated to the engineers responsible for the affected systems, as well the business stakeholders for those systems.
This is clearly not a simple or a “make-work” position. It requires — or would cause the individual to develop — a significant technical breadth and depth. It would also have a real impact on the enterprise’s IT budget by allowing a better approximation of the true cost for a given change in a given system. Equally important, it would help to avoid the cascading chain of errors that often occurs when a fix or an improvement is hastily (and often silently) slipped into production.
This role would be a great training position for chief software architects. Someone who has spent a year or two in this position would have a tremendous understanding of the existing enterprise technical architecture. He or she would also have a great appreciation for the interconnectedness of systems and the often-large and sometimes-intractable consequences of seemingly small decisions regarding software architecture, design and implementation. Most important, the maintenance architect would keep all these production systems in mind when developing a new program, lowering the barriers to integration and deployment.
Almost as important, since this job would be seen as the fast track to becoming a software architect — the most prestigious job in software engineering — you’d have your best and brightest IT engineers competing to fill that slot. If your enterprise is large enough, you might even want to make this a team of two or three maintenance architects. That would make it easier to deal with the vast IT production infrastructure; it would train more software architects; and — if you stagger promotions in and out of the group — it would maintain continuity and ease knowledge transfer.
So, if your organization is spending more than 50% of its IT budget on maintenance, give serious consideration to setting up this position and staffing it with an outstanding IT engineer — the position will likely pay for itself within the first 12 months. And if your organization is spending less than 50 percent of its IT budget on maintenance, look around carefully — there’s a good chance that you’ll find someone who is, in effect, filling this role already.
[Adapted from an article I originally wrote for the online version of Baseline.]
About the Author: bfwebster
Webster is Principal and Founder at at Bruce F. Webster & Associates, as well as an Adjunct Professor for the BYU Computer Science Department. He works with organizations to help them with troubled or failed information technology (IT) projects. He has also worked in several dozen legal cases as a consultant and as a testifying expert, both in the United States and Japan. He can be reached at 303.502.4141 or at bwebster@bfwa.com.