A few years later — in 1982 — my close friend Wayne Holder hired me into his nascent software company, Oasis Systems, in part to help with his existing and planned word processing utilities (The Word Plus, Punctuation + Style), but mostly to develop computer games. And we did, developing Sundog: Frozen Legacy on the Apple II, a game for which I still get e-mails (and which Wayne is even now working on resurrecting for modern platforms). In January 1984, a few months before Sundog shipped, we were invited by Guy Kawasaki to come up to Apple to see a preview of the Mac and to talk about what software we could port to the Mac. Through my connections with computer stores in San Diego, I was able to get a personal loan of a Mac for a few days at home prior to the official announcement in Cupertino later that month, which Wayne and I attended as well. That was my first time seeing Steve Jobs in person, and it remains a memorable highlight of my professional life.

When the Mac shipped a few days later, I went down to the one computer store in San Diego that I knew would be getting machines from Apple. I took $3000 in cash with me and managed to convince the store owner — a friend — to let me have one of the three Macs he had to sell. Through a connection with Phil Lemmons — editor-in-chief at BYTE — I ended up writing the official BYTE review of the 128K Macintosh (August 1984 issue). By the end of 1984, I was writing full-time for BYTE, including on-going coverage of the Macintosh, particularly once my BYTE column started in mid-1985. After a few years of writing for BYTE, I switched to writing for Macworld magazine. Steve was now long-gone from Apple, and Apple was having some of its own problems going forward.

But in late 1987, I was contacted by Addison-Wesley. They were interested in having me write a book about Steve Jobs’ new project at NeXT. Folks at NeXT had apparently suggested me to Addison-Wesley, probably due to my writing at BYTE and Macworld. I leapt at the opportunity, particularly since in coincided with our family moving from Utah to just outside Santa Cruz (where I would be doing technical writing for Borland on a consulting basis). Once there, I found myself invited to visit NeXT HQ on Deer Creek Road, sit in on meetings, and attend the 0.3 NeXTstep Dev Camp. And, yes, that meant getting actual face time with Steve Jobs as well — not a lot, but this was a man whose creations had been impacting my personal and professional life for over a decade at this point.

The writing of the book dragged out as I waited to get my hands on an actual NeXT cube, which finally happened (if I recall correctly) at the end of 1988 or early 1989. I wrote the first several drafts of the book on that NeXT cube itself. The book came out in the fall of 1989; it remains the single most successful book I’ve ever written, due to the intense interest in NeXT itself, more than any particular writing skills or technical insight on my part.

The following year, I found myself working with a world-class typographer (Mike Parker) and graphic designer (Vic Spindler) to create a design-oriented desktop publishing system. I was doing all the software prototyping on my NeXT cube, and we made the decision to make the NeXT our first target platform. For five years — 1990 to 1995 — I served as chief architect and CTO at Pages Software Inc, where we developed Pages by Pages and then WebPages, while spending nearly two years just trying to raise venture funding. We closed on funding at the start of 1992 and shipped our first version of Pages in early 1994. We quickly sold all that we were going to in the all-too-small NeXTstep market. My frustrations at seeing larger firm try to leverage off of NeXT’s incredible innovations led to an op-ed piece in the November 1994 issue of BYTE, “Whither NextStep?” The day that issue came out was the last time that Steve Jobs and I spoke — he called me from the back of a car somewhere to ask me what the hell I was doing writing that. I said, telling the truth. Pages would close its door the next year, unable to secure additional funding to move its technology to Windows.

When Steve engineered his brilliant reverse takeover of Apple — getting Apple to buy NeXT for $400 million, then slowly moving himself into the CEO seat — I was not optimistic. I still had unconditional praise for the NextStep technology, but I was dubious about Steve’s ability to sell technology to markets and to compete with Microsoft.

Boy, was I wrong. I was not only wrong about his abilities at Apple, I was wrong in my BYTE article about NextStep being on a downward slope. NextStep, of course, was the foundation of Mac OS X, and Steve transformed Apple into the most-admired, most-imitated, and most-valuable company in the world. And I was tickled that, when Apple brought out its own word processor, it was named “Pages”. Steve had always liked that name when we were developing (and shipping) our own product years before; glad he was able to use it.

To quote John Perry Barlow over on FB, “The world is suddenly a less interesting place.” ..bruce w..

[1] The first was an HP-67 card-reading programmable calculator.

[Cross-posted from And Still I Persist]

Now, only two months after the iPad actually started shipping, AT&T is ending that plan as of June 7th, and I find it very hard to believe that Apple didn’t know this would happen, possibly before the iPad started shipping.

I ordered an iPad from the Apple Store back on May 22nd; it still has not shipped, which means that I am unlikely to get it before June 7th, and therefore will not even be able to ‘grandfather’ in under the $30/unlimited plan. There must be at least a few hundred thousand (if not more) people in the same position, all of whom ordered (and paid money) with the expectation of the availability of the $30/month unlimited data plan, but who will find that this data plan is no longer available when their iPads finally arrive. I do consider this classic bait-and-switch (and I’m not the only one);  I would not be surprised to see a ‘race to the courthouse’ from numerous law firms in an effort to establish class action lawsuits against Apple and AT&T.

I think that Apple is underestimating the anger, bad will and litigation that this move is likely to generate; I know I’m pretty appalled.  ..bruce..

And so, from hour to hour, we ripe and ripe,
And then, from hour to hour, we rot and rot;
And thereby hangs a tale.

– William Shakespeare, As You Like It, Act II, Scene vii.

I have observed a pattern (or anti-pattern) in IT engineering that looks something like this:

Let’s call this the arc of engineering, since that’s more compact and elegant than the strangely shaped curve with what appears to be a single inflection point of engineering. “Arc” in any case conveys the essential sense: that system quality (however you wish to define that) rises over time to a peak value and then starts to decline.

This came to mind because I recently put a new hard drive in my laptop and have been going through the tedious drill of reinstalling all the various applications and utilities that I am used to using. With some software, I want — or have to use — the latest versions. So, for example, my versions of Firefox, Thunderbird, Quicken, and so on are all current, not to mention my anti-virus/firewall software.

For others, however, I either don’t need — or specifically don’t want — the newest versions. The laptop’s restore disks installed Windows XP, for which I’m grateful; I have no plans whatsoever to upgrade to Vista. Similarly, I installed MS Office 2003; I don’t own and don’t plan to buy Office 2007. As far as I can tell, Microsoft pretty much hit the top of its engineering arc for both Windows and Office a few years ago. In fact, the early to mid-2000s may well represent the peak of quality, features, and user acceptability for these product lines. Given the on-going end-user unhappiness with Vista and Office 2007 — and the corporate resistance to upgrades to either — Microsoft’s user base may never again be as happy with these product lines as they were a few years ago.

This is not an anti-Microsoft screed, because I have observed a similar effect in other companies and product lines. For example, I was a very early Macintosh adopter and champion, though I also was quite critical of its initial limitations. Apple continued to make improvements and changes to the Macintosh product line and finally reached what I considered at the time to be a close-to-perfect Mac: the Macintosh IIcx. It was solid, reliable, and very stable. Apple followed up with the Macintosh IIci, which used the same form factor, but offered several improvements over the IIcx, though there were some software stability problems due to the rewritten ROM and related hardware changes.

And then came the Macintosh IIfx, which was supposed to be even better — but the IIfx soon became well known for its hardware problems, in particular its tendency to blow out power supplies. Other Mac II models — the IIvi, IIsi, and IIvx — each had their own problems and limitations. From there, Apple descended into the product-line chaos known as Quadra/Centris/Performa, a marketing disaster from which Apple would not recover completely until Steve Jobs came back to Apple and introduced the iMac.

In this same time period, Apple had to deal with the fact that its operating system — System 7, introduced in 1991 — was under-featured, underwhelming, and built on aging technology. System 7 would lose its graphical UI edge with the introduction in 1992 of MS Windows 3.1 and would pretty much lose the operating systems wars completely a few years later with the introduction of MS Windows 95. It would not be until Apple’s introduction in 2002 of the Jaguar (10.2) version of Mac OS X — a heavily updated and retooled version of the NeXTstep OS — that Apple would regain technical and UI leadership in operating systems.

This same pattern can appear in internal IT projects. The system under development goes into production and continues to improve over time. Then at some point, the system’s overall quality (functionality, performance, reliability) starts to decline and maintenance costs begin to go up. At some point the organization decides to replace or re-engineer the system, sometimes successfully, sometimes not.

So, why does this happen? I believe several factors are or can be involved. (To simply the discussion, I’ll use “system” to mean the software, IT system, product, or product line under discussion; likewise, I’ll use “developer” to mean the company and/or IT development group responsible for creating, enhancing, and maintaining the system.)

The developer loses conceptual control of the system. In other words, no person or group of people understands the overall architecture, design, and operating constraints of the system. Thus the developer can no longer ensure that on-going changes to the system are consistent and compatible with existing aspects of the system or with other changes being made. This in turn can come about for several reasons:

• The system grows too large and complex. Sheer code size is not necessarily a problem (though it doesn’t help); on the other hand, the degree of fragmentation and interdependencies can be. (See this Vista post-mortem for some examples.)
• Those who really understand the system have left or moved on to other projects. The best and brightest IT engineers — particularly architects and designers — usually prefer to work on new systems. That means once this system has shipped (or gone into production), the people who created the system in the first place and who understand it best may well want to move on to other things, inside or outside of the company. Those who replace them — and those who stay behind — may be quite talented but may not understand all the system’s nuances and requirements. Eventually, they may decide to move on as well. With each generation of turnover, more institutional memory is lost.
• The development organization no long meshes with the system. Conway’s law states that “any piece of software reflects the organizational structure that produced it.” Since the IT group responsible for the system often changes once the system ships or goes into production, the organization of the new group responsible for the system may not match the system’s structure, causing gaps in knowledge and responsibility, plus a tendency — conscious or not — to reshape the system to match the new organization.

Software rot sets in. Software rot is a real term of art within software engineering, referring to the well-known tendency of systems in production to become less reliable over time for a variety of reasons having mostly to do with piecemeal and uncoordinated changes to the system, or growing incompatibilities with external systems.

The enhanced system finally outgrows its original foundation. When the developer seeks to expand or improve the system, it often does so without properly rearchitecting and redesigning the system first. As a result, the original system eventually may be stressed beyond its original intent and begins to suffer from functionality, reliability and/or performance problems. Mike Parker, one of the co-founders of Pages Software Inc. (read the 3rd paragraph), once described this syndrome as “trying to build a castle on the foundation of an outhouse.”

Market or business needs shift beyond the product’s fundamental design. A developer creates and enhances a system to meet a specific set of business/market requirements. However, there often comes a time when new or expanded requirements exceed the capabilities of the system’s original architecture and design. This can lead to the “outgrowing the foundation” situation described above, with the resulting problems. Or it can lead to a true re-engineering or replacement effort, which may well lead to the final two problems below.

The developer begins to add “blue sky”/”kitchen sink” enhancements. Once the “door is open” on changes to the system — or features for the new/re-engineered system — the requests pour in from all directions. Without rigorous control over these enhancements, the development effort can quickly bloat in every aspect: architecture, design, code size, schedule, staff, and defects.

Backward compatibility is maintained at all costs. Supporting backwards compatibility in a system can be a bit like trying to drive a car while pedaling a wheel at the same time. Apple abandoned its old arc of engineering and created a new one under Mac OS X by first forcing binary backwards compatibility into a separate mode (the “Classic” environment) and then dropping the Classic environment altogether for Intel-based Macs running Mac OS X. Microsoft — which has struggled with the backwards compatibility issue for almost its entire OS life — has announced that for “Windows 7″, it will in effect follow Apple’s lead by not supporting binary (and to a certain extent source) backwards compatibility in Windows 7 but instead using virtualization technology (switching to a backwards-compatible OS) for older application.

I’m sure there are other reasons I haven’t listed here, but it does show why the arc of engineering exists. Comments? ..bruce..

Two disappointed believers,
Two people playing the game.
Negotiations and love songs
Are often mistaken for one and the same.

– “Train in the Distance”, Paul Simon

I used to have arguments with Carol Teasley, one of my mentors, regarding software development methodologies. She contended that there were no really good or effective methodologies, because if there were, everyone would use them and they would work. I, in turn, contended that there were several effective methodologies, but that they were just badly applied for the most part.

Then while watching the movie “A Beautiful Mind” (about Nobel Laureate John Forbes Nash, a brilliant mathematician who struggled with schizophrenia), I was struck by the sequence that represented (not entirely accurately) some of his insights about game theory and multi-player equilibrium points. It occurred to me that software development within a typical corporate/government organization is really an instance of multi-player game theory, with several general classes of players. The real issue in organizational software development is not the development methodology being used. That methodology, at best, is necessary but not sufficient for a successful project; it may actually be irrelevant, and at worst it may be a hindrance.

The real issue is the game that’s being played.

Organizational software development, I believe, can be thought of as an n-player game, where each set of players has voiced and unvoiced (and possibly even unconscious) goals, many of which are incompatible with those of one or more other players.

At the most abstract level, the fundamental division is among IT developers (Geeks), upper management (Suits), and end-users (Users). Put very roughly, the Geeks work as if they’re playing Halo, the Suits work as if they’re playing poker, and the Users are just trying to balance their checkbooks. The result is that the three groups mostly baffle and frustrate one another; they can’t even agree on what game they’re playing, much less what constitutes winning.

Adding to the problem is that each group has generally negative perceptions of the other two, which in turn shape the games. For example, Geeks tend see Suits as “marketing weasels” who make impossible promises (to Users) or demands (of the Geeks) that the Geeks can’t possibly meet. Geeks look upon Users with a mixture of condescension and dismissal, though that elevates to horror and dismay if the Geeks have to directly provide tech support to the Users.

Suits tend to see Geeks as overgrown, naive, but possibly bright children who never actually deliver anything. They tend to see Users as annoying sheep, except when the Users control the purse-strings, at which point the most far-fetched User demand is seen (and portrayed by the Suit) as eminently possible.

The Users tend to see Geeks as irascible wizards who may or may not deliver the magic necessary to make their own lives easier. They tend to see Suits as corrupt nobles who hold not just power but also the wizards’ collective leashes.

Now, these are of course blatant and gross oversimplifications, but ones with a foundation in reality; were it not so, the Dilbert comic strip would have died a quiet death long since.

My next set of posts will look a bit closer at these abstract classes (Geek, Suit, User) and then will look at some of the real-world patterns (internal development, external development, and so on).