“Ground-truth documents”

A great post by Eric S. Raymond[*] (yes, that esr) on what he terms ground-truth documents:

Here is an example: AIVDM/AIVDO protocol decoding. It describes the behavior of Marine AIS radios; I wrote it as preparation for coding the GPSD project’s AIS driver. It isn’t exactly or completely a hardware-interface specification, and some of its claims are derived from standards documents and not yet tested – but the point is that it tells you which claims have been tested and which have not. It also tells you where the observed behavior of AIS doesn’t match the standards.

Casting about semi-consciously for a way to distinguish this from a “design document”, I found one. What this is, is a “ground-truth document”.

The thing about ground-truth documents is that they don’t make promises, don’t erect requirements, and don’t talk about the future. They’re just the facts, ma’am. They describe what is, warts and all. Mine evolved into the best single reference on the AIS protocols anywhere, and has since been used as a spec by at least three decoder projects other than GPSD itself.

The practice that goes with this term is simple: always put your ground-truth document together before you start on production code (test tools to reverse-engineer the device are not production code). Maintain it with the code, treat it as the authority for how the code should behave, and when the code doesn’t behave that way treat the divergence as a bug. When your knowledge about how the device behaves changes, change the code second; change the ground-truth document first. (Of course you have it under version control, so you also have a history of your knowledge of the device.)

Go read his whole post and then ask yourself: what ground-truth documents should you have in place (and under change control) in your IT project? ..bruce..

[*] D’oh! Not only did I misspell esr’s last name (I originally wrote “Reynolds”), I did so with “The Cathedral and the Bazaar” sitting in a bookshelf at eye-level 4′ behind my monitor. Senility in the young is a sad thing.

About the Author: bfwebster

Webster is Principal and Founder at at Bruce F. Webster & Associates, as well as an Adjunct Professor for the BYU Computer Science Department. He works with organizations to help them with troubled or failed information technology (IT) projects. He has also worked in several dozen legal cases as a consultant and as a testifying expert, both in the United States and Japan. He can be reached at 303.502.4141 or at bwebster@bfwa.com.