Tue 12 Jun 2007

The Thing About Specs

There's been a bit of a thread going round in one circle of the blog-space lately following a somewhat flawed post from Dare Obasanjo about the Atom Publishing Protocol (APP). Putting a phrase like "...[Microsoft] will likely standardize on a different RESTful protocol" early in a post is bound to get the attention of anybody who is concentrating. It just sounds so much like the typical Microsoft result of not using existing standards and reinventing poorly.

Developments since then have been interesting and it provides an interesting case study in the use and abuse of technical specifications.

The response to this post has been interesting. Reasoned technical responses from Bill de hÓra and Joe Gregorio address things in a balanced fashion. The more I thought about the original post, though, the more I was thinking similar to Tim Bray's response. I'm not particularly inclined to give Microsoft or anybody who's drunk their Kool-aid the benefit of the doubt any longer. There's too much negative evidence on record.

Then the landscape changed. Dare wrote a reasoned response yesterday that explains a lot more about what he intended in the original post. It's worth a read and the follow-ups to this should be interesting. But this brings me to my main point....

Read the spec and think about it!

It transpires that some of Dare's original comments weren't based on a careful reading of the APP specification. There was a lot of jumping to conclusions being done from looking at the GData implementation and not a lot of questioning the results.

Technical specifications are both a curse and a blessing. A well-written spec captures all the requirements and even explicitly identifies some things that fall outside the scope of the spec (e.g. the "security implications" section in a lot of IETF standards). What you miss out on is the reasoning behind the decisions. Why did they choose one method over another? Spoilt for choice, or hidden problems in the alternative? Are extensions encouraged or is this viewed as a limited problem domain specification and things outside that domain should look elsewhere? The sort of things an implementor has to worry about.

For Atom (syndication) and Atom Publishing, the mailing list is a great resource to divine the logic behind the decisions. I've written before about the downside of that particular list and I stand by those comments to this day. However, things do appear to have improved lately and when I revisited the archives for an extended read, there was a lot less obvious acrimony. Possibly because they were much closer to final draft time and all the focus was on editing for quality and a last round or two of bike-shedding.

The problem remains, though: to create an implementation that uses something like Atom Publishing as an interface requires a lot of thinking, research and experimentation. This is clear from reading the responses to Dare's original post (a lot of which are linked in his second post, above). People like Bill and Joe and James Snell all mention in their posts how they have thought and debated how to map APP onto their problems. They seem to have all approached it with the "this should work if we can just work out a couple of items" approach that is necessary to reuse. As for the alternative ("we can't copy an example from the manual; it's not a perfect fit. we'll have to invent our own")... well... let's see how that works out for the group within Microsoft. Surely the difference here is the difference between engineering and being merely a tool user? Isn't there some pride in being able to join a problem with a non-hacky solution? Not to mention the benefits that result. The right attitude makes solving a lot of problems easier.

One day, good practices will emerge from Atom implementations. Some of them might even be slightly cursed as "best practices", stifling innovation somewhat but providing buzzword compliance for other purposes. There will be a legacy of attempts, failures and successes of varying degrees. That day isn't here yet. So it's that period where everybody evaluating APP has to read the specification, think about it for a while, ask questions, read the archives, think some more, write some code. Repeat.

This isn't a new or isolated issue, although without writing pages of text, I'm struggling to write both the general essay and tie it to the example at hand. To pick merely one example from personal experience, I've always (in recent years) found the TCP RFC to be fairly clear. Subsequent extensions of TCP also seem easy to understand. However, I have the benefit of being able to use the experience of the Stevens books, or reading existing implementations and experimenting with TCP networks. Without those, something like the TCP/IP tutorial in RFC 1180 would have been invaluable (it still is a pretty compact overview of how the Internet works on some level). Would TCP have been so clear back in 1981 if I was only used to using the Network Control Program method of connecting to ARPANET? It would have been possible to work it out, but it would take effort -- if I was starting a personal computer company, how strong would the temptation be to invent my own protocol or use some other one instead?

Topics: technology/xml/atom, software/design