Wed 5 Sep 2007

On Software Design: Practical Experience

It might turn out to be a week of posts related to software development. I have a few half-written pieces and lists of ideas in a folder, whence was born yesterday's initial debugging post and today's design observations.

Software design (and its ties to implementation) is not a new topic and there are few clearly right or wrong answers. I'd like to focus a bit on the internal process; what I go through when I'm tackling a design problem. The dirtily practical over the theoretical and recommended, so the speak.

What Sort Of Design?

Software design comes in a number of flavours. We might be building a new product from a blank slate. Working to a functional specification, making internal design decisions that conform to external requirements. Adding a new feature into an existing product. Rewriting existing code to try to fix a number of small (or large) problems that have crept in, one small piece at a time.

Of these things, fixing existing code is probably most common; starting from an empty whiteboard, by far the least frequent (if you find yourself thinking "unfortunately" there, bear in mind that there is sometimes nothing more terrible than an unanswered request and a blank whiteboard). Still, these are all different scales of the same problem: work out what needs to be done, find a solution, be convinced (and convince others) that the solution is good enough, implement the solution.

So, let's pay no more mind to the scale of the design. In fact, I've just wasted a whole section right here even bringing it up in the first place.

Design Is Debugging In Disguise

You are trying to solve a problem when you are writing code. Whether you go through a conscious design phase or sort of fill in the details in your head as you go (remembering that we aren't talking about only one scale of problem here, so both extremes can happen), you are starting from problem exists and hoping to end up at problem solved. Hey... it's the same as debugging!

I wrote a whole post yesterday about making sure you are debugging the right problem. The same holds true for design. I'm guilty of plenty of bad designs over the years — and not just in the distant past — and, if I care to be totally honest about the causes, one of the two main ones is failing to really have an understanding of what the true problem was. Either through incomplete knowledge or by misidentifying the root issue altogether.

Do regular reality checks of this assumption (that you understand the problem). Knowledge accumulates as you attack a problem. A lunchtime conversation with colleagues might reveal something you didn't know or provide a new connection to some other problem. At the my last corporate job, switching to a new role forced me to be at least somewhat aware of a lot more projects and a lot of code I hadn't looked at in a few years. Some over-arching design missteps became apparent at that point, but without being shoved into that situation, I would never have spotted them (and nobody else was in the position to do so, either). This is why code review and design review works: you get input from a group that has a broader base of knowledge than any individual.

Design Requires Pragmatism

Perfect solutions do exist. They often live in universe called Textbooks and Research Papers. The real world is a little more complicated with imperfect data, unusual requirements, unknown timings, unreliable equipment... the list goes on.

When choosing amongst possible design alternatives, be clear to yourself about what constitutes a good enough solution. We all implicitly understand the difference between a "once off" solution and something that is designed continual production use. But in those latter cases, what are the important features you are trying to improve. What can you trade away to achieve that goal?

Maintainability, extensibility, usability, performance. Those are four things that each work somewhat in opposition to the others. Going for every last microsecond of performance at the code of maintainability or the possibility of easily adding features later might not be the best approach in all cases. If you're part of a large team (implicitly true in Open Source work), what counts as "fast enough"? Do you want to be maybe one of only two or three people who are willing and able to work on this code in the foreseeable future? Of course, sometimes performance is important in this code path or internal messiness is required to make the external API more user-friendly, but know when that is the path you have taken.

Be conscious of which aspects you are trading off against each other and your decision will be informed. Focusing on only one improvement axis and your inattention might come back and bite everybody in the hindquarters.

(Above, I mentioned two main areas where my failing designs fall guilty. This is the second one: a lack of pragmatism about stopping at sufficient, rather than ideal.)

Still To Come

This essay has become long enough for now. In the near future, though, I want to write a bit more about the other side of this coin: looking at designs from other people, reviewing and improving them, along with teaching and managing in this domain. I've screwed up enough to learn some lessons here. Might as well pass them along.

Topics: software/design, software/django