niksilver.com

Sometimes you can trust too much. Wherever I’ve worked I’ve been involved in a few examples where we listened to the customer, trusted them to know what they wanted, given it to them, and they’ve regretted it. We have delivered anti-features.

Most recently at Guardian Unlimited our (previous) homepage had clever layout rules whose logic sometimes overrode the content that editors entered. The result was that editors were often confused that the page didn’t render according to what they had put in the system. The clever layout rules had been devised by close collaboration with the original editors and graphic designers — the expert users. They reasoned that they didn’t want anyone to unbalance the layout by entering inappropriate combinations of text and images.

But these layout rules had been forgotten over time, hence the confusion years later. Consequently the tech team was often called up regarding a supposed bug (”I’ve put this image in but it doesn’t appear”), and effort was expended only to discover that it was in fact a feature — much to the caller’s amazement. Our micro-lesson there was to give the end users the freedom they naturally expected, including the freedom to decide for themselves what was a balanced layout. If what they produced was unbalanced then the designers would steer them back in the right direction — a much more human corrective.

Those clever layout rules were an anti-feature. They were additional functionality that actually made users’ lives worse. Eventually we removed them.

Anti-features happen in highly experienced mega-corporations, too. In November 2006 Joel O’Software started what became known as “the Windows shutdown crapfest”. He compared the three shutdown options of Apple’s OS X with the astonishing nine or more shutdown options of Microsoft’s Windows Vista. Not only is that confusing for the user, but it was also incredibly painful to develop — Moishe Letvinn, a former Microsoft engineer, tells the sorry story.

But at least the Vista shutdown anti-feature made the problem visible. In the layout logic example, and others I know of, there is silent intelligence at work that leads to confusion and frustration without giving the user any visibility at all of what’s going on.

Anti-features are time-consuming to build in, because any features are time-consuming to build in. But anti-features also consume additional time in their removal.

One way to prevent anti-features is to help the end user determine the long-term consequences of what they want. Of course, you’d hope they’d think of that themselves, but you can’t avoid your responsibilities to the project as a whole if you see something that others have missed.

Another way is to adhere even more fervently to the Agile mantras of delivering early (before every last sub-feature is there), keeping it simple, and focusing ruthlessly on only the highest value work. This way we deliver first a front page without clever corrective layout logic, or one or two shutdown options only, and consider further enhancements later if we find we need them. Suggesting and doing this is easier said than done, of course, but if everyone trusts each other to listen honestly to what they have to say then it’s more likely the decisions made will be the best ones.

Meanwhile we can at least ask ourselves each time: Is this a feature or an anti-feature?

An aphorism I heard recently seems to be particularly memorable: “there’s nothing so permanent as temporary”. However, it wasn’t originally referring to software — it comes from a builder who is rebuilding the kitchen of friends. He’s from Azerbaijan, and my friends are fond of quoting him in full, to give the words maximum colour: “As we say in Azerbaijan, there’s nothing so permanent as temporary”.

It is, however, as relevant to software as it is to building (and probably to many other areas of life). The constant pressure to deliver means there is very rarely the opportunity to go back and improve a nasty historical workaround which is causing problems today. However, there are some strategies we might consider…

1. Automated testing

If the nasty programmatic stickytape is relatively small then a comprehensive suite of automatic tests should enable you to make the change relatively safely and painlessly. Of course, you need to have built up that suite of tests in the first place. And the “relatively small” caveat is important, because only then can the software people fit the replacement activty into their daily work without disrupting any schedules. If there’s no external change then this is an example of refactoring.

If the nasty workaround isn’t so small, but really does need replacing, then a much more concerted effort is needed. A plan needs to be carefully devised which allows slow piecemeal replacement. The point of the plan should be to take baby steps towards the end goal; each step should also leave the system in a stable state, because you never know when you might have to delay implementing any subsequent step. However, if everyone in the team knows the plan then the end goal can be achieved with minimum disruption to the business’s schedules.

2. Act like a trusted professional

I find trust and transparency is an increasing part of the software projects I’m involved with. Part of this is that the technical people want to give their customers options, and are keen to explain the pros and cons, so an informed decision can be made.

However, while this is usually excellent, there can be times when it is too much or undesirable. Stakeholders don’t necessarily want to be given options for every single thing, and sometimes it’s right for a technologist to make an executive decision without referring back — because they’re a professional, and because they are trusted (and employed) to be professional. In this regard it’s sometimes the responsibility of a technologist not even to entertain the possibility of a temporary workaround knowing it will become permanent.

The decisions I’ve been close to in this regard tend to be architecture or technology decisions. For example, in my current work we have, roughly speaking, legacy technologies and modern technologies. Sometimes a feature requirement would arise which is quicker to implement in the legacy technology — but of course we knew it would be more costly in the long run. If the difference in timescales was not wildly divergent then I’d encourage my team to choose the modern technology and not to offer up a choice. These days I need to do that much less, because by building up that base of modern technology it’s now easier to expand and extend that for future features. By making hard decisions in our professional capacity earlier on we’ve made it easier to make the right decisions in the future.

3. Bite the bullet

A third way to remove the temporary workaround is to just be honest about the work involved and the cost to the business. Easier said than done, but an open and frank discussion followed by adjustment and agreement will ensure the issue is examined properly and supported more widely.

One example I remember is a particularly troublesome database table. Originally conceived as an optimisation with a few hundred rows, it grew over time to be a real albatross, slowing both development and production work, and running to over 19 million rows. However, we couldn’t replace it without a lot of effort, because its tentacles were everywhere; we needed to schedule real time for it, and that would mean less time for other, more visible, work.

But when we presented the plan, two things sugared the pill. First, we had long cited the table as the reason for previous work taking longer than anyone would like, so the cost of its presence was already felt tangibly. Second, we ensured our plan was broken down using the “baby steps” approach outlined above — it would take a long time, but it would take out no more than 10% of our resources at any moment, and we could always suspend work for a period if absolutely necessary. The plan won support, was executed, and after several months our DBA typed the SQL “drop” command and we popped a bottle of champagne.

Meanwhile, back in the kitchen…

Of course, all that is assuming the temporary workaround really does need to be replaced. In my friends’ kitchen they are actually quite happy with the drawer dividers they’ve requisitioned as a spicerack. Aside from anything else, it provokes conversation and allows them to talk about their Azerbaijani builder’s many philosophies. He seems to have so many pithy sayings — all of which begin “As we say in Azerbaijan…” — that they’re beginning to suspect he may be making them up as he goes along. Still, if he does have to make a hasty exit from the building trade there may be an opening for him in the software industry.

There’s more to software development than getting a software project delivered to the users’ satisfaction, on time and on budget. (As if that wasn’t enough.) Sometimes someone has to ensure the system lasts for the long term. But when that happens it’s a balancing act: between the project and the product.

The project and the product might start at the same point, but they usually finish at very different points. A project finishes with the last milestone release of the code — and a party. The product finishes with the opposite: the last byte of code being removed from the servers — and a party only if it was really nasty software.

The balance between project and product is something that I see again and again in many different forms. Here are some examples from different stages of the project lifecycle…

Tools: Java verus Ruby

There’s an amusing little spat going on over at Michael Coté’s blog. It’s a fairly traditional tussle of Java versus Ruby (and other scripting tools), in which Coté accuses Java people of being overly concerned with internal architecture, while Ruby (and LAMP and Django) people are more concerned with delivering the goods. But Robert Cooper adds the telling observation that the maintenance of the delivered goods could well be a nightmare if it comes from the scripters, because it often involves a “mish mosh” of technologies with non-standard dependencies.

Robert’s point, more generally, is that getting the application out the door is only part of what a software team needs to worry about, and the choice of tools has a bearing on life beyond the project end.

People: Internal and external

Having an internal software team is a wonderful thing. Among other things you can build up a huge amount of expertise, both technical and business-specific, which helps everything go much more smoothly. But sometimes bringing in consultants from out the outside has a benefit, too, and beyond mere extra bodies. Consultants get to see many different projects, technologies, businesses and people over a very short time, and they bring this accumulated expertise with them.

Mixing internal and external people like this can work really well. In terms of ideas, external people can bring ideas and propose directions that might otherwise not have been considered. Internal people are all too aware of the long term implications of these ideas in their organisation and can stop the crazier ones before they get too far. In terms of making the best use of time, external people are often particularly driven by project deadlines, perhaps because they’re often only employed for projects, so it’s all they ever know. Internal people can see the time spent now but also have a very good feel for time likely to be spent in the future, so can determine better when additional time within a project is actually investment for the future.

Sometimes work is outsourced entirely to external teams. Sometimes work is undertaken entirely by internal people. When the two kinds of people mix there is creative conflict, and the balance between project and product is tested.

Release: Releases rarely happen once

I once worked on a six month project with a big bang release. At the end of the project was a single release which took three days. That wasn’t three days of integration, testing and launch. That was three days from the point when the software was entirely ready for the end users, to the point where it was actually available to those users.

In terms of the project this was fine: on the scale of a six month project, a three day release is negligible. But when we came to release 1.0.1 and release 1.0.2 it was a bit excessive, and by release 1.0.16 it was actually a serious business issue.

There are many lessons to be learnt from this, but one is that we had erred towards the project too much and our lack of consideration for the product caused us problems very, very quickly.

Soup to nuts

It’s too easy to forget about the product in the rush to the end of the project. At one time I realised I was so negligent in this area I stuck a large message on my monitor in 36 point font:

…and how will that be maintained?

Balancing project versus product affects everything from who you involve at the start through to how you hit your deadlines and on to how it affects the rest of your working life (and possibly your non-working life if you find yourself on call). There isn’t a one-size-fits-all answer, but at least if we are aware of the tension between the project and product we have a better chance of making more informed decisions.