The Agile treadmill: Command and control in disguise?

Written by Nik

Agile comes with tons of literature on how to organise work at a very detailed level. How much of this treadmill is activity for activity’s sake, rituals, and religious manifestations of an ‘Agile Sub-Culture’ aimed at integrating a growing workforce? Can this relentless ‘heart beat’ and this esoteric jargon stifle innovation and alienate the very people Agile purports to help? Could more slack benefit a discipline which ultimately relies on people’s intelligence and creativity?

That’s how we set the scene a few days ago when I led a discussion among some peers from different companies entitled “The Agile treadmill: Command and control in disguise?” It was organised by IndigoBlue as part of their Second Wednesday series, under the Chatham House Rule. This is summary of what we discussed, with fictionalised initials for each participant.

In this article

Origins

When people talk about how Agile they are I find this usually coincides with how rigorous their processes are. Some of the largest, most successful Agile projects I know have an exhausting hothouse atmosphere, where everyone’s role is tightly defined and cannot be deviated from.

Have we created a kind of Animal Farm, where a system designed to set us free has actually enslaved us? Does it make a difference if Agile has been introduced by developers or management? Is effective Agile synonymous with a hothouse environment?

Between us we had an awful lot of experience to help answer these questions.

How useful is Agile process?

There was a general agreement with TC that there is some real value in having standard processes and practices:

  • Rewriting the rulebook every time you start a project or programme is wasteful if you can capture the processes and practices that are known to work;
  • It’s easier to bring in external resources if well-known language and practices are being used in the team.

But practices can be unnecessary, and there was recognition that many practitioners have a near-religious attachment to them, and to particular ways of working. As people managing programmes of work we often don’t have the discipline to recognise what the right level of discipline is… and isn’t.

There was also concern that Agile created an alienating language. NA indicated his particular discomfort over the word “customer” because that was often not how people saw themselves or their relationship with the tech teams.

Religious Agile, variable results

So no-one was anti-process (nor anti-Agile process), and we know it has produced good results. Our own major rebuild and redesign at the Guardian is one example of that, with a very highly regimented process and an outcome that at least met the aims and expectations. There are plenty of others.

On the other hand we collectively had experience of strong-process projects which weren’t delivering.

FC talked about Agile projects in his organisation. Most of the smaller ones were fine; these were projects with just a handful of developers. But the company’s most significant piece of work was having serious problems. It was now in its third year — nothing necessarily wrong with that, we said, after all some projects are just very large. And what had been delivered so far? Not much. Ah, we see the problem. And yet, he said, to the people on the ground this was a successful Agile project, because they were doing all the right things: daily stand-ups, rigorous unit testing, prioritised iterations… all the paraphernalia of an Agile project. They couldn’t see there was a problem.

AS reported a similar experience. “When I walked into the organisation they proudly walked me through their very rigorous process. ‘But nine out of your ten projects are on Red’, I said, ‘What is that process doing for you?'”

So what is Agile, anyway?

“Is Agile a process?” asked AS, rhetorically, “No. Is there process in it? Yes.” Process helps a project fit into the overall programme, added FC.

BJ went a bit further: successful Agile is about business change, not about what happens in the software team. The Agile approach to a large project is to get just enough clarity to embark on it, which probably isn’t every last detail. The stakeholders need to have the confidence to deal with uncertainty, and that sufficient uncertainty will continuously be resolved as the project progresses.

[And with that I’m going to drop the capital A from Agile unless I’m referring to by-the-book developer-level Agile.]

RN then came in with his take on this. He works in a highly bureaucratic company where technology is driven by heavyweight specs, a focus by high level people on low level details, and delivered entirely by specialist (external) suppliers. Yet his board are continually wanting things delivered faster, to greater quality, and he sees agile as the solution to this. It means persuading his board to think about what they want at the right level — business value, not the contents of drop-down menus — and providing reassurance through clear, regular deliverables. It also means getting his suppliers to deliver in that manner. But all of that is entirely independent of unit tests, story cards, and so on.

I wondered if this meant a top-down mandate of agile was more successful than a bottom up one. AS clarified: it’s successful if it’s business-led, not management-led. LB linked this to Kotter’s 8 steps to successful change, and in particular the first step: there has to be a sense of urgency.

I confessed that our processes in the Guardian team today are substantially less religious (or rigorous) than they were during the time of our major rebuild programme. I used to be able to confidently say we are Agile and point to intensive pair programme, lots of story cards, and so on. Today pairing is inconsistent, automated unit tests aren’t used in non-critical satellite code, and teams don’t always work to iterations. I explained this changed because our business had changed. The emphasis is no longer on transforming our platform but now much more on innovation and on smaller projects with a much wider variety of stakeholders. This makes the control freak within me distinctly uncomfortable (AS had previously commented that in many organisations command-and-control provides confidence) but we generally deliver projects as intended and have happy stakeholders, so I wasn’t going to wade in to change it. BK commented that this seemed perfectly appropriate, and still looked like an agile business.

Seeing the vision, from top to bottom

Successful agile is something that needs to permeate the organisation, from top to bottom. It’s about delivering value, and the vision for what needs to be delivered must be seen and understood at all levels. Many of our experiences of projects going awry were also experiences of where the project vision wasn’t seen by the people doing the day-to-day work.

TC told how his IT department decided it was going to achieve CMMI Level 2. After two years they had accumulated 10 rooms’ worth of documentation, and their productivity had dropped like a stone. And yes, he could measure that drop in productivity, because the team essentially did nothing else for those two years than produce documentation. The board inevitably said: Never do that again. CMMI accreditation may be useful, but it’s got to be for a purpose.

FC talked about his troubled project in its third year. “I revisited the original outcomes document a short while ago, and I couldn’t see how the activities around me are ever going to achieve those outcomes. When I ask one of the team leads what they’ve delivered, and how much there is to go, I’m just waved in the direction of a vast wall of index cards.”

AS talked about a daily scrum he attended: a two-minute meeting in which the team lead got a task update from each team member while he stood with his back to the story card board. “I had to take him to one side and tell him that he needed to be constantly referring people to the project outcomes, not just getting task updates.”

He also told the story of a business manager who wanted a change rolled out midweek — unprecedented for the organisation, as it would involve taking the website down for two hours. This was flatly refused by the Head of Systems. When they discussed the matter the issues were this: the business manager could show that if the change rolled out in the desired slot the company was projected to make an extra £1m, and if the slot was missed there wouldn’t be another opportunity for six months; the Head of Systems, meanwhile, had a target of 99.9% uptime and two hours downtime would mean he’d forfeit his bonus. “Our business must organise itself so that we are all focused on the same goals.”

What did we learn?

You may ask what our learnings were. I can confidently say there were no learnings whatsoever because “learnings” isn’t a real word. But there were probably lots of lessons. Here are three key things I came away with.

  1. Religious Agile, with an optional hothouse atmosphere, is entirely independent of a successful agile project. There are successful projects with and without religious Agile, and there are unsuccessful projects with and without religious Agile. This is underscored by…
  2. Agile is about business change. It really only works when all the stakeholders focus on the same business-level goals, understand what their part in it is (and isn’t), can deal with uncertainty, and continually work to reduce that uncertainty exactly when necessary.
  3. The business-level vision must be clear and permeate all levels. This is a corollary to the last point, but to me is significant enough to pull out. The people on the ground have to be able to relate all their activities to the business’s end goal, otherwise they will define success by other criteria — perhaps how well they adhere to a process — which are different from the actual success criteria.

Thanks to all the participants for making it such a stimulating morning.