FOREWORD (by me): Have you ever seen one of those posters they put up on college campuses with “SEX” or “FREE BEER” written in 120 point type at the top of the page immediately followed by the words, “there, we got your attention” in 10 point type. Anyone who reads my blog knows that despite all title theory to the contrary I like strange titles that don’t tell you what the article is about. Today’s title is so far afield that it sounds like one of those college posters. It probably isn’t, but since I’m employing an agile writing style for today’s post I’ll just have to wait and see where we end up. Finally, a word of caution, I’m in a bit of a cynical mood today.

Coding in circles

Using a protractor and the science of finger-counting I recently determined that slightly over half of all software life cycles are circles (55.6%), most of the rest are variants on the line theme (horizontal, vertical, or perhaps a wandering line). You can see this for yourself by doing a google image search for “software life cycle”. These are happy pictures that convey a sense of calm and order, and in the case of the line a sense of progress. Mostly though they just make my head ache.

People draw diagrams like these for one of three reasons 1) they need to fool their bosses into believing they have an orderly and well defined process, 2) they need to comfort themselves that, regardless of how chaotic the current development process is, someday it will be orderly, or 3) they work in a cube and need to keep looking busy all day (if they worked in a private office they could just browse for porn on the Internet).

Now, add the word “agile” to the search and you’ll notice something completely different: an absence of either circles or lines. Almost no one is willing to provide a diagram at all, and those who do invariably have cyclical connected graphs or three dimensional stacked thingies. I can think of three possible conclusions to draw from this: 1) people who use agile processes have no idea what they are doing, 2) they really take themselves seriously when they describe the software life cycle as managed chaos, or 3) all of them really suck at making cool diagrams.

A day at the office

Process diagrams are inherently superficial, so perhaps not drawing one is the sign of true maturity. I’ve made sure to include plenty in this article just in case that’s true. Imagine the process diagram for this fairly typical little hack I delivered recently:

• Days 1 and 2: Read the description of the project on the plane to the US. It was meaningless so mostly I just watched movies in Spanish.
• Days 3-5: Interviewed the alleged stake-holders and figured out that only one Senior VP had any idea what the project was about. I eventually tracked down the SVP and got it straightened out. The goal was to take totally meaningless data and organize it into a very accurate summary to demonstrate “where the organization was at.” Since none of it made sense, I used my bracers of +5 consultant-meta-think and figured out that he needed charts to make him look good in his leadership meetings.
• Day 6: Mocked up some sexy graphs and dashboards, got buy-in and was informed that it was critical that the information be kept up to date “real-time”. These kinds of projects almost never need to be real-time. Mostly the graphs just get cut and pasted into a weekly status email. His feeling was that it was an “organizational imperative” that people be able to keep the data in the “forefront of their thinking” and that “always-on, real-time availability” would make that happen. With so much jargon it was hard for me to wade out of the room, but at least I got sign-off on the layouts.
• Days 7-22: I did all of the following (via cooperatively multitasking):
  • Filled out the HR Form-O-Rama.
  • Engaged in a running battle with IT to gain access to the servers with the critical data and set up a deployment platform.
  • Wrote a simulation framework to stand in for the data I would eventually get off the “secure” servers.
  • Schooled the locals in foosball with my “snake” shot.
  • Researched tools to help draw the graphs.
  • Ranted about the lame development practices of the organization.
  • Answered 236.3 questions about India.
  • Wrote some code using Java, JavaScript, HTML, XML, SQL and virtually every other letter of the alphabet (especially capital J and lowercase i).
  • Helped fix several things completely unrelated to the project.
  • Drank approximately 18.006 liters of Coke.
  • Gave a lunch-talk on unit testing.
  • Remembered that I should actually write some unit tests and went back and added them the easiest parts of the system.
• Day 23: Spent most of the day in a meeting with the SVP and a bunch of his pissed-off staff members. It was decided that yes, indeed, the project would go forward even though it would force them to subvert their security.
• Days 24-26: Flew back to India
• Days 27-40: Hand-held IT through deployment and fixed approximately 23 bugs none of which were in any of the code I had written unit tests for. Actually, most of this time was spent doing other things (like preparing my bill and playing World of Warcraft). It just took a long time for them to “reconfigure their security environment” to support the new project.

Safety through incompetence

Even though this just involved one person (me) I doubt anyone could accurately diagram my development process. If someone could it would be either Picasso or perhaps M. C. Escher. Drawing diagrams just seems like a form of self-delusion, so I can see how the agile approach may be better. This probably does little to help make whatever random point I set out to make about agile processes but it does remind me of an observation I’ve made about why software gets written. Here are my findings on the reasons for software projects (to five digits of imprecision):

• 11.301% of software is written to save money.
• 13.852% of software is written to make money.
• 24.718% of software is written to make someone in the company (or some department) look good to their peers or bosses.
• 50.129% of software is written as a panic reaction to solve a problem that no one understands.

That’s right, most projects (almost 75% if you believe those numbers) are a huge waste of time and energy and should be abandoned as soon as humanly possible. Back in the days of my father they had a well designed process that deftly protected the organization against these projects. It was called the “waterfall” software development process. While it’s been academically dead for decades, all of the circle and line diagrams continue to pay homage to it. I suspect this means it’s a fairly dominant form in the wild. Managers like the waterfall model for the same reasons that tourists like real waterfalls, they are simple and powerful and beautiful to look at. They are much less fun when you go down one.

What the waterfall does well is to keep useless projects from resulting in useless code that needs to be maintained. I’m not sure if that’s the real purpose, but it’s certainly a great side benefit. It may sound inefficient to pay a lot of engineers to get started on projects, do a bunch of analysis and design, and finally abandon the whole thing when something else becomes a higher priority, but every line of code they don’t write is another line that can’t break! Back in the 90s, some senior engineers got through years at a time without ever having any code see the light of day. Even trade unions can’t promise that kind of job security (perhaps that explains ISO 9000).

Agile to the rescue

The one group that always had mixed feelings about the waterfall model was consultants. Consultants can’t afford to show up, work for months at a time and produce no real results (unless they’re Accenture). Having teams of well dressed twenty-somethings around is great for morale, but even the most clueless CIO eventually has to have them kick out some code. The waterfall model also works against the consulting business model. In consulting you need the new blood pushing as many billable hours as possible, so you want them on the clock early in the schedule. Unfortunately, you can’t have fresh college grads doing those first few months of “analysis” and “design”. It’s just not credible.

Luckily for the consulting industry there is now a better alternative: agile! Designed primarily by consultants*, the various agile processes solve the core issues of the waterfall approach. This is achieved by adopting the doe-eyed belief that code is intrinsically a good thing and that what companies need is more of it. With agile you don’t ask the company to look out into their future and figure out what they really need. As I’ve already noted companies suck at that anyway, so you stop asking. With agile you ask them to tell you what they want right now so you can give it to them right now! This is one of those so “so brilliant I can’t believe we didn’t think of it before” kinds of ideas. It’s like Marijuana legalization meets software process.

From a practical standpoint agile solves the two problems consultants had with the waterfall process. They get to have very tangible results they can point to, and they eliminate the project management nightmare of not having all their people on the project as early as possible. At the same time, companies get what they think they want. It’s a win-win situation.

Just enough rope

The danger of agile processes is not that they won’t work, it’s that they will work too well. Code, it seems to me, is for knowledge companies what oil was for heavy industry: vitally important but with long-term negative side-effects. When you put up your first factory it’s no big deal, the Swomee-Swans keep singing and a little smog is a small price to pay for your Thneeds (which everyone needs), but add another and another and pretty soon the Brown Bar-ba-loots are gone and you’re arguing about global warming. Walk in to any medium to large knowledge company over 6 years old and you can see the equivalent effect of code. There are systems and databases and processes and interactions that will have grown so complex that no one really understands them any more. Real information will have somehow been replaced with data that no one trusts and the work-arounds will outnumber the procedures. There will be a running argument about reengineering the whole thing from scratch.

You may be tempted to believe that the added efficiency of agile processes may help eliminate the coding logjam by increasing the focus on bug fixing and increasing the chances that badly designed components will get reworked. Of course you may also be tempted to believe you can fly if you’ve just taken a hit of PCP, but that won’t keep you from falling 20 stories to your death. No, my suspicion is that as companies move into agile processes they will find themselves digging even deeper holes from which to extract themselves. Even more cynically, rather than then coming to understand that what they need is not more software but better thinking about how much they can manage and what it should do, these same companies will instead assume that agile processes are to blame and gradually turn them into waterfalls just like they have done with every other process. When that comes to pass all those lovely circle diagrams will seem almost prophetic.

Have a happy day!

AFTERWORD (by me): Well, when I started I actually did have a plan to tie the title in with the article, but, as sometimes happens in agile processes, I later decided to pull out that part of the article and save it for a later post when I can be a little more serious. I decided to leave the free-beer style title since I spent so much time putting together the image for it:)