Low software quality? Poor requirements may be to blame

Note: this post was originally published on InfoWorld.com.

When I read articles about improving the quality of software, I read a lot about new programming techniques and languages, as well as new ways to manage projects and requirements. But there have been times in my career when I was asked, as a developer, to work on a project that was destined for failure before I had written a line of code. While projects this extreme have thankfully been rare, times when I had to rescue a project from bad initial expectations were more common - common enough that I decided to get an MBA to combat the problem (more on that in my next post).

Apparently it’s not my imagination. In his book "Software Estimation" (Microsoft Press, 2006), Steve McConnell discusses the top factors that might cause a software project estimate to be incorrect. For example, he compared whether the presence or absence of a mature process would affect estimates as much as, say, which software tools you use.

Based on the number of articles and blogs available, you’d think that programmer quality would be on top of the list, followed by which tools the developers used. Oddly enough, programmer capability came in third on the list, not first, beaten by both software complexity and requirement analyst capability. Software tools came in a distant 10th. Improving our software tools is not where we should be spending a significant portion of our time if we want to improve our software quality.

We are addressing the first problem through the adoption of agile techniques. Building software using agile techniques by itself doesn’t make software less complex than building software using waterfall methodologies, but it does limit the complexity the development team needs to worry about at any one time.

What about improving the requirement analyst capability? About the best improvement I’ve seen in this area is that our user stories are now in the format of: “As a <type of user> I want to <goal> because <reason>.” When you look at advancements in technologies, programming languages, and project management techniques over the years, the corresponding advancements in requirement gathering and communication are more than a bit underwhelming.

What about business analysts? Isn’t it their job to translate business requirements into something actionable by the development team? In theory, yes. But there are two reasons why adding more business analysts to the software development process is not the answer to improving requirement gathering.

The first is that business analysts must provide significant value to the software development process in order to provide more good than harm. Remember playing “telephone” as a child? You may have had a different name for it, but basically it’s the game where everyone lines up in a row and a phrase is whispered from one child to the next until everyone has heard the message. Then the messages are compared, and everyone laughs at how jumbled the message gets.

Communication within the business world is similar: The more people there are between the two endpoints (in this case, the business sponsors and the software developers), the more messages get garbled and priorities get colored by the intermediaries. Some BAs, usually subject matter experts, can add more to the process than they take away, but these BAs are the exception, not the rule.

The second, related reason not to invest in more BAs to solve the requirements problem is that nearly all of the best solutions that I have designed or built came about during brainstorming sessions between the business sponsors and the development team. When business stakeholders who have a thorough understanding of the business problem they’re trying to solve collaborate with software developers and architects with a thorough understanding of what’s possible, magic can happen. When you put people (or processes) in between the two groups most responsible for getting the project done you (at best) doom yourself to mediocrity.

What can we do to improve our ability to gather and document our requirements in a meaningful way? Nothing quick or easy, I’m afraid. But in next week’s post (and throughout the life of this blog) I’ll go over potential solutions

On hiatus

I have a new blog with InfoWorld! My contract with them allows me to re-post my entries here too, so I will not be abandoning this blog, but I do have to wait 15 days before I'm able to do so. That means that there will be a few weeks before I'm able to start posting here again. I plan to start posting here again before the month is over, but if you don't want to wait that long, you can check out my InfoWorld blog here: http://www.infoworld.com/blog/software-project-success/