4 common mistakes in rolling out purchased business software

If your job is to make software, and you're not living under a rock, you've probably heard dozens of stories of software projects that never made it to market because of uncontrolled scope creep, poor software quality, blown budgets, and the list goes on. In theory, purchased software should be safer, but I've witnessed more failures of rolling out purchased software than I have of custom software, despite the fact that I've been involved in roughly the same percentage of each type of project. What can go wrong? Here are four problems I have either witnessed or was brought in to fix.

Solving all problems at once

When companies first get their hands on new software, their instinct is to try to solve all of their problems at once. They'll jump head first into the new system but get lost in all of the possibilities of how to use it. After taking much longer to start than expected, they'll discover that their elaborate designs are invalid because of some basic understandings of how the system works. Instead, companies should focus on starting slow and simple, building on successes and learning from failures.

One time I saw this first hand was at a company that wished to use Team Foundation Server (TFS) from Microsoft as its project/work tracking software. In most companies, this would not be a big deal. This one, though, was going to try to customize TFS so that it could replace two custom-built work tracking software systems, custom-built time tracking software, and dozens of spreadsheets. When the company tried to tackle everything at once, they got overwhelmed with all of the problems that come with such an overhaul. After we reduced the scope for phase 1, the project started moving more quickly for the organization and allowed us to make continuous progress towards fixing all of the problems.

Choosing the wrong place to start

When companies purchase software to solve multiple problems, they typically jump in trying to solve the wrong problem. They'll either try to solve the business' most pressing need, or they use the software to fix an issue for a particularly assertive stakeholder. Even the best-designed software has a learning curve, and companies have to learn when it is appropriate to use the software and when another solution should be found. Instead, companies should focus on solving a non-trivial problem that the software is particularly well-suited to fix. This allows the company to build on positive experiences and provides a foundation to solve more difficult problems in the future.

Perhaps the most glaring example of this I've run across in my career dealt with a SharePoint implementation. My company was called in to help a client get started with SharePoint, since they had a great need for simplifying internal website creation and getting a handle on their document management. SharePoint is great at both of these things. But instead of starting on one of these projects, they tried to shoehorn a workflow system into SharePoint that stretched the abilities of the system. The implementation was so bad that the company almost abandoned the system entirely. 

Trying to customize the system too much

Purchased software can only do so much. In order to make the software appeal to as wide an audience as possible, the software makers will generally make the product as generic as possible. This means that businesses are faced with the choice of either customizing the system or living with a less-than-ideal solution. Most businesses will lean towards the former when they should really be focusing on the latter. Customizing third-party systems should be done as sparingly as possible, since any customization makes supporting the system more difficult in the long run.

This situation is actually quite common, especially with software whose main selling points are flexibility and configurability. You could pretty easily use the previous example as an example of why customizing the system too much is not a good thing. Instead, I will focus on some customizations I did as a consultant for a company trying to implement a Content Management System (CMS). Our client had products, but worked with distributors and so didn't want to post prices on their site, otherwise an e-Commerce system would have been appropriate. We had two choices - use the CMS as-is and ask our client to manually create pages for each product, or modify the CMS to make it easier to load and update multiple products at a time. We chose the latter approach. We did the best we could, but we never were able to get the system to work reliably, and the user interface was hard to use. We would have been much better off building an add-on that merely looked like it was a part of the system.

Not monitoring usage/progress

If you put new software in front of users, they will usually happily continue what they were doing before. They want to get their work done, and learning new software doesn't do that, at least not in the short term. If you make it impossible to carry on as before, they will try to use the new system, but in ways that you never would have expected. If either of these are allowed to continue, your new system won't be used very well if at all. Here the maxim "trust but verify" should be remembered if you want your new software to succeed.

An example I witnessed of this was a particularly difficult implementation of a Microsoft Dynamics CRM (Customer Relationship Management) solution. The project champion pushed for adoption of this system and customized several areas within CRM specifically for this company, but not only failed to follow up to ensure the users were using the system properly, but also neglected to train managers on how to use the system or why it is important to the company. When the project champion left the company, the software predictably fell into disuse to the point where no one understood why the company continued using it.

And you'll notice that documentation didn't make the list...

Many managers place a heavy emphasis on creating usable and complete documentation, yet I've never had a project succeed or fail because of the presence or absence of documentation. Motivated people will do the right thing regardless of documentation and unmotivated people won't check the documentation regardless of circumstances. Documentation is important, but it's a highly overrated component of software project success or failure.

Conclusion

If you want your software customization project to succeed, start by implementing a small and simple part of the system first whenever possible. Find a good fit for both your company and the software to help people understand how to work within the system. Have a roadmap for where you want to go, and communicate it clearly and consistently so people stay on track. Finally, monitor everyone so you stay on track to both ensure that people are using the system but also that the usage appropriately solves the problem at hand. If you don't do these things, your software either won't be used or won't be used well, so you won't get much out of your investment.

Unclear specs? Maybe you need to stop writing specs.

I read a blog post recently from a company describing how their product can help developers be more productive. I thought it was fairly interesting in that it talked about creating better specs and creating prototypes to help everyone understand the problem being solved. But it suffered from one fundamental flaw:

In the most successful software projects that I've ever worked on, we didn't write any specifications at all.

If you work in traditional software environments, the idea that you wouldn't have business analysts writing specifications for the developers is probably quite shocking. But skipping specifications works so well that I'd be hard pressed, as a developer, to work from a spec again. How did we make it work?

Generally, the software architects were responsible for all leadership roles on the software team. In addition to deciding how the system would be built on the software side, they were also responsible for making sure that the software met the needs of the business, communicating status to all interested parties, and ensuring that the software met established time and budget goals. Someone serving as the product owner oversaw the entire process, setting priorities and making decisions on behalf of the user team. What are the advantages of this approach?

Less staff needed

The typical process to create software specifications is as follows:

1. The product owner has an idea for a new change or feature and either writes up a summary or talks to a business analyst.
2. The business analyst spends time to understand the business specifications, and gets clarification from the product owner as needed. Then the business analyst writes business specifications describing the changes desired for the software system.
3. A senior developer or architect takes the business specifications and writes technical specifications describing in detail how the system should be changed to meet the business specifications.
4. The developers take the time to understand the technical specifications and clarify issues with the architect and business analyst. The BA will go back to the product owner as necessary. The developers then implement a solution.

If the BA is an expert in both the business processes and technology, and if the developers truly cannot understand the business need, this process makes sense. Unfortunately, in my experience neither of these is true. BAs are rarely experts in technology and developers can understand the business need. Yes, by severely reducing or eliminating the BAs from this process, you put additional work and responsibility onto both the product owner and the developers. But if you reach a common understanding of what you are trying to accomplish and are flexible with the details, then BAs and specs typically get in the way.

Less rework needed

If there is one constant in life, it's that things will change. Changes in software occur because of misunderstandings, changes in business context, changes in management, additional features needed, and so forth. Writing code to be easily changeable is more difficult than writing code to meet the need at the moment. Therefore, being able to anticipate where changes might occur is key to making software easy to maintain in the future. When the development team understands the business, then anticipating change becomes relatively easy. When developers are limited to implementing technical specifications, anticipating change can be almost impossible.

Unleash creativity

Some of the best solutions I've created resulted from conversations between me and the product owner when we were just tossing around ideas - me from the point of view of what's possible and the product owner from the point of view of what's needed. Specifications can gum up this creative process. Both product owners and business analysts will unnecessarily simplify their requests in a well-meaning but misguided attempt to put the developers in a position to succeed, resulting in a subpar product.

Less miscommunication

The single biggest problem in communication is the illusion that it has taken place.
-George Bernard Shaw

The idea behind specifications, both business and technical, is to ensure that all interested parties have the same understanding of what is supposed to be created. Surprisingly, detailed technical specifications are often inadequate for this task. My best guess is that people, when presented with a large number of small details, fail to see the big picture. This results in software that may meet expectations but doesn't meet the business or customer needs.

What about testing?

Many people, when reading this post, would wonder how testing is done, since the business specifications are often given to the testing team to create test plans. Testing is like programming in that it's much harder to work with vague instructions, but if you have the right people in place, vague instructions result in much higher quality. I have not worked with testers much of my career, but most of my experience with testing teams is that when they have specific testing plans, they only test using those test plans. But users act in unexpected ways. Testing teams that are given free rein to think like the users find much more than testing teams who follow a test plan. Here again it is better to ask teams to accomplish a goal than finish a series of tasks.

Conclusion

Software efforts are the most successful when:

1. The people who are building the software have direct access to the people using it and vice versa
2. Everyone has the same big picture understanding of the problems being solved with the new software
3. Effort spent on directly making the product better is maximized, while other activities are minimized

Writing specifications gets in the way of all three of these. If your developers are reasonably intelligent, then skipping writing formal specifications will improve your overall team productivity. If you have lower quality developers, or the product owners have little interest in being a part of the creation process, or you have a large team and need to keep everyone moving in the same direction, then specifications may be helpful. But don't create them just because by having them you are following best practices. In many situations, they are no longer needed.

Creating a leadership team to make your software project a success

In this, my last post in my Leadership vs. Management series (a week later than I intended due to the holidays), it's time to put it all together and talk about what leadership and management are needed for the successful completion of a project. This is going to be a difficult topic to tackle in a single blog post since the leadership team you'd put in place for a 200-hour project in a small company would look very different from a leadership team built for a company-wide mission-critical project for a Fortune 500 company. Nevertheless, I can still address some of the commonalities between the two.

Leadership

The leadership needed for a software project is relatively straightforward:

1. You need a Product Owner to serve as the final decision maker to ensure that the software meets business objectives, including functionality, timelines, and budget. He/she does this by keeping the team focused on the features and functionality that the software needs to accomplish its goals. He/she also must decide when a blown budget means that the project should be scrapped, when it should be scaled down, and when to forge ahead anyway.
2. You need a Chief Technical Architect to serve as the final arbiter between all technical groups (developers, server administrators, database architects, etc.). These groups have very different goals in their jobs and you need a leader who can balance the needs of the project as a whole, primarily (though not entirely) ensuring that the developers are able to deliver functionality without putting security holes or data issues into the system.

I've seen companies try to use multiple people for each of these leadership roles, but that approach almost always fails. When multiple people serve as the product owner, business objectives get ignored because an arbitrary budget isn't being met and important features get ignored in favor of ones pushed by more assertive sponsors. Having multiple architects almost always means having hours of discussion over security/stability vs. speed rather than delivering value.

I don't think it's necessary to have separate people serve as the business lead and the technical lead, but that requires one person to be an expert on the business as well as an expert in technology. Difficult, though not impossible.

For larger projects, it is possible to have leaders of smaller components of the software development effort, but they should always be answerable to the product owner and chief technical architect.

Management

While the leadership needed for a successful software project completion is relatively straightforward, the appropriate level of management is anything but. The number of managers needed will not only vary by project size and purpose, but also by company type, skill level of the workers, and the methodology used for project completion. However, here are a few guidelines:

• Waterfall projects generally need more management than Agile ones. It is quite possible, with the right approach and tools, for leaders to manage an Agile project without any help from a dedicated manager.
• Related to the first point, when time and budget are critical to your project's success, having a dedicated project manager can be extremely helpful.
• Be careful assuming that your leader can manage the project well (or vice versa). Management and leadership skills are quite different, so if you try to make your leaders manage the project, make sure they have the skills to do so beforehand.
• Nothing says "I don't trust you" like putting in excessive amounts of management. If you think you need a large number of managers for the number of employees you have, maybe you need new employees (or at least better training).

Summary

You need someone to lead the software project from a business point of view and you need someone to lead the technical effort. Committees don't work well here. Management should be kept to a minimum, though what this means for you will vary. On many small projects, you may not need any particular person to serve as a dedicated manager. On large projects, you may need several managers, each managing one portion of the delivery effort.

Other posts in this series

June: An introduction to Leadership vs. Management

July: Is a project manager the right person to put in charge of your software project?

August: How do you manage an Agile software project?

September: How to leverage a project administrator effectively

October: Getting programmers to see the value in management

November: The Peter Principle and software architects

December: Creating a leadership team to make your software project a success