Sunday, November 30, 2014

Peter Principle and Software Architects

It's time to address an uncomfortable truth within IT: technology professionals have a tough time finding, retaining, and promoting people with leadership ability. Instead, technology professionals, especially in consulting, tend to promote people based on their ability to do their current jobs and their knowledge of technology. This type of promotion leads to the Peter Principle, which is the idea that people get promoted until they cannot do their job anymore, which leads to them failing and getting fired. It also leads to business leaders losing their faith that IT can manage themselves, causing IT to lose their place at the executive table.

The problem is that typically people aren't promoted to a job that requires the same skills. Managing people well requires the ability to understand people so they can be motivated to do their best work, which only rarely requires a manager to be able to do a job better than the employees can. A leader (CIO, CTO, etc.) must understand their industry and know how to react to industry trends, and must be able to create plans for the managers to execute. But instead of promoting people based on their ability to do their new job, they're too often promoted based on their ability to do the job that they have. This makes sense to a certain extent - if you have a team member who clearly excels at their job more than any of their peers, would you promote one of the lesser skilled team members?

Within technology departments and companies there is another issue that is contributing to the problem. As I mentioned earlier, promotions within technology areas tend to occur based on a candidate's knowledge of technology. I would argue that knowledge of technology has surprisingly little bearing on how well a software product is built (though a detailed explanation of why that is the case will need to wait for another day), and it certainly has no bearing on turning around an underperforming employee's performance.

Let's examine a fairly common example of the Peter Principle at work: promoting a good software developer to a software architect. Many companies treat a software architect as a senior software developer, but if done correctly, the positions are quite different.

New TechnologiesMust understand the details of new and emerging technologies and how they can solve a particular problemMust understand at a high level how new and emerging technologies can solve a particular business problem
Business RequirementsMust take a micro-level business requirement and anticipate end-user needs (such as form field validation)Must take a business need and envision how that could be solved (with or without software)
Return On Investment (ROI)Does not need to understand ROINeeds to find the right balance between the best software solution and keeping costs appropriately low
People ManagementNeeds to be able to communicate with peers about the specifics of technology decisionsNeeds to be able to communicate with and persuade people with a wide range of experience, from developers to project managers to business stakeholders
CodeMust write clean, clear, understandable, and maintainable codeMust understand why developers focus on code quality and must be able to communicate these reasons to business stakeholders
DebuggingMust be able to find and fix problems on a micro-levelMust be able to look for macro-level logical fallacies in business and technical requirements

Looking at this list, it's not hard to see why people promote their best developers to be architects. On the surface, it appears as though the architect must do everything that the developer does, except have a higher-level understanding and have better communication skills. And I will be the first to admit that having developer skills would help an architect do their job well. But here are some problems that occur when you assume that the best developer can be an architect:
  • A good developer needs to be extremely detailed. They must understand where a user might make a mistake on a single form field, and understand how one line of code might cause problems several hundred lines later. If an architect tried to think at this level of detail, he/she would be extremely slow to make decisions as he/she gathered all the information needed to understand the problem.
  • Similarly, a developer must be able to communicate very small issues to other developers and to the architect. To contrast, an architect must understand his/her audience and provide the level of detail necessary for that person to understand the issue at hand. Just because a developer can communicate well does not mean that he/she can communicate as an architect should.
  • A good developer should spend time on their own learning about new and emerging technologies so they can work with them on a moment's notice. An architect must instead have at least some knowledge over a much wider array of subjects, both technology- and business-related.
In other words, the detail orientation that is needed to be a truly fantastic developer can at times be a hindrance for an architect. Instead of promoting the best developer on your team to be an architect, focus on finding the big-picture developers on your team and training them to be architects. To make this work, you'll also need to build a company structure where there are multiple means for advancement and build a culture where being excellent at a career without being promoted is okay.

You should also take the same approach when promoting people to leadership and management positions within your organization. Promote people based on their ability to do their new position rather than their old. Your company will be better for it.

Other posts in this series

November: The Peter Principle and software architects

Sunday, November 16, 2014

Programming beyond writing code (Part 1)

I recently came across an article by Laura Klein titled "Your Job Is Not to Write Code". I read the article because I happen to agree - it's a software developer's job to solve problems, and too many developers focus too much on the code and not on the problem at hand. I was hoping for another point of view, or at least something I could share on Twitter. Instead, I read a condescending and destructive post from a myopic product manager. I'm going to write a blog post soon about why software developers should be problem solvers before coders, but today I want to respond to Ms. Klein's post.

Quote #1
Your job is to improve our product for our users. If you want to get technical about it, your job is to improve our product for our users in a way that improves the key metrics of the company. But honestly, you don't always have a lot of control over that second bit. You do, however, have an enormous amount of control over the first bit!
One of the reasons I got into programming in the first place was that I was excited about building solutions to make people's lives easier. Then after a while I realized that it's generally not the programmers who set the strategy, set the scope, determine the metrics for success, or even come up with the general design. But it's these things that really make the most difference in the overall quality of the product. It's this lack of control that led me to getting an MBA and searching for several years for the right position where I did have the control to make a real difference. So I disagree quite heartily about a developer/programmer/engineer having an "enormous" amount of control over anything other than code quality.

Quote #2
For one thing, you need to make sure that the code you write (writing code is still one of the main things you will do when doing your job, by the way) runs the way it should, even on users' machines. [Emphasis in original.]
Don't get me wrong - I've definitely seen the attitude that "it works on my machine so it's not my problem". But I've also seen the attitude of "I don't want to hear 'it works on my machine'! That's just an excuse, and a pathetic one at that!" The truth is I simply can't always predict how my code will react in other environments and you don't have the budget for me to try. So I will make my best attempt to prevent these types of issues from happening, but realize that problems that only crop up on other people's machines will happen from time to time. And don't be surprised if it takes a long time to fix - any programmer will tell you how incredibly difficult it is to find and fix a problem that he/she can't recreate.

Quote #3
Did you know that our users probably don't have brand new MacBook Airs with giant Thunderbolt monitors set at the highest resolution possible and running the latest version of Chrome? I checked. A lot of them have Internet Explorer on 4 year old laptops, so sometimes things you build don't work right on their machines. They're still our users, and it's still your job to improve the product for them, so please make sure that the code you wrote works in a reasonable number of environments.
On its own, this statement is pretty fair. No programmer likes to dumb down their code because of IE's stupidity. Given the tone of the rest of the blog post, though, I feel like I have to defend the programmer's point of view here, too. "Works in a reasonable number of environments" is an extremely vague term. What is "reasonable" to a product manager who is fielding complaints from users is almost always different than to a developer whose primary focus is putting in the latest whiz-bang features. It's important to take a realistic look at the actual number of people with problems. Are we truly talking about "a lot" of people? Or are there just three people who complain a lot? (And yes, I've been in situations where people have said that "a lot of people have problems" when in reality it was a very small but very vocal portion of the total users.)

Also realize that by putting in work-arounds for other browsers, you are increasing the costs of development, increasing the costs of testing, and increasing the costs of support. These costs increase exponentially the older the browser you want to support. You are also increasing the chance that the user experience for people on the latest and greatest technologies will be degraded. In other words, you can't make a website easily accessible to everyone, give everyone the best experience possible, and do so at a reasonably low cost. Therefore it is absolutely critical that everyone stays on the same page as to when these additional costs are worth it and when they are not.

Quote #4
So, to avoid [having problems crop up in production], you need to check your changes in production. Every time.
Here's another quote that if found in an otherwise normal blog post, I wouldn't think twice about it. Given the attitude in the post, though, it gave me pause. To explain why, here's a situation that happened to me earlier this week:

In the process of customizing Microsoft's Team Foundation Server (TFS) to be a time tracking system as well as a work tracking system, I needed to make a couple of unrelated changes. One of them was a configuration change to TFS, the other a code change to a customization I built. I made the changes in our development environment and everything worked fine. I pushed the changes to production, tested each change, and everything still worked fine. But when users got their hands on it, they uncovered a problem when taking a particular action that touched both of my changes.

Most people would understand that problems like this happen. This is true especially when you consider that, in this situation, I had the choice of either spending 4-8 hours really testing the app hard or spending 30 minutes diagnosing the problem, fixing the bug, and installing that fix. Some people assume that just because a bug pops up that wasn't found in testing, that the problem must have been an oversight of the developer pushing up code. And yes, I'm making assumptions here, but the tone of Ms. Klein's post makes me think that she's in the latter camp.

Quote #5
Of course, in order to check your changes in production, you're going to need to make sure that your code actually gets merged and pushed into production. I mean, you can't really check your changes in production if you just let them sit unpushed for hours or days. Push your code. Get it into production. Then run it and check it.
In principle, I agree. But if you actually use these words, you're more likely to do harm than good. If you are condescending towards your developers, their mindset will tend towards pain avoidance rather than problem-solving. And which do you think would avoid more pain, delaying pushing up code or pushing up code for someone who apparently thinks she knows your job better than you do?

(On a side note, I actually got scolded twice in the last couple months by a coworker who didn't like that I pushed up code outside of my determined release schedule, despite the fact that I did so to help her team out. Really? The usual response towards someone who goes out of their way is "thank you", not "why is it acceptable for you to not follow an arbitrary process".)

Quote #6
Another thing to remember is that sometimes users do surprising things, which means that it's not enough just to test that your code works under perfect conditions. You need to make sure that it does something reasonable even in error cases and zero data states and when the user does something you might not expect, like use the back button or make two accounts by mistake.
The vast majority of programmers know to test for certain conditions, both for usability and for security. But we can't anticipate everything. I wish I could attribute the source, but the quote "if you make something idiot-proof, someone will make a better idiot" comes to mind. Ms. Klein, as a product manager, should know that better than anyone. And like my response to quote #4, the law of diminishing returns applies here too. I could spend 8 hours trying to break an application or I could spend 30 minutes to find the cause of that same problem and fix it. There are times when spending the extra time is appropriate. There are others when it is not. It is unwise to always assume the correct course of action is one or the other.

Quote #7
There's one more important part to your job. You need to make sure that we can measure whether we're all doing our jobs well. That means adding metrics and analytics so that we can test the effects of our changes. If you expect the code you are writing to improve user engagement (by improving the user experience in some key way), then you need to have a way to learn whether or not you succeeded. How else will you know if your job is done? Because, as I've mentioned, your job isn't done until you've improved the product for our users.
There are two problems with this statement. The first is that a programmer may certainly suggest that this would be a good feature to put in, but if you read my response to quote #1, you'll realize that whether the feature actually goes in is generally not the call of the programmer/engineer. It is a business/design decision.

The second problem is I've never worked on a project that was big enough, or had a budget large enough, to warrant such an investment. I've certainly been in situations when such tracking would be useful, but at the end of the day, most people are going to throw Google Analytics at the website and call it a day. If you have the budget to write your own tracking, great! Have at it! But don't take it for granted that this is a feature that MUST go into a system! For most products the benefit is just not worth the expense.

Quote #8 (Article Subtitle)
I am lucky enough to work with a small team of fantastic engineers who truly care about their customers. If you are not that lucky, this letter is for you to share with your engineering team.
My advice for you is that even if you feel the need to share this with your engineering team, don't. There's very little chance you will do any good, but a very great chance you will do some harm. Here's what will happen:

If your team is clearly apathetic, then showing them this article will make you feel better for a short time, but then you'll get even more frustrated after it does no good. Your team will tear this blog post apart (sometimes making excuses in the process) and lose respect for you.

If you have a young development team, reading this article will cause them to get paranoid that they aren't doing a good job and they'll get overly-analytical about their own work. They will then start making mistakes by trying too hard and pushing too quickly, blowing your budget out of the water and killing your timelines.

If you have a good team, then they will be annoyed with the fact that you wasted their time to read an article that was full of condescension and devoid of any real solutions. Good leadership requires you to understand problems and find solutions. This article does neither beyond just "work better".

But as I said, I actually agree with more points in the article than not. I just think the tone is horrifyingly destructive. In the next few weeks I'll come out with a post with a different take on this problem.

Sunday, November 9, 2014

Hiring in service-oriented vs. innovative environments

This is the seventh in a series of eight posts that examines the relationship between business and IT. Based on an article examining manufacturing by Steven Wheelright and Robert Hayes, I have talked about four different levels (or stages) at which IT can interact with the rest of the business:
  • Stage 1: Internally Neutral (Minimize IT’s Negative Potential)
  • Stage 2: Externally Neutral (Parity with Competitors or IT as a Service Provider)
  • Stage 3: Internally Supportive (Provides Credible Support to the Business Strategy)
  • Stage 4: Externally Supportive (Provides an Important Contribution to the Competitive Success of the Organization)
In this month's continuation of this series, I'll talk about how hiring practices differ among the environments that are worth talking about. (Followers of this series will recall that Stage 3 is unsustainable.) 

Hiring in a dysfunctional environment

If you're unfortunate enough to work in a dysfunctional Stage 1 environment where the goal is to minimize the risk of an IT failure, you should be prepared to do one of two things:
  1. Hire someone who only meets the minimum requirements for the job
  2. Hire someone who is well-qualified for the job, but be prepared to pay well over market rate
Obviously you wouldn't want to hire someone who is not qualified for the job. But why wouldn't you want to hire someone who is well-qualified for market rate? Stage 1 environments are miserable places to work and well-qualified job applicants in technology generally have plenty of other places to work. Looking for and training a new employee is difficult and expensive. If you're going to hire someone who can get a job elsewhere, you're going to need to pay them well enough to ignore the bad work environment.

When would you choose one over the other? Unless you are actively trying to change the culture of the organization, it is unlikely that you'll have the budget to overpay someone, so you may have no choice but to look for the barely-competent candidates.

Hiring in a service-oriented environment

If your goal as a technology department is to provide a service to the rest of the business, you're probably tempted to go after the superstar technology expert when you have an open position. Try to avoid this temptation. Superstar experts tend to want to work on cutting-edge projects that only result when someone with knowledge of what's possible with technology either drives a project or closely collaborates with the business person driving the project. When the cutting-edge innovation isn't baked into the design, these experts tend either to lose interest in the project or go rogue and add their own innovations.

Instead, you have two choices:
  1. Hire a solid contributor at market rates
  2. Hire an above-average contributor, either at an above-market salary or find other outlets for creativity
Choice #1 is the lower risk, lower reward scenario of these two. If you choose to hire an above-average contributor and try to find other ways to make the work rewarding for them, be sure that this is communicated well during the interview process and don't oversell the position.

Hiring in an innovative environment

If you have a truly innovative environment where collaboration occurs with the rest of the business, your best bet will be to hire superstar technology people with a passion for problem-solving. These types of people will fit in with the culture of the business best.

However, you may want to consider adding one or two average contributors to the team. You typically want your innovators innovating, but most programmers need to do quite a bit of non-innovative work as a part of their day-to-day jobs. (Don't we all?) Hiring someone who can get this work done will free your best people to do their best work and raise the productivity of the team as a whole.

Sunday, November 2, 2014

Wanting to move to Agile? Certifications are useless.

A couple of days ago, I read an interesting article about how Agile certificates are actively destructive to the Agile community. One of the most important points he made was this: because Agile certifications aren't sufficient to prove whether you are an expert or not, companies that hire certified "experts" to launch their Agile efforts will fail, and therefore believe that Agile is not a valid approach. While I agree with most of what the author said, he missed an important point. There is a significant difference between being able to lead an Agile project and helping a company with an Agile transformation. To see why, we need first to look at what a certification does and doesn't cover.

What certifications show

Good certifications generally show an individual's skill with a particular subject, i.e. whether they understand the terminology and theory about that particular approach or technology. Even the best technology certifications, though, typically do a poor job showing an individual's knowledge of a particular subject, i.e. whether they can make the necessary adaptions to the approach or technology to solve the problem at hand. (A further discussion about skills vs. knowledge can be found in an earlier blog post about age discrimination in IT.)

Why skills aren't enough

If you are in an organization that already is using Agile to great success, plugging in someone with the skills to get started but without the knowledge to lead may not do that much damage. Existing processes and co-workers will often be enough to overcome knowledge deficiencies. But Agile, at its best, is a cultural change, not a procedural one. The minute you start trying to make Agile a procedural change within your organization you've failed. Instead, Agile is more about increasing collaboration and maximizing flexibility than following a process. (For an explanation why, please read my post about the subject from earlier this year.)

Why this can be a difficult cultural change

A manager's job is to ensure that his or her people have what they need in order to succeed. Most managers I know achieve this by doing two things: making sure that their people have the processes and procedures in place to do their job predictably well, and creating and monitoring metrics that demonstrate their team's effectiveness. Adopting an Agile culture makes these two goals extremely difficult. How can you create processes when you're trying to increase collaboration with outside teams? How can you measure success when maximizing flexibility makes monitoring time and scope, the two most common IT project metrics, obsolete?

The other significant challenge that teams face when moving from Waterfall to Agile is that Agile projects have immeasurably more uncertainty. With Agile, scope is always changing, responsibility is shared, and there is no "definition of done". Selling this to a group of people who have a (false) sense of security with a fixed-scope, fixed-budget project can be overwhelming to say the least.

All these sound like Agile skills. What's missing?

If you try to implement Agile in these environments, you're going to run into significant challenges from other people who, either intentionally or unintentionally, sabotage the change effort due to their own discomfort. To get through this, you need a change management expert who can create the sense of urgency necessary to get people behind the change effort. You will have much better luck achieving a transformation with a change management expert who is not up on the latest Agile frameworks than an Agile expert who is not familiar with managing people through a change.


If you are starting, or planning to start, a transformation within your company to a more Agile mindset, don't worry too much about Agile certifications. You want someone leading the project who has done it successfully before. If, in the case of the original article, you're in HR and don't have the skills to evaluate candidates, then get references and hire the people who have already achieved what you're looking to do. Then let the certification be a tie-breaker in case you really need one.