When being under an estimate can be a problem

Recently, I wrote about how software projects should be measured by other criteria than time and budget.  Time and budget measurements aren’t going away, though, so we need to make sure that we act appropriately to make sure that focusing too much on time and budget harms our ability to deliver business value.

The problems associated with being on a project over the original estimate are relatively straightforward.  Pressure to meet a pre-determined schedule could result in all parties involved cutting corners, which could result in poor designs, incomplete implementations, inadequately tested features, and so on.  Very likely these results are already familiar to you.

But what happens if the software development process is significantly under budget?  (Yes, it happens.  I’ve been involved in several.)  Generally, one of two things happens:

1. The project is completed significantly under budget
2. Scope creep is allowed and unnecessary features are added

The first one is not a problem.  The estimate was, after all, just an estimate.  If you’re good at estimating, you should be under half the time and over half the time, assuming a reasonable tolerance for error each way.  But what about the second result?  For reasons I wrote about in an earlier post, adding features just because you can doesn’t mean that you should.  These features, if not thought-out properly (and they usually aren’t), will likely add maintenance costs and additional points of failure.  If any stakeholders in the project must use the entire budget, define some “nice-to-have” features ahead of time.  This way these features can the thought and planning they deserve.

How tech-focused do we need our developers to be?

In several earlier posts, I have written about how I think it’s time for software developers to stop thinking so much about the technology they use and start focusing on the business problems they solve.  There are two primary reasons for this:

1. Writing maintainable software isn’t easy, but it’s getting easier every day.  Integrated Development Environments, such as Visual Studio or Eclipse, give developers the tools they need to write, refactor, and debug software at their fingertips.  Modern programming languages give developers needed functionality out-of-the-box, while abstracting some of the lower-level concepts, to help developers be more productive than ever before.  Finally, and perhaps more importantly, with the thousands of technical bloggers and bulletin boards out there, it’s easier than ever to find answers to your questions using your favorite search engine.
2. The average technologist doesn’t know about the inner workings of a business process, nor do they seem to care.  "Tell me what to build and I’ll build it" is an attitude that’s all too common among technologists today.  On the flip side, too many business people know little to nothing about the technology that they’re using and managing.  Their attitude seems to be "that's a technology issue, so the technology people should deal with it".  Someone needs to bridge the gap in order to create solutions that bring out the best of both worlds.

Business analysts are supposed to fill this gap, but too many times true business analysts are simply not present, or not focused on what truly makes a project a success.  I've heard about some cases where business analysts function as middle-men that just get in the way, because they don’t know enough about either the business or the technology.  Project managers should be able to fill this gap too, if they are to truly have the knowledge necessary to be the ones primarily responsible for delivering a product.  Time and budget constraints get in the way, though.

Should it fall on either the developer or software architect to fill the gap?  I used to think so, but I used to work in a consulting company where the developer worked directly with the client, with no business analysts or project managers to step in and help.  In that particular case, it absolutely makes sense for all of the developers to go take a business class or three and be versed in both business and technology.

But what about situations where business analysts and project managers are present?  As much as development is getting easier, it’s still hard.  It took me four years of study and experience before I felt like I knew enough to be fully comfortable as a technologist, so I started branching out by getting an MBA.  But I know full well that I don’t have the knowledge to create a site with the scalability needs of Google, or the perfection needs of a pacemaker.  The need for programmers to know other items not directly related to a particular technology is high as well, such as software readability, predicting scalability, etc.

I think the answer is that the hard-core technologist isn’t going away, despite the rise of business/technology hybrids.  Instead, I see software teams made up of a mix of strong technologists and adequate technologists with a strong business background.  It would help greatly if we could also have all people in the development process gain more specific technology and implementation knowledge.  This is not a small task, but a necessary one if we are going to enable our technology to meet business needs more easily.

Good is the enemy of great

The first two sentences of Good to Great: Why Some Companies Make the Leap... and Others Don't by Jim Collins are:

Good is the enemy of great.

And that is one of the key reasons why we have so little that becomes great.

At some point I may write a blog about the book as a whole in some blog post, but today I’m just going to focus on those two sentences.  When I first read them, I had to put the book down and think about those two sentences before being able to move on.  I’ve encountered this in so many different ways.  Once one has achieved a certain level of competence, it’s difficult to continue to push boundaries in order to go beyond that.  Everyone has a different idea what is "great", though.  Is great creating the highest quality product possible?  Or is it finding the best balance between quality and cost?  To make matters more complicated, what we define as "great" will change as we are better able to provide services.  Before we can agree whether we have so little that becomes great, we must first define it.  Personally, I see greatness in people who can go beyond their typical job role and provide some extra benefit to their customers.  Take these examples:

A good developer gets the job done in a relatively short amount of time, and they are able to find most of their bugs in their testing process.  They understand and use programming best practices to avoid errors and help the next person viewing their code.  A great developer can find bugs just by looking at code, and errors found during testing come as a surprise.  They also understand when not to use a particular best practice if it does not apply to that particular situation.

A good designer can take a color palette and a set of business requirements and come up with a design that impresses users on first glance.  A great designer can understand the needs of the end user to create a design that is easy and natural to use both the first use and the thousandth use.

A good project manager can tell if a project is at risk of not meeting stakeholders’ expectations and can balance the needs between different groups involved, such as developers, designers, and stakeholders.  A great project manager can anticipate and mitigate problems before they arrive, as well as keep the entire team focused on what’s really important in a project.

A good business analyst takes business specifications and turns them into software specifications that the team can understand.  A great business analyst takes business requirements, creates a holistic business solution.  They then can create documentation for the software that supports this business solution.

So how is the good the enemy of great?  In all of the cases I’ve outlined above, not only can one can make a career out of being good, but some managers might think that striving for greatness detracts from the main problem at hand.  (My definitions of greatness involve going beyond traditional job roles, which might not be appreciated by no-nonsense, “get-it-done” type managers.)  Because most people can make a career out of being good, they limit themselves and never reach for true greatness.

Going beyond Collins' point, I would also argue that being good encourages complacency.  Continued complacency leads to adequacy.  Continued adequacy leads to stagnation.  In our ever-changing world, stagnation leads to mediocrity and worse.  It’s no coincidence that all of the companies Collins profiled achieved greatness by shaking up the status quo.  Don’t accept something that is merely good when you can strive for something great.  You may not reap the benefits today, but you will someday.

The difference between a "project" and "product" in software development

What is the difference between delivering software products and completing software projects?  Both delivering a software product and completing a software project have the same end product in mind – creating working software.  However, the approaches are significantly different.

A software project’s success is primarily measured by one concept: completing all desired functionality within time and budget.  The obvious benefit to a project-based mentality is that having a (reasonably) fixed cost and duration it makes it easier to budget money and resources for future projects.  However, as I wrote in a previous post, time and budget measurements should be secondary considerations to the actual business value delivered.  If the project mentality is taken too far, it can lead to a deliberate attempt to limit needed functionality if it is determined after scope is laid out.

A software product’s success is primarily determined by the goals of the software itself.  If, and only if, the software does what it was intended to, it was a success.  The benefit to this mentality is that the core mindset becomes delivering software that exceeds expectations.  Scope creep no longer becomes an issue – if functionality that is needed is found after the scoping process, the project is re-scoped.  The development team and business stakeholders can re-evaluate priorities based on business needs and current budget without needing to concern themselves with a pre-determined budget.  A drawback is that product-based teams tend to increase scope as time goes on, which makes budgeting and resource planning more difficult.  It is also more difficult to determine success criteria in a software product environment, since this will vary from project to project.

While there are definitely benefits to a project-based approach, I would like to see more of a product-based mentality among technology teams.  We as technologists need to be more concerned with the software functionality as a whole; not in the sense of “does it do what we were asked to build” but “does it do what it’s supposed to do from a business point of view”.  Of course, time and budget are important, otherwise the methods for determining which projects to pursue start losing their relevancy.  But a greater product-based approach would help bridge the gulf between business and IT.

Beware the Expert Fallacy

The fact that the relationship between business and technology is so contentious at times is due to behaviors on both the business and technology sides.  I’ve been writing mainly about the technology side primarily due to my familiarity with it, and not due to its relative importance.  If business and technology are to work together as one team, though, business leaders need to change too.

A friend of mine told me of a conversation they had with a co-worker that exemplifies a serious problem in the business community when it comes to technology.  In this scenario, person #1 (whom I’ll call "John") hired an outside firm to redesign a website, and person #2 (whom I’ll call "Jane") was looking at the results.

John: I’d like you to take a look at the design for our new web site.

Jane: I’d love to, our current site could use a new look.

John: Here it is.  I think they did a great job, didn’t they?

Jane: Yes they did.  I like the colors and it’s much easier to read than our old site.  I have a question, though.  This new design put a secondary product line under a different tab.  I know that’s technically correct, but are any of our users going to be able to find it there?

John: Don’t think about it too much.  This design was created by experts. [Emphasis added.]

I come across John’s attitude all too often.  Technologists are often experts in technology, but they too often know little about your particular business, or in this case, they know too little about marketing.  I wasn’t involved in the design process of this particular project, but I’m reasonably certain that the design team created a site using an existing color palette and the design specifications given to them, but didn’t give much thought into how people would want to use the site.  The result was a site organization that made sense to the business team but not to the end user.

I can tell you from personal experience that Jane’s observations are not unique.  The design team created a mediocre design, but why didn’t John say something?  Essentially, John put too much responsibility on the technology team to build a business product.  Technology can be confusing at times for anyone, so trusting the technology team to technology seems like a safe thing to do.  The problem is that while the design team knew the technology, they didn’t know the business.  In this case, no one, neither the design team nor the business team, was really focusing on the user’s experience.  Because the user’s experience was overlooked, user satisfaction will almost certainly suffer.

The solution is straightforward, though not simple: the business team must not view technology as its own area, but instead integrate the thoughts on technology into the thoughts about the business.  Any software that is created should involve as much communication as possible between the technology team, business team, and a representative sample of end users.  By working together, and not treating technologists as experts in anything that seems technology related, businesses can create software applications that are a pleasure to use.

Mixing jQuery Mobile and ASP.NET WebForms

I’m going to depart from my usual post subjects and get a little more technical.  I recently had an unpleasant experience trying to use jQuery Mobile with ASP.NET WebForms.  If you're thinking about doing the same, I recommend stopping right now and start using ASP.NET MVC.  To see why, read on.

To start, before I had gotten into jQuery Mobile, I had assumed that it was simply a slimmed-down version of jQuery built specifically for mobile devices.  I was wrong.  JQuery Mobile is built on top of traditional jQuery, and it is intended to simplify the styling of web sites to make them look and behave more like native mobile applications with as little development effort as possible.  For example, by only adding “data-role=’listview’” to a <ul> tag, you can specify that the <li> tags underneath show up in a typical mobile stacked list.  (You can check out the jQuery Mobile documentation to see an example.)

Anyway, I started with a typical login control on the first page, created some list pages, and created a couple of data entry pages.  It worked extremely well at first.  All of my page navigations loaded smoothly into the next page, and the user interface styling happened exactly as advertised.  Fantastic!  I was quite pleased at this point.

Then I added a “Logout” LinkButton.  It wouldn’t post back.  In order to get the page transitions to work, jQuery Mobile, when navigating between pages, loads the next page using JavaScript, then slides the new page into the view of the user.  JQuery Mobile was apparently doing its magic on all hyperlinks on the page, including my LinkButton.  A Google search gave me several suggestions on making it work, ranging from unbinding the control to adding an attribute “data-role=’none’” to the control.  None of which worked.  So I made the logout button redirect a page that logged the user out of the application.  That’s not how I would have liked to implement that functionality, but it’s better to have full functionality with a less-than-ideal implementation than an ideal implementation with a less-than-ideal set of features.

So then I started working on the data entry pages.  On one page, the user is required to enter a zip code, and a dropdown is populated with the matching counties.  Since I didn’t want to fight ViewState or turn it off altogether, I decided to use an UpdatePanel to refresh the values.  Changing the text of the zip code textbox didn’t cause a postback, but instead caused JavaScript errors.  The errors made no sense to me, so I went back to Google.  It didn’t take long (about 2 minutes, I think) for me to realize that UpdatePanels and jQuery Mobile really can’t work together.  Even if I had been able to get the JavaScript working, the CSS classes that jQuery Mobile would add to the page elements would be lost when an UpdatePanel is refreshed.  Sigh.  That wasn’t a huge deal – I turned off ViewState on the dropdown, bound it on every page postback, and wrote my own AJAX (using the ASP.NET AJAX stuff) to populate the dropdown.  I don’t like UpdatePanels anyway, so I wasn’t too heartbroken.

Then I went back to the list pages.  The navigation was working well, but I needed to give the users the ability to claim items off the list.  We wanted a certain look, so I used ImageButtons for the task.  Unfortunately, there were errors in the postback process, which made it difficult for the browser to send back the CommandName and CommandArgument from the ImageButton properly.  Ok, so time to do that manually too, using hidden fields to store what would have been the CommandName and CommandArgument that should have been returned by using the button.

Then I noticed that after I had logged out, I couldn’t log back in.  I did some digging, at it appeared that the AJAX load of the page confused ASP.NET into thinking that any postbacks after logging out should go to the logout page first.  So no matter what happened, the user would be logged out immediately after trying to log into the application a second time.  Then I noticed the same problem on some of my list pages – the browser would try to post back to the page in the URL, not the page on the screen.  At that point I gave up and removed the jQuery Mobile from the application.

JQuery Mobile seemed like a great tool with a lot of promise.  It did not mix with ASP.NET WebForms, however.  JQuery Mobile was built for a pure web environment, not a web environment with the goofy postback model that WebForms uses.  If you're looking to do mix the two, save yourself some time and frustration and skip one or the other.  The next time I do a mobile web project, I will definitely use ASP.NET MVC with jQuery Mobile.  The integration between the two should be much more natural.

Aligning business and IT strategy is wrong-headed

Up until recently, I’ve been talking (and thinking) about how to align IT and business strategies.  I’ve come to realize that this is the wrong approach for one simple reason: it implies that IT and the rest of the business have separate strategies.  What we need to do is make the strategies of the IT department and each business function the same strategy.  That won’t be easy, though, given the disparate goals of each group.  To get an idea why, let’s look at strategies and approaches that are common in the IT industry:

1. Do extremely high quality work, because software outages and security leaks reflects poorly on the team and can lead to long hours at unexpected times
2. Focus on creating methods that elicit the best requirements from the business stakeholder and minimize changes, such as Agile and Scrum
3. Create methods and tools that make technology-centric functions (like creating a new server or writing code) more efficient and less time-consuming

These strategies are not business-focused, they’re technology focused.  To see why, let’s examine them:

1. It is often the case that doing the absolute best work possible isn’t appropriate for a given situation.  Technologists often want to do more work than necessary up front to avoid problems which they believe reflect poorly on them.  (And yes, there’s a sense of pride emphasized in the wrong place, too.)
2. In most of my consulting experiences, the customer wants to keep the costs low as much as possible.  This is understandable to a point, but in order to cut costs, the requirements gathering process gets neglected, causing poor user experiences and costly rework.
3. I think it’s great that technologists are continually working to improve their craft.  It’s great that we’re able to deliver more functionality with less work because of these efforts.  But the focus on understanding and using software is coming at the expense of knowing and understanding business needs.

So how is aligning IT and business strategy wrong-headed?  Because it implies that IT and business strategies are different strategies that need to be brought together when we should be finding ways of making the two the same strategy.  IT and business need to work together in order to find the right balance between cost, security, efficiency, performance, and all other aspects of software development.  How would the above scenarios change with a different approach?

1. Create software whose quality reflects the need of the moment.  Single-use, non-mission-critical software doesn't need to be as robust as high-use, mission-critical applications.  Always have a cost/benefit analysis in mind when deciding how robust to make your application.
2. Instead of focusing on gathering the requirements from the business leader, we as technologists need to do a better job of gathering requirements from the actual end user.  Knowing your customer is critical to getting the user experience right the first time.
3. We need to make sure that we aren’t just focused on delivering better code, though, and be sure we’re focused on delivering better software.  To start, there should be as many user groups focused on software usage as there are on software writing.

So how do we make this happen?  Both business stakeholders and technologists are contributing to the problem by pushing their own agendas over the good for everyone.  Merely telling technologists that they need to be more business focused, or the business stakeholders that they need to be more tech-savvy, is more likely to elicit negative feedback than it is to solve any issues.  I like the idea of breaking up IT so it is no longer its own separate department.  In that case, we would still need a corporate IT department that would set company-level policies and procedures, but most of the work would be done by technology professionals within each department.  This way the people doing the actual technology implementation would be more familiar with their subject matter, reducing communication problems and the technology-first mentality.  Also, we can more easily make sure that the technology implementation team has the business strategies and priorities foremost in their minds.