Chris Doig's Blog

Thoughts on Managing In-House Software Development Projects

Managing in-house software development is very different from managing software developed for resale. To be sure, there are many things in common, but there are some very significant differences. This article looks at some of the problems unique to managing in-house software, and offers some thoughts on solutions.

The aim of any IT department is to "solve the customer's problem", either by buying software or developing in-house. Usually it's better to buy prepackaged software because this reduces risk and implementation time, and can supply prepackaged expertise you don't have. But large software purchases require an experienced project manager, or you can be the victim of enthusiastic selling. On the other had, there are times when you will undertake in-house software development, for example

• No commercial software is close enough to meeting the requirements, e.g. projects that handle departmental workflows. These are typically small, highly specific projects and there are many of them. Typically they are implemented via a web browser or with the Notes client.
• The project is so small that it would take longer to find and evaluate a commercial product than it would take to develop it in house. This presupposes you have a good developer, good libraries that the developer uses (or a similar project that he is well acquainted with and can copy from) and a rapid application development environment.
• You have developers with a good track record of successful projects, very good understanding of the problem, and of the technology used. Remember that engineering lore states that when a developer gets to the end of a project, then they know how it should have been done.
• The commercial package is ridiculously expensive. I once had a case of implementing a certain feature using Lotus Notes. A commercial company quoted me the equivalent of 3000 hours of developer time for a site license. The developer took some existing code and had a similar solution working in less than 100 hours.

The aim of in-house software development is to produce successful projects in a short a time using as few resources as possible. When it comes to measuring the success of software projects all sorts of numbers are thrown about. They range from studies that claim only 25% of software projects succeed, to one IT director who told me that every single one of his software projects succeeded. So what defines success? Is success meeting requirements, or is it more than this? I would argue that a successful software project has all of the following attributes to some degree:

• The project must be put into production, must meet customer requirements, and must be continuously used by employees.
• The software must be easy to use, and not require extensive training (Reasonable training is acceptable.)
• The software must not frustrate users. Users must find relatively few bugs, and they must not loose data.
• The software must be capable of being extended. When employees start using in-house developed software, it doesn't take long for enhancement requests to arrive. If the project is difficult to maintain and extend, there may be a serious danger of it being only a short term success. So the internal design of a software project is very important.

Now that we have a successful project defined, we need to look at the four elements that go into making a software project successful: Scope, Resources, Time and Quality. Bear in mind that you can never control more than 3 of the 4 elements. Sometimes you can't even control that many!

Scope

A big problem for -in-house software development is project scope. What exactly is going to be done? Project scope should be defined by a Requirements Specification (more in a future article), but often is not. This leaves the way open to scope creep, especially from inexperienced developers who are eager to please. Mr Customer, would you like this feature? It will take no time to add. Usually the answer is yes, and most of the time the developer severely under estimates how long it takes to add. Scope creep is a perennial problem for all software development.

Probably the best way of dealing with scope creep is to push new requests into future releases. You avoid saying no to the customer, yet you are still able to meet the original delivery dates. Frequent small releases allow the customer to steer development as it unfolds.

Resources

When considering resources thoughts naturally turn to outsourcing. While there certainly are projects that can be successfully outsourced, I think that it can easily be a disaster. The main problem is communication between the developers and the customer. It is very important to have the project lead developer talk face to face with the customer. By all means have the developer's manager in meetings etc, but the lead developer must take final responsibility for communication - the manager acts only a facilitator. One on one communication and an immersion the corporate culture give in-house developers a much better chance of developing a successful project.

While offshore developer teams loose out on face to face communication, they also loose out on phone conversations. Usually there is a language or accent barrier, and often a time zone barrier as well. While offshore teams can meet requirements specifications, you can still have a project failure on your hands. I tend to think that offshore development works best where there is a well defined project specification, and continuous communication is not really required.

Whether a team is local or off shore, small, tightly focused teams of excellent coders will be orders of magnitude more successful than larger groups of average coders. Always bear in mind that when you add new people to a project, you increase the communications overhead; there comes a point where adding new coders can actually slow the project down.

In any IT department there tend to be a few large software projects and many small ones. One problem that occurs when you have individual developers working on small projects is they can go off on tangents. These tangents range from a non-optimum architecture though to spending significant time on something that is not project related at all. Here is a lesson I learned the hard way about software architecture...

Many years ago I was managing a developer who was writing software to capture measurement log output of production line machinery (they were making small metal objects on the production line). This software tracked tolerance trends, and if they drifted too far in one direction would shut the line down and prevent scrap from being manufactured. In those days disk space was expensive (I did say this was a long time ago!) and the developer came up with a complicated relational structure designed to reduce disk space. The developer overhead of maintaining this structure was so high that it doomed the project. Looking back I learned several lessons that apply to in-house developed applications:

• As a project manager pay close attention until you are satisfied with the software architecture and the work done.
• Hardware is much cheaper than software development. Always design to reduce software development costs, not hardware. (Also you usually should design for easy of maintenance, and not speed of the code).
• When logging data, you don't need (or even want) a relational structure. The overhead is not worth it.
• The simplest solution is usually the best.

This experience taught me why it is vital that as a project manager you know and understand your developers. If you understand the architecture and can suggest alternatives, you are ahead in the game.

Another tangent is having a developer get side tracked by something that interests them, and loose focus on the required project. While it is important to maintain a developer's interest in a project, discipline is also required. If this gets too far out of hand, the only option may be to let them go, not a pleasant thing to do.

Finally, while priorities do change, as far as possible avoid rapid switching between projects. It seriously reduces a developers effectiveness.

Time

Usually time is the toughest element to manage. One technique is to set frequent, small milestones. This avoids the problem of having a project at 95% done for weeks or months. Then when reality strikes at the last minute it drops back to 50% done.

Another technique is small, frequent releases to production. This  leads to a much higher customer satisfaction than few larger releases. And they allow the customer to better steer the direction of the project.

Always remember to build in contingencies for those unknowns that creep into any project. For example, never budget on a programmer spending more that 75% of their time on their "top" project.

Quality

How well does the software meets the customer's real needs? The problem is a customer sometimes can't articulate those needs very well. Also things come up in a project that nobody can possibly predict at the start.

Good (user) software requires a good interface. Remember that developers are code artists, not graphics artists. A developer can spend weeks tweaking a user interface (UI), and still end up something totally unusable like with red text on a blue background! Contract with an experienced a graphics artist / UI specialist. It is often cheaper, especially when you take the reduced training costs into account. While code reuse is very important, how many people consider UI reuse? Develop a standard user interface theme for all your projects, and free your developers from designing a new UI for every project. Not only do you save developer costs, but you reduce end user training costs as well.

The rate at which the customer finds bugs in production should be minimized. Of course software should include error handling, and part of that error handling must notify the developers when something goes wrong. Log the error to the database, and send an email to the groups responsible for the software. (Note: the error handling in production databases should always use a group, and not names. When people leave the company it is much easier to find all the groups they belonged to and update the names. If names are recorded in the database, these names often don't get updated, and the errors are missed.) When developers are notified by email they will often fix the problem before the end user even notices it. Also, because they don't like being pestered by error notifications, they try to solve the problems that cause the errors in the first place, in effect climbing the CMMI ladder. This is exactly what Microsoft and others are doing when a program crashes and Windows asks if you want to send an error report.

Reducing project risk

There are many articles on reducing project risk on the web, so let me simply summarize a few simple rules of thumb here

• Minimize the unknowns. Go though the project looking for things you don't know, and get rid of them. For example, reduce the number of new technologies, and match developers to the technologies proposed.
• Define the scope of the project well. Minimize scope creep. When new requirements push the scope, put them in a future version.
• Make frequent, small production releases so the customer can steer the development direction. For Lotus Notes development, this could easily be as often as once a week.
• Simplify. To paraphrase Occams Razor: when faced with several similar choices, choose the simplest. Why? Well, if you made a wrong choice and you chose the simplest, it means you throw away the least amount of work.
• Reduce coupling between parts of the project. It is a lot easier to make several little widgets that one big widget. It is interesting to see Microsoft go against this with Exchange 2003 which requires active directory. Linking Active Directory to Exchange has made it so much harder to deploy that two years after the introduction, less than 50% of the customer base has upgraded.

A Few Final Thoughts

We all know the story of the carrot and the stick. Simply put, you are in a much better position when employees want to use the new system, rather than forcing them to use it. And you can reach that heavenly state of project manager bliss by following one simple rule: You must deliver more from the new project that the old way cost. Whether the old way was a manual method or simply an outdated system, the new way must have a reward/cost ration greater than one.

When rolling out a new project, identify the "opinion leaders" amongst the users. If you can get them to endorse the project you are well on your way to a success. Bear in mind that they are not the actual customer, but are members of the user community. Although conventional wisdom says "start with your least critical users", it can be a good idea to roll it out to the opinion leaders first.

• You have the time to hand hold them while bugs are worked out.
• They feel important, part of the team. They build in a commitment to the success of the project.
• When others see the opinion leaders endorsing the project, they tend to follow. The first time I did this, I even had end users asking when they would be upgraded.

Never forget end user training aiming to train at least 80% of your users. It can make or break a project. Of course having decent user help is also important (user help techniques are the subject of a future article).

I hope these thoughts and real world anecdotes give you some good ideas and help you make a success managing in-house software development. projects.

Acknowledgement: I am indebted to Ray Hoving of Ray Hoving Associates for some of the ideas in this article.