In my role as Chief Technology Officer of an independent online agency, I have oftentimes found that one of the most difficult parts of my job is estimating how long it will take us to develop custom software. Certainly our clients expect, and deserve, to have some idea of when they can expect their e-commerce system, content management system, or online recruiting platform to be ready for launch.
The software development industry is notorious for late delivery. Even the great software companies have this problem (think about how many times Microsoft has delayed new releases of Windows for example). And, after a new product is launched, it's always buggy. This is true for nearly any program - from the simplest contact management application to an operating system.
A major part of the reason for this is that it's awfully difficult to estimate with any precision how long it will take to build something that hasn't been built the same way before. Every software project is different in its way, and unless you're going to exactly imitate another system that's been used for years, you really can't know how long it will take to get a project to the point where everybody's relatively happy with it. Joel Spolsky believes it takes 10 years to get software right.
Scott Rosenberg wrote a book that's very high on my to-read list entitled Dreaming in Code: Two Dozen Programmers, Three Years, 4,732 Bugs, and One Quest for Transcendent Software. The book tells the story of a major open source software project that was started roughly four years ago by some well known and well regarded programmers to write a new kind of open source Personal Information Management (PIM) tool. Salon.com has an interview with Scott that turned me onto this (I ready about the book elsewhere already, but this interview made me want to read it). Some of the key takeaways:
- PIM systems have been built many times before, and yet this team of "stud" programmers still found it impossible to estimate how long it would take
- It's not just about how long it will take to build what you want, but also how do you know when it's done? You can always tweak and improve what you're doing. At some point you have to say "this is good enough."
- Any software project that is trying to do ANYTHING new is impossible to estimate accurately
- Given the choice, programmers will always prefer to write new code than to use somebody else's code, even if somebody else's code has been used by many people and is therefore much more thoroughly tested
- It is likely that it will be faster for the programmer to code his/her own stuff than to use somebody else's stuff
- However, the new code will also be a lot buggier and difficult to maintain longer term
- The best way to create software is to have just one person do it. As soon as you start adding multiple programmers, the complexity of software development rises exponentially making managing a software project like "herding cats".
- The state of software development has not radically improved in the last 20 years. It hasn't really gotten easier.