A software product is inseparable from the underlying software process which resulted in creating it. Though bugs are technical in nature, it is the software development process that has the most impact on bugs. To illustrate this, see what happens in a new software project, when Raphus Cucullatus, nicknamed Dodo, is given the responsibility of managing the project.
All is well: Dodo kick-starts the new project with a team of experienced developers. Dodo thinks that software can be produced under tight deadlines, and creates a project plan. The customer is satisfied with the plan.
Monitoring based on effort spent: Dodo monitors the project by effort spent rather than the progress made. The customer is happy to see the considerable amount of effort spent in the project.
Focus on functionality completion: Dodo feels quite happy to closely follow the functionality completion. He accepts all features that the customer requests. The customer is glad that the software has a good set of features.
NFRs become a problem: However, during the last few months, when approaching milestones, and when integration and systems testing are done, all kinds of non-functional issues (performance, stability, etc.) arise, and bug counts shoot up. Dodo tells the customers that everything is going well and that the software will be shipped in time.
Increasing pressure: Dodo is unhappy with the rising bug counts, and increases pressure on the development and testing teams. However, no progress seems to happen, and bug counts keep going up no matter how many bugs are fixed.
Adding more people: When it becomes doubtful that the project will meet the release deadline, he tells the customer that the team is facing a few “minor” issues. He pacifies the customers, saying that more programmers have been added to the project. The situation only worsens.
Process — what process? When the house is burning, there is no place for etiquette and manners, so he removes unnecessary “progress bottlenecks” and “niceties” such as peer reviews and impact analysis for change requests. He is a practical man, and focuses on “shipping the software” within the sacred milestone date.
Delay is inevitable: When it is clear that the software is unstable, and milestone dates cannot be met, he tells the customer that the software will be ready for release after a delay of a few months. The customer is very unhappy.
Ship on an “as-is basis”: When it dawns on him that he cannot postpone the release any more, Dodo tells the customer that there are “minor glitches”, and the software is ready for release. The customer is frustrated with the quality of the shipped software, and escalates the issue to the management.
Fire the programmers: The management forms a “task force” to conduct a “root-cause-analysis” for the whole problem. The team finds that clearly it is the poor quality of technical people in the team that is the cause of all the problems (to quote from the report, “The quality of software is as good as the quality of its people”). The team further suggests adopting agile methods, and using “the latest technologies” to avoid repeating this mistake in the future.
The management fires the architect, the team leads and key developers; further, it decides to adopt the “agile methodology” for all new projects, and suggests using only Java and .NET technologies in the future.
Promotion: Dodo has gained experience in managing software projects, and the management finds that he has special skills in “crisis management”! He gets promoted and assigned a bigger team with a new and larger software project to manage.
It starts all over again: Ever optimistic, Dodo prepares a “reasonable” estimate based on his “past experiences” in managing projects…
This story is not to disrespect any managers. Also, I am not cynical; what I described is reality. Optimistic estimation of effort for software development, and poor project management, are the two most important factors that result in buggy software (poor “craftsmanship” is another important cause, but that’s a topic for another column). To put it simply, when software is developed under “undue pressure” and “getting the work done” becomes the priority, quality is naturally compromised — and that is the major cause for software bugs.
