Technical Debt Ceiling
The US Congress and the White House are engaging in a historic debate over the federal debt ceiling. All problems tend to be accumulated over time; the longtime negligence and lack of action because of political inconvenience are the reasons for the current Federal debt crisis. Software development shares a remarkably similar pattern.
US National Debt
Technical Debt is like government debt: it carries an interest cost, comes from short term rapid enhancements without long term planning, and is ultimately caused by the lack of a balanced approach between addressing urgent issues vs. important issues.
Some level of technical debt can be beneficial to a startup company: it actually enables an accelerated development timeframe, reduces the occurrence of over-engineering, and generates immediate ROI in rapid delivery of software values to clients --- after all, the market needed those features yesterday.
However, once the technical debt is accumulated to a certain level, it will lead to performance issues and reduce the capability of adding new features, and it will eventually hinder the scalability, reliability and flexibility of the entire software system. No great structure can be built on an unstable foundation.
In order to have a balanced technical debt, or achieve technical surplus, we should first be able to measure it. I’d like to introduce a technical debt measurement standard within the Scrum framework --- A software system’s technical debt can be measured by the user story points that could be created, but are not being worked on due to the time spent serving existing technical debt through code refactoring. Gross Development Product (GDP) should be the annual user story points output for a software company. So the technical debt ratio should be calculated by: Technical debt ratio = User Story Points used to serve technical debt / GDP.
If a software team has to devote the effort of the equivalent of 2000 user story points to refactor existing codes, then the technical debt of this software system is 2000. If the same development team can produce 10,000 user story points per year, the technical debt to annual GDP ratio for this company is 20%, which could be at a sustainable level depending on the industry the company is in. Every software company needs to establish its comfortable technical debt ratio ceiling, maybe 10% of the GDP or 30% of the GDP, it can be different from industry to industry, but each software company must reach consensus on the technical debt ratio ceiling, and have a plan to reduce technical debt when approaching this ceiling.
A development team should always use evolutionary product design in software development. Continuous refactoring is always recommended over a situation that requires stopping all feature development and only working on technical debt. Through this continuous effort, technical debt can be reduced without shutting down all new feature development.
The best way to avoid technical debt, and even sustain a technical surplus, is to create and maintain a sound architectural pattern in the architectural design phase. Coders following a solid software design pattern can avoid technical debt in the first place, even under the heavy pressure of deadlines.
Scrum Methodology not only provides guidance for us in building new features, it also provides a similar principle to address existing technical debt, which is through incremental iterations, weaving architectural improvement inside feature building. The best way to avoid crisis is to never put us into one in the first place. If we do find ourselves in a crisis, we should try to use this as an opportunity. Always use the crisis to bring greater good to the software system, never let any crisis go to waste. The solutions created to respond to the crisis should not only solve those immediate issues, but also address the underlying long-term architectural deficit, rather than kicking the can down the road, like our members of Congress.
Economies boom and bust; software platforms and software companies rise and fall. A repeating scenario is that when a legacy software system becomes so obsolete (too much debt and baggage) that it’s easier for the market to find a new company to build a new system to replace the legacy software and along with it, the old company.
Therefore, the management of technical debt for a software company should be part of the company’s strategic thinking. It’s time for your company’s Barack Obama and John Boehner to work together on this.
By Isaac Shi
July 17, 2011
