AI Makes Tech Debt More Expensive

There is an emerging belief that AI will make tech debt less relevant. Since it’s getting easier to write code, and easier to clean up code, wouldn’t it make sense that the typical company can handle a little more debt?

The opposite is true – AI has significantly increased the real cost of carrying tech debt. The key impact to notice is that generative AI dramatically widens the gap in velocity between ‘low-debt’ coding and ‘high-debt’ coding.

Companies with relatively young, high-quality codebases benefit the most from generative AI tools, while companies with gnarly, legacy codebases will struggle to adopt them. In other words, the penalty for having a ‘high-debt’ codebase is now larger than ever.

GenAI can’t handle High Complexity

If you’ve tried tools like Cursor or Aider for professional coding, you know that their performance is highly sensitive to the complexity of the code you’re working on. They provide a dramatic speedup when applying pre-existing patterns, and when making use of existing interfaces or module relationships. However, in ‘high-debt’ environments with subtle control flow, long-range dependencies, and unexpected patterns, they struggle to generate a useful response.

Not only does a complex codebase make it harder for the model to generate a coherent response, it also makes it harder for the developer to formulate a coherent request.

This experience has lead most developers to “watch and wait” for the tools to improve until they can handle ‘production-level’ complexity in software. While genAI is improving dramatically in some sense (multi-modal, faster, smaller, cheaper), this barrier has so far proven more stubborn, with o1 seeing relatively low adoption and multi-agent frameworks failing to significantly scale reasoning capabilities.

Make Your Tools Work for You

Instead of trying to force genAI tools to tackle thorny issues in legacy codebases, human experts should do the work of refactoring legacy code until genAI can operate on it smoothly. When direct refactoring is still too risky, teams can adjust their development strategy with approaches like strangler fig to build greenfield modules which can benefit immediately from genAI tooling.

This implies a shift to a higher level of abstraction for a modern development team. A product should be owned by a lean team of experts, focused primarily on the architecture of their code rather than the implementation details.

AI-Friendly Approaches

In essence, the goal should be to unblock your AI tools as much as possible. One reliable way to do this is to spend time breaking your system down into cohesive and coherent modules, each interacting through an explicit interface.

A useful heuristic for evaluating a set of modules is to use them to explain your core features and data flows in natural language. You should be able to concisely describe current and planned functionality. You might also want to set up visibility and enforcement to make progress toward your desired architecture.

A modern development team should work to maintain and evolve a system of well-defined modules which robustly model the needs of their domain. Day-to-day feature work should then be done on top of this foundation with maximum leverage from generative AI tooling.

‍

Quality Leads to Speed

It’s now more valuable than ever to have a high-quality codebase. Investment into modular architecture lets teams take full advantage of generative AI tools, achieving rapid feature development while simultaneously building higher quality software.