Tech Debt visits the City of Codeville
In the heart of a vast open area, in a hidden corner of the world, there was a growing town named Codeville. In its early days, the town was a marvel, built swiftly with vision and dreams. Its roads were straightforward, houses well-constructed, and all its systems operated in harmony. Travellers from far and wide came to admire its brilliance.
However, as the city grew, the founders started taking shortcuts. Instead of repairing the old bridge, they would hastily add planks just to keep it standing. Instead of planning new streets, they built them wherever they found space, leading to a maze of confusing alleys. Over time, these shortcuts accumulated and became part of the town’s architecture, referred to by the locals as the Debt Districts.
Soon, the Debt Districts started showing their strain. The old bridge creaked worryingly, and the streets became congested. The newer parts of the city, built atop the old, inherited its vulnerabilities. Cracks began to show, and systems failed. The once-prosperous town was now a shadow of its former self.
Then came the day when the town's main square, a beautiful area once loved by all, collapsed due to a weak foundation. The calamity served as a wake-up call for the residents of Codeville.
With a united resolve, the city decided to address their growing "debt." They brought in architects and planners, who advised them to take a step back, repair the foundations, and restructure the city. The process was slow and often painful, as they had to undo and redo much of what was hastily built. But with time, Codeville started finding its former glory.
Introduction: A tale of Codeville
Much like the rebuilding journey of Codeville, software development often encounters hurdles. These hurdles, often called 'Technical Debt', can jeopardise the future of projects and the businesses they support. Through this story, let's journey together to understand what technical debt is, why it matters, and how to manage it effectively.
The birth of 'Technical Debt'
In 1992, Ward Cunningham compared taking shortcuts in software to building up financial debt. In his paper on The WyCash portfolio management system, he highlighted the core challenge facing developers: "Every minute spent on not-quite-right code counts as interest on that debt." The longer you delay addressing this debt, the more costly it becomes.
Imagine building a bridge in haste. While it may serve its immediate purpose, the flaws in its construction will, in time, require more repair effort than if it had been built correctly from the start.
In software development, Technical Debt refers to the consequences of quick solutions over optimal ones, leading to software that over time becomes challenging and expensive to update or modify.
Tech Debt example
Consider a web application built in 2000, using what was best and popular at that time, that never got upgraded:
Frontend: Built using plain HTML, CSS, and JavaScript without any frontend framework.
Backend: The application's logic was crafted using PHP 5.3, without leveraging any modern MVC frameworks like Laravel, Symfony.
Database: Data was stored using MySQL 5.1.
While the application might still function today, opting not to upgrade, at the earliest opportunity possible, incurs certain costs, naming only few:
User experience (UX): The plain JavaScript and limited AJAX capabilities meant the site was less interactive. With the rise of single-page applications using Angular, React, Vue.js or Blazor, users came to expect a seamless browsing experience.
Performance issues: Newer versions of the web framework introduce optimizations, making applications run faster and more responsively.
Security vulnerabilities: The older version of the framework and database had known vulnerabilities, thereby posing risks to your application and organisation.
Maintenance overhead: Developers face challenges maintaining and understanding dated code. New team members may be less acquainted with the old version, leading to extended training periods. Some features or plugins, standard with newer frameworks, might necessitate custom development in your older version.
System limitations: New modern third-party tools and plugins might not always be compatible with the older tech stack, limiting potential enhancements.
Missed opportunities: With a contemporary database and framework, implementing features like personalised user recommendations based on browsing history, or effective logging and auditing for compliance, would be straightforward. Yet, with such an outdated system, these either run too slowly, are too resource-intensive, or are virtually impossible.
Expensive: Some old software necessitate licenses are typically pricier than other open-source alternatives. Nowadays, using platforms like Docker can offer more cost-effective hosting solutions.
An application built two decades ago isn't inherently obsolete. With consistent maintenance and timely upgrades, such software can remain relevant and efficient in today's tech landscape. The accumulated knowledge and expertise from its long history can be its greatest strength. However, if neglected and left to age without attention, its chances, if any, of retaining relevance dramatically decrease.
Why is it important?
While developers might feel the pinch of tech debt the most, its implications are far-reaching:
Business operations: Delayed updates can affect business operations, leading to loss of revenue and customer trust.
Team morale: Constantly firefighting issues reduces team morale and hampers innovation.
Customers: Inefficient systems can lead to a poor user experience, driving customers away.
Understanding technical debt matters for two primary reasons: its domain-specific reference and its personal nature. While the metaphor might not resonate with every business, its consequences - like potholes that evolve into gaping road hazards - affect entire communities. Think of the repercussions: damaged vehicles, raised insurance premiums, and worst of all, tragic accidents.
Why does Technical Debt accumulate?
Three decades on, the tech world continues wrestling with technical debt, even in the face of countless resources advocating for clean code and best practices. Why does this challenge persist? The answer lies in understanding the dynamics at play from both top-down and bottom-up perspectives.
Top-Down view: The leadership disconnect
Most organizations exist to grow and profit. Their leaders hire various experts to drive this growth. For most companies, the development team's output plays a crucial role. However, a disconnect often arises between the ground realities and the leadership's understanding.
Distorted communications: In hierarchical organisations, the flow of information from one level to another can sometimes undergo subtle changes. A team might communicate certain challenges they are facing, but as this information is relayed upwards through various levels, its essence might get modified due to a variety of reasons, such as unintentional over-simplifications or the nuances getting lost in translation. Consequently, by the time the message reaches the top tier, it might not fully capture the initial concerns raised at the ground level.
Avoidance and consequence: When challenges aren't communicated effectively, they're likely to be ignored or sidelined. This burying of issues can lead to significant future problems. By the time the depth of the issue is realised, it may be too late for a simple fix, leading to a ripple effect of complications.
Unified vision: Successful software projects demand cohesion between business objectives and technical execution. Without clear vision, transparency and shared understanding across levels, discrepancies arise, often leading to avoidable technical debt.
True leadership requires transparency. Recognizing and addressing issues, rather than masking them, is the cornerstone of successful software engineering projects.
Bottom-Up view: On-the-ground challenges
The drive to achieve rapid growth and deliver quick results often pushes organisations to compromise on code quality. Combine this with evolving technologies, team turnovers, the inevitable knowledge gaps that arise, and maintaining older systems becomes a daunting task.
Key Challenges:
Rapid delivery over quality: The push for speed can overshadow the importance of quality.
Evolution and turnover: Technologies evolve, and with team members coming and going, knowledge continuity is disrupted.
Lack of oversight and expertise: Without regular reviews, mentoring, and a focus on skill development, code quality can suffer.
Communication barriers: Even when development managers raise red flags, these alarms are sometimes ignored or deferred by higher-ups.
Drawing a parallel with our town analogy, businesses might focus on constructing and monetizing houses, neglecting community essentials like schools or hospitals. A town without these facilities prioritises profit over the well-being of its inhabitants. Similarly, in the software world, neglecting foundational quality can lead to monumental challenges down the line.
It's crucial to remember that no structure, be it a building or software, is meant to last indefinitely without the need for adjustments.
Google's Technical Debt case study
When seeking tangible examples of why and how technical debt accumulates, industry giants, often referred to as the FAANG companies, serve as invaluable case studies. Their vast and intricate software systems, combined with the necessity to constantly innovate, make them prime subjects for understanding the complexities of technical debt.
One standout example is Google. In 2023, they unveiled a comprehensive paper detailing their approach to defining, measuring, and managing technical debt. This can be accessed at Google's Technical Debt Paper. The dedication Google exhibited by investing four years to thoroughly comprehend and address their technical debt underscores the significance and complexity of the issue. This deep dive into Google's strategies provides a segue into a broader discussion on the inevitability of technical debt in software development and the need for proactive management strategies.
This realisation brings us to the question of management - acknowledging that while technical debt cannot be entirely prevented, it must be meticulously managed to minimize its impact.
Addressing the elephant in the room
The best way to address technical debt is not have it in the first place.
The ideal strategy to manage technical debt is to anticipate it and incorporate proactive measures into the development process. It's about treating software as a continuously evolving entity, requiring regular maintenance and foresight. Similar to routine vehicle services ensuring its efficiency and safety, regular software reviews and updates are essential for its health and sustainability.
Here are some actionable steps for managing tech debt effectively:
Emphasize communication: Effective communication is paramount. Encourage developers to articulate issues in a way that resonates with stakeholders, ensuring that technical and business sides are aligned on tech debt implications.
Focus on user needs: Focus on delivering functionality that aligns with actual user needs rather than perceived business requirements. Validate ideas through user research and prototyping.
Tackle work effectively: Ensure that work items have clear definitions, with well defined requirements, user stories, acceptance criteria, and test scenarios. Agile methodologies can be particularly effective in managing tech debt by incorporating regular reflection and improvement into the development cycle. Apply refactoring as an ongoing practice to adapt the codebase proactively. Encourage collaborative practices such as pair programming and code reviews, and consider mob programming for complex issues.
Make collaborative decisions: Involve the whole team in architectural decisions to foster ownership and ensure that designs reflect on-the-ground realities. Ensure that all decisions are made publicly and with transparency. This approach helps future team members understand the rationale behind past decisions, which may not seem optimal in hindsight but were the best given the context at the time.
Establish clear guidelines: Simply asking for clean code is insufficient. No developer intentionally crafts poor code. Provide your developers with explicit processes, coding standards and best practices to align individual efforts with your project's objectives and industry benchmarks.
Leverage tools effectively: Make use of tools like SonarQube, Klockwork, SQUORE, or built-in metrics and analysis tools in IDEs, to monitor and address technical debt in real-time. These tools can provide insights that pre-empt debt accumulation.
Prioritize regular reviews: Implement a systematic approach to review your codebase and architectural design at regular intervals. This ensures ongoing quality control and identifies opportunities for refinement. Encourage teams to adopt a proactive stance on technical oversight, integrating these evaluations into the development lifecycle. Additionally, consider periodic external audits to provide a third-party perspective on adherence to industry standards and best practices.
Invest in training and leadership development: Regular training is key to avoiding technical debt. Teach your team good coding habits and how to design systems that are easy to maintain, following the best coding and design practices. Look for the leaders in your team - the ones who are really good but also down-to-earth, smart and passionate - and help them lead the way. They can show everyone how to do things better, making sure that the team's work gets better over time.
Integrate technical debt management: Allocating a specific portion of development time to manage technical debt, typically between 10-20%, could require meticulous planning. It’s crucial to determine whether this is a temporary or a permanent commitment within each sprint, as well as how to measure this allocation effectively. With time also set aside for learning (10%) and sprint meetings (10%), the remaining time for core development can significantly diminish. Integrating technical debt resolution as a standard part of development tasks - including as enabler features or work items - helps prevent it from being overlooked or treated as an afterthought. Should the time allocated for learning and innovation prove insufficient for sparking creativity, it could be repurposed to address technical debt, thereby ensuring continuous improvement.
A shift in perspective
While 'Technical Debt' has been the go-to term, industry leaders like Kent Beck, the creator of Extreme Programming and an Agile Manifesto signatory, advocate for a change in narrative. Kent now speaks of Friction instead of technical debt when addressing business stakeholders.
Yet, perhaps we shouldn't be hung up on labels at all. Instead, our attention should be devoted to the actions we take - prioritizing regular maintenance and ongoing refinement; service your software the same way you service your car.
The label 'Tech Debt' often becomes a convenient excuse, a procrastinator's promise: "We'll tag it with a TODO and revisit it later." That "later" remains elusive, allowing developers and stakeholders to deflect responsibility. It's less about assigning blame and more about understanding it as an 'enabling work,' acknowledging the past without being hindered by it.
What Technical Debt isn't
It is important to highlight that not everything should be classed as tech debt. Just because tomorrow a new framework version is release, it does not make your code base old and obsolete.
Technical Debt isn't:
Simpler code in newer versions: Just because newer versions of a language or framework allow for more concise or simplified code, it doesn't automatically make the previous verbose version technical debt.
Initial architectural decisions: An architectural choice that was appropriate and fit the purpose when the software was initially built isn't technical debt. If, with time, the original architecture doesn't align with emerging requirements, it's more about the system evolving and less about it being a debt. Adapting to new needs can be viewed as a feature enhancement or system evolution rather than a debt. Think of it as a house renovation to accommodate a growing family rather than a fault in the original construction.
Changing best practices: The tech industry is dynamic, with best practices evolving over time. A practice or tool that was considered state-of-the-art a few years ago might be less favoured today. This evolution doesn't retroactively label the previous practice as technical debt.
Code you disagree with: Personal preferences or disagreements with coding practices don't equate to technical debt. Just because you might've approached a problem differently doesn't mean the existing solution is inherently flawed or in debt.
Lack of latest features: Not having the newest features or functionalities that contemporary software might have isn't necessarily technical debt. If the software serves its intended purpose efficiently and effectively, it's doing its job.
Conclusion: The sustainable road ahead
The importance of discussing and addressing tech debt lies in its foundational role in software development. Just as our world leans towards sustainability in all aspects, software development should be no different.
The tale of Codeville serves as a stark reminder of the repercussions of neglect. It's not just about building; it's about building the right thing in the right way, reviewing, and adapting. It's a lesson every stakeholder, from developers to business heads, must internalise.
Good software is built with passion, when everyone, from top to bottom, cares about what and how they build products and services.
Do not make space for Tech Debt, do not allow it to enter the Tech Debt lane, and certainly do not allow it to enter the town. Make your town or city a success story that you can tell the entire world about it.
Extras - Codeville, a pun-intended explanation
At the heart of our discussion lies a simple play on words that encapsulates our message: 'Ville' in French means 'city' - bustling hubs of activity, growth, and innovation. Yet, remove just one 'l' and you're left with 'vile', an adjective that means horrid and unpleasant in English. Through this pun, we arrive at the core appeal of this article: to implore not just coders, but everyone involved in the creation and maintenance of software, to steer away from the 'vile' - from creating code that becomes a bane rather than a boon.