Software-intensive systems require continuous architecture refactoring in response to the company’s continuously evolving business strategy.
Architecture provides the linchpin between business strategy and technology strategy. Previously, we’ve explored how deep the relationship is between architecture choices and the available business strategy options. As the world is constantly evolving, we need to evolve with it. This means not only adjusting our business strategy to changes but also, by extension, changing the architecture of our offerings and platform.
Especially leaders with a background in mechanics or electronics often fail to understand the continuous nature of digital technologies. For atoms-based products, the typical process consists of a product development project, followed by a start of production where all changes to the product are stopped to effectively scale production. There may be smaller follow-up projects to address reliability issues or drive down cost by reengineering expensive parts, but the default state is “don’t touch.”
Many seek to treat software in the same way, which is a challenge. We often use external components, ranging from operating systems to databases and from open-source packages to highly specialized commercial extensions, that are subject to continuous evolution. We need to incorporate these new versions when they become available to stay compatible with other systems that our offerings interact with, to ensure safety and (cyber)security as well as from a branding perspective where we want to be seen to run in the front of the pack.
In addition, the feedback from the field, through our sales staff and customer support, as well as the automated data collection in our products, provides a constant stream of requests for changes, updates and extensions. Often starting with modifications to deal with quality issues, companies rapidly move to more frequent releases that also include new functionality.
Such frequent releases require changes to the architecture. This constant evolution of the architecture is also referred to as refactoring and presented as the solution part of the architecture technical debt problem. Although technical debt is often viewed as the shortcuts taken by architects and engineers to rapidly solve a problem, one of its primary sources is simply domain creep: the problem we initially solved and designed the architecture for has evolved and changed. We need to respond to this by evolving and refactoring the architecture of our offerings.
Again, for those with backgrounds outside of the digital realm, architecture refactoring may easily seem an infrequent, special event. In mechanics and electronics, these refactorings are often caused by some supplier sunsetting a component, forcing us to upgrade or switch to another supplier, requiring architecture refactoring. In software-intensive systems, however, there’s a constant flow of architecture technical debt items requiring architecture refactoring.
Our empirical research on the topic (including work by multiple PhD students and postdocs, covering well over a decade) has resulted in many learnings, of which I’ll discuss three here. First, we will do architecture refactoring whether we like it or not. Companies that fail to continuously invest in architecture refactoring tend to experience increasing debt levels in their systems, resulting in less productive development, high quality assurance cost and high-profile failures in the field. When productivity and quality become real issues, leadership decides that something needs to be done, which boils down to either a major overhaul of the existing architecture or a fresh start by building a new platform. Either approach means investing major resources into what, in effect, is architecture refactoring. In practice, it’s much better to allocate 10-25 percent of R&D resources to architecture refactoring continuously and avoid the really bad times.
Second, although the general belief is that software ages over time, the fact is that this is a choice, not an inevitability. With sufficient investment in architecture refactoring, the software can stay up to par for decades and as long as there’s a business need for it. However, it does require continuous investment in architecture refactoring and risk-taking in cases where major architecture design decisions, eg the database used for the system, need to be refactored.
Third, everyone loves a greenfield approach where we start from scratch. Few realize what a high-risk endeavor starting from scratch really is. A new architecture with new technologies brings its own set of risk factors as we have no experience with these ‘modern’ technologies. Also, the amount of domain knowledge embedded in our legacy systems is typically vastly underestimated – sometimes with a factor of 2 to 10!
Another risk is that, while the new architecture is under development, the legacy system needs to be maintained and evolved as that’s what’s running the business and bringing in the revenue. This means the organization needs to split its resources, as a result of which it easily ends up in the worst of both worlds: the new architecture is delayed due to resources being redirected to the legacy system. Similarly, as the legacy system isn’t receiving sufficient resources, customers are complaining and may leave for competitors. I’ve worked with companies where the new architecture and the legacy system had been co-existing for a decade or more! And by the time we’re finally ready to fully switch to the new architecture, it’s already outdated and people start to talk about starting from scratch again.
My advice to virtually any company I work with is to not start a new platform, stop thinking in terms of platform generations and instead focus on the current platform and invest sufficient refactoring resources into it so that you don’t need to start from scratch again.
Software-intensive systems require continuous architecture refactoring in response to the company’s continuously evolving business strategy. Failing to cater to this will cause productivity and quality concerns over time and will require us to address this anyway by a major architecture overhaul or starting a high-risk and draining greenfield initiative. As we will be paying the effort anyway, it’s much better to continuously invest 10-25 percent of resources into refactoring and ensuring the offerings are always up to par technically and in relation to the business strategy. To end with a quote by Philip Johnson: the future of architecture is culture. Make sure you choose the right ones!
