Applying AI with an architectural mindset makes it a genuine force multiplier. Applying it without one makes it a complexity multiplier.
At Alten, we’ve been doing legacy system modernization projects for decades. Generative AI has significantly changed the way we approach these projects. Stories about creating new applications in hours instead of months have been competing with each other over the productivity gains AI can bring. However, most of these stories focus on greenfield development. But what does AI actually mean for legacy systems? Should we continue maintaining existing code or should we start from scratch and throw away the old code and prompt our way to a new codebase?
As is always the case with these types of questions, it depends. If we were to prompt: “Make me a new operating system with better performance than Windows and Linux,” the result wouldn’t be usable. These systems are far too complex, too deeply embedded in real-world constraints unavailable in an LLM. On the other hand, if we’re dealing with a small application of, say, 10 KLOC, then a rewrite from scratch might be a viable option.
This tells us that for most systems, we need to keep working with existing legacy codebases, making improvements and extending their functionality. But what’s the essential gain brought by AI-assisted software development? And what are the pitfalls?
Unstable feedback loop
Before the advent of generative AI, we were already dealing with what can be described as the “vicious cycle of legacy systems.” This vicious cycle begins when the code needs to be changed to support a new requirement or to fix a problem. The team must decide whether there’s time to refactor the existing code and realize a future-ready design. In many cases, project pressure wins and refactoring is postponed in favor of adding code on top of what already exists. This short-term solution makes the system larger and increasingly more difficult to understand.
The practice of adding code instead of refactoring creates an unstable feedback loop in which complexity increases over time, productivity decreases, risk grows and consequently, the cost of change rises. Companies have struggled and some have even gone bankrupt over this, as they were unable to ship new features and lost ground to their competitors. The public sector suffers from this too. Governments have been unable to implement new legislation because the civil-service systems that need to be updated are no longer fully understood by anyone. Experience confirms that tightly coupled, poorly structured architectures are a core predictor of poor software delivery performance and that teams operating under them consistently struggle to deploy safely and independently.
The root cause of this lies in the interaction between two factors: the quality of the software architecture and our capacity to understand it. Well-designed software is significantly easier to navigate than poorly designed software – that much is true. But even a well-designed system eventually grows to a size and complexity that exceeds what any individual can hold in their head. Our cognitive capacity is the scaling constraint. A more experienced software engineer can navigate a legacy codebase more effectively than a junior one, and a well-structured system is easier for both. But as systems grow, even the most experienced engineer may be outperformed by an LLM in raw comprehension of an unfamiliar codebase.
A well-defined architecture compensates for our limited cognition by making local reasoning possible. When a system is decomposed into components with clear responsibilities and well-defined interfaces, an engineer doesn’t need to understand the entire codebase to reason confidently about a specific change. The engineer only needs to understand the relevant component and its contract with the rest of the system. Complexity is contained rather than unbounded. Architecture also externalizes design intent that would otherwise exist only in the minds of the engineers who built the system, creating a shared understanding that outlasts any individual and survives team turnover.
The absence of a well-defined architecture achieves the opposite. Complexity is unbounded. To understand one thing, an engineer must understand everything. This creates a hard dependency on the engineers who originally built the system. Knowledge that’s rarely documented and that walks out the door when those people leave. The result is a direct and measurable impact on time to market: It simply takes too long for a new team to become productive.
Coupled with our limited ability to understand complex systems is another factor: risk aversion. Because engineers don’t fully understand the current software or the impact of a code change, only ‘safe’ changes are made, which inherently increases the architecture’s complexity over time. These are incremental patches that avoid touching what’s fragile but add another layer of accidental complexity. New technical debt is created that’s unlikely to be paid back.
AI-assisted decomposition
So, what happens when we simply let AI solve an issue? It will figure out a solution and provide a pull request consisting of a lot of new code. The issue appears to be resolved, while in fact, the problem has grown bigger. We’re adding even more code than before the AI era, further increasing the complexity of the codebase. Without structure, AI becomes an accelerator of the very cycle we’re trying to escape.
The only scalable way to deal with complex systems is to decompose the architecture into components that are loosely coupled and have high cohesion. This allows both humans and AI-assisted tooling to reason about the system effectively. It also provides a clear standard for evaluating whether an AI-generated solution is compliant with the intended design.
Consider a monolithic climate control system where control loop logic is scattered across the user interface, driver software, configuration and API modules. A first step toward decomposition is extracting a dedicated control loop component with a well-defined interface. Once isolated, both engineers and AI agents can reason about control loop behavior independently, test it in isolation and change it without fear of cascading side effects. The scope of any change, and the scope of what AI needs to understand to make that change correctly, shrinks drastically.
This is also where AI brings unique value in the legacy context, beyond just generating new code. AI tools can analyze existing codebases, identify clusters of related functionalities, detect hidden dependencies and help create a plan to isolate the control loop component, while also mitigating regression risk by generating test cases at multiple levels of the test pyramid. In this way, AI can actively assist in the process of decomposition itself.
Before the AI era, this kind of architectural refactoring was expensive and slow. It required deep familiarity with the existing system, significant manual effort to build test coverage from scratch and long-term horizons to absorb unexpected setbacks, which is why many organizations deferred it indefinitely. AI changed this. It can rapidly analyze an unfamiliar codebase, help generate the test coverage needed to make changes safely and support the incremental work of decomposition at a pace that was previously impractical for most teams.
New division of labor
If AI is better at handling complexity than us, why do we still need to manage it? Even though AI handles complexity better than humans, it isn’t immune to it. Poorly structured “Big Ball of Mud” architectures are characterized by cyclic dependencies, low cohesion and high coupling. In these systems, even AI struggles to determine the impact of a change.
One practical constraint is model context capacity: Very large codebases spanning millions of lines across hundreds of modules still exceed what can be loaded into a single model context. The good news is that context window sizes have grown considerably. Models nowadays support hundreds of thousands of tokens or more. For the very largest systems, unfortunately, this still provides only partial coverage.
The more fundamental issue is context noise. In a poorly structured codebase, it’s difficult to identify which parts of the code are actually relevant to a given change and no retrieval strategy can reliably surface what the structure doesn’t isolate. In “Big Ball of Mud” architectures, important details are buried in a large amount of unrelated code, making it harder for AI to reason correctly and increasing the likelihood of hallucinations. In contrast, in a well-designed system, most of what ends up in context is directly relevant, raising the signal-to-noise ratio and allowing AI to reason more accurately.
Structure also becomes critical as soon as development moves from a single agent to many. In an agentic environment, multiple agents working simultaneously on the same codebase will, without clear boundaries, inevitably make conflicting changes to independent features. Components with well-defined responsibilities allow work to be safely parallelized: Feature development can be decomposed into a coordinated sequence of steps, each scoped to individual components, with an orchestration layer managing dependencies between them. This isn’t a nice-to-have; it’s the only approach that scales as the number of agents grows.
The relationship between structure and quality points in the same direction. A well-structured architecture allows behavior to be tested at different levels of abstraction. Individual components can be verified in isolation, which makes it easier to cover edge cases and performance characteristics. This shifts the burden of validation from the entire system to a manageable subset. Even when AI generates the test cases, human judgment remains essential. A person reviewing the behavior of a single, well-defined component is in a far better position to do that than a person reviewing the behavior of an entire system.
This leads to a new and explicit division of labor between humans and AI. AI excels at navigating complexity: reading unfamiliar code, identifying patterns, generating implementations and spotting potential issues across large bodies of code. Humans own the architectural intent, the decisions about how a system should be structured, what its components are responsible for and where the boundaries lie. Human judgment is also irreplaceable in validation: deciding whether an AI-generated solution is not just functional but architecturally sound.
Managing legacy systems is fundamentally different from greenfield development. In legacy system management, the name of the game is the understandability of the architecture. Without this, the complexity of the architecture will continue to grow and we’ll remain trapped in the vicious cycle of legacy systems, with or without AI.
Breaking the cycle
The fundamentals of software engineering haven’t changed with the advent of AI-assisted tooling. The wisdom built up over the past decades stays valid. Specification skills, architectural thinking and solid verification and validation practices are as essential as ever. With AI-assisted tooling, these artefacts become the primary input to software development, not a byproduct of it. Alongside these enduring skills, working effectively with AI requires new competencies: the ability to formulate precise prompts, to evaluate AI-generated code critically and to define architectural guardrails that keep automated generation aligned with design intent.
The vicious cycle of legacy systems isn’t new. What’s new is that the cycle has, for most organizations, been running long enough that escaping it with conventional methods has become practically impossible. Each iteration has added complexity, raised the cost of change and narrowed the window for meaningful improvement.
AI introduces, for the first time, a credible means of breaking that cycle. It can absorb the complexity that has accumulated over decades, reading unfamiliar code, identifying structural weaknesses and generating the tests that make safe refactoring possible at a speed and scale no team of engineers can match. But reversing the cycle requires architectural intent. Without it, AI simply accelerates the forward direction: more code, faster, with less understanding.
Applying AI with an architectural mindset makes it a genuine force multiplier. Applying it without one makes it a complexity multiplier. The difference between those two outcomes is architecture.
