The kaizen AI generator stage is based on the principle that systems should never stand still but continuously adapt and improve, explains Jan Bosch.
In this series, we’ve explored the journey toward the AI-driven company. First, we looked at the business process view and recently, we’ve focused on the R&D process. Here, we see an evolution from AI assistants to compensators, superchargers and finally system generators. Each stage represents a deeper integration of AI into how we design, build and deliver software-intensive systems.
The final step in this maturity ladder is the kaizen AI generator. Borrowing from the Japanese concept of kaizen, meaning “continuous improvement,” this stage goes beyond creating a system once. Instead, the AI agent team remains engaged, constantly monitoring, experimenting and regenerating the system to ensure it improves itself over time.
Whereas the AI system generator focuses on producing a complete, tested and documented system from an initial prompt, the kaizen AI generator takes it a step further. It stays active after deployment, embedded into the lifecycle of the system. In many ways, the AI agents that make up the solution are part of the system, not outside it. They don’t just hand over a product; they act as co-creators and custodians of performance. They continuously gather feedback, test new approaches and regenerate components to keep the system aligned with business goals and operational realities.
Although no companies in our interview study had reached this stage in their AI R&D maturity, several discussed different aspects of what a kaizen AI generator would incorporate. There are several aspects we’ll discuss, including monitoring, experimentation, regeneration and self-healing.
First, for a set of AI agents to continuously improve the system they’re responsible for, it’s necessary to continuously monitor the system. This means that AI agents track performance metrics, user behaviors and external conditions in real-time. It allows these agents to identify opportunities for improvement and detect early signs of degradation. It also lets them identify situations or contexts where the system performs mediocre and develop specific solutions for those situations.
The second aspect is concerned with experimentation at scale. We typically lack the models and theories to predict the impact of changes before we implement them. However, the impact might be negative and we don’t want everyone to be negatively affected by changes with negative outcomes. The best approach to addressing this is to employ experimentation. Like digital scientists, kaizen AI generators design and run experiments to explore alternative ways of realizing functionality. Typical techniques that humans already use and that AI agents will also employ include A/B tests, simulations and reinforcement learning to validate improvements.
As the AI agents continuously seek to improve the system, they’ll need to regenerate code frequently. For example, when experiments looking to realize improvements are successful, the system automatically regenerates code for relevant components or subsystems, as well as the documentation and test suites. Although research into automatically identifying technical debt is ongoing, using AI agents continuously allows for managing technical debt proactively rather than accumulating it over the years.
Of course, even systems that don’t change have issues and may experience failures. In this case, fully autonomous systems need the ability to heal themselves and be resilient in the face of failures. When failures occur, the agents adapt in real-time, restoring functionality while also learning from the incident to prevent recurrence. We can imagine a situation where serious system failures initially require human involvement, but the agents learn from that involvement and, over time, can resolve the failures autonomously.
The benefit of adopting kaizen AI generators is that they create a major improvement in how companies build and operate systems. These systems will experience continuous performance improvements as they can evolve daily. In addition, proactive regeneration of code keeps the codebase fresh, clean and aligned with current needs, reducing technical debt. Third, as long as the system can accurately measure its performance, it can adapt much faster to shifting customer demands, regulatory changes or market shocks, as a change in the ‘reward function’ will automatically lead to the system accelerating its experimentation to improve performance. Finally, the same techniques can be used for autonomous correction and self-healing to improve uptime and reliability. In essence, kaizen AI generators shift the mindset from projects that end to systems that are alive.
As always, the model isn’t without its challenges. One of the major ones is concerned with governance and compliance, as regulators and customers will demand guarantees of safety, traceability and accountability in continuously changing code. A second major concern is goal alignment, as the optimization targets of AI agents must be tightly coupled to business strategy. In our experience, companies struggle significantly in capturing these targets in quantitative terms and often fall into the “worthwhile many” trap. Finally, the approach requires a cultural shift so that the humans in the organization trust the system and the continuous flow of changes and improvements.
Although it’s the highest level of the R&D maturity ladder, this approach requires a set of preconditions not required by earlier steps. First, we need to have monitoring in place to ensure relevant telemetry and observability. Second, we need to see to it that any autonomous experimentation by the system is safe, so we need to start in areas of system functionality that have limited associated risk. The same is the case for core regeneration, where initially we want to focus on non-critical components before broadening. Finally, we need to have guardrails in place, such as compliance checks, audit trails and human-in-the-loop mechanisms, to ensure that the system can be trusted. However, in the end, it requires the organization to adopt an AI-first culture where AI agents are seen as collaborators, rather than tools.
The kaizen AI generator stage represents the culmination of the AI-driven company journey for R&D. This stage is based on the principle that systems should never stand still but continuously adapt and advance. Rather than generating it once and then updating only when it can no longer be avoided, the approach should be for the system to continuously evolve and improve. As we discussed in the context of the three-layer product model in earlier posts, differentiation is continuously eroding and the only way to avoid it is to continuously experiment, improve and evolve. To end with a quote by Daoud Abdel Hadi, “Generative AI is just the beginning; AI agents are what comes next.”
