The AI-driven company: AI compensators

Product development requires several different skill sets. When focusing on the software part of a product, we can recognize the problem domain, technology, infrastructure and people as four main areas of competence. For a team to be effective, it needs to understand the problem or application domain so that it can realize new functions or features in such a way that it makes sense to users and provides value to customers. Obviously, it also needs to understand the technology required to realize the functionality or feature. This technology includes what the team is expected to create or evolve, but also the libraries, frameworks and components that it uses. Infrastructure is concerned with the tooling, build chains, coordination with other teams and parts of the overall product and the infrastructure needed for releasing the product or its latest version to customers. Fourth, teams consist of humans who need to work together in ways that are effective and productive, which requires a skill set as well.

Of course, in addition to these four, we also need competencies around regulatory compliance, the business model or models of the company, the variations in needs between different customers, as well as a host of other aspects. This is one of the reasons that it often takes quite a bit of time for new employees to become productive in a way that provides maximum value. Through training, explicit mentoring and simple osmosis, people need to absorb all this knowledge to be effective.

The challenge is that there are many companies and contexts in which teams are asked to take on projects for which they lack the necessary knowledge and/or skills. Traditionally, teams would either scramble to educate themselves on the job, often causing suboptimal results due to lack of knowledge, or bring in experts who, though knowledgeable on their own topic, have difficulty balancing their expertise with other areas of knowledge that interact and need to be considered.

With the emergence of LLM-based AI agents, we can take another approach: the team receives an additional, virtual, team member in the form of an AI agent that has the knowledge and competence the rest of the team is lacking. We refer to this as the AI compensator stage. An AI agent becomes a virtual contributor to a team of software engineers and provides competence to the team that the other members lack.

In our interview study, we’ve seen several examples. In some cases, it’s about using new tools or frameworks that the team hasn’t used before and where the agent can provide knowledge and support in getting to use them effectively. In other cases, it’s concerned with domain knowledge. The team may be asked to work on an application in a domain where they have limited experience and understanding and the agent can provide details on the peculiarities of the domain. Finally, the agent may support the team with a part of the development process that they don’t have much experience with. For example, when building software subject to regulatory compliance, evidence needs to be collected to show that the software satisfies the regulations and an agent can then provide input on the necessary steps, activities and artefacts.

As we structure the input from the interviews, we can organize the tasks of the AI agent around specific roles, depending on where the team lacks relevant skills, ie product owner, engineer, quality assurance and project management. In addition, there are some cross-functional aspects that can’t be mapped to a specific role.

For the product owner (PO) role, the primary focus is concerned with domain knowledge. The AI agent can support the PO with stakeholder interaction as well as market scanning to ensure the team has a good understanding of the needs, wants and wishes. In addition, these potential development tasks need to be prioritized and put in a backlog that makes sense both from a priority and a development sequencing perspective.

For engineers, there are of course the same tasks as in the AI assistance stage, such as generating code, creating test cases, supporting code reviews and generating documentation. However, the AI agent can also provide knowledge on the use of new frameworks, libraries and other reusable assets originating inside or outside the company. This form of knowledge retrieval can be very effective, especially as it can be connected to the architecture of the system. The latter allows for early identification of mismatches between the system architecture and the way external components are structured.

Quality assurance is a critical part of any development initiative. With the emergence of continuous deployment, or DevOps, the automation of quality assurance is increasingly important, but there’s always a risk that specific issues are missed and flow through to the field. AI agents support the identification of areas that aren’t well covered. When we have a solid flow back to the company from systems out in the field, AI agents can help warn about quality issues identified at run-time.

In addition, for systems subject to regulatory compliance, quality assurance often plays a central role in collecting the necessary information to ensure that the company and its products stay compliant. AI agents can automate or at least significantly support the regulatory compliance process. In cases where the company is subject to multiple, potentially conflicting regulations, different AI agents can negotiate with each other to identify the main areas of contention. These areas can then be highlighted and resolved by the humans involved in the process.

The project management or scrum master role is, of course, predominantly focused on humans and their behavior. AI agents can, however, keep track of metrics and KPIs, identify risks such as too much work in progress and scope creep, remind teams of Agile principles (assuming you’re using Agile) and even support the facilitation of meetings.

Finally, there are areas where an AI agent can support the team with tasks not necessarily assigned to a single role. For instance, it can generate all the documentation that’s needed for different contexts or convert technical issues into business language. And, for anyone new entering the team, having a “buddy” that you can ask questions without feeling like an idiot can be extremely helpful in rapidly onboarding new team members.

In the second stage, the AI compensator, one or more AI agents support teams in areas where they lack skills or competencies. However, it isn’t a binary state. Teams and individuals may have a significant level of proficiency and still benefit from AI agents to complement them and compensate for team members. This may easily read as a future scenario, but I think it’s important to realize that many of these capabilities are available today. And if you’re not using them already, you’re doing yourself, your employer and your customers a disservice. To end with a quote from Bill Gates: “Agents are not only going to change how everyone interacts with computers. They’re also going to upend the software industry, bringing about the biggest revolution in computing since we went from typing commands to tapping on icons.”