The AI-driven company: static ML

Interestingly, in the companies I work with, there’s still some hesitation to include machine learning models in products. This is especially the case when there’s some form of safety certification associated with the product; we don’t always know how to ensure that the ML models won’t jeopardize critical product characteristics.

That said, we need to put AI into our products and the first step often is to include a static ML model that’s trained outside of the product and that’s simply integrated as a component. Of course, the ML model needs input data for inference and to provide its results somewhere, but by and large, it can work as a relatively standard software component, except for its behavior being probabilistic instead of algorithmic.

Many companies struggle with collecting relevant data, selecting the right model architecture, ensuring data quality, training the model and validating it before being able to deploy it. It should be noted, however, that sites like Hugging Face provide access to numerous open-source ML models. Rather than going through the trouble of doing everything from scratch internally, I think it makes much more sense to try to find an open-source model and use it straight off or do some fine-tuning.

Although a great starting point, the challenge of static ML is, of course, exactly that: it’s static. And every piece of technology, including software, eventually becomes obsolete when kept static. So we need to understand the nature of AI in this sense, as software and especially AI commoditize much faster than other technologies. So, as a first step, using pretrained, potentially open-source, models is a great start, but it’s important not to stay there. That’s why our maturity ladder still has four additional steps.

It’s easy to understand why many companies use this as a natural starting point. Pretrained models are easy to plug in, as they mostly resemble yet another open-source component. Consequently, the internal investment in ML and AI can be minimal. Second, as the model comes from the outside, it has typically been used by many others and its performance is often well understood. Also, as many others use the model, it may even be periodically updated by others, providing benefits. Especially in situations where a company quickly wants to provide a “smart” feature, using existing pretrained models offers a fast path to deployment and gaining some differentiation in the market.

Differentiation, however, is always temporary; competitors will soon catch up and offer the same functionality. The challenge with relying on external, pretrained models is that these artefacts put constraints on the architecture of our system, which may make it difficult to build solutions that evolve more easily. So, the quick win may come with architectural technical debt that will need to be paid off at some point in the future.

Furthermore, many companies starting with AI often insufficiently appreciate that there’s more to using ML models than simply inserting them into the system. One of the key challenges is access to high-quality data that can be used for inference. Although most companies feel like they’re drowning in data, I’ve been part of many situations where, despite the large data assets, the data needed for a specific use case simply wasn’t available. Or, if it was, it had the wrong accuracy, quality, frequency or some other invalidating characteristic.

Even within this first step, companies evolve through a number of stages. The first is to simply embed a pretrained model sourced from the outside, either commercially or from the open-source community. Once some experience has been gained with it, companies realize that periodic retraining is required, and a process is introduced to collect data for retraining and to trigger it periodically or based on certain events. The third is when fine-tuning or even personalization for specific customers is added to the process and the model (or multiple variants thereof) is deployed in such a way that customers get an optimized user experience.

This seems like an easy, straightforward evolution process. However, each substep requires changes in the architecture, the data management infrastructure, the tooling and, of course, the governance of product development and the regulatory compliance, as a minimum.

For companies starting with AI in their products, one of the key factors is to start small. The first experiments should be low-risk and with minimal impact on customers in case things go sideways. Also, the solution should be instrumented extensively so that enough data is available to evaluate the performance of the model continuously. Finally, we need guardrails and associated infrastructure in place so that we can manage versioning and rollback in case the latest model version doesn’t perform in deployment.

When moving toward incorporating AI and ML in the product, most companies start with embedding pretrained models sourced from outside the company. This is a great first step, but it’s only a foot in the door. It’s important to keep evolving toward the higher steps in our maturity model. The challenge is that the initial differentiation that might have been gained by introducing a “smart feature” will either commoditize quickly or require constant evolution. In the next steps, we’ll increase the level of advancedness, but for now, focus on where you are in your organization and where you need to move to. To end with a quote from Salesforce: “When deploying AI, whether you focus on top-line growth or bottom-line profitability, start with the customer and work backward.” ML has to support a customer and his or her needs.