Software quality is about much more than code

Ger Cloudt’s first encounter with programming involved the use of punch cards during his electronics studies at Fontys Venlo University of Applied Sciences in the early 1980s. After graduating, he embarked on a career as a digital electronics engineer, focusing on both designing digital circuitry and developing software to control microprocessors. “This was in assembler, and I remember I created truly unstructured spaghetti code,” Cloudt recalls. “Naturally, this made it exceedingly difficult to troubleshoot and fix bugs, teaching me a tough lesson that there must be a better way.”

Fortunately, during his second assignment, Cloudt was paired with an experienced mentor who taught him to begin with structured pseudocode and then convert it into assembler. “This was the first time I experienced that structure can facilitate the creation of robust code and make debugging easier.” The experience eventually led him to transition to software development a few years later.

Process versus skill

Cloudt went on to work at Philips Medical Systems as a software development engineer and later as a software architect, where he learned how processes and skills complement each other. “To execute actions, you need a certain skill level, while to achieve results, actions must be structured by a process. However, the importance of process or skill depends on the type of task. On the one hand, there are tasks like assembly-line work or building Ikea furniture you bought, which involve a strict process but minimal skill requirements. On the other hand, tasks such as painting the Mona Lisa, as Leonardo da Vinci did, rely less on processes but require a high skill level that few possess.”

During this period, Cloudt observed a strong emphasis on processes in software development. “This was the period of the Capability Maturity Model’s emergence, aimed at improving software development processes. However, even with processes in place, skills remain essential. In the pursuit of high CMM levels, undervaluing skill is a real danger.” This insight was further reinforced when Cloudt transitioned to management roles at Philips Medical Systems, leading teams of sixty people. “Achieving a specific CMM level quickly turns into a goal in itself, and as Goodhart’s Law states: when a measure becomes a target, it ceases to be a good measure. I increasingly believed that skill level is more important than processes for software engineers. A process can facilitate applying your skills, but with inadequate skills, no process will help.”

Cloudt subsequently learned about the importance of transparency. “In my first quality management role, I had to look at an issue involving the integration of two distinct software stacks. One team developed NFC software, another worked on software for a secure element. Integrating both turned out to be a challenge. When I looked at it deeper, I discovered that although the teams were testing their software, test failures weren’t monitored systematically. So we created daily updated dashboards showing test results, and the developers had daily discussions of the outcomes. We even shared the dashboards with the customer. Naturally, everything appeared red initially, but this served as a strong incentive for the developers to improve. Consequently, the project succeeded.”

Learning by sharing

In his role as a software R&D manager at Bosch, Cloudt started to feel the need to share his insights on software quality. He began by sharing articles on the company’s internal social network, as well as on Linkedin. “I received a lot of positive feedback, particularly from Bosch colleagues,” he says. “So in 2020, I decided to write a book, ‘What is software quality?’. This experience was very enriching, as it made much of my implicit knowledge explicit and revealed gaps in my knowledge as well.”

In a quality committee at Bosch, Cloudt met a young graduate with a Master’s degree in quality management. When he asked whether the graduate had taken a course on software quality, the answer was negative. “This prompted me to approach the Engineering Doctorate program at Eindhoven University of Technology, where they invited me to give a guest lecture. Eventually, I became a lecturer for a quality management course.” Cloudt also began speaking about software quality at events, such as a Bits&Chips event in 2021 and he’s currently launching two training programs at the High Tech Institute, one for engineers and one for managers. His current role is software quality manager for the Digital Application Platform development at ASML.

Measuring software quality

Software quality as such isn’t measurable, Cloudt maintains, due to the concept’s diversity. “You can measure some specific aspects of software quality, known as ‘modeled quality.’ These include cyclomatic complexity of code, dependencies, code coverage, line count and open bugs. Such metrics are useful, but everyone who sets targets on them should be wary of Goodhart’s Law.”

An essential part of quality remains unmeasurable: transcendent quality. To illustrate this, Cloudt compares it to evaluating a painting. “You can measure paint thickness and canvas dimensions, but you can’t measure the painting’s beauty. The same applies to software quality: you can measure code coverage by your unit tests, but that doesn’t determine whether the tests are good. You need an expert opinion for this, supported by the modeled quality you measure.”

When people think about software quality, they often mention aspects such as modularity, clean code and usability. These are examples of design quality (eg modularity, maintainability and separation of concerns), code quality (eg clean code, portability and unit tests) and product quality (eg usability, security and reliability). However, according to Cloudt, these three types of quality require a frequently overlooked element: organizational quality. “This type of quality determines whether your organization is able to build high-quality software. Aspects such as software craftsmanship, mature processes, collaboration and culture are vital to organizational quality. Never underestimate culture. An organization should foster an environment where software engineers can thrive and deliver excellent design, code and product quality.”

Intended and implemented design

There are several well-known best practices for developing high-quality software, including test-driven development (TDD) and pair programming, alongside static code analysis. Cloudt also adds something less common: static design analysis. “Many people don’t realize that there’s a difference between the intended design and the implemented design of software. Software architects document their intended design in UML models. However, a gap often exists between this intended design and its implementation in code. Keeping this gap small is a best practice. Tools can check for consistency between your code and UML models, issuing warnings when discrepancies arise.”

This gap between intended and implemented design often emerges under time constraints, for example due to project deadlines. “In such cases, you take a shortcut by ‘hacking’ a solution that allows you to meet the deadline, with less emphasis on quality,” Cloudt explains. “This is a deliberate choice to introduce technical debt due to time pressure. While this might be the only immediate solution, addressing this technical debt later is crucial. After the release is delivered, you should set aside some time to develop a proper, high-quality solution. Unfortunately, this doesn’t occur often. Managers should recognize the need to give developers time to reduce this gap and this technical debt to prevent future issues. Through their decisions, managers significantly contribute to organizational quality, directly influencing software quality.”

This article was written in close collaboration with High Tech Institute.