Jan Bosch is a research center director, professor, consultant and angel investor in startups. You can contact him at jan@janbosch.com.

Opinion

Machines that think: regulation meets silicon

Reading time: 8 minutes

This is part 13 in a series on how AI is crossing out of software and into the physical world. This time, Jan Bosch looks at how rules actually land on chips, models and products.

The debate about AI regulation is almost always conducted as a question of quantity: too much or too little; innovation versus protection; Europe cautious, America permissive, China directed. I find this framing nearly useless, because it skips the question that determines everything downstream. Law can’t regulate a capability; it can only attach obligations to something it can identify, count and audit. So, the interesting question is not how much regulation there is; it’s what the regulation attaches to. For AI, the answer turns out to be compute, documents and shipments.

Start with compute, because it’s the most revealing choice. Article 51 of the EU AI Act presumes that a general-purpose model has high-impact capabilities and therefore poses systemic risk when the cumulative compute used to train it exceeds 1025 floating-point operations. Recital 111 says that total includes pre-training, synthetic data generation and fine-tuning. In a 2026 paper in the European Journal of Risk Regulation, Samuel Carey notes that roughly 25 models had crossed that line as of 2025, with 63 reaching 1024 by the end of 2024.

The critique Carey assembles is, I think, correct and worth stating plainly. Treating computational power as a proxy for capability asserts a false equivalence. The threshold is backward-looking, derived from observing past models, while its purpose is forward-looking. It assumes risk scales with training compute, when inference-time compute can increase a model’s autonomy without adding a single FLOP to the training total, and when a modest model wired into critical infrastructure can be systemically consequential while a very large one writing poetry isn’t. Borrowing from German sociologist and philosopher Niklas Luhmann, Carey calls the threshold an “empty formula-like obscurity.”

And the arithmetic isn’t even settled. Does compute spent on an earlier version count toward a later one? Does fine-tuning after market placement add to the original total or start a new count? Are parallel branches of a training run summed or counted separately? The text doesn’t say, and the answers decide which side of the line real models fall on.

Why then choose it? Because it’s auditable. A regulator can ask a provider for a number and cross-check it against a cloud invoice. That’s not a trivial virtue – it may be the only property that makes a threshold enforceable at all. But we should be honest about the trade: The law selected the quantity it could verify over the quantity that mattered, because a rule you can’t verify isn’t a rule.

Now look at what happened to the parts of the Act that were about whether systems actually work. The Digital Omnibus on AI, agreed in May 2026, deferred the high-risk obligations substantially. Stand-alone high-risk systems under Annex III – recruitment, credit scoring, education – moved from 2 August 2026 to 2 December 2027. High-risk AI embedded in regulated products under Annex I – medical devices, machinery, vehicles – moved from 2 August 2027 to 2 August 2028. The deadline for member states to establish regulatory sandboxes slipped to August 2027.

What didn’t move: the Article 5 prohibitions, in force since February 2025. The general-purpose model obligations, applying since August 2025. The Article 50 transparency duties, live since August 2026.

Set those two lists beside each other and the pattern is unmistakable. What landed on time was bans and labels. What slipped was conformity assessment – the machinery for establishing that a high-risk system does what it claims.

That isn’t hypocrisy; it’s capacity. Prohibiting a practice costs a sentence of legislative text and is checked by observing whether anyone does it. Requiring that a credit-scoring system pass a conformity assessment requires harmonized standards that don’t yet exist, notified bodies with staff who don’t yet exist and auditors trained in a discipline that’s itself being invented. Europe wrote the obligation before it built the institution, discovered the gap and bought eighteen months. Anyone who has shipped a regulated product will recognize the maneuver.

However, there’s a more specific and troubling consequence, and it concerns exactly the machines this series has been about. The Omnibus also narrowed the definition of a safety component, so that some AI in regulated products isn’t high-risk unless its failure would endanger health and safety – and it largely exempted AI systems embedded in products covered by the Machinery Regulation from the AI Act. Now recall the fourth part in this series. The 2025 revision of ISO 10218 admits humanoid robots as industrial robots on their manipulator characteristics, while their mobility hazards sit outside the standard’s scope.

Put those together. A walking machine on a factory floor is partly carved out of the AI Act on the grounds that machinery law covers it, and partly outside machinery law’s reach on the grounds that its mode of failure isn’t what the standard was written for. Nobody decided this. Two regimes each assumed the other had it, and the gap is precisely where physical AI lives.

Which brings me to where regulation does grip hard, and it’s not the model layer at all. You can’t embargo weights. They copy perfectly, cross borders as a file and leak. What you can embargo is a package, a tool, a shipment – and that’s what has actually been done. US export controls on advanced accelerators are, by a wide margin, the most consequential AI regulation currently operating anywhere in the world, and they work because they attach to objects that have to physically move.

The enforcement detail is instructive. In May 2026 the Bureau of Industry and Security confirmed that license requirements for advanced AI chips reach companies headquartered in China or with a Chinese parent, including their subsidiaries outside China. Asked whether existing license requirements were still being enforced after the repeal of the AI Diffusion framework, BIS responded: “The answer is yes.” Top Blackwell parts remain barred for China; H200 sales were permitted by a decision in December, a part roughly six times the capability of the H20 previously allowed.

Note what that is. Not a safety regime, not a transparency obligation – trade policy applied to the substrate from the first part of this series, biting harder than anything in the AI Act, because a shipment can be stopped at a border and a file can’t.

So, the structural finding of this part of the series: AI governance is becoming industrial policy, not because anyone chose that, but because jurisdiction grips matter. The law went looking for something solid to attach to and found silicon.

For startups, the practical reading is more specific than the usual complaint about compliance burden. You’re almost certainly not a systemic-risk provider; 1025 FLOP isn’t a threshold a seed-stage company crosses. Your exposure is as a provider or deployer of high-risk systems, and that just moved to December 2027. Treat that as a reprieve rather than a reprieve from thinking: The characteristic failure is to read a deferral as permission to skip documentation, then assemble two years of technical files in a panic because a customer’s procurement team asks for them. Enterprise buyers will demand conformity evidence well before the regulator does.

The genuine unfairness is the ambiguity. A startup fine-tuning an open model can’t currently determine whether it inherits provider obligations, because the counting rules for post-market fine-tuning are unresolved. Large firms absorb that with counsel. Small ones absorb it with founder attention, which is the scarcer resource.

And there’s a real business in the gap. If conformity assessment arrives in December 2027 and the auditor capacity to deliver it doesn’t yet exist, then tooling that maintains technical documentation and captures evidence continuously is a product with a legislated delivery date – an unusually legible market.

For incumbents, compliance as a moat is real but usually described wrongly. The advantage isn’t that compliance is expensive – expense is symmetrical. The advantage is the ability to absorb change. When a deadline moves eighteen months, a large firm reallocates a program; a small one re-plans its roadmap, renegotiates with investors and loses a quarter to the exercise. Regulatory instability is itself a barrier to entry, and it’s one that nobody legislated.

The second advantage deserves naming without melodrama. Harmonized standards are written by committees, and committees are populated by organizations that can afford to send people to them for years. The firms in the room will write workable standards – and they’ll write standards workable for firms like themselves. This is how standardization has always functioned. It matters more than it used to, because the standard now determines who may place a product on the market at all.

For society, the deferrals aren’t a scandal, but they do have a democratic cost that I don’t think has been properly counted. Every slipped deadline transfers a decision from a legislature to whoever fills the vacuum with practice. The firms deploying these systems in 2027 will have established what “reasonable” looks like before any auditor arrives to disagree, and auditors tend to ratify incumbent practice because there’s nothing else to measure against. Delay isn’t neutral; it’s a reallocation of authority from the elected to the operational.

And observe the scope of what we’re actually governing. A compute threshold reaches about twenty-five companies. The provisions that would touch an ordinary citizen’s life – the hiring filter, the credit decision, the benefits assessment – are the ones that moved to the end of 2027. We’ve built a regime that regulates the frontier, which is glamorous and concentrated, while postponing the regulation of the middle, which is mundane, diffuse, and where nearly all of the harm will occur.

Which brings me to Lawrence Lessig, who published an essay called “Code is law” in Harvard Magazine in January 2000 and spent the following decade being proved right. His argument was that in cyberspace, architecture regulates behavior – that software and hardware constrain us as effectively as statute and that we’d find ourselves governed by code we never voted on.

Twenty-six years later, legislators have fully absorbed the lesson and drawn the logical conclusion. If architecture regulates, then regulate the architecture. Hence a threshold denominated in floating-point operations. Hence license requirements on packaging capacity and export classifications for accelerators. Law couldn’t get a grip on behavior or on weights, so it reached down the stack until it found something that had to pass through a port.

That’s a real answer to Lessig’s problem, and it arrives with a cost on the same ledger. The most binding instrument of AI policy in the world today is a licensing decision about a shipment of chips – made by executive action, revised in weeks, with no parliamentary vote, no conformity assessment and no appeal. We’ve succeeded in regulating the architecture. We should be clear-eyed that we did it by moving the decision into the one branch of government that moves at the speed of the technology, which is also the one with the least obligation to explain itself.

Lessig’s worry was that we’d end up governed by arrangements we never chose. It would be an unfortunate irony if the solution to that problem turned out to share the property.

Related content