The Model Leaves the Lab
Road to CEPAS 2026 — Episode 5
At the end of the last episode I made a promise. Buried in the discussion section of our own 2023 paper is a reference to European medical-device regulation that I cited and then walked straight past. I said Episode 5 would pick it up and not move on. So here we are.
Regulatory sentences are easy to walk past because they read like housekeeping. You build a model, you report an AUC, and somewhere near the end a reviewer-pleasing sentence acknowledges that turning this into something used on real babies would mean engaging with European device law. Cite the regulations, nod, move to the conclusion. I have done exactly this. It’s not dishonest, but it does quietly imply that the regulation is the last, dull step, A lot of paperwork between a finished model and the bedside.
Yet, it isn’t the last step. It’s a different question entirely, and it’s the one the whole series has been circling.
The sentence I should have sat with
Our discussion makes an argument I now think is the most important thing in the paper, and it’s not the AUC. It says, roughly, that to build a medical device you have to specify an intended use together with a performance claim — and the performance claim has to be good enough to identify any hazard that comes from using the device in practice.
Read that again with Episode 4 in mind. We spent a whole episode on the alarm policy and the precision problem, the finding that at the operating point we chose, most alarms are not followed by sepsis. In the research frame, that’s a PPV number, a point on a curve, a thing you note and contextualise. In the regulatory frame it is not a number. It is a hazard. A false alarm that triggers a workup, or antibiotics, or a clinician’s decision to treat is over-treatment, and over-treatment is precisely the kind of harm the performance claim is supposed to be sensitive enough to catch. The regulation takes the thing we treated as a metric and reclassifies it as a risk to a patient.
That reframing is the whole episode. Everything below is consequence.
Which regulation, and why it matters
The first thing to settle is which regulation even applies, because Europe has two and they classify by completely different logic. The in-vitro diagnostic regulation governs devices whose medical purpose depends on examining specimens — a sample taken from the body, run through a test. The medical device regulation, the MDR, governs everything else, including software that stands on its own.
The word “diagnostic” pulls the reflex toward the in-vitro side, sepsis, blood cultures, diagnosis everywhere. But our model never touches a specimen. It runs on continuously monitored physiology and data already in the record, and produces a score that informs a clinical decision. That is software as a medical device, squarely under the MDR. The in-vitro regulation, despite the diagnostic subject matter, simply isn’t the instrument that governs a thing like this.
This isn’t pedantry. Which regulation you’re under decides everything downstream, what evidence you owe, whether an independent body has to audit you, how long it takes, what it costs. So: software, under the MDR. What does the MDR do with software that informs a clinical decision?
Rule 11, and why this lands high
The MDR has a single rule — Rule 11 — that classifies software, and it is notorious for pushing almost everything upward. The logic runs by worst credible harm, not by how often the software is right. Software that provides information used in diagnostic or therapeutic decisions starts at Class IIa. If the decisions it informs could cause death or irreversible harm, it’s Class III. If serious deterioration, Class IIb. And there’s a second limb: software that monitors vital physiological parameters, where the nature of the variation in those parameters could put the patient in immediate danger, lands at IIb.
Walk our model through that. It monitors continuous vital-sign physiology. The variation it watches for is the early signature of a baby becoming septic, a state where deterioration can be immediate and severe. The decision it informs is whether to start antibiotics in a preterm infant, where error in either direction has consequences. On a conservative reading, this is not low-risk software. Class IIb is a very plausible landing point; depending on the exact intended use and performance claim, a Class III argument is not impossible.
Class IIb is not Class I. Class I you can largely self-declare. From IIa upward an independent notified body audits your quality system and reviews your technical file before anything is allowed near a patient. The thing that pushes you over that line is not the sophistication of the model. It’s the severity of the decision it touches. A simpler, more interpretable model with the same intended use lands in exactly the same class, a point that should sound familiar to anyone who read Episode 3.
“It works in a simulation” is the start of the conversation
There’s a story that should be taught alongside every sepsis-prediction paper, and it isn’t ours. The Epic Sepsis Model was built into one of the most widely used electronic record systems in the world and deployed across hundreds of American hospitals, about as far from “research artifact” as a model can get. Then an external team at Michigan validated it independently and found it missed roughly two-thirds of sepsis cases at the threshold in use. Deployed everywhere; externally validated almost nowhere until someone finally checked.
That gap — between deployed and validated — is the gap the regulation exists to close. And it’s the gap our own staged plan, the one I cited approvingly in the paper, is built around. We named the path: prospective validation first, then a clinical validation study, then the hard outcomes, mortality, sepsis episodes, antibiotic days. That ladder isn’t bureaucratic decoration. Each rung is a different claim about the world, and the regulation is essentially asking you to climb it in order and show your work at each step. A retrospective AUC, however good, is a claim about a dataset. It is not yet a claim about a baby.
The model in this lineage that actually climbed the ladder is the old one. The heart-rate-characteristics work I traced back in Episode 1 ran a genuine prospective randomised trial — 3003 very-low-birth-weight infants across nine NICUs, with a reduction in mortality when the monitor was displayed, published in 2011. Fifteen years later it remains the canonical prospective randomised trial in this literature. Much of the field since, ours included, has stopped at the rung marked “good ROC curve.” The regulation is, in a sense, just the formal version of the question that trial answered and the rest of us have been deferring: does using this change what happens to the patient?
The ground is moving while I write this
I should be honest that the regulatory picture in 2026 is not settled, and a research diary should say so rather than pretend the law is a fixed monument. The AI Act now adds a second layer: an AI medical device that requires notified-body assessment will generally fall into the high-risk AI category, which our model, on the reading above, would. That brings requirements around data governance, representativeness of the training data, transparency, human oversight, logging, and post-market monitoring. The direction of travel is toward avoiding two wholly separate assessments and integrating the AI Act requirements into the existing product-conformity architecture where possible, rather than running them in parallel. The exact timing and mechanics remain in flux: the high-risk AI timelines and a December 2025 proposal to simplify the MDR/IVDR framework, which secondary commentary reads as touching the software-classification logic, including Rule 11, are still moving through the system.
So the exact procedural path may move, and the specific class boundaries with it. What won’t shift is the underlying logic: severity of the decision drives the obligation, and deployment is gated on evidence the field has mostly not produced.
I’m flagging this partly for accuracy and partly because it matters for the talk in Lyon. The details will have moved again by October. The argument won’t.
Back to 3am
I ended the last episode at a bedside: a score crosses a threshold in front of a clinician at three in the morning, and the only question that matters is whether that becomes earlier treatment or just one more thing beeping.
The regulation is what stands between those two outcomes, and I’ve spent this episode learning that I’d been treating it as the wrong kind of thing. Not paperwork after the science. A second, harder science — the one that asks not “can the model tell true from false” but “when this fires at 3am and someone acts on it, is the baby better off.” Our paper has an answer to the first question and, honestly, an IOU on the second.
Next episode I want to follow that IOU somewhere specific: what a real clinical validation of a model like ours would actually have to look like, and why almost nobody has done one.
How I used Claude
The regulatory reading in this episode is not my home turf, and that shaped the division of labour. I asked Claude to pull the current state of the EU device rules — the MDR classification logic, the software rule, the in-vitro versus medical-device distinction — and the 2026 developments around the AI Act, because this is exactly the kind of material that dates fast and that I’d otherwise half-remember from a conference slide. Claude surfaced the Rule 11 mechanics, the Epic Sepsis Model external-validation figure, and the live status of the AI Act overlay and the late-2025 simplification proposal.
The argument is mine. The decision to open on our own paper’s “intended use plus performance claim” sentence, and to read the false-alarm rate from Episode 4 as a regulatory hazard rather than a metric, was the spine I wanted before any research happened; the regulation just gave it teeth. I had Claude check the Rule 11 classification reasoning against current guidance rather than take my reconstruction of it on trust, because a confidently wrong regulatory class is worse than no class. Where the law is genuinely unsettled in 2026, I’ve tried to say so plainly rather than launder uncertainty into authority — and that paragraph got read adversarially before it went in.
Next: Episode 6 — what a real clinical validation of a model like this would have to look like, and why, fifteen years on, that 2011 trial still stands largely alone.
