Episode 5 ended with a statement. Our paper showed that the model can separate higher-risk periods from lower-risk with useful but far from perfect accuracy, it flagged roughly half of late-onset sepsis episodes before clinical suspicion without instant alarm fatigue. That is the discrimination question, the easier half. The harder half, does acting on that alarm at three in the morning leave the baby better off? This is still open, because the paper does not answer it and could not have. That is not a modeling question, it is a trial question. This episode is that quest, to go to somewhere concrete.

The place it leads is a single study run between 2004 and 2010, and the uncomfortable fact that, fifteen years on, it is still pretty much the only one of its kind.

The rung we are standing on

In the discussion of our 2023 model, we sketched the road a prediction like ours has to travel to earn a place at the bedside. It runs in three stages: retrospective development and internal validation. (which is where our published model sits) Then prospective silent validation, confirming the score stays accurate on new patients while clinicians cannot yet see it; and only then a clinical-impact trial, which asks the different question of whether displaying the score, or acting on it, actually changes what happens to the baby. An AI model is a retrospective simulation. It looks back over years of saved oxygen-saturation and heart-rate data, replays them minute by minute, and asks how often the alarm would have fired before the clinician noticed anything. The answer we know, but a replay is not a trial. Care has not changed. No baby was treated differently because the number went up.

Where this episode stands is the next step, the clinical-impact trial, the patient outcomes, the very thing a good model cannot deliver on its own, and it is where almost nobody in this field has climbed. To understand what it costs come there, it helps to look at the one team that did.

HeRO(‘s) trial

Between April 2004 and May 2010, across nine American NICUs, 3,003 very low birth weight infants were randomised to have a continuously updated heart-rate risk score either displayed at the bedside or recorded but hidden (Moorman et al., 2011). The score was the HeRO index: a number expressing the fold-increase in the risk of sepsis over the next 24 hours, derived from subtle changes in heart rate, reduced variability and transient decelerations, that tend to appear before a preterm infant looks clinically septic. It is not our model. But it is the same species of thing: a continuous physiological risk score, updated hourly, sitting on a monitor next to a sick baby.

The result that everyone remembers is a mortality reduction. Infants whose score was displayed died less often, 8.1% versus 10.2%, a hazard ratio of 0.78 and an estimated number-needed-to-monitor of 48. In the pre-specified subgroup of infants below 1,000 grams, the estimated benefit was larger, falling from 17.6% to 13.2%, a number-needed-to-monitor of 23. For a device that changes nothing about the baby except what the clinician can see, that is a striking finding, and it is the reason HeRO is the reference point for this entire conversation.

What the trial actually measured

There is a wrinkle worth being precise about and it matters more here than anywhere.

Mortality was not the trial’s primary outcome. It was a secondary one. The registered primary endpoint was a composite, the number of days a baby was alive and off the ventilator in the 120 days after randomisation, and on that endpoint the trial did not reach significance. Displayed-score infants gained an average of 2.3 such days, with a p-value of 0.08. By the trial’s own pre-specified terms, that is a miss.

So the canonical trial in this field is, read strictly, both positive and negative at once: it missed on the outcome it was built to measure and hit on one listed beneath. This holds two possibilities open at the same time. A composite can obscure a real mortality effect if its components move in different directions, the ventilator half adding noise that drags the whole below the threshold. Equally, a significant secondary finding after a negative primary endpoint can itself be a chance result, and mortality was one of several secondaries. The confidence interval for the mortality benefit only just excluded no effect; the p-value was 0.04. A Dutch systematic review weighing exactly this rated the trial’s risk of bias high, on performance and detection bias and the absence of correction for multiple testing, and concluded that the evidence does not justify putting HRC monitoring into routine care without another trial (Koppens et al., 2023). None of this makes the finding worthless. It makes it what it is: clinically important, internally coherent, and statistically less secure than its prominence in the later literature suggests.

The point that survives either reading is the one that matters most for designing the next study: how much rides on a decision made before a single baby is enrolled, which endpoint you name as primary. The HeRO team reasoned their way to a composite that seemed clinically sensible in 2004. Whether that was the right call is still arguable, which is precisely why the endpoint is the first thing a validation study has to get right, and the thing hardest to know in advance.

The counterfactual problem, which is the real difficulty

The HeRO trial deliberately did not tell anyone what to do. Clinicians were taught what the score meant and told that a rising number should prompt them to look at the baby and consider tests, but no specific action was required by the protocol. The authors call this, in their own discussion, “a debatable weakness.” I would put it more strongly: it is the exact gap this whole series is about. The trial tested the display of a number. It did not test a defined response to that number. If the mortality effect was causal at all, it must have operated through nine units’ worth of clinicians each responding, in their own way, to the score in front of them.

And you cannot cleanly measure what that response was worth. To know whether the alarm helped, you would want to know how much earlier sepsis was caught, but the biological moment a preterm infant’s sepsis begins is unobservable; there is no clock that starts. So time-to-onset, the thing you would most want to anchor to, cannot be measured directly, and the trial says as much: the mechanism of the benefit could not be proven, only inferred.

That is not a counsel of despair. A new trial cannot see the true onset of sepsis, but it can measure the operational chain the alert is meant to shorten: time from alert to bedside assessment, from alert to blood culture, from alert to antibiotics; physiological deterioration in the hours after an alert; escalation to organ support; adherence to whatever response protocol is specified. None of that pins down biological onset, but it would make the mechanism far less opaque than it was in 2011.

What the 2011 team could show is indirect, and it is worth laying out because it is the texture a real impact trial lives in:

• The amount of proven sepsis was the same in both arms. Displaying the score did not catch more sepsis: 358 affected infants in the displayed group, 379 in the control group, no meaningful difference.

• Among the babies who did develop proven sepsis, mortality in the following 30 days was lower where the score had been displayed, 10.0% versus 16.1%. The score did not find more infections; the most that can be said is that it may have helped clinicians act sooner on the infections already there, and “helped” is an interpretation the design cannot confirm.

• That came at a cost, though a less certain one than the mortality figures. Displayed-score infants underwent slightly more blood cultures, a difference of borderline significance, and accumulated slightly more antibiotic days, a difference that was not statistically significant at all.

“Act sooner on the infections already there” is as far as the strongest trial in the field can take the claim. That is not a failure of the trial. It is the limit of what a trial of this design can prove, and it is the same limit any future study, including any study of our model, would run into.

So what would a real trial of our model take

Lay the HeRO experience against our 2023 paper and the shape of the required study comes into focus. It is sobering.

• A response protocol, not just an alarm. Our published alarm framework, an eight-hour refractory period and escalating thresholds, defines when the number speaks, not what the clinician does when it does. If the goal is a validated response, and not simply another displayed number, that response has to be written, agreed and fixed before enrolment, so that the thing under test is the whole pathway from alert to action.

• An endpoint chosen in full view of HeRO’s split result. Days-alive-and-ventilator-free is a good composite; it also failed to reach significance, while the mortality signal that did reach significance was fragile. Neither is a template; both are a warning about how much the choice decides in advance.

• A randomisation design that reckons with contamination. HeRO randomised individual babies, which is clean statistically but leaky in practice: the same nurses and doctors care for both arms, and a nurse or clinician who learns to read the score cannot unlearn it on the hidden babies, they are much smarter than we think. Randomising whole units, or phasing the score in unit by unit over time, can reduce that cross-arm contamination, but at the cost of far larger samples and vulnerability to centre effects and secular trends. It is a trade-off, not an upgrade.

• A scale no single centre commands. HeRO enrolled 3,003 infants across nine units over six years and still produced a mortality estimate whose confidence interval only just excluded no effect. A future trial’s size cannot simply be read off HeRO; it depends on the endpoint, the baseline risk, the expected effect, the clustering and the adherence, and on whether the intervention is the score alone or the score-plus-response bundle. What is certain is that the arithmetic points away from any one hospital, and our model was built in one hospital, on one hospital’s data.

• The onset you still cannot see. Even a perfectly designed, fully funded trial inherits the unobservable start of sepsis. Measure the operational chain above and the mechanism grows clearer; it does not become transparent.

None of this is a reason not to do it. It is a description of what doing it properly costs, and of why the distance between a good model and a proven pathway is measured in years and thousands of patients rather than in another analysis of the data already in hand.

A note on how HeRO reached the bedside anyway

Worth tethering back to Episode 5’s regulatory thread: HeRO did not wait for the trial to reach patients. It entered the trial already 510(k)-cleared, for its intended use as an ECG-derived measure of reduced heart-rate variability and transient decelerations, not for diagnosing sepsis and not for improving survival. That clearance rested, as the 510(k) route does, on substantial equivalence to an existing device and on adequate safety and performance within that stated intended use.

A 510(k) is the main US regulatory route through which many medical devices receive FDA clearance. The name refers to section 510(k) of the Federal Food, Drug, and Cosmetic Act. Rather than proving clinical benefit from scratch, the manufacturer generally has to show that the device is substantially equivalent to a legally marketed “predicate” device: it has the same intended use and comparable safety and performance. Clearance therefore does not necessarily mean that the device has been shown in a randomized trial to improve patient outcomes.

The randomised trial addressed a separate question the clearance never touched: clinical utility, whether showing the number to a clinician at three in the morning changes the night for the better. Regulatory clearance and clinical benefit are different claims, established by different means. Only the benefit claim necessarily calls for a clinical-impact trial, though what regulators demand in practice varies with the device, its intended use and the jurisdiction.

The second question, hiding inside the first

Even HeRO leaves its own question open. When the smallest survivors were followed up at 18 to 22 months, the composite outcome of death or neurodevelopmental impairment was not significantly reduced, roughly 39% in the displayed group against 44% in controls, a relative risk near 0.87 that did not clear significance (Schelonka et al., 2020). Two caveats matter. The outcome was available for only about 72% of eligible infants, and attrition on that scale limits what can be concluded. And the analysis is of a composite of death or impairment, not of the survivors themselves; asking whether the extra survivors were better off means conditioning on survival, which brings its own selection problem. What can fairly be said is narrower: the follow-up did not show that the mortality benefit was matched by an improvement on the death-or-impairment composite. (A later analysis did find a benefit in the narrower subgroup of extremely preterm infants who developed sepsis, King et al., 2021, a real but subgroup-level signal a single trial cannot settle.) So the question recurses: the study that suggested displaying the score saves lives raised a further one, did improved survival come with improved, or at least not worse, neurodevelopmental outcomes, that it was not built to answer.

Why, fifteen years on, there is still only one

Before leaning on the word “only,” I went looking for a second, and so had others. The Dutch systematic review already mentioned searched four databases and found fifteen papers on HRC monitoring in preterm infants; across all of them, exactly one randomised trial with clinical outcomes, the HeRO trial, reported three times (Koppens et al., 2023). My own search of the trials registry turned up no second completed randomised outcome trial. The nearest newer entry is an observational, model-development pilot rather than a second impact trial (NCT07254559).

One trial and many second looks into the same dataset, the septicaemia-mortality question (Fairchild et al., 2013), the neurodevelopmental follow-up, whether the score performed equally across Black and White infants (Sullivan et al., 2023). It stands alone not because the question stopped mattering, the same review calls plainly for a large international RCT, but because a trial like this is enormous, slow and hard to fund, and because a result positive enough to be quoted let the field feel the question was answered when it was answered only halfway.

That is the trial that would take. The number can already tell you the baby’s risk has risen. One large randomised trial suggests that displaying this particular score may reduce mortality, a real and hard-won signal, but a single fragile result, not a settled clinical pathway. What no trial has yet shown, for HeRO and still less for a single-centre model built on a replay, is which move at the bedside, at three in the morning, turns a rising number into a better morning. Until a study is built to answer that, the model is a very good question still waiting for an answer.

How I used Claude and ChatGPT

The argument, the framing and every editorial decision here are mine, including the choice to treat the endpoint correction as a sidebar rather than the spine, to keep the FDA point to a single paragraph, and to fold the neurodevelopmental follow-up in as a second open question rather than let it take over.

I used Claude to interrogate and cross-check the papers: it surfaced the primary-versus-secondary-endpoint distinction I had been carrying loosely, verified the figures and the systematic review’s risk-of-bias findings, and ran the registry and literature searches behind the “only one” claim, flagging the newer observational entry rather than letting the tidy version stand. The interpretation, and the responsibility for it, are mine.

ChatGPT made the infographic and reviewed the manuscript for content and readability, removed the em-dashes and proposed technical adaptations.

Next: with the trial mapped, the question is who would ever run it, and what it means to build toward a validated response protocol when the field’s incentives reward the next model over the missing trial. That is Episode 7.

Regulatory sentences are easy to walk past because they read like housekeeping. You build a model, you report an AUC, and somewhere near the end a reviewer-pleasing sentence acknowledges that turning this into something used on real babies would mean engaging with European device law. Cite the regulations, nod, move to the conclusion. I have done exactly this. It’s not dishonest, but it does quietly imply that the regulation is the last, dull step, A lot of paperwork between a finished model and the bedside.

Yet, it isn’t the last step. It’s a different question entirely, and it’s the one the whole series has been circling.

The sentence I should have sat with

Our discussion makes an argument I now think is the most important thing in the paper, and it’s not the AUC. It says, roughly, that to build a medical device you have to specify an intended use together with a performance claim — and the performance claim has to be good enough to identify any hazard that comes from using the device in practice.

Read that again with Episode 4 in mind. We spent a whole episode on the alarm policy and the precision problem, the finding that at the operating point we chose, most alarms are not followed by sepsis. In the research frame, that’s a PPV number, a point on a curve, a thing you note and contextualise. In the regulatory frame it is not a number. It is a hazard. A false alarm that triggers a workup, or antibiotics, or a clinician’s decision to treat is over-treatment, and over-treatment is precisely the kind of harm the performance claim is supposed to be sensitive enough to catch. The regulation takes the thing we treated as a metric and reclassifies it as a risk to a patient.

That reframing is the whole episode. Everything below is consequence.

Which regulation, and why it matters

The first thing to settle is which regulation even applies, because Europe has two and they classify by completely different logic. The in-vitro diagnostic regulation governs devices whose medical purpose depends on examining specimens — a sample taken from the body, run through a test. The medical device regulation, the MDR, governs everything else, including software that stands on its own.

The word “diagnostic” pulls the reflex toward the in-vitro side, sepsis, blood cultures, diagnosis everywhere. But our model never touches a specimen. It runs on continuously monitored physiology and data already in the record, and produces a score that informs a clinical decision. That is software as a medical device, squarely under the MDR. The in-vitro regulation, despite the diagnostic subject matter, simply isn’t the instrument that governs a thing like this.

This isn’t pedantry. Which regulation you’re under decides everything downstream, what evidence you owe, whether an independent body has to audit you, how long it takes, what it costs. So: software, under the MDR. What does the MDR do with software that informs a clinical decision?

Rule 11, and why this lands high

The MDR has a single rule — Rule 11 — that classifies software, and it is notorious for pushing almost everything upward. The logic runs by worst credible harm, not by how often the software is right. Software that provides information used in diagnostic or therapeutic decisions starts at Class IIa. If the decisions it informs could cause death or irreversible harm, it’s Class III. If serious deterioration, Class IIb. And there’s a second limb: software that monitors vital physiological parameters, where the nature of the variation in those parameters could put the patient in immediate danger, lands at IIb.

Walk our model through that. It monitors continuous vital-sign physiology. The variation it watches for is the early signature of a baby becoming septic, a state where deterioration can be immediate and severe. The decision it informs is whether to start antibiotics in a preterm infant, where error in either direction has consequences. On a conservative reading, this is not low-risk software. Class IIb is a very plausible landing point; depending on the exact intended use and performance claim, a Class III argument is not impossible.

Class IIb is not Class I. Class I you can largely self-declare. From IIa upward an independent notified body audits your quality system and reviews your technical file before anything is allowed near a patient. The thing that pushes you over that line is not the sophistication of the model. It’s the severity of the decision it touches. A simpler, more interpretable model with the same intended use lands in exactly the same class, a point that should sound familiar to anyone who read Episode 3.

“It works in a simulation” is the start of the conversation

There’s a story that should be taught alongside every sepsis-prediction paper, and it isn’t ours. The Epic Sepsis Model was built into one of the most widely used electronic record systems in the world and deployed across hundreds of American hospitals, about as far from “research artifact” as a model can get. Then an external team at Michigan validated it independently and found it missed roughly two-thirds of sepsis cases at the threshold in use. Deployed everywhere; externally validated almost nowhere until someone finally checked.

That gap — between deployed and validated — is the gap the regulation exists to close. And it’s the gap our own staged plan, the one I cited approvingly in the paper, is built around. We named the path: prospective validation first, then a clinical validation study, then the hard outcomes, mortality, sepsis episodes, antibiotic days. That ladder isn’t bureaucratic decoration. Each rung is a different claim about the world, and the regulation is essentially asking you to climb it in order and show your work at each step. A retrospective AUC, however good, is a claim about a dataset. It is not yet a claim about a baby.

The model in this lineage that actually climbed the ladder is the old one. The heart-rate-characteristics work I traced back in Episode 1 ran a genuine prospective randomised trial — 3003 very-low-birth-weight infants across nine NICUs, with a reduction in mortality when the monitor was displayed, published in 2011. Fifteen years later it remains the canonical prospective randomised trial in this literature. Much of the field since, ours included, has stopped at the rung marked “good ROC curve.” The regulation is, in a sense, just the formal version of the question that trial answered and the rest of us have been deferring: does using this change what happens to the patient?

The ground is moving while I write this

I should be honest that the regulatory picture in 2026 is not settled, and a research diary should say so rather than pretend the law is a fixed monument. The AI Act now adds a second layer: an AI medical device that requires notified-body assessment will generally fall into the high-risk AI category, which our model, on the reading above, would. That brings requirements around data governance, representativeness of the training data, transparency, human oversight, logging, and post-market monitoring. The direction of travel is toward avoiding two wholly separate assessments and integrating the AI Act requirements into the existing product-conformity architecture where possible, rather than running them in parallel. The exact timing and mechanics remain in flux: the high-risk AI timelines and a December 2025 proposal to simplify the MDR/IVDR framework, which secondary commentary reads as touching the software-classification logic, including Rule 11, are still moving through the system.

So the exact procedural path may move, and the specific class boundaries with it. What won’t shift is the underlying logic: severity of the decision drives the obligation, and deployment is gated on evidence the field has mostly not produced.

I’m flagging this partly for accuracy and partly because it matters for the talk in Lyon. The details will have moved again by October. The argument won’t.

Back to 3am

I ended the last episode at a bedside: a score crosses a threshold in front of a clinician at three in the morning, and the only question that matters is whether that becomes earlier treatment or just one more thing beeping.

The regulation is what stands between those two outcomes, and I’ve spent this episode learning that I’d been treating it as the wrong kind of thing. Not paperwork after the science. A second, harder science — the one that asks not “can the model tell true from false” but “when this fires at 3am and someone acts on it, is the baby better off.” Our paper has an answer to the first question and, honestly, an IOU on the second.

Next episode I want to follow that IOU somewhere specific: what a real clinical validation of a model like ours would actually have to look like, and why almost nobody has done one.

How I used Claude

The regulatory reading in this episode is not my home turf, and that shaped the division of labour. I asked Claude to pull the current state of the EU device rules — the MDR classification logic, the software rule, the in-vitro versus medical-device distinction — and the 2026 developments around the AI Act, because this is exactly the kind of material that dates fast and that I’d otherwise half-remember from a conference slide. Claude surfaced the Rule 11 mechanics, the Epic Sepsis Model external-validation figure, and the live status of the AI Act overlay and the late-2025 simplification proposal.

The argument is mine. The decision to open on our own paper’s “intended use plus performance claim” sentence, and to read the false-alarm rate from Episode 4 as a regulatory hazard rather than a metric, was the spine I wanted before any research happened; the regulation just gave it teeth. I had Claude check the Rule 11 classification reasoning against current guidance rather than take my reconstruction of it on trust, because a confidently wrong regulatory class is worse than no class. Where the law is genuinely unsettled in 2026, I’ve tried to say so plainly rather than launder uncertainty into authority — and that paragraph got read adversarially before it went in.

Next: Episode 6 — what a real clinical validation of a model like this would have to look like, and why, fifteen years on, that 2011 trial still stands largely alone.

For three episodes I have been reading through other people’s papers. I mapped the field, then read four of its anchor models side by side and argued that the two decisions that matter most in any of these systems, which signals you give the model, and how you turn its scores into alarms, are the two decisions almost no one documents clearly.

It would be too easy to leave the argument there. The most credible place to test it is a paper I cannot hide behind a citation: my own.

So this episode turns the same reading onto van den Berg et al. 2023, the WKZ Utrecht late-onset sepsis model I am senior author on. The same two decisions — signal stack and alarm policy — with the candour raised, because I owe it to you that after the last episode, and because the limitations of our own paper point directly to where this series goes next.

I will state the obvious first, so it is not mistaken later for false modesty: I think it is good work. It was done by incredibly smart people from our AI for health department and many more before that in the ADAM program. It is the largest unselected single-centre cohort in this corner of the literature, and it carries the most extensive retrospective impact simulation anyone has done for LOS prediction to date. I stand behind both claims. But “good work with real limitations” has two halves, and the second is the reason for this episode.

The number we didn’t put in the abstract

The abstract reports that the algorithm detects LOS in at least 47% of patients before clinical suspicion, without exceeding an alarm-fatigue threshold of three alarms per day. That is the sentence most readers take away, and the one most likely to be quoted.

At that same operating point, the precision is 3.93% on the training set and 4.41% on the test set. This number is not in the abstract.

A precision of 4% means that of every hundred alarms the system raises, roughly ninety-six are false, and not at some reckless sensitivity setting, but at the operating point we chose precisely because it matched the alarm burden a NICU already tolerates. The defence the paper offers is genuine and I still hold to it: three alarms a day is approximately how often the WKZ team already considers sepsis during routine care, so the system adds little net burden even when most alarms do not pan out. We did a small (unpublished) study where we found that 80% of the documented sepsis evaluations did not result in a blood culture. A bedside clinician already works with a high base rate of “check, probably nothing.”

Still, the asymmetry in the writing is worth naming. A detection fraction went into the abstract; the false-alarm rate stayed in Table 3. They come from the same alarm configuration, but they summarize different aspects of it: the fraction of LOS episodes flagged before the culture, and the positive predictive value of the alarms generated by that policy. We foregrounded the more flattering summary. That is not data manipulation — every figure is in the paper for anyone who reads as far as the table — but it is a framing decision, and an honest account of the paper has to treat it as one. If “prediction is not utility” means anything, it means the 4% deserves at least as much prominence as the 47%.

Signal stack: the strength and the limitation are one decision

The first of the two decisions is what the model is allowed to see.

Heart rate and oxygen saturation, and nothing else. Low-frequency, one sample a minute, the values that scroll across every monitor on the ward. We deliberately set aside the inputs that would have raised the AUC: CRP, blood pressure, white cell counts. The reason was principled. A clinician orders a CRP only when they already suspect something, so a model trained on CRP learns to detect clinical suspicion and then claims credit for predicting it. We were strict about excluding anything carrying that prior knowledge, and I would defend that strictness without hesitation. It is the best decision in the paper.

It is also the source of two of the paper’s hardest limitations, and the point worth dwelling on is that the strength and the limitations are not separate items to be weighed against each other. They are one decision seen from two sides.

Because the model keys on physiological deterioration alone, it cannot say why a baby is deteriorating. The paper states this directly: the algorithm may not be specific to sepsis and could also respond to other inflammatory conditions such as NEC. A model built on the body’s distress signal will respond to distress, whatever its cause. We call it a sepsis model; it is more accurately a deterioration model aimed at a sepsis question.

The same minimalism is part of why the precision is so low. With two physiological streams and no specific marker, there is a hard ceiling on how cleanly the model can separate the baby who is becoming septic from the one who is briefly unstable for any of a dozen ordinary reasons. The interpretable seven-feature logistic regression is not underpowered for want of a better architecture, the supplement shows XGBoost and GAMs reach the same performance, with no significant difference between them. It sits at that ceiling because the signal stack we chose, for good reasons, cannot see past it. The principled choice and the (disappointing) precision are the same fact stated twice.

Alarm policy: an operational convenience that became a convention

The second decision is the one I now think we explained least, and it is the one Episode 3 identified as under-documented across the whole field.

A model produces a score every hour. A score is not an alarm. Turning one into the other requires a policy, and ours had three parts:

• A refractory period. Once an alarm fires, further alarms are silenced for a set window, so a stably high-risk baby does not alarm every hour. We set that window to eight hours, on the stated reasoning that eight hours is the length of a nursing shift at the WKZ NICU.

• A true-positive window. An alarm counts as catching the sepsis if it falls anywhere from 24 hours before to 12 hours after the blood culture.

• A multi-threshold escalation rule, in which a higher-risk alarm can break through the silence imposed by a lower-risk one.

The justification for the refractory period is the revealing part. Eight hours, because that is a shift. This is an operational convenience, a reasonable one for a working ward, but it is not a clinically derived parameter. No one demonstrated that eight hours is where early detection and alarm fatigue trade off best. It is where the staff rotation happens to fall.

Whether that matters is something the supplement lets us check, and the answer is that it does, moderately. Varying the refractory period to 4 or 12 hours (Tables S7 and S8) moves recall and precision, not dramatically, but enough to show that the headline figure depends on a number chosen for rostering reasons. The true-positive window matters more. Our default window spans 36 hours and counts an alarm up to 12 hours after the culture as a successful early warning. Restrict it to the stricter interval from 12 hours before the culture to the culture itself and multi-alarm recall falls from the high-50s and 60s into the high-30s to mid-50s (Table S3). The “at least 47%” is real, but it rests on a generous definition of what counts as catching the episode in time.

This is more than an internal quibble because the policy did not stay internal. Episode 3 documented that this alarm framework, the eight-hour silencing, the multi-threshold escalation, the alarms-per-patient-day accounting, was taken up by other groups. Yang 2024 cites our paper and reuses both the eight-hour period and the escalation logic; Meeus echoes the alarm-burden accounting. I presented that propagation in Episode 3 as a point of pride. Seen from the other direction, though, a refractory period justified by our shift pattern has propagated into other people’s models as a convention. The element we documented least rigorously is the one the field borrowed most directly. That ought to make all of us cautious about how much of a shared methodology is in fact a shared assumption.

The proxy at the centre of everything

Beneath both decisions lies a single substitution that the whole impact story depends on.

We do not know the moment a clinician first suspected sepsis; it is not reliably recorded. So we used the timestamp of the blood culture as a stand-in for that moment “t = 0”. Every claim about detecting LOS before clinical suspicion is, more precisely, a claim about detecting it before the blood culture was drawn.

The paper concedes this directly: because actual suspicion could arise earlier, using the culture time as the proxy can lead to an overestimation of the early-warning performance. It is the cleanest limitation in the paper. The entire early-warning proposition rests on a proxy the authors themselves flag as probably flattering. If a clinician was already uneasy an hour before drawing the culture, part of our measured lead time is an artifact of when the culture was recorded, not of when the model spoke first.

There is a smaller version of the same problem in the data. A number of blood cultures arrived timestamped at exactly midnight, an artifact of records where the time was missing and defaulted to 00:00. We imputed those to noon, closer to morning rounds, less likely to be far off. It is a defensible fix, and we confirmed that dropping the contaminated records did not change the results. But it means that a portion of our t = 0 anchors, the reference point the whole simulation turns on, are reasoned estimates of what the clock read.

While re-reading the supplement for this episode I also found something I cannot fully explain. The train/test baseline table reports a CRP-above-10mg/L rate in the control group of roughly 80%, against 8.5% in the main-text table — a figure that cannot be right for a control group, and one the page appears to print twice with differing control numbers. I cannot reconstruct from the published PDF exactly what went wrong in producing that table. It is probably proofing fault, but I would be a poor critic of my own paper if I only noticed the errors that flatter it.

One further definitional point quietly inflates the false-alarm count. We labelled only culture-positive cases as sepsis. Culture-negative “clinical sepsis”, babies treated as septic who never grew an organism, sits in the control group. When the model fires on one of them it is scored as a false positive, even where a clinician might have agreed with it. So some unknown fraction of the 96% of “false” alarms may not be false. That cuts in our favour, but a muddied outcome definition is muddied whichever way it leans.

Simulation is not impact

This brings me to the claim I am at once most proud of and most wary of. We described our longitudinal alarm analysis as the most extensive clinical impact assessment in the LOS prediction literature to date.

It is worth being clear about what kind of object it is. It is a retrospective simulation run on historical data, the model applied to records of babies whose outcomes were already settled, asking what the alarms would have done. No clinician saw a score. No decision changed. Nothing that happened to those babies happened because of this model.

The paper’s final move is its most honest. Listing what a real evaluation would require, the authors note that counterfactuals — would this sepsis have been caught later without the model? — are not measured. That is the thesis of this entire series, stated in our own paper, in our own words, two years before I began writing these episodes. We demonstrated prediction. We simulated impact. We did not show utility, and we said so.

That gap, between the most extensive impact simulation and actual clinical impact, is not a flaw in the paper. It is the frontier at which the paper honestly stops. Every model in Episode 2’s field map stops at roughly the same line. The open question is no longer whether we can predict LOS early; the field has answered that many times over, ours included, within an AUC band everyone now clusters in. The question is what happens when one of these systems is placed in front of a clinician at three in the morning and a score crosses a threshold, whether the early warning becomes earlier treatment, or simply one more thing beeping.

That is deployment. And deployment, in Europe, is where the model stops being a research artifact and becomes something the regulator has views on. Berg 2023 gestures at this already: two footnotes in the discussion point to the EU Medical Device and In-Vitro Diagnostic Regulations, almost in passing, as the frame to which any performance claim would eventually have to answer.

We dropped those two footnotes and moved on. Episode 5 will pick them up and not move on.

How I used Claude

For this episode the work was reading rather than searching. I gave Claude the Berg 2023 paper and its supplementary materials and asked it to build the critique against the primary source instead of from my memory of my own paper. That was the right instinct, because reading the supplement closely brought back things I had half-forgotten. The sensitivity of the headline recall to the true-positive window, the comparison in Table S3 between the strict and generous intervals, is set out in the supplement, and having it in front of me again sharpened the alarm-policy section. Claude also pointed out the implausible control CRP figure in the supplementary baseline table, which I had never noticed.

The structural decision, to organise around the two decisions from Episode 3 rather than re-run all eight dimensions, was mine, and I want to be exact about that, because this section is the easiest place in the series to quietly overstate what the tool did. Claude set out both options and argued for the two-decisions structure; I chose it because we had already done the eight-dimension grid in Episode 3 and repeating it would have read as padding. The judgements about what counts as a real limitation versus a defensible choice are mine, including the uncomfortable one about what we put in the abstract.

Next: Episode 5 — the model leaves the lab. What European medical-device regulation actually requires of a system like this, and why “it works in a simulation” is where the conversation begins rather than ends.

Laat een reactie achter

In Episode 2, I mapped the field and came away with three claims. One of them was technical convergence, across every group doing continuous-physiology machine learning for late-onset sepsis in preterm infants, the headline AUCs land between 0.78 and 0.88. I called the convergence real and said it wasn’t a methodological artefact.

I want to take that back a little. Not because the numbers are wrong, they’re right. But because lining four AUCs up in a row and calling them a convergence quietly implies they are four readings of the same thing. They are not. This episode is the close read that shows why and it lands on a more useful question than “how high is the AUC.”

I promised at the end of Episode 2 that this one would go deep on two decisions: the signal stack and the alarm policy. I’m keeping that promise, because after reading the four anchor papers side by side, I’m convinced those two decisions matter more than anything in the modelling and they are exactly the two things we need to discuss.

The four papers, for reference: our own from WKZ Utrecht (Berg 2023), the multi-NICU study from UVA (Kausch 2023), the Eindhoven/Máxima paper (Yang 2024), and the Antwerp/Innocens paper (Meeus 2024). Full eight-dimension comparison is on the companion page; here I’ll walk the argument.

The first decision: which signals, how fast, from what

Before any machine learning happens, you choose a signal stack. It sounds like a technicality. It is the single most consequential design decision in the whole field, and it’s really three decisions wearing one coat: which signals, sampled how fast, from which device.

Two of the four papers contain a finding on this that I think is a bit underrated, and in both cases it’s a sentence or two that the paper itself doesn’t dwell on.

The first is from Kausch. They built their model, POWS, and then asked whether it mattered if the heart rate came from an ECG or from a pulse oximeter. It didn’t. Performance was within about 0.01 AUC either way. That sounds like a footnote. It isn’t. It means a continuous-physiology sepsis model can run on a standalone pulse oximeter, no ECG leads, no specialised monitor. For a well-resourced NICU that’s a convenience. For everywhere else, it’s the difference between a deployable tool and a research curiosity. The most important sentence in the paper for global deployment is a sensitivity analysis.

The second is from Yang, they ran a sampling-rate ablation: the same pipeline fed raw waveforms, then 1 Hz vitals, then one sample per minute, then one per hour. Raw waveforms scored 0.886. One Hertz scored 0.875, basically the same. One sample per minute, 0.825. One per hour, 0.687, the signal essentially gone. Two lessons fall out of that. You do not need waveform processing; minute-resolution numerical data, the kind any modern monitor already exports, gets you almost all the way. But once you summarise to hourly aggregates, the signal collapses. So there is probably a floor, minute-by-minute, and it is now empirically settled where it sits.

Put those together and the signal-stack picture is clearer than the AUCs suggest. More channels is not better. Yang uses three signals, Kausch two, and they bracket the field. Meeus uses seven, heart rate, respiration, oxygen saturation, perfusion index, temperature, FiO₂, glucose, and it does not buy a cleanly higher comparable number. What moves performance is sampling resolution and a well-chosen feature like Kausch’s heart-rate–oxygen cross-correlation. Not the length of the signal list.

The AUCs are not measuring the same thing

Here are the four AUC headline numbers. Berg: 0.79. Kausch: about 0.79 on external validation. Yang: 0.875. Meeus: 0.93, statistically significant, for what that matters. In Episode 2 I let those sit together as a band. Read the papers and the band falls apart, not because any number is wrong, but because each one answers a different question.

• Berg’s 0.79 is a cross-sectional AUC at the moment of clinical suspicion, on a whole-NICU population of every infant 32 weeks or under, with culture-positive sepsis as the outcome. (For honesty’s sake: our cross-sectional figure is 0.73 on the training set and 0.79 on test — at or below the floor of the band I quoted in Episode 2.)

• Kausch’s 0.79 is a prediction of sepsis within the next 24 hours, on a VLBW-only cohort, a narrower, sicker slice, and crucially it is an external number, the model tested at hospitals it was not trained on. (!)

• Yang’s 0.875 is measured 6 hours before the sepsis workup, on a cohort of 119 infants that was deliberately cleaned to clear cases and clear controls.

• Meeus’s 0.93 is for a model predicting late-onset sepsis and necrotising enterocolitis jointly, not LOS alone, and it is computed on cross-validated time windows, a metric the Meeus paper itself flags as optimistic on heavily imbalanced data.

Four different outcomes, or horizons, or cohorts, or evaluation units. The numbers being close is a coincidence of four separate measurements, not agreement between four readings of one quantity. “Similar, but not the same” is the honest description, and it’s why the subtitle of this episode is what it is.

So if AUC won’t carry a fair comparison, what will? The metric that survives translation between these papers is a clinical one: what fraction of sepsis episodes does the model catch before the clinician does. On that metric the picture is consistent. Berg detects roughly 47 to 60 percent of patients before clinical suspicion. Meeus catches 69 percent of all episodes and 81 percent of severe ones, with a median head start around 10 hours. Kausch sees risk climb significantly 23 to 24 hours ahead. Yang reports 96 percent, but patient-wise, on 119 infants, with a specificity of 19 percent, so it belongs in a different bucket than the others.

Set Yang’s small-cohort outlier aside and the field converges on something like half to two-thirds of late-onset sepsis episodes detectable hours before clinical recognition. That is the real state of the art. It is a less impressive number than “AUC 0.9,” and a far more honest one. It is also the number a clinician can actually reason about.

The second decision: the alarm policy

Say you have a model. It emits a risk score every hour for every baby on the unit. When does a clinician get told? That rule, the alarm policy, is what converts an AUC into something a human being at a bedside actually experiences. And it is chosen, across these four papers, in four genuinely different ways, mostly without much justification for why.

• Berg generates hourly scores and sets thresholds to fixed unit-wide alarm rates. A new alarm is muted for an 8-hour refractory period, the length of a nursing shift at our NICU, unless a higher threshold is crossed, in which case it escalates. False alarms are reported as alarms per patient-day.

• Kausch uses a lighter rule: an alert switches on at a threshold and stays on until 24 hours pass with no further crossing. No refractory period, no escalation tiers.

• Yang adopts Berg’s framework more or less wholesale, same 8-hour silencing, same multi-threshold escalation.

• Meeus aggregates hourly predictions into 24-hour buckets and reports alarm-days per week, which echoes Berg’s alarm-burden accounting without the refractory machinery.

Here is the point that matters. The detection-fraction headlines I just quoted, Berg’s 47 percent, Yang’s 96 percent, are not, mostly, a gap between models. They are a gap between alarm policies and cohorts. Change the refractory rule, the threshold scheme, the true-positive window, and the same underlying model produces a very different headline. The alarm policy is the highest-leverage decision in turning a model into a deployable tool, and it is the one most often relegated to a methods paragraph and never defended.

I’ll claim one thing for our group here, and leave the rest for a later episode. The alarm-fatigue framework from the Berg paper, hourly prediction, multi-threshold escalation, an 8-hour shift-length refractory period, false-alarm burden counted as alarms per patient-day, has quietly become a small convention. Yang adopted it directly. Meeus echoed its accounting. The modelling across the four groups is roughly equivalent; what propagated was a way of evaluating a model for deployment. That’s worth saying plainly. Whether our own paper holds up to a harder look is a question for Episode 4.

What this leaves us with

The two decisions that most determine whether one of these models could ever run at a bedside, what signals you feed it, and how you decide when to alarm, are precisely the two decisions the literature documents least clearly. The modelling, by contrast, is well described, carefully compared, and largely settled: across these papers, architecture barely matters once the features are right.

That inversion is this episode’s reason to exist. We have spent fifteen years getting very good at the part that turned out not to be the bottleneck, and we write almost casually about the two parts that are.

There is a coincidence at the centre of this field that looks like consensus. Four ROC curves, similar but not the same. The convergence is real as an observation and misleading as a measurement, and the more useful question was never “how high is the AUC” but “caught before the clinician, how often, on whom, under what alarm rule.” From here the series turns toward deployment, and toward the European regulatory frame that decides whether any of this reaches a patient.

How I used Claude

This episode is a four-paper close read, and Claude did the structural work of it with me. I uploaded the information form the four anchor papers, plus from the three supporting-cast papers from the Episode 2 map, and we built an eight-dimension comparison across them: cohort design, signal stack, ML methodology, validation rigour, performance, alarm policy, honest limitations, unique contributions. That comparison grid is what this post is distilled from, and it’s published in full on the companion page.

Two moments are worth reporting honestly. First, Claude caught that the handover note I started from described work, a finished comparison sitting in an earlier chat, that it then couldn’t actually find, it wasn’t published yet, it stopped and said so rather than inventing it. We rebuilt the analysis from the papers instead.

Second, I asked it to check the licensing of the four PDFs before we used them, which are open access, which are all-rights-reserved, and it worked through them paper by paper. The short version that came out of that: discussing and comparing papers is unconstrained, redistributing them is not. That’s why this series links to DOIs and hosts no PDFs.

It also caught a detail I’d have glossed: our own Berg paper’s AUC is 0.73 on training and 0.79 on test, at or below the floor of the “0.78–0.88 band” I quoted last episode. That’s not an embarrassment to bury. It’s the cleanest single piece of evidence for this episode’s whole argument.

This post is part of “Road to CEPAS 2026,” a transparent research diary as I prepare a 20-minute talk for the CEPAS 2026 congress in Lyon. Episode 1 traced the HeRO/Moorman lineage; Episode 2 mapped the field. Episode 4 turns the same close-reading on our own work — Berg 2023, warts and all. The full eight-dimension comparison and methodology log for this episode is at neonatology.ai/sepsis/episode-03.

If I’ve misread one of these four papers, or weighted a comparison wrongly, tell me — the whole point of doing this transparently is to fix what I get wrong.

Laat een reactie achter

I’m senior author on one of the more recent ML papers for late-onset sepsis prediction in preterm infants. I collaborate with several of the others doing this work. I thought I knew the field, kept up with literature.

In Episode 1 I traced the lineage from HeRO and the Moorman trial through to the present. The question I’m building toward for CEPAS 2026 is harder: where does the field actually stand right now? Not where the headlines say it is. Where the literature actually says it is.

What follows is the result of a staged literature search with Claude. Five search rounds, roughly 200 abstracts triaged, full text on seven anchor papers, two systematic reviews. The chat log is the source material; this post is what came out the other end.

First surprise: the field has its own dialect

I started where any clinician would. PubMed, “neonatal sepsis” AND “machine learning”, filtered for the last 15 years. 101 papers. The top of the list looked sensible, Kainth’s 2024 systematic review, a handful of clinical-features models, the usual review pieces.

Then I narrowed to “late-onset sepsis” AND “preterm” AND ML. Forty papers. Four obvious anchors: Berg, Kausch, Yang, Meeus. I thought I had the field.

Three more searches to be sure and the actual count was at least seven academic groups, plus a second systematic review I hadn’t seen.

The problem was in the vocabulary. My searches were in machine learning terminology — “machine learning”, “prediction model”, “algorithm”. But a substantial part of the field uses physiology terminology — “heart rate characteristics”, “HRC index”, “HRV”, “visibility graph”. Those papers don’t reliably surface on ML keyword searches, even though they’re doing the same job for the same patients.

The clearest evidence was the discovery of two separate systematic reviews of the same clinical problem, written within months of each other, with almost no overlap in cited literature:

• Kainth et al. 2024 (Pediatr Infect Dis J): 19 studies, pooled AUC 0.94. Reviews ML models using clinical and laboratory features. Explicitly excludes vital-signs-only studies.

• Koppens et al. 2023 (Neonatology, Amsterdam UMC): 15 studies, 8,230 infants. Reviews HRC monitoring for LOS in preterm. This is the vital-signs branch.

The two reviews do not cite each other. They are reviewing the same clinical problem, early detection of LOS in preterm infants, through two different methodological lenses, and the camps don’t talk. Calling them Branch A (continuous vital signs) and Branch B (snapshot clinical and lab features) is the best way to keep them straight. Our paper is Branch A.

Searching the right field requires speaking its dialect. That was the first lesson, and one I wish I’d known when I started Episode 1.

The complete map

After five staged searches and a careful read of both systematic reviews, the active groups in Branch A, continuous-physiology ML for LOS in preterm infants, are:

• UVA / Med Predictive Science (US) — Kausch SL et al. 2023, Pediatric Research. DOI: 10.1038/s41390-022-02444-7. The HeRO heritage; only multi-NICU external validation in the field.

• WKZ Utrecht (NL) — van den Berg M et al. 2023, Computers in Biology and Medicine. DOI: 10.1016/j.compbiomed.2023.107156. Largest single-center cohort; alarm-fatigue framework.

• Eindhoven / Máxima (NL) — Yang M et al. 2024, Computer Methods and Programs in Biomedicine. DOI: 10.1016/j.cmpb.2024.108335. Most thorough feature ablation; Sino-Dutch collaboration.

• Antwerp / Innocens (BE) — Meeus M et al. 2024, Journal of Pediatrics. DOI: 10.1016/j.jpeds.2023.113869. Joint LOS+NEC prediction; only commercial spinoff.

• Karolinska / KTH (SE) — Honoré A et al. 2023, Acta Paediatrica. DOI: 10.1111/apa.16660. Bridges both branches; senior co-authors wrote the Persad 2021 SR.

• Rennes (FR) — Leon C et al. 2021, IEEE Journal of Biomedical and Health Informatics. DOI: 10.1109/JBHI.2020.3021662. Visibility-graph HRV; methodologically distinctive.

• Lausanne (CH) — Rio L et al. 2022, Pediatric Research. DOI: 10.1038/s41390-021-01913-9. Only independent HeRO validation outside the US.

Two synthesis papers anchor the field: Koppens 2023 (Branch A) and Kainth 2024 (Branch B).

A few observations on this map.

Geography. Four of the seven groups are in continental Western Europe, two Dutch centres, plus Belgium, France, Switzerland, and one is in Sweden if you count the Karolinska bridge group. One is in the US. Couldn’t find any in Asia, Africa, or Latin America. I checked specifically for Chinese Branch A work, because the Chinese neonatology literature is data-rich and methodologically capable. The Chinese research bet in this space is firmly on Branch B, lab-feature and clinical-variable nomograms. The continuous-physiology ML problem is, almost entirely, a Western European and American research phenomenon.

Time. The field has gone a bit quiet in 2024–2026. I checked the 184 most recent papers after the Kainth SR cutoff. Almost none are new Branch A LOS prediction work from the major groups. The work has broadened, to NEC, to catheter-related bloodstream infections, to LMIC adaptation, to regulatory questions, but it has not deepened. There has been no major new model paper from any of the seven groups in roughly 18 months.

What every group is finding

This is the part I expected to be messy. It turned out to be cleaner than I thought.

Across all seven anchor papers, different signal stacks (1-channel to 7-channel), different sampling rates (1/h to 250 Hz), different ML methods (logistic regression to deep learning), different cohort sizes (51 to 3,151), AUCs converge on 0.78–0.88 at the moment of clinical suspicion. The single external validation in the field (Kausch trained on UVA, tested on Columbia and St Louis) drops AUC by about 0.03 across centres. The convergence is real, and it isn’t a methodological artifact.

The more clinically meaningful number is what fraction of LOS episodes are detected before clinical recognition:

• Berg 2023: 47–60% of patients detected pre-clinically, using a multi-threshold alarm policy.

• Kausch 2023: significant risk elevation 23–24 hours before blood culture.

• Yang 2024: 96% patient-wise detection before clinical suspicion, but on a small cohort (n=119).

• Meeus 2023: 69% of all episodes, 81% of severe episodes; median time gain 10 hours.

The consensus is this: roughly half to two-thirds of LOS episodes appear to be detectable hours before clinical recognition with current technology. That is the technical answer to the central question of the field.

Two methodological points that matter more than they get credit for:

Yang’s sampling-rate ablation is the most underrated paper in this list. They compared raw waveforms (AUC 0.886) → 1 Hz vitals (0.875) → 1/min vitals (0.825) → 1/h vitals (0.687). The 1/min-to-1/h jump destroys the signal. The 1 Hz-to-1/min jump costs about 5 AUC points. What this tells you is that minute-by-minute data works fine; once you summarize to hourly aggregates, the signal collapses. That’s the floor of deployable signal resolution, and it’s empirically settled now.

Kausch’s pulse-rate-equals-ECG finding is the most underrated result. They showed that POWS performance was within 0.01 AUC whether the heart-rate signal came from ECG or from pulse oximetry. That means a Branch A model can run on a standalone pulse oximeter, no ECG leads, no specialized monitor. For low-resource deployment, that is the difference between a research curiosity and a deployable tool.

What every group is choosing not to do

A consistent finding across the papers that compared multiple ML methods: none of them found that complex models beat simple ones. Berg compared logistic regression, GAMs, and XGBoost, all within 0.01 AUC, chose LR for interpretability. Kausch compared LR with cubic splines, neural networks, XGBoost, and random forest, all within 0.01 AUC, chose LR (”equal performance, better explainability”). Yang compared seven methods including two deep-learning architectures, gradient boosting won; deep learning actually underperformed on their cohort.

This is unusual for an ML field in 2024–2026. It means LOS prediction is not a deep-learning-shaped problem (yet?). It is a feature-engineering problem with a clear physiological signal stack, and once you have the right features, linear and tree-based methods reach the same ceiling. The interesting work is in choosing features, not architectures.

Said differently: the bottleneck in this field is not algorithm sophistication. It’s the things ML alone can’t fix.

Where I think the WKZ contributed

I’ll claim one specific thing for our group, because reading the comparative literature makes it visible in a way it wasn’t when we published.

The Berg 2023 paper proposed a specific framework for evaluating these models the way a clinician would actually experience them: continuous hourly prediction, multi-threshold alarms with an 8-hour refractory period (the length of a NICU shift), and explicit accounting for false-alarm burden as alarms per patient-day. The point was to stop optimizing for AUC alone and start optimizing for what bedside clinicians would tolerate.

Within 12 months, two other groups had adopted the same framework. Yang 2024 cites Berg explicitly as the basis for their multi-threshold alarm policy. Meeus 2023 echoes the alarm-rate accounting. The 8-hour shift-length refractory period and the threshold-escalation rule are now a small but real convention in this corner of the field.

I’m not claiming we solved LOS prediction. The actual modelling work is roughly equivalent across the major groups. What I’ll claim is that WKZ set a convention for how to evaluate these models for deployment, and the field adopted it. That’s worth saying.

The honest gap: the deployment vacuum

Here is what is not in the literature, despite 15 years of work and seven productive academic groups:

• Zero prospective validation studies of any of these models in current clinical use.

• One randomized controlled trial. The Moorman 2011 HeRO trial. Fourteen years old. Mortality benefit ARR 2.07%, NNTM 49 (95% CI 24–15,484). High risk of bias per the Cochrane assessment. The long-term follow-up (King 2021) showed persistent mortality reduction at 18–22 months, but also an unexplained increase in deafness in the HRC-monitored arm (4.4% vs 0.5%), possibly aminoglycoside-related.

• One independent external validation of the only commercial product. Rio 2021, single-centre Lausanne data, showed real-world HeRO performance is strongly gestational-age-dependent: sensitivity 76% in infants below 28 weeks, falling to 25% above 32 weeks. The optimal threshold in their cohort was 2.76, not the FDA-cleared 2.0.

• One commercial deployment outside HeRO: Innocens BV, the Antwerp spinoff. Currently navigating the European MDR/CE certification pathway. Vrijlandt et al. 2026 (Erasmus MC) is the first published paper to think through what that regulatory pathway looks like for paediatric clinical decision support in Europe.

The Koppens 2023 SR lands the policy verdict most directly: “methodological weaknesses and limited generalizability do not justify implementation of HRC in clinical care. A large international RCT is warranted.”

That is the gap. We have seven academic groups, four major model papers, two systematic reviews, one underpowered 2011 RCT, one independent external validation showing real-world drift, and one commercial product navigating European regulation. The technical question “can we predict LOS pre-clinically with ML?” has been answered, “reasonably well”. The deployment question is wide open.

What this means for CEPAS

I came into this search expecting to summarize the technical state of the art. I’m leaving it convinced that’s the wrong talk to give.

The technical state of the art is converged. AUCs at 0.78–0.88. Half to two-thirds of LOS episodes detectable hours before clinical recognition. Feature engineering, not architecture, drives the ceiling. Pulse-rate-only signal stacks work. Alarm-fatigue frameworks are now field convention.

What is missing is everything that turns a model into a deployable clinical tool: prospective trials, international validation, regulatory clarity, deployment infrastructure outside high-income academic centres, and an honest accounting of what real-world performance looks like once you leave your training cohort.

Episode 3 will go deeper on the head-to-head between the major model papers, specifically on signal stack choices and alarm policy design, because those decisions matter more than the ML methodology and almost nobody writes about them clearly. After that, the series turns toward deployment and the European regulatory frame.

If you spotted a group I missed in the map, or have a paper I should have anchored on, please tell me. The whole point of doing this transparently is to fix what I get wrong.

Laat een reactie achter

This post is part of “Road to CEPAS 2026”, a transparent research diary as I prepare a 20-minute talk for the CEPAS 2026 congress in Lyon. Episode 1 traced the HeRO/Moorman lineage; this episode maps the current state of the field. The full chat transcript that produced this post will be available on neonatology.ai.

Abonneer nu

I’ve been quietly uneasy about it ever since.

Not about the speaking part, that’s familiar territory. The unease is about what I’m supposed to say. Because if I’m honest with the audience, and I intend to be, the talk has to begin with a problem most of us in this field have stopped acknowledging out loud.

ChatGPT Image 2

We can predict. So what?

We can predict late-onset sepsis in preterm neonates reasonably well, and we have been able to for years. The literature is full of models. ROC curves climbing toward 0.80, 0.85, occasionally higher with enough feature engineering and a cooperative dataset. That includes — full disclosure — our own paper from 2023 in Computers in Biology and Medicine. AUC 0.73 at the moment of clinical suspicion. A longitudinal impact simulation suggesting we could have flagged 47% of LOS cases ahead of the bedside team, while keeping false alarms below three per day. A genuinely useful exercise in clinical impact assessment, more than most papers in the space at the time.

It’s a good paper. I’m proud of the work. It is also a model that, as I write this in the spring of 2026, is not being used to take care of a single baby. Not in our NICU. Not anywhere.

That is the honest problem. Not can AI predict neonatal sepsis or at least clinical deterioration, that question is largely settled. The honest problem is the next one. Once we have the prediction, what do we actually do with it?

The HeRO question

Consider the HeRO monitor — heart rate characteristics, FDA-cleared, with a randomized trial showing absolute mortality reduction in very low birthweight infants. That trial was published in 2011. Fifteen years on, HeRO is on the market, the evidence is real, and yet ask any group of neonatologists how they respond when the score climbs and you’ll get a different answer at every cot. Order an IL-6? A CRP? Both? Neither? Increase observation? Start empiric antibiotics? Wait? Blood culture?

This is not a HeRO problem. It is the problem of every prediction tool we have built, including ours. We have invested decades into the predict half of the equation and almost nothing into the act half. There is no validated response protocol. No agreed-upon downstream pathway. No randomized comparison of “alarm fires → action A” versus “alarm fires → action B.” The score lights up and the clinical reasoning that follows is, essentially, vibes.

That is the gap I want to talk about in Lyon. Not the success stories. The gap.

What this project is

So, between now and 31 October 2026, I’m going to build the talk in public. Twenty-some weeks. Posts on this Substack, a growing public literature base, an AI-assisted mini systematic review along the way, and the slides themselves at the end. Everything sourced. Everything visible. Nothing dressed up.

I’m doing it with Claude as a research partner. Not as a gimmick, as the actual working method. PubMed searches, evidence tables, critical re-reads of papers (including my own), regulatory landscape mapping, slide logic. Every post will include a short “How I used Claude” section so you can see exactly where the AI helped and where it didn’t. If you want to copy the workflow for your own project, you’ll have everything you need.

The thesis I’ll keep returning to is simple. Prediction is not clinical utility, and the field has not yet built the bridge between them. That bridge is regulatory, methodological, behavioural, and economic. Each of the next posts is a piece of it.

How I used Claude in this post

I drafted the framing and the argument. Claude turned the bullet points of the project brief, the CEPAS invitation, the paper details, the “dal” I find myself in, into structured prose, kept the word count in range, and pushed back on a softer opening I’d originally written. I edited every paragraph. The unease is mine.

What’s next

Episode 2 lands soon: Searching the Evidence — Live. I’ll run a real PubMed search for LOS prediction models with Claude using the PubMed MCP, on screen, queries and all. If you want to follow along or copy the method, subscribe, and meet me in Lyon, virtually or otherwise.

More will appear soon @ neonatology.ai.

NB the images will become more happy along the way