Every few years, one industry becomes the proving ground where cloud computing’s next hard problem gets solved under pressure. Finance stress-tested encryption and uptime. Retail stress-tested elasticity. The current proving ground is healthcare, and the problem is one every enterprise running AI will face shortly: how do you deploy machine learning in production when a regulator can demand an explanation for any individual output, years after the fact?
American health insurers are living that requirement right now, and the architectures emerging from their scramble are worth every cloud practitioner’s attention.
The forcing function
The pressure comes from money and audits. US insurers covering older adults are paid according to risk scores derived from members’ recorded diagnoses, and those diagnoses are increasingly extracted from clinical documents by AI. The government has responded to years of inflated records with an audit regime of real consequence: roughly two thousand certified coders, quarterly cycles, AI-assisted review on the regulator’s own side, and error rates extrapolated from samples to entire contracts. Federal reviews published in March 2026 found 81 to 91 percent of sampled high-risk codes unsupported at three plans, and one major insurer settled with the Department of Justice for 117.7 million dollars over how its diagnosis records were assembled.
The technical translation: any AI whose output feeds a risk score is now, functionally, a regulated system. Its conclusions must survive third-party inspection at the level of the individual inference. “The model is 94 percent accurate in aggregate” is not an answer an auditor accepts about one specific patient’s diabetes code.
The architecture that survives inspection
What passes inspection is instructive. The systems gaining ground in this market pair neural components, which read messy clinical language, with symbolic components, which validate findings against explicit clinical rules and emit a traceable justification for every output. The pattern is a working example of explainable AI in medical coding, and its production shape looks like this.
The neural layer handles perception: parsing notes, resolving abbreviations, locating candidate evidence in years of unstructured text. The symbolic layer handles judgment: does the located evidence actually satisfy the documented criteria for this diagnosis? Every accepted output ships with its receipts, the source sentence, the rule satisfied, the confidence, and the checkpoint where a human confirmed it. Rejection is a first-class outcome: the system flags codes that should be removed as readily as codes that could be added, because one-directional correction is exactly what prosecutors treat as intent.
For cloud architects, the interesting part is what this does to the stack. Explainability stops being a model property and becomes an infrastructure property. You need immutable storage for source documents, lineage metadata joining every inference to its evidence, versioning of both models and rule sets so that a decision made in 2024 can be reconstructed under 2024’s logic, and audit-grade logging that survives years of retention. The inference itself becomes a small part of a large evidentiary pipeline.
Operational lessons that generalise
Three practices from healthcare’s front line transfer cleanly to any regulated AI workload.
Design for reconstruction, not just monitoring. Observability tells you what the system is doing now. Regulators ask what it did, and why, on a specific input long ago. That requires deterministic replay: pinned model versions, pinned rule versions, pinned source data. Teams that treat model updates like schema migrations, versioned, reversible, documented, pass audits. Teams that hot-swap weights do not.
Make the human checkpoint real. Every framework converging on high-risk AI, from American health regulators to the EU AI Act, requires meaningful human oversight. In production, that means the review interface must show the evidence, not just the conclusion, and must record the reviewer’s decision as part of the lineage. A rubber-stamp UI is a compliance liability wearing a workflow’s clothes.
Budget for the boring layer. The healthcare vendors winning enterprise deals report that the differentiator is rarely model accuracy, where leaders cluster within a few points, but the evidentiary machinery around it. Certifications, retention, access controls, reconstruction guarantees. It is unglamorous cloud engineering, and it is now the moat.
The forecast
The direction of travel is unambiguous. The EU AI Act classifies most consequential systems as high-risk with traceability obligations. Financial regulators expect explainable model decisions. Even advertising and hiring platforms face demands to justify individual outputs. Healthcare simply arrived first, because the audits there came with clawbacks measured in contract-wide percentages.
Which means the reference architecture being hammered out in medical coding, perception by neural network, validation by explicit rules, evidence attached to every inference, humans at recorded checkpoints, replay guaranteed for years, is not a niche pattern. It is an early build of the standard enterprise AI stack for any workload where someone can later ask, under oath or under audit, why the machine said what it said.
The cloud industry has spent a decade making AI fast, cheap, and scalable. Healthcare is now demonstrating the next requirement: making it answerable. The organisations that treat that as an architecture problem today will find the coming decade of regulation considerably less exciting, in the best possible way.
