Clinical leadership adopts an AI tool for radiology triage, sepsis prediction, or prior authorization review. The tool is FDA-cleared. The vendor has a BAA. Legal has reviewed the contract. The CISO has signed off. The deployment goes live.

Six months later, the vendor updates the model. Nobody in the hospital's clinical governance structure knows it happened. The tool's behavior has shifted in ways the institution cannot observe, did not authorize, and cannot document.

Under the FDA's December 2024 guidance on Predetermined Change Control Plans forAI-enabled medical devices, that is called operating outside your cleared predicate. The  responsibility lands with the hospital, not the vendor. Clinical leadership adopts an AI tool for radiology triage, sepsis prediction, or prior authorization review. The tool is FDA-cleared. The vendor has a BAA. Legal has reviewed the contract. The CISO has signed off. The deployment goes live.

Six months later, the vendor updates the model. Nobody inthe hospital's clinical governance structure knows it happened. The tool'sbehavior has shifted in ways the institution cannot observe, did not authorize,and cannot document.

Under the FDA's December 2024 guidance on PredeterminedChange Control Plans for AI-enabled medical devices, that is called operating outside your cleared predicate. The responsibility lands with the hospital, not the vendor.

The risks described in this post sit at the intersection ofclinical governance and infrastructure strategy. Solving them requires bothsides of that conversation: a CMIO or CNO who understands what's at stakeclinically, and an infrastructure or IT leader who controls where and howinference actually runs.

What the FDA Changed, and Why It Reaches Your Infrastructure Team

The FDA's December 2024 guidance established the Predetermined Change Control Plan, or PCCP. Because AI models need to improve over time, the FDA created a pathway for manufacturers to define in advance what kinds of changes are permissible without a new regulatory submission.Changes within the plan can proceed. Changes outside it require going back to the FDA.

The FDA has authorized more than 1,450 AI and machine learning-based medical devices through end-2025. Fewer than one in five has a PCCP. The rest are effectively locked, meaning any behavioral change to those models from any source, including vendor-side updates to a cloud inference backend, potentially requires regulatory action before the device can continue in clinical use.

The deployer is the hospital. Deployer obligations under FDA guidance include ensuring the device operates in accordance with its intended use, maintaining conditions consistent with the cleared design, and monitoring for performance changes. When a cloud inference vendor updates a model and the hospital cannot document what changed, when it changed, or whether the change fell within the scope of the clearance, the hospital carries the compliance gap.

Does your clinical AI setup give you the visibility and control to know what version of a model is running, whether its behavior matches how it was validated, and whether any changes occurred without your authorization? For most health systems running clinical AI on managed cloud inference services, the answer is no, and that is an infrastructure problem before it is a legal one.

"We Have a BAA" Covers Different Ground Than Most Teams Assume

A BAA addresses data privacy obligations. Model governance is separate territory.

A BAA specifies how a vendor handles protected health information. It does not define which version of a model runs against that information, whether the vendor can update the model without notice, or whether the hospital holds audit rights over inference behavior.

BAAs are also scoped instruments. They cover specific data flows and specific services. When a clinical AI tool routes patient data through an inference endpoint not explicitly covered by the BAA, or when audio from an ambient documentation session is processed by a sub-processor not named in the agreement, the hospital carries a potential Privacy Rule exposure that upstream due diligence does not resolve.

Closing that gap requires knowing, with specificity, where PHI flows during inference, what protections exist at each point, and whether the hospital controls or can audit each of those points. Managed cloud inference typically does not provide that visibility.

What the Epic Sepsis Model Taught the Industry

A widely cited study examined the real-world performance of Epic's sepsis prediction model at a major academic medical center. Results were significantly worse than Epic's published figures, with sensitivity much lower than expected and alert volumes high enough to create fatigue that may have worsened clinical outcomes rather than improved them.

Epic subsequently required hospitals to retrain on local data. The institutions that caught the performance gap earliest had their own model monitoring in place, with visibility into what the model was actually doing in their environment, independent of what the vendor reported. That visibility is only possible when the institution controls enough of the infrastructure to observe it.

A model running entirely on vendor-managed systems with no independent audit capability produces a clinical risk before it produces a regulatory one.

What Infrastructure Control Actually Means in Practice

Knowing exactly what model version is in production, from your own systems and not the vendor's. If answering that question requires a call to the vendor, the hospital does not have the documentation baseline that PCCP compliance and post-market monitoring both require.

Controlling when and whether model updates are applied. Notification rights matter, but veto rights over deployment timing matter more.A vendor that can push an update to a production clinical AI tool on their own schedule is operating a regulated medical device on their terms.

Owning your audit logs. Post-market monitoring requires the institution to maintain evidence of model behavior over time. Logs that live in a vendor's environment, accessible only through vendor tooling, are not fully under the hospital's control, and in an adverse event scenario that distinction carries real weight.

Running inference in consistent physical conditions. Modern AI inference hardware is thermally sensitive. High-density AI chips in data centers not designed to cool them properly will throttle under load, reducing operating speed to manage heat. A model that throttles mid-inference produces different outputs than one running at full capacity. For a device validated under specific conditions, that variability is a reproducibility problem. Liquid-cooled infrastructure, purpose-built for inference-class density, eliminates that variability. Air-cooled legacy data centers and standard retail colocation configurations do not.

Keeping inference close to your data. Clinical AI inference running against Epic or Cerner data needs to return results within the clinician's workflow window, often under a second. When inference runs in a distant cloud region, network transit alone can consume most of that budget before the model has processed a single token. Private network paths to a metro-edge facility, physically close to where your EHR lives, solve both the latency problem and reduce the PHI transit exposure that multi-hop cloud routing creates.

Where the Institutions That Have Worked Through This Have Landed

Mayo Clinic's AI infrastructure for pathology and clinical decision support is physically on-site, under Mayo's operational control, processing patient data on dedicated hardware. The institution controls model versioning, execution conditions, and audit logging.

Eli Lilly built its own liquid-cooled AI infrastructure for drug discovery on-premises. The proprietary compound and genomic data running through inference is the source of competitive value, and keeping it on infrastructure the firm controls is a core requirement.

Bristol Myers Squibb moved its AI infrastructure into a dedicated colocation environment and reports meaningful cost savings compared to its prior cloud model, along with the control and audit capability that cloud inference did not provide.

What these institutions share is having taken seriously the question of what happens when AI informs decisions that carry clinical, financial, or legal consequences, and building infrastructure that reflects the answer.

For health systems that cannot build at the scale of a MayoClinic or Eli Lilly, purpose-built colocation is the practical path. Colovore's facilities were designed specifically for this class of workload. Liquid-cooled from the ground up, not retrofitted. Certified under ISO 27001 and SOC 2 TypeII, with HIPAA, PCI DSS, and FedRAMP support built into facility design.Positioned in metro markets, including Chicago, with the carrier infrastructure and private network access that clinical AI data residency requirements demand.Scalable from an initial deployment to multi-megawatt without re-platforming or renegotiating as AI embeds deeper into clinical operations.

The Chicago campus adds something specifically relevant for health systems in that market: 54 megawatts of critical capacity across Aurora and West Chicago, with ORD01 available December 2026 and additional halls coming online through 2028. For health systems evaluating their AI infrastructure runway over the next two to five years, that capacity road map matters. AI-ready space in the Chicago metro is constrained and getting tighter. The institutions securing infrastructure now are the ones that will not be rebuilding their AI strategy around a capacity problem two years from now.

The Question to Ask Before the Next Clinical AI Deployment

Before the next deployment, three questions are worth asking. If this tool's model changes, will we know? If it changes in a way that affects clinical performance, do we have monitoring in place to catch it? If a regulator asks us to demonstrate the tool is operating in accordance with its cleared predicate, can we answer from our own systems?

Uncertainty on any of those points is an infrastructure question waiting to be asked.

The FDA's December 2024 guidance formalized a logic that was always present in how cleared medical devices are supposed to be deployed. The execution environment, previously treated as a technical detail, is now a compliance variable. Health systems that recognize this before their clinicalAI strategy is locked will be in a fundamentally stronger position than those that recognize it after.

The risks described in this post sit at the intersection of clinical governance and infrastructure strategy. Solving them requires both sides of that conversation: a CMIO or CNO who understands what's at stake clinically, and an infrastructure or IT leader who controls where and how inference actually runs.

‍

This post is part of Colovore's ongoing series on the coming AI inference divide, the structural shift separating where AI is trained from where it runs in production at enterprise scale.

For the full analysis, including industry-specific use cases and the specialized silicon landscape, download the complete strategy paper.

‍