Multicenter Evaluation of an Interoperable System for Automated Guideline Adherence Monitoring in ICUs.

Summary

Across five university hospitals and 82,000 ICU episodes, an interoperable system that digitally encodes guideline recommendations achieved 97% accuracy in assessing applicability and adherence—significantly outperforming expert human review—and processed data at massive scale. Real-world adherence varied by site and over time, influenced by documentation quality and evolving knowledge.

Key Findings

• Automated system accuracy 97.0% vs human 86.6% (p < 0.001) for applicability/adherence identification.
• Throughput >2000 patient-days/second vs manual 2 patient-days/minute.
• Adherence varied across sites and over time, influenced by documentation inconsistencies and evolving knowledge.
• Six recommendations from 41 guidelines successfully translated into standardized digital format across disparate EHRs.

Clinical Implications

Hospitals can deploy interoperable, digitally encoded guideline engines to monitor adherence at scale, identify gaps, and target interventions; robust structured documentation is critical to enable accurate automation.

Limitations

• Retrospective design limits causal inference on adherence determinants.
• Dependence on structured and consistent documentation; unstructured data may reduce performance.

Future Directions

Expand to broader guideline sets, incorporate unstructured data (NLP), and test prospective deployment with feedback loops to improve adherence and outcomes.