AbstractObjectiveTo create and validate a simple and transferable machine learning model from electronic health record data to accurately predict clinical deterioration in patients with covid-19 ...across institutions, through use of a novel paradigm for model development and code sharing.DesignRetrospective cohort study.SettingOne US hospital during 2015-21 was used for model training and internal validation. External validation was conducted on patients admitted to hospital with covid-19 at 12 other US medical centers during 2020-21.Participants33 119 adults (≥18 years) admitted to hospital with respiratory distress or covid-19.Main outcome measuresAn ensemble of linear models was trained on the development cohort to predict a composite outcome of clinical deterioration within the first five days of hospital admission, defined as in-hospital mortality or any of three treatments indicating severe illness: mechanical ventilation, heated high flow nasal cannula, or intravenous vasopressors. The model was based on nine clinical and personal characteristic variables selected from 2686 variables available in the electronic health record. Internal and external validation performance was measured using the area under the receiver operating characteristic curve (AUROC) and the expected calibration error—the difference between predicted risk and actual risk. Potential bed day savings were estimated by calculating how many bed days hospitals could save per patient if low risk patients identified by the model were discharged early.Results9291 covid-19 related hospital admissions at 13 medical centers were used for model validation, of which 1510 (16.3%) were related to the primary outcome. When the model was applied to the internal validation cohort, it achieved an AUROC of 0.80 (95% confidence interval 0.77 to 0.84) and an expected calibration error of 0.01 (95% confidence interval 0.00 to 0.02). Performance was consistent when validated in the 12 external medical centers (AUROC range 0.77-0.84), across subgroups of sex, age, race, and ethnicity (AUROC range 0.78-0.84), and across quarters (AUROC range 0.73-0.83). Using the model to triage low risk patients could potentially save up to 7.8 bed days per patient resulting from early discharge.ConclusionA model to predict clinical deterioration was developed rapidly in response to the covid-19 pandemic at a single hospital, was applied externally without the sharing of data, and performed well across multiple medical centers, patient subgroups, and time periods, showing its potential as a tool for use in optimizing healthcare resources.
The impact of alcohol or opioid use disorders on medication dosing for procedural sedation in the emergency department (ED) is unclear, as most of the literature is from gastrointestinal endoscopy. ...Exploring how these patient factors affect sedative and analgesic medications may inform more nuanced sedation strategies in the emergency department.
This was a retrospective chart-review cohort study across five EDs from 2015 to 2020. Included were adult patients who underwent procedural sedation in the ED, categorized into three a priori groups: alcohol use disorder (AUD), opioid use disorder (OUD), and individuals with neither (non-SUD). Wilcoxon test was used to compare the time-averaged dose of agents between groups. Logistic regression was used to model multi-agent sedations. The propofol time-averaged dose was the primary outcome. Secondary outcomes included other agents, sedation duration, and switching to other agents.
2725 sedations were included in the analysis. 59 patients had a history of AUD, and 40 had a history of OUD. Time-averaged doses of medications did not differ significantly between AUD and non-SUD patients. Likewise, patients with OUD did not receive different doses of medications compared to non-SUD. The propofol doses for non-SUD, AUD, and OUD were 0.033 IQR 0.04; 0.042 IQR 0.05; and 0.058 IQR 0.04 mg/kg*min, respectively. Sedation duration was not different across groups. Having AUD or OUD is not associated with increased odds of requiring multiple sedative agents.
Although sedation in patients with AUD or OUD may be associated with significant case bias, these patient factors did not significantly alter outcomes compared to the general population. This study suggests there is no evidence to proactively adjust medication strategy in ED patients with AUD or OUD.
•Alcohol and opioid use disorder may not affect med dosing in ED procedural sedation.•Substance use patient factors may not result in longer sedation duration.•Significant case bias should be considered when planning sedation strategy.•SNOMED CT and social history can be used to cohort patients with great accuracy.
The study sought to determine availability and use of structured override reasons for drug-drug interaction (DDI) alerts in electronic health records.
We collected data on DDI alerts and override ...reasons from 10 clinical sites across the United States using a variety of electronic health records. We used a multistage iterative card sort method to categorize the override reasons from all sites and identified best practices.
Our methodology established 177 unique override reasons across the 10 sites. The number of coded override reasons at each site ranged from 3 to 100. Many sites offered override reasons not relevant to DDIs. Twelve categories of override reasons were identified. Three categories accounted for 78% of all overrides: "will monitor or take precautions," "not clinically significant," and "benefit outweighs risk."
We found wide variability in override reasons between sites and many opportunities to improve alerts. Some override reasons were irrelevant to DDIs. Many override reasons attested to a future action (eg, decreasing a dose or ordering monitoring tests), which requires an additional step after the alert is overridden, unless the alert is made actionable. Some override reasons deferred to another party, although override reasons often are not visible to other users. Many override reasons stated that the alert was inaccurate, suggesting that specificity of alerts could be improved.
Organizations should improve the options available to providers who choose to override DDI alerts. DDI alerting systems should be actionable and alerts should be tailored to the patient and drug pairs.
To determine if a real-time monitoring system with automated clinician alerts improves 3-hour sepsis bundle adherence.
Prospective, pragmatic clinical trial. Allocation alternated every 7 days.
...Quaternary hospital from December 1, 2020 to November 30, 2021.
Adult emergency department or inpatients meeting objective sepsis criteria triggered an electronic medical record (EMR)-embedded best practice advisory. Enrollment occurred when clinicians acknowledged the advisory indicating they felt sepsis was likely.
Real-time automated EMR monitoring identified suspected sepsis patients with incomplete bundle measures within 1-hour of completion deadlines and generated reminder pages. Clinicians responsible for intervention group patients received reminder pages; no pages were sent for controls. The primary analysis cohort was the subset of enrolled patients at risk of bundle nonadherent care that had reminder pages generated.
The primary outcome was orders for all 3-hour bundle elements within guideline time limits. Secondary outcomes included guideline-adherent delivery of all 3-hour bundle elements, 28-day mortality, antibiotic discontinuation within 48-hours, and pathogen recovery from any culture within 7 days of time-zero. Among 3,269 enrolled patients, 1,377 had reminder pages generated and were included in the primary analysis. There were 670 (48.7%) at-risk patients randomized to paging alerts and 707 (51.3%) to control. Bundle-adherent orders were placed for 198 intervention patients (29.6%) versus 149 (21.1%) controls (difference: 8.5%; 95% CI, 3.9-13.1%; p = 0.0003). Bundle-adherent care was delivered for 152 (22.7%) intervention versus 121 (17.1%) control patients (difference: 5.6%; 95% CI, 1.4-9.8%; p = 0.0095). Mortality was similar between groups (8.4% vs 8.3%), as were early antibiotic discontinuation (35.1% vs 33.4%) and pan-culture negativity (69.0% vs 68.2%).
Real-time monitoring and paging alerts significantly increased orders for and delivery of guideline-adherent care for suspected sepsis patients at risk of 3-hour bundle nonadherence. The trial was underpowered to determine whether adherence affected mortality. Despite enrolling patients with clinically suspected sepsis, early antibiotic discontinuation and pan-culture negativity were common, highlighting challenges in identifying appropriate patients for sepsis bundle application.
Objective: The United States Office of the National Coordinator for Health Information Technology sponsored the development of a “high-priority” list of drug-drug interactions (DDIs) to be used for ...clinical decision support. We assessed current adoption of this list and current alerting practice for these DDIs with regard to alert implementation (presence or absence of an alert) and display (alert appearance as interruptive or passive).
Materials and methods: We conducted evaluations of electronic health records (EHRs) at a convenience sample of health care organizations across the United States using a standardized testing protocol with simulated orders.
Results: Evaluations of 19 systems were conducted at 13 sites using 14 different EHRs. Across systems, 69% of the high-priority DDI pairs produced alerts. Implementation and display of the DDI alerts tested varied between systems, even when the same EHR vendor was used. Across the drug pairs evaluated, implementation and display of DDI alerts differed, ranging from 27% (4/15) to 93% (14/15) implementation.
Discussion: Currently, there is no standard of care covering which DDI alerts to implement or how to display them to providers. Opportunities to improve DDI alerting include using differential displays based on DDI severity, establishing improved lists of clinically significant DDIs, and thoroughly reviewing organizational implementation decisions regarding DDIs.
Conclusion: DDI alerting is clinically important but not standardized. There is significant room for improvement and standardization around evidence-based DDIs.
Abstract
Background
Isolation of hospitalized persons under investigation (PUIs) for coronavirus disease 2019 (COVID-19) reduces nosocomial transmission risk. Efficient evaluation of PUIs is needed ...to preserve scarce healthcare resources. We describe the development, implementation, and outcomes of an inpatient diagnostic algorithm and clinical decision support system (CDSS) to evaluate PUIs.
Methods
We conducted a pre-post study of CORAL (COvid Risk cALculator), a CDSS that guides frontline clinicians through a risk-stratified COVID-19 diagnostic workup, removes transmission-based precautions when workup is complete and negative, and triages complex cases to infectious diseases (ID) physician review. Before CORAL, ID physicians reviewed all PUI records to guide workup and precautions. After CORAL, frontline clinicians evaluated PUIs directly using CORAL. We compared pre- and post-CORAL frequency of repeated severe acute respiratory syndrome coronavirus 2 nucleic acid amplification tests (NAATs), time from NAAT result to PUI status discontinuation, total duration of PUI status, and ID physician work hours, using linear and logistic regression, adjusted for COVID-19 incidence.
Results
Fewer PUIs underwent repeated testing after an initial negative NAAT after CORAL than before CORAL (54% vs 67%, respectively; adjusted odd ratio, 0.53 95% confidence interval, .44–.63; P < .01). CORAL significantly reduced average time to PUI status discontinuation (adjusted difference standard error, −7.4 0.8 hours per patient), total duration of PUI status (−19.5 1.9 hours per patient), and average ID physician work-hours (−57.4 2.0 hours per day) (all P < .01). No patients had a positive NAAT result within 7 days after discontinuation of precautions via CORAL.
Conclusions
CORAL is an efficient and effective CDSS to guide frontline clinicians through the diagnostic evaluation of PUIs and safe discontinuation of precautions.
The COvid Risk cALculator diagnostic algorithm and clinical decision support system substantially reduced duration of transmission-based precautions for persons under investigation for coronavirus disease 2019, and the time infectious diseases physicians spent evaluating them in a large academic medical center.
Abstract
Objectives
To improve clinical decision support (CDS) by allowing users to provide real-time feedback when they interact with CDS tools and by creating processes for responding to and ...acting on this feedback.
Methods
Two organizations implemented similar real-time feedback tools and processes in their electronic health record and gathered data over a 30-month period. At both sites, users could provide feedback by using Likert feedback links embedded in all end-user facing alerts, with results stored outside the electronic health record, and provide feedback as a comment when they overrode an alert. Both systems are monitored daily by clinical informatics teams.
Results
The two sites received 2,639 Likert feedback comments and 623,270 override comments over a 30-month period. Through four case studies, we describe our use of end-user feedback to rapidly respond to build errors, as well as identifying inaccurate knowledge management, user-interface issues, and unique workflows.
Conclusion
Feedback on CDS tools can be solicited in multiple ways, and it contains valuable and actionable suggestions to improve CDS alerts. Additionally, end users appreciate knowing their feedback is being received and may also make other suggestions to improve the electronic health record. Incorporation of end-user feedback into CDS monitoring, evaluation, and remediation is a way to improve CDS.
Abstract
Objectives
To evaluate the use of a provider ordering alert to improve laboratory efficiency and reduce costs.
Methods
We conducted a retrospective study to assess the use of an ...institutional reflex panel for monoclonal gammopathy evaluation. We then created a clinical decision support (CDS) alert to educate and encourage providers to change their less-efficient orders to the reflex panel.
Results
Our retrospective analysis demonstrated that an institutional reflex panel could be safely substituted for a less-efficient and higher-cost panel. The implemented CDS alert resulted in 79% of providers changing their high-cost order panel to an order panel based on the reflex algorithm.
Conclusions
The validated decision support alert demonstrated high levels of provider acceptance and directly led to operational and cost savings within the laboratory. Furthermore, these studies highlight the value of laboratory involvement with CDS efforts to provide agile and targeted provider ordering assistance.
Patients with limited English proficiency (LEP) face multiple barriers and are at risk for worse health outcomes compared with patients with English proficiency (EP). In sepsis, a major cause of ...mortality in the US, the association of LEP with health outcomes is not widely explored.
To assess the association between LEP and inpatient mortality among patients with sepsis and test the hypothesis that LEP would be associated with higher mortality rates.
This retrospective cohort study of hospitalized patients with sepsis included those who met the Centers for Disease Control and Prevention's sepsis criteria, received antibiotics within 24 hours, and were admitted through the emergency department. Data were collected from the electronic medical records of a large New England tertiary care center from January 1, 2016, to December 31, 2019. Data were analyzed from January 8, 2021, to March 2, 2023.
Limited English proficiency, gathered via self-reported language preference in electronic medical records.
The primary outcome was inpatient mortality. The analysis used multivariable generalized estimating equation models with propensity score adjustment and analysis of covariance to analyze the association between LEP and inpatient mortality due to sepsis.
A total of 2709 patients met the inclusion criteria, with a mean (SD) age of 65.0 (16.2) years; 1523 (56.2%) were men and 327 (12.1%) had LEP. Nine patients (0.3%) were American Indian or Alaska Native, 101 (3.7%) were Asian, 314 (11.6%) were Black, 226 (8.3%) were Hispanic, 38 (1.4%) were Native Hawaiian or Other Pacific Islander or of other race or ethnicity, 1968 (72.6%) were White, and 6 (0.2%) were multiracial. Unadjusted mortality included 466 of 2382 patients with EP (19.6%) and 69 of 327 with LEP (21.1%). No significant difference was found in mortality odds for the LEP compared with EP groups (odds ratio OR, 1.12 95% CI, 0.88-1.42). When stratified by race and ethnicity, odds of inpatient mortality for patients with LEP were significantly higher among the non-Hispanic White subgroup (OR, 1.76 95% CI, 1.41-2.21). This significant difference was also present in adjusted analyses (adjusted OR, 1.56 95% CI, 1.02-2.39). No significant differences were found in inpatient mortality between LEP and EP in the racial and ethnic minority subgroup (OR, 0.99 95% CI, 0.63-1.58; adjusted OR, 0.91 95% CI, 0.56-1.48).
In a large diverse academic medical center, LEP had no significant association overall with sepsis mortality. In a subgroup analysis, LEP was associated with increased mortality among individuals identifying as non-Hispanic White. This finding highlights a potential language-based inequity in sepsis care. Further studies are needed to understand drivers of this inequity, how it may manifest in other diverse health systems, and to inform equitable care models for patients with LEP.
PDF
