U.S. hospitals are required by CMS to publicly report CLABSI, CAUTI, C.diffficile, MRSA bacteremia, and selected SSIs for benchmarking and pay-for-performance programs. It is unclear, however, to ...what extent these conditions capture the full breadth of serious healthcare-associated infections (HAIs). CDC's hospital-onset Adult Sepsis Event (HO-ASE) definition could facilitate more comprehensive and efficient surveillance for serious HAIs, but the overlap between HO-ASE and currently reportable HAIs is unknown.
We retrospectively assessed the overlap between HO-ASEs and reportable HAIs among adults hospitalized between June 2015-June 2018 in 3 hospitals. Medical record reviews were conducted for 110 randomly selected HO-ASE cases to determine clinical correlates.
Amongst 282,441 hospitalized patients, 2,301 (0.8%) met HO-ASE criteria and 1,260 (0.4%) had reportable HAIs. In-hospital mortality rates were higher with HO-ASEs than reportable HAIs (28.6% vs 12.9%). Mortality rates for HO-ASE missed by reportable HAIs were substantially higher than mortality rates for reportable HAIs missed by HO-ASE (28.1% vs 6.3%). Reportable HAIs were only present in 334/2,301 (14.5%) HO-ASEs, most commonly CLABSIs (6.0% of HO-ASEs), C.difficile (5.0%), and CAUTI (3.0%). On medical record review, most HO-ASEs were caused by pneumonia (39.1%, of which only 34.9% were ventilator-associated), bloodstream infections (17.4%, of which only 10.5% were central line-associated), non-C.difficile intra-abdominal infections (14.5%), urinary infections (7.3%, of which 87.5% were catheter-associated), and skin/soft tissue infections (6.4%).
CDC's HO-ASE definition detects many serious nosocomial infections missed by currently reportable HAIs. HO-ASE surveillance could increase the efficiency and clinical significance of surveillance while identifying new targets for prevention.
Procedural competency is essential to the practice of emergency medicine. However, there are limited data quantifying emergency department procedural volumes to inform the work of educators and ...credentialing bodies. In this study, we characterize procedural scope and volume in a regional health care system and compare rates between practice settings and over time.
Cross-sectional data were acquired from electronic medical records of a regional health care system from March 2017 through February 2022. Nonspecific entries, esoteric procedures, and nonprocedural clinical skills were excluded. Procedural rates were compared: (1) between academic and community hospitals, (2) across study years, and (3) across seasons. Analyses were repeated for pediatric encounters, and with study year 4 removed to assess the influence of the first year of the coronavirus disease 2019 pandemic on results.
There were 131,976 instances of 40 qualifying procedures in 1,979,935 unique visits across 9 EDs. Several high-acuity procedures had similar rates in academic and community settings, including cardiac pacing, cricothyrotomy, and lateral canthotomy. Year-over-year procedural rates were stable or increasing for most procedures, with a notable exception of lumbar puncture. Most procedures did not have significant seasonal variation, and most findings were stable when study year 4 was removed from the analysis.
All procedures were performed in all settings and rates of several emergent procedures were similar in both settings, underscoring the importance of broad procedural competence for all emergency physicians. Educators and credentialing organizations can use these data to inform decisions regarding curriculum design and certification requirements.
Early notification of admissions from the emergency department (ED) may allow hospitals to plan for inpatient bed demand. This study aimed to assess Epic’s ED Likelihood to Occupy an Inpatient Bed ...predictive model and its application in improving hospital bed planning workflows.
All ED adult (18 years and older) visits from September 2021 to August 2022 at a large regional health care system were included. The primary outcome was inpatient admission. The predictive model is a random forest algorithm that uses demographic and clinical features. The model was implemented prospectively, with scores generated every 15 minutes. The area under the receiver operator curves (AUROC) and precision-recall curves (AUPRC) were calculated using the maximum score prior to the outcome and for each prediction independently. Test characteristics and lead time were calculated over a range of model score thresholds.
Over 11 months, 329,194 encounters were evaluated, with an incidence of inpatient admission of 25.4%. The encounter-level AUROC was 0.849 (95% confidence interval CI, 0.848 to 0.851), and the AUPRC was 0.643 (95% CI, 0.640 to 0.647). With a prediction horizon of 6 hours, the AUROC was 0.758 (95% CI, 0.758 to 0.759,) and the AUPRC was 0.470 (95% CI, 0.469 to 0.471). At a predictive model threshold of 40, the sensitivity was 0.49, the positive predictive value was 0.65, and the median lead-time warning was 127 minutes before the inpatient bed request.
The Epic ED Likelihood to Occupy an Inpatient Bed model may improve hospital bed planning workflows. Further study is needed to determine its operational effect.
Malfunctions in Clinical Decision Support (CDS) systems occur due to a multitude of reasons, and often go unnoticed, leading to potentially poor outcomes. Our goal was to identify malfunctions within ...CDS systems.
We evaluated 6 anomaly detection models: (1) Poisson Changepoint Model, (2) Autoregressive Integrated Moving Average (ARIMA) Model, (3) Hierarchical Divisive Changepoint (HDC) Model, (4) Bayesian Changepoint Model, (5) Seasonal Hybrid Extreme Studentized Deviate (SHESD) Model, and (6) E-Divisive with Median (EDM) Model and characterized their ability to find known anomalies. We analyzed 4 CDS alerts with known malfunctions from the Longitudinal Medical Record (LMR) and Epic® (Epic Systems Corporation, Madison, WI, USA) at Brigham and Women's Hospital, Boston, MA. The 4 rules recommend lead testing in children, aspirin therapy in patients with coronary artery disease, pneumococcal vaccination in immunocompromised adults and thyroid testing in patients taking amiodarone.
Poisson changepoint, ARIMA, HDC, Bayesian changepoint and the SHESD model were able to detect anomalies in an alert for lead screening in children and in an alert for pneumococcal conjugate vaccine in immunocompromised adults. EDM was able to detect anomalies in an alert for monitoring thyroid function in patients on amiodarone.
Malfunctions/anomalies occur frequently in CDS alert systems. It is important to be able to detect such anomalies promptly. Anomaly detection models are useful tools to aid such detections.
Abstract
Objective
Surviving Sepsis guidelines recommend blood cultures before administration of intravenous (IV) antibiotics for patients with sepsis or moderate to high risk of bacteremia. Clinical ...decision support (CDS) that reminds emergency department (ED) providers to obtain blood cultures when ordering IV antibiotics may lead to improvements in this process measure.
Methods
This was a multicenter causal impact analysis comparing timely blood culture collections prior to IV antibiotics for adult ED patients 1 year before and after a CDS intervention implementation in the electronic health record. A Bayesian structured time-series model compared daily timely blood cultures collected compared to a forecasted synthetic control. Mixed effects models evaluated the impact of the intervention controlling for confounders.
Results
The analysis included 54 538 patients over 2 years. In the baseline phase, 46.1% had blood cultures prior to IV antibiotics, compared to 58.8% after the intervention. Causal impact analysis determined an absolute increase of 13.1% (95% CI 10.4–15.7%) of timely blood culture collections overall, although the difference in patients with a sepsis diagnosis or who met CDC Adult Sepsis Event criteria was not significant, absolute difference 8.0% (95% CI −0.2 to 15.8). Blood culture positivity increased in the intervention phase, and contamination rates were similar in both study phases.
Discussion
CDS improved blood culture collection before IV antibiotics in the ED, without increasing overutilization.
Conclusion
A simple CDS alert increased timely blood culture collections in ED patients for whom concern for infection was high enough to warrant IV antibiotics.
There is considerable variation in ordering practices for the initial laboratory evaluation of monoclonal gammopathies (MGs) despite clear society guidelines to include serum free light chain (sFLC) ...testing. We assessed the ability of a clinical decision support (CDS) alert to improve guideline compliance and analyzed its clinical impact.
We designed and deployed a targeted CDS alert to educate and prompt providers to order an sFLC assay when ordering serum protein electrophoresis (SPEP) testing.
The alert was highly effective at increasing the co-ordering of SPEP and sFLC testing. Preimplementation, 62.8% of all SPEP evaluations included sFLC testing, while nearly 90% of evaluations included an sFLC assay postimplementation. In patients with no prior sFLC testing, analysis of sFLC orders prompted by the alert led to the determination that 28.9% (800/2,769) of these patients had an abnormal κ/λ ratio. In 452 of these patients, the sFLC assay provided the only laboratory evidence of a monoclonal protein. Moreover, within this population, there were numerous instances of new diagnoses of multiple myeloma and other MGs.
The CDS alert increased compliance with society guidelines and improved the diagnostic evaluation of patients with suspected MGs.
•Transferrin is a common test that may be overutilized due to ordering confusion.•Transferrin provides no independent information compared to concurrent iron studies.•An electronic health record ...alert was effective at improving transferrin utilization.
An important cause of laboratory test misordering and overutilization is clinician confusion between tests with similar sounding names or similar indications. We identified an area of test ordering confusion with iron studies that involves total iron binding capacity (TIBC), transferrin, and transferrin saturation. We observed concurrent ordering of direct transferrin along with TIBC at many hospitals within our health system and suspected this was unnecessary.
We extracted patient test results for transferrin, TIBC and other biomarkers. Using these data, we evaluated both patterns of test utilization and test result concordance. We implemented a clinical decision support (CDS) alert to discourage unnecessary orders for direct transferrin.
Using linear regression, we were able to predict transferrin from either TIBC alone or TIBC with other analytes with a high degree of accuracy, demonstrating that in most cases, direct transferrin in combination with TIBC provides little if any additional diagnostic information beyond TIBC alone. The CDS alert proved highly effective in reducing transferrin test utilization at four different hospitals.
Concurrent ordering of direct transferrin and TIBC should usually be avoided. Removal of transferrin or TIBC from the test menu or implementation of CDS may improve utilization of these tests.
Abstract
Background
Rule-base clinical decision support alerts are known to malfunction, but tools for discovering malfunctions are limited.
Objective
Investigate whether user override comments can ...be used to discover malfunctions.
Methods
We manually classified all rules in our database with at least 10 override comments into 3 categories based on a sample of override comments: “broken,” “not broken, but could be improved,” and “not broken.” We used 3 methods (frequency of comments, cranky word list heuristic, and a Naïve Bayes classifier trained on a sample of comments) to automatically rank rules based on features of their override comments. We evaluated each ranking using the manual classification as truth.
Results
Of the rules investigated, 62 were broken, 13 could be improved, and the remaining 45 were not broken. Frequency of comments performed worse than a random ranking, with precision at 20 of 8 and AUC = 0.487. The cranky comments heuristic performed better with precision at 20 of 16 and AUC = 0.723. The Naïve Bayes classifier had precision at 20 of 17 and AUC = 0.738.
Discussion
Override comments uncovered malfunctions in 26% of all rules active in our system. This is a lower bound on total malfunctions and much higher than expected. Even for low-resource organizations, reviewing comments identified by the cranky word list heuristic may be an effective and feasible way of finding broken alerts.
Conclusion
Override comments are a rich data source for finding alerts that are broken or could be improved. If possible, we recommend monitoring all override comments on a regular basis.
Tetanus is the most common vaccination given in the emergency department; yet, administrations of tetanus vaccine boosters in the ED may not comply with the US Centers for Disease Control and ...Prevention’s recommended vaccination schedule. We implemented a clinical decision support alert in the electronic health record that warned providers when ordering a tetanus vaccine if a prior one had been given within 10 years and studied its efficacy to reduce potentially unnecessary vaccines in the ED.
This was a retrospective, quasi-experimental, 1-group, pretest-posttest study in 3 hospital EDs in Boston, MA. We studied adult patients for whom tetanus vaccines were ordered despite a history of vaccination within the prior 10 years. We compared the number of potentially unnecessary tetanus vaccine administrations in a baseline phase (when the clinical decision support alert was not visible) versus an intervention phase.
Of eligible patients, 22.1% (95% confidence interval CI 21.8% to 22.4%) had prior tetanus vaccines within 5 years, 12.8% (95% CI 12.5% to 13.0%) within 5 to 10 years, 3.8% (95% CI 3.6% to 3.9%) more than 10 years ago, and 61.3% (95% CI 60.9% to 61.7%) had no prior tetanus vaccination documentation. Of 60,983 encounters, 337 met the inclusion criteria. A tetanus vaccination was administered in 91% (95% CI 87% to 96%) of encounters in the baseline phase, compared to 55% (95% CI 47% to 62%) during the intervention. The absolute risk reduction was 36.7% (95% CI 28.0% to 45.4%), and the number of encounters needed to alert to avoid 1 potentially unnecessary tetanus vaccine (number needed to treat) was 2.7 (95% CI 2.2% to 3.6%). For patients with tetanus vaccines within the prior 5 years, the absolute risk reduction was 47.9% (95% CI 35.5 % to 60.3%) and the number needed to treat was 2.1 (95% CI 1.7% to 2.8%).
A clinical decision support alert that warns ED clinicians that a patient may have an up-to-date tetanus vaccination status reduces potentially unnecessary vaccinations.
Delays in the second dose of antibiotics in the emergency department (ED) are associated with increased morbidity and mortality in patients with serious infections. We analyzed the influence of ...clinical decision support to prevent delays in second doses of broad-spectrum antibiotics in the ED.
We allocated adult patients who received cefepime or piperacillin/tazobactam in 9 EDs within an integrated health care system to an electronic alert that reminded ED clinicians to reorder antibiotics at the appropriate interval vs usual care. The primary outcome was a median delay in antibiotic administration. Secondary outcomes were rates of intensive care unit (ICU) admission, hospital mortality, and hospital length of stay. We included a post hoc secondary outcome of frequency of major delay (>25% of expected interval for second antibiotic dose).
A total of 1,113 ED patients treated with cefepime or piperacillin/tazobactam were enrolled in the study, of whom 420 remained under ED care when their second dose was due and were included in the final analysis. The clinical decision support tool was associated with reduced antibiotic delays (median difference 35 minutes, 95% confidence interval CI, 5 to 65). There were no differences in ICU transfers, inpatient mortality, or hospital length of stay. The clinical decision support tool was associated with decreased probability of major delay (absolute risk reduction 13%, 95% CI, 6 to 20).
The implementation of a clinical decision support alert reminding clinicians to reorder second doses of antibiotics was associated with a reduction in the length and frequency of antibiotic delays in the ED. There was no effect on the rates of ICU transfers, inpatient mortality, or hospital length of stay.
PDF
