There is an increasing interest in clinical prediction tools that can achieve high prediction accuracy and provide explanations of the factors leading to increased risk of adverse outcomes. However, ...approaches to explaining complex machine learning (ML) models are rarely informed by end-user needs and user evaluations of model interpretability are lacking in the healthcare domain. We used extended revisions of previously-published theoretical frameworks to propose a framework for the design of user-centered displays of explanations. This new framework served as the basis for qualitative inquiries and design review sessions with critical care nurses and physicians that informed the design of a user-centered explanation display for an ML-based prediction tool.
We used our framework to propose explanation displays for predictions from a pediatric intensive care unit (PICU) in-hospital mortality risk model. Proposed displays were based on a model-agnostic, instance-level explanation approach based on feature influence, as determined by Shapley values. Focus group sessions solicited critical care provider feedback on the proposed displays, which were then revised accordingly.
The proposed displays were perceived as useful tools in assessing model predictions. However, specific explanation goals and information needs varied by clinical role and level of predictive modeling knowledge. Providers preferred explanation displays that required less information processing effort and could support the information needs of a variety of users. Providing supporting information to assist in interpretation was seen as critical for fostering provider understanding and acceptance of the predictions and explanations. The user-centered explanation display for the PICU in-hospital mortality risk model incorporated elements from the initial displays along with enhancements suggested by providers.
We proposed a framework for the design of user-centered displays of explanations for ML models. We used the proposed framework to motivate the design of a user-centered display of an explanation for predictions from a PICU in-hospital mortality risk model. Positive feedback from focus group participants provides preliminary support for the use of model-agnostic, instance-level explanations of feature influence as an approach to understand ML model predictions in healthcare and advances the discussion on how to effectively communicate ML model information to healthcare providers.
Celotno besedilo
Dostopno za:
DOBA, IZUM, KILJ, NUK, PILJ, PNG, SAZU, SIK, UILJ, UKNU, UL, UM, UPUK
Big data has the capacity to transform both pediatric healthcare delivery and research, but its potential has yet to be fully realized. Curation of large multi-institutional datasets of high-quality ...data has allowed for significant advances in the timeliness of quality improvement efforts. Improved access to large datasets and computational power have also paved the way for the development of high-performing, data-driven decision support tools and precision medicine approaches. However, implementation of these approaches and tools into pediatric practice has been hindered by challenges in our ability to adequately capture the heterogeneity of the pediatric population as well as the nuanced complexities of pediatric diseases such as sepsis. Moreover, there are large gaps in knowledge and definitive evidence demonstrating the utility, usability, and effectiveness of these types of tools in pediatric practice, which presents significant challenges to provider willingness to leverage these solutions. The next wave of transformation for pediatric healthcare delivery and research through big data and sophisticated analytics will require focusing efforts on strategies to overcome cultural barriers to adoption and acceptance. IMPACT: Big data from EHRs can be used to drive improvement in pediatric clinical care. Clinical decision support, artificial intelligence, machine learning, and precision medicine can transform pediatric care using big data from the EHR. This article provides a review of barriers and enablers for the effective use of data analytics in pediatric clinical care using pediatric sepsis as a use case. The impact of this review is that it will inform influencers of pediatric care about the importance of current trends in data analytics and its use in improving outcomes of care through EHR-based strategies.
Esophageal cancer is the seventh most common type of cancer in the world, the sixth leading cause of cancer-related death and its incidence is expected to rise 140% in the world in a period of ...10 years until 2025. The overall incidence is higher in males, while data about prognosis and survival are not well established yet. The goal of this study was to carry out a comprehensive analysis of differences between sexes and other covariates in patients diagnosed with primary esophageal cancer. Data from 2005 to 2020 were obtained from the University Hospitals (UH) Seidman Cancer Center and from 2005 to 2018 from SEER. Patients were categorized according to histological subtype and divided according to sex. Pearson Chi-square test was used to compare variables of interest by sex and the influence of sex on survival was assessed by Kaplan Meier, log rank tests and Cox proportional hazards regression models. A total of 1205 patients were used for analysis. Sex differences in all types were found for age at diagnosis, histology, smoking status and prescriptions of NSAIDs and in SCC for age at diagnosis and alcoholism. Survival analysis didn't showed differences between males and females on univariable and multivariable models. Males have a higher incidence of Esophageal Cancer and its two main subtypes but none of the comprehensive set of variables analyzed showed to be strongly or unique correlated with this sex difference in incidence nor are they associated with a sex difference in survival.
Lung cancer is the leading cause of cancer-related death and the second most often diagnosed malignancy worldwide. Males have higher incidence of lung cancer and higher mortality. It is hypothesized ...that the sex differences in survival are primarily driven by a better response of females to treatment. The primary objective of this work is to analyze and describe outcome differences between males and females diagnosed with having lung cancer.
Data were obtained from a large hybrid academic-community practice institution and validated with Surveillance, Epidemiology, and End Results (SEER). The initial cohort included patients aged more than or equal to 18 years diagnosed with having primary malignant lung cancer. Patients were excluded from the analysis if they had an unknown diagnosis date, were missing sex, or had prior history of cancer. Chi-square, t test, and Kruskal-Wallis tests were used to compare characteristics of males and females. Risks were estimated by logistic and Cox regressions.
A total of 8909 patients from our institution and 725,018 in SEER were analyzed. Male-to-female ratio was 1.0. Females were more likely to undergo surgery and less likely to be treated with immunotherapy. Females had higher rates of documented psychological affections, depression, anxiety, urinary tract infection, hypothyroidism, and hyperthyroidism, while displaying lower rates of acute kidney injury, myocardial infarction, and myocarditis. Paired multivariable models revealed a lower risk of death for females in SEER (hazard ratio for females = 0.84, confidence interval: 0.69–1.02, p = 0.08) and equal risks in our institution (hazard ratio for females = 0.84, confidence interval: 0.69–1.02, p = 0.08).
Female sex was associated with higher surgical rates, lower immunotherapy use rates, higher rates of endocrinologic complications after immunotherapy use, and higher rates of psychological disorders.
Racial disparities have been reported for breast cancer and cardiovascular disease (CVD) outcomes. The determinants of racial disparities in CVD outcomes are not yet fully understood. We aimed to ...examine the impact of individual and neighborhood-level social determinants of health (SDOH) on the racial disparities in major adverse cardiovascular events (MACE; consisting of heart failure, acute coronary syndrome, atrial fibrillation, and ischemic stroke) among female patients with breast cancer.
This 10-year longitudinal retrospective study was based on a cancer informatics platform with electronic medical record supplementation. We included women aged ≥18 years diagnosed with breast cancer. SDOH were obtained from LexisNexis, and consisted of the domains of social and community context, neighborhood and built environment, education access and quality, and economic stability. Race-agnostic (overall data with race as a feature) and race-specific machine learning models were developed to account for and rank the SDOH impact in 2-year MACE.
We included 4,309 patients (765 non-Hispanic Black NHB; 3,321 non-Hispanic white). In the race-agnostic model (C-index, 0.79; 95% CI, 0.78-0.80), the 5 most important adverse SDOH variables were neighborhood median household income (SHapley Additive exPlanations SHAP score SS, 0.07), neighborhood crime index (SS = 0.06), number of transportation properties in the household (SS = 0.05), neighborhood burglary index (SS = 0.04), and neighborhood median home values (SS = 0.03). Race was not significantly associated with MACE when adverse SDOH were included as covariates (adjusted subdistribution hazard ratio, 1.22; 95% CI, 0.91-1.64). NHB patients were more likely to have unfavorable SDOH conditions for 8 of the 10 most important SDOH variables for the MACE prediction.
Neighborhood and built environment variables are the most important SDOH predictors for 2-year MACE, and NHB patients were more likely to have unfavorable SDOH conditions. This finding reinforces that race is a social construct.
Display omitted
•Proposed a method to automatically classify symptom severity in psychiatric reports.•Question-answers from reports are the most important source of information.•Best predictive ...models automatically selected features prevalent in literature.
In response to the challenges set forth by the CEGS N-GRID 2016 Shared Task in Clinical Natural Language Processing, we describe a framework to automatically classify initial psychiatric evaluation records to one of four positive valence system severities: absent, mild, moderate, or severe. We used a dataset provided by the event organizers to develop a framework comprised of natural language processing (NLP) modules and 3 predictive models (two decision tree models and one Bayesian network model) used in the competition. We also developed two additional predictive models for comparison purpose. To evaluate our framework, we employed a blind test dataset provided by the 2016 CEGS N-GRID. The predictive scores, measured by the macro averaged-inverse normalized mean absolute error score, from the two decision trees and Naïve Bayes models were 82.56%, 82.18%, and 80.56%, respectively. The proposed framework in this paper can potentially be applied to other predictive tasks for processing initial psychiatric evaluation records, such as predicting 30-day psychiatric readmissions.
We aimed to gain a better understanding of how standardization of laboratory data can impact predictive model performance in multi-site datasets. We hypothesized that standardizing local laboratory ...codes to logical observation identifiers names and codes (LOINC) would produce predictive models that significantly outperform those learned utilizing local laboratory codes.
We predicted 30-day hospital readmission for a set of heart failure-specific visits to 13 hospitals from 2008 to 2012. Laboratory test results were extracted and then manually cleaned and mapped to LOINC. We extracted features to summarize laboratory data for each patient and used a training dataset (2008-2011) to learn models using a variety of feature selection techniques and classifiers. We evaluated our hypothesis by comparing model performance on an independent test dataset (2012).
Models that utilized LOINC performed significantly better than models that utilized local laboratory test codes, regardless of the feature selection technique and classifier approach used.
We quantitatively demonstrated the positive impact of standardizing multi-site laboratory data to LOINC prior to use in predictive models. We used our findings to argue for the need for detailed reporting of data standardization procedures in predictive modeling, especially in studies leveraging multi-site datasets extracted from electronic health records.
Abstract only
Introduction:
Cardiovascular disease (CVD) prevalence and cancer mortality are higher in non-Hispanic Blacks (NHB) relative to other racial subpopulations.
Hypothesis:
Among females ...with BC (≥18 years), there are racial differences in cardiac events (CEV) which can be explained by adverse social determinants of health (SDOH).
Methods:
Data were obtained from a Cleveland area integrated health care systems informatics platform. (2005-2019). Zip-code level demographic features were extracted using the SDOH Database from the Agency for Healthcare Research and Quality. The CEVs included were heart failure (HF), acute coronary syndrome (ACS), and ischemic stroke (IS). Multivariable Cox proportional hazards regression models were used to examine racial disparities in CEVs. Race-stratified (NHB and non-Hispanic Whites NHW) multivariable logistic regression was performed to assess the role of each SDOH in association with CEV.
Results:
This study included 9,022 females with breast cancer (BC) of which 18.8% were NHB, and 48.7% had a CEV. The racial differences in CEV and the role of zip-code level SDOH are presented in
Table 1.
These racial differences were not explained by the proportion of people unemployed in the zip code (p=0.42, race*unemployment % cross-product), proportional of people in the zip code with only a high school diploma (p=0.66, race*education % cross-product), and proportion of people in the zip code who use public transport (p=0.24, race * transport % cross-product).
Conclusion:
There are racial differences in the risk of ACS and IS after a BC diagnosis, but this does not seem to be explained by the 3 adverse zip-code level SDOH included in this study. Therefore, studies with individual-level SDOH and including other domains are necessary to better understand their relationship with CEVs.
Objective:
Technology-dependent children with medical complexity (CMC) are frequently admitted to the pediatric intensive care unit (PICU). The social risk factors for high PICU utilization in these ...children are not well described. The objective of this study was to describe the relationship between race, ethnicity, insurance status, estimated household income, and PICU admission following the placement of a tracheostomy and/or gastrostomy (GT) in CMC.
Study Design:
This was a retrospective multicenter study of children <19 years requiring tracheostomy and/or GT placement discharged from a hospital contributing to the Pediatric Health Information System (PHIS) database between January 2016 and March 2019. Primary predictors included estimated household income, insurance status, and race/ethnicity. Additional predictor variables collected included patient age, sex, number of chronic complex conditions (CCC), history of prematurity, and discharge disposition following index hospitalization. The primary outcome was need for PICU readmission within 30 days of hospital discharge. Secondary outcomes included repeated PICU admissions and total hospital costs within 1 year of tracheostomy and/or GT placement.
Results:
Patients requiring a PICU readmission within 30 days of index hospitalization for tracheostomy or GT placement accounted for 6% of the 20,085 included subjects. In multivariate analyses, public insurance OR 1.28 (95% C.I. 1.12–1.47),
p
< 0.001 was associated with PICU readmission within 30 days of hospital discharge while living below the federal poverty threshold (FPT) was associated with a lower odds of 30-day PICU readmission OR 0.7 (95% C.I. 0.51–0.95),
p
= 0.0267. Over 20% (
n
= 4,197) of children required multiple (>1) PICU admissions within one year from index hospitalization. In multivariate analysis, Black children OR 1.20 (95% C.I. 1.10–1.32),
p
< 0.001 and those with public insurance OR 1.34 (95% C.I. 1.24–1.46),
p
< 0.001 had higher odds of multiple PICU admissions. Social risk factors were not associated with total hospital costs accrued within 1 year of tracheostomy and/or GT placement.
Conclusions:
In a multicenter cohort study, Black children and those with public insurance had higher PICU utilization following tracheostomy and/or GT placement. Future research should target improving healthcare outcomes in these high-risk populations.
PDF
