Abstract only
e14114
Background: Watson for Oncology (WfO) is an artificial intelligence-based clinical decision-support system which provides therapeutic options and associated scientific evidence ...to cancer-treating physicians. Oncologists at Bumrungrad International Hospital (BIH) have used WfO since 2015. We examined the association between concordance of WfO therapeutic options and BIH treatment decisions with short-term clinical outcomes for lung cancer patients. Methods: This study included lung cancer patients seen at BIH for treatment and follow-up care and for whom WfO was used from 2015 to 2018. Charts were reviewed for concordance with WfO, documentation of disease progression, response to treatment, and survival. We evaluated concordance between oncologists’ treatments and therapeutic options listed as “recommended” by WfO. We evaluated association between WfO concordance and partial or complete response rates over a 24-month period by comparison of proportions with odds ratio. Progression-free survival (PFS, time from diagnosis until progression or death) was evaluated by Kaplan-Meier log-rank test. Results: Seventy-nine lung cancer patients were included. We identified a trend towards higher response rates in concordant cases (59.2%, N = 32), as compared to discordant (48.0%, N = 12), with an odds ratio of 1.56 (see table). There was not a significant difference in PFS between concordant and discordant cohorts. Conclusions: In this small-cohort, retrospective study, lung cancer patients receiving treatments that are concordant with WfO recommended therapeutic options trended towards higher response rates than patients with discordant treatments. Use of a clinical decision-support system may help support cancer-treating physicians in delivering best practice and evidence-based care that may improve short-term outcomes. Prospective studies with larger samples and other cancer types are underway. Table: see text
Abstract Background/Purpose The management of asymptomatic congenital lung lesions is controversial. It is unclear whether elective resection provides a significant benefit. We sought to determine ...whether early vs delayed resection of asymptomatic congenital lung malformations resulted in complications. Methods Institutional billing records were queried for patients with lung malformations over a 10-year period. Medical records were reviewed for demographics, type of anomaly, symptoms, management, and procedural or disease-related complications. Results Eighty-seven patients were identified. The diagnoses included congenital cystic adenomatoid malformation (41%), bronchogenic cyst (19.3%), sequestration (13.2%), and congenital lobar emphysema (12.0%). Fifty patients were observed for some period. Eleven became symptomatic, and 47 underwent resection at a mean age of 11 months. There was no difference in the type of resection, length of hospitalization, or complication rate between patients who underwent early vs delayed resection. There were no occurrences of malignancy or death. Conclusions In our series, there was no difference in measurable outcomes between early and delayed resection of congenital lung lesions. These data provide some support for a management strategy that might include observation with delayed resection for asymptomatic patients.
The COVID-19 pandemic presented a challenge to inpatient safety. It is unknown whether there were spillover effects due to COVID-19 into non-COVID-19 care and safety. We sought to evaluate the ...changes in inpatient Agency for Healthcare Research and Quality patient safety indicators (PSIs) in the United States before and during the first surge of the pandemic among patients admitted without COVID-19.
We analyzed trends in PSIs from January 2019 to June 2020 in patients without COVID-19 using data from IBM MarketScan Commercial Database. We included members of employer-sponsored or Medicare supplemental health plans with inpatient, non-COVID-19 admissions. The primary outcomes were risk-adjusted composite and individual PSIs.
We analyzed 1,869,430 patients admitted without COVID-19. Among patients without COVID-19, the composite PSI score was not significantly different when comparing the first surge (Q2 2020) to the prepandemic period (e.g., Q2 2020 score of 2.46 95% confidence interval {CI}, 2.34-2.58 versus Q1 2020 score of 2.37 95% CI, 2.27-2.46; P = 0.22). Individual PSIs for these patients during Q2 2020 were also not significantly different, except in-hospital fall with hip fracture (e.g., Q2 2020 was 3.42 95% CI, 3.34-3.49 versus Q4 2019 was 2.45 95% CI, 2.40-2.50; P = 0.01).
The first surge of COVID-19 was not associated with worse inpatient safety for patients without COVID-19, highlighting the ability of the healthcare system to respond to the initial surge of the pandemic.
Patients have diverse health information needs, and secure messaging through patient portals is an emerging means by which such needs are expressed and met. As patient portal adoption increases, ...growing volumes of secure messages may burden healthcare providers. Automated classification could expedite portal message triage and answering. We created four automated classifiers based on word content and natural language processing techniques to identify health information needs in 1000 patient-generated portal messages. Logistic regression and random forest classifiers detected single information needs well, with area under the curves of 0.804-0.914. A logistic regression classifier accurately found the set of needs within a message, with a Jaccard index of 0.859 (95% Confidence Interval: (0.847, 0.871)). Automated classification of consumer health information needs expressed in patient portal messages is feasible and may allow direct linking to relevant resources or creation of institutional resources for commonly expressed needs.
Screening patients for eligibility for clinical trials is labor intensive. It requires abstraction of data elements from multiple components of the longitudinal health record and matching them to ...inclusion and exclusion criteria for each trial. Artificial intelligence (AI) systems have been developed to improve the efficiency and accuracy of this process.
This study aims to evaluate the ability of an AI clinical decision support system (CDSS) to identify eligible patients for a set of clinical trials.
This study included the deidentified data from a cohort of patients with breast cancer seen at the medical oncology clinic of an academic medical center between May and July 2017 and assessed patient eligibility for 4 breast cancer clinical trials. CDSS eligibility screening performance was validated against manual screening. Accuracy, sensitivity, specificity, positive predictive value, and negative predictive value for eligibility determinations were calculated. Disagreements between manual screeners and the CDSS were examined to identify sources of discrepancies. Interrater reliability between manual reviewers was analyzed using Cohen (pairwise) and Fleiss (three-way) κ, and the significance of differences was determined by Wilcoxon signed-rank test.
In total, 318 patients with breast cancer were included. Interrater reliability for manual screening ranged from 0.60-0.77, indicating substantial agreement. The overall accuracy of breast cancer trial eligibility determinations by the CDSS was 87.6%. CDSS sensitivity was 81.1% and specificity was 89%.
The AI CDSS in this study demonstrated accuracy, sensitivity, and specificity of greater than 80% in determining the eligibility of patients for breast cancer clinical trials. CDSSs can accurately exclude ineligible patients for clinical trials and offer the potential to increase screening efficiency and accuracy. Additional research is needed to explore whether increased efficiency in screening and trial matching translates to improvements in trial enrollment, accruals, feasibility assessments, and cost.
The objective of this technical study was to evaluate the performance of an artificial intelligence (AI)-based system for clinical trials matching for a cohort of lung cancer patients in an ...Australian cancer hospital.
A lung cancer cohort was derived from clinical data from patients attending an Australian cancer hospital. Ten phases I-III clinical trials registered on clinicaltrials.gov and open to lung cancer patients at this institution were utilized for assessments. The trial matching system performance was compared to a gold standard established by clinician consensus for trial eligibility.
The study included 102 lung cancer patients. The trial matching system evaluated 7252 patient attributes (per patient median 74, range 53-100) against 11 467 individual trial eligibility criteria (per trial median 597, range 243-4132). Median time for the system to run a query and return results was 15.5 s (range 7.2-37.8). In establishing the gold standard, clinician interrater agreement was high (Cohen's kappa 0.70-1.00). On a per-patient basis, the performance of the trial matching system for eligibility was as follows: accuracy, 91.6%; recall (sensitivity), 83.3%; precision (positive predictive value), 76.5%; negative predictive value, 95.7%; and specificity, 93.8%.
The AI-based clinical trial matching system allows efficient and reliable screening of cancer patients for clinical trials with 95.7% accuracy for exclusion and 91.6% accuracy for overall eligibility assessment; however, clinician input and oversight are still required. The automated system demonstrates promise as a clinical decision support tool to prescreen a large patient cohort to identify subjects suitable for further assessment.
Patient portal adoption has rapidly increased, and portal usage has been associated with patients' sociodemographics, health literacy, and education. Research on patient portals has primarily focused ...on the outpatient setting. We explored whether health literacy and education were associated with portal usage in an inpatient population. Among 60,159 admissions in 2012-2013, 23.3% of patients reported limited health literacy; 50.4% reported some post-secondary education; 34.4% were registered for the portal; and 23.4% of registered patients used the portal during hospitalization. Probability of registration and inpatient portal use increased with educational attainment. Health literacy was associated with registration but not inpatient use. Among admissions with inpatient use, educational attainment was associated with viewing health record data, and health literacy was associated use of appointment and health education tools. The inpatient setting may provide an opportunity to overcome barriers to patient portal adoption and reduce disparities in use of health information technologies.
This article describes the system architecture, training, initial use, and performance of Watson Assistant (WA), an artificial intelligence-based conversational agent, accessible within Micromedex
.
...The number and frequency of
(target of a user's query) triggered in WA during its initial use were examined; intents triggered over 9 months were compared to the frequency of topics accessed via keyword search of Micromedex. Accuracy of WA intents assigned to 400 queries was compared to assignments by 2 independent subject matter experts (SMEs), with inter-rater reliability measured by Cohen's kappa.
In over 126 000 conversations with WA, intents most frequently triggered involved dosing (
= 30 239, 23.9%) and administration (
= 14 520, 11.5%). SMEs with substantial inter-rater agreement (kappa = 0.71) agreed with intent mapping in 247 of 400 queries (62%), including 16 queries related to content that WA and SMEs agreed was unavailable in WA. SMEs found 57 (14%) of 400 queries incorrectly mapped by WA; 112 (28%) queries unanswerable by WA included queries that were either ambiguous, contained unrecognized typographical errors, or addressed topics unavailable to WA. Of the queries answerable by WA (288), SMEs determined 231 (80%) were correctly linked to an intent.
A conversational agent successfully linked most queries to intents in Micromedex. Ongoing system training seeks to widen the scope of WA and improve matching capabilities.
WA enabled Micromedex users to obtain answers to many medication-related questions using natural language, with the conversational agent facilitating mapping to a broader distribution of topics than standard keyword searches.
Explicit guidelines are needed to develop safe and effective patient portals. This paper proposes general principles, policies, and procedures for patient portal functionality based on ...MyHealthAtVanderbilt (MHAV), a robust portal for Vanderbilt University Medical Center. We describe policies and procedures designed to govern popular portal functions, address common user concerns, and support adoption. We present the results of our approach as overall and function-specific usage data. Five years after implementation, MHAV has over 129,800 users; 45% have used bi-directional messaging; 52% have viewed test results and 45% have viewed other medical record data; 30% have accessed health education materials; 39% have scheduled appointments; and 29% have managed a medical bill. Our policies and procedures have supported widespread adoption and use of MHAV. We believe other healthcare organizations could employ our general guidelines and lessons learned to facilitate portal implementation and usage.
PDF
