Abstract only
e14070
Background: Watson for Oncology (WfO) is an artificial intelligence-based clinical decision-support system that offers potential therapeutic options to cancer-treating ...physicians. We reviewed studies of concordance between therapeutic options offered by WfO and treatment decisions made by individual clinicians (IC) and multidisciplinary tumor boards (MTB) in practice in gynecological cancers. Methods: We searched PubMed and an internal database to identify peer-reviewed WfO concordance studies of gynecological cancers published between 01/01/2015 and 06/30/2019. Concordance was defined as agreement between therapeutic options recommended or offered for consideration by WfO and treatment decisions made by IC or MTB. Mean concordance was calculated as a weighted average based on the number of patients per study. Statistical significance was evaluated by z-test of two proportions. Results: Our search identified 5 retrospective studies with 635 patients with cervical and ovarian cancers in China and Thailand; 4 compared WfO to MTB and 1 to IC. Overall WfO concordance with MTB and IC for both cancers was 77.2% (SD 11.6%). The concordance between MTB and WfO in cervical and ovarian cancers was 80.5% and 86.2%, respectively ( P = .21); IC concordance with WfO in cervical and ovarian cancers was 65.2% and 73.2%, respectively ( P = .18). MTB concordance with WfO for both cancers combined was 81.5%, significantly higher than the 67.9% IC concordance with WfO for both cancers ( P = .01). Conclusions: Studies of cervical and ovarian cancers demonstrated a statistically significantly higher concordance of MTB and WfO than IC and WFO, suggesting a role for WfO in supporting treatment-decision making in gynecological cancers that aligns with decisions made by MTB. Larger prospective studies are needed to evaluate the technical performance, usability, workflow integration, and clinical impact of WfO in gynecological cancers.Table: see text
Abstract
Increased scrutiny of artificial intelligence (AI) applications in healthcare highlights the need for real-world evaluations for effectiveness and unintended consequences. The complexity of ...healthcare, compounded by the user- and context-dependent nature of AI applications, calls for a multifaceted approach beyond traditional in silico evaluation of AI. We propose an interdisciplinary, phased research framework for evaluation of AI implementations in healthcare. We draw analogies to and highlight differences from the clinical trial phases for drugs and medical devices, and we present study design and methodological guidance for each stage.
Background
The first surge of the COVID-19 pandemic entirely altered healthcare delivery. Whether this also altered the receipt of high- and low-value care is unknown.
Objective
To test the ...association between the April through June 2020 surge of COVID-19 and various high- and low-value care measures to determine how the delivery of care changed.
Design
Difference in differences analysis, examining the difference in quality measures between the April through June 2020 surge quarter and the January through March 2020 quarter with the same 2 quarters’ difference the year prior.
Participants
Adults in the MarketScan® Commercial Database and Medicare Supplemental Database.
Main Measures
Fifteen low-value and 16 high-value quality measures aggregated into 8 clinical quality composites (4 of these low-value).
Key Results
We analyzed 9,352,569 adults. Mean age was 44 years (SD, 15.03), 52% were female, and 75% were employed. Receipt of nearly every type of low-value care decreased during the surge. For example, low-value cancer screening decreased 0.86% (95% CI, −1.03 to −0.69). Use of opioid medications for back and neck pain (DiD +0.94 95% CI, +0.82 to +1.07) and use of opioid medications for headache (DiD +0.38 95% CI, 0.07 to 0.69) were the only two measures to increase. Nearly all high-value care measures also decreased. For example, high-value diabetes care decreased 9.75% (95% CI, −10.79 to −8.71).
Conclusions
The first COVID-19 surge was associated with receipt of less low-value care and substantially less high-value care for most measures, with the notable exception of increases in low-value opioid use.
IBM(R) Watson for Oncology (WfO) is a clinical decision-support system (CDSS) that provides evidence-informed therapeutic options to cancer-treating clinicians. A panel of experienced oncologists ...compared CDSS treatment options to treatment decisions made by clinicians to characterize the quality of CDSS therapeutic options and decisions made in practice.
This study included patients treated between 1/2017 and 7/2018 for breast, colon, lung, and rectal cancers at Bumrungrad International Hospital (BIH), Thailand. Treatments selected by clinicians were paired with therapeutic options presented by the CDSS and coded to mask the origin of options presented. The panel rated the acceptability of each treatment in the pair by consensus, with acceptability defined as compliant with BIH's institutional practices. Descriptive statistics characterized the study population and treatment-decision evaluations by cancer type and stage.
Nearly 60% (187) of 313 treatment pairs for breast, lung, colon, and rectal cancers were identical or equally acceptable, with 70% (219) of WfO therapeutic options identical to, or acceptable alternatives to, BIH therapy. In 30% of cases (94), 1 or both treatment options were rated as unacceptable. Of 32 cases where both WfO and BIH options were acceptable, WfO was preferred in 18 cases and BIH in 14 cases. Colorectal cancers exhibited the highest proportion of identical or equally acceptable treatments; stage IV cancers demonstrated the lowest.
This study demonstrates that a system designed in the US to support, rather than replace, cancer-treating clinicians provides therapeutic options which are generally consistent with recommendations from oncologists outside the US.
Abstract only
e18081
Background: The Instituto do Câncer do Ceará (ICC), a 160-bed oncology hospital located in Brazil, serves approximately 23,000 patients monthly. In December of 2017, ICC ...implemented Watson for Oncology (WFO), an artificial intelligence (AI)-based clinical decision-support (CDS) tool to help enhance personalized cancer care. As of December 2018, 903 cases involving mainly breast, prostate and gastric cancers were entered in WFO. The purpose of this study was to investigate how implementation of WFO and use by oncologists affects clinical decision-making and workflow. Methods: 7 oncologists who employed WfO during and after the patients’ first visit were recruited to complete a survey regarding usability, decision-making and workflow. The group consisted of 1 urologist, 3 gastric surgeons, 1 gynecologist, 1 breast surgeon, 1 head-neck surgeon. Survey questions integrated the CDS Five Rights framework. Results: Most oncologists agreed that WFO is easy to understand and provides complete, relevant and actionable information at an appropriate time (Table). Opinions on the impact on treatment decisions varied. 71.4% expressed positive statements (agree or strongly agree) pertaining to the use of WFO. Conclusions: In this study, oncologists felt WFO met 5 Rights expectations for CDS; 57% felt that WFO exceed expectations. Further research is needed to understand how variation in experience affects decision impact. Table: see text
Abstract only
e20006
Background: Clinical decision-support systems (CDSS) provide up-to-date evidence to practitioners overwhelmed by the deluge of clinical findings. Few studies, however, have ...evaluated their impact on patient outcomes. This cross-sectional retrospective study was conducted to: 1) measure concordance of real-world clinical decisions with therapeutic options from the IBM Watson for Oncology (WfO) CDSS and the Chinese Society of Clinical Oncology (CSCO) guidelines, 2) the effect of concordance on objective response rate (ORR). Methods: Health records from patients receiving 1
st
line treatment at Peking University International Hospital Oncology Center in China between January 2016 and December 2018 (69 stage IV NSCLC, 30 with SCLC) with documented tumor progression after 2+ cycles of treatment, were reviewed to determine concordance of actual treatment with WfO therapeutic options and CSCO guidelines. Patients’ treatments were grouped as concordant with: WfO+CSCO, WFO only, CSCO only, or neither. ORR, defined as partial or complete response after 2+ treatment cycles (RECIST criteria)was determined for each group. Results: For NSCLC, ORR ranged from 21.4% for discordance with both WfO and CSCO to 100.0% for WfO only concordance. For SCLC, ORR ranged from 37.5% for CSCO only concordance to 73.3% for WfO+CSCO concordance (Table). The main reasons for discordance were: 1
st
generation TKIs like Gefitinib and Erlotinib (vs. Osimertinib) are standard of care for EGFR mutant patients in China, (2) Local CSCO guideline drugs like Lobaplatin and Icotinib are not included in WfO. Conclusions: This study provides preliminary evidence to suggest that treatment concordance with WfO may be associated with improved ORR in some cases of NSCLC (WfO only) and SCLC (WfO + CSCO). ORR in NSCLC patients who were discordant with both WFO and CSCO guidelines was the lowest at 21.4%. Larger studies are needed to understand the effect of guideline and WfO concordance on ORR. Table: see text
Abstract only
e14061
Background: Advances in artificial intelligence (AI) continue to expand capabilities within the healthcare domain, particularly in the discipline of oncology. Watson For Oncology ...(WfO) is an AI-enabled clinical decision support system that presents potential therapeutic options for cancer-treating physicians. The objectives of this study were to identify non-user physicians’ expectations, perceived challenges and benefits of WfO use in Brazil. Methods: The study took place at Instituto do Câncer do Ceará (ICC), a Brazilian oncology hospital that implemented WfO in December 2017, but not all physicians adopted the tool. Physicians who had not used WfO (n = 5) were recruited through purposive sampling identified with the assistance of local research personnel. Semi-structured interviews were conducted in Portuguese and later de-identified and transcribed into English. A thematic analysis of interview data based on grounded theory by two members of the research team with extensive experience in qualitative data analysis was conducted. Results: Non-user physicians had positive perceptions about WfO, along with several concerns and uncertainties. They expected that WfO would be easy to learn, useful, and helpful. Physicians perceived that WfO would provide a more standardized approach to treatment than care without it. They also believed that WfO would play a supportive and not a substitute role in care especially for complex cases in which the physicians had more in-depth knowledge of a patient and already had an established patient-provider relationship. Physicians did expect WfO use to negatively impact productivity, specifically through longer office times per patient because of the need to enter data and review recommendations. Physicians questioned whether the use of WfO would negatively impact their autonomy and role in providing care. Finally, physicians also questioned whether the treatment suggested by WfO would fit the social context of a low-middle income country such as Brazil with limited technological and economic resources. Conclusions: The implementation of US-developed AI technologies, such as WfO, should be further explored in different social and economic contexts. Physician concerns about productivity and autonomy need to be assessed and addressed in AI implementation; one strategy is to leverage previous lessons learned from electronic health record (EHR) implementations. This study is a critical step in understanding potential user perspectives in adopting a new AI tool in different social contexts.
Sickle cell disease (SCD) is a chronic condition affecting over 100 000 individuals in the United States, predominantly from vulnerable populations. Clinical practice guidelines, written for ...providers, have low adherence. This study explored knowledge about guidelines; desire for guidelines; and how technology could support guideline awareness and adherence, examining current technology uses, and user preferences to inform design of a patient-centered guidelines application in a chronic disease.
This cross-sectional mixed-methods study involved semi-structured interviews, surveys, and focus groups of adolescents and adults with SCD. We evaluated interest, preferences, and anticipated benefits or barriers of a patient-centered adaptation of SCD practice guidelines; prospective technology uses for health; and barriers to technology utilization.
Forty-seven individuals completed surveys and interviews, and 39 participated in three separate focus groups. Most participants (91%) were unaware of SCD guidelines, but almost all (96%) expressed interest in a guidelines application, identifying benefits (knowledge, activation, individualization, and rewards), and barriers (poor information, low motivation, and resource limitations). Current technology health uses included information access, care coordination, and reminders about health-related actions. Prospective technology uses included informational messaging and timely alerts. Barriers to technology use included lack of interest, lack of utility, and preference for direct communication.
This study's findings can inform the design of clinical practice guideline applications, suggesting a promising role for technology to engage patients, facilitate care decisions and actions, and improve outcomes.
Abstract only
e19071
Background: Programs to address disparities in cancer care outcomes in resource-limited settings require attention to social determinants of health (SDoH) to achieve successful ...clinical care implementation. The Instituto de Câncer do Ceará, the largest cancer center in northeastern Brazil, has implemented a Social Responsibility Agenda (SRA) to guide equitable cancer care delivery. This goal of this study was to develop a framework for an implementation science (IS) study evaluating the longitudinal impact of the SRA on cancer outcomes. Methods: We outlined a mixed-methods and participatory study incorporating a process model, the Consolidated Framework for Implementation Research (CFIR) and the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) evaluation framework. A list of constructs and links to measurement tools associated with IS models were identified to guide the study phases. Results: We established a logic model to guide in evaluating the health and economic impact of the SRA. We identified >30 constructs and measures across domains of IS models. The table shows a driver diagram to inform the framework. Conclusions: Understanding determinants, key drivers and change concepts are important initial steps in an ongoing evaluation of the impact of evidence based SDoH interventions to address cancer disparities. Table: see text
PDF
