The COVID-19 pandemic accelerated the adoption of remote patient monitoring technology, which offers exciting opportunities for expanded connected care at a distance. However, while the mode of ...clinicians' interactions with patients and their health data has transformed, the larger framework of how we deliver care is still driven by a model of episodic care that does not facilitate this new frontier. Fully realizing a transformation to a system of continuous connected care augmented by remote monitoring technology will require a shift in clinicians' and health systems' approach to care delivery technology and its associated data volume and complexity. In this article, we present a solution that organizes and optimizes the interaction of automated technologies with human oversight, allowing for the maximal use of data-rich tools while preserving the pieces of medical care considered uniquely human. We review implications of this "augmented continuous connected care" model of remote patient monitoring for clinical practice and offer human-centered design-informed next steps to encourage innovation around these important issues.
To evaluate demographic biases in diagnostic accuracy and health advice between generative artificial intelligence (AI) (ChatGPT GPT-4) and traditional symptom checkers like WebMD.
Combination ...symptom and demographic vignettes were developed for 27 most common symptom complaints. Standardized prompts, written from a patient perspective, with varying demographic permutations of age, sex, and race/ethnicity were entered into ChatGPT (GPT-4) between July and August 2023. In total, 3 runs of 540 ChatGPT prompts were compared to the corresponding WebMD Symptom Checker output using a mixed-methods approach. In addition to diagnostic correctness, the associated text generated by ChatGPT was analyzed for readability (using Flesch-Kincaid Grade Level) and qualitative aspects like disclaimers and demographic tailoring.
ChatGPT matched WebMD in 91% of diagnoses, with a 24% top diagnosis match rate. Diagnostic accuracy was not significantly different across demographic groups, including age, race/ethnicity, and sex. ChatGPT's urgent care recommendations and demographic tailoring were presented significantly more to 75-year-olds versus 25-year-olds (P < .01) but were not statistically different among race/ethnicity and sex groups. The GPT text was suitable for college students, with no significant demographic variability.
The use of non-health-tailored generative AI, like ChatGPT, for simple symptom-checking functions provides comparable diagnostic accuracy to commercially available symptom checkers and does not demonstrate significant demographic bias in this setting. The text accompanying differential diagnoses, however, suggests demographic tailoring that could potentially introduce bias.
These results highlight the need for continued rigorous evaluation of AI-driven medical platforms, focusing on demographic biases to ensure equitable care.
Abstract Lifestyle behavior changes can prevent progression of prediabetes to diabetes but providers often are not able to effectively counsel about preventive lifestyle changes. We developed and ...pilot tested the Avoiding Diabetes Thru Action Plan Targeting (ADAPT) program to enhance primary care providers' counseling about behavior change for patients with prediabetes. Primary care providers in two urban academic practices and their patients with prediabetes were recruited to participate in the ADAPT study, an unblinded randomized pragmatic trial to test the effectiveness of the ADAPT program, including a streamlined electronic medical record-based goal setting tool. Providers were randomized to intervention or control arms; eligible patients whose providers were in the intervention arm received the ADAPT program. Physical activity (the primary outcome) was measured using pedometers, and data were gathered about patients' diet, weight and glycemic control. A total of 54 patients were randomized and analyzed as part of the 6-month ADAPT study (2010–2012, New York, NY). Those in the intervention group showed an increase total daily steps compared to those in the control group (+ 1418 vs − 598, p = 0.007) at 6 months. There was also a trend towards weight loss in the intervention compared to the control group (− 1.0 lbs. vs. 3.0 lbs., p = 0.11), although no change in glycemic control. The ADAPT study is among the first to use standard electronic medical record tools to embed goal setting into realistic primary care workflows and to demonstrate a significant improvement in prediabetes patients' physical activity.
Nonadherence to cardiovascular medications such as statins is a common, important problem. Clinicians currently rely on intuition to identify medication nonadherence. The visit-to-visit variability ...(VVV) of low-density lipoprotein (LDL) cholesterol might represent an opportunity to identify statin nonadherence with greater accuracy. We examined the clinical and pharmacy data from 782 members of the Boston Medical Center Health Plan, seen at either the Boston Medical Center or its affiliated community health centers, who were taking statins and had ≥3 LDL cholesterol measurements from 2008 to 2011. The LDL cholesterol VVV (defined by the within-patient SD) was categorized into quintiles. Multivariate logistic regression models were generated with statin nonadherence (defined by the standard 80% pharmacy refill-based medication possession ratio threshold) as the dependent variable. The proportion of statin nonadherence increased across the quintiles of LDL cholesterol VVV (64.3%, 71.2%, 89.2%, 92.3%, 91.7%). Higher quintiles of LDL cholesterol VVV had a strong positive association with statin nonadherence, with an adjusted odds ratio of 3.4 (95% confidence interval 1.7 to 7.1) in the highest versus lowest quintile of LDL cholesterol VVV. The age- and gender-adjusted model had poor discrimination (C-statistic 0.62, 95% confidence interval 0.57 to 0.67), but the final adjusted model (age, gender, race, mean LDL cholesterol) demonstrated good discrimination (C-statistic 0.75, 95% confidence interval 0.71 to 0.79) between the adherent and nonadherent patients. In conclusion, the VVV of LDL cholesterol demonstrated a strong association with statin nonadherence in a clinic setting. Furthermore, a VVV of LDL cholesterol-based model had good discrimination characteristics for statin nonadherence. Research is needed to validate and generalize these findings to other populations and biomarkers.
There is consensus that incorporating clinical decision support into electronic health records will improve quality of care, contain costs, and reduce overtreatment, but this potential has yet to be ...demonstrated in clinical trials.
To assess the influence of a customized evidence-based clinical decision support tool on the management of respiratory tract infections and on the effectiveness of integrating evidence at the point of care.
In a randomized clinical trial, we implemented 2 well-validated integrated clinical prediction rules, namely, the Walsh rule for streptococcal pharyngitis and the Heckerling rule for pneumonia. INTERVENTIONS AND MAIN OUTCOMES AND MEASURES: The intervention group had access to the integrated clinical prediction rule tool and chose whether to complete risk score calculators, order medications, and generate progress notes to assist with complex decision making at the point of care.
The intervention group completed the integrated clinical prediction rule tool in 57.5% of visits. Providers in the intervention group were significantly less likely to order antibiotics than the control group (age-adjusted relative risk, 0.74; 95% CI, 0.60-0.92). The absolute risk of the intervention was 9.2%, and the number needed to treat was 10.8. The intervention group was significantly less likely to order rapid streptococcal tests compared with the control group (relative risk, 0.75; 95% CI, 0.58-0.97; P= .03).
The integrated clinical prediction rule process for integrating complex evidence-based clinical decision report tools is of relevant importance for national initiatives, such as Meaningful Use. TRIAL REGISTRATION clinicaltrials.gov Identifier: NCT01386047.
Clinical decision support (CDS) is a valuable feature of electronic health records (EHRs) designed to improve quality and safety. However, due to the complexities of system design and inconsistent ...results, CDS tools may inadvertently increase alert fatigue and contribute to physician burnout. A/B testing, or rapid-cycle randomized tests, is a useful method that can be applied to the EHR in order to rapidly understand and iteratively improve design choices embedded within CDS tools.
This paper describes how rapid randomized controlled trials (RCTs) embedded within EHRs can be used to quickly ascertain the superiority of potential CDS design changes to improve their usability, reduce alert fatigue, and promote quality of care.
A multistep process combining tools from user-centered design, A/B testing, and implementation science was used to understand, ideate, prototype, test, analyze, and improve each candidate CDS. CDS engagement metrics (alert views, acceptance rates) were used to evaluate which CDS version is superior.
To demonstrate the impact of the process, 2 experiments are highlighted. First, after multiple rounds of usability testing, a revised CDS influenza alert was tested against usual care CDS in a rapid (~6 weeks) RCT. The new alert text resulted in minimal impact on reducing firings per patients per day, but this failure triggered another round of review that identified key technical improvements (ie, removal of dismissal button and firings in procedural areas) that led to a dramatic decrease in firings per patient per day (23.1 to 7.3). In the second experiment, the process was used to test 3 versions (financial, quality, regulatory) of text supporting tobacco cessation alerts as well as 3 supporting images. Based on 3 rounds of RCTs, there was no significant difference in acceptance rates based on the framing of the messages or addition of images.
These experiments support the potential for this new process to rapidly develop, deploy, and rigorously evaluate CDS within an EHR. We also identified important considerations in applying these methods. This approach may be an important tool for improving the impact of and experience with CDS.
Flu alert trial: ClinicalTrials.gov NCT03415425; https://clinicaltrials.gov/ct2/show/NCT03415425. Tobacco alert trial: ClinicalTrials.gov NCT03714191; https://clinicaltrials.gov/ct2/show/NCT03714191.
The seminal Dietary Approaches to Stopping Hypertension (DASH) study demonstrated the effectiveness of diet to control hypertension; however, the effective implementation and dissemination of its ...principles have been limited.
This study aimed to determine the feasibility and effectiveness of a DASH mobile health intervention. We hypothesized that combining Bluetooth-enabled data collection, social networks, and a human coach with a smartphone DASH app (DASH Mobile) would be an effective medium for the delivery of the DASH program.
We conducted a single-arm pilot study from August 2015 through August 2016, using a pre-post evaluation design to evaluate the feasibility and preliminary effectiveness of a smartphone version of DASH that incorporated a human health coach. Participants were recruited both online and offline.
A total of 17 patients participated in this study; they had a mean age of 59 years (SD 6) and 10 (60%) were women. Participants were engaged with the app; in the 120 days of the study, the mean number of logged blood pressure measurements was 63 (SD 46), the mean number of recorded weight measurements was 52 (SD 45), and participants recorded a mean of 55 step counts (SD 36). Coaching phone calls had a high completion rate (74/102, 73%). The mean number of servings documented per patient for the dietary assessment was 709 (SD 541), and patients set a mean number of 5 (SD 2) goals. Mean systolic and diastolic blood pressure, heart rate, weight, body mass index, and step count did not significantly change over time (P>.10 for all parameters).
In this pilot study, we found that participants were engaged with an interactive mobile app that promoted healthy behaviors to treat hypertension. We did not find a difference in the physiological outcomes, but were underpowered to identify such changes.
The integration of behavioral economics (BE) principles and electronic health records (EHRs) using clinical decision support (CDS) tools is a novel approach to improving health outcomes. Meanwhile, ...the American Geriatrics Society has created the Choosing Wisely (CW) initiative to promote less aggressive glycemic targets and reduction in pharmacologic therapy in older adults with type 2 diabetes mellitus. To date, few studies have shown the effectiveness of combined BE and EHR approaches for managing chronic conditions, and none have addressed guideline-driven deprescribing specifically in type 2 diabetes. We previously conducted a pilot study aimed at promoting appropriate CW guideline adherence using BE nudges and EHRs embedded within CDS tools at 5 clinics within the New York University Langone Health (NYULH) system. The BE-EHR module intervention was tested for usability, adoption, and early effectiveness. Preliminary results suggested a modest improvement of 5.1% in CW compliance.
This paper presents the protocol for a study that will investigate the effectiveness of a BE-EHR module intervention that leverages BE nudges with EHR technology and CDS tools to reduce overtreatment of type 2 diabetes in adults aged 76 years and older, per the CW guideline.
A pragmatic, investigator-blind, cluster randomized controlled trial was designed to evaluate the BE-EHR module. A total of 66 NYULH clinics will be randomized 1:1 to receive for 18 months either (1) a 6-component BE-EHR module intervention + standard care within the NYULH EHR, or (2) standard care only. The intervention will be administered to clinicians during any patient encounter (eg, in person, telemedicine, medication refill, etc). The primary outcome will be patient-level CW compliance. Secondary outcomes will measure the frequency of intervention component firings within the NYULH EHR, and provider utilization and interaction with the BE-EHR module components.
Study recruitment commenced on December 7, 2020, with the activation of all 6 BE-EHR components in the NYULH EHR.
This study will test the effectiveness of a previously developed, iteratively refined, user-tested, and pilot-tested BE-EHR module aimed at providing appropriate diabetes care to elderly adults, compared to usual care via a cluster randomized controlled trial. This innovative research will be the first pragmatic randomized controlled trial to use BE principles embedded within the EHR and delivered using CDS tools to specifically promote CW guideline adherence in type 2 diabetes. The study will also collect valuable information on clinician workflow and interaction with the BE-EHR module, guiding future research in optimizing the timely delivery of BE nudges within CDS tools. This work will address the effectiveness of BE-inspired interventions in diabetes and chronic disease management.
ClinicalTrials.gov NCT04181307; https://clinicaltrials.gov/ct2/show/NCT04181307.
DERR1-10.2196/28723.
Design thinking and human-centered design approaches have become increasingly common in health care literature, particularly in relation to health information technology (HIT), as a pathway toward ...the development of usable, diffusible tools and processes. There is a need in academic medical centers tasked with digital innovation for a comprehensive process model to guide development that incorporates current industry trends, including design thinking and lean and agile approaches to digital development.
This study aims to describe the foundations and phases of our model for user-centered HIT development.
Based on our experience, we established an integrated approach and rigorous process for HIT development that leverages design thinking and lean and agile strategies in a pragmatic way while preserving methodological integrity in support of academic research goals.
A four-phased pragmatic process model was developed for user-centered digital development in HIT.
The model for user-centered HIT development that we developed is the culmination of diverse innovation projects and represents a multiphased, high-fidelity process for making more creative, flexible, efficient, and effective tools. This model is a critical step in building a rigorous approach to HIT design that incorporates a multidisciplinary, pragmatic perspective combined with academic research practices and state-of-the-art approaches to digital product development to meet the unique needs of health care.
Indiscriminate use of predictive models incorporating race can reinforce biases present in source data and lead to an exacerbation of health disparities. In some countries, such as the United States, ...there is therefore a push to remove race from prediction models; however, there are still many prediction models that use race as an input. Biomedical informaticists who are given the responsibility of using these predictive models in healthcare environments are likely to be faced with questions like how to deal with race covariates in these models. Thus, there is a need for a pragmatic framework to help model users think through how to include race in their chosen model so as to avoid inadvertently exacerbating disparities. In this paper, we use the case study of lung cancer screening to propose a simple framework to guide how model users can approach the use (or non-use) of race inputs in the predictive models they are tasked with leveraging in electronic health records and clinical workflows.
PDF
