With rising smartphone ownership, mobile health applications (mHealth apps) have the potential to support high-need, high-cost populations in managing their health. While the number of available ...mHealth apps has grown substantially, no clear strategy has emerged on how providers should evaluate and recommend such apps to patients. Key stakeholders, including medical professional societies, insurers, and policy makers, have largely avoided formally recommending apps, which forces patients to obtain recommendations from other sources. To help stakeholders overcome barriers to reviewing and recommending apps, we evaluated 137 patient-facing mHealth apps-those intended for use by patients to manage their health-that were highly rated by consumers and recommended by experts and that targeted high-need, high-cost populations. We found that there is a wide variety of apps in the marketplace but that few apps address the needs of the patients who could benefit the most. We also found that consumers' ratings were poor indications of apps' clinical utility or usability and that most apps did not respond appropriately when a user entered potentially dangerous health information. Going forward, data privacy and security will continue to be major concerns in the dissemination of mHealth apps.
Mobile health applications ("apps") have rapidly proliferated, yet their ability to improve outcomes for patients remains unclear. A validated tool that addresses apps' potentially important ...dimensions has not been available to patients and clinicians. The objective of this study was to develop and preliminarily assess a usable, valid, and open-source rating tool to objectively measure the risks and benefits of health apps. We accomplished this by using a Delphi process, where we constructed an app rating tool called THESIS that could promote informed app selection. We used a systematic process to select chronic disease apps with ≥4 stars and <4-stars and then rated them with THESIS to examine the tool's interrater reliability and internal consistency. We rated 211 apps, finding they performed fair overall (3.02 out of 5 95% CI, 2.96-3.09), but especially poorly for privacy/security (2.21 out of 5 95% CI, 2.11-2.32), interoperability (1.75 95% CI, 1.59-1.91), and availability in multiple languages (1.43 out of 5 95% CI, 1.30-1.56). Ratings using THESIS had fair interrater reliability (κ = 0.3-0.6) and excellent scale reliability (ɑ = 0.85). Correlation with traditional star ratings was low (r = 0.24), suggesting THESIS captures issues beyond general user acceptance. Preliminary testing of THESIS suggests apps that serve patients with chronic disease could perform much better, particularly in privacy/security and interoperability. THESIS warrants further testing and may guide software and policymakers to further improve app performance, so apps can more consistently improve patient outcomes.
Despite the broad adoption of electronic health record (EHR) systems across the continuum of care, safety problems persist.
To measure the safety performance of operational EHRs in hospitals across ...the country during a 10-year period.
This case series included all US adult hospitals nationwide that used the National Quality Forum Health IT Safety Measure EHR computerized physician order entry safety test administered by the Leapfrog Group between 2009 and 2018. Data were analyzed from July 1, 2018 to December 1, 2019.
The Health IT Safety Measure test, which uses simulated medication orders that have either injured or killed patients previously to evaluate how well hospital EHRs could identify medication errors with potential for patient harm.
Descriptive statistics for performance on the assessment test over time were calculated at the overall test score level, type of decision support category level, and EHR vendor level.
Among 8657 hospital-years observed during the study, mean (SD) scores on the overall test increased from 53.9% (18.3%) in 2009 to 65.6% (15.4%) in 2018. Mean (SD) hospital score for the categories representing basic clinical decision support increased from 69.8% (20.8%) in 2009 to 85.6% (14.9%) in 2018. For the categories representing advanced clinical decision support, the mean (SD) score increased from 29.6% (22.4%) in 2009 to 46.1% (21.6%) in 2018. There was considerable variation in test performance by EHR.
These findings suggest that despite broad adoption and optimization of EHR systems in hospitals, wide variation in the safety performance of operational EHR systems remains across a large sample of hospitals and EHR vendors. Hospitals using some EHR vendors had significantly higher test scores. Overall, substantial safety risk persists in current hospital EHR systems.
Failure to follow-up nonurgent, clinically significant test results (CSTRs) is an ambulatory patient safety concern. Tools within electronic health records (EHRs) may facilitate test result ...acknowledgment, but their utility with regard to nonurgent CSTRs is unclear. We measured use of an acknowledgment tool by 146 primary care physicians (PCPs) at 13 network-affiliated practices that use the same EHR. We then surveyed PCPs to assess use of, satisfaction with, and desired enhancements to the acknowledgment tool. The rate of acknowledgment of non-urgent CSTRs by PCPs was 78%. Of 73 survey respondents, 72 reported taking one or more actions after reviewing a CSTR; fewer (40–75%) reported that using the acknowledgment tool was helpful for a specific purpose. Forty-six (64%) were satisfied with the tool. Both satisfied and nonsatisfied PCPs reported that enhancements linking acknowledgment to routine actions would be useful. EHR vendors should consider enhancements to acknowledgment functionality to ensure follow-up of nonurgent CSTRs.
BACKGROUND:Patient-centered medical home (PCMH) has gained prominence as a promising model to encourage improved primary care delivery. There is a paucity of studies that evaluate the impact of ...payment models in the PCMH.
OBJECTIVES:We sought to examine whether coupling coordinated, team-based care transformation plan with a novel reimbursement model affects outcomes related to expenditures and utilization.
RESEARCH DESIGN:Interrupted time-series model with a difference-in-differences approach to assess differences between intervention and control groups, across time periods attributable to PCMH transformation and/or payment change.
RESULTS:Although results were modest and mixed overall, PCMH with payment reform is associated with a reduction of $1.04 (P=0.0347) per member per month (PMPM) in pharmacy expenditures. Patients with hypertension, hyperlipidemia, diabetes, and coronary atherosclerosis enrolled in PCMH without payment reform experienced reductions in emergency department visits of 2.16 (P<0.0001), 2.42 (P<0.0001), 3.98 (P<0.0001), and 3.61 (P<0.0001) per 1000 per month. Modest increases in inpatient admission were seen among these patients in PCMH either with or without payment reform. Patients 65 and older enrolled in PMCH without payment reform experienced reductions in pharmacy expenditures $2.35 (P=0.0077) PMPM with a parallel reduction in pharmacy standardized cost of $2.81 (P=0.0174) PMPM indicative of a reduction in the intensity of drug utilization.
CONCLUSIONS:We conclude that PCMH implementation coupled with an innovative payment arrangement generated mixed results with modest improvements with respect to pharmacy expenditures, but no overall financial improvement. However, we did see improvement within specific groups, especially older patients and those with chronic conditions.
Rising ownership of smartphones and tablets across social and demographic groups has made mobile applications, or apps, a potentially promising tool for engaging patients in their health care, ...particularly those with high health care needs. Through a systematic search of iOS (Apple) and Android app stores and an analysis of apps targeting individuals with chronic illnesses, we assessed the degree to which apps are likely to be useful in patient engagement efforts. Usefulness was determined based on the following criteria: description of engagement, relevance to the targeted patient population, consumer ratings and reviews, and most recent app update. Among the 1,046 health care-related, patient-facing applications identified by our search, 43 percent of iOS apps and 27 percent of Android apps appeared likely to be useful. We also developed criteria for evaluating the patient engagement, quality, and safety of mobile apps.
Evidence-based point-of-care information (POCI) tools can facilitate patient safety and care by helping clinicians to answer disease state and drug information questions in less time and with less ...effort. However, these tools may also be visually challenging to navigate or lack the comprehensiveness needed to sufficiently address a medical issue.
This study aimed to collect clinicians' feedback and directly observe their use of the combined POCI tool DynaMed and Micromedex with Watson, now known as DynaMedex. EBSCO partnered with IBM Watson Health, now known as Merative, to develop the combined tool as a resource for clinicians. We aimed to identify areas for refinement based on participant feedback and examine participant perceptions to inform further development.
Participants (N=43) within varying clinical roles and specialties were recruited from Brigham and Women's Hospital and Massachusetts General Hospital in Boston, Massachusetts, United States, between August 10, 2021, and December 16, 2021, to take part in usability sessions aimed at evaluating the efficiency and effectiveness of, as well as satisfaction with, the DynaMed and Micromedex with Watson tool. Usability testing methods, including think aloud and observations of user behavior, were used to identify challenges regarding the combined tool. Data collection included measurements of time on task; task ease; satisfaction with the answer; posttest feedback on likes, dislikes, and perceived reliability of the tool; and interest in recommending the tool to a colleague.
On a 7-point Likert scale, pharmacists rated ease (mean 5.98, SD 1.38) and satisfaction (mean 6.31, SD 1.34) with the combined POCI tool higher than the physicians, nurse practitioner, and physician's assistants (ease: mean 5.57, SD 1.64, and satisfaction: mean 5.82, SD 1.60). Pharmacists spent longer (mean 2 minutes, 26 seconds, SD 1 minute, 41 seconds) on average finding an answer to their question than the physicians, nurse practitioner, and physician's assistants (mean 1 minute, 40 seconds, SD 1 minute, 23 seconds).
Overall, the tool performed well, but this usability evaluation identified multiple opportunities for improvement that would help inexperienced users.
Background
Clinician perceptions of a newly implemented electronic health record play an important role in its success or failure.
Objective
To measure changes in primary care clinician attitudes ...toward an electronic health record during the first year following implementation.
Design
Longitudinal survey.
Participants
86 primary care clinicians surveyed between December 2006 and January 2008.
Measurements
Perceived impact on overall quality of care, patient safety, communication, and efficiency at 1, 3, 6, and 12 months following implementation.
Results
Response rates for months 1, 3, 6, and 12 were 92%, 95%, 90%, and 82%, respectively. The proportion of clinicians agreeing that the EHR improved the overall quality of care (63% to 86%; p < 0.001), reduced medication-related errors (72% to 81%; p = 0.03), improved follow-up of test results (62% to 87%; p < 0.001), and improved communication among clinicians (72% to 93%; p < 0.001) increased from month 1 to month 12. During the same time period, a decreasing proportion of clinicians agreed that the EHR reduced the quality of patient interactions (49% to 33%; p = 0.001), resulted in longer patient visits (68% to 51%; p = 0.001), and increased time spent on medical documentation (78% to 68%; p = 0.006). Significant improvements in perceptions related to test result follow-up were first detected at 6 months, while those related to overall quality, efficiency, and communication were first identified at 12 months.
Conclusions
Primary care clinicians report increasingly positive perceptions of a new electronic health record within 1 year of implementation across a spectrum of domains of care.
Display omitted
► We evaluated an innovative application for communicating genetic results. ► Application allows users to access knowledge updates to reported variants. ► Optimizing accessibility and ...clinical relevance of genetic data is essential. ► Incorporating user focused design can optimize value of application. ► Providers felt the application had high value for delivering clinical care.
The complexity and rapid growth of genetic data demand investment in information technology to support effective use of this information. Creating infrastructure to communicate genetic information to healthcare providers and enable them to manage that data can positively affect a patient’s care in many ways. However, genetic data are complex and present many challenges. We report on the usability of a novel application designed to assist providers in receiving and managing a patient’s genetic profile, including ongoing updated interpretations of the genetic variants in those patients. Because these interpretations are constantly evolving, managing them represents a challenge. We conducted usability tests with potential users of this application and reported findings to the application development team, many of which were addressed in subsequent versions. Clinicians were excited about the value this tool provides in pushing out variant updates to providers and overall gave the application high usability ratings, but had some difficulty interpreting elements of the interface. Many issues identified required relatively little development effort to fix suggesting that consistently incorporating this type of analysis in the development process can be highly beneficial. For genetic decision support applications, our findings suggest the importance of designing a system that can deliver the most current knowledge and highlight the significance of new genetic information for clinical care. Our results demonstrate that using a development and design process that is user focused helped optimize the value of this application for personalized medicine.
Abstract
Background
Substantial research has been performed about the impact of computerized physician order entry on medication safety in the inpatient setting; however, relatively little has been ...done in ambulatory care, where most medications are prescribed.
Objective
To outline the development and piloting process of the Ambulatory Electronic Health Record (EHR) Evaluation Tool and to report the quantitative and qualitative results from the pilot.
Methods
The Ambulatory EHR Evaluation Tool closely mirrors the inpatient version of the tool, which is administered by The Leapfrog Group. The tool was piloted with seven clinics in the United States, each using a different EHR. The tool consists of a medication safety test and a medication reconciliation module. For the medication test, clinics entered test patients and associated test orders into their EHR and recorded any decision support they received. An overall percentage score of unsafe orders detected, and order category scores were provided to clinics. For the medication reconciliation module, clinics demonstrated how their EHR electronically detected discrepancies between two medication lists.
Results
For the medication safety test, the clinics correctly alerted on 54.6% of unsafe medication orders. Clinics scored highest in the drug allergy (100%) and drug–drug interaction (89.3%) categories. Lower scoring categories included drug age (39.3%) and therapeutic duplication (39.3%). None of the clinics alerted for the drug laboratory or drug monitoring orders. In the medication reconciliation module, three (42.8%) clinics had an EHR-based medication reconciliation function; however, only one of those clinics could demonstrate it during the pilot.
Conclusion
Clinics struggled in areas of advanced decision support such as drug age, drug laboratory, and drub monitoring. Most clinics did not have an EHR-based medication reconciliation function and this process was dependent on accessing patients' medication lists. Wider use of this tool could improve outpatient medication safety and can inform vendors about areas of improvement.
PDF
