A promise of machine learning in health care is the avoidance of biases in diagnosis and treatment; a computer algorithm could objectively synthesize and interpret the data in the medical record. ...Integration of machine learning with clinical decision support tools, such as computerized alerts or diagnostic support, may offer physicians and others who provide health care targeted and timely information that can improve clinical decisions. Machine learning algorithms, however, may also be subject to biases. The biases include those related to missing data and patients not identified by algorithms, sample size and underestimation, and misclassification and measurement error. There is concern that biases and deficiencies in the data used by machine learning algorithms may contribute to socioeconomic disparities in health care. This Special Communication outlines the potential biases that may be introduced into machine learning-based clinical decision support tools that use electronic health record data and proposes potential solutions to the problems of overreliance on automation, algorithms based on biased data, and algorithms that do not provide information that is clinically meaningful. Existing health care disparities should not be amplified by thoughtless or excessive reliance on machines.
Objective
To develop new classification criteria for systemic lupus erythematosus (SLE) jointly supported by the European League Against Rheumatism (EULAR) and the American College of Rheumatology ...(ACR).
Methods
This international initiative had four phases. 1) Evaluation of antinuclear antibody (ANA) as an entry criterion through systematic review and meta‐regression of the literature and criteria generation through an international Delphi exercise, an early patient cohort, and a patient survey. 2) Criteria reduction by Delphi and nominal group technique exercises. 3) Criteria definition and weighting based on criterion performance and on results of a multi‐criteria decision analysis. 4) Refinement of weights and threshold scores in a new derivation cohort of 1,001 subjects and validation compared with previous criteria in a new validation cohort of 1,270 subjects.
Results
The 2019 EULAR/ACR classification criteria for SLE include positive ANA at least once as obligatory entry criterion; followed by additive weighted criteria grouped in 7 clinical (constitutional, hematologic, neuropsychiatric, mucocutaneous, serosal, musculoskeletal, renal) and 3 immunologic (antiphospholipid antibodies, complement proteins, SLE‐specific antibodies) domains, and weighted from 2 to 10. Patients accumulating ≥10 points are classified. In the validation cohort, the new criteria had a sensitivity of 96.1% and specificity of 93.4%, compared with 82.8% sensitivity and 93.4% specificity of the ACR 1997 and 96.7% sensitivity and 83.7% specificity of the Systemic Lupus International Collaborating Clinics 2012 criteria.
Conclusion
These new classification criteria were developed using rigorous methodology with multidisciplinary and international input, and have excellent sensitivity and specificity. Use of ANA entry criterion, hierarchically clustered, and weighted criteria reflects current thinking about SLE and provides an improved foundation for SLE research.
Video
To develop new classification criteria for systemic lupus erythematosus (SLE) jointly supported by the European League Against Rheumatism (EULAR) and the American College of Rheumatology (ACR).
This ...international initiative had four phases. (1) Evaluation of antinuclear antibody (ANA) as an entry criterion through systematic review and meta-regression of the literature and criteria generation through an international Delphi exercise, an early patient cohort and a patient survey. (2) Criteria reduction by Delphi and nominal group technique exercises. (3) Criteria definition and weighting based on criterion performance and on results of a multi-criteria decision analysis. (4) Refinement of weights and threshold scores in a new derivation cohort of 1001 subjects and validation compared with previous criteria in a new validation cohort of 1270 subjects.
The 2019 EULAR/ACR classification criteria for SLE include positive ANA at least once as obligatory entry criterion; followed by additive weighted criteria grouped in seven clinical (constitutional, haematological, neuropsychiatric, mucocutaneous, serosal, musculoskeletal, renal) and three immunological (antiphospholipid antibodies, complement proteins, SLE-specific antibodies) domains, and weighted from 2 to 10. Patients accumulating ≥10 points are classified. In the validation cohort, the new criteria had a sensitivity of 96.1% and specificity of 93.4%, compared with 82.8% sensitivity and 93.4% specificity of the ACR 1997 and 96.7% sensitivity and 83.7% specificity of the Systemic Lupus International Collaborating Clinics 2012 criteria.
These new classification criteria were developed using rigorous methodology with multidisciplinary and international input, and have excellent sensitivity and specificity. Use of ANA entry criterion, hierarchically clustered and weighted criteria reflect current thinking about SLE and provide an improved foundation for SLE research.
The SARS-CoV-2 global pandemic resulted in major disruptions to medical care. We aimed to understand changes in outpatient care delivery and use of telemedicine in U.S. rheumatology practices during ...this period. Rheumatology Informatics System Effectiveness (RISE) is a national, EHR-enabled registry that passively collects data on all patients seen by participating practices. Included practices were required to have been participating in RISE from January 2019 through August 2020 (
N
= 213). We compared total visit counts and telemedicine visits during March–August 2020 to March–August 2019 and stratified by locations in states with shelter-in-place (SIP) orders. We assessed characteristics of patients within each practice, including primary rheumatic diagnosis and disease activity scores, where available. We included 213 practices with 945,160 patients. Overall, we found visit counts decreased by 10.9% (from 1,302,455 to 1,161,051) between March and August 2020 compared to 2019; this drop was most dramatic during the month of April (− 22.3%). Telemedicine visits increased from 0% to a mean of 12.1%. Practices in SIP states had more dramatic decreases in visits, (11.5% vs. 5.3%). We found no major differences in primary diagnoses or disease activity across the two periods. We detected a meaningful decrease in rheumatology visits in March–August 2020 during the SARS-CoV-2 global pandemic compared to the year prior with a concomitant increase in the use of telemedicine. Future work should address possible adverse consequences to patient outcomes due to decreased contact with clinicians.
This article reviews the evolution of quality measurement in rheumatology, highlighting new health-information technology infrastructure and standards that are enabling unprecedented innovation in ...this field.
Spurred by landmark legislation that ties physician payment to value, the widespread use of electronic health records, and standards such as the Quality Data Model, quality measurement in rheumatology is rapidly evolving. Rather than relying on retrospective assessments of care gathered through administrative claims or manual chart abstraction, new electronic clinical quality measures (eCQMs) allow automated data capture from electronic health records. At the same time, qualified clinical data registries, like the American College of Rheumatology's Rheumatology Informatics System for Effectiveness registry, are enabling large-scale implementation of eCQMs across national electronic health record networks with real-time performance feedback to clinicians. Although successful examples of eCQM development and implementation in rheumatology and other fields exist, there also remain challenges, such as lack of health system data interoperability and problems with measure accuracy.
Quality measurement and improvement is increasingly an essential component of rheumatology practice. Advances in health information technology are likely to continue to make implementation of eCQMs easier and measurement more clinically meaningful and accurate in coming years.
There is a great and growing need to ascertain what exactly is the state of a patient, in terms of disease progression, actual care practices, pathology, adverse events, and much more, beyond the ...paucity of data available in structured medical record data. Ascertaining these harder-to-reach data elements is now critical for the accurate phenotyping of complex traits, detection of adverse outcomes, efficacy of off-label drug use, and longitudinal patient surveillance. Clinical notes often contain the most detailed and relevant digital information about individual patients, the nuances of their diseases, the treatment strategies selected by physicians, and the resulting outcomes. However, notes remain largely unused for research because they contain Protected Health Information (PHI), which is synonymous with individually identifying data. Previous clinical note de-identification approaches have been rigid and still too inaccurate to see any substantial real-world use, primarily because they have been trained with too small medical text corpora. To build a new de-identification tool, we created the largest manually annotated clinical note corpus for PHI and develop a customizable open-source de-identification software called Philter ("Protected Health Information filter"). Here we describe the design and evaluation of Philter, and show how it offers substantial real-world improvements over prior methods.
ObjectivesContraception is crucial for safely timing pregnancies in patients with SLE. This study investigated predictors of contraception documentation in patients with SLE, and the alignment of ...contraception practices with the 2020 American College of Rheumatology (ACR) guidelines, within the Rheumatology Informatics System for Effectiveness (RISE) registry.Materials and methodsFemale patients (aged 18–44 years) with SLE were identified via International Classification of Diseases (ICD)-9/ICD-10 coding within the RISE registry, which includes data from rheumatology clinics across the USA. Eligible patients were required to have ≥1 clinical visit in 2019 (prepandemic) or between 1 April 2020 and 30 March 2021 (mid-pandemic). Adjusted multilevel logistic modelling assessed patient, provider and practice characteristics for associations with contraception documentation. Contraception patterns were identified and compared with the 2020 ACR guidelines.ResultsContraception documentation rates were similar in the prepandemic and mid-pandemic groups (8.1% and 8.5%, respectively). Higher documentation rates were found in women who were younger, White, and had more visits, as well as those seen within a health system, by a female provider, and within specific regions and electronic health record (EHR) systems. Prescription of a teratogenic medication did not influence contraception documentation or type. Oestrogen-containing contraceptives were prescribed less often to women at high risk for thrombosis (26.2% with thrombotic risk vs 60.6% without, p<0.0001) and history of lupus nephritis (LN) (53.8% with history of LN vs 63.2% without, p=0.024).ConclusionsPractices participating in the RISE registry do not currently record contraception in the large majority of women with SLE, although increased documentation in some EHRs suggests that system changes may improve rates of documentation. Women at higher risk for thrombosis were less likely to receive oestrogen, suggesting that warnings against oestrogen use has impacted contraception prescription, although the limited documentation and limited contraception among women taking teratogenic medications suggest a high unmet need.
ObjectiveAccurate identification of lupus nephritis (LN) cases is essential for patient management, research and public health initiatives. However, LN diagnosis codes in electronic health records ...(EHRs) are underused, hindering efficient identification. We investigated the current performance of International Classification of Diseases (ICD) codes, 9th and 10th editions (ICD9/10), for identifying prevalent LN, and developed scoring systems to increase identification of LN that are adaptable to settings with and without LN ICD codes.MethodsTraining and test sets derived from EHR data from a large health system. An external set comprised data from the EHR of a second large health system. Adults with ICD9/10 codes for SLE were included. LN cases were ascertained through manual chart reviews conducted by rheumatologists. Two definitions of LN were used: strict (definite LN) and inclusive (definite, potential or diagnostic uncertainty). Gradient boosting models including structured EHR fields were used for predictor selection. Two logistic regression-based scoring systems were developed (‘LN-Code’ included LN ICD codes and ‘LN-No Code’ did not), calibrated and validated using standard performance metrics.ResultsA total of 4152 patients from University of California San Francisco Medical Center and 370 patients from Zuckerberg San Francisco General Hospital and Trauma Center met the eligibility criteria. Mean age was 50 years, 87% were female. LN diagnosis codes demonstrated low sensitivity (43–73%) but high specificity (92–97%). LN-Code achieved an area under the curve (AUC) of 0.93 and a sensitivity of 0.88 for identifying LN using the inclusive definition. LN-No Code reached an AUC of 0.91 and a sensitivity of 0.95 (0.97 for the strict definition). Both scoring systems had good external validity, calibration and performance across racial and ethnic groups.ConclusionsThis study quantified the underutilisation of LN diagnosis codes in EHRs and introduced two adaptable scoring systems to enhance LN identification. Further validation in diverse healthcare settings is essential to ensure their broader applicability.
In 2005, the Healthcare Effectiveness Data and Information Set (HEDIS) introduced a quality measure to assess the receipt of disease-modifying antirheumatic drugs (DMARDs) among patients with ...rheumatoid arthritis (RA).
To identify sociodemographic, community, and health plan factors associated with DMARD receipt among Medicare managed care enrollees.
We analyzed individual-level HEDIS data for 93,143 patients who were at least 65 years old with at least 2 diagnoses of RA within a measurement year (during 2005-2008). Logistic regression models with generalized estimating equations were used to determine factors associated with DMARD receipt and logistic regression was used to adjust health plan performance for case mix.
Receipt or nonreceipt of DMARD.
The mean age of patients was 74 years; 75% were women and 82% were white. Overall performance on the HEDIS measure for RA was 59% in 2005, increasing to 67% in 2008 (P for trend <.001). The largest difference in performance was based on age: patients aged 85 years and older had a 30 percentage point lower rate of DMARD receipt (95% confidence interval CI, -29 to -32 points; P < .001), compared with patients 65 to 69 years of age, even after adjusting for other factors. Lower percentage point rates were also found for patients who were men (-3 points; 95% CI, -5 to -2 points; P < .001), of black race (-4 points; 95% CI, -6 to -2 points; P < .001), with low personal income (-6 points; 95% CI, -8 to -5 points; P < .001), with the lowest zip code-based socioeconomic status (-4 points; 95% CI, -6 to 2 points; P < .001), or enrolled in for-profit health plans (-4 points; 95% CI, -7 to 0 points; P < .001); and in the Middle Atlantic region (-7 points; 95% CI, -13 to -2 points; P < .001) and South Atlantic regions (-11 points; 95% CI, -20 to -3 points; P < .001) as compared with the Pacific region. Performance varied widely by health plan, ranging from 16% to 87%.
Among Medicare managed care enrollees carrying a diagnosis of RA between 2005 and 2008, 63% received a DMARD. Receipt of DMARDs varied based on demographic factors, socioeconomic status, geographic location, and health plan.
Behçet's disease (BD), a chronic systemic vasculitis, has distinct geographical and ethnic variation. Data regarding the epidemiology of patients with BD in the U.S. are limited; therefore, we sought ...to describe BD patient characteristics and medication use in the U.S., and compared them with data from patients from endemic regions.
We conducted a cross-sectional study using data from the RISE registry (2014-2018). Patients aged ≥ 18 years with BD were included. Sociodemographic and treatment information was extracted. We compared patients from the RISE registry to data from other published studies of patients with BD from endemic areas.
One thousand three hundred twenty-three subjects with BD from the RISE registry were included. Mean age was 48.7 ± 16.3 years, female to male ratio was 3.8:1, and 66.7% were White. The most frequently used medications included glucocorticoids (67.6%) and colchicine (55.0%). Infliximab and adalimumab were the most used biologics (14.5% and 14.1%, respectively); 3.2% of patients used apremilast. The RISE registry had more women (79.3%), and patients were older compared to previously published BD studies from endemic areas. Methotrexate and TNFi were more commonly reported in RISE (21.8% and 29.4%) compared to studies from Egypt and Turkey. Colchicine, cyclosporine, and cyclophosphamide were more commonly used in cohorts from Egypt, Turkey, and Iran.
Findings from the largest BD dataset in the U.S. suggest that BD patients are predominantly female. Further research is needed to explore the reasons for the higher prevalence of BD among women in the U.S. and its possible impact on disease severity and management.
PDF
