Carl Moons and colleagues provide a checklist and background explanation for critically appraising and extracting data from systematic reviews of prognostic and diagnostic prediction modelling ...studies. Please see later in the article for the Editors' Summary.
Ten events per variable (EPV) is a widely advocated minimal criterion for sample size considerations in logistic regression analysis. Of three previous simulation studies that examined this minimal ...EPV criterion only one supports the use of a minimum of 10 EPV. In this paper, we examine the reasons for substantial differences between these extensive simulation studies.
The current study uses Monte Carlo simulations to evaluate small sample bias, coverage of confidence intervals and mean square error of logit coefficients. Logistic regression models fitted by maximum likelihood and a modified estimation procedure, known as Firth's correction, are compared.
The results show that besides EPV, the problems associated with low EPV depend on other factors such as the total sample size. It is also demonstrated that simulation results can be dominated by even a few simulated data sets for which the prediction of the outcome by the covariates is perfect ('separation'). We reveal that different approaches for identifying and handling separation leads to substantially different simulation results. We further show that Firth's correction can be used to improve the accuracy of regression coefficients and alleviate the problems associated with separation.
The current evidence supporting EPV rules for binary logistic regression is weak. Given our findings, there is an urgent need for new research to provide guidance for supporting sample size considerations for binary logistic regression analysis.
Binary logistic regression is one of the most frequently applied statistical approaches for developing clinical prediction models. Developers of such models often rely on an Events Per Variable ...criterion (EPV), notably EPV ≥10, to determine the minimal sample size required and the maximum number of candidate predictors that can be examined. We present an extensive simulation study in which we studied the influence of EPV, events fraction, number of candidate predictors, the correlations and distributions of candidate predictor variables, area under the ROC curve, and predictor effects on out-of-sample predictive performance of prediction models. The out-of-sample performance (calibration, discrimination and probability prediction error) of developed prediction models was studied before and after regression shrinkage and variable selection. The results indicate that EPV does not have a strong relation with metrics of predictive performance, and is not an appropriate criterion for (binary) prediction model development studies. We show that out-of-sample predictive performance can better be approximated by considering the number of predictors, the total sample size and the events fraction. We propose that the development of new sample size criteria for prediction models should be based on these three parameters, and provide suggestions for improving sample size determination.
Latent class models (LCMs) combine the results of multiple diagnostic tests through a statistical model to obtain estimates of disease prevalence and diagnostic test accuracy in situations where ...there is no single, accurate reference standard. We performed a systematic review of the methodology and reporting of LCMs in diagnostic accuracy studies. This review shows that the use of LCMs in such studies increased sharply in the past decade, notably in the domain of infectious diseases (overall contribution: 59%). The 64 reviewed studies used a range of differently specified parametric latent variable models, applying Bayesian and frequentist methods. The critical assumption underlying the majority of LCM applications (61%) is that the test observations must be independent within 2 classes. Because violations of this assumption can lead to biased estimates of accuracy and prevalence, performing and reporting checks of whether assumptions are met is essential. Unfortunately, our review shows that 28% of the included studies failed to report any information that enables verification of model assumptions or performance. Because of the lack of information on model fit and adequate evidence "external" to the LCMs, it is often difficult for readers to judge the validity of LCM-based inferences and conclusions reached.
Summary Background A physician is frequently unable to distinguish bacterial from viral infections. ImmunoXpert is a novel assay combining three proteins: tumour necrosis factor-related ...apoptosis-inducing ligand (TRAIL), interferon gamma induced protein-10 (IP-10), and C-reactive protein (CRP). We aimed to externally validate the diagnostic accuracy of this assay in differentiating between bacterial and viral infections and to compare this test with commonly used biomarkers. Methods In this prospective, double-blind, international, multicentre study, we recruited children aged 2–60 months with lower respiratory tract infection or clinical presentation of fever without source at four hospitals in the Netherlands and two hospitals in Israel. A panel of three experienced paediatricians adjudicated a reference standard diagnosis for all patients (ie, bacterial or viral infection) using all available clinical and laboratory information, including a 28-day follow-up assessment. The panel was masked to the assay results. We identified majority diagnosis when two of three panel members agreed on a diagnosis and unanimous diagnosis when all three panel members agreed on the diagnosis. We calculated the diagnostic performance (ie, sensitivity, specificity, positive predictive value, and negative predictive value) of the index test in differentiating between bacterial (index test positive) and viral (index test negative) infection by comparing the test classification with the reference standard outcome. Findings Between Oct 16, 2013 and March 1, 2015, we recruited 777 children, of whom 577 (mean age 21 months, 56% male) were assessed. The majority of the panel diagnosed 71 cases as bacterial infections and 435 as viral infections. In another 71 patients there was an inconclusive panel diagnosis. The assay distinguished bacterial from viral infections with a sensitivity of 86·7% (95% CI 75·8–93·1), a specificity of 91·1% (87·9–93·6), a positive predictive value of 60·5% (49·9–70·1), and a negative predictive value of 97·8% (95·6–98·9). In the more clear cases with unanimous panel diagnosis (n=354), sensitivity was 87·8% (74·5–94·7), specificity 93·0% (89·6–95·3), positive predictive value 62·1% (49·2–73·4), and negative predictive value 98·3% (96·1–99·3). Interpretation This external validation study shows the diagnostic value of a three-host protein-based assay to differentiate between bacterial and viral infections in children with lower respiratory tract infection or fever without source. This diagnostic based on CRP, TRAIL, and IP-10 has the potential to reduce antibiotic misuse in young children. Funding MeMed Diagnostics.
Meta-analyses are increasingly used for synthesis of evidence from biomedical research, and often include an assessment of publication bias based on visual or analytical detection of asymmetry in ...funnel plots. We studied the influence of different normalisation approaches, sample size and intervention effects on funnel plot asymmetry, using empirical datasets and illustrative simulations. We found that funnel plots of the Standardized Mean Difference (SMD) plotted against the standard error (SE) are susceptible to distortion, leading to overestimation of the existence and extent of publication bias. Distortion was more severe when the primary studies had a small sample size and when an intervention effect was present. We show that using the Normalised Mean Difference measure as effect size (when possible), or plotting the SMD against a sample size-based precision estimate, are more reliable alternatives. We conclude that funnel plots using the SMD in combination with the SE are unsuitable for publication bias assessments and can lead to false-positive results.
Subcutaneous or Transvenous Defibrillator Therapy Knops, Reinoud E; Olde Nordkamp, Louise R A; Delnoy, Peter-Paul H M ...
New England journal of medicine/The New England journal of medicine,
08/2020, Volume:
383, Issue:
6
Journal Article
Peer reviewed
Open access
The subcutaneous implantable cardioverter-defibrillator (ICD) was designed to avoid complications related to the transvenous ICD lead by using an entirely extrathoracic placement. Evidence comparing ...these systems has been based primarily on observational studies.
We conducted a noninferiority trial in which patients with an indication for an ICD but no indication for pacing were assigned to receive a subcutaneous ICD or transvenous ICD. The primary end point was the composite of device-related complications and inappropriate shocks; the noninferiority margin for the upper boundary of the 95% confidence interval for the hazard ratio (subcutaneous ICD vs. transvenous ICD) was 1.45. A superiority analysis was prespecified if noninferiority was established. Secondary end points included death and appropriate shocks.
A total of 849 patients (426 in the subcutaneous ICD group and 423 in the transvenous ICD group) were included in the analyses. At a median follow-up of 49.1 months, a primary end-point event occurred in 68 patients in the subcutaneous ICD group and in 68 patients in the transvenous ICD group (48-month Kaplan-Meier estimated cumulative incidence, 15.1% and 15.7%, respectively; hazard ratio, 0.99; 95% confidence interval CI, 0.71 to 1.39; P = 0.01 for noninferiority; P = 0.95 for superiority). Device-related complications occurred in 31 patients in the subcutaneous ICD group and in 44 in the transvenous ICD group (hazard ratio, 0.69; 95% CI, 0.44 to 1.09); inappropriate shocks occurred in 41 and 29 patients, respectively (hazard ratio, 1.43; 95% CI, 0.89 to 2.30). Death occurred in 83 patients in the subcutaneous ICD group and in 68 in the transvenous ICD group (hazard ratio, 1.23; 95% CI, 0.89 to 1.70); appropriate shocks occurred in 83 and 57 patients, respectively (hazard ratio, 1.52; 95% CI, 1.08 to 2.12).
In patients with an indication for an ICD but no indication for pacing, the subcutaneous ICD was noninferior to the transvenous ICD with respect to device-related complications and inappropriate shocks. (Funded by Boston Scientific; PRAETORIAN ClinicalTrials.gov number, NCT01296022.).
Fibrinogen concentrate might partly restore coagulation defects and reduce intraoperative bleeding.
To determine whether fibrinogen concentrate infusion dosed to achieve a plasma fibrinogen level of ...2.5 g/L in high-risk cardiac surgery patients with intraoperative bleeding reduces intraoperative blood loss.
A randomized, placebo-controlled, double-blind clinical trial conducted in Isala Zwolle, the Netherlands (February 2011-January 2015), involving patients undergoing elective, high-risk cardiac surgery (ie, combined coronary artery bypass graft CABG surgery and valve repair or replacement surgery, the replacement of multiple valves, aortic root reconstruction, or reconstruction of the ascending aorta or aortic arch) with intraoperative bleeding (blood volume between 60 and 250 mL suctioned from the thoracic cavity in a period of 5 minutes) were randomized to receive either fibrinogen concentrate or placebo.
Intravenous, single-dose administration of fibrinogen concentrate (n = 60) or placebo (n = 60), targeted to achieve a postinfusion plasma fibrinogen level of 2.5 g/L.
The primary outcome was blood loss in milliliters between intervention (ie, after removal of cardiopulmonary bypass) and closure of chest. Safety variables (within 30 days) included: in-hospital mortality, myocardial infarction, cerebrovascular accident or transient ischemic attack, renal insufficiency or failure, venous thromboembolism, pulmonary embolism, and operative complications.
Among 120 patients (mean age; 71 SD, 10 years, 37 women 31%) included in the study, combined CABG and valve repair or replacement surgery comprised 72% of procedures and had a mean (SD) cardiopulmonary bypass time of 200 minutes (83) minutes. For the primary outcome, median blood loss in the fibrinogen group was 50 mL (interquartile range IQR, 29-100 mL) compared with 70 mL (IQR, 33-145 mL) in the control group (P = .19), the absolute difference 20 mL (95% CI, -13 to 35 mL). There were 6 cases of stroke or transient ischemic attack (4 in the fibrinogen group); 4 myocardial infarctions (3 in the fibrinogen group); 2 deaths (both in the fibrinogen group); 5 cases with renal insufficiency or failure (3 in the fibrinogen group); and 9 cases with reoperative thoracotomy (4 in the fibrinogen group).
Among patients with intraoperative bleeding during high-risk cardiac surgery, administration of fibrinogen concentrate, compared with placebo, resulted in no significant difference in the amount of intraoperative blood loss.
clinicaltrials.gov Identifier: NCT01124981 and EudraCT No: 2009-018086-12.
Electromagnetic navigation bronchoscopy (ENB) has been widely adopted as a guidance technique for biopsy of peripheral lung nodules. However, ENB is limited by the lack of real-time confirmation of ...the biopsy devices. Intraprocedural cone-beam computed tomography (CBCT) imaging can be utilized to assess or confirm the location of biopsy devices. The aim of this study is to determine the safety and diagnostic yield (DY) of image fusion of intraprocedural CBCT data with live fluoroscopy (augmented fluoroscopy) during ENB-guided biopsy of peripheral lung nodules.
Data from 75 consecutive patients who underwent biopsy with ENB was collected retrospectively. Patients underwent CBCT imaging while temporarily suspending mechanical ventilation. CBCT data were acquired and 3-dimensional segmentation of nodules was performed using commercially available software (OncoSuite). During ENB, the segmented lesions were projected and fused with live fluoroscopy enabling real-time 3-dimensional guidance.
A total of 93 lesions with a median size of 16.0 mm were biopsied in 75 consecutive patients. The overall DY by lesion was 83.7% (95% confidence interval, 74.8%-89.9%). Multivariate regression analysis showed no independent correlation between lesion size, lesion location, lesion visibility under standard fluoroscopy, and the presence of a bronchus sign with DY. Pneumothorax occurred in 3 patients (4%).
Intraprocedural CBCT imaging with augmented fluoroscopy is feasible and effective and is associated with high DY during ENB-guided biopsies.
In practice, the diagnostic workup usually starts with a patient with particular symptoms or signs, who is suspected of having a particular target disease. In a sequence of steps, an array of ...diagnostic information is commonly documented. The diagnostic information conveyed by different results from patient history, physical examination, and subsequent testing is to varying extents overlapping and thus mutually dependent. This implies that the diagnostic potential of a test or biomarker is conditional on the information obtained from previous tests. A key question about the accuracy of a diagnostic test/biomarker is whether that test improves the diagnostic workup beyond already available diagnostic test results. This second report in a series of 4 gives an overview of several methods to quantify the added value of a new diagnostic test or biomarker, including the area under the ROC curve, net reclassification improvement, integrated discrimination improvement, predictiveness curve, and decision curve analysis. Each of these methods is illustrated with the use of empirical data. We reiterate that reporting on the relative increase in discrimination and disease classification is relevant to obtain insight into the incremental value of a diagnostic test or biomarker. We also recommend the use of decision-analytic measures to express the accuracy of an entire diagnostic workup in an informative way.
PDF
