The use of core outcome sets (COS) ensures that researchers measure and report those outcomes that are most likely to be relevant to users of their research. Several hundred COS projects have been ...systematically identified to date, but there has been no formal quality assessment of these studies. The Core Outcome Set-STAndards for Development (COS-STAD) project aimed to identify minimum standards for the design of a COS study agreed upon by an international group, while other specific guidance exists for the final reporting of COS development studies (Core Outcome Set-STAndards for Reporting COS-STAR).
An international group of experienced COS developers, methodologists, journal editors, potential users of COS (clinical trialists, systematic reviewers, and clinical guideline developers), and patient representatives produced the COS-STAD recommendations to help improve the quality of COS development and support the assessment of whether a COS had been developed using a reasonable approach. An open survey of experts generated an initial list of items, which was refined by a 2-round Delphi survey involving nearly 250 participants representing key stakeholder groups. Participants assigned importance ratings for each item using a 1-9 scale. Consensus that an item should be included in the set of minimum standards was defined as at least 70% of the voting participants from each stakeholder group providing a score between 7 and 9. The Delphi survey was followed by a consensus discussion with the study management group representing multiple stakeholder groups. COS-STAD contains 11 minimum standards that are the minimum design recommendations for all COS development projects. The recommendations focus on 3 key domains: the scope, the stakeholders, and the consensus process.
The COS-STAD project has established 11 minimum standards to be followed by COS developers when planning their projects and by users when deciding whether a COS has been developed using reasonable methods.
A prognostic model should not enter clinical practice unless it has been demonstrated that it performs a useful role. External validation denotes evaluation of model performance in a sample ...independent of that used to develop the model. Unlike for logistic regression models, external validation of Cox models is sparsely treated in the literature. Successful validation of a model means achieving satisfactory discrimination and calibration (prediction accuracy) in the validation sample. Validating Cox models is not straightforward because event probabilities are estimated relative to an unspecified baseline function.
We describe statistical approaches to external validation of a published Cox model according to the level of published information, specifically (1) the prognostic index only, (2) the prognostic index together with Kaplan-Meier curves for risk groups, and (3) the first two plus the baseline survival curve (the estimated survival function at the mean prognostic index across the sample). The most challenging task, requiring level 3 information, is assessing calibration, for which we suggest a method of approximating the baseline survival function.
We apply the methods to two comparable datasets in primary breast cancer, treating one as derivation and the other as validation sample. Results are presented for discrimination and calibration. We demonstrate plots of survival probabilities that can assist model evaluation.
Our validation methods are applicable to a wide range of prognostic studies and provide researchers with a toolkit for external validation of a published Cox model.
Alternatively, one can directly estimate the PAF original formula "(O-E)/O" using results from a multivariable logistic regression model. 7 As an example of the latter approach, the authors of a ...recent BMJ paper 8 calculated the population attributable fraction (PAF) of concurrent benzodiazepine/opioid use for the risk of opioid overdose in a retrospective analysis of claim data. 8 Valid 95% confidence intervals for PAF should take into account the uncertainty in both the observed and expected number of cases. 7 The PAF formula with adjusted RR is easily generalised to exposures with more than two levels. 9 In a cohort study the PAF for the effect of maternal overweight and obesity on infant mortality in relation to normal weight was estimated as 11%. ...for preventive exposures one can reverse the coding: RR is now the adjusted risk ratio for no exposure and pc is the prevalence of no exposure among cases.
The CONSORT Statement for randomised controlled clinical trials was one of the first guidelines developed in response to this need 24,25. Since publication, an increasing number of leading journals ...have supported CONSORT as part of their instructions to authors 26,27. ...convincing evidence is emerging that CONSORT improves the quality and transparency of reports of clinical trials 28,29.
Systematic reviews (SRs) can help decision makers interpret the deluge of published biomedical literature. However, a SR may be of limited use if the methods used to conduct the SR are flawed, and ...reporting of the SR is incomplete. To our knowledge, since 2004 there has been no cross-sectional study of the prevalence, focus, and completeness of reporting of SRs across different specialties. Therefore, the aim of our study was to investigate the epidemiological and reporting characteristics of a more recent cross-section of SRs.
We searched MEDLINE to identify potentially eligible SRs indexed during the month of February 2014. Citations were screened using prespecified eligibility criteria. Epidemiological and reporting characteristics of a random sample of 300 SRs were extracted by one reviewer, with a 10% sample extracted in duplicate. We compared characteristics of Cochrane versus non-Cochrane reviews, and the 2014 sample of SRs versus a 2004 sample of SRs. We identified 682 SRs, suggesting that more than 8,000 SRs are being indexed in MEDLINE annually, corresponding to a 3-fold increase over the last decade. The majority of SRs addressed a therapeutic question and were conducted by authors based in China, the UK, or the US; they included a median of 15 studies involving 2,072 participants. Meta-analysis was performed in 63% of SRs, mostly using standard pairwise methods. Study risk of bias/quality assessment was performed in 70% of SRs but was rarely incorporated into the analysis (16%). Few SRs (7%) searched sources of unpublished data, and the risk of publication bias was considered in less than half of SRs. Reporting quality was highly variable; at least a third of SRs did not report use of a SR protocol, eligibility criteria relating to publication status, years of coverage of the search, a full Boolean search logic for at least one database, methods for data extraction, methods for study risk of bias assessment, a primary outcome, an abstract conclusion that incorporated study limitations, or the funding source of the SR. Cochrane SRs, which accounted for 15% of the sample, had more complete reporting than all other types of SRs. Reporting has generally improved since 2004, but remains suboptimal for many characteristics.
An increasing number of SRs are being published, and many are poorly conducted and reported. Strategies are needed to help reduce this avoidable waste in research.
Evidence shows the quality of reporting of randomised controlled trials is not optimal. The lack of transparent reporting impedes readers from judging the reliability and validity of trial findings, ...prevents researchers from extracting information for systematic reviews, and results in research waste. The Consolidated Standards of Reporting Trials (CONSORT) statement was developed to improve the reporting of randomised controlled trials. The primary focus of the statement was on parallel group trials with two treatment groups. Crossover trials are a particular type of trial for chronic conditions in which participants are randomised to a sequence of interventions. They are a useful and efficient design because participants act as their own control. However, the reporting of crossover trials has been variable and incomplete, which hinders their use in clinical decision making and by future researchers. We present the CONSORT extension to randomised crossover trials, which aims to facilitate better reporting of crossover trials. The CONSORT 2010 checklist is revised for crossover designs, and introduces a modified flowchart and baseline table to enhance transparency. Examples of good reporting and evidence based rationale for CONSORT crossover checklist items are provided.
The REMARK (Reporting Recommendations for Tumor Marker Prognostic Studies) guideline includes a checklist which aims to improve the reporting of these types of studies. Here, we expand on the REMARK ...checklist to enhance its use and effectiveness through better understanding of the intent of each item and why the information is important to report. Each checklist item of the REMARK guideline is explained in detail and accompanied by published examples of good reporting. The paper provides a comprehensive overview to educate on good reporting and provide a valuable reference of issues to consider when designing, conducting, and analyzing tumor marker studies and prognostic studies in medicine in general.
To our knowledge, no publication providing overarching guidance on the conduct of systematic reviews of observational studies of etiology exists.
Conducting Systematic Reviews and Meta-Analyses of ...Observational Studies of Etiology (COSMOS-E) provides guidance on all steps in systematic reviews of observational studies of etiology, from shaping the research question, defining exposure and outcomes, to assessing the risk of bias and statistical analysis. The writing group included researchers experienced in meta-analyses and observational studies of etiology. Standard peer-review was performed. While the structure of systematic reviews of observational studies on etiology may be similar to that for systematic reviews of randomised controlled trials, there are specific tasks within each component that differ. Examples include assessment for confounding, selection bias, and information bias. In systematic reviews of observational studies of etiology, combining studies in meta-analysis may lead to more precise estimates, but such greater precision does not automatically remedy potential bias. Thorough exploration of sources of heterogeneity is key when assessing the validity of estimates and causality.
As many reviews of observational studies on etiology are being performed, this document may provide researchers with guidance on how to conduct and analyse such reviews.
PDF
