Background
Randomised trials use the play of chance to assign participants to comparison groups. The unpredictability of the process, if not subverted, should prevent systematic differences between ...comparison groups (selection bias). Differences due to chance will still occur and these are minimised by randomising a sufficiently large number of people.
Objectives
To assess the effects of randomisation and concealment of allocation on the results of healthcare studies.
Search methods
We searched the Cochrane Methodology Register, MEDLINE, SciSearch and reference lists up to September 2009. In addition, we screened articles citing included studies (ISI Science Citation Index) and papers related to included studies (PubMed).
Selection criteria
Eligible study designs were cohorts of studies, systematic reviews or meta‐analyses of healthcare interventions that compared random allocation versus non‐random allocation or adequate versus inadequate/unclear concealment of allocation in randomised trials. Outcomes of interest were the magnitude and direction of estimates of effect and imbalances in prognostic factors.
Data collection and analysis
We retrieved and assessed studies that appeared to meet the inclusion criteria independently. At least two review authors independently appraised methodological quality and extracted information. We prepared tabular summaries of the results for each comparison and assessed the results across studies qualitatively to identify common trends or discrepancies.
Main results
A total of 18 studies (systematic reviews or meta‐analyses) met our inclusion criteria. Ten compared random allocation versus non‐random allocation and nine compared adequate versus inadequate or unclear concealment of allocation within controlled trials. All studies were at high risk of bias.
For the comparison of randomised versus non‐randomised studies, four comparisons yielded inconclusive results (differed between outcomes or different modes of analysis); three comparisons showed similar results for random and non‐random allocation; two comparisons had larger estimates of effect in non‐randomised studies than in randomised trials; and two comparisons had larger estimates of effect in randomised than in non‐randomised studies.
Five studies found larger estimates of effect in trials with inadequate concealment of allocation than in trials with adequate concealment. The four other studies did not find statistically significant differences.
Authors' conclusions
The results of randomised and non‐randomised studies sometimes differed. In some instances non‐randomised studies yielded larger estimates of effect and in other instances randomised trials yielded larger estimates of effect. The results of controlled trials with adequate and inadequate/unclear concealment of allocation sometimes differed. When differences occurred, most often trials with inadequate or unclear allocation concealment yielded larger estimates of effects relative to controlled trials with adequate allocation concealment. However, it is not generally possible to predict the magnitude, or even the direction, of possible selection biases and consequent distortions of treatment effects from studies with non‐random allocation or controlled trials with inadequate or unclear allocation concealment.
Background
s of presentations at scientific meetings are usually available only in conference proceedings. If subsequent full publication of results reported in these s is based on the magnitude or ...direction of the results, publication bias may result. Publication bias creates problems for those conducting systematic reviews or relying on the published literature for evidence about health and social care.
Objectives
To systematically review reports of studies that have examined the proportion of meeting s and other summaries that are subsequently published in full, the time between meeting presentation and full publication, and factors associated with full publication.
Search methods
We searched MEDLINE, Embase, the Cochrane Library, Science Citation Index, reference lists, and author files. The most recent search was done in February 2016 for this substantial update to our earlier Cochrane Methodology Review (published in 2007).
Selection criteria
We included reports of methodology research that examined the proportion of biomedical results initially presented as s or in summary form that were subsequently published. Searches for full publications had to be at least two years after meeting presentation.
Data collection and analysis
Two review authors extracted data and assessed risk of bias. We calculated the proportion of s published in full using a random‐effects model. Dichotomous variables were analyzed using risk ratio (RR), with multivariable models taking into account various characteristics of the reports. We assessed time to publication using Kaplan‐Meier survival analyses.
Main results
Combining data from 425 reports (307,028 s) resulted in an overall full publication proportion of 37.3% (95% confidence interval (CI), 35.3% to 39.3%) with varying lengths of follow‐up. This is significantly lower than that found in our 2007 review (44.5%. 95% CI, 43.9% to 45.1%). Using a survival analyses to estimate the proportion of s that would be published in full by 10 years produced proportions of 46.4% for all studies; 68.7% for randomized and controlled trials and 44.9% for other studies. Three hundred and fifty‐three reports were at high risk of bias on one or more items, but only 32 reports were considered at high risk of bias overall.
Forty‐five reports (15,783 s) with 'positive' results (defined as any 'significant' result) showed an association with full publication (RR = 1.31; 95% CI 1.23 to 1.40), as did 'positive' results defined as a result favoring the experimental treatment (RR =1.17; 95% CI 1.07 to 1.28) in 34 reports (8794 s). Results emanating from randomized or controlled trials showed the same pattern for both definitions (RR = 1.21; 95% CI 1.10 to 1.32 (15 reports and 2616 s) and RR = 1.17; 95% CI, 1.04 to 1.32 (13 reports and 2307 s), respectively.
Other factors associated with full publication include oral presentation (RR = 1.46; 95% CI 1.40 to 1.52; studied in 143 reports with 115,910 s); acceptance for meeting presentation (RR = 1.65; 95% CI 1.48 to 1.85; 22 reports with 22,319 s); randomized trial design (RR = 1.51; 95% CI 1.36 to 1.67; 47 reports with 28,928 s); and basic research (RR = 0.78; 95% CI 0.74 to 0.82; 92 reports with 97,372 s). s originating at an academic setting were associated with full publication (RR = 1.60; 95% CI 1.34 to 1.92; 34 reports with 16,913 s), as were those considered to be of higher quality (RR = 1.46; 95% CI 1.23 to 1.73; 12 reports with 3364 s), or having high impact (RR = 1.60; 95% CI 1.41 to 1.82; 11 reports with 6982 s). Sensitivity analyses excluding reports that were s themselves or classified as having a high risk of bias did not change these findings in any important way.
In considering the reports of the methodology research that we included in this review, we found that reports published in English or from a native English‐speaking country found significantly higher proportions of studies published in full, but that there was no association with year of report publication. The findings correspond to a proportion of s published in full of 31.9% for all reports, 40.5% for reports in English, 42.9% for reports from native English‐speaking countries, and 52.2% for both these covariates combined.
Authors' conclusions
More than half of results from s, and almost a third of randomized trial results initially presented as s fail to be published in full and this problem does not appear to be decreasing over time. Publication bias is present in that 'positive' results were more frequently published than 'not positive' results. Reports of methodology research written in English showed that a higher proportion of s had been published in full, as did those from native English‐speaking countries, suggesting that studies from non‐native English‐speaking countries may be underrepresented in the scientific literature. After the considerable work involved in adding in the more than 300 additional studies found by the February 2016 searches, we chose not to update the search again because additional searches are unlikely to change these overall conclusions in any important way.
Abstract Background Lack of standardization of outcome measures limits the usefulness of clinical trial evidence to inform health care decisions. This can be addressed by agreeing on a minimum core ...set of outcome measures per health condition, containing measures relevant to patients and decision makers. Since 1992, the Outcome Measures in Rheumatology (OMERACT) consensus initiative has successfully developed core sets for many rheumatologic conditions, actively involving patients since 2002. Its expanding scope required an explicit formulation of its underlying conceptual framework and process. Methods Literature searches and iterative consensus process (surveys and group meetings) of stakeholders including patients, health professionals, and methodologists within and outside rheumatology. Results To comprehensively sample patient-centered and intervention-specific outcomes, a framework emerged that comprises three core “Areas,” namely Death, Life Impact, and Pathophysiological Manifestations; and one strongly recommended Resource Use. Through literature review and consensus process, core set development for any specific health condition starts by identifying at least one core “Domain” within each of the Areas to formulate the “Core Domain Set.” Next, at least one applicable measurement instrument for each core Domain is identified to formulate a “Core Outcome Measurement Set.” Each instrument must prove to be truthful (valid), discriminative, and feasible. In 2012, 96% of the voting participants ( n = 125) at the OMERACT 11 consensus conference endorsed this model and process. Conclusion The OMERACT Filter 2.0 explicitly describes a comprehensive conceptual framework and a recommended process to develop core outcome measurement sets for rheumatology likely to be useful as a template in other areas of health care.
Background
Pulmonary rehabilitation is a proven, effective intervention for people with chronic respiratory diseases including chronic obstructive pulmonary disease (COPD), interstitial lung disease ...(ILD) and bronchiectasis. However, relatively few people attend or complete a program, due to factors including a lack of programs, issues associated with travel and transport, and other health issues. Traditionally, pulmonary rehabilitation is delivered in‐person on an outpatient basis at a hospital or other healthcare facility (referred to as centre‐based pulmonary rehabilitation). Newer, alternative modes of pulmonary rehabilitation delivery include home‐based models and the use of telehealth.
Telerehabilitation is the delivery of rehabilitation services at a distance, using information and communication technology. To date, there has not been a comprehensive assessment of the clinical efficacy or safety of telerehabilitation, or its ability to improve uptake and access to rehabilitation services, for people with chronic respiratory disease.
Objectives
To determine the effectiveness and safety of telerehabilitation for people with chronic respiratory disease.
Search methods
We searched the Cochrane Airways Trials Register, and the Cochrane Central Register of Controlled Trials; six databases including MEDLINE and Embase; and three trials registries, up to 30 November 2020. We checked reference lists of all included studies for additional references, and handsearched relevant respiratory journals and meeting s.
Selection criteria
All randomised controlled trials and controlled clinical trials of telerehabilitation for the delivery of pulmonary rehabilitation were eligible for inclusion. The telerehabilitation intervention was required to include exercise training, with at least 50% of the rehabilitation intervention being delivered by telerehabilitation.
Data collection and analysis
We used standard methods recommended by Cochrane. We assessed the risk of bias for all studies, and used the ROBINS‐I tool to assess bias in non‐randomised controlled clinical trials. We assessed the certainty of evidence with GRADE. Comparisons were telerehabilitation compared to traditional in‐person (centre‐based) pulmonary rehabilitation, and telerehabilitation compared to no rehabilitation. We analysed studies of telerehabilitation for maintenance rehabilitation separately from trials of telerehabilitation for initial primary pulmonary rehabilitation.
Main results
We included a total of 15 studies (32 reports) with 1904 participants, using five different models of telerehabilitation. Almost all (99%) participants had chronic obstructive pulmonary disease (COPD). Three studies were controlled clinical trials. For primary pulmonary rehabilitation, there was probably little or no difference between telerehabilitation and in‐person pulmonary rehabilitation for exercise capacity measured as 6‐Minute Walking Distance (6MWD) (mean difference (MD) 0.06 metres (m), 95% confidence interval (CI) ‐10.82 m to 10.94 m; 556 participants; four studies; moderate‐certainty evidence). There may also be little or no difference for quality of life measured with the St George's Respiratory Questionnaire (SGRQ) total score (MD ‐1.26, 95% CI ‐3.97 to 1.45; 274 participants; two studies; low‐certainty evidence), or for breathlessness on the Chronic Respiratory Questionnaire (CRQ) dyspnoea domain score (MD 0.13, 95% CI ‐0.13 to 0.40; 426 participants; three studies; low‐certainty evidence). Participants were more likely to complete a program of telerehabilitation, with a 93% completion rate (95% CI 90% to 96%), compared to a 70% completion rate for in‐person rehabilitation. When compared to no rehabilitation control, trials of primary telerehabilitation may increase exercise capacity on 6MWD (MD 22.17 m, 95% CI ‐38.89 m to 83.23 m; 94 participants; two studies; low‐certainty evidence) and may also increase 6MWD when delivered as maintenance rehabilitation (MD 78.1 m, 95% CI 49.6 m to 106.6 m; 209 participants; two studies; low‐certainty evidence). No adverse effects of telerehabilitation were noted over and above any reported for in‐person rehabilitation or no rehabilitation.
Authors' conclusions
This review suggests that primary pulmonary rehabilitation, or maintenance rehabilitation, delivered via telerehabilitation for people with chronic respiratory disease achieves outcomes similar to those of traditional centre‐based pulmonary rehabilitation, with no safety issues identified. However, the certainty of the evidence provided by this review is limited by the small number of studies, of varying telerehabilitation models, with relatively few participants. Future research should consider the clinical effect of telerehabilitation for individuals with chronic respiratory diseases other than COPD, the duration of benefit of telerehabilitation beyond the period of the intervention, and the economic cost of telerehabilitation.
Background
Improper practices and unprofessional conduct in clinical research have been shown to waste a significant portion of healthcare funds and harm public health.
Objectives
Our objective was ...to evaluate the effectiveness of educational or policy interventions in research integrity or responsible conduct of research on the behaviour and attitudes of researchers in health and other research areas.
Search methods
We searched the CENTRAL, MEDLINE, LILACS and CINAHL health research bibliographical databases, as well as the Academic Search Complete, AGRICOLA, GeoRef, PsycINFO, ERIC, SCOPUS and Web of Science databases. We performed the last search on 15 April 2015 and the search was limited to articles published between 1990 and 2014, inclusive. We also searched conference proceedings and s from research integrity conferences and specialized websites. We handsearched 14 journals that regularly publish research integrity research.
Selection criteria
We included studies that measured the effects of one or more interventions, i.e. any direct or indirect procedure that may have an impact on research integrity and responsible conduct of research in its broadest sense, where participants were any stakeholders in research and publication processes, from students to policy makers. We included randomized and non‐randomized controlled trials, such as controlled before‐and‐after studies, with comparisons of outcomes in the intervention versus non‐intervention group or before versus after the intervention. Studies without a control group were not included in the review.
Data collection and analysis
We used the standard methodological procedures expected by Cochrane. To assess the risk of bias in non‐randomized studies, we used a modified Cochrane tool, in which we used four out of six original domains (blinding, incomplete outcome data, selective outcome reporting, other sources of bias) and two additional domains (comparability of groups and confounding factors). We categorized our primary outcome into the following levels: 1) organizational change attributable to intervention, 2) behavioural change, 3) acquisition of knowledge/skills and 4) modification of attitudes/perceptions. The secondary outcome was participants' reaction to the intervention.
Main results
Thirty‐one studies involving 9571 participants, described in 33 articles, met the inclusion criteria. All were published in English. Fifteen studies were randomized controlled trials, nine were controlled before‐and‐after studies, four were non‐equivalent controlled studies with a historical control, one was a non‐equivalent controlled study with a post‐test only and two were non‐equivalent controlled studies with pre‐ and post‐test findings for the intervention group and post‐test for the control group. Twenty‐one studies assessed the effects of interventions related to plagiarism and 10 studies assessed interventions in research integrity/ethics. Participants included undergraduates, postgraduates and academics from a range of research disciplines and countries, and the studies assessed different types of outcomes.
We judged most of the included randomized controlled trials to have a high risk of bias in at least one of the assessed domains, and in the case of non‐randomized trials there were no attempts to alleviate the potential biases inherent in the non‐randomized designs.
We identified a range of interventions aimed at reducing research misconduct. Most interventions involved some kind of training, but methods and content varied greatly and included face‐to‐face and online lectures, interactive online modules, discussion groups, homework and practical exercises. Most studies did not use standardized or validated outcome measures and it was impossible to synthesize findings from studies with such diverse interventions, outcomes and participants. Overall, there is very low quality evidence that various methods of training in research integrity had some effects on participants' attitudes to ethical issues but minimal (or short‐lived) effects on their knowledge. Training about plagiarism and paraphrasing had varying effects on participants' attitudes towards plagiarism and their confidence in avoiding it, but training that included practical exercises appeared to be more effective. Training on plagiarism had inconsistent effects on participants' knowledge about and ability to recognize plagiarism. Active training, particularly if it involved practical exercises or use of text‐matching software, generally decreased the occurrence of plagiarism although results were not consistent. The design of a journal's author contribution form affected the truthfulness of information supplied about individuals' contributions and the proportion of listed contributors who met authorship criteria. We identified no studies testing interventions for outcomes at the organizational level. The numbers of events and the magnitude of intervention effects were generally small, so the evidence is likely to be imprecise. No adverse effects were reported.
Authors' conclusions
The evidence base relating to interventions to improve research integrity is incomplete and the studies that have been done are heterogeneous, inappropriate for meta‐analyses and their applicability to other settings and population is uncertain. Many studies had a high risk of bias because of the choice of study design and interventions were often inadequately reported. Even when randomized designs were used, findings were difficult to generalize. Due to the very low quality of evidence, the effects of training in responsible conduct of research on reducing research misconduct are uncertain. Low quality evidence indicates that training about plagiarism, especially if it involves practical exercises and use of text‐matching software, may reduce the occurrence of plagiarism.
Abstract
The Academic Research Consortium (ARC)-2 initiative revisited the clinical and angiographic end point definitions in coronary device trials, proposed in 2007, to make them more suitable for ...use in clinical trials that include increasingly complex lesion and patient populations and incorporate novel devices such as bioresorbable vascular scaffolds. In addition, recommendations for the incorporation of patient-related outcomes in clinical trials are proposed. Academic Research Consortium-2 is a collaborative effort between academic research organizations in the United States and Europe, device manufacturers, and European, US, and Asian regulatory bodies. Several in-person meetings were held to discuss the changes that have occurred in the device landscape and in clinical trials and regulatory pathways in the last decade. The consensus-based end point definitions in this document are endorsed by the stakeholders of this document and strongly advocated for clinical trial purposes. This Academic Research Consortium-2 document provides further standardization of end point definitions for coronary device trials, incorporating advances in technology and knowledge. Their use will aid interpretation of trial outcomes and comparison among studies, thus facilitating the evaluation of the safety and effectiveness of these devices.
Randomised trials use the play of chance to assign participants to comparison groups. The unpredictability of the process, if not subverted, should prevent systematic differences between comparison ...groups (selection bias), provided that a sufficient number of people are randomised.
To assess the effects of randomisation and concealment of allocation on the results of healthcare trials.
We searched the Cochrane Methodology Register, MEDLINE, SciSearch, reference lists up to August 2000 and used personal communication.
Cohorts of trials, systematic reviews or meta-analyses of healthcare interventions that compared outcomes or prognostic factors for one of the following comparisons: randomised versus non-randomised trials, randomised trials with adequately versus inadequately concealed allocation, or high versus low quality trials where selection bias could not be separated from other sources of bias.
One of us went through all of the citations in the Cochrane Methodology Register and accumulated reference lists. Studies that appeared to meet the inclusion criteria were retrieved and assessed independently by two of the reviewers. The methodological quality of included studies was appraised and information extracted by one of us and checked by a second. Tabular summaries of the results were prepared for each comparison and the results across studies were assessed qualitatively to identify common trends or discrepancies.
We identified 32 studies including over 3000 trials. Twenty-two studies compared randomised versus non-randomised trials, three compared adequately versus inadequately concealed allocation, and nine compared high versus low quality trials (some studies included more than one comparison). Five studies were of high methodological quality. In 15 of the 22 studies that compared randomised and non-randomised trials of the same intervention, important differences were found in the estimates of effect. Some of these differences were due to a poorer prognosis in the control groups in the non-randomised trials. The results of the other seven studies that compared randomised and non-randomised trials across different interventions are less clear. Comparisons of adequately and inadequately concealed allocation in randomised trials of the same intervention provided high quality evidence that concealment can be crucial in achieving similar treatment groups and, therefore, unbiased estimates of treatment effects. Studies with inadequate concealment tended to overestimate treatment effects. Comparisons of high and low quality trials of the same intervention have found important differences in estimates of effect, but it is not possible to determine the extent to which these differences can be attributed to randomisation or concealment of allocation. Omitting comparisons between randomised trials and non-randomised trials using historical controls did not substantially alter the results or conclusions of our review.
On average, non-randomised trials and randomised trials with inadequate concealment of allocation tend to result in larger estimates of effect than randomised trials with adequately concealed allocation. However, it is not generally possible to predict the magnitude, or even the direction, of possible selection biases and consequent distortions of treatment effects.
The goal of this consensus conference, sponsored by the American Association of Plastic Surgeons, was to perform a systematic review and meta-analysis of controlled trials to examine both the ...benefits and risks of surgical treatment and surgical prevention of upper and lower extremity lymphedema.
The panel met in Boston for a 3-day, face-to-face meeting in July of 2017. After an exhaustive review of the existing literature, the authors created consensus recommendations using the Grading of Recommendations, Assessment, Development and Evaluation criteria. Important directions for future research were also identified.
There is evidence to support that lymphovenous anastomosis can be effective in reducing severity of lymphedema (grade 1C). There is evidence to support that vascular lymph node transplantation can be effective in reducing severity of lymphedema (grade 1B). Currently, there is no consensus on which procedure (lymphovenous bypass versus vascular lymph node transplantation) is more effective (grade 2C). A few studies show that prophylactic lymphovenous bypass in patients undergoing extremity lymphadenectomy may reduce the incidence of lymphedema (grade 1B). More studies with longer follow-up are required to confirm this benefit. Debulking procedures such as liposuction are effective in addressing a nonfluid component such as fat involving lymphedema (grade 1C). There is a role for liposuction combined with physiologic procedures although the timing of each procedure is currently unresolved (grade 1C).
Many studies seem to support some efficacy of lymphovenous bypass and vascular lymph node transplantation. Many studies show the important role of lymphedema therapy and other procedures such as liposuction and debulking. The management of lymphedema is a challenging field with many promising advances. However, many questions remain unanswered.
•A meta-analysis of prebiotic and probiotic trials for depression and anxiety.•Prebiotics did not differ from placebo for depression or anxiety.•Probiotics yielded small but significant effects for ...depression and anxiety.•Probiotic effects were larger for clinical than community samples for depression.•More studies of clinical samples are needed fully to evaluate therapeutic potential.
With growing interest in the gut microbiome, prebiotics and probiotics have received considerable attention as potential treatments for depression and anxiety. We conducted a random-effects meta-analysis of 34 controlled clinical trials evaluating the effects of prebiotics and probiotics on depression and anxiety. Prebiotics did not differ from placebo for depression (d = −.08, p = .51) or anxiety (d = .12, p = .11). Probiotics yielded small but significant effects for depression (d = −.24, p < .01) and anxiety (d = −.10, p = .03). Sample type was a moderator for probiotics and depression, with a larger effect observed for clinical/medical samples (d = −.45, p < .001) than community ones. This effect increased to medium-to-large in a preliminary analysis restricted to psychiatric samples (d = −.73, p < .001). There is general support for antidepressant and anxiolytic effects of probiotics, but the pooled effects were reduced by the paucity of trials with clinical samples. Additional randomized clinical trials with psychiatric samples are necessary fully to evaluate their therapeutic potential.
PDF
