Pragmatic randomized controlled trials (RCTs) mimic usual clinical practice and they are critical to inform decision-making by patients, clinicians and policy-makers in real-world settings. Pragmatic ...RCTs assess effectiveness of available medicines, while explanatory RCTs assess efficacy of investigational medicines. Explanatory and pragmatic are the extremes of a continuum. This debate article seeks to evaluate and provide recommendation on how to characterize pragmatic RCTs in light of the current landscape of RCTs. It is supported by findings from a PubMed search conducted in August 2017, which retrieved 615 RCTs self-labeled in their titles as "pragmatic" or "naturalistic". We focused on 89 of these trials that assessed medicines (drugs or biologics).
36% of these 89 trials were placebo-controlled, performed before licensing of the medicine, or done in a single-center. In our opinion, such RCTs overtly deviate from usual care and pragmatism. It follows, that the use of the term 'pragmatic' to describe them, conveys a misleading message to patients and clinicians. Furthermore, many other trials among the 615 coined as 'pragmatic' and assessing other types of intervention are plausibly not very pragmatic; however, this is impossible for a reader to tell without access to the full protocol and insider knowledge of the trial conduct. The degree of pragmatism should be evaluated by the trial investigators themselves using the PRECIS-2 tool, a tool that comprises 9 domains, each scored from 1 (very explanatory) to 5 (very pragmatic).
To allow for a more appropriate characterization of the degree of pragmatism in clinical research, submissions of RCTs to funders, research ethics committees and to peer-reviewed journals should include a PRECIS-2 tool assessment done by the trial investigators. Clarity and accuracy on the extent to which a RCT is pragmatic will help understand how much it is relevant to real-world practice.
To systematically assess the robustness of reported postacute SARS-CoV-2 infection health outcomes in children.
A search on PubMed and Web of Science was conducted to identify studies published up to ...22 January 2022 that reported on postacute SARS-CoV-2 infection health outcomes in children (<18 years) with follow-up of ≥2 months since detection of infection or ≥1 month since recovery from acute illness. We assessed the consideration of confounding bias and causality, as well as the risk of bias.
21 studies including 81 896 children reported up to 97 symptoms with follow-up periods of 2.0-11.5 months. Fifteen studies had no control group. The reported proportion of children with post-COVID syndrome was between 0% and 66.5% in children with SARS-CoV-2 infection (n=16 986) and between 2.0% and 53.3% in children without SARS-CoV-2 infection (n=64 910). Only two studies made a clear causal interpretation of an association between SARS-CoV-2 infection and the main outcome of 'post-COVID syndrome' and provided recommendations regarding prevention measures. The robustness of all 21 studies was seriously limited due to an overall critical risk of bias.
The robustness of reported postacute SARS-CoV-2 infection health outcomes in children is seriously limited, at least in all the published articles we could identify. None of the studies provided evidence with reasonable certainty on whether SARS-CoV-2 infection has an impact on postacute health outcomes, let alone to what extent. Children and their families urgently need much more reliable and methodologically robust evidence to address their concerns and improve care.
Purpose
To study the landscape of funding in intensive care research and assess whether the reported outcomes of industry-funded randomized controlled trials (RCTs) are more favorable.
Methods
We ...systematically assembled meta-analyses evaluating any type of intervention in the critical care setting and reporting the source of funding for each included RCT. Furthermore, when the intervention was a drug or biologic, we searched also the original RCT articles, when their funding information was unavailable in the meta-analysis. We then qualitatively summarized the sources of funding. For binary outcomes, separate summary odds ratios were calculated for trials with and without industry funding. We then calculated the ratio of odds ratios (RORs) and the summary ROR (sROR) across topics. ROR < 1 implies that the experimental intervention is relatively more favorable in trials with industry funding compared with trials without industry funding. For RCTs included in the ROR analysis, we also examined the conclusions of their abstract.
Results
Across 67 topics with 568 RCTs, 88 were funded by industry and another 73 had both industry and non-profit funding. Across 33 topics with binary outcomes, the sROR was 1.10 95% CI (0.96–1.26),
I
2
= 1%. Conclusions were not significantly more commonly unfavorable for the experimental arm interventions in industry-funded trials (21.3%) compared with trials without industry funding (18.2%).
Conclusion
Industry-funded RCTs are the minority in intensive care. We found no evidence that industry-funded trials in intensive care yield more favorable results or are less likely to reach unfavorable conclusions.
Pragmatic trials provide decision-oriented, real-world evidence that is highly applicable and generalizable. The interest in real-world evidence is fueled by the assumption that effects in the ..."real-world" are different to effects obtained under artificial, controlled, research conditions as often used for traditional explanatory trials. However, it is unknown which features of pragmatism, generalizability, and applicability would be responsible for such differences. There is a need to provide empirical evidence and promote meta-research to answer these fundamental questions on the pragmatism of randomized trials and real-world evidence. Here, we describe the rationale and design of the PragMeta database which pursues this goal ( www.PragMeta.org ).
PragMeta is a non-commercial, open data platform and infrastructure to facilitate research on pragmatic trials. It collects and shares data from published randomized trials that either have a specific design feature or other characteristic related to pragmatism or they form clusters of trials addressing the same research question but having different aspects of pragmatism. This lays the foundation to determine the relationship of various features of pragmatism, generalizability, and applicability with intervention effects or other trial characteristics. The database contains trial data actively collected for PragMeta but also allows to import and link existing datasets of trials collected for other purposes, forming a large-scale meta-database. PragMeta captures data on (1) trial and design characteristics (e.g., sample size, population, intervention/comparison, outcome, longitudinal structure, blinding), (2) effects estimates, and (3) various determinants of pragmatism (e.g., the use of routinely collected data) and ratings from established tools used to determine pragmatism (e.g., the PRagmatic-Explanatory Continuum Indicator Summary 2; PRECIS-2). PragMeta is continuously provided online, inviting the meta-research community to collaborate, contribute, and/or use the database. As of April 2023, PragMeta contains data from > 700 trials, mostly with assessments on pragmatism.
PragMeta will inform a better understanding of pragmatism and the generation and interpretation of real-world evidence.
Ten simple rules for good research practice Schwab, Simon; Janiaud, Perrine; Dayan, Michael ...
PLOS computational biology/PLoS computational biology,
06/2022, Volume:
18, Issue:
6
Journal Article
•Access to precise and useful information is important for all clinical trials, regardless of whether they assess medicines or other interventions•An important tool to provide this information are ...clinical trial registers.•There are different models proposed by the US NIH and the IDEAL framework for describing the development stages of behavioural interventions, surgery, devices, radiation oncology, or physiotherapy.•However, more than 21,000 non-medicines trials are described in ClinicalTrials.gov with the phases of clinical development of medicines. This is also common in publish reports•─This situation can be corrected by both registers and journals teams
Background:
Pragmatic trials are increasingly recognized for providing real-world evidence on treatment choices.
Objective:
The objective of this study is to investigate the use and characteristics ...of pragmatic trials in multiple sclerosis (MS).
Methods:
Systematic literature search and analysis of pragmatic trials on any intervention published up to 2022. The assessment of pragmatism with PRECIS-2 (PRagmatic Explanatory Continuum Indicator Summary-2) is performed.
Results:
We identified 48 pragmatic trials published 1967–2022 that included a median of 82 participants (interquartile range (IQR) = 42–160) to assess typically supportive care interventions (n = 41; 85%). Only seven trials assessed drugs (15%). Only three trials (6%) included >500 participants. Trials were mostly from the United Kingdom (n = 18; 38%), Italy (n = 6; 13%), the United States and Denmark (each n = 5; 10%). Primary outcomes were diverse, for example, quality-of-life, physical functioning, or disease activity. Only 1 trial (2%) used routinely collected data for outcome ascertainment. No trial was very pragmatic in all design aspects, but 14 trials (29%) were widely pragmatic (i.e. PRECIS-2 score ⩾ 4/5 in all domains).
Conclusion:
Only few and mostly small pragmatic trials exist in MS which rarely assess drugs. Despite the widely available routine data infrastructures, very few trials utilize them. There is an urgent need to leverage the potential of this pioneering study design to provide useful randomized real-world evidence.
The validity of observational studies and their meta-analyses is contested. Here, we aimed to appraise thousands of meta-analyses of observational studies using a pre-specified set of quantitative ...criteria that assess the significance, amount, consistency, and bias of the evidence. We also aimed to compare results from meta-analyses of observational studies against meta-analyses of randomized controlled trials (RCTs) and Mendelian randomization (MR) studies.
We retrieved from PubMed (last update, November 19, 2020) umbrella reviews including meta-analyses of observational studies assessing putative risk or protective factors, regardless of the nature of the exposure and health outcome. We extracted information on 7 quantitative criteria that reflect the level of statistical support, the amount of data, the consistency across different studies, and hints pointing to potential bias. These criteria were level of statistical significance (pre-categorized according to 10
, 0.001, and 0.05 p-value thresholds), sample size, statistical significance for the largest study, 95% prediction intervals, between-study heterogeneity, and the results of tests for small study effects and for excess significance.
3744 associations (in 57 umbrella reviews) assessed by a median number of 7 (interquartile range 4 to 11) observational studies were eligible. Most associations were statistically significant at P < 0.05 (61.1%, 2289/3744). Only 2.6% of associations had P < 10
, ≥1000 cases (or ≥20,000 participants for continuous factors), P < 0.05 in the largest study, 95% prediction interval excluding the null, and no large between-study heterogeneity, small study effects, or excess significance. Across the 57 topics, large heterogeneity was observed in the proportion of associations fulfilling various quantitative criteria. The quantitative criteria were mostly independent from one another. Across 62 associations assessed in both RCTs and in observational studies, 37.1% had effect estimates in opposite directions and 43.5% had effect estimates differing beyond chance in the two designs. Across 94 comparisons assessed in both MR and observational studies, such discrepancies occurred in 30.8% and 54.7%, respectively.
Acknowledging that no gold-standard exists to judge whether an observational association is genuine, statistically significant results are common in observational studies, but they are rarely convincing or corroborated by randomized evidence.
A potential larger perceived placebo effect in children compared with adults could influence the detection of the treatment effect and the extrapolation of the treatment benefit from adults to ...children. This study aims to explore this potential difference, using a meta-epidemiological approach.
A systematic review of the literature was done to identify trials included in meta-analyses evaluating a drug intervention with separate data for adults and children. The standardized mean change and the proportion of responders (binary outcomes) were used to calculate the perceived placebo effect. A meta-regression analysis was conducted to test for the difference between adults and children of the perceived placebo effect.
For binary outcomes, the perceived placebo effect was significantly more favorable in children compared with adults (β = 0.13; P = 0.001). Parallel group trials (β = -1.83; P < 0.001), subjective outcomes (β = -0.76; P < 0.001), and the disease type significantly influenced the perceived placebo effect.
The perceived placebo effect is different between adults and children for binary outcomes. This difference seems to be influenced by the design, the disease, and outcomes. Calibration of new studies for children should consider cautiously the placebo effect in children.
PDF
