Clinical researchers have often preferred to use a fixed effects model for the primary interpretation of a meta-analysis. Heterogeneity is usually assessed via the well known Q and I2 statistics, ...along with the random effects estimate they imply. In recent years, alternative methods for quantifying heterogeneity have been proposed, that are based on a 'generalised' Q statistic.
We review 18 IPD meta-analyses of RCTs into treatments for cancer, in order to quantify the amount of heterogeneity present and also to discuss practical methods for explaining heterogeneity.
Differing results were obtained when the standard Q and I2 statistics were used to test for the presence of heterogeneity. The two meta-analyses with the largest amount of heterogeneity were investigated further, and on inspection the straightforward application of a random effects model was not deemed appropriate. Compared to the standard Q statistic, the generalised Q statistic provided a more accurate platform for estimating the amount of heterogeneity in the 18 meta-analyses.
Explaining heterogeneity via the pre-specification of trial subgroups, graphical diagnostic tools and sensitivity analyses produced a more desirable outcome than an automatic application of the random effects model. Generalised Q statistic methods for quantifying and adjusting for heterogeneity should be incorporated as standard into statistical software. Software is provided to help achieve this aim.
In systematic reviews and meta-analyses, time-to-event outcomes are most appropriately analysed using hazard ratios (HRs). In the absence of individual patient data (IPD), methods are available to ...obtain HRs and/or associated statistics by carefully manipulating published or other summary data. Awareness and adoption of these methods is somewhat limited, perhaps because they are published in the statistical literature using statistical notation.
This paper aims to 'translate' the methods for estimating a HR and associated statistics from published time-to-event-analyses into less statistical and more practical guidance and provide a corresponding, easy-to-use calculations spreadsheet, to facilitate the computational aspects.
A wider audience should be able to understand published time-to-event data in individual trial reports and use it more appropriately in meta-analysis. When faced with particular circumstances, readers can refer to the relevant sections of the paper. The spreadsheet can be used to assist them in carrying out the calculations.
The methods cannot circumvent the potential biases associated with relying on published data for systematic reviews and meta-analysis. However, this practical guide should improve the quality of the analysis and subsequent interpretation of systematic reviews and meta-analyses that include time-to-event outcomes.
It remains unclear when standard systematic reviews and meta-analyses that rely on published aggregate data (AD) can provide robust clinical conclusions. We aimed to compare the results from a large ...cohort of systematic reviews and meta-analyses based on individual participant data (IPD) with meta-analyses of published AD, to establish when the latter are most likely to be reliable and when the IPD approach might be required.
We used 18 cancer systematic reviews that included IPD meta-analyses: all of those completed and published by the Meta-analysis Group of the MRC Clinical Trials Unit from 1991 to 2010. We extracted or estimated hazard ratios (HRs) and standard errors (SEs) for survival from trial reports and compared these with IPD equivalents at both the trial and meta-analysis level. We also extracted or estimated the number of events. We used paired t tests to assess whether HRs and SEs from published AD differed on average from those from IPD. We assessed agreement, and whether this was associated with trial or meta-analysis characteristics, using the approach of Bland and Altman. The 18 systematic reviews comprised 238 unique trials or trial comparisons, including 37,082 participants. A HR and SE could be generated for 127 trials, representing 53% of the trials and approximately 79% of eligible participants. On average, trial HRs derived from published AD were slightly more in favour of the research interventions than those from IPD (HRAD to HRIPD ratio = 0.95, p = 0.007), but the limits of agreement show that for individual trials, the HRs could deviate substantially. These limits narrowed with an increasing number of participants (p < 0.001) or a greater number (p < 0.001) or proportion (p < 0.001) of events in the AD. On average, meta-analysis HRs from published AD slightly tended to favour the research interventions whether based on fixed-effect (HRAD to HRIPD ratio = 0.97, p = 0.088) or random-effects (HRAD to HRIPD ratio = 0.96, p = 0.044) models, but the limits of agreement show that for individual meta-analyses, agreement was much more variable. These limits tended to narrow with an increasing number (p = 0.077) or proportion of events (p = 0.11) in the AD. However, even when the information size of the AD was large, individual meta-analysis HRs could still differ from their IPD equivalents by a relative 10% in favour of the research intervention to 5% in favour of control. We utilised the results to construct a decision tree for assessing whether an AD meta-analysis includes sufficient information, and when estimates of effects are most likely to be reliable. A lack of power at the meta-analysis level may have prevented us identifying additional factors associated with the reliability of AD meta-analyses, and we cannot be sure that our results are generalisable to all outcomes and effect measures.
In this study we found that HRs from published AD were most likely to agree with those from IPD when the information size was large. Based on these findings, we provide guidance for determining systematically when standard AD meta-analysis will likely generate robust clinical conclusions, and when the IPD approach will add considerable value.
Identifying which individuals benefit most from particular treatments or other interventions underpins so-called personalised or stratified medicine. However, single trials are typically underpowered ...for exploring whether participant characteristics, such as age or disease severity, determine an individual’s response to treatment. A meta-analysis of multiple trials, particularly one where individual participant data (IPD) are available, provides greater power to investigate interactions between participant characteristics (covariates) and treatment effects. We use a published IPD meta-analysis to illustrate three broad approaches used for testing such interactions. Based on another systematic review of recently published IPD meta-analyses, we also show that all three approaches can be applied to aggregate data as well as IPD. We also summarise which methods of analysing and presenting interactions are in current use, and describe their advantages and disadvantages. We recommend that testing for interactions using within-trials information alone (the deft approach) becomes standard practice, alongside graphical presentation that directly visualises this.
Summary Background Results from large randomised controlled trials combining docetaxel or bisphosphonates with standard of care in hormone-sensitive prostate cancer have emerged. In order to ...investigate the effects of these therapies and to respond to emerging evidence, we aimed to systematically review all relevant trials using a framework for adaptive meta-analysis. Methods For this systematic review and meta-analysis, we searched MEDLINE, Embase, LILACS, and the Cochrane Central Register of Controlled Trials, trial registers, conference proceedings, review articles, and reference lists of trial publications for all relevant randomised controlled trials (published, unpublished, and ongoing) comparing either standard of care with or without docetaxel or standard of care with or without bisphosphonates for men with high-risk localised or metastatic hormone-sensitive prostate cancer. For each trial, we extracted hazard ratios (HRs) of the effects of docetaxel or bisphosphonates on survival (time from randomisation until death from any cause) and failure-free survival (time from randomisation to biochemical or clinical failure or death from any cause) from published trial reports or presentations or obtained them directly from trial investigators. HRs were combined using the fixed-effect model (Mantel-Haenzsel). Findings We identified five eligible randomised controlled trials of docetaxel in men with metastatic (M1) disease. Results from three (CHAARTED, GETUG-15, STAMPEDE) of these trials (2992 93% of 3206 men randomised) showed that the addition of docetaxel to standard of care improved survival. The HR of 0·77 (95% CI 0·68–0·87; p<0·0001) translates to an absolute improvement in 4-year survival of 9% (95% CI 5–14). Docetaxel in addition to standard of care also improved failure-free survival, with the HR of 0·64 (0·58–0·70; p<0·0001) translating into a reduction in absolute 4-year failure rates of 16% (95% CI 12–19). We identified 11 trials of docetaxel for men with locally advanced disease (M0). Survival results from three (GETUG-12, RTOG 0521, STAMPEDE) of these trials (2121 53% of 3978 men) showed no evidence of a benefit from the addition of docetaxel (HR 0·87 95% CI 0·69–1·09; p=0·218), whereas failure-free survival data from four (GETUG-12, RTOG 0521, STAMPEDE, TAX 3501) of these trials (2348 59% of 3978 men) showed that docetaxel improved failure-free survival (0·70 0·61–0·81; p<0·0001), which translates into a reduced absolute 4-year failure rate of 8% (5–10). We identified seven eligible randomised controlled trials of bisphosphonates for men with M1 disease. Survival results from three of these trials (2740 88% of 3109 men) showed that addition of bisphosphonates improved survival (0·88 0·79–0·98; p=0·025), which translates to 5% (1–8) absolute improvement, but this result was influenced by the positive result of one trial of sodium clodronate, and we found no evidence of a benefit from the addition of zoledronic acid (0·94 0·83–1·07; p=0·323), which translates to an absolute improvement in survival of 2% (−3 to 7). Of 17 trials of bisphosphonates for men with M0 disease, survival results from four trials (4079 66% of 6220 men) showed no evidence of benefit from the addition of bisphosphonates (1·03 0·89–1·18; p=0·724) or zoledronic acid (0·98 0·82–1·16; p=0·782). Failure-free survival definitions were too inconsistent for formal meta-analyses for the bisphosphonate trials. Interpretation The addition of docetaxel to standard of care should be considered standard care for men with M1 hormone-sensitive prostate cancer who are starting treatment for the first time. More evidence on the effects of docetaxel on survival is needed in the M0 disease setting. No evidence exists to suggest that zoledronic acid improves survival in men with M1 or M0 disease, and any potential benefit is probably small. Funding Medical Research Council UK.
It is unclear whether adjuvant or early salvage radiotherapy following radical prostatectomy is more appropriate for men who present with localised or locally advanced prostate cancer. We aimed to ...prospectively plan a systematic review of randomised controlled trials (RCTs) comparing these radiotherapy approaches.
We used a prospective framework for adaptive meta-analysis (FAME), starting the review process while eligible trials were ongoing. RCTs were eligible if they aimed to compare immediate adjuvant radiotherapy versus early salvage radiotherapy, following radical prostatectomy in men (age ≥18 years) with intermediate-risk or high-risk, localised or locally advanced prostate cancer. We searched trial registers and conference proceedings until July 8, 2020, to identify eligible RCTs. By establishing the ARTISTIC collaboration with relevant trialists, we were able to anticipate when eligible trial results would emerge, and we developed and registered a protocol with PROSPERO before knowledge of the trial results (CRD42019132669). We used a harmonised definition of event-free survival, as the time from randomisation until the first evidence of either biochemical progression (prostate-specific antigen PSA ≥0·4 ng/mL and rising after completion of any postoperative radiotherapy), clinical or radiological progression, initiation of a non-trial treatment, death from prostate cancer, or a PSA level of at least 2·0 ng/mL at any time after randomisation. We predicted when we would have sufficient power to assess whether adjuvant radiotherapy was superior to early salvage radiotherapy. Investigators supplied results for event-free survival, both overall and within predefined patient subgroups. Hazard ratios (HRs) for the effects of radiotherapy timing on event-free survival and subgroup interactions were combined using fixed-effect meta-analysis.
We identified three eligible trials and were able to obtain updated results for event-free survival for 2153 patients recruited between November, 2007, and December, 2016. Median follow-up ranged from 60 months to 78 months, with a maximum follow-up of 132 months. 1075 patients were randomly assigned to receive adjuvant radiotherapy and 1078 to a policy of early salvage radiotherapy, of whom 421 (39·1%) had commenced treatment at the time of analysis. Patient characteristics were balanced within trials and overall. Median age was similar between trials at 64 or 65 years (with IQRs ranging from 59 to 68 years) across the three trials and most patients (1671 77·6%) had a Gleason score of 7. All trials were assessed as having low risk of bias. Based on 270 events, the meta-analysis showed no evidence that event-free survival was improved with adjuvant radiotherapy compared with early salvage radiotherapy (HR 0·95, 95% CI 0·75–1·21; p=0·70), with only a 1 percentage point (95% CI −2 to 3) change in 5-year event-free survival (89% vs 88%). Results were consistent across trials (heterogeneity p=0·18; I2=42%).
This collaborative and prospectively designed systematic review and meta-analysis suggests that adjuvant radiotherapy does not improve event-free survival in men with localised or locally advanced prostate cancer. Until data on long-term outcomes are available, early salvage treatment would seem the preferable treatment policy as it offers the opportunity to spare many men radiotherapy and its associated side-effects.
UK Medical Research Council.
Many trials are evaluating therapies for men with metastatic hormone-sensitive prostate cancer (mHSPC).
To systematically review trials of prostate radiotherapy.
Using a prospective framework ...(framework for adaptive meta-analysis FAME), we prespecified methods before any trial results were known. We searched extensively for eligible trials and asked investigators when results would be available. We could then anticipate that a definitive meta-analysis of the effects of prostate radiotherapy was possible. We obtained prepublication, unpublished, and harmonised results from investigators.
We included trials that randomised men to prostate radiotherapy and androgen deprivation therapy (ADT) or ADT only.
Hazard ratios (HRs) for the effects of prostate radiotherapy on survival, progression-free survival (PFS), failure-free survival (FFS), biochemical progression, and subgroup interactions were combined using fixed-effect meta-analysis.
We identified one ongoing (PEACE-1) and two completed (HORRAD and STAMPEDE) eligible trials. Pooled results of the latter (2126 men; 90% of those eligible) showed no overall improvement in survival (HR=0.92, 95% confidence interval CI 0.81–1.04, p=0.195) or PFS (HR=0.94, 95% CI 0.84–1.05, p=0.238) with prostate radiotherapy. There was an overall improvement in biochemical progression (HR=0.74, 95% CI 0.67–0.82, p=0.94×10−8) and FFS (HR=0.76, 95% CI 0.69–0.84, p=0.64×10−7), equivalent to ∼10% benefit at 3yr. The effect of prostate radiotherapy varied by metastatic burden—a pattern consistent across trials and outcome measures, including survival (<5, ≥5; interaction HR=1.47, 95% CI 1.11–1.94, p=0.007). There was 7% improvement in 3-yr survival in men with fewer than five bone metastases.
Prostate radiotherapy should be considered for men with mHSPC with a low metastatic burden.
Prostate cancer that has spread to other parts of the body (metastases) is usually treated with hormone therapy. In men with fewer than five bone metastases, addition of prostate radiotherapy helped them live longer and should be considered.
This prospectively designed systematic review showed that prostate radiotherapy did not improve survival in unselected men. A difference in effect by metastatic burden was evident across included trials and outcomes; 3-yr survival increased by 7% in men with fewer than five bone metastases.
A recent prospective meta-analysis demonstrated that interleukin-6 antagonists are associated with lower all-cause mortality in hospitalised patients with COVID-19, compared with usual care or ...placebo. However, emerging evidence suggests that clinicians are favouring the use of tocilizumab over sarilumab. A new randomised comparison of these agents from the REMAP-CAP trial shows similar effects on in-hospital mortality. Therefore, we initiated a network meta-analysis, to estimate pairwise associations between tocilizumab, sarilumab and usual care or placebo with 28-day mortality, in COVID-19 patients receiving concomitant corticosteroids and ventilation, based on all available direct and indirect evidence. Eligible trials randomised hospitalised patients with COVID-19 that compared tocilizumab or sarilumab with usual care or placebo in the prospective meta-analysis or that directly compared tocilizumab with sarilumab. Data were restricted to patients receiving corticosteroids and either non-invasive or invasive ventilation at randomisation. One trial (REMAP-CAP) was identified that directly compared tocilizumab with sarilumab and supplied results on all-cause mortality at 28-days. This network meta-analysis was based on 898 eligible patients (278 deaths) from REMAP-CAP and 3710 eligible patients from 18 trials (1278 deaths) from the prospective meta-analysis. Summary ORs were similar for tocilizumab 0·82 0·71-0·95, p = 0·008 and sarilumab 0·80 0·61-1·04, p = 0·09 compared with usual care or placebo. The summary OR for 28-day mortality comparing tocilizumab with sarilumab was 1·03 95%CI 0·81-1·32, p = 0·80. The p-value for the global test of inconsistency was 0·28. Administration of either tocilizumab or sarilumab was associated with lower 28-day all-cause mortality compared with usual care or placebo. The association is not dependent on the choice of interleukin-6 receptor antagonist.
The vast majority of systematic reviews are planned retrospectively, once most eligible trials have completed and reported, and are based on aggregate data that can be extracted from publications. ...Prior knowledge of trial results can introduce bias into both review and meta-analysis methods, and the omission of unpublished data can lead to reporting biases. We present a collaborative framework for prospective, adaptive meta-analysis (FAME) of aggregate data to provide results that are less prone to bias. Also, with FAME, we monitor how evidence from trials is accumulating, to anticipate the earliest opportunity for a potentially definitive meta-analysis.
We developed and piloted FAME alongside 4 systematic reviews in prostate cancer, which allowed us to refine the key principles. These are to: (1) start the systematic review process early, while trials are ongoing or yet to report; (2) liaise with trial investigators to develop a detailed picture of all eligible trials; (3) prospectively assess the earliest possible timing for reliable meta-analysis based on the accumulating aggregate data; (4) develop and register (or publish) the systematic review protocol before trials produce results and seek appropriate aggregate data; (5) interpret meta-analysis results taking account of both available and unavailable data; and (6) assess the value of updating the systematic review and meta-analysis. These principles are illustrated via a hypothetical review and their application to 3 published systematic reviews.
FAME can reduce the potential for bias, and produce more timely, thorough and reliable systematic reviews of aggregate data.
PDF
