Randomised controlled trials (RCTs) are considered the gold standard when evaluating the causal effects of healthcare interventions. When RCTs cannot be used (e.g. ethically difficult), the ...interrupted time series (ITS) design is a possible alternative. ITS is one of the strongest quasi-experimental designs. The aim of this methodological study was to describe how ITS designs were being used, the design characteristics, and reporting in the healthcare setting.
We searched MEDLINE for reports of ITS designs published in 2015 which had a minimum of two data points collected pre-intervention and one post-intervention. There was no restriction on participants, language of study, or type of outcome. Data were summarised using appropriate summary statistics.
One hundred and sixteen studies were included in the study. Interventions evaluated were mainly programs 41 (35%) and policies 32 (28%). Data were usually collected at monthly intervals, 74 (64%). Of the 115 studies that reported an analysis, the most common method was segmented regression (78%), 55% considered autocorrelation, and only seven reported a sample size calculation. Estimation of intervention effects were reported as change in slope (84%) and change in level (70%) and 21% reported long-term change in levels.
This methodological study identified problems in the reporting of design features and results of ITS studies, and highlights the need for future work in the development of formal reporting guidelines and methodological work.
Dentists prescribe approximately 10% of antibiotics dispensed in UK community pharmacies. Despite clear clinical guidance, dentists often prescribe antibiotics inappropriately. This ...cluster-randomised controlled trial used routinely collected National Health Service (NHS) dental prescribing and treatment claim data to compare the impact of individualised audit and feedback (A&F) interventions on dentists' antibiotic prescribing rates.
All 795 antibiotic prescribing NHS general dental practices in Scotland were included. Practices were randomised to the control (practices = 163; dentists = 567) or A&F intervention group (practices = 632; dentists = 1,999). A&F intervention practices were allocated to one of two A&F groups: (1) individualised graphical A&F comprising a line graph plotting an individual dentist's monthly antibiotic prescribing rate (practices = 316; dentists = 1,001); or (2) individualised graphical A&F plus a written behaviour change message synthesising and reiterating national guidance recommendations for dental antibiotic prescribing (practices = 316; dentists = 998). Intervention practices were also simultaneously randomised to receive A&F: (i) with or without a health board comparator comprising the addition of a line to the graphical A&F plotting the monthly antibiotic prescribing rate of all dentists in the health board; and (ii) delivered at 0 and 6 mo or at 0, 6, and 9 mo, giving a total of eight intervention groups. The primary outcome, measured by the trial statistician who was blinded to allocation, was the total number of antibiotic items dispensed per 100 NHS treatment claims over the 12 mo post-delivery of the baseline A&F. Primary outcome data was available for 152 control practices (dentists = 438) and 609 intervention practices (dentists = 1,550). At baseline, the number of antibiotic items prescribed per 100 NHS treatment claims was 8.3 in the control group and 8.5 in the intervention group. At follow-up, antibiotic prescribing had decreased by 0.4 antibiotic items per 100 NHS treatment claims in control practices and by 1.0 in intervention practices. This represents a significant reduction (-5.7%; 95% CI -10.2% to -1.1%; p = 0.01) in dentists' prescribing rate in the intervention group relative to the control group. Intervention subgroup analyses found a 6.1% reduction in the antibiotic prescribing rate of dentists who had received the written behaviour change message relative to dentists who had not (95% CI -10.4% to -1.9%; p = 0.01). There was no significant between-group difference in the prescribing rate of dentists who received a health board comparator relative to those who did not (-4.3%; 95% CI -8.6% to 0.1%; p = 0.06), nor between dentists who received A&F at 0 and 6 mo relative to those who received A&F at 0, 6, and 9 mo (0.02%; 95% CI -4.2% to 4.2%; p = 0.99). The key limitations relate to the use of routinely collected datasets which did not allow evaluation of any effects on inappropriate prescribing.
A&F derived from routinely collected datasets led to a significant reduction in the antibiotic prescribing rate of dentists.
Current Controlled Trials ISRCTN49204710.
Background
The opportunity to improve care by delivering decision support to clinicians at the point of care represents one of the main incentives for implementing sophisticated clinical information ...systems. Previous reviews of computer reminder and decision support systems have reported mixed effects, possibly because they did not distinguish point of care computer reminders from e‐mail alerts, computer‐generated paper reminders, and other modes of delivering ‘computer reminders’.
Objectives
To evaluate the effects on processes and outcomes of care attributable to on‐screen computer reminders delivered to clinicians at the point of care.
Search methods
We searched the Cochrane EPOC Group Trials register, MEDLINE, EMBASE and CINAHL and CENTRAL to July 2008, and scanned bibliographies from key articles.
Selection criteria
Studies of a reminder delivered via a computer system routinely used by clinicians, with a randomised or quasi‐randomised design and reporting at least one outcome involving a clinical endpoint or adherence to a recommended process of care.
Data collection and analysis
Two authors independently screened studies for eligibility and ed data. For each study, we calculated the median improvement in adherence to target processes of care and also identified the outcome with the largest such improvement. We then calculated the median absolute improvement in process adherence across all studies using both the median outcome from each study and the best outcome.
Main results
Twenty‐eight studies (reporting a total of thirty‐two comparisons) were included. Computer reminders achieved a median improvement in process adherence of 4.2% (interquartile range (IQR): 0.8% to 18.8%) across all reported process outcomes, 3.3% (IQR: 0.5% to 10.6%) for medication ordering, 3.8% (IQR: 0.5% to 6.6%) for vaccinations, and 3.8% (IQR: 0.4% to 16.3%) for test ordering. In a sensitivity analysis using the best outcome from each study, the median improvement was 5.6% (IQR: 2.0% to 19.2%) across all process measures and 6.2% (IQR: 3.0% to 28.0%) across measures of medication ordering.
In the eight comparisons that reported dichotomous clinical endpoints, intervention patients experienced a median absolute improvement of 2.5% (IQR: 1.3% to 4.2%). Blood pressure was the most commonly reported clinical endpoint, with intervention patients experiencing a median reduction in their systolic blood pressure of 1.0 mmHg (IQR: 2.3 mmHg reduction to 2.0 mmHg increase).
Authors' conclusions
Point of care computer reminders generally achieve small to modest improvements in provider behaviour. A minority of interventions showed larger effects, but no specific reminder or contextual features were significantly associated with effect magnitude. Further research must identify design features and contextual factors consistently associated with larger improvements in provider behaviour if computer reminders are to succeed on more than a trial and error basis.
Non-randomised studies of the effects of interventions are critical to many areas of healthcare evaluation, but their results may be biased. It is therefore important to understand and appraise their ...strengths and weaknesses. We developed ROBINS-I (“Risk Of Bias In Non-randomised Studies - of Interventions”), a new tool for evaluating risk of bias in estimates of the comparative effectiveness (harm or benefit) of interventions from studies that did not use randomisation to allocate units (individuals or clusters of individuals) to comparison groups. The tool will be particularly useful to those undertaking systematic reviews that include non-randomised studies.
This trial of varicose-vein treatments showed no substantial differences in quality of life 6 months after ultrasound-guided foam sclerotherapy, endovenous laser ablation, or surgery, but ...disease-specific quality of life was slightly worse after foam treatment than after surgery.
Ultrasound-guided foam sclerotherapy and thermal ablation techniques such as endovenous laser ablation have become widely used alternatives to surgery for the treatment of varicose veins. Previous randomized trials and meta-analyses have shown these treatments to be effective in terms of short-term technical success and clinician-reported outcomes.
1
–
19
Clinical practice guidelines recommend the use of patient-reported quality of life to assess the outcomes of treatment of varicose veins.
20
Quality of life was a primary outcome measure in two small randomized trials that compared surgery and endovenous laser ablation,
5
,
9
but to our knowledge, it has not been assessed as a primary . . .
There is no agreement which outcomes should be measured when investigating interventions for periodontal diseases. It is difficult to compare or combine studies with different outcomes; resulting in ...research wastage and uncertainty for patients and healthcare professionals.
Develop a core outcome set (COS) relevant to key stakeholders for use in effectiveness trials investigating prevention and management of periodontal diseases.
Mixed method study involving literature review; online Delphi Study; and face-to-face consensus meeting.
Key stakeholders: patients, dentists, hygienist/therapists, periodontists, researchers.
The literature review identified 37 unique outcomes. Delphi round 1: 20 patients and 51 dental professional and researchers prioritised 25 and suggested an additional 11 outcomes. Delphi round 2: from the resulting 36 outcomes, 13 patients and 39 dental professionals and researchers prioritised 22 outcomes. A face-to-face consensus meeting was hosted in Dundee, Scotland by an independent chair. Eight patients and six dental professional and researchers participated. The final COS contains: Probing depths, Quality of life, Quantified levels of gingivitis, Quantified levels of plaque, Tooth loss.
Implementation of this COS will ensure the results of future effectiveness trials for periodontal diseases are more relevant to patients and dental professionals, reducing research wastage. This could reduce uncertainty for patients and dental professionals by ensuring the evidence used to inform their choices is meaningful to them. It could also strengthen the quality and certainty of the evidence about the relative effectiveness of interventions.
COMET Database: http://www.comet-initiative.org/studies/details/265?result=true.
An interrupted time series design is a powerful quasi-experimental approach for evaluating effects of interventions introduced at a specific point in time. To utilize the strength of this design, a ...modification to standard regression analysis, such as segmented regression, is required. In segmented regression analysis, the change in intercept and/or slope from pre- to post-intervention is estimated and used to test causal hypotheses about the intervention. We illustrate segmented regression using data from a previously published study that evaluated the effectiveness of a collaborative intervention to improve quality in pre-hospital ambulance care for acute myocardial infarction (AMI) and stroke. In the original analysis, a standard regression model was used with time as a continuous variable. We contrast the results from this standard regression analysis with those from segmented regression analysis. We discuss the limitations of the former and advantages of the latter, as well as the challenges of using segmented regression in analysing complex quality improvement interventions.
Based on the estimated change in intercept and slope from pre- to post-intervention using segmented regression, we found insufficient evidence of a statistically significant effect on quality of care for stroke, although potential clinically important effects for AMI cannot be ruled out.
Segmented regression analysis is the recommended approach for analysing data from an interrupted time series study. Several modifications to the basic segmented regression analysis approach are available to deal with challenges arising in the evaluation of complex quality improvement interventions.
Comparing the relative effectiveness of interventions across glaucoma trials can be problematic due to differences in definitions of outcomes. We sought to identify a key set of clinical outcomes and ...reach consensus on how best to measure them from the perspective of glaucoma experts.
A 2-round electronic Delphi survey was conducted. Round 1 involved 25 items identified from a systematic review. Round 2 was developed based on information gathered in round 1. A 10-point Likert scale was used to quantify importance and consensus of outcomes (7 outcomes) and ways of measuring them (44 measures). Experts were identified through 2 glaucoma societies membership-the UK and Eire Glaucoma Society and the European Glaucoma Society. A Nominal Group Technique (NGT) followed the Delphi process. Results were analyzed using descriptive statistics.
A total of 65 participants completed round 1 out of 320; of whom 56 completed round 2 (86%). Agreement on the importance of outcomes was reached on 48/51 items (94%). Intraocular pressure (IOP), visual field (VF), safety, and anatomic outcomes were classified as highly important. Regarding methods of measurement of IOP, "mean follow-up IOP" using Goldmann applanation tonometry achieved the highest importance, whereas for evaluating VFs "global index mean deviation/defect (MD)" and "rate of VF progression" were the most important. Retinal nerve fiber layer (RNFL) thickness measured by optical coherence tomography (OCT) was identified as highly important. The NGT results reached consensus on "change of IOP (mean of 3 consecutive measurements taken at fixed time of day) from baseline," change of VF-MD values (3 reliable VFs at baseline and follow-up visit) from baseline, and change of RNFL thickness (2 good quality OCT images) from baseline.
Consensus was reached among glaucoma experts on how best to measure IOP, VF, and anatomic outcomes in glaucoma randomized controlled trials.
In a randomized trial, 798 patients with varicose veins were assigned to undergo laser ablation, foam sclerotherapy, or surgery. At 5 years, disease-specific quality of life was better after laser ...ablation or surgery than after foam sclerotherapy, and cost-effectiveness analysis favored laser ablation.
To assess baseline ocular parameters in the prediction of long-term intraocular pressure (IOP) control after clear lens extraction (CLE) or laser peripheral iridotomy (LPI) in patients with primary ...angle closure (PAC) disease using data from the Effectiveness of Early Lens Extraction for the treatment of primary angle-closure glaucoma (EAGLE) tria.
This study is a secondary analysis of EAGLE data where we define the primary outcome of 'good responders' as those with IOP<21 mm Hg without requiring additional surgery and 'optimal responders' as those who in addition were medication free, at 36-month follow-up. Primary analysis was conducted using a multivariate logistic regression model to assess how randomised interventions and ocular parameters predict treatment response.
A total of 369 patients (182 in CLE arm and 187 in LPI arm) completed the 36-month follow-up examination. After CLE, 90% met our predefined 'good response' criterion compared with 67% in the LPI arm, and 66% met 'optimal response' criterion compared with 18% in the LPI arm, with significantly longer drops/surgery-free survival time (p<0.05 for all). Patients randomised to CLE (OR=10.1 (6.1 to 16.8)), Chinese (OR=2.3 (1.3 to 3.9)), and those who had not previously used glaucoma drops (OR=2.8 (1.6 to 4.8)) were more likely to maintain long-term optimal IOP response over 36 months.
Patients with primary angle closure glaucoma/PAC are 10 times more likely to maintain drop-free good IOP control with initial CLE surgery than LPI. Non-Chinese ethnicity, higher baseline IOP and using glaucoma drops prior to randomisation are predictors of worse long-term IOP response.
PDF
