ObjectivesRobust research of complex interventions designed to promote mental well-being in later life is required to inform service development. An essential component is ensuring that such ...interventions are delivered as intended. We present a detailed description of the design and implementation of a fidelity assessment within a trial of one such intervention (Lifestyle Matters). The findings help to explain the trial results and also inform the design of embedded fidelity assessments within future evaluations of complex interventions.DesignWe conducted a mixed-method fidelity assessment embedded as part of a multicentre pragmatic randomised controlled trial. A conceptual fidelity framework was developed from the Behaviour Change Consortium framework. From this the fidelity assessment was designed. The resulting instrument assessed the following parameters: intervention design, training, supervision; and delivery, receipt and enactment of the intervention.InterventionThe Lifestyle Matters intervention was designed to assist older people to improve and sustain mental well-being through participation in meaningful activity. The aim is to enable participants to engage in both new and neglected activities through a mix of facilitated group meetings and individual sessions.ResultsThe fidelity assessment demonstrated that the intervention was delivered as per protocol for the group component and was tailored to meet individual needs. There was substantial inter-rater agreement for training; and group member performance 0.72; and moderate agreement for facilitator performance 0.55. It was not possible to determine whether small declines seen in facilitator performance were due to facilitator drift or moderating factors such as group dynamics or participant characteristics.ConclusionsThe assessment methods adequately measured criteria identified as being significant indicators of fidelity. Adherence during training, delivery and supervision was good. The subjective nature of identification and rating observed behaviours was the main challenge. Future research should explore alternative methods of assessing fidelity in trials of complex interventions.Trial registration numberISRCTN67209155.
Harms, also known as adverse events (AEs), are recorded and monitored in randomised controlled trials (RCTs) to ensure participants' safety. Harms are recorded poorly or inconsistently in RCTs of ...Behaviour Change Interventions (BCI); however, limited guidance exists on how to record harms in BCI trials. This qualitative study explored experiences and perspectives from multi-disciplinary trial experts on recording harms in BCI trials.
Data were collected through fifteen in-depth semi-structured qualitative interviews and three focus groups with thirty-two participants who work in the delivery and oversight of clinical trials. Participants included multi-disciplinary staff from eight CTUs, Chief investigators, and patient and public representatives. Interviews and focus group recordings were transcribed verbatim and thematic analysis was used to analyse the transcripts.
Five themes were identified, namely perception and understanding of harm, proportionate reporting and plausibility, the need for a multi-disciplinary approach, language of BCI harms and complex harms for complex interventions. Participants strongly believed harms should be recorded in BCI trials; however, making decisions on "how and what to record as harms" was difficult. Recording irrelevant harms placed a high burden on trial staff and participants, drained trial resources and was perceived as for little purpose. Participants believed proportionate recording was required that focused on events with a strong plausible link to the intervention. Multi-disciplinary trial team input was essential for identifying and collecting harms; however, this was difficult in practice due to lack of knowledge on harms from BCIs, lack of input or difference in opinion. The medical language of harms was recognised as a poor fit for BCI trial harms but was familiar and established within internal processes. Future guidance on this topic would be welcomed and could include summarised literature.
Recording harms or adverse events in behaviour change intervention trials is complex and challenging; multi-disciplinary experts in trial design and implementation welcome forthcoming guidance on this topic. Issues include the high burden of recording irrelevant harms and use of definitions originally designed for drug trials. Proportionate recording of harms focused on events with a strong plausible link to the intervention and multi-disciplinary team input into decision making are essential.
Explainer animations are a means to communicate aspects of clinical trials to participants in a more engaging and accessible way. Delivered well these have the potential to enhance recruitment and ...retention. The range of media technology used to deliver this material is expanding rapidly but is highly fragmented. Usage of explainer animations across the UK is unknown, the aim of this research was to determine current usage across the 52 registered UK Clinical Research Collaboration (UKCRC) Clinical Trials Units (CTUs) to understand the current landscape and any barriers that could be preventing wider uptake of this functionality.
A survey link was emailed to all UKCRC CTU Directors and Trial Management Leads to ascertain current usage of explainer animations within their CTU. The survey ran between 01 February 2023 and 07 March 2023.
Responses were received from 35 CTUs-representing a response rate of 67%. 24 CTUs (69%) reported that they had created/used at least one explainer animation within their unit, although the usage, cost, length and production activities varied among the units.
The survey showed that a high proportion of the UKCRC CTUs have used explainer animations to provide information to participants about clinical studies. For those not using the technology yet, the most common reasons cited were a lack of expertise, lack of resources and costs to produce them. One of the desired outcomes of this project is the creation of a free-to-use library of animations to encourage wider uptake and avoid duplication.
There exist many different methods of allocating participants to treatment groups during a randomised controlled trial. Although there is research that explores trial characteristics that are ...associated with the choice of method, there is still a lot of variety in practice not explained. This study used qualitative methods to explore more deeply the motivations behind researchers' choice of randomisation, and which features of the method they use to evaluate the performance of these methods.
Data was collected from online focus groups with various stakeholders involved in the randomisation process. Focus groups were recorded and then transcribed verbatim. A thematic analysis was used to analyse the transcripts.
Twenty-five participants from twenty clinical trials units across the UK were recruited to take part in one of four focus groups. Four main themes were identified: how randomisation methods are selected; researchers' opinions of the different methods; which features of the method are desirable and ways to measure method features. Most researchers agree that the randomisation method should be selected based on key trial characteristics; however, for many, a unit standard is in place. Opinions of methods were varied with some participants favouring stratified blocks and others favouring minimisation. This was generally due to researchers' perception of the effect these methods had on balance and predictability. Generally, predictability was considered more important than balance as adjustments cannot be made for it; however, most researchers felt that the importance of these two methods was dependent on the design of the study. Balance is usually evaluated by tabulating variables by treatment arm and looking for perceived imbalances, predictability was generally considered much harder to measure, partly due to differing definitions.
There is a wide variety in practice on how randomisation methods are selected and researcher's opinions on methods. The difference in practice observed when looking at randomisation method selection can be explained by a difference in unit practice, and also by a difference in researchers prioritisation of balance and predictability. The findings of this study show a need for more guidance on randomisation method selection.
Abstract
Background
Blinding is an established approach in clinical trials which aims to minimise the risk of performance and detection bias. There is little empirical evidence to guide UK clinical ...trials units (CTUs) about the practice of blinding statisticians. Guidelines recommend that statisticians remain blinded to allocation prior to the final analysis. As these guidelines are not based on empirical evidence, this study undertook a qualitative investigation relating to when and how statisticians should be blinded in clinical trials.
Methods
Data were collected through online focus groups with various stakeholders who work in the delivery and oversight of clinical trials. Recordings of the focus groups were transcribed verbatim and thematic analysis was used to analyse the transcripts.
Results
Thirty-seven participants from 19 CTUs participated in one of six focus groups. Four main themes were identified, namely statistical models of work, factors affecting the decision to blind statisticians, benefits of blinding/not blinding statisticians and practicalities. Factors influencing the decision to blind the statistician included available resources, study design and types of intervention and outcomes and analysis.
Although blinding of the statistician is perceived as a desirable mitigation against bias, there was uncertainty about the extent to which an unblinded statistician might impart bias. Instead, in most cases, the insight that the statistician offers was deemed more important to delivery of a trial than the risk of bias they may introduce if unblinded.
Blinding of statisticians was only considered achievable with the appropriate resource and staffing, which were not always available. In many cases, a standard approach to blinding was therefore considered unrealistic and impractical; hence the need for a proportionate risk assessment approach identifying possible mitigations.
Conclusions
There was wide variation in practice between UK CTUs regarding the blinding of trial statisticians. A risk assessment approach would enable CTUs to identify risks associated with unblinded statisticians conducting the final analysis and alternative mitigation strategies. The findings of this study will be used to design guidance and a tool to support this risk assessment process.
Abstract
Background
Barriers to mental health research participation are well documented including distrust of services and research; and stigma surrounding mental health. They can contribute to a ...lack of diversity amongst participants in mental health research, which threatens the generalisability of knowledge. Given the recent widespread use of the internet in medical research, this study aimed to explore the perspectives of key partners on the use of online (e.g. social media) and offline (e.g. in-person) recruitment as an approach to improving diversity in mental health randomised controlled trials (RCTs).
Methods
Face-to-face and online interviews/focus groups with researchers working in mental health and Patient and Public Involvement partners in the United Kingdom. Recordings were transcribed and analysed using a combination of inductive and deductive thematic analysis.
Results
Three focus groups and three interviews were conducted with a total N = 23 participants. Four overarching themes were identified: (1) recruitment reach; (2) Demographic factors that affect selection of recruitment method; (3) safety of technology, and; (4) practical challenges. Five main factors were identified that affect the choice of recruitment method: age, complexity of mental health problem and stigma, cultural and ethnicity differences and digital divide. The use of online methods was considered more accessible to people who may feel stigmatised by their mental health condition and with a benefit of reaching a wider population. However, a common view amongst participants was that online methods require closer data monitoring for quality of responders, are not fully secure and less trustworthy compared to offline methods that enable participants to build relationships with health providers. Funding, staff time and experience, organisational support, and technical issues such as spam or phishing emails were highlighted as practical challenges facing online recruitment. All participants agreed that using a hybrid approach tailored to the population under study is paramount.
Conclusions
This study highlighted the importance of offering a flexible and multifaceted recruitment approach by integrating online with offline methods to support inclusivity and widening participation in mental health research. The findings will be used to develop considerations for researchers designing RCTs to improve recruitment in mental health research.
Celotno besedilo
Dostopno za:
DOBA, IZUM, KILJ, NUK, PILJ, PNG, SAZU, SIK, UILJ, UKNU, UL, UM, UPUK
Group interventions are interventions delivered to groups of people rather than to individuals and are used in healthcare for mental health recovery, behaviour change, peer support, self-management ...and/or health education. Evaluating group interventions in randomised controlled trials (RCTs) presents trialists with a set of practical problems, which are not present in RCTs of one-to-one interventions and which may not be immediately obvious.
Case-based approach summarising Sheffield trials unit's experience in the design and implementation of five group interventions. We reviewed participant recruitment and attrition, facilitator training and attrition, attendance at the group sessions, group size and fidelity aspects across five RCTs.
Median recruitment across the five trials was 3.2 (range 1.7-21.0) participants per site per month. Group intervention trials involve a delay in starting the intervention for some participants, until sufficient numbers are available to start a group. There was no evidence that the timing of consent, relative to randomisation, affected post-randomisation attrition which was a matter of concern for all trial teams. Group facilitator attrition was common in studies where facilitators were employed by the health system rather than the by the grant holder and led to the early closure of one trial; research sites responded by training 'back-up' and new facilitators. Trials specified that participants had to attend a median of 62.5% (range 16.7%-80%) of sessions, in order to receive a 'therapeutic dose'; a median of 76.7% (range 42.9%-97.8%) received a therapeutic dose. Across the five trials, 75.3% of all sessions went ahead without the pre-specified ideal group size. A variety of methods were used to assess the fidelity of group interventions at a group and individual level across the five trials.
This is the first paper to provide an empirical basis for planning group intervention trials. Investigators should expect delays/difficulties in recruiting groups of the optimal size, plan for both facilitator and participant attrition, and consider how group attendance and group size affects treatment fidelity.
ISRCTN17993825 registered on 11/10/2016, ISRCTN28645428 registered on 11/04/2012, ISRCTN61215213 registered on 11/05/2011, ISRCTN67209155 registered on 22/03/2012, ISRCTN19447796 registered on 20/03/2014.
Abstract Background Ensuring diversity in clinical trials can be a challenge, which may be exacerbated when recruiting vulnerable populations, such as participants with mental health illness. As ...recruitment continues to be the major cause of trial delays, researchers are turning to online recruitment strategies, e.g. social media, to reach a wider population and reduce recruitment time and costs. There is mixed evidence for the use of online recruitment strategies; therefore, the REcruitment in Mental health trials: broadening the ‘net’, opportunities for INclusivity through online methoDs (RE-MIND) study aimed to identify evidence and provide guidance for use of online strategies in recruitment to mental health trials, with a focus on whether online strategies can enhance inclusivity. This commentary, as part of the RE-MIND study, focusses on providing recommendations for recruitment strategy selection in future research with the aim to improve trial efficiency. A mixed-methods approach was employed involving three work packages: (I) an evidence review of a cohort of 97 recently published randomised controlled trials/feasibility or pilot studies in mental health to assess the impact of online versus offline recruitment; (II) a qualitative study investigating the experiences of n = 23 key stakeholders on use of an online recruitment approach in mental health clinical trials; (III) combining the results of WP1 and WP2 to produce recommendations on the use of an online recruitment strategy in mental health clinical trials. The findings from WP1 and 2 have been published elsewhere; this commentary represents the results of the third work package. Conclusion For external validity, clinical trial participants should reflect the populations that will ultimately receive the interventions being tested, if proven effective. To guide researchers on their options for inclusive recruitment strategies, we have developed a list of considerations and practical recommendations on how to maximise the use of online recruitment methods.
People need high-quality information to make decisions about research participation. Providing information in written format alone is conventional but may not be the most effective and acceptable ...approach. We developed a structure for the presentation of information using multimedia which included generic and trial-specific content. Our aim was to embed 'Studies Within A Trial' (SWATs) across multiple ongoing trials to test whether multimedia presentation of patient information led to better rates of recruitment.
Five trials included a SWAT and randomised their participants to receive a multimedia presentation alongside standard information, or standard written information alone. We collected data on trial recruitment, acceptance and retention and analysed the pooled results using random effects meta-analysis, with the primary outcome defined as the proportion of participants randomised following an invitation to take part.
Five SWATs provided data on the primary outcome of proportion of participants randomised. Multimedia alongside written information results in little or no difference in recruitment rates (pooled odds ratio = 0.96, 95% CI: 0.79 to 1.17, p-value = 0.671, I
= 0%). There was no effect on any other outcomes.
Multimedia alongside written information did not improve trial recruitment rates.
ISRCTN71952900, ISRCTN 06710391, ISRCTN 17160087, ISRCTN05926847, ISRCTN62869767.
Celotno besedilo
Dostopno za:
DOBA, IZUM, KILJ, NUK, PILJ, PNG, SAZU, SIK, UILJ, UKNU, UL, UM, UPUK
Existing guidelines recommend statisticians remain blinded to treatment allocation prior to the final analysis and that any interim analyses should be conducted by a separate team from the one ...undertaking the final analysis. However, there remains substantial variation in practice between UK Clinical Trials Units (CTUs) when it comes to blinding statisticians. Therefore, the aim of this study was to develop guidance to advise CTUs on a risk-proportionate approach to blinding statisticians within clinical trials.
This study employed a mixed methods approach involving three stages: (I) a quantitative study using a cohort of 200 studies (from a major UK funder published between 2016 and 2020) to assess the impact of blinding statisticians on the proportion of trials reporting a statistically significant finding for the primary outcome(s); (II) a qualitative study using focus groups to determine the perspectives of key stakeholders on the practice of blinding trial statisticians; and (III) combining the results of stages I and II, along with a stakeholder meeting, to develop guidance for UK CTUs.
After screening abstracts, 179 trials were included for review. The results of the primary analysis showed no evidence that involvement of an unblinded trial statistician was associated with the likelihood of statistically significant findings being reported, odds ratio (OR) 1.02 (95% confidence interval (CI) 0.49 to 2.13). Six focus groups were conducted, with 37 participants. The triangulation between stages I and II resulted in developing 40 provisional statements. These were rated independently by the stakeholder group prior to the meeting. Ten statements reached agreement with no agreement on 30 statements. At the meeting, various factors were identified that could influence the decision of blinding the statistician, including timing, study design, types of intervention and practicalities. Guidance including 21 recommendations/considerations was developed alongside a Risk Assessment Tool to provide CTUs with a framework for assessing the risks associated with blinding/not blinding statisticians and for identifying appropriate mitigation strategies.
This is the first study to develop a guidance document to enhance the understanding of blinding statisticians and to provide a framework for the decision-making process. The key finding was that the decision to blind statisticians should be based on the benefits and risks associated with a particular trial.
PDF
