Designing implementation research can be a complex and daunting task, especially for applied health researchers who have not received specialist training in implementation science. We developed the ...Implementation Science Research Development (ImpRes) tool and supplementary guide to address this challenge and provide researchers with a systematic approach to designing implementation research.
A multi-method and multi-stage approach was employed. An international, multidisciplinary expert panel engaged in an iterative brainstorming and consensus-building process to generate core domains of the ImpRes tool, representing core implementation science principles and concepts that researchers should consider when designing implementation research. Simultaneously, an iterative process of reviewing the literature and expert input informed the development and content of the tool. Once consensus had been reached, specialist expert input was sought on involving and engaging patients/service users; and economic evaluation. ImpRes was then applied to 15 implementation and improvement science projects across the National Institute of Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care (CLAHRC) South London, a research organisation in London, UK. Researchers who applied the ImpRes tool completed an 11-item questionnaire evaluating its structure, content and usefulness.
Consensus was reached on ten implementation science domains to be considered when designing implementation research. These include implementation theories, frameworks and models, determinants of implementation, implementation strategies, implementation outcomes and unintended consequences. Researchers who used the ImpRes tool found it useful for identifying project areas where implementation science is lacking (median 5/5, IQR 4-5) and for improving the quality of implementation research (median 4/5, IQR 4-5) and agreed that it contained the key components that should be considered when designing implementation research (median 4/5, IQR 4-4). Qualitative feedback from researchers who applied the ImpRes tool indicated that a supplementary guide was needed to facilitate use of the tool.
We have developed a feasible and acceptable tool, and supplementary guide, to facilitate consideration and incorporation of core principles and concepts of implementation science in applied health implementation research. Future research is needed to establish whether application of the tool and guide has an effect on the quality of implementation research.
Background
Telemental health care has been rapidly adopted for maintaining services during the COVID-19 pandemic, and a substantial interest is now being devoted in its future role. Service planning ...and policy making for recovery from the pandemic and beyond should draw on both COVID-19 experiences and the substantial research evidence accumulated before this pandemic.
Objective
We aim to conduct an umbrella review of systematic reviews available on the literature and evidence-based guidance on telemental health, including both qualitative and quantitative literature.
Methods
Three databases were searched between January 2010 and August 2020 for systematic reviews meeting the predefined criteria. The retrieved reviews were independently screened, and those meeting the inclusion criteria were synthesized and assessed for risk of bias. Narrative synthesis was used to report these findings.
Results
In total, 19 systematic reviews met the inclusion criteria. A total of 15 reviews examined clinical effectiveness, 8 reported on the aspects of telemental health implementation, 10 reported on acceptability to service users and clinicians, 2 reported on cost-effectiveness, and 1 reported on guidance. Most reviews were assessed to be of low quality. The findings suggested that video-based communication could be as effective and acceptable as face-to-face formats, at least in the short term. Evidence on the extent of digital exclusion and how it can be overcome and that on some significant contexts, such as children and young people’s services and inpatient settings, was found to be lacking.
Conclusions
This umbrella review suggests that telemental health has the potential to be an effective and acceptable form of service delivery. However, we found limited evidence on the impact of its large-scale implementation across catchment areas. Combining previous evidence and COVID-19 experiences may allow realistic planning for the future implementation of telemental health.
Early in 2020, mental health services had to rapidly shift from face-to-face models of care to delivering the majority of treatments remotely (by video or phone call or occasionally messaging) due to ...the COVID-19 pandemic. This resulted in several challenges for staff and patients, but also in benefits such as convenience or increased access for people with impaired mobility or in rural areas. There is a need to understand the extent and impacts of telemental health implementation, and barriers and facilitators to its effective and acceptable use. This is relevant both to future emergency adoption of telemental health and to debates on its future use in routine mental health care.
To investigate the adoption and impacts of telemental health approaches during the COVID-19 pandemic, and facilitators and barriers to optimal implementation.
Four databases (PubMed, PsycINFO, CINAHL, and Web of Science) were searched for primary research relating to remote working, mental health care, and the COVID-19 pandemic. Preprint servers were also searched. Results of studies were synthesized using framework synthesis.
A total of 77 papers met our inclusion criteria. In most studies, the majority of contacts could be transferred to a remote form during the pandemic, and good acceptability to service users and clinicians tended to be reported, at least where the alternative to remote contacts was interrupting care. However, a range of impediments to dealing optimal care by this means were also identified.
Implementation of telemental health allowed some continuing support to the majority of service users during the COVID-19 pandemic and has value in an emergency situation. However, not all service users can be reached by this means, and better evidence is now needed on long-term impacts on therapeutic relationships and quality of care, and on impacts on groups at risk of digital exclusion and how to mitigate these.
PROSPERO International prospective register of systematic reviews CRD42021211025; https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42021211025.
IntroductionThe need for quantitative criteria to appraise the quality of implementation research has recently been highlighted to improve methodological rigour. The Implementation Science Research ...development (ImpRes) tool and supplementary guide provide methodological guidance and recommendations on how to design high-quality implementation research. This protocol reports on the development of the Implementation Science Research Project Appraisal Criteria (ImpResPAC) tool, a quantitative appraisal tool, developed based on the structure and content of the ImpRes tool and supplementary guide, to evaluate the conceptual and methodological quality of implementation research.Methods and analysisThis study employs a three-stage sequential mixed-methods design. During stage 1, the research team will map core domains of the ImpRes tool, guidance and recommendations contained in the supplementary guide and within the literature, to ImpResPAC. In stage 2, an international multidisciplinary expert group, recruited through purposive sampling, will inform the refinement of ImpResPAC, including content, scoring system and user instructions. In stage 3, an extensive psychometric evaluation of ImpResPAC, that was created in stage 1 and refined in stage 2, will be conducted. The scaling assumptions (inter-item and item-total correlations), reliability (internal consistency, inter-rater) and validity (construct and convergent validity) will be investigated by applying ImpResPAC to 50 protocols published in Implementation Science. We envisage developing ImpResPAC in this way will provide implementation research stakeholders, primarily grant reviewers and educators, a comprehensive, transparent and fair appraisal of the conceptual and methodological quality of implementation research, increasing the likelihood of funding research that will generate knowledge and contribute to the advancement of the field.Ethics and disseminationThis study will involve human participants. This study has been registered and minimal risk ethical clearance granted by The Research Ethics Office, King’s College London (reference number MRA-20/21-20807). Participants will receive written information on the study via email and will provide e-consent if they wish to participate. We will use traditional academic modalities of dissemination (eg, conferences and publications).
IntroductionBeing on a waiting list for elective (planned) cardiac surgery can be physically and psychologically challenging for patients. Research suggests that stress associated with waiting for ...surgery is dependent on different individual and contextual factors. However, most data on patients’ experiences of waiting for surgery and preferences for waiting list management derives from non-cardiac clinical populations. The aim of the current study is to explore patients’ experiences of being on a waiting list for elective cardiac surgery, and their views on how the waiting experience could be improved in the future. This work will inform the patient management strategy during the waiting period for surgery across the four major hospitals in London directly involved in this study, and potentially beyond by transferring learning to other services.Methods and analysisThis is a mixed-methods study that will collect quantitative and qualitative data using a cross-sectional online survey. Patients who are on waiting lists for elective surgery across four major cardiac surgery departments in London hospitals, and are at least 18 years old, will be invited by their healthcare team via text message or letter to complete the survey. The target sample size of non-randomly selected participants will be 268. Bivariable and multivariable regression models will be used to assess associations between survey items measuring the impact of the cardiac condition on specific life domains (eg, daily activities, social and family relationships, hobbies, sexual life), anxiety and depression symptoms as measured by the Patient Health Questionnaire-4 and survey items evaluating experiences of health services. Data on experience and preferences for improvements to the waiting experience will be analysed with qualitative content analysis using an inductive approach.Ethics and disseminationThis study was reviewed and granted ethical approval by the East of England—East Cambridge Research Ethics Committee. Findings from this study will be disseminated through peer-reviewed journals, a research website and social media and with an online event engaging patients, members of the public, healthcare professionals and other relevant stakeholders.Trial registration numbNCT05996640
Mental health information systems are, in general, inadequate and unreliable in India. We have developed key mental health indicators for measuring mental health service coverage in primary care. ...This study aims to evaluate the use of these new indicators in seven health care facilities in Sehore District of Madhya Pradesh in India.
The study employed a mixed methods approach. We conducted: a qualitative study (n = 26) with health workers, Health Management Information Systems (HMIS) staff, project coordinators and supervisors; a review of case records (n = 61 at time 1 and n = 74 at time 2); and a structured questionnaire (n = 26) with health workers. The performance, user-friendliness, appropriateness, perceived utility and sustainability of the use of new mental health indicators was assessed.
High levels of completion, and correctness of completion, of the new mental health indicators were found for the case records. The simplicity of the forms, as well as technical support from the project team, contributed to acceptability and feasibility of implementation. Perceived sustainability of the new forms was, however, affected by the overstretched primary care staff. Further work is needed to support the integration of mental health with routine HMIS.
This study demonstrated that the implementation of key mental health service delivery indicators in Sehore District primary care facilities was feasible. Technical assistance was imperative in maintaining the performance of the indicators over the two studied time points. The integration of mental health indicators in routine health information systems, and political buy-in, are needed to monitor and sustain community mental health programmes in India.
Despite an increasing number of training opportunities in implementation science becoming available, the demand for training amongst researchers and practitioners is unmet. To address this training ...shortfall, we developed the King's College London 'Implementation Science Masterclass' (ISM), an innovative 2-day programme (and currently the largest of its kind in Europe), developed and delivered by an international faculty of implementation experts.
This paper describes the ISM and provides delegates' quantitative and qualitative evaluations (gathered through a survey at the end of the ISM) and faculty reflections over the period it has been running (2014-2019).
Across the 6-year evaluation, a total of 501 delegates have attended the ISM, with numbers increasing yearly from 40 (in 2014) to 147 (in 2019). Delegates represent a diversity of backgrounds and 29 countries from across the world. The overall response rate for the delegate survey was 64.5% (323/501). Annually, the ISM has been rated 'highly' in terms of delegates' overall impression (92%), clear and relevant learning objectives (90% and 94%, respectively), the course duration (85%), pace (86%) and academic level 87%), and the support provided on the day (92%). Seventy-one percent of delegates reported the ISM would have an impact on how they approached their future work. Qualitative feedback revealed key strengths include the opportunities to meet with an international and diverse pool of experts and individuals working in the field, the interactive nature of the workshops and training sessions, and the breadth of topics and contexts covered.
Yearly, the UK ISM has grown, both in size and in its international reach. Rated consistently favourably by delegates, the ISM helps to tackle current training demands from all those interested in learning and building their skills in implementation science. Evaluation of the ISM will continue to be an annual iterative process, reflective of changes in the evidence base and delegates changing needs as the field evolves.
The uptake of clinical practice guidelines (CPGs) is inconsistent, despite their potential to improve the quality of health care and patient outcomes. Some guideline producers have addressed this ...problem by developing tools to encourage faster adoption of new guidelines. This review focuses on the effectiveness of tools developed and disseminated by guideline producers to improve the uptake of their CPGs.
To evaluate the effectiveness of implementation tools developed and disseminated by guideline producers, which accompany or follow the publication of a CPG, to promote uptake. A secondary objective is to determine which approaches to guideline implementation are most effective.
We searched the Cochrane Effective Practice and Organisation of Care (EPOC) Group Specialised Register, Cochrane Central Register of Controlled Trials (CENTRAL); NHS Economic Evaluation Database, HTA Database; MEDLINE and MEDLINE In-Process and other non-indexed citations; Embase; PsycINFO; CINAHL; Dissertations and Theses, ProQuest; Index to Theses; Science Citation Index Expanded, ISI Web of Knowledge; Conference Proceedings Citation Index - Science, ISI Web of Knowledge; Health Management Information Consortium (HMIC), and NHS Evidence up to February 2016. We also searched trials registers, reference lists of included studies and relevant websites.
We included randomised controlled trials (RCTs) and cluster-RCTs, controlled before-and-after studies (CBAs) and interrupted time series (ITS) studies evaluating the effects of guideline implementation tools developed by recognised guideline producers to improve the uptake of their own guidelines. The guideline could target any clinical area.
Two review authors independently extracted data and assessed the risk of bias of each included study using the Cochrane 'Risk of bias' criteria. We graded our confidence in the evidence using the approach recommended by the GRADE working group. The clinical conditions targeted and the implementation tools used were too heterogenous to combine data for meta-analysis. We report the median absolute risk difference (ARD) and interquartile range (IQR) for the main outcome of adherence to guidelines.
We included four cluster-RCTs that were conducted in the Netherlands, France, the USA and Canada. These studies evaluated the effects of tools developed by national guideline producers to implement their CPGs. The implementation tools evaluated targeted healthcare professionals; none targeted healthcare organisations or patients.One study used two short educational workshops tailored to barriers. In three studies the intervention consisted of the provision of paper-based educational materials, order forms or reminders, or both. The clinical condition, type of healthcare professional, and behaviour targeted by the CPG varied across studies.Two of the four included studies reported data on healthcare professionals' adherence to guidelines. A guideline tool developed by the producers of a guideline probably leads to increased adherence to the guidelines; median ARD (IQR) was 0.135 (0.115 and 0.159 for the two studies respectively) at an average four-week follow-up (moderate certainty evidence), which indicates a median 13.5% greater adherence to guidelines in the intervention group. Providing healthcare professionals with a tool to improve implementation of a guideline may lead to little or no difference in costs to the health service.
Implementation tools developed by recognised guideline producers probably lead to improved healthcare professionals' adherence to guidelines in the management of non-specific low back pain and ordering thyroid-function tests. There are limited data on the relative costs of implementing these interventions.There are no studies evaluating the effectiveness of interventions targeting the organisation of care (e.g. benchmarking tools, costing templates, etc.), or for mass media interventions. We could not draw any conclusions about our second objective, the comparative effectiveness of implementation tools, due to the small number of studies, the heterogeneity between interventions, and the clinical conditions that were targeted.
Highlights • Business partners participated in the Responsibility Deal for reputational reasons. • Partners frequently chose pledges reflecting work they were already doing. • The Responsibility Deal ...is likely to have limited ‘added value’. • Government needs to set out a clear vision for the RD. • These findings have implications for the development of other voluntary agreements.
goulding l., furze g. & birks y. (2010) Randomized controlled trials of interventions to change maladaptive illness beliefs in people with coronary heart disease: systematic review. Journal of ...Advanced Nursing 66(5), 946–961.
Title. Randomized controlled trials of interventions to change maladaptive illness beliefs in people with coronary heart disease: systematic review.
Aim. This paper is a report of a systematic review of randomized controlled trials of interventions to change maladaptive illness beliefs in people with coronary heart disease, and was conducted to determine whether such interventions were effective in changing maladaptive beliefs, and to assess any consequent change in coping and outcome.
Background. An increasing body of evidence suggests that faulty beliefs can lead to maladaptive behaviours and, in turn, to poor outcomes. However, the effectiveness of interventions to change such faulty illness beliefs in people with coronary heart disease is unknown.
Data sources. Multiple data bases were searched using a systematic search strategy. In addition, reference lists of included papers were checked and key authors in the field contacted.
Review methods. The systematic review included randomized controlled trials with adults of any age with a diagnosis of coronary heart disease and an intervention aimed at changing cardiac beliefs. The primary outcome measured was change in beliefs about coronary heart disease.
Results. Thirteen trials met the inclusion criteria. Owing to the heterogeneity of these studies, quantitative synthesis was not practicable. Descriptive synthesis of the results suggested that cognitive behavioural and counselling/education interventions can be effective in changing beliefs. The effects of changing beliefs on behavioural, functional and psychological outcomes remain unclear.
Conclusion. While some interventions may be effective in changing beliefs in people with coronary heart disease, the effect of these changes on outcome is not clear. Further high quality research is required before firmer guidance can be given to clinicians on the most effective method to dispel cardiac misconceptions.
PDF
