Context and implementation of health interventions have received increasing attention over the past decade, in particular with respect to their influence on the effectiveness and reach of complex ...interventions. The underlying concepts are both considered partially mature, limiting their operationalization in research and practice. We conducted systematic literature searches and pragmatic utility (PU) concept analyses to provide a state-of-the-art assessment of the concepts of “context” and “implementation” in the health sciences to create a common understanding for their use within systematic reviews and HTA.
We performed two separate searches, one for context (EMBASE, MEDLINE) and the other for implementation (Google Scholar) to identify relevant models, theories and frameworks. 17 publications on context and 35 articles on implementation met our inclusion criteria. PU concept analysis comprises three guiding principles: selection of the literature, organization and structuring of the literature, and asking analytic questions of the literature. Both concepts were analyzed according to four features of conceptual maturity, i.e., consensual definitions, clear characteristics, fully described preconditions and outcomes, and delineated boundaries.
Context and implementation are highly intertwined, with both concepts influencing and interacting with each other. Context is defined as a set of characteristics and circumstances that surround the implementation effort. Implementation is conceptualized as a planned and deliberately initiated effort with the intention to put an intervention into practice. The concept of implementation presents largely consensual definitions and relatively well-defined boundaries, while distinguishing features, preconditions and outcomes are not yet fully articulated. In contrast, definitions of context vary widely, and boundaries with neighbouring concepts, such as setting and environment, are blurred; characteristics, preconditions and outcomes are ill-defined. Therefore, the maturity of both concepts should be further improved to facilitate operationalization in systematic reviews and HTAs.
Dem Kontext und der Implementierung von Gesundheitsinterventionen wurde während des vergangenen Jahrzehnts erhöhte Aufmerksamkeit zuteil - insbesondere was den Einfluss auf Effektivität und Reichweite komplexer Interventionen anbelangt. Dem aktuellen Forschungsstand nach können beide zugrunde liegenden Konzepte als noch nicht voll ausdifferenziert betrachtet werden, was sich negativ auf die Operationalisierung in Forschung und Praxis auswirkt. Um dem entgegenzuwirken, wurde eine systematische Literatursuche sowie eine „Pragmatic-Utility-“ (PU-) Konzeptanalyse durchgeführt, die die Verwendung beider Konzepte in den Gesundheitswissenschaften darstellen soll. Übergeordnetes Ziel ist es, durch den Austausch über die unterschiedlichen Konzeptualisierungen ein gemeinsames Verständnis beider Konzepte in systematischen Reviews und Health Technology Assessments (HTAs) zu schaffen. Es wurden jeweils zwei separate Suchen für Kontext (EMBASE, MEDLINE) und Implementierung (Google Scholar) durchgeführt, um relevante Modelle, Theorien und Frameworks zu identifizieren. Dabei stimmten 17 Publikationen zu Kontext und 35 zu Implementierung mit unseren Einschlusskriterien überein. Die Literatur wurde den drei Grundprinzipien der PU folgend (1) ausgewählt, (2) organisiert und strukturiert sowie (3) analytischen Fragen unterworfen. Zudem wurden beide Konzepte bezüglich vier vordefinierter Kriterien von konzeptioneller Reife analysiert (übereinstimmende Definitionen, klare Charakteristika, vollständig umschriebene Vorbedingungen und Ergebnisse sowie eindeutige Grenzen). Die Analyse zeigt, dass Kontext und Implementierung zwei stark miteinander verflochtene Konzepte sind, die interagieren. Kontext bezieht sich dabei auf die einen Implementierungsversuch umgebenden Charakteristika und Umstände. Implementierung hingegen wird als geplante und bewusst initiierte Anstrengung verstanden, die darauf abzielt, eine Intervention in die Praxis zu überführen. Das Konzept der Implementierung zeichnet sich durch größtenteils übereinstimmende Definitionen sowie relativ klar definierte Grenzen gegenüber anderen Konzepten aus, wohingegen abgrenzende Eigenschaften, Vorbedingungen sowie dem Konzept folgende Ergebnisse noch nicht ausreichend artikuliert sind. Im Gegensatz dazu variieren die Definitionen von Kontext stark, wodurch eine Abgrenzung von benachbarten Konzepten wie Setting und Umwelt erschwert wird; auch sind die Charakteristika, Vorbedingungen sowie Ergebnisse des Konzepts unklar. Beide Konzepte sollten dahingehend weiterentwickelt werden, dass sie eine einheitliche Verwendung in systematischen Reviews, HTAs sowie in der Praxis erleichtern.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UL, UM, UPCLJ, UPUK
Abstract Objective Cognitive behaviour therapy (CBT) is an effective treatment for chronic fatigue syndrome (CFS). Main aim was to determine whether treatment effects were maintained up to 10 years ...after treatment. Methods Participants ( n = 583) of previously published studies on the effects of CBT for CFS were contacted for a long-term follow-up assessment. They completed questionnaires on main outcomes fatigue severity (CIS) and physical functioning (SF-36). The course of these outcomes since post-treatment assessment was examined using mixed model analyses. Results Between 21 and 125 months after finishing CBT, 511 persons (response rate 88%) completed a follow-up assessment. At follow-up, mean fatigue severity was significantly increased to 37.60 (SD = 12.76) and mean physical functioning significantly decreased to 73.16 (SD = 23.56) compared to post-treatment assessment. At follow-up still 37% of the participants had fatigue scores in the normal range and 70% were not impaired in physical functioning. Conclusion Positive effects of CBT for CFS on fatigue and physical functioning were partly sustained at long-term follow-up. However, a subgroup of patients once again reported severe fatigue, and compromised physical functioning. Further research should elucidate the reasons for this deterioration to facilitate the development of treatment strategies for relapse prevention.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UL, UM, UPCLJ, UPUK
Objectives: The INTEGRATE-HTA project provided methodology to evaluate complex technologies. This study provides guidance on how to retrieve and critically appraise available evidence on moderators ...and predictors of treatment effects and on patient preferences for treatment outcomes as a source of complexity.
Methods: Search filters for PubMed were developed by hand-searching a large volume of articles reporting on relevant aspects. Search terms were retrieved from selected papers and algorithmically combined to find the optimal combination of search terms. For the development of the appraisal checklists literature was searched in PubMed and Google Scholar together with citation chasing. For the CHecklist for the Appraisal of Moderators and Predictors (CHAMP) a Delphi procedure was used to value a set of eligible appraisal criteria retrieved from the literature.
Results: Search filters were developed optimized for different accuracy measures. The final version of CHAMP consists of a seventeen questions covering the design, analysis, results and transferability of results of moderator and predictor analysis. The final checklist for appraisal of literature on patient preferences for treatment outcomes consist of six questions meant to help the user to identify relevant quality issues together with a guidance toward existing tools concerning the appraisal of specific preference elicitation methods.
Conclusions: Incorporating knowledge on subgroups for whom a specific treatment will produce more benefit holds the promise of better targeting and, ultimately, enhancing overall effectiveness and efficiency of healthcare technology. Finally, incorporating information on preferences for treatment outcomes will foster health technology assessment that addresses outcomes that are important to patients.
Background:
Stakeholders are people with an interest in a topic. Internationally, stakeholder involvement in palliative care research and health technology assessment requires development. ...Stakeholder involvement adds value throughout research (from prioritising topics to disseminating findings). Philosophies and understandings about the best ways to involve stakeholders in research differ internationally. Stakeholder involvement took place in seven countries (England, Germany, Italy, Lithuania, the Netherlands, Norway and Poland). Findings informed a project that developed concepts and methods for health technology assessment and applied these to evaluate models of palliative care service delivery.
Aims:
To report on stakeholder involvement in the INTEGRATE-HTA project and how issues identified informed project development.
Design:
Using stakeholder consultation or a qualitative research design, as appropriate locally, stakeholders in seven countries acted as ‘advisors’ to aid researchers’ decision making. Thematic analysis was used to identify key issues across countries.
Setting/participants:
A total of 132 stakeholders (82 professionals and 50 ‘lay’ people) aged ⩾18 participated in individual face-to-face or telephone interviews, consultation meetings or focus groups.
Results:
Different stakeholder involvement methods were used successfully to identify key issues in palliative care. A total of 23 issues common to three or more countries informed decisions about the intervention and comparator of interest, sub questions and specific assessments within the health technology assessment.
Conclusion:
Stakeholders, including patients and families undergoing palliative care, can inform project decision making using various involvement methods according to the local context. Researchers should consider local understandings about stakeholder involvement as views of appropriate and feasible methods vary. Methods for stakeholder involvement, especially consultation, need further development.
Full text
Available for:
NUK, OILJ, SAZU, UKNU, UL, UM, UPUK
Cognitive behavioural therapy (CBT) is an evidence-based treatment for chronic fatigue syndrome (CFS). Stepped care for CFS, consisting of a minimal intervention followed by face-to-face CBT, was ...found efficacious when tested in a CFS specialist centre. Stepped care implemented in a community-based mental health centre (MHC) has not yet been evaluated.
(1) To test the effectiveness of stepped care for CFS implemented in a MHC at post-treatment and at long-term follow-up; and (2) compare post-treatment outcomes of implemented stepped care with treatment outcomes of a CFS specialist centre.
An uncontrolled study was used to test effectiveness of stepped care implemented in a MHC (n = 123). The outcomes of implemented care were compared with the outcomes of specialist care reported in previous studies (n = 583). Data on outcomes from implemented stepped care were gathered at post-treatment and at long-term follow-up. Mixed models were used as method of analysis.
Fatigue decreased and physical functioning increased significantly following implemented stepped care (both p < .001). The follow-up was completed by 94 patients (78%) within 1-6 years after treatment. Treatment effects were sustained to follow-up. Patients in the MHC showed less improvement directly following stepped care compared with patients in a CFS specialist centre (p < .01).
Implemented stepped care for CFS is effective with sustained treatment gains at long-term follow-up. There is room for improvement when compared with outcomes of a CFS specialist centre. Some suggestions are made on how to improve stepped care.
Background
Treatment effects differ across patients. To guide selection of treatments for patients, it is essential to acknowledge these differences and identify moderators or predictors. Our aim was ...to generate optimal search strategies (commonly known as filters) for PubMed to retrieve papers identifying moderators and predictors of treatment effects.
Methods
Six journals were hand‐searched for articles on moderators or predictors. Selected articles were randomly allocated to a development and validation set. Search terms were extracted from the development set and tested for their performance. Search filters were created from combinations of these terms and tested in the validation set.
Results
Of 4407 articles, 198 were considered to be relevant. The most sensitive filter in the development set ‘(“Epidemiologic Methods” MeSH OR assign* OR control*tiab OR trial*tiab) AND therapy*sh’ yielded in the validation set a sensitivity of 89% 88%–90% and a specificity of 80% 79%–82%.
Conclusions
The search filters created in this study can help to efficiently retrieve evidence on moderators and predictors of treatment effect. Testing of the filters in multiple domains should reveal robustness across disciplines. These filters can facilitate the retrieval of evidence on moderators and predictors of treatment effects, helping the implementation of stratified or personalised health care.
Full text
Available for:
BFBNIB, DOBA, FZAB, GIS, IJS, IZUM, KILJ, NLZOH, NUK, OILJ, PILJ, PNG, SAZU, SBCE, SBMB, SIK, UILJ, UKNU, UL, UM, UPUK
INTRODUCTION:
Priority setting in health care has been long recognized as an intrinsically complex and value-laden process. Yet, Health Technology Assessment (HTA) agencies presently employ value ...assessment frameworks that are ill-fitted to capture the range and diversity of stakeholder values, and thereby risk to compromise the legitimacy of their recommendations. We propose ‘evidence-informed deliberative processes’ as an alternative framework with the aim to enhance this legitimacy.
METHODS:
The framework is based on an integration of two increasingly popular and complementary frameworks for priority setting: multi-criteria decision analysis (MCDA) and accountability for reasonableness (A4R), Evidence-informed deliberative processes are, on the one hand, based on early, continued stakeholder deliberation to learn about the importance of relevant social values. On the other hand, they are based on rational decision-making – through evidence-informed evaluation of the identified values.
RESULTS:
The framework has important implications for how HTA agencies should ideally organize their processes. Firstly, HTA agencies should take the responsibility to organize stakeholder involvement. Second, agencies are advised to integrate their assessment and appraisal phase, allowing for the timely collection of evidence on values that are considered relevant. Third, HTA agencies should subject their specification of decision-making criteria to public scrutiny. Fourth, agencies are advised to use a checklist of potentially relevant criteria, and to provide argumentation how each criterion affected the recommendation. Fifth, HTA agencies must publish their argumentation and install options for appeal.
CONCLUSIONS:
Adopting ‘evidence-informed deliberative processes’ as a value assessment framework could be an important step forward for HTA agencies to optimize the legitimacy of their priority setting decisions. Agencies can incorporate elements according to their needs and affordances.
Abstract Objective A subgroup of patients with Chronic Fatigue Syndrome (CFS) has cognitive impairments, reflected by deviant neuropsychological test performance. However, abnormal test scores can ...also be caused by suboptimal effort. We hypothesized that worse neuropsychological test performance and underperformance were related to each other and to a smaller reduction in fatigue, functional impairments, physical limitations and higher dropout rates following cognitive behavior therapy (CBT) for CFS. Methods Data were drawn from a previous trial, in which CFS patients were randomized to two conditions; 1) guided self-instruction and additional CBT (n = 84) or 2) waiting period followed by regular CBT for CFS (n = 85). Underperformance was assessed using the Amsterdam Short Term Memory Test (< 84). To test neuropsychological test performance, the Symbol Digit Modalities Task, a simple reaction time task and a choice reaction time task were used. Interaction effects were determined between underperformance and neuropsychological test performance on therapy outcomes. Results Underperformance was associated to worse neuropsychological test performance, but there were no significant interaction effects of these two factors by therapy on fatigue severity, functional impairments and physical limitations, but there was a significant main effect of underperformance on functional impairments, physical limitations and dropout rates. Conclusion Underperformance or neuropsychological test performance was not related to the change in fatigue, functional impairments, and physical limitations following CBT for CFS. However, underperforming patients did drop out more often. Therapists should pay attention to beliefs and behavioral or environmental factors that might maintain underperformance and increase the risk of dropout.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UL, UM, UPCLJ, UPUK
Objectives: Internationally, funders require stakeholder involvement throughout health technology assessment (HTA). We report successes, challenges, and lessons learned from extensive stakeholder ...involvement throughout a palliative care case study that demonstrates new concepts and methods for HTA.
Methods: A 5-step “INTEGRATE-HTA Model” developed within the INTEGRATE-HTA project guided the case study. Using convenience or purposive sampling or directly / indirectly identifying and approaching individuals / groups, stakeholders participated in qualitative research or consultation meetings. During scoping, 132 stakeholders, aged ≥ 18 years in seven countries (England, Italy, Germany, The Netherlands, Norway, Lithuania, and Poland), highlighted key issues in palliative care that assisted identification of the intervention and comparator. Subsequently stakeholders in four countries participated in face–face, telephone and / or video Skype meetings to inform evidence collection and / or review assessment results. An applicability assessment to identify contextual and implementation barriers and enablers for the case study findings involved twelve professionals in the three countries. Finally, thirteen stakeholders participated in a mock decision-making meeting in England.
Results: Views about the best methods of stakeholder involvement vary internationally. Stakeholders make valuable contributions in all stages of HTA; assisting decision making about interventions, comparators, research questions; providing evidence and insights into findings, gap analyses and applicability assessments. Key challenges exist regarding inclusivity, time, and resource use.
Conclusion: Stakeholder involvement is feasible and worthwhile throughout HTA, sometimes providing unique insights. Various methods can be used to include stakeholders, although challenges exist. Recognition of stakeholder expertise and further guidance about stakeholder consultation methods is needed.
PDF
