•Economic considerations influence implementation of evidence-based mental health practices.•Decision makers and other stakeholders need cost information to make implementation decisions.•Economic ...evaluation of implementation strategies differs from traditional economic evaluation.•Economic evaluation of implementation is critical for efficient use of organizational resources.
Implementation researchers have made notable progress in developing and testing implementation strategies (i.e., highly-specified methods used to help providers improve uptake of mental health evidence-based practices: EBPs). Yet, implementation strategies are not widely applied in healthcare organizations to improve delivery of EBPs. Economic considerations are a key factor influencing the use of implementation strategies to deliver and sustain mental health evidence-based practices, in part because many health care leaders are reluctant to invest in ongoing implementation strategy support without knowing the return-on-investment. Comparative economic evaluation of implementation strategies provides critical information for payers, policymakers, and providers to make informed decisions if specific strategies are an efficient use of scarce organizational resources. Currently, few implementation studies include implementation cost data and even fewer conduct comparative economic analyses of implementation strategies. This summary will introduce clinicians, researchers and other health professionals to the economic evaluation in implementation science. We provide an overview of different economic evaluation methods, discuss differences between economic evaluation in health services and implementation science. We also highlight approaches and frameworks to guide economic evaluation of implementation, provide an example for a cognitive-behavioral therapy program and discuss recommendations.
Guidance from economic evaluations on which implementation strategies represent the best return on investment will be critical to advancing the Triple Aim of health care: improving patient care and ...population health while minimizing per-capita cost. The results of traditional (quantitative) economic evaluations are limited by a remaining "qualitative residual" of contextual information and stakeholders perspectives, which cannot be captured by monetary values alone and is particularly prevalent in implementation science research. The emergence of qualitative methods for economic evaluation offers a promising solution.
To maximize the contributions of economic evaluations to implementation science, we recommend that researchers embrace a mixed-methods research agenda that merges traditional quantitative approaches with innovative, contextually grounded qualitative methods. Such studies are exceedingly rare at present. To assist implementation scientists in making use of mixed methods in this research context, we present an adapted taxonomy of mixed-method studies relevant to economic evaluation. We then illustrate the application of mixed methods in a recently completed cost-effectiveness evaluation, making use of an adapted version of reporting standards for economic evaluations.
By incorporating qualitative methods, implementation researchers can enrich their economic evaluations with detailed, context-specific information that tells the full story of the costs and impacts of implementation. We end by providing suggestions for building a research agenda in mixed-method economic evaluation, along with more resources and training to support investigators who wish to answer our call to action.
Innovative approaches are needed to maximise the uptake and sustainment of evidence-based practices in a variety of health service contexts. This protocol describes a study that will seek to ...characterise the potential of one such approach, user-centred design (UCD), which is an emerging field that seeks to ground the design of an innovation in information about the people who will ultimately use that innovation. The use of UCD to enhance strategies for implementation of health services, although promising, requires a multidisciplinary perspective based on a firm understanding of how experts from each discipline perceives the interrelatedness and suitability of these strategies.
This online study will use a combination of purposive and snowball sampling to recruit a sample of implementation experts (n = 30) and UCD experts (n = 30). These participants will each complete a concept mapping task, which is a mixed-method conceptualisation technique that will allow for identification of distinct clusters of implementation and/or UCD strategies. The research team has selected a set of implementation strategies and UCD strategies that each participant will sort and rate on dimensions of importance and feasibility. Data analyses will focus on describing the sample, identifying related clusters of strategies, and examining the convergences, divergences, and potential for collaboration between implementation science and UCD.
By leading to a better understanding of the overlap between implementation science and UCD, grounded within established theoretical frameworks, this study holds promise for improving the impact and sustainability of evidence-based health services in community settings.
Innovative approaches are needed to maximize fit between the characteristics of evidence-based practices (EBPs), implementation strategies that support EBP use, and contexts in which EBPs are ...implemented. Standard approaches to implementation offer few ways to address such issues of fit. We characterized the potential for collaboration with experts from a relevant complementary approach, user-centered design (UCD), to increase successful implementation.
Using purposive and snowball sampling, we recruited 56 experts in implementation (
= 34) or UCD (
= 22). Participants had 5+ years of professional experience (
= 10.31), worked across many settings (e.g., healthcare, education, human services), and were mostly female (59%) and white (73%). Each participant completed a web-based concept mapping structured conceptualization task. They sorted strategies from established compilations for implementation (36 strategies) and UCD (30 strategies) into distinct clusters, then rated the importance and feasibility of each strategy.
We used multidimensional scaling techniques to examine patterns in the sorting of strategies. Based on conceptual clarity and fit with established implementation frameworks, we selected a final set of 10 clusters (i.e., groups of strategies), with five implementation-only clusters, two UCD-only clusters, and three trans-discipline clusters. The highest-priority activities (i.e., above-average importance and feasibility) were the trans-discipline clusters plus facilitate change and monitor change. Implementation and UCD experts sorted strategies into similar clusters, but each gave higher importance and feasibility ratings to strategies/clusters from their own discipline.
In this concept mapping study, experts in implementation and UCD had perspectives that both converged (e.g., trans-discipline clusters, which were all rated as high-priority) and diverged (e.g., in importance/feasibility ratings). The results provide a shared understanding of the alignment between implementation science and UCD, which can increase the impact and sustainability of EBP implementation efforts. Implications for improved collaboration among implementation and UCD experts are discussed.
Objective: Researchers have identified several family-based treatments that hold considerable promise in reducing serious juvenile offending; however, these treatments remain underutilized by youth ...service systems. In the present study, we used meta-analysis to summarize the findings of research on family-based treatments for serious juvenile offenders. Method: We conducted a multilevel meta-analysis that modeled dependencies between multiple effect sizes from the same study. The meta-analysis synthesized 324 effect sizes from 28 studies that met inclusion criteria. Potential moderators (e.g., characteristics of samples, treatments, methods, and measures) were entered as fixed effects in the meta-analytic model. Results: Across studies, family-based treatments produced modest, yet long-lasting, treatment effects (mean d = 0.25 for antisocial behavior, 0.24 overall) relative to comparison conditions. Furthermore, certain characteristics moderated the magnitude of treatment effects; for example, measures of substance use showed the largest effects and measures of peer relationships showed the smallest effects. Conclusions: Policymakers, administrators, and treatment providers may find it useful to consider the effects of family-based treatments for serious juvenile offenders in their selection of treatments for this population. In addition, investigators who seek to develop and study such treatments may wish to consider the current findings in their future research efforts.
What is the public health significance of this article?
Family-based treatments have the potential to reduce the social and financial consequences of juvenile crime while improving public safety. The results of the present meta-analysis suggest that family-based treatments for serious juvenile offenders produce modest but long-lasting effects when compared to usual community services.
Sexual and gender minority (SGM) individuals experience high rates of homelessness and criminal justice system involvement, underscoring the need for supportive housing services. To explore the ...service needs of this population, we interviewed providers (n = 11) and clients (n = 10) from eight supportive housing organizations working with SGM populations in Los Angeles County, California, USA. We used the Consolidated Framework for Implementation Research to synthesize interview responses into themes (by domain and cross-cutting). Takeaways included the need for investment in systems of care for vulnerable SGM populations; the particular marginalization of Trans individuals and providers that serve them; the roles of supportive housing staff, residents, and leadership in cultivating an affirming environment; the prevalence of discrimination and stigma within supportive housing programs and broader society; and the complex interrelationships among SGM identity, homelessness, and criminal justice system involvement. These findings have important implications for supportive housing services and related policy.
Youth antisocial behavior exacts a tremendous toll on society and often persists into adulthood. Although researchers have identified a number of psychosocial interventions that prevent or reduce ...youth antisocial behavior in the short term, evidence of long-term intervention benefits has only recently become available. In addition, research on such interventions spans two substantial but largely separate bodies of literature: prevention and therapy. The present study used meta-analysis to integrate research on the long-term effects of preventive and therapeutic interventions for youth antisocial behavior and examined potential moderators of these effects. Results from 66 intervention trials (i.e., 34 prevention, 32 therapy) indicated that a broad range of youth psychosocial interventions demonstrated modest effects on antisocial behavior (mean d=0.31, 95% confidence interval=0.23–0.39) for at least one year beyond the end of interventions relative to control conditions. Among other findings, moderator analyses revealed that inclusion of a peer group intervention component was associated with reduced intervention effects for samples consisting predominantly of boys or older youths. The results of this study have important implications for service providers, administrators, and policymakers involved in the implementation of preventive and therapeutic interventions targeting youth antisocial behavior.
•The study integrates prevention and therapy research on youth antisocial behavior.•Modest long-term benefits were evident for 66 intervention trials (mean d=0.31).•Several interactions between participant, intervention, and methodological factors moderated effects.•For example, peer interventions were less effective for samples with more boys or older youths.•Results have important implications for intervention selection and development.
Implementation researchers are increasingly using economic evaluation to explore the benefits produced by implementing evidence-based practices (EBPs) in healthcare settings. However, the findings of ...typical economic evaluations (e.g., based on clinical trials) are rarely sufficient to inform decisions about how health service organizations and policymakers should finance investments in EBPs. This paper describes how economic evaluations can be translated into policy and practice through complementary research on financing strategies that support EBP implementation and sustainment.
We provide an overview of EBP implementation financing, which outlines key financing and health service delivery system stakeholders and their points of decision-making. We then illustrate how economic evaluations have informed decisions about EBP implementation and sustainment with three case examples: (1) use of Pay-for-Success financing to implement multisystemic therapy in underserved areas of Colorado, USA, based in part on the strength of evidence from economic evaluations; (2) an alternative payment model to sustain evidence-based oncology care, developed by the US Centers for Medicare and Medicaid Services through simulations of economic impact; and (3) use of a recently developed fiscal mapping process to collaboratively match financing strategies and needs during a pragmatic clinical trial for a newly adapted family support intervention for opioid use disorder.
EBP financing strategies can help overcome cost-related barriers to implementing and sustaining EBPs by translating economic evaluation results into policy and practice. We present a research agenda to advance understanding of financing strategies in five key areas raised by our case examples: (1) maximize the relevance of economic evaluations for real-world EBP implementation; (2) study ongoing changes in financing systems as part of economic evaluations; (3) identify the conditions under which a given financing strategy is most beneficial; (4) explore the use and impacts of financing strategies across pre-implementation, active implementation, and sustainment phases; and (5) advance research efforts through strong partnerships with stakeholder groups while attending to issues of power imbalance and transparency. Attention to these research areas will develop a robust body of scholarship around EBP financing strategies and, ultimately, enable greater public health impacts of EBPs.
Problematic sexual behavior in youth represents a significant public health problem in need of evidence-based treatments. Unfortunately, such treatments are not available in most communities.
This ...study used a mixed quantitative-qualitative approach to investigate the economics of the implementation of Problematic Sexual Behavior – Cognitive-Behavioral Therapy (PSB-CBT), an evidence-based treatment for problem sexual behaviors in youth.
Youth (N = 413) participated in PSB-CBT at six program sites in youth service agencies across the United States.
We used cost-effectiveness ratios (CERs) to compare the direct and indirect costs of PSB-CBT to self- and caregiver-reported youth clinical outcomes (i.e., problem sexual behavior as well as secondary behavioral health problems). CERs represented the cost of achieving one standard unit of change on a measure (i.e., d = 1.0). The design and interpretation of those quantitative analyses were informed by qualitative themes about program costs and benefits that were derived from interviews with 59 therapists, administrators, and stakeholders.
CERs (i.e., $ per SD) were $1,772 per youth for problem sexual behavior and ranged from $2,867 to $4,899 per youth for secondary outcomes. These quantitative results, considered alongside the qualitative perspectives of interviewees, suggested that the implementation of PSB-CBT was cost-effective. The results were robust to uncertainty in key parameters under most, but not all, conditions.
The results have important implications for decisions made by administrators, policymakers, and therapists regarding use of community-based approaches to address problematic sexual behavior of youth.
Objective: Although there is evidence that the positive impact of multisystemic therapy for problem sexual behaviors (MST-PSB) reaches as far as young adulthood, the longer-term effects of MST-PSB ...into midlife are unknown. The present study examined criminal and civil court outcomes for sexually offending youths who participated on average 24.9 years earlier in a clinical trial of MST-PSB (Borduin et al., Journal of Consulting and Clinical Psychology, 2009, 77, p. 26). Method: Participants were 48 individuals who were originally randomized to MST-PSB or usual community services (UCS) and were at high risk of continued criminality. Arrest, incarceration, and civil suit data were obtained in middle adulthood when participants averaged 39.4 years of age. Results: Intent-to-treat analyses showed that MST-PSB participants had 85% fewer sexual offenses and 70% fewer nonsexual offenses than did UCS participants. In addition, MST-PSB participants were sentenced to 46% fewer days of incarceration and had 62% fewer family-related civil suits. Moreover, the favorable effects of MST-PSB on participants' crimes and civil suits were mediated by improved peer and family relations during treatment. Conclusion: The current study represents the longest and most comprehensive follow-up to date of an MST-PSB clinical trial and demonstrates that the positive effects of an evidence-based youth treatment for sexual crimes can last well into adulthood. Implications of the findings for policymakers, service providers, and researchers are discussed.
What is the public health significance of this article?
This study demonstrates that a comprehensive, family-based treatment for youth sexual offending can have a lasting impact on participants' lives. The findings are useful for policymakers and service providers to consider in their selection of mental health interventions for youths who engage in sexual offenses.
PDF
