There is extensive health and public health literature on the 'evidence-policy gap', exploring the frustrating experiences of scientists trying to secure a response to the problems and solutions they ...raise and identifying the need for better evidence to reduce policymaker uncertainty. We offer a new perspective by using policy theory to propose research with greater impact, identifying the need to use persuasion to reduce ambiguity, and to adapt to multi-level policymaking systems.We identify insights from secondary data, namely systematic reviews, critical analysis and policy theories relevant to evidence-based policymaking. The studies are drawn primarily from countries such as the United States, United Kingdom, Canada, Australia and New Zealand. We combine empirical and normative elements to identify the ways in which scientists can, do and could influence policy.We identify two important dilemmas, for scientists and researchers, that arise from our initial advice. First, effective actors combine evidence with manipulative emotional appeals to influence the policy agenda - should scientists do the same, or would the reputational costs outweigh the policy benefits? Second, when adapting to multi-level policymaking, should scientists prioritise 'evidence-based' policymaking above other factors? The latter includes governance principles such the 'co-production' of policy between local public bodies, interest groups and service users. This process may be based primarily on values and involve actors with no commitment to a hierarchy of evidence.We conclude that successful engagement in 'evidence-based policymaking' requires pragmatism, combining scientific evidence with governance principles, and persuasion to translate complex evidence into simple stories. To maximise the use of scientific evidence in health and public health policy, researchers should recognise the tendency of policymakers to base judgements on their beliefs, and shortcuts based on their emotions and familiarity with information; learn 'where the action is', and be prepared to engage in long-term strategies to be able to influence policy; and, in both cases, decide how far you are willing to go to persuade policymakers to act and secure a hierarchy of evidence underpinning policy. These are value-driven and political, not just 'evidence-based', choices.
Coproduction, a collaborative model of research that includes stakeholders in the research process, has been widely advocated as a means of facilitating research use and impact. We summarise the ...arguments in favour of coproduction, the different approaches to establishing coproductive work and their costs, and offer some advice as to when and how to consider coproduction.
Despite the multiplicity of reasons and incentives to coproduce, there is little consensus about what coproduction is, why we do it, what effects we are trying to achieve, or the best coproduction techniques to achieve policy, practice or population health change. Furthermore, coproduction is not free risk or cost. Tensions can arise throughout coproduced research processes between the different interests involved. We identify five types of costs associated with coproduced research affecting the research itself, the research process, professional risks for researchers and stakeholders, personal risks for researchers and stakeholders, and risks to the wider cause of scholarship. Yet, these costs are rarely referred to in the literature, which generally calls for greater inclusion of stakeholders in research processes, focusing exclusively on potential positives. There are few tools to help researchers avoid or alleviate risks to themselves and their stakeholders.
First, we recommend identifying specific motivations for coproduction and clarifying exactly which outcomes are required for whom for any particular piece of research. Second, we suggest selecting strategies specifically designed to enable these outcomes to be achieved, and properly evaluated. Finally, in the absence of strong evidence about the impact and process of coproduction, we advise a cautious approach to coproduction. This would involve conscious and reflective research practice, evaluation of how coproduced research practices change outcomes, and exploration of the costs and benefits of coproduction. We propose some preliminary advice to help decide when coproduction is likely to be more or less useful.
Despite a growth in knowledge translation (KT) or exchange activities, and a smaller growth in their evaluations, it remains challenging to identify evidence of efficacy. This could be due to ...well-documented political and logistical difficulties involved in evaluating knowledge exchange interventions. By bringing in theory from science and technology studies (STS), Borst et al
offer a new way of thinking about this problem. Most KT evaluations draw on health research traditions; centralising comparability, efficacy, and so on. Borst et al propose focusing on the work it takes to move knowledge over boundaries between these communities, seeing relationships as interactions, not just conduits for evidence. They show how 'context' can be understood as a mutual creation, not a static environment; and that institutions shape behaviours, rather than merely being sites or platforms for evidence mobilisation. Seeing KT as a creative, active practice opens new ways to design and evaluate KT mechanisms.
Public health (PH) policymakers are encouraged to use evidence in the decision-making process. However, little is known about what types of evidence policymakers working in local settings prefer to ...use. This study aims to evaluate policymakers' needs and sources of information, at regional and local levels. An electronic survey with telephone follow-up was carried out among PH policymakers and evidence producers ( n = 152) working in a large UK city. Respondents were asked which types of evidence they used regularly, found most useful and what were their main sources of information. Semi-structured interviews ( n = 23) added were analysed quantitatively in addition to the categorical data generated by the survey. Policymakers use a much greater range of evidence and information than is often indicated in the literature on evidence-based policy. Local data were by far the most used ( n = 95%) and most valued ( n = 85%) type of information, followed by practice guidelines. The main sources of information were Government websites (84%), followed by information obtained through personal contacts (71%), including PH professionals, council officers and politicians. Academics were rarely consulted and research evidence was rarely seen as directly relevant. Conclusions: Policymakers use a wider range of evidence types than previously discussed in the literature. Although local data were most valued by policymakers, results suggest that these were accessed through personal contacts, rather than specialized organizations. Systems to provide local high-quality evidence for PH policy should be supported.
Public health policies sometimes have unexpected effects. Understanding how policies and interventions lead to outcomes is essential if policymakers and researchers are to intervene effectively and ...reduce harmful and other unintended consequences (UCs) of their actions. Yet, evaluating complex mechanisms and outcomes is challenging, even before considering how to predict assess and understand outcomes and UCs when interventions are scaled up. We aimed to explore with UK policymakers why some policies have UCs, and how researchers and policymakers should respond.
We convened a one-day workshop with 14 people involved in developing, implementing or evaluating social and public health policies, and/or evaluating possible unintended effects. This included senior evaluators, policymakers from government and associated agencies, and researchers, covering policy domains from public health, social policy, poverty, and international development.
Policymakers suggested UCs happen for a range of reasons: poor policy design, unclear articulation of policy mechanisms or goals, or unclear or inappropriate evidence use, including evaluation techniques. While not always avoidable, it was felt that UCs could be partially mitigated by better use of theory and evidence, better involvement of stakeholders in concurrent design and evaluation of policies, and appropriate evaluation systems.
UCs can be used to explore the mechanisms underpinning social change caused by public health policies. Articulating these mechanisms is essential for truly evidence-informed decision-making, to enable informed debate about policy options, and to develop evaluation techniques. Future work includes trying to develop a holistic stakeholder-led evaluation process.
Variants in the cystic fibrosis transmembrane conductance regulator gene (CFTR) result in cystic fibrosis-a lethal autosomal recessive disorder. Missense variants that alter a single amino acid in ...the CFTR protein are among the most common cystic fibrosis variants, yet tools for accurately predicting molecular consequences of missense variants have been limited to date. AlphaMissense (AM) is a new technology that predicts the pathogenicity of missense variants based on dual learned protein structure and evolutionary features. Here, we evaluated the ability of AM to predict the pathogenicity of CFTR missense variants. AM predicted a high pathogenicity for CFTR residues overall, resulting in a high false positive rate and fair classification performance on CF variants from the CFTR2.org database. AM pathogenicity score correlated modestly with pathogenicity metrics from persons with CF including sweat chloride level, pancreatic insufficiency rate, and Pseudomonas aeruginosa infection rate. Correlation was also modest with CFTR trafficking and folding competency in vitro. By contrast, the AM score correlated well with CFTR channel function in vitro-demonstrating the dual structure and evolutionary training approach learns important functional information despite lacking such data during training. Different performance across metrics indicated AM may determine if polymorphisms in CFTR are recessive CF variants yet cannot differentiate mechanistic effects or the nature of pathophysiology. Finally, AM predictions offered limited utility to inform on the pharmacological response of CF variants i.e., theratype. Development of new approaches to differentiate the biochemical and pharmacological properties of CFTR variants is therefore still needed to refine the targeting of emerging precision CF therapeutics.
There has been a proliferation of frameworks with a common goal of bridging the gap between evidence, policy, and practice, but few aim to specifically guide evaluations of academic-policy ...engagement. We present the modification of an action framework for the purpose of selecting, developing and evaluating interventions for academic-policy engagement.
We build on the conceptual work of an existing framework known as SPIRIT (Supporting Policy In Health with Research: an Intervention Trial), developed for the evaluation of strategies intended to increase the use of research in health policy. Our aim was to modify SPIRIT, (i) to be applicable beyond health policy contexts, for example encompassing social, environmental, and economic policy impacts and (ii) to address broader dynamics of academic-policy engagement. We used an iterative approach through literature reviews and consultation with multiple stakeholders from Higher Education Institutions (HEIs) and policy professionals working at different levels of government and across geographical contexts in England, alongside our evaluation activities in the Capabilities in Academic Policy Engagement (CAPE) programme.
Our modifications expand upon Redman et al.'s original framework, for example adding a domain of 'Impacts and Sustainability' to capture continued activities required in the achievement of desirable outcomes. The modified framework fulfils the criteria for a useful action framework, having a clear purpose, being informed by existing understandings, being capable of guiding targeted interventions, and providing a structure to build further knowledge.
The modified SPIRIT framework is designed to be meaningful and accessible for people working across varied contexts in the evidence-policy ecosystem. It has potential applications in how academic-policy engagement interventions might be developed, evaluated, facilitated and improved, to ultimately support the use of evidence in decision-making.
Patients with cystic fibrosis (CF) harboring the P67L variant in the cystic fibrosis transmembrane conductance regulator (CFTR) often exhibit a typical CF phenotype, including severe respiratory ...compromise. This rare mutation (reported in <300 patients worldwide) responds robustly to CFTR correctors, such as lumacaftor and tezacaftor, with rescue in model systems that far exceed what can be achieved for the archetypical CFTR mutant F508del. However, the specific molecular consequences of the P67L mutation are poorly characterized. In this study, we conducted biochemical measurements following low-temperature growth and/or intragenic suppression, which suggest a mechanism underlying P67L that (1) shares key pathogenic features with F508del, including off-pathway (non-native) folding intermediates, (2) is linked to folding stability of nucleotide-binding domains 1 and 2, and (3) demonstrates pharmacologic rescue that requires domains in the carboxyl half of the protein. We also investigated the “lasso” helices 1 and 2, which occur immediately upstream of P67. Based on limited proteolysis, pulse chase, and molecular dynamics analysis of full-length CFTR and a series of deletion constructs, we argue that P67L and other maturational processing (class 2) defects impair the integrity of the lasso motif and confer misfolding of downstream domains. Thus, amino-terminal missense variants elicit a conformational change throughout CFTR that abrogates maturation while providing a robust substrate for pharmacologic repair.
Understanding how policies lead to changes in health systems and in practice helps policymakers and researchers to intervene more successfully. Yet identifying all the possible changes that occur as ...a result of a new policy is challenging not only methodologically and logistically, as limited resources are available to conduct indefinite evaluations, but also theoretically, as a complete mapping and attribution of post-hoc changes requires a full understanding of the mechanisms underpinning all change. One option is to identify possible changes across a number of policy impact domains.
Using a Policy Impact Framework, we brought together data from media, documents and interviews to identify changes to midwifery policy, practice and provision, following the launch of a new global policy initiative, the State of the World's Midwifery (SoWMy 2014) report published in 2014. We used these identified impacts to develop a map of the mechanisms underpinning these changes.
SoWMy 2014 contributed to a number of changes at national levels, including increased status of midwifery within national governments, improved curricula and training opportunities for midwives, and improved provision of and access to midwifery-led care. These contributions were attributed to SoWMy 2014 via mechanisms such as stakeholder interaction and acquisition of government support, holding national and international dissemination and training events, and a perceived global momentum around supporting midwifery provision. Policy initiatives of this kind can lead to changes in national and international policy dialogue and practice. We identify factors and mechanisms that are likely to increase the scope and scale of these changes, at contextual, national and global levels.
Identifying changes following a policy using a policy impact framework can help researchers and policymakers understand why policies have the effect they do. This is important information for those wishing to increase the effectiveness of future policies and interventions.
As pressures to maximize research funding grow, biomedical research funders are increasingly tasked with demonstrating the long-term and real-world impacts of their funded research investments. Over ...the past three decades, research impact assessments (RIA) have emerged as an important tool for analysing the impacts of research by incorporating logic models, frameworks and indicators to track measures of knowledge production, capacity-building, development of research products, adoption of research into clinical guidelines and policies, and the realization of health, economic and social benefits. While there are currently several models for RIA within the literature, less attention has been paid to how funders can practically select and implement a RIA model to demonstrate the impacts of their own research portfolios. In this paper, a literature review was performed to understand (1) which research funders have performed RIAs of their research portfolios to date; (2) how funders have designed their assessments, including the models and tools they have used; (3) what challenges to and facilitators of success have funders found when adopting the RIA model to their own portfolio; and (4) who participates in the assessments. Forty-four papers from both published and grey literature were found to meet the review criteria and were examined in detail. There is a growing culture of RIA among funders, and included papers spanned a diverse set of funders from 10 countries or regions. Over half of funders (59.1%) used a framework to conduct their assessment, and a variety of methods for collecting impact data were reported. Issues of methodological rigour were observed across studies in the review, and this was related to numerous challenges funders faced in designing timely RIAs with quality impact data. Over a third of articles (36.4%) included input from stakeholders, yet only one article reported surveying patients and members of the public as part of the assessment. To advance RIA among funders, we offer several recommendations for increasing the methodological rigour of RIAs and suggestions for future research, and call for a careful reflection of the voices needed in an impact assessment to ensure that RIAs are having a meaningful impact on patients and the public.
PDF
