SYRCLE's risk of bias tool for animal studies Hooijmans, Carlijn R; Rovers, Maroeska M; de Vries, Rob B M ...
BMC medical research methodology,
03/2014, Volume:
14, Issue:
1
Journal Article
Peer reviewed
Open access
Systematic Reviews (SRs) of experimental animal studies are not yet common practice, but awareness of the merits of conducting such SRs is steadily increasing. As animal intervention studies differ ...from randomized clinical trials (RCT) in many aspects, the methodology for SRs of clinical trials needs to be adapted and optimized for animal intervention studies. The Cochrane Collaboration developed a Risk of Bias (RoB) tool to establish consistency and avoid discrepancies in assessing the methodological quality of RCTs. A similar initiative is warranted in the field of animal experimentation.
We provide an RoB tool for animal intervention studies (SYRCLE's RoB tool). This tool is based on the Cochrane RoB tool and has been adjusted for aspects of bias that play a specific role in animal intervention studies. To enhance transparency and applicability, we formulated signalling questions to facilitate judgment.
The resulting RoB tool for animal studies contains 10 entries. These entries are related to selection bias, performance bias, detection bias, attrition bias, reporting bias and other biases. Half these items are in agreement with the items in the Cochrane RoB tool. Most of the variations between the two tools are due to differences in design between RCTs and animal studies. Shortcomings in, or unfamiliarity with, specific aspects of experimental design of animal studies compared to clinical studies also play a role.
SYRCLE's RoB tool is an adapted version of the Cochrane RoB tool. Widespread adoption and implementation of this tool will facilitate and improve critical appraisal of evidence from animal studies. This may subsequently enhance the efficiency of translating animal research into clinical practice and increase awareness of the necessity of improving the methodological quality of animal studies.
Drug development is currently hampered by high attrition rates; many developed treatments fail during clinical testing. Part of the attrition may be due to low animal-to-human translational success ...rates; so-called "translational failure". As far as we know, no systematic overview of published translational success rates exists.
The following research question was examined: "What is the observed range of the animal-to-human translational success (and failure) rates within the currently available empirical evidence?". We searched PubMed and Embase on 16 October 2017. We included reviews and all other types of "umbrella"-studies of meta-data quantitatively comparing the translational results of studies including at least two species with one being human. We supplemented our database searches with additional strategies. All abstracts and full-text papers were screened by two independent reviewers. Our scoping review comprises 121 references, with various units of measurement: compound or intervention (k = 104), study/experiment (k = 10), and symptom or event (k = 7). Diagnostic statistics corresponded with binary and continuous definitions of successful translation. Binary definitions comprise percentages below twofold error, percentages accurately predicted, and predictive values. Quantitative definitions comprise correlation/regression (r
) and meta-analyses (percentage overlap of 95% confidence intervals). Translational success rates ranged from 0 to 100%.
The wide range of translational success rates observed in our study might indicate that translational success is unpredictable; i.e. it might be unclear upfront if the results of primary animal studies will contribute to translational knowledge. However, the risk of bias of the included studies was high, and much of the included evidence is old, while newer models have become available. Therefore, the reliability of the cumulative evidence from current papers on this topic is insufficient. Further in-depth "umbrella"-studies of translational success rates are still warranted. These are needed to evaluate the probabilistic evidence for predictivity of animal studies for the human situation more reliably, and to determine which factors affect this process.
Drug research with animal models is expensive, time-consuming and translation to clinical trials is often poor, resulting in a desire to replace, reduce, and refine the use of animal models. One ...approach to replace and reduce the use of animal models is to use in vitro cell-culture models. To study bone physiology, bone diseases and drugs, many studies have been published using osteoblast-osteoclast co-cultures. The use of osteoblast-osteoclast co-cultures is usually not clearly mentioned in the title and abstract, making it difficult to identify these studies without a systematic search and thorough review. As a result, researchers are all developing their own methods, leading to conceptually similar studies with many methodological differences and, as a consequence, incomparable results. The aim of this study was to systematically review existing osteoblast-osteoclast co-culture studies published up to 6 January 2020, and to give an overview of their methods, predetermined outcome measures (formation and resorption, and ALP and TRAP quantification as surrogate markers for formation and resorption, respectively), and other useful parameters for analysis. Information regarding these outcome measures was extracted and collected in a database, and each study was further evaluated on whether both the osteoblasts and osteoclasts were analyzed using relevant outcome measures. From these studies, additional details on methods, cells and culture conditions were extracted into a second database to allow searching on more characteristics. The two databases presented in this publication provide an unprecedented amount of information on cells, culture conditions and analytical techniques for using and studying osteoblast-osteoclast co-cultures. They allow researchers to identify publications relevant to their specific needs and allow easy validation and comparison with existing literature. Finally, we provide the information and tools necessary for others to use, manipulate and expand the databases for their needs.
Laboratory animal studies are used in a wide range of human health related research areas, such as basic biomedical research, drug research, experimental surgery and environmental health. The results ...of these studies can be used to inform decisions regarding clinical research in humans, for example the decision to proceed to clinical trials. If the research question relates to potential harms with no expectation of benefit (e.g., toxicology), studies in experimental animals may provide the only relevant or controlled data and directly inform clinical management decisions. Systematic reviews and meta-analyses are important tools to provide robust and informative evidence summaries of these animal studies. Rating how certain we are about the evidence could provide important information about the translational probability of findings in experimental animal studies to clinical practice and probably improve it. Evidence summaries and certainty in the evidence ratings could also be used (1) to support selection of interventions with best therapeutic potential to be tested in clinical trials, (2) to justify a regulatory decision limiting human exposure (to drug or toxin), or to (3) support decisions on the utility of further animal experiments. The Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) approach is the most widely used framework to rate the certainty in the evidence and strength of health care recommendations. Here we present how the GRADE approach could be used to rate the certainty in the evidence of preclinical animal studies in the context of therapeutic interventions. We also discuss the methodological challenges that we identified, and for which further work is needed. Examples are defining the importance of consistency within and across animal species and using GRADE's indirectness domain as a tool to predict translation from animal models to humans.
Despite advancements in surgical technique and perioperative care, intestinal anastomoses still have a 10-15 per cent risk of leakage, which results in considerable morbidity and/or mortality. Recent ...animal studies have suggested that administration of butyrate to the anastomotic site results in enhanced anastomotic strength, which may prevent leakage. This systematic review and meta-analysis summarises current evidence concerning the effect of butyrate administration on anastomotic healing and will form a scientific basis for the development of new research into this subject.
Animal studies on the effect of butyrate-based interventions in models of intestinal anastomotic healing were systematically retrieved from online databases. Bibliographical data, study characteristics and outcome data were extracted, and internal validity of the studies was assessed. Outcomes studied through meta-analysis concerned: anastomotic strength, anastomotic leakage, collagen metabolism and general histologic parameters of wound healing.
A comprehensive search and selection identified 19 relevant studies containing 41 individual comparisons. Design and conduct of most experiments were poorly reported resulting in an unclear risk of bias. Meta-analyses showed that butyrate administration significantly increases anastomotic strength (SMD 1.24, 0.88 to 1.61), collagen synthesis (SMD 1.44, 0.72 to 2.15) and collagen maturation, making anastomoses less prone to leakage in the early postoperative period (OR 0.37, 0.15 to 0.93).
This systematic review and meta-analysis shows that there is potential ground to investigate the use of butyrate in clinical trials to prevent anastomotic leakage in intestinal surgery. However, more research is necessary to define the best application form, dosage and administration route.
The methodological quality of animal studies is an important factor hampering the translation of results from animal studies to a clinical setting. Systematic reviews of animal studies may provide a ...suitable method to assess and thereby improve their methodological quality.
The aims of this study were: 1) to evaluate the risk of bias assessment in animal-based systematic reviews, and 2) to study the internal validity of the primary animal studies included in these systematic reviews.
We systematically searched Pubmed and Embase for SRs of preclinical animal studies published between 2005 and 2012.
A total of 91 systematic reviews met our inclusion criteria. The risk of bias was assessed in 48 (52.7%) of these 91 systematic reviews. Thirty-three (36.3%) SRs provided sufficient information to evaluate the internal validity of the included studies. Of the evaluated primary studies, 24.6% was randomized, 14.6% reported blinding of the investigator/caretaker, 23.9% blinded the outcome assessment, and 23.1% reported drop-outs.
To improve the translation of animal data to clinical practice, systematic reviews of animal studies are worthwhile, but the internal validity of primary animal studies needs to be improved. Furthermore, risk of bias should be assessed by systematic reviews of animal studies to provide insight into the reliability of the available evidence.
Low frequency magnetic field (LF MF) exposure is recurrently suggested to have the ability to induce health effects in society. Therefore, in vitro model systems are used to investigate biological ...effects of exposure. LF MF induced changes of the cellular calcium homeostasis are frequently hypothesised to be the possible target, but this hypothesis is both substantiated and rejected by numerous studies in literature. Despite the large amount of data, no systematic analysis of in vitro studies has been conducted to address the strength of evidence for an association between LF MF exposure and calcium homeostasis. Our systematic review, with inclusion of 42 studies, showed evidence for an association of LF MF with internal calcium concentrations and calcium oscillation patterns. The oscillation frequency increased, while the amplitude and the percentage of oscillating cells remained constant. The intracellular calcium concentration increased (SMD 0.351, 95% CI 0.126, 0.576). Subgroup analysis revealed heterogeneous effects associated with the exposure frequency, magnetic flux density and duration. Moreover, we found support for the presence of MF-sensitive cell types. Nevertheless, some of the included studies may introduce a great risk of bias as a result of uncontrolled or not reported exposure conditions, temperature ranges and ambient fields. In addition, mathematical calculations of the parasitic induced electric fields (IEFs) disclosed their association with increased intracellular calcium. Our results demonstrate that LF MF might influence the calcium homeostasis in cells in vitro, but the risk of bias and high heterogeneity (I2>75%) weakens the analyses. Therefore any potential clinical implications await further investigation.
•First systematic review that links electromagnetic exposure (EMF) to altered calcium homeostasis•EMF exposure could induce a small increase of intracellular calcium.•Exposure characteristics and cell type are indicators for the effect size.•Uncontrolled induced electric fields (IEFs) are strongly associated with intracellular calcium.
Microdialysis is a method to study the extracellular space in vivo, based on the principle of diffusion. It can be used to measure various small molecules including the neuroregulator adenosine. ...Baseline levels of the compounds measured with microdialysis vary over studies. We systematically reviewed the literature to investigate the full range of reported baseline concentrations of adenosine and adenosine monophosphate in microdialysates. We performed a meta‐regression analysis to study the influence of flow rate, probe membrane surface area, species, brain area and anaesthesia versus freely behaving, on the adenosine concentration. Baseline adenosine concentrations in microdialysates ranged from 0.8 to 2100 nM. There was limited evidence on baseline adenosine monophosphate concentrations in microdialysates. Across studies, we found effects of flow rate and anaesthesia versus freely behaving on dialysate adenosine concentrations (p ≤ 0.001), but not of probe membrane surface, species, or brain area (p ≥ 0.14). With increasing flow rate, adenosine concentrations decreased. With anaesthesia, adenosine concentrations increased. The effect of other predictor variables on baseline adenosine concentrations, for example, post‐surgical recovery time, could not be analysed because of a lack of reported data. This study shows that meta‐regression can be used as an alternative to new animal experiments to answer research questions in the field of neurochemistry. However, current levels of reporting of primary studies are insufficient to reach the full potential of this approach; 63 out of 133 studies could not be included in the analysis because of insufficient reporting, and several potentially relevant factors had to be excluded from the analyses. The level of reporting of experimental detail needs to improve.
Microdialysis has long been used to measure neurotransmitters, with overwhelming variations in experimental designs. A systematic literature review is the optimal method to acquire a complete overview. We systematically reviewed the literature for baseline concentrations of adenosine in microdialysates. We then performed a meta‐regression analysis to study the effects of variations in experimental design. This study shows that meta‐regression can be used as an alternative to new animal experiments to answer research questions in the field of neurochemistry. However, current levels of reporting of primary studies are insufficient to reach the full potential of this approach; 63 out of 133 studies could not be included in the analysis because of insufficient reporting, and several potentially relevant factors had to be excluded from the analyses. Therefore, the standard of experimental detail reporting needs to improve. Picture by Julia M.L. Menon, syringe and vial adapted from Servier medical art.
In February 2008, the results of the PRObiotics in PAncreatitis TRIAl (PROPATRIA) were published. This study investigated the use of probiotics in patients suffering from severe acute pancreatitis. ...No differences between the groups were found for any of the primary endpoints. However, mortality in the probiotics group was significantly higher than in the placebo group. This result was unexpected in light of the results of the animal studies referred to in the trial protocol. We used the methods of systematic review and meta-analysis to take a closer look at the relation between the animal studies on probiotics and pancreatitis and the PROPATRIA-trial, focussing on indications for harmful effects and efficacy.
Both PubMed and Embase were searched for original articles concerning the effects of probiotics in experimental acute pancreatitis, yielding thirteen studies that met the inclusion criteria. Data on mortality, bacterial translocation and histological damage to the pancreas were extracted, as well as study quality indicators. Meta-analysis of the four animal studies published before PROPATRIA showed that probiotic supplementation did not diminish mortality, reduced the overall histopathological score of the pancreas and reduced bacterial translocation to pancreas and mesenteric lymph nodes. Comparable results were found when all relevant studies published so far were taken into account.
A more thorough analysis of all relevant animal studies carried out before (and after) the publication of the study protocol of the PROPATRIA trial could not have predicted the harmful effects of probiotics found in the PROPATRIA-trial. Moreover, meta-analysis of the preclinical animal studies did show evidence for efficacy. It may be suggested, however, that the most appropriate animal experiments in relation to the design of the human trial have not yet been conducted, which compromises a fair comparison between the results of the animal studies and the PROPATRIA trial.
Good Cell and Tissue Culture Practice (GCCP) 2.0 is an updated guidance document from GCCP 1.0 (published by ECVAM in 2005), which was developed for practical use in the laboratory to assure the ...reproducibility of in vitro (cell-based) work. The update in the guidance was essential as cell models have advanced dramatically to more complex culture systems and need more comprehensive quality management to ensure reproducibility and high-quality scientific data. This document describes six main principles to consider when performing cell culture including characterization and maintenance of essential characteristics, quality management, documentation and reporting, safety, education and training, and ethics. The document does not intend to impose detailed procedures but to describe potential quality issues. It is foreseen that the document will require further updates as the science and technologies evolve over time.
PDF
