The movement towards open science is a consequence of seemingly pervasive failures to replicate previous research. This transition comes with great benefits but also significant challenges that are ...likely to affect those who carry out the research, usually early career researchers (ECRs). Here, we describe key benefits, including reputational gains, increased chances of publication, and a broader increase in the reliability of research. The increased chances of publication are supported by exploratory analyses indicating null findings are substantially more likely to be published via open registered reports in comparison to more conventional methods. These benefits are balanced by challenges that we have encountered and that involve increased costs in terms of flexibility, time, and issues with the current incentive structure, all of which seem to affect ECRs acutely. Although there are major obstacles to the early adoption of open science, overall open science practices should benefit both the ECR and improve the quality of research. We review 3 benefits and 3 challenges and provide suggestions from the perspective of ECRs for moving towards open science practices, which we believe scientists and institutions at all levels would do well to consider.
Celotno besedilo
Dostopno za:
DOBA, IZUM, KILJ, NUK, PILJ, PNG, SAZU, SIK, UILJ, UKNU, UL, UM, UPUK
We currently observe a disconcerting phenomenon in machine learning studies in psychiatry: While we would expect larger samples to yield better results due to the availability of more data, larger ...machine learning studies consistently show much weaker performance than the numerous small-scale studies. Here, we systematically investigated this effect focusing on one of the most heavily studied questions in the field, namely the classification of patients suffering from Major Depressive Disorder (MDD) and healthy controls based on neuroimaging data. Drawing upon structural MRI data from a balanced sample of N = 1868 MDD patients and healthy controls from our recent international Predictive Analytics Competition (PAC), we first trained and tested a classification model on the full dataset which yielded an accuracy of 61%. Next, we mimicked the process by which researchers would draw samples of various sizes (N = 4 to N = 150) from the population and showed a strong risk of misestimation. Specifically, for small sample sizes (N = 20), we observe accuracies of up to 95%. For medium sample sizes (N = 100) accuracies up to 75% were found. Importantly, further investigation showed that sufficiently large test sets effectively protect against performance misestimation whereas larger datasets per se do not. While these results question the validity of a substantial part of the current literature, we outline the relatively low-cost remedy of larger test sets, which is readily available in most cases.
There is increasing interest in exploring the use of functional MRI neurofeedback (fMRI-NF) as a therapeutic technique for a range of neurological conditions such as stroke and Parkinson's disease ...(PD). One main therapeutic potential of fMRI-NF is to enhance volitional control of damaged or dysfunctional neural nodes and networks via a closed-loop feedback model using mental imagery as the catalyst of self-regulation. The choice of target node/network and direction of regulation (increase or decrease activity) are central design considerations in fMRI-NF studies. Whilst it remains unclear whether the primary motor cortex (M1) can be activated during motor imagery, the supplementary motor area (SMA) has been robustly activated during motor imagery. Such differences in the regulation potential between primary and supplementary motor cortex are important because these areas can be differentially affected by a stroke or PD, and the choice of fMRI-NF target and grade of self-regulation of activity likely have substantial influence on the clinical effects and cost effectiveness of NF-based interventions. In this study we therefore investigated firstly whether healthy subjects would be able to achieve self-regulation of the hand-representation areas of M1 and the SMA using fMRI-NF training. There was a significant decrease in M1 neural activity during fMRI-NF, whereas SMA neural activity was increased, albeit not with the predicated graded effect. This study has important implications for fMRI-NF protocols that employ motor imagery to modulate activity in specific target regions of the brain and to determine how they may be tailored for neurorehabilitation.
•Graded fMRI neurofeedback can separate self-regulation effects from general cognitive ones.•There is a significant negative BOLD response in M1 during kinaesthetic motor imagery.•SMA shows robust positive BOLD responses during kinaesthetic motor imagery.•Participants can target discrete levels of BOLD response in SMA using kinaesthetic motor imagery.•For our paradigm neurofeedback provides only minimal extra control of BOLD signals in SMA.
Statistical power is essential for robust science and replicability, but a meta-analysis by Button et al. in 2013 diagnosed a "power failure" for neuroscience. In contrast, Nord et al. ( J Neurosci ...37: 8051-8061, 2017) reanalyzed these data and suggested that some studies feature high power. We illustrate how publication and researcher bias might have inflated power estimates, and review recently introduced techniques that can improve analysis pipelines and increase power in neuroscience studies.
Background:
The effects of electroencephalography (EEG) and functional magnetic resonance imaging (fMRI)-neurofeedback on brain activation and behaviors have been studied extensively in the past. ...More recently, researchers have begun to investigate the effects of functional near-infrared spectroscopy-based neurofeedback (fNIRS-neurofeedback). FNIRS is a functional neuroimaging technique based on brain hemodynamics, which is easy to use, portable, inexpensive, and has reduced sensitivity to movement artifacts.
Method:
We provide the first systematic review and database of fNIRS-neurofeedback studies, synthesizing findings from 22 peer-reviewed studies (including a total of
N
= 441 participants; 337 healthy, 104 patients). We (1) give a comprehensive overview of how fNIRS-neurofeedback training protocols were implemented, (2) review the online signal-processing methods used, (3) evaluate the quality of studies using pre-set methodological and reporting quality criteria and also present statistical sensitivity/power analyses, (4) investigate the effectiveness of fNIRS-neurofeedback in modulating brain activation, and (5) review its effectiveness in changing behavior in healthy and pathological populations.
Results and discussion:
(1–2) Published studies are heterogeneous (e.g., neurofeedback targets, investigated populations, applied training protocols, and methods). (3) Large randomized controlled trials are still lacking. In view of the novelty of the field, the quality of the published studies is moderate. We identified room for improvement in reporting important information and statistical power to detect realistic effects. (4) Several studies show that people can regulate hemodynamic signals from cortical brain regions with fNIRS-neurofeedback and (5) these studies indicate the feasibility of modulating motor control and prefrontal brain functioning in healthy participants and ameliorating symptoms in clinical populations (stroke, ADHD, autism, and social anxiety). However, valid conclusions about specificity or potential clinical utility are premature.
Conclusion:
Due to the advantages of practicability and relatively low cost, fNIRS-neurofeedback might provide a suitable and powerful alternative to EEG and fMRI neurofeedback and has great potential for clinical translation of neurofeedback. Together with more rigorous research and reporting practices, further methodological improvements may lead to a more solid understanding of fNIRS-neurofeedback. Future research will benefit from exploiting the advantages of fNIRS, which offers unique opportunities for neurofeedback research.
•First systematic review including studies employing EEG and fMRI-based protocols.•Data from 24 studies (480 patients in experimental and 194 in the control groups).•Most studies showing symptom ...improvement superior to the control group(s).•Positive linear trends showing improved study quality across years.
Major depressive disorder (MDD) is the leading cause of disability worldwide. Neurofeedback training has been suggested as a potential additional treatment option for MDD patients not reaching remission from standard care (i.e., psychopharmacology and psychotherapy). Here we systematically reviewed neurofeedback studies employing electroencephalography, or functional magnetic resonance-based protocols in depressive patients. Of 585 initially screened studies, 24 were included in our final sample (N = 480 patients in experimental and N = 194 in the control groups completing the primary endpoint). We evaluated the clinical efficacy across studies and attempted to group studies according to the control condition categories currently used in the field that affect clinical outcomes in group comparisons. In most studies, MDD patients showed symptom improvement superior to the control group(s). However, most articles did not comply with the most stringent study quality and reporting practices. We conclude with recommendations on best practices for experimental designs and reporting standards for neurofeedback training.
A key objective in the field of translational psychiatry over the past few decades has been to identify the brain correlates of major depressive disorder (MDD). Identifying measurable indicators of ...brain processes associated with MDD could facilitate the detection of individuals at risk, and the development of novel treatments, the monitoring of treatment effects, and predicting who might benefit most from treatments that target specific brain mechanisms. However, despite intensive neuroimaging research towards this effort, underpowered studies and a lack of reproducible findings have hindered progress. Here, we discuss the work of the ENIGMA Major Depressive Disorder (MDD) Consortium, which was established to address issues of poor replication, unreliable results, and overestimation of effect sizes in previous studies. The ENIGMA MDD Consortium currently includes data from 45 MDD study cohorts from 14 countries across six continents. The primary aim of ENIGMA MDD is to identify structural and functional brain alterations associated with MDD that can be reliably detected and replicated across cohorts worldwide. A secondary goal is to investigate how demographic, genetic, clinical, psychological, and environmental factors affect these associations. In this review, we summarize findings of the ENIGMA MDD disease working group to date and discuss future directions. We also highlight the challenges and benefits of large-scale data sharing for mental health research.
Transparent communication of research is key to foster understanding within and beyond the scientific community. An increased focus on reporting effect sizes in addition to p value–based significance ...statements or Bayes Factors may improve scientific communication with the general public. Across three studies (N = 652), we compared subjective informativeness ratings for five effect sizes, Bayes Factor, and commonly used significance statements. Results showed that Cohen’s U3 was rated as most informative. For example, 440 participants (69%) found U3 more informative than Cohen’s d, while 95 (15%) found d more informative than U3, with 99 participants (16%) finding both effect sizes equally informative. This effect was not moderated by level of education. We therefore suggest that in general, Cohen’s U3 is used when scientific findings are communicated. However, the choice of the effect size may vary depending on what a researcher wants to highlight (e.g. differences or similarities).
Functional magnetic resonance imaging neurofeedback (fMRI-NF) training of areas involved in emotion processing can reduce depressive symptoms by over 40% on the Hamilton Depression Rating Scale ...(HDRS). However, it remains unclear if this efficacy is specific to feedback from emotion-regulating regions. We tested in a single-blind, randomized, controlled trial if upregulation of emotion areas (NFE) yields superior efficacy compared to upregulation of a control region activated by visual scenes (NFS). Forty-three moderately to severely depressed medicated patients were randomly assigned to five sessions augmentation treatment of either NFE or NFS training. At primary outcome (week 12) no significant group mean HDRS difference was found (B = -0.415 95% CI -4.847 to 4.016, p = 0.848) for the 32 completers (16 per group). However, across groups depressive symptoms decreased by 43%, and 38% of patients remitted. These improvements lasted until follow-up (week 18). Both groups upregulated target regions to a similar extent. Further, clinical improvement was correlated with an increase in self-efficacy scores. However, the interpretation of clinical improvements remains limited due to lack of a sham-control group. We thus surveyed effects reported for accepted augmentation therapies in depression. Data indicated that our findings exceed expected regression to the mean and placebo effects that have been reported for drug trials and other sham-controlled high-technology interventions. Taken together, we suggest that the experience of successful self-regulation during fMRI-NF training may be therapeutic. We conclude that if fMRI-NF is effective for depression, self-regulation training of higher visual areas may provide an effective alternative.
PDF
