Once considered mere noise, fMRI-based functional connectivity has become a major neuroscience tool in part due to early studies demonstrating its reliability. These fundamental studies revealed only ...the tip of the iceberg; over the past decade, many test-retest reliability studies have continued to add nuance to our understanding of this complex topic. A summary of these diverse and at times contradictory perspectives is needed.
We aimed to summarize the existing knowledge regarding test-retest reliability of functional connectivity at the most basic unit of analysis: the individual edge level. This entailed (1) a meta-analytic estimate of reliability and (2) a review of factors influencing reliability.
A search of Scopus was conducted to identify studies that estimated edge-level test-retest reliability. To facilitate comparisons across studies, eligibility was restricted to studies measuring reliability via the intraclass correlation coefficient (ICC). The meta-analysis included a random effects pooled estimate of mean edge-level ICC, with studies nested within datasets. The review included a narrative summary of factors influencing edge-level ICC.
From an initial pool of 212 studies, 44 studies were identified for the qualitative review and 25 studies for quantitative meta-analysis. On average, individual edges exhibited a “poor” ICC of 0.29 (95% CI = 0.23 to 0.36). The most reliable measurements tended to involve: (1) stronger, within-network, cortical edges, (2) eyes open, awake, and active recordings, (3) more within-subject data, (4) shorter test-retest intervals, (5) no artifact correction (likely due in part to reliable artifact), and (6) full correlation-based connectivity with shrinkage.
This study represents the first meta-analysis and systematic review investigating test-retest reliability of edge-level functional connectivity. Key findings suggest there is room for improvement, but care should be taken to avoid promoting reliability at the expense of validity. By pooling existing knowledge regarding this key facet of accuracy, this study supports broader efforts to improve inferences in the field.
•We present the first meta-analytic review of functional connectivity reliability.•On average, individual connections exhibit a “poor” ICC of 0.29.•Study characteristics that influence reliability are summarized and discussed.•Findings suggest room for improvement and need to also consider validity.
Functional magnetic resonance imaging (fMRI) studies typically collapse data from many subjects, but brain functional organization varies between individuals. Here we establish that this individual ...variability is both robust and reliable, using data from the Human Connectome Project to demonstrate that functional connectivity profiles act as a 'fingerprint' that can accurately identify subjects from a large group. Identification was successful across scan sessions and even between task and rest conditions, indicating that an individual's connectivity profile is intrinsic, and can be used to distinguish that individual regardless of how the brain is engaged during imaging. Characteristic connectivity patterns were distributed throughout the brain, but the frontoparietal network emerged as most distinctive. Furthermore, we show that connectivity profiles predict levels of fluid intelligence: the same networks that were most discriminating of individuals were also most predictive of cognitive behavior. Results indicate the potential to draw inferences about single subjects on the basis of functional connectivity fMRI.
Neuroimaging is a fast-developing research area in which anatomical and functional images of human brains are collected using techniques such as functional magnetic resonance imaging (fMRI), ...diffusion tensor imaging (DTI), and electroencephalography (EEG). Technical advances and large-scale data sets have allowed for the development of models capable of predicting individual differences in traits and behavior using brain connectivity measures derived from neuroimaging data. Here, we present connectome-based predictive modeling (CPM), a data-driven protocol for developing predictive models of brain-behavior relationships from connectivity data using cross-validation. This protocol includes the following steps: (i) feature selection, (ii) feature summarization, (iii) model building, and (iv) assessment of prediction significance. We also include suggestions for visualizing the most predictive features (i.e., brain connections). The final result should be a generalizable model that takes brain connectivity data as input and generates predictions of behavioral measures in novel subjects, accounting for a considerable amount of the variance in these measures. It has been demonstrated that the CPM protocol performs as well as or better than many of the existing approaches in brain-behavior prediction. As CPM focuses on linear modeling and a purely data-driven approach, neuroscientists with limited or no experience in machine learning or optimization will find it easy to implement these protocols. Depending on the volume of data to be processed, the protocol can take 10-100 min for model building, 1-48 h for permutation testing, and 10-20 min for visualization of results.
Although attention plays a ubiquitous role in perception and cognition, researchers lack a simple way to measure a person's overall attentional abilities. Because behavioral measures are diverse and ...difficult to standardize, we pursued a neuromarker of an important aspect of attention, sustained attention, using functional magnetic resonance imaging. To this end, we identified functional brain networks whose strength during a sustained attention task predicted individual differences in performance. Models based on these networks generalized to previously unseen individuals, even predicting performance from resting-state connectivity alone. Furthermore, these same models predicted a clinical measure of attention--symptoms of attention deficit hyperactivity disorder--from resting-state connectivity in an independent sample of children and adolescents. These results demonstrate that whole-brain functional network strength provides a broadly applicable neuromarker of sustained attention.
Recent work has begun to relate individual differences in brain functional organization to human behaviors and cognition, but the best brain state to reveal such relationships remains an open ...question. In two large, independent data sets, we here show that cognitive tasks amplify trait-relevant individual differences in patterns of functional connectivity, such that predictive models built from task fMRI data outperform models built from resting-state fMRI data. Further, certain tasks consistently yield better predictions of fluid intelligence than others, and the task that generates the best-performing models varies by sex. By considering task-induced brain state and sex, the best-performing model explains over 20% of the variance in fluid intelligence scores, as compared to <6% of variance explained by rest-based models. This suggests that identifying and inducing the right brain state in a given group can better reveal brain-behavior relationships, motivating a paradigm shift from rest- to task-based functional connectivity analyses.
Synaptic loss and deficits in functional connectivity are hypothesized to contribute to symptoms associated with major depressive disorder (MDD) and post-traumatic stress disorder (PTSD). The ...synaptic vesicle glycoprotein 2A (SV2A) can be used to index the number of nerve terminals, an indirect estimate of synaptic density. Here, we used positron emission tomography (PET) with the SV2A radioligand
CUCB-J to examine synaptic density in n = 26 unmedicated individuals with MDD, PTSD, or comorbid MDD/PTSD. The severity of depressive symptoms was inversely correlated with SV2A density, and individuals with high levels of depression showing lower SV2A density compared to healthy controls (n = 21). SV2A density was also associated with aberrant network function, as measured by magnetic resonance imaging (MRI) functional connectivity. This is the first in vivo evidence linking lower synaptic density to network alterations and symptoms of depression. Our findings provide further incentive to evaluate interventions that restore synaptic connections to treat depression.
Pioneering work in human neuroscience has relied on the ability to map brain function using task-based fMRI, but the empirical validity of these inferential methods is still being characterized. A ...recent landmark study by Eklund and colleagues showed that popular multiple comparison corrections based on cluster extent suffer from unexpectedly low specificity (i.e., high false positive rate). Yet that study’s focus on specificity, while important, is incomplete. The validity of a method depends also on its sensitivity (i.e., true positive rate or power), yet the sensitivity of cluster correction remains poorly understood. Here, we assessed the sensitivity of gold-standard nonparametric cluster correction by resampling real data from five tasks in the Human Connectome Project and comparing results with those from the full “ground truth” datasets (n = 480–493). Critically, we found that sensitivity after correction is lower than may be practical for many fMRI applications. In particular, sensitivity to medium-sized effects (|Cohen’s d| = 0.5) was less than 20% across tasks on average, about three times smaller than without any correction. Furthermore, cluster extent correction exhibited a spatial bias in sensitivity that was independent of effect size. In comparison, correction based on the Threshold-Free Cluster Enhancement (TFCE) statistic approximately doubled sensitivity across tasks but increased spatial bias. These results suggest that we have, until now, only measured the tip of the iceberg in the activation-mapping literature due to our goal of limiting the familywise error rate through cluster extent-based inference. There is a need to revise our practices to improve sensitivity; we therefore conclude with a list of modern strategies to boost sensitivity while maintaining respectable specificity in future investigations.
The goal of human brain mapping has long been to delineate the functional subunits in the brain and elucidate the functional role of each of these brain regions. Recent work has focused on ...whole-brain parcellation of functional Magnetic Resonance Imaging (fMRI) data to identify these subunits and create a functional atlas. Functional connectivity approaches to understand the brain at the network level require such an atlas to assess connections between parcels and extract network properties. While no single functional atlas has emerged as the dominant atlas to date, there remains an underlying assumption that such an atlas exists. Using fMRI data from a highly sampled subject as well as two independent replication data sets, we demonstrate that functional parcellations based on fMRI connectivity data reconfigure substantially and in a meaningful manner, according to brain state.
•Functional parcel boundaries reconfigure with cognitive state.•Parcel reconfigurations are robust and reliable across sessions and subjects.•Parcel sizes can significantly predict task condition and task performance.•State-dependent functional atlases have implications for functional connectivity.
•We propose a smooth graph learning framework for functional connectivity estimation.•Smooth graph functional connectivity provides a natural and sparse graph representation.•Improves ...connectome-based identification (fingerprinting).•Smooth graph functional connectivity predicts fluid intelligence.
Functional connectivity (FC) estimated from functional magnetic resonance imaging (fMRI) signals is important in understanding neural representation and information processing in cortical networks. However, due to a lack of ”ground truth” FC pattern, the reliability and robustness of FC estimates are usually examined in downstream FC analysis tasks, such as performing participant’s identification (also known as ”fingerprinting”). In this paper, we propose to learn FC via a smooth graph learning framework. In particular, we treat each time frame of the fMRI time series as a graph signal on an underlying functional brain graph, and estimate the smooth graph functional connectivity (SGFC) by learning the weighted graph adjacency matrix based on graph signal smoothness assumption. We demonstrate that our approach gives rise to a natural and sparse graph representation of FC from which reliable graph measures can be extracted. Reliability of SGFC is evaluated in the context of fingerprinting and compared to correlation FC (CFC). SGFC achieves higher fingerprinting accuracy across several different experiment settings; the improvement is even more significant when a shorter fMRI scanning length is used for FC estimation. In addition to being reliable, we also validate the cognitive relevance of SGFC by using it to predict fluid intelligence. Finally, in evaluating topological measures of the sparse graph, SGFC reveals a more small-world and modular structure compared to CFC. Together, our results suggest that the smooth graph learning framework produces a naturally sparse, reliable, and cognitive-relevant representation of functional connectivity.
The large-scale organization of the brain has features of complex networks that can be quantified using network measures from graph theory. However, many network measures were designed to be ...calculated on binary graphs, whereas functional brain organization is typically inferred from a continuous measure of correlations in temporal signal between brain regions. Thresholding is a necessary step to use binary graphs derived from functional connectivity data. However, there is no current consensus on what threshold to use, and network measures and group contrasts may be unstable across thresholds. Nevertheless, whole-brain network analyses are being applied widely with findings typically reported at an arbitrary threshold or range of thresholds. This study sought to evaluate the stability of network measures across thresholds in a large resting state functional connectivity dataset. Network measures were evaluated across absolute (correlation-based) and proportional (sparsity-based) thresholds, and compared between sex and age groups. Overall, network measures were found to be unstable across absolute thresholds. For example, the direction of group differences in a given network measure may change depending on the threshold. Network measures were found to be more stable across proportional thresholds. These results demonstrate that caution should be used when applying thresholds to functional connectivity data and when interpreting results from binary graph models.
•Network measures were found to be unstable across absolute thresholds.•For instance, the direction of significant group differences can reverse across thresholds.•Network measures were found to be more stable across proportional thresholds.•Caution should be used when applying thresholds to functional connectivity data.
PDF
