A large number of mathematical models have been proposed to describe the measured signal in diffusion‐weighted (DW) magnetic resonance imaging (MRI). However, model comparison to date focuses only on ...specific subclasses, e.g. compartment models or signal models, and little or no information is available in the literature on how performance varies among the different types of models. To address this deficiency, we organized the ‘White Matter Modeling Challenge’ during the International Symposium on Biomedical Imaging (ISBI) 2015 conference. This competition aimed to compare a range of different kinds of models in their ability to explain a large range of measurable in vivo DW human brain data. Specifically, we assessed the ability of models to predict the DW signal accurately for new diffusion gradients and b values. We did not evaluate the accuracy of estimated model parameters, as a ground truth is hard to obtain. We used the Connectome scanner at the Massachusetts General Hospital, using gradient strengths of up to 300 mT/m and a broad set of diffusion times. We focused on assessing the DW signal prediction in two regions: the genu in the corpus callosum, where the fibres are relatively straight and parallel, and the fornix, where the configuration of fibres is more complex. The challenge participants had access to three‐quarters of the dataset and their models were ranked on their ability to predict the remaining unseen quarter of the data. The challenge provided a unique opportunity for a quantitative comparison of diverse methods from multiple groups worldwide. The comparison of the challenge entries reveals interesting trends that could potentially influence the next generation of diffusion‐based quantitative MRI techniques. The first is that signal models do not necessarily outperform tissue models; in fact, of those tested, tissue models rank highest on average. The second is that assuming a non‐Gaussian (rather than purely Gaussian) noise model provides little improvement in prediction of unseen data, although it is possible that this may still have a beneficial effect on estimated parameter values. The third is that preprocessing the training data, here by omitting signal outliers, and using signal‐predicting strategies, such as bootstrapping or cross‐validation, could benefit the model fitting. The analysis in this study provides a benchmark for other models and the data remain available to build up a more complete comparison in the future.
The ISBI 2015 ‘White Matter Modeling Challenge’ provided a unique opportunity for a quantitative comparison of diverse models from groups worldwide. Competitors had partial access to data (
|Gmax|=300 mT/m, many Δ) and ranking was based on missing data prediction. Ranking reveals that signal models do not necessarily outperform tissue models. Additionally, assuming a non‐Gaussian noise model provides little benefit, but shows that data preprocessing and signal predicting strategies could benefit the model fitting. With our publicly available data, this analysis provides a benchmark for other future models.
A large number of mathematical models have been proposed to describe the measured signal in diffusion-weighted (DW) magnetic resonance imaging (MRI). However, model comparison to date focuses only on ...specific subclasses, and little or no information is available in the literature on how performance varies among the different types of models. To address this deficiency we organized the " White Matter Modeling Challenge " during the International Symposium on Biomedical Imaging (ISBI) 2015 conference. This competition aimed to compare a range of different kinds of model in their ability to explain measured in vivo DW human brain data. We focus specifically on the challenge of explaining a large range of measurable data. We used the Connectome scanner at the Massachusetts General Hospital, using gradients strengths of up to 300 mT/m and a broad set of diffusion times. We focused on assessing the DW signal prediction in two regions: the genu in the corpus callosum, where the fibres are relatively straight and parallel, and the fornix, where the configuration of fibres is more complex. The challenge participants had access to three-quarters of the dataset, and their models were ranked on their ability to predict the remaining unseen quarter of the data. The challenge provided a unique opportunity for quantitative comparison of a diverse set of methods from multiple groups worldwide. The comparison of the challenge entries reveals important trends and conclusions that influence the next generation of diffusion-based quantitative MRI techniques. The first is that signal models do not necessarily outperform tissue models; in fact tissue models on average rank highest of those tested. The second is that assuming a non-Gaussian (rather than a purely Gaussian) noise model provides little benefit. The third is that preprocessing the training data (here, omitting signal outliers) and using signal predicting strategies, such as bootstrapping or cross-validation, could benefit the model fitting. The analysis in this study provides a benchmark for other models and the data remains available to build up a more complete comparison over future years.
A large number of mathematical models have been proposed to describe the measured signal in diffusion-weighted (DW) magnetic resonance imaging (MRI) and infer properties about the white matter ...microstructure. However, a head-to-head comparison of DW-MRI models is critically missing in the field. To address this deficiency, we organized the "White Matter Modeling Challenge" during the International Symposium on Biomedical Imaging (ISBI) 2015 conference. This competition aimed at identifying the DW-MRI models that best predict unseen DW data. in vivo DW-MRI data was acquired on the Connectom scanner at the A.A.Martinos Center (Massachusetts General Hospital) using gradients strength of up to 300 mT/m and a broad set of diffusion times. We focused on assessing the DW signal prediction in two regions: the genu in the corpus callosum, where the fibres are relatively straight and parallel, and the fornix, where the configuration of fibres is more complex. The challenge participants had access to three-quarters of the whole dataset, and their models were ranked on their ability to predict the remaining unseen quarter of data. In this paper we provide both an overview and a more in-depth description of each evaluated model, report the challenge results, and infer trends about the model characteristics that were associated with high model ranking. This work provides a much needed benchmark for DW-MRI models. The acquired data and model details for signal prediction evaluation are provided online to encourage a larger scale assessment of diffusion models in the future.
PDF
