Detailed whole brain segmentation is an essential quantitative technique in medical image analysis, which provides a non-invasive way of measuring brain regions from a clinical acquired structural ...magnetic resonance imaging (MRI). Recently, deep convolution neural network (CNN) has been applied to whole brain segmentation. However, restricted by current GPU memory, 2D based methods, downsampling based 3D CNN methods, and patch-based high-resolution 3D CNN methods have been the de facto standard solutions. 3D patch-based high resolution methods typically yield superior performance among CNN approaches on detailed whole brain segmentation (>100 labels), however, whose performance are still commonly inferior compared with state-of-the-art multi-atlas segmentation methods (MAS) due to the following challenges: (1) a single network is typically used to learn both spatial and contextual information for the patches, (2) limited manually traced whole brain volumes are available (typically less than 50) for training a network. In this work, we propose the spatially localized atlas network tiles (SLANT) method to distribute multiple independent 3D fully convolutional networks (FCN) for high-resolution whole brain segmentation. To address the first challenge, multiple spatially distributed networks were used in the SLANT method, in which each network learned contextual information for a fixed spatial location. To address the second challenge, auxiliary labels on 5111 initially unlabeled scans were created by multi-atlas segmentation for training. Since the method integrated multiple traditional medical image processing methods with deep learning, we developed a containerized pipeline to deploy the end-to-end solution. From the results, the proposed method achieved superior performance compared with multi-atlas segmentation methods, while reducing the computational time from >30 h to 15 min. The method has been made available in open source (https://github.com/MASILab/SLANTbrainSeg).
Display omitted
•SLANT method distributes multiple independent 3D deep networks.•SLANT was proposed for high-resolution whole brain segmentation.•Better than multi-atlas segmentation, while reducing time to 15 minutes.•Auxiliary labels on 5111 initially unlabeled scans were used for training.•The method has been made in Docker and available in open source.
•We propose a novel Faster Mean-shift algorithm, which accelerates the embedding clustering based one-stage holistic cell instance segmentation and tracking.•The proposed Faster Mean-shift achieved ...7-10 times speed-up compared with the state-of-the-art embedding based cell instance segmentation and tracking algorithm.•We propose the new online seed optimization policy (OSOP) and early stopping strategy to achieve the best computational speed with optimized memory consumption, compared with previous GPU-based benchmarks.•Comprehensive simulations for in-depth theoretical analysis, as well as empirical validations on four ISBI cell tracking challenge data-set have been performed in this study.
Display omitted
Recently, single-stage embedding based deep learning algorithms gain increasing attention in cell segmentation and tracking. Compared with the traditional “segment-then-associate” two-stage approach, a single-stage algorithm not only simultaneously achieves consistent instance cell segmentation and tracking but also gains superior performance when distinguishing ambiguous pixels on boundaries and overlaps. However, the deployment of an embedding based algorithm is restricted by slow inference speed (e.g., ≈1–2 min per frame). In this study, we propose a novel Faster Mean-shift algorithm, which tackles the computational bottleneck of embedding based cell segmentation and tracking. Different from previous GPU-accelerated fast mean-shift algorithms, a new online seed optimization policy (OSOP) is introduced to adaptively determine the minimal number of seeds, accelerate computation, and save GPU memory. With both embedding simulation and empirical validation via the four cohorts from the ISBI cell tracking challenge, the proposed Faster Mean-shift algorithm achieved 7–10 times speedup compared to the state-of-the-art embedding based cell instance segmentation and tracking algorithm. Our Faster Mean-shift algorithm also achieved the highest computational speed compared to other GPU benchmarks with optimized memory consumption. The Faster Mean-shift is a plug-and-play model, which can be employed on other pixel embedding based clustering inference for medical image analysis. (Plug-and-play model is publicly available: https://github.com/masqm/Faster-Mean-Shift)
Transformer, one of the latest technological advances of deep learning, has gained prevalence in natural language processing or computer vision. Since medical imaging bear some resemblance to ...computer vision, it is natural to inquire about the status quo of Transformers in medical imaging and ask the question: can the Transformer models transform medical imaging? In this paper, we attempt to make a response to the inquiry. After a brief introduction of the fundamentals of Transformers, especially in comparison with convolutional neural networks (CNNs), and highlighting key defining properties that characterize the Transformers, we offer a comprehensive review of the state-of-the-art Transformer-based approaches for medical imaging and exhibit current research progresses made in the areas of medical image segmentation, recognition, detection, registration, reconstruction, enhancement, etc. In particular, what distinguishes our review lies in its organization based on the Transformer’s key defining properties, which are mostly derived from comparing the Transformer and CNN, and its type of architecture, which specifies the manner in which the Transformer and CNN are combined, all helping the readers to best understand the rationale behind the reviewed approaches. We conclude with discussions of future perspectives.
•Can the Transformer models transform medical imaging? We attempt to make a response to the inquiry.•We make a comparative review of key properties and current progresses of medical vision transformers.•We present future perspectives of medical vision transformers.
Diffusion magnetic resonance imaging (dMRI) is widely used to probe tissue microstructure, and is currently the only non-invasive way to measure the brain's fiber architecture. While a large number ...of approaches to recover the intra-voxel fiber structure have been utilized in the scientific community, a direct, 3D, quantitative validation of these methods against relevant histological fiber geometries is lacking. In this study, we investigate how well different high angular resolution diffusion imaging (HARDI) models and reconstruction methods predict the ground-truth histologically defined fiber orientation distribution (FOD), as well as investigate their behavior over a range of physical and experimental conditions. The dMRI methods tested include constrained spherical deconvolution (CSD), Q-ball imaging (QBI), diffusion orientation transform (DOT), persistent angular structure (PAS), and neurite orientation dispersion and density imaging (NODDI) methods. Evaluation criteria focus on overall agreement in FOD shape, correct assessment of the number of fiber populations, and angular accuracy in orientation. In addition, we make comparisons of the histological orientation dispersion with the fiber spread determined from the dMRI methods. As a general result, no HARDI method outperformed others in all quality criteria, with many showing tradeoffs in reconstruction accuracy. All reconstruction techniques describe the overall continuous angular structure of the histological FOD quite well, with good to moderate correlation (median angular correlation coefficient > 0.70) in both single- and multiple-fiber voxels. However, no method is consistently successful at extracting discrete measures of the number and orientations of FOD peaks. The major inaccuracies of all techniques tend to be in extracting local maxima of the FOD, resulting in either false positive or false negative peaks. Median angular errors are ∼10° for the primary fiber direction and ∼20° for the secondary fiber, if present. For most methods, these results did not vary strongly over a wide range of acquisition parameters (number of diffusion weighting directions and b value). Regardless of acquisition parameters, all methods show improved successes at resolving multiple fiber compartments in a voxel when fiber populations cross at near-orthogonal angles, with no method adequately capturing low to moderate angle (<60°) crossing fibers. Finally, most methods are limited in their ability to capture orientation dispersion, resulting in low to moderate, yet statistically significant, correlation with histologically-derived dispersion with both HARDI and NODDI methodologies. Together, these results provide quantitative measures of the reliability and limitations of dMRI reconstruction methods and can be used to identify relative advantages of competing approaches as well as potential strategies for improving accuracy.
•3D histological validation of diffusion MRI measures of fiber orientation.•All methods capture the overall structure of the FOD quite well.•Most inaccuracies occur when extracting discrete peaks from the FOD.•No method consistently resolves fibers crossing at low to moderate angles.•Measures of dispersion show modest correlation with histological measures.
Diffusion magnetic resonance images may suffer from geometric distortions due to susceptibility induced off resonance fields, which cause geometric mismatch with anatomical images and ultimately ...affect subsequent quantification of microstructural or connectivity indices. State-of-the art diffusion distortion correction methods typically require data acquired with reverse phase encoding directions, resulting in varying magnitudes and orientations of distortion, which allow estimation of an undistorted volume. Alternatively, additional field maps acquisitions can be used along with sequence information to determine warping fields. However, not all imaging protocols include these additional scans and cannot take advantage of state-of-the art distortion correction. To avoid additional acquisitions, structural MRI (undistorted scans) can be used as registration targets for intensity driven correction. In this study, we aim to (1) enable susceptibility distortion correction with historical and/or limited diffusion datasets that do not include specific sequences for distortion correction and (2) avoid the computationally intensive registration procedure typically required for distortion correction using structural scans. To achieve these aims, we use deep learning (3D U-nets) to synthesize an undistorted b0 image that matches geometry of structural T1w images and intensity contrasts from diffusion images. Importantly, the training dataset is heterogenous, consisting of varying acquisitions of both structural and diffusion. We apply our approach to a withheld test set and show that distortions are successfully corrected after processing. We quantitatively evaluate the proposed distortion correction and intensity-based registration against state-of-the-art distortion correction (FSL topup). The results illustrate that the proposed pipeline results in b0 images that are geometrically similar to non-distorted structural images, and more closely match state-of-the-art correction with additional acquisitions. In addition, we show generalizability of the proposed approach to datasets that were not in the original training / validation / testing datasets. These datasets included varying populations, contrasts, resolutions, and magnitudes and orientations of distortion and show efficacious distortion correction. The method is available as a Singularity container, source code, and an executable trained model to facilitate evaluation.
Data-driven machine learning (ML) has emerged as a promising approach for building accurate and robust statistical models from medical data, which is collected in huge volumes by modern healthcare ...systems. Existing medical data is not fully exploited by ML primarily because it sits in data silos and privacy concerns restrict access to this data. However, without access to sufficient data, ML will be prevented from reaching its full potential and, ultimately, from making the transition from research to clinical practice. This paper considers key factors contributing to this issue, explores how federated learning (FL) may provide a solution for the future of digital health and highlights the challenges and considerations that need to be addressed.
Executive cognitive functions, including working memory, cognitive flexibility, and inhibition, are impaired in schizophrenia. Executive functions rely on coordinated information processing between ...the prefrontal cortex (PFC) and thalamus, particularly the mediodorsal nucleus. This raises the possibility that anatomical connectivity between the PFC and mediodorsal thalamus may be 1) reduced in schizophrenia and 2) related to deficits in executive function. The current investigation tested these hypotheses.
Forty-five healthy subjects and 62 patients with a schizophrenia spectrum disorder completed a battery of tests of executive function and underwent diffusion-weighted imaging. Probabilistic tractography was used to quantify anatomical connectivity between six cortical regions, including PFC, and the thalamus. Thalamocortical anatomical connectivity was compared between healthy subjects and patients with schizophrenia using region-of-interest and voxelwise approaches, and the association between PFC-thalamic anatomical connectivity and severity of executive function impairment was examined in patients.
Anatomical connectivity between the thalamus and PFC was reduced in schizophrenia. Voxelwise analysis localized the reduction to areas of the mediodorsal thalamus connected to lateral PFC. Reduced PFC-thalamic connectivity in schizophrenia correlated with impaired working memory but not cognitive flexibility and inhibition. In contrast to reduced PFC-thalamic connectivity, thalamic connectivity with somatosensory and occipital cortices was increased in schizophrenia.
The results are consistent with models implicating disrupted PFC-thalamic connectivity in the pathophysiology of schizophrenia and mechanisms of cognitive impairment. PFC-thalamic anatomical connectivity may be an important target for procognitive interventions. Further work is needed to determine the implications of increased thalamic connectivity with sensory cortex.
Diffusion MRI fiber tractography has been increasingly used to map the structural connectivity of the human brain. However, this technique is not without limitations; for example, there is a growing ...concern over anatomically correlated bias in tractography findings. In this study, we demonstrate that there is a bias for fiber tracking algorithms to terminate preferentially on gyral crowns, rather than the banks of sulci. We investigate this issue by comparing diffusion MRI (dMRI) tractography with equivalent measures made on myelin‐stained histological sections. We begin by investigating the orientation and trajectories of axons near the white matter/gray matter boundary, and the density of axons entering the cortex at different locations along gyral blades. These results are compared with dMRI orientations and tract densities at the same locations, where we find a significant gyral bias in many gyral blades across the brain. This effect is shown for a range of tracking algorithms, both deterministic and probabilistic, and multiple diffusion models, including the diffusion tensor and a high angular resolution diffusion imaging technique. Additionally, the gyral bias occurs for a range of diffusion weightings, and even for very high‐resolution datasets. The bias could significantly affect connectivity results using the current generation of tracking algorithms.
The management of indeterminate pulmonary nodules (IPNs) remains challenging, resulting in invasive procedures and delays in diagnosis and treatment. Strategies to decrease the rate of unnecessary ...invasive procedures and optimize surveillance regimens are needed.
To develop and validate a deep learning method to improve the management of IPNs.
A Lung Cancer Prediction Convolutional Neural Network model was trained using computed tomography images of IPNs from the National Lung Screening Trial, internally validated, and externally tested on cohorts from two academic institutions.
The areas under the receiver operating characteristic curve in the external validation cohorts were 83.5% (95% confidence interval CI, 75.4-90.7%) and 91.9% (95% CI, 88.7-94.7%), compared with 78.1% (95% CI, 68.7-86.4%) and 81.9 (95% CI, 76.1-87.1%), respectively, for a commonly used clinical risk model for incidental nodules. Using 5% and 65% malignancy thresholds defining low- and high-risk categories, the overall net reclassifications in the validation cohorts for cancers and benign nodules compared with the Mayo model were 0.34 (Vanderbilt) and 0.30 (Oxford) as a rule-in test, and 0.33 (Vanderbilt) and 0.58 (Oxford) as a rule-out test. Compared with traditional risk prediction models, the Lung Cancer Prediction Convolutional Neural Network was associated with improved accuracy in predicting the likelihood of disease at each threshold of management and in our external validation cohorts.
This study demonstrates that this deep learning algorithm can correctly reclassify IPNs into low- or high-risk categories in more than a third of cancers and benign nodules when compared with conventional risk models, potentially reducing the number of unnecessary invasive procedures and delays in diagnosis.
PDF
