Spinal cord segmentation provides measures of atrophy and facilitates group analysis via inter-subject correspondence. Automatizing this procedure enables studies with large throughput and minimizes ...user bias. Although several automatic segmentation methods exist, they are often restricted in terms of image contrast and field-of-view. This paper presents a new automatic segmentation method (PropSeg) optimized for robustness, accuracy and speed. The algorithm is based on the propagation of a deformable model and is divided into three parts: firstly, an initialization step detects the spinal cord position and orientation using a circular Hough transform on multiple axial slices rostral and caudal to the starting plane and builds an initial elliptical tubular mesh. Secondly, a low-resolution deformable model is propagated along the spinal cord. To deal with highly variable contrast levels between the spinal cord and the cerebrospinal fluid, the deformation is coupled with a local contrast-to-noise adaptation at each iteration. Thirdly, a refinement process and a global deformation are applied on the propagated mesh to provide an accurate segmentation of the spinal cord. Validation was performed in 15 healthy subjects and two patients with spinal cord injury, using T1- and T2-weighted images of the entire spinal cord and on multiecho T2*-weighted images. Our method was compared against manual segmentation and against an active surface method. Results show high precision for all the MR sequences. Dice coefficients were 0.9 for the T1- and T2-weighted cohorts and 0.86 for the T2*-weighted images. The proposed method runs in less than 1min on a normal computer and can be used to quantify morphological features such as cross-sectional area along the whole spinal cord.
•Automatic spinal cord segmentation method using propagated deformable models•Works on T1- and T2-weighted MR images and any field of view•High accuracy, similar to the inter-rater variability•Robust towards local lack of contrast between spinal cord and CSF
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UL, UM, UPCLJ, UPUK
3.
Deep learning workflow in radiology: a primer Montagnon, Emmanuel; Cerny, Milena; Cadrin-Chênevert, Alexandre ...
Insights into imaging,
02/2020, Volume:
11, Issue:
1
Journal Article
Peer reviewed
Open access
Interest for deep learning in radiology has increased tremendously in the past decade due to the high achievable performance for various computer vision tasks such as detection, segmentation, ...classification, monitoring, and prediction. This article provides step-by-step practical guidance for conducting a project that involves deep learning in radiology, from defining specifications, to deployment and scaling. Specifically, the objectives of this article are to provide an overview of clinical use cases of deep learning, describe the composition of multi-disciplinary team, and summarize current approaches to patient, data, model, and hardware selection. Key ideas will be illustrated by examples from a prototypical project on imaging of colorectal liver metastasis. This article illustrates the workflow for liver lesion detection, segmentation, classification, monitoring, and prediction of tumor recurrence and patient survival. Challenges are discussed, including ethical considerations, cohorting, data collection, anonymization, and availability of expert annotations. The practical guidance may be adapted to any project that requires automated medical image analysis.
Full text
Available for:
IZUM, KILJ, NUK, PILJ, PNG, SAZU, UL, UM, UPUK
•Image segmentation pipeline based on Fully Convolutional Networks (FCN) and ResNets is proposed.•FCN can serve as a pre-processor to normalize medical imaging input data.•A trainable FCN is an ...alternative to hand-designed, modality specific pre-processing steps.•Our pipeline obtains or matches state-of-the-art performance on 3 segmentation datasets.
Display omitted
In this paper, we introduce a simple, yet powerful pipeline for medical image segmentation that combines Fully Convolutional Networks (FCNs) with Fully Convolutional Residual Networks (FC-ResNets). We propose and examine a design that takes particular advantage of recent advances in the understanding of both Convolutional Neural Networks as well as ResNets. Our approach focuses upon the importance of a trainable pre-processing when using FC-ResNets and we show that a low-capacity FCN model can serve as a pre-processor to normalize medical input data. In our image segmentation pipeline, we use FCNs to obtain normalized images, which are then iteratively refined by means of a FC-ResNet to generate a segmentation prediction. As in other fully convolutional approaches, our pipeline can be used off-the-shelf on different image modalities. We show that using this pipeline, we exhibit state-of-the-art performance on the challenging Electron Microscopy benchmark, when compared to other 2D methods. We improve segmentation results on CT images of liver lesions, when contrasting with standard FCN methods. Moreover, when applying our 2D pipeline on a challenging 3D MRI prostate segmentation challenge we reach results that are competitive even when compared to 3D methods. The obtained results illustrate the strong potential and versatility of the pipeline by achieving accurate segmentations on a variety of image modalities and different anatomical regions.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UL, UM, UPCLJ, UPUK, ZRSKP
Magnetic resonance imaging (MRI) plays a critical role in the planning and monitoring of hepatocellular carcinomas (HCC) treated with locoregional therapies, in order to assess disease progression or ...recurrence. Dynamic contrast-enhanced (DCE)-MRI sequences offer temporal data on tumor enhancement characteristics which has strong prognostic value. Yet, predicting follow-up DCE-MR images from which tumor enhancement and viability can be measured, before treatment of HCC actually begins, remains an unsolved problem given the complexity of spatial and temporal information. We propose an approach to predict future DCE-MRI examinations following transarterial chemoembolization (TACE) by learning the spatio-temporal features related to HCC response from pre-TACE images. A novel Spatial-Temporal Discriminant Graph Neural Network (STDGNN) based on graph convolutional networks is presented. First, embeddings of viable, equivocal and non-viable HCCs are separated within a joint low-dimensional latent space, which is created using a discriminant neural network representing tumor-specific features. Spatial tumoral features from independent MRI volumes are then extracted with a structural branch, while dynamic features are extracted from the multi-phase sequence with a separate temporal branch. The model extracts spatio-temporal features by a joint minimization of the network branches. At testing, a pre-TACE diagnostic DCE-MRI is embedded on the discriminant spatio-temporal latent space, which is then translated to the follow-up domain space, thus allowing to predict the post-TACE DCE-MRI describing HCC treatment response. A dataset of 366 HCC's from liver cancer patients was used to train and test the model using DCE-MRI examinations with associated pathological outcomes, with the spatio-temporal framework yielding 93.5% classification accuracy in response identification, and generating follow-up images yielding insignificant differences in perfusion parameters compared to ground-truth post-TACE examinations.
Full text
Available for:
DOBA, IZUM, KILJ, NUK, PILJ, PNG, SAZU, SIK, UILJ, UKNU, UL, UM, UPUK
Purpose
During MR-guided neurosurgical procedures, several factors may limit the acquisition of additional MR sequences, which are needed by neurosurgeons to adjust surgical plans or ensure complete ...tumor resection. Automatically synthesized MR contrasts generated from other available heterogeneous MR sequences could alleviate timing constraints.
Methods
We propose a new multimodal MR synthesis approach leveraging a combination of MR modalities presenting glioblastomas to generate an additional modality. The proposed learning approach relies on a least square GAN (LSGAN) using an unsupervised contrastive learning strategy. We incorporate a contrastive encoder, which extracts an invariant contrastive representation from augmented pairs of the generated and real target MR contrasts. This contrastive representation describes a pair of features for each input channel, allowing to regularize the generator to be invariant to the high-frequency orientations. Moreover, when training the generator, we impose on the LSGAN loss another term reformulated as the combination of a reconstruction and a novel perception loss based on a pair of features.
Results
When compared to other multimodal MR synthesis approaches evaluated on the BraTS’18 brain dataset, the model yields the highest Dice score with
0.748
±
0.04
and achieves the lowest variability information of
2.1
±
1.11
, with a probability rand index score of
0.84
±
0.03
and a global consistency error of
0.17
±
0.04
.
Conclusion
The proposed model allows to generate reliable MR contrasts with enhanced tumors on the synthesized image using a brain tumor dataset (BraTS’18). In future work, we will perform a clinical evaluation of residual tumor segmentations during MR-guided neurosurgeries, where limited MR contrasts will be acquired during the procedure.
Full text
Available for:
EMUNI, FIS, FZAB, GEOZS, GIS, IJS, IMTLJ, KILJ, KISLJ, MFDPS, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, SBMB, SBNM, UKNU, UL, UM, UPUK, VKSCE, ZAGLJ
In radiation oncology, predicting patient risk stratification allows specialization of therapy intensification as well as selecting between systemic and regional treatments, all of which helps to ...improve patient outcome and quality of life. Deep learning offers an advantage over traditional radiomics for medical image processing by learning salient features from training data originating from multiple datasets. However, while their large capacity allows to combine high-level medical imaging data for outcome prediction, they lack generalization to be used across institutions. In this work, a pseudo-volumetric convolutional neural network with a deep preprocessor module and self-attention (PreSANet) is proposed for the prediction of distant metastasis, locoregional recurrence, and overall survival occurrence probabilities within the 10 year follow-up time frame for head and neck cancer patients with squamous cell carcinoma. The model is capable of processing multi-modal inputs of variable scan length, as well as integrating patient data in the prediction model. These proposed architectural features and additional modalities all serve to extract additional information from the available data when availability to additional samples is limited. This model was trained on the public Cancer Imaging Archive Head-Neck-PET-CT dataset consisting of 298 patients undergoing curative radio/chemo-radiotherapy and acquired from 4 different institutions. The model was further validated on an internal retrospective dataset with 371 patients acquired from one of the institutions in the training dataset. An extensive set of ablation experiments were performed to test the utility of the proposed model characteristics, achieving an AUROC of Formula: see text, Formula: see text and Formula: see text for DM, LR and OS respectively on the public TCIA Head-Neck-PET-CT dataset. External validation was performed on a retrospective dataset with 371 patients, achieving Formula: see text AUROC in all outcomes. To test for model generalization across sites, a validation scheme consisting of single site-holdout and cross-validation combining both datasets was used. The mean accuracy across 4 institutions obtained was Formula: see text, Formula: see text and Formula: see text for DM, LR and OS respectively. The proposed model demonstrates an effective method for tumor outcome prediction for multi-site, multi-modal combining both volumetric data and structured patient clinical data.
Full text
Available for:
IZUM, KILJ, NUK, PILJ, PNG, SAZU, UL, UM, UPUK
Purpose
In adolescent idiopathic scoliosis (AIS), non-invasive surgical techniques such as anterior vertebral body tethering (AVBT) enable to treat patients with mild and severe degrees of deformity ...while maintaining lower lumbar motion by avoiding spinal fusion. However, multiple features and characteristics affect the overall patient outcome, notably the 3D spine geometry and bone maturity, but also from decisions taken intra-operatively such as the selected tethered vertebral levels, which makes it difficult to anticipate the patient response.
Methods
We propose here a forecasting method which can be used during AVBT surgery, exploiting the spatio-temporal features extracted from a dynamic networks. The model learns the corrective effect from the spine’s different segments while taking under account the time differences in the initial diagnosis and between the serial acquisitions taken before and during surgery. Clinical parameters are integrated through an attention-based decoder, allowing to associate geometrical features to patient status. Long-term relationships allow to ensure regularity in geometrical curve prediction, using a manifold-based smoothness term to regularize geometrical outputs, capturing the temporal variations of spine correction.
Results
A dataset of 695 3D spine reconstructions was used to train the network, which was evaluated on a hold-out dataset of 72 scoliosis patients using the baseline 3D reconstruction obtained prior to surgery, yielding an overall reconstruction error of
1.8
±
0.8
mm based on pre-identified landmarks on vertebral bodies. The model was also tested prospectively on a separate cohort of 15 AIS patients, demonstrating the integration within the OR theatre.
Conclusion
The proposed predictive network allows to intra-operatively anticipate the geometrical response of the spine to AVBT procedures using the dynamic features.
Full text
Available for:
EMUNI, FIS, FZAB, GEOZS, GIS, IJS, IMTLJ, KILJ, KISLJ, MFDPS, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, SBMB, SBNM, UKNU, UL, UM, UPUK, VKSCE, ZAGLJ
Purpose
Understanding how to classify and quantify three-dimensional (3D) spinal deformities remains an open question in adolescent idiopathic scoliosis. The objective of this study was to perform a ...3D manifold characterization of scoliotic spines demonstrating thoracic deformations using a novel geometric and intuitive statistical tool to determine patterns in pathological cases.
Methods
Personalized 3D reconstructions of thoracic (T)/lumbar (L) spines from a cohort of 170 Lenke Type-1 patients were analyzed with a non-linear manifold embedding algorithm in order to reduce the high-dimensionality of the data, using statistical properties of neighbouring spine models. We extracted sub-groups of the data from the underlying manifold structure using an unsupervised clustering algorithm to understand the inherent distribution and determine classes of pathologies which appear from the low-dimensional space.
Results
For Lenke Type-1 patients, four clusters were detected from the low-dimensional manifold of 3D models: (1) normal kyphosis (T) with hyper-lordosis (L) and high Cobb angles (37 cases), (2) low kyphosis (T) and normal lordosis (L), with high rotation of plane of maximum curvature (55 cases), (3) hypo-kyphotic (T) and hyper-lordosis (L) (21 cases) and (4) hyper-kyphotic (T) with strong vertebral rotation (57 cases). Results show the manifold representation can potentially be useful for classification of 3D spinal pathologies such as idiopathic scoliosis and serve as a tool for understanding the progression of deformities in longitudinal studies.
Conclusions
Quantitative evaluation illustrates that the complex space of spine variability can be modeled by a low-dimensional manifold and shows the existence of an additional hyper-kyphotic subgroup from the cohort of 3D spine reconstructions of Lenke Type-1 patients when compared with previous findings on the 3D classification of spinal deformities.
Full text
Available for:
EMUNI, FIS, FZAB, GEOZS, GIS, IJS, IMTLJ, KILJ, KISLJ, MFDPS, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, SBMB, SBNM, UKNU, UL, UM, UPUK, VKSCE, ZAGLJ
In brain tumor surgery, the quality and safety of the procedure can be impacted by intra-operative tissue deformation, called brain shift. Brain shift can move the surgical targets and other vital ...structures such as blood vessels, thus invalidating the pre-surgical plan. Intra-operative ultrasound (iUS) is a convenient and cost-effective imaging tool to track brain shift and tumor resection. Accurate image registration techniques that update pre-surgical MRI based on iUS are crucial but challenging. The MICCAI Challenge 2018 for Correction of Brain shift with Intra-Operative UltraSound (CuRIOUS2018) provided a public platform to benchmark MRI-iUS registration algorithms on newly released clinical datasets. In this work, we present the data, setup, evaluation, and results of CuRIOUS 2018, which received 6 fully automated algorithms from leading academic and industrial research groups. All algorithms were first trained with the public RESECT database, and then ranked based on a test dataset of 10 additional cases with identical data curation and annotation protocols as the RESECT database. The article compares the results of all participating teams and discusses the insights gained from the challenge, as well as future work.
PDF
