One of the distinct characteristics of radiologists reading multiparametric prostate MR scans, using reporting systems like PI-RADS v2.1, is to score individual types of MR modalities, including ...T2-weighted, diffusion-weighted, and dynamic contrast-enhanced, and then combine these image-modality-specific scores using standardised decision rules to predict the likelihood of clinically significant cancer. This work aims to demonstrate that it is feasible for low-dimensional parametric models to model such decision rules in the proposed Combiner networks, without compromising the accuracy of predicting radiologic labels. First, we demonstrate that either a linear mixture model or a nonlinear stacking model is sufficient to model PI-RADS decision rules for localising prostate cancer. Second, parameters of these combining models are proposed as hyperparameters, weighing independent representations of individual image modalities in the Combiner network training, as opposed to end-to-end modality ensemble. A HyperCombiner network is developed to train a single image segmentation network that can be conditioned on these hyperparameters during inference for much-improved efficiency. Experimental results based on 751 cases from 651 patients compare the proposed rule-modelling approaches with other commonly-adopted end-to-end networks, in this downstream application of automating radiologist labelling on multiparametric MR. By acquiring and interpreting the modality combining rules, specifically the linear-weights or odds ratios associated with individual image modalities, three clinical applications are quantitatively presented and contextualised in the prostate cancer segmentation application, including modality availability assessment, importance quantification and rule discovery.
Total plaque volume (TPV) measured from 3D carotid ultrasound has been shown to be able to predict cardiovascular events and is sensitive in detecting treatment effects. Manual plaque segmentation ...was performed in previous studies to quantify TPV, but is tedious, requires long training times and is prone to observer variability. This article introduces the first 3D direct volume-based level-set algorithm to segment plaques from 3D carotid ultrasound images. The plaque surfaces were first initialized based on the lumen and outer wall boundaries generated by a previously described semi-automatic algorithm and then deformed by a direct three-dimensional sparse field level-set algorithm, which enforced the longitudinal continuity of the segmented plaque surfaces. This is a marked advantage as compared to a previously proposed 2D slice-by-slice plaque segmentation method. In plaque boundary initialization, the previous technique performed a search on lines connecting corresponding point pairs of the outer wall and lumen boundaries. A limitation of this initialization strategy was that an inaccurate initial plaque boundary would be generated if the plaque was not enclosed entirely by the wall and lumen boundaries. A mechanism is proposed to extend the search range in order to capture the entire plaque if the outer wall boundary lies on a weak edge in the 3D ultrasound image. The proposed method was compared with the previously described 2D slice-by-slice plaque segmentation method in 26 three-dimensional carotid ultrasound images containing 27 plaques with volumes ranging from 12.5 to 450.0 mm3. The manually segmented plaque boundaries serve as the surrogate gold standard. Segmentation accuracy was quantified by volume-, area- and distance-based metrics, including absolute plaque volume difference (|ΔPV|), Dice similarity coefficient (DSC), mean and maximum absolute distance (MAD and MAXD). The proposed direct 3D plaque segmentation algorithm was associated with a significantly lower |ΔPV|, MAD and MAXD, and a significantly higher DSC compared to the previously described slice-by-slice algorithm (|ΔPV|:p=0.012, DSC: p=2.1×10−4, MAD: p=1.3×10−4, MAXD: p=5.2×10−4). The proposed 3D volume-based algorithm required 72±22 s to segment a plaque, which is 40% lower than the 2D slice-by-slice algorithm (114±18 s). The proposed automatic plaque segmentation method generates accurate and reproducible boundaries efficiently and will allow for streamlining plaque quantification based on 3D ultrasound images.
Abstract Rapid progression in total plaque area and volume measured from ultrasound images has been shown to be associated with an elevated risk of cardiovascular events. Since atherosclerosis is ...focal and predominantly occurring at bifurcation, biomarkers that are able to quantify the spatial distribution of vessel-wall-plus-plaque thickness ( VET ) change may allow for more sensitive detection of treatment effect. The goal of this paper is to develop simple and sensitive biomarkers to quantify the responsiveness to therapies based on the spatial distribution of VWT-Change on the entire 2D carotid standardized map previously described. Point-wise VWT-Changes computed for each patient was reordered lexicographically to a high-dimensional data node in a graph. A graph-based random walk framework was applied with the novel Weighted Cosine (WCos) similarity function introduced, which was tailored for quantification of responsiveness to therapy. The converging probability of each data node to the VWT regression template in the random walk process served as a scalar descriptor for VWT responsiveness to treatment. The WCos-based biomarker was 14 times more sensitive than the mean VWT-Change in discriminating responsive and unresponsive subjects based on the the p-values obtained in T-tests. The proposed framework was extended to quantify where VWT-Change occurred by including multiple VWT-Change distribution templates representing focal changes at different regions. Experimental results show that the framework was effective in classifying carotid arteries with focal VWT-Change at different locations and may facilitate future investigations to correlate risk of cardiovascular events with the location where focal VWT-Change occurs.
A good semantic segmentation method for visual scene understanding should consider both accuracy and efficiency. However, the existing networks tend to concentrate only on segmentation results but ...not on simplifying the network. As a result, a heavy network will be made and it is difficult to deploy such heavy network on some hardware with limited memory. To address this problem, we in this paper develop a novel architecture by involving the recursive block to reduce parameters and improve prediction, as recursive block can improve performance without introducing new parameters for additional convolutions. In detail, for the purpose of mitigating the difficulty of training recursive block, we have adopted a residual unit to give the data more choices to flow through and utilize concatenation layer to combine the output maps of the recursive convolution layers with same resolution but different field-of-views. As a result, richer semantic information can be included in the feature maps, which is good to achieve satisfying pixel-wise prediction. Meriting from the above strategy, we also extend it to enhance Mask-RCNN for instance segmentation. Extensive simulations based on different benchmark datasets, such as DeepFashion, Cityscapes and PASCAL VOC 2012, show that our method can improve segmentation results as well as reduce the parameters.
Targeted prostate biopsy guided by multiparametric magnetic resonance imaging (mpMRI) detects more clinically significant lesions than conventional systemic biopsy. Lesion segmentation is required ...for planning MRI-targeted biopsies. The requirement for integrating image features available in T2-weighted and diffusion-weighted images poses a challenge in prostate lesion segmentation from mpMRI.
A flexible and efficient multistream fusion encoder is proposed in this work to facilitate the multiscale fusion of features from multiple imaging streams. A patch-based loss function is introduced to improve the accuracy in segmenting small lesions.
The proposed multistream encoder fuses features extracted in the three imaging streams at each layer of the network, thereby allowing improved feature maps to propagate downstream and benefit segmentation performance. The fusion is achieved through a spatial attention map generated by optimally weighting the contribution of the convolution outputs from each stream. This design provides flexibility for the network to highlight image modalities according to their relative influence on the segmentation performance. The encoder also performs multiscale integration by highlighting the input feature maps (low-level features) with the spatial attention maps generated from convolution outputs (high-level features). The Dice similarity coefficient (DSC), serving as a cost function, is less sensitive to incorrect segmentation for small lesions. We address this issue by introducing a patch-based loss function that provides an average of the DSCs obtained from local image patches. This local average DSC is equally sensitive to large and small lesions, as the patch-based DSCs associated with small and large lesions have equal weights in this average DSC.
The framework was evaluated in 931 sets of images acquired in several clinical studies at two centers in Hong Kong and the United Kingdom. In particular, the training, validation, and test sets contain 615, 144, and 172 sets of images, respectively. The proposed framework outperformed single-stream networks and three recently proposed multistream networks, attaining F
scores of 82.2 and 87.6% in the lesion and patient levels, respectively. The average inference time for an axial image was 11.8 ms.
The accuracy and efficiency afforded by the proposed framework would accelerate the MRI interpretation workflow of MRI-targeted biopsy and focal therapies.
This paper describes a framework for vascular image segmentation evaluation. Since the size of vessel wall and plaque burden is defined by the lumen and wall boundaries in vascular segmentation, ...these two boundaries should be considered as a pair in statistical evaluation of a segmentation algorithm. This work proposed statistical metrics to evaluate the difference of local vessel wall thickness (VWT) produced by manual and algorithm-based semi-automatic segmentation methods (ΔT) with the local segmentation standard deviation of the wall and lumen boundaries considered. ΔT was further approximately decomposed into the local wall and lumen boundary differences (ΔW and ΔL respectively) in order to provide information regarding which of the wall and lumen segmentation errors contribute more to the VWT difference. In this study, the lumen and wall boundaries in 3D carotid ultrasound images acquired for 21 subjects were each segmented five times manually and by a level-set segmentation algorithm. The (absolute) difference measures (i.e., ΔT, ΔW, ΔL and their absolute values) and the pooled local standard deviation of manually and algorithmically segmented wall and lumen boundaries were computed for each subject and represented in a 2D standardized map. The local accuracy and variability of the segmentation algorithm at each point can be quantified by the average of these metrics for the whole group of subjects and visualized on the 2D standardized map. Based on the results shown on the 2D standardized map, a variety of strategies, such as adding anchor points and adjusting weights of different forces in the algorithm, can be introduced to improve the accuracy and variability of the algorithm.
The fusion of multiview data sets, in which features of each sample are categorized into distinct groups, is increasingly important in the big data era. Successful multiview learning approaches have ...mechanisms to enforce consensus and/or complementarity among views. This article introduces a framework called the consensus and complementarity-based multiview latent space projection (MVLSP-2C) that enforces both principles simultaneously. Consensus is established by extracting and representing information shared by all views in a shared latent space, whereas complementarity among views is achieved by the representation in view-specific spaces. As the diversity of the multiview feature representation benefits classification performance, MVLSP-2C minimizes the similarity between the shared and view-specific representations, thereby improving diversity. The driving principle of MVLSP-2C is that the latent space representation is obtained by optimally projecting it to match the original feature space representation on a view-by-view basis. Unlike pairwise consensus methods that enforce consistency between two views, matching on a view-by-view basis allows extensions to settings with more than two views. A related and important advantage of this per-view matching design is that a class view can be readily incorporated to learn a supervised representation that facilitates subsequent classification. As the class view is added without an assumption on the exclusivity of classes, MVLSP-2C is equally applicable to multiclass single-label and multilabel classifications. MVLSP-2C further optimizes the integration of latent variables based on their correlation. Extensive experiments in multiclass and multiview image datasets show that MVLSP-2C produces more accurate classification results as compared to state-of-the-art methods.
This study sought to determine the feasibility of using Simultaneous Non-contrast Angiography and intraPlaque Hemorrhage (SNAP) to detect the lipid-rich/necrotic core (LRNC), and develop a machine ...learning based algorithm to segment plaque components on SNAP images.
Sixty-eight patients (age: 58±9 years, 24 males) with carotid artery atherosclerotic plaque were imaged on a 3 T MR scanner with both traditional multi-contrast vessel wall MR sequences (TOF, T1W, and T2W) and 3D SNAP sequence. The manual segmentations of carotid plaque components including LRNC, intraplaque hemorrhage (IPH), calcification (CA) and fibrous tissue (FT) on traditional multi-contrast images were used as reference. By utilizing the intensity and morphological information from SNAP, a machine learning based two steps algorithm was developed to firstly identify LRNC (with or without IPH), CA and FT, and then segmented IPH from LRNC. Ten-fold cross-validation was used to evaluate the performance of proposed method. The overall pixel-wise accuracy, the slice-wise sensitivity & specificity & Youden's index, and the Pearson's correlation coefficient of the component area between the proposed method and the manual segmentation were reported.
In the first step, all tested classifiers (Naive Bayes (NB), Support Vector Machine (SVM), Random Forest (RF), Gradient Boosting Decision Tree (GBDT) and Artificial Neural Network (ANN)) had overall pixel-wise accuracy higher than 0.88. For RF, GBDT and ANN classifiers, the correlation coefficients of areas were all higher than 0.82 (p < 0.001) for LRNC and 0.79 for CA (p < 0.001), and the Youden's indexes were all higher than 0.79 for LRNC and 0.76 for CA, which were better than that of NB and SVM. In the second step, the overall pixel-wise accuracy was higher than 0.78 for the five classifiers, and RF achieved the highest Youden's index (0.69) with the correlation coefficients as 0.63 (p < 0.001).
The RF is the overall best classifier for our proposed method, and the feasibility of using SNAP to identify plaque components, including LRNC, IPH, CA, and FT has been validated. The proposed segmentation method using a single SNAP sequence might be a promising tool for atherosclerotic plaque components assessment.
Purpose
Vitamin B deficiency has been identified as a risk factor for vascular events. However, the reduction of vascular events was not shown in large randomized controlled trials evaluating ...B‐Vitamin therapy. There is an important requirement to develop sensitive biomarkers to be used as efficacy targets for B‐Vitamin therapy as well as other dietary treatments and lifestyle regimes that are being developed. Carotid vessel‐wall‐plus‐plaque thickness change (VWT‐Change) measured from 3D ultrasound has been shown to be sensitive to atorvastatin therapies in previous studies. However, B‐Vitamin treatment is expected to confer a smaller beneficial effect in carotid atherosclerosis than the strong dose of atorvastatin. This paper introduces a sensitive atherosclerosis biomarker based on the weighted mean VWT‐Change measurement from 3D ultrasound with a purpose to detect statistically significant effect of B‐Vitamin therapy.
Methods
Of the 56 subjects analyzed in this study, 27 were randomized to receive a B‐Vitamin tablet daily and 29 received a placebo tablet daily. Participants were scanned at baseline and 1.9 ± 0.8 yr later. The 3D VWT map at each scanning session was computed by matching the outer wall and lumen surfaces on a point‐by‐point basis. The 3D annual VWT‐Change maps were obtained by first registering the 3D VWT maps obtained at the baseline and follow‐up scanning sessions, and then taking the point‐wise difference in VWT and dividing the result by the years elapsed from the baseline to the follow‐up scanning session. The 3D VWT‐Change maps constructed for all patients were mapped to a 2D carotid template to adjust for the anatomic variability of the arteries. A weight at each point of the carotid template was assigned based on the degree of correlation between the VWT‐Change measurements exhibited at that point and the treatment received (i.e., B‐Vitamin or placebo) quantified by mutual information. The weighted mean of VWT‐Change for each patient, denoted by ΔVWT¯Weighted, was computed according to this weight. T‐tests were performed to compare the sensitivity of ΔVWT¯Weighted with existing biomarkers in detecting treatment effects. These biomarkers included changes in intima‐media thickness (IMT), total plaque area (TPA), vessel wall volume (VWV), unweighted average of VWT‐Change (ΔVWT¯) and a previously described biomarker, denoted by ΔVWT¯S, that quantifies the mean VWT‐Change specific to regions of interest identified by a feature selection algorithm.
Results
Among the six biomarkers evaluated, the effect of B Vitamins was detected only by ΔVWT¯Weighted in this cohort (P=4.4×10−3). The sample sizes per treatment group required to detect an effect as large as exhibited in this study were 139, 178, 41 for ΔVWV, ΔVWT¯ and ΔVWT¯Weighted respectively.
Conclusion
The proposed weighted mean of VWT‐Change is more sensitive than existing biomarkers in detecting treatment effects. This measurement tool will allow for many proof‐of‐principal studies to be performed for various novel treatments before a more costly study involving a larger population is held to validate the results.
Measurements of vessel-wall-plus-plaque thickness (
VWT
) from 3D carotid ultrasound have been shown to be sensitive to the effect of pharmaceutical interventions. Since the geometry of carotid ...arteries is highly subject-specific, quantitative comparison of the distributions of point-wise
VWT
measured for different patients or for the same patients at different ultrasound scanning sessions requires the development of a mapping strategy to adjust for the geometric variability of different carotid surface models. In this paper, we present an algorithm mapping each 3D carotid surface to a 2D carotid template with an emphasis on preserving the local geometry of the carotid surface by minimizing local angular distortion. The previously described arc-length scaling (AL) approach was applied to generate an initial 2D
VWT
map. Using results established in the quasi-conformal theory, a new map was computed to compensate for the angular distortion incurred in AL mapping. As the 2D carotid template lies on an L-shaped non-convex domain, one-to-one correspondence of the mapping operation was not guaranteed. To address this issue, an iterative Beltrami differential chopping and smoothing procedure was developed to enforce bijectivity. Evaluations performed in the 20 carotid surface models showed that the reduction in average angular distortion made by the proposed algorithm was highly significant (
P
= 2.06 × 10
−5
). This study is the first study showing that a bijective conformal map to a non-convex domain can be obtained using the iterative Beltrami differential chopping and smoothing procedure. The improved consistency exhibited in the 2D
VWT
map generated by the proposed algorithm will allow for unbiased quantitative comparisons of
VWT
as well as local geometric and hemodynamic quantities in population studies.
