Nowcasting of convective storms is urgently needed yet rather challenging. Current nowcasting methods are mostly based on radar echo extrapolation, which suffer from the insufficiency of input ...information and ineffectiveness of model architecture. A novel deep‐learning (DL) model, FURENet, is designed for extracting information from multiple input variables to make predictions. Polarimetric radar variables, KDP and ZDR, which provide extra microphysics and dynamic structure information of storms, are fed into the model to improve nowcasting. Two representative cases indicate that KDP and ZDR can help the DL model better forecast convective organization and initiation. Quantitative statistical evaluation shows using FURENet, KDP, and ZDR synergistically improve nowcasting skills (CSI score) by 13.2% and 17.4% for the lead time of 30 and 60 min, respectively. Further evaluation shows the microphysical information provided by the polarimetric variables can enhance the DL model in understanding the evolution of convective storms and making more trustable nowcasts.
Plain Language Summary
Severe convective precipitation is a major cause of many hazards. However, very short‐term forecasting, i.e., nowcasting, of convective precipitation is rather challenging. Current nowcasting methods suffer from insufficiency of physics information of input data and ineffectiveness of model architecture. As an advanced observing tool, polarimetric weather radar can provide crucial microphysics and dynamic structure information of convective precipitation systems. To incorporate polarimetric radar variables into the nowcasting task, this study proposes a novel model architecture termed FURENet based on deep learning. FURENet uses U‐Net as a flexible backbone, and is specially designed to facilitate exploiting information from multiple input variables. By training the model with polarimetric radar variables (KDP and ZDR) as input, significant improvement of forecasting the initiation, development and evolution of convective storms is achieved. The results also show the effectiveness of model architecture.
Key Points
A deep‐learning approach termed FURENet is proposed for convective precipitation nowcasting with multiple input variables
Polarimetric radar variables are used to provide microphysics and dynamic structure information of convective storms in the model
Experiments with FURENet show significant improvement on nowcasting performance
Purpose
Radiation therapy (RT) is a common treatment option for head and neck (HaN) cancer. An important step involved in RT planning is the delineation of organs‐at‐risks (OARs) based on HaN ...computed tomography (CT). However, manually delineating OARs is time‐consuming as each slice of CT images needs to be individually examined and a typical CT consists of hundreds of slices. Automating OARs segmentation has the benefit of both reducing the time and improving the quality of RT planning. Existing anatomy autosegmentation algorithms use primarily atlas‐based methods, which require sophisticated atlas creation and cannot adequately account for anatomy variations among patients. In this work, we propose an end‐to‐end, atlas‐free three‐dimensional (3D) convolutional deep learning framework for fast and fully automated whole‐volume HaN anatomy segmentation.
Methods
Our deep learning model, called AnatomyNet, segments OARs from head and neck CT images in an end‐to‐end fashion, receiving whole‐volume HaN CT images as input and generating masks of all OARs of interest in one shot. AnatomyNet is built upon the popular 3D U‐net architecture, but extends it in three important ways: (a) a new encoding scheme to allow autosegmentation on whole‐volume CT images instead of local patches or subsets of slices, (b) incorporating 3D squeeze‐and‐excitation residual blocks in encoding layers for better feature representation, and (c) a new loss function combining Dice scores and focal loss to facilitate the training of the neural model. These features are designed to address two main challenges in deep learning‐based HaN segmentation: (a) segmenting small anatomies (i.e., optic chiasm and optic nerves) occupying only a few slices, and (b) training with inconsistent data annotations with missing ground truth for some anatomical structures.
Results
We collected 261 HaN CT images to train AnatomyNet and used MICCAI Head and Neck Auto Segmentation Challenge 2015 as a benchmark dataset to evaluate the performance of AnatomyNet. The objective is to segment nine anatomies: brain stem, chiasm, mandible, optic nerve left, optic nerve right, parotid gland left, parotid gland right, submandibular gland left, and submandibular gland right. Compared to previous state‐of‐the‐art results from the MICCAI 2015 competition, AnatomyNet increases Dice similarity coefficient by 3.3% on average. AnatomyNet takes about 0.12 s to fully segment a head and neck CT image of dimension 178 × 302 × 225, significantly faster than previous methods. In addition, the model is able to process whole‐volume CT images and delineate all OARs in one pass, requiring little pre‐ or postprocessing.
Conclusion
Deep learning models offer a feasible solution to the problem of delineating OARs from CT images. We demonstrate that our proposed model can improve segmentation accuracy and simplify the autosegmentation pipeline. With this method, it is possible to delineate OARs of a head and neck CT within a fraction of a second.
Purpose
Automatic segmentation of organs‐at‐risk (OARs) is a key step in radiation treatment planning to reduce human efforts and bias. Deep convolutional neural networks (DCNN) have shown great ...success in many medical image segmentation applications but there are still challenges in dealing with large 3D images for optimal results. The purpose of this study is to develop a novel DCNN method for thoracic OARs segmentation using cropped 3D images.
Methods
To segment the five organs (left and right lungs, heart, esophagus and spinal cord) from the thoracic CT scans, preprocessing to unify the voxel spacing and intensity was first performed, a 3D U‐Net was then trained on resampled thoracic images to localize each organ, then the original images were cropped to only contain one organ and served as the input to each individual organ segmentation network. The segmentation maps were then merged to get the final results. The network structures were optimized for each step, as well as the training and testing strategies. A novel testing augmentation with multiple iterations of image cropping was used. The networks were trained on 36 thoracic CT scans with expert annotations provided by the organizers of the 2017 AAPM Thoracic Auto‐segmentation Challenge and tested on the challenge testing dataset as well as a private dataset.
Results
The proposed method earned second place in the live phase of the challenge and first place in the subsequent ongoing phase using a newly developed testing augmentation approach. It showed superior‐than‐human performance on average in terms of Dice scores (spinal cord: 0.893 ± 0.044, right lung: 0.972 ± 0.021, left lung: 0.979 ± 0.008, heart: 0.925 ± 0.015, esophagus: 0.726 ± 0.094), mean surface distance (spinal cord: 0.662 ± 0.248 mm, right lung: 0.933 ± 0.574 mm, left lung: 0.586 ± 0.285 mm, heart: 2.297 ± 0.492 mm, esophagus: 2.341 ± 2.380 mm) and 95% Hausdorff distance (spinal cord: 1.893 ± 0.627 mm, right lung: 3.958 ± 2.845 mm, left lung: 2.103 ± 0.938 mm, heart: 6.570 ± 1.501 mm, esophagus: 8.714 ± 10.588 mm). It also achieved good performance in the private dataset and reduced the editing time to 7.5 min per patient following automatic segmentation.
Conclusions
The proposed DCNN method demonstrated good performance in automatic OAR segmentation from thoracic CT scans. It has the potential for eventual clinical adoption of deep learning in radiation treatment planning due to improved accuracy and reduced cost for OAR segmentation.
•A novel deep learning network was proposed based on the classical U-Net model to accurately segment the optic disc from colour fundus images.•A sub-network and a decoding convolutional block were ...introduced to provide additional key features and highlight the morphological changes of the target objects in convolutional feature maps.•Experiment results on both the global field-of-view fundus images and their local disc versions from the MESSIDOR, ORIGA, and REFUGE datasets demonstrated that the developed network achieved promising performance and outperformed some existing segmentation networks.
Accurate segmentation of the optic disc (OD) regions from colour fundus images is a critical procedure for computer-aided diagnosis of glaucoma. We present a novel deep learning network to automatically identify the OD regions. On the basis of the classical U-Net framework, we define a unique sub-network and a decoding convolutional block. The sub-network is used to preserve important textures and facilitate their detections, while the decoding block is used to improve the contrast of the regions-of-interest with their background. We integrate these two components into the classical U-Net framework to improve the accuracy and reliability of segmenting the OD regions depicted on colour fundus images. We train and evaluate the developed network using three publicly available datasets (i.e., MESSIDOR, ORIGA, and REFUGE). The results on an independent testing set (n = 1,970 images) show a segmentation performance with an average Dice similarity coefficient (DSC), intersection over union (IOU), and Matthew's correlation coefficient (MCC) of 0.9377, 0.8854, and 0.9383 when trained on the global field-of-view images, respectively, and 0.9735, 0.9494, and 0.9594 when trained on the local disc region images. When compared with the other three classical networks (i.e., the U-Net, M-Net, and Deeplabv3) on the same testing datasets, the developed network demonstrates a relatively higher performance.
Purpose
Clinical implementation of magnetic resonance imaging (MRI)‐only radiotherapy requires a method to derive synthetic CT image (S‐CT) for dose calculation. This study investigated the ...feasibility of building a deep convolutional neural network for MRI‐based S‐CT generation and evaluated the dosimetric accuracy on prostate IMRT planning.
Methods
A paired CT and T2‐weighted MR images were acquired from each of 51 prostate cancer patients. Fifteen pairs were randomly chosen as tested set and the remaining 36 pairs as training set. The training subjects were augmented by applying artificial deformations and feed to a two‐dimensional U‐net which contains 23 convolutional layers and 25.29 million trainable parameters. The U‐net represents a nonlinear function with input an MR slice and output the corresponding S‐CT slice. The mean absolute error (MAE) of Hounsfield unit (HU) between the true CT and S‐CT images was used to evaluate the HU estimation accuracy. IMRT plans with dose 79.2 Gy prescribed to the PTV were applied using the true CT images. The true CT images then were replaced by the S‐CT images and the dose matrices were recalculated on the same plan and compared to the one obtained from the true CT using gamma index analysis and absolute point dose discrepancy.
Results
The U‐net was trained from scratch in 58.67 h using a GP100‐GPU. The computation time for generating a new S‐CT volume image was 3.84–7.65 s. Within body, the (mean ± SD) of MAE was (29.96 ± 4.87) HU. The 1%/1 mm and 2%/2 mm gamma pass rates were over 98.03% and 99.36% respectively. The DVH parameters discrepancy was less than 0.87% and the maximum point dose discrepancy within PTV was less than 1.01% respect to the prescription.
Conclusion
The U‐net can generate S‐CT images from conventional MR image within seconds with high dosimetric accuracy for prostate IMRT plan.
Summary
Periodic road crack monitoring is an essential procedure for effective pavement management. Highly efficient and accurate crack measurements are key research topics in both academia and ...industry. Automatic methods gradually replaced traditional manual surveys for more reliable evaluation outputs and better efficiency, whereas the devices are not available to all functional classes of pavements and different departments considering the high cost versus the limited budget. Recently, the widespread use of smartphones and digital cameras made it possible to collect pavement surface crack images at an affordable price in easier ways. However, the qualities of these crack images are diversely influenced by the noises from pavement background, roadways, and so forth. Thus, traditional methods usually fail to extract accurate crack information from pavement images. Therefore, this research proposes a state‐of‐the‐art pixelwise crack detection architecture called CrackU‐net, which is featured by its utilization of advanced deep convolutional neural network technology. CrackU‐net achieved pixelwise crack detection through convolution, pooling, transpose convolution, and concatenation operations, forming the “U”‐shaped model architecture. The model is trained and validated by 3,000 pavement crack images, in which 2,400 for training and 600 for validating, using the Adam algorithm. CrackU‐net has the performance of loss = 0.025, accuracy = 0.9901, precision = 0.9856, recall = 0.9798, and F‐measure = 0.9842 with learning rate of 10−2. Meanwhile, the false‐positive crack detection problem is avoided in CrackU‐net. Therefore, CrackU‐net outperforms both traditional approaches and fully convolutional network (FCN) and U‐net for pixelwise crack detections.
Purpose
Owing to histologic complexities of brain tumors, its diagnosis requires the use of multimodalities to obtain valuable structural information so that brain tumor subregions can be properly ...delineated. In current clinical workflow, physicians typically perform slice‐by‐slice delineation of brain tumor subregions, which is a time‐consuming process and also more susceptible to intra‐ and inter‐rater variabilities possibly leading to misclassification. To deal with this issue, this study aims to develop an automatic segmentation of brain tumor in MR images using deep learning.
Method
In this study, we develop a context deep‐supervised U‐Net to segment brain tumor subregions. A context block which aggregates multiscale contextual information for dense segmentation was proposed. This approach enlarges the effective receptive field of convolutional neural networks, which, in turn, improves the segmentation accuracy of brain tumor subregions. We performed the fivefold cross‐validation on the Brain Tumor Segmentation Challenge (BraTS) 2020 training dataset. The BraTS 2020 testing datasets were obtained via BraTS online website as a hold‐out test. For BraTS, the evaluation system divides the tumor into three regions: whole tumor (WT), tumor core (TC), and enhancing tumor (ET). The performance of our proposed method was compared against two state‐of‐the‐arts CNN networks in terms of segmentation accuracy via Dice similarity coefficient (DSC) and Hausdorff distance (HD). The tumor volumes generated by our proposed method were compared with manually contoured volumes via Bland–Altman plots and Pearson analysis.
Results
The proposed method achieved the segmentation results with a DSC of 0.923 ± 0.047, 0.893 ± 0.176, and 0.846 ± 0.165 and a 95% HD95 of 3.946 ± 7.041, 3.981 ± 6.670, and 10.128 ± 51.136 mm on WT, TC, and ET, respectively. Experimental results demonstrate that our method achieved comparable to significantly (p < 0.05) better segmentation accuracies than other two state‐of‐the‐arts CNN networks. Pearson correlation analysis showed a high positive correlation between the tumor volumes generated by proposed method and manual contour.
Conclusion
Overall qualitative and quantitative results of this work demonstrate the potential of translating proposed technique into clinical practice for segmenting brain tumor subregions, and further facilitate brain tumor radiotherapy workflow.
Summary
The image dehazing stage is used significantly as a preprocessing step for various applications such as remote sensing and long range imaging and automatic driver assistance system. Images ...acquired under low illumination, fog and snow conditions frequently show qualities like low contrast and low brightness, which genuinely influence the enhanced visualization on natural eyes and extraordinarily limit the exhibition of different machine vision frameworks. The images that are captured in low‐light or heavy fog might have salient features that cannot be extracted using standard computer vision systems. A good way to get the enhanced image is to determine the transmission map (haze density or low illumination parameters) of air‐light media from each pixel of the input image. In this article, an improved U‐Net architecture is proposed to enhance images and provide robust performance metrics against the existing methods. In this model, the pooling operations in generalized U‐Net architecture are replaced by discrete wavelet transform based on up and down samplings. An attention module is developed by fusing both up and down samples to identify the missing information of low‐level features in up‐samples. The proposed architecture for U‐Net tested with different datasets: See‐in‐the‐Dark (SID) dataset, Exclusively Dark Image Dataset (ExDark), Realistic Single Image Dehazing (RESIDE) dataset, and few real‐time images and achieves superior performance metrics in terms of PSNR, MSE, and SSI when compared to the other state‐of‐art methods.
Purpose
Positron emission tomography (PET) is an essential technique in many clinical applications that allows for quantitative imaging at the molecular level. This study aims to develop a denoising ...method using a novel dilated convolutional neural network (CNN) to recover full‐count images from low‐count images.
Methods
We adopted similar hierarchical structures as the conventional U‐Net and incorporated dilated kernels in each convolution to allow the network to observe larger, more robust features within the image without the requirement of downsampling and upsampling internal representations. Our dNet was trained alongside a U‐Net for comparison. Both models were evaluated using a leave‐one‐out cross‐validation procedure on a dataset of 35 subjects (~3500 slabs), which were obtained from an ongoing 18F‐Fluorodeoxyglucose (FDG) study. Low‐count PET data (10% count) were generated by randomly selecting one‐tenth of all events in the associated listmode file. Analysis was done on the static image from the last 10 minutes of emission data. Both low‐count PET and full‐count PET were reconstructed using ordered subset expectation maximization (OSEM). Objective image quality metrics, including mean absolute percent error (MAPE), peak signal‐to‐noise ratio (PSNR), and structural similarity index metric (SSIM), were used to analyze the deep learning methods. Both deep learning methods were further compared to a traditional Gaussian filtering method. Further, region of interest (ROI) quantitative analysis was also used to compare U‐Net and dNet architectures.
Results
Both the U‐Net and our proposed network were successfully trained to synthesize full‐count PET images from the generated low‐count PET images. Compared to low‐count PET and Gaussian filtering, both deep learning methods improved MAPE, PSNR, and SSIM. Our dNet also systematically outperformed U‐Net on all three metrics (MAPE: 4.99 ± 0.68 vs 5.31 ± 0.76, P < 0.01; PSNR: 31.55 ± 1.31 dB vs 31.05 ± 1.39, P < 0.01; SSIM: 0.9513 ± 0.0154 vs 0.9447 ± 0.0178, P < 0.01). ROI quantification showed greater quantitative improvements using dNet over U‐Net.
Conclusion
This study proposed a novel approach of using dilated convolutions for recovering full‐count PET images from low‐count PET images.
PDF
