Nowcasting of convective storms is urgently needed yet rather challenging. Current nowcasting methods are mostly based on radar echo extrapolation, which suffer from the insufficiency of input ...information and ineffectiveness of model architecture. A novel deep‐learning (DL) model, FURENet, is designed for extracting information from multiple input variables to make predictions. Polarimetric radar variables, KDP and ZDR, which provide extra microphysics and dynamic structure information of storms, are fed into the model to improve nowcasting. Two representative cases indicate that KDP and ZDR can help the DL model better forecast convective organization and initiation. Quantitative statistical evaluation shows using FURENet, KDP, and ZDR synergistically improve nowcasting skills (CSI score) by 13.2% and 17.4% for the lead time of 30 and 60 min, respectively. Further evaluation shows the microphysical information provided by the polarimetric variables can enhance the DL model in understanding the evolution of convective storms and making more trustable nowcasts.
Plain Language Summary
Severe convective precipitation is a major cause of many hazards. However, very short‐term forecasting, i.e., nowcasting, of convective precipitation is rather challenging. Current nowcasting methods suffer from insufficiency of physics information of input data and ineffectiveness of model architecture. As an advanced observing tool, polarimetric weather radar can provide crucial microphysics and dynamic structure information of convective precipitation systems. To incorporate polarimetric radar variables into the nowcasting task, this study proposes a novel model architecture termed FURENet based on deep learning. FURENet uses U‐Net as a flexible backbone, and is specially designed to facilitate exploiting information from multiple input variables. By training the model with polarimetric radar variables (KDP and ZDR) as input, significant improvement of forecasting the initiation, development and evolution of convective storms is achieved. The results also show the effectiveness of model architecture.
Key Points
A deep‐learning approach termed FURENet is proposed for convective precipitation nowcasting with multiple input variables
Polarimetric radar variables are used to provide microphysics and dynamic structure information of convective storms in the model
Experiments with FURENet show significant improvement on nowcasting performance
Purpose
Radiation therapy (RT) is a common treatment option for head and neck (HaN) cancer. An important step involved in RT planning is the delineation of organs‐at‐risks (OARs) based on HaN ...computed tomography (CT). However, manually delineating OARs is time‐consuming as each slice of CT images needs to be individually examined and a typical CT consists of hundreds of slices. Automating OARs segmentation has the benefit of both reducing the time and improving the quality of RT planning. Existing anatomy autosegmentation algorithms use primarily atlas‐based methods, which require sophisticated atlas creation and cannot adequately account for anatomy variations among patients. In this work, we propose an end‐to‐end, atlas‐free three‐dimensional (3D) convolutional deep learning framework for fast and fully automated whole‐volume HaN anatomy segmentation.
Methods
Our deep learning model, called AnatomyNet, segments OARs from head and neck CT images in an end‐to‐end fashion, receiving whole‐volume HaN CT images as input and generating masks of all OARs of interest in one shot. AnatomyNet is built upon the popular 3D U‐net architecture, but extends it in three important ways: (a) a new encoding scheme to allow autosegmentation on whole‐volume CT images instead of local patches or subsets of slices, (b) incorporating 3D squeeze‐and‐excitation residual blocks in encoding layers for better feature representation, and (c) a new loss function combining Dice scores and focal loss to facilitate the training of the neural model. These features are designed to address two main challenges in deep learning‐based HaN segmentation: (a) segmenting small anatomies (i.e., optic chiasm and optic nerves) occupying only a few slices, and (b) training with inconsistent data annotations with missing ground truth for some anatomical structures.
Results
We collected 261 HaN CT images to train AnatomyNet and used MICCAI Head and Neck Auto Segmentation Challenge 2015 as a benchmark dataset to evaluate the performance of AnatomyNet. The objective is to segment nine anatomies: brain stem, chiasm, mandible, optic nerve left, optic nerve right, parotid gland left, parotid gland right, submandibular gland left, and submandibular gland right. Compared to previous state‐of‐the‐art results from the MICCAI 2015 competition, AnatomyNet increases Dice similarity coefficient by 3.3% on average. AnatomyNet takes about 0.12 s to fully segment a head and neck CT image of dimension 178 × 302 × 225, significantly faster than previous methods. In addition, the model is able to process whole‐volume CT images and delineate all OARs in one pass, requiring little pre‐ or postprocessing.
Conclusion
Deep learning models offer a feasible solution to the problem of delineating OARs from CT images. We demonstrate that our proposed model can improve segmentation accuracy and simplify the autosegmentation pipeline. With this method, it is possible to delineate OARs of a head and neck CT within a fraction of a second.
Purpose
Automatic segmentation of organs‐at‐risk (OARs) is a key step in radiation treatment planning to reduce human efforts and bias. Deep convolutional neural networks (DCNN) have shown great ...success in many medical image segmentation applications but there are still challenges in dealing with large 3D images for optimal results. The purpose of this study is to develop a novel DCNN method for thoracic OARs segmentation using cropped 3D images.
Methods
To segment the five organs (left and right lungs, heart, esophagus and spinal cord) from the thoracic CT scans, preprocessing to unify the voxel spacing and intensity was first performed, a 3D U‐Net was then trained on resampled thoracic images to localize each organ, then the original images were cropped to only contain one organ and served as the input to each individual organ segmentation network. The segmentation maps were then merged to get the final results. The network structures were optimized for each step, as well as the training and testing strategies. A novel testing augmentation with multiple iterations of image cropping was used. The networks were trained on 36 thoracic CT scans with expert annotations provided by the organizers of the 2017 AAPM Thoracic Auto‐segmentation Challenge and tested on the challenge testing dataset as well as a private dataset.
Results
The proposed method earned second place in the live phase of the challenge and first place in the subsequent ongoing phase using a newly developed testing augmentation approach. It showed superior‐than‐human performance on average in terms of Dice scores (spinal cord: 0.893 ± 0.044, right lung: 0.972 ± 0.021, left lung: 0.979 ± 0.008, heart: 0.925 ± 0.015, esophagus: 0.726 ± 0.094), mean surface distance (spinal cord: 0.662 ± 0.248 mm, right lung: 0.933 ± 0.574 mm, left lung: 0.586 ± 0.285 mm, heart: 2.297 ± 0.492 mm, esophagus: 2.341 ± 2.380 mm) and 95% Hausdorff distance (spinal cord: 1.893 ± 0.627 mm, right lung: 3.958 ± 2.845 mm, left lung: 2.103 ± 0.938 mm, heart: 6.570 ± 1.501 mm, esophagus: 8.714 ± 10.588 mm). It also achieved good performance in the private dataset and reduced the editing time to 7.5 min per patient following automatic segmentation.
Conclusions
The proposed DCNN method demonstrated good performance in automatic OAR segmentation from thoracic CT scans. It has the potential for eventual clinical adoption of deep learning in radiation treatment planning due to improved accuracy and reduced cost for OAR segmentation.
•A novel deep learning network was proposed based on the classical U-Net model to accurately segment the optic disc from colour fundus images.•A sub-network and a decoding convolutional block were ...introduced to provide additional key features and highlight the morphological changes of the target objects in convolutional feature maps.•Experiment results on both the global field-of-view fundus images and their local disc versions from the MESSIDOR, ORIGA, and REFUGE datasets demonstrated that the developed network achieved promising performance and outperformed some existing segmentation networks.
Accurate segmentation of the optic disc (OD) regions from colour fundus images is a critical procedure for computer-aided diagnosis of glaucoma. We present a novel deep learning network to automatically identify the OD regions. On the basis of the classical U-Net framework, we define a unique sub-network and a decoding convolutional block. The sub-network is used to preserve important textures and facilitate their detections, while the decoding block is used to improve the contrast of the regions-of-interest with their background. We integrate these two components into the classical U-Net framework to improve the accuracy and reliability of segmenting the OD regions depicted on colour fundus images. We train and evaluate the developed network using three publicly available datasets (i.e., MESSIDOR, ORIGA, and REFUGE). The results on an independent testing set (n = 1,970 images) show a segmentation performance with an average Dice similarity coefficient (DSC), intersection over union (IOU), and Matthew's correlation coefficient (MCC) of 0.9377, 0.8854, and 0.9383 when trained on the global field-of-view images, respectively, and 0.9735, 0.9494, and 0.9594 when trained on the local disc region images. When compared with the other three classical networks (i.e., the U-Net, M-Net, and Deeplabv3) on the same testing datasets, the developed network demonstrates a relatively higher performance.
Summary
Periodic road crack monitoring is an essential procedure for effective pavement management. Highly efficient and accurate crack measurements are key research topics in both academia and ...industry. Automatic methods gradually replaced traditional manual surveys for more reliable evaluation outputs and better efficiency, whereas the devices are not available to all functional classes of pavements and different departments considering the high cost versus the limited budget. Recently, the widespread use of smartphones and digital cameras made it possible to collect pavement surface crack images at an affordable price in easier ways. However, the qualities of these crack images are diversely influenced by the noises from pavement background, roadways, and so forth. Thus, traditional methods usually fail to extract accurate crack information from pavement images. Therefore, this research proposes a state‐of‐the‐art pixelwise crack detection architecture called CrackU‐net, which is featured by its utilization of advanced deep convolutional neural network technology. CrackU‐net achieved pixelwise crack detection through convolution, pooling, transpose convolution, and concatenation operations, forming the “U”‐shaped model architecture. The model is trained and validated by 3,000 pavement crack images, in which 2,400 for training and 600 for validating, using the Adam algorithm. CrackU‐net has the performance of loss = 0.025, accuracy = 0.9901, precision = 0.9856, recall = 0.9798, and F‐measure = 0.9842 with learning rate of 10−2. Meanwhile, the false‐positive crack detection problem is avoided in CrackU‐net. Therefore, CrackU‐net outperforms both traditional approaches and fully convolutional network (FCN) and U‐net for pixelwise crack detections.
Purpose
Owing to histologic complexities of brain tumors, its diagnosis requires the use of multimodalities to obtain valuable structural information so that brain tumor subregions can be properly ...delineated. In current clinical workflow, physicians typically perform slice‐by‐slice delineation of brain tumor subregions, which is a time‐consuming process and also more susceptible to intra‐ and inter‐rater variabilities possibly leading to misclassification. To deal with this issue, this study aims to develop an automatic segmentation of brain tumor in MR images using deep learning.
Method
In this study, we develop a context deep‐supervised U‐Net to segment brain tumor subregions. A context block which aggregates multiscale contextual information for dense segmentation was proposed. This approach enlarges the effective receptive field of convolutional neural networks, which, in turn, improves the segmentation accuracy of brain tumor subregions. We performed the fivefold cross‐validation on the Brain Tumor Segmentation Challenge (BraTS) 2020 training dataset. The BraTS 2020 testing datasets were obtained via BraTS online website as a hold‐out test. For BraTS, the evaluation system divides the tumor into three regions: whole tumor (WT), tumor core (TC), and enhancing tumor (ET). The performance of our proposed method was compared against two state‐of‐the‐arts CNN networks in terms of segmentation accuracy via Dice similarity coefficient (DSC) and Hausdorff distance (HD). The tumor volumes generated by our proposed method were compared with manually contoured volumes via Bland–Altman plots and Pearson analysis.
Results
The proposed method achieved the segmentation results with a DSC of 0.923 ± 0.047, 0.893 ± 0.176, and 0.846 ± 0.165 and a 95% HD95 of 3.946 ± 7.041, 3.981 ± 6.670, and 10.128 ± 51.136 mm on WT, TC, and ET, respectively. Experimental results demonstrate that our method achieved comparable to significantly (p < 0.05) better segmentation accuracies than other two state‐of‐the‐arts CNN networks. Pearson correlation analysis showed a high positive correlation between the tumor volumes generated by proposed method and manual contour.
Conclusion
Overall qualitative and quantitative results of this work demonstrate the potential of translating proposed technique into clinical practice for segmenting brain tumor subregions, and further facilitate brain tumor radiotherapy workflow.
Summary
The image dehazing stage is used significantly as a preprocessing step for various applications such as remote sensing and long range imaging and automatic driver assistance system. Images ...acquired under low illumination, fog and snow conditions frequently show qualities like low contrast and low brightness, which genuinely influence the enhanced visualization on natural eyes and extraordinarily limit the exhibition of different machine vision frameworks. The images that are captured in low‐light or heavy fog might have salient features that cannot be extracted using standard computer vision systems. A good way to get the enhanced image is to determine the transmission map (haze density or low illumination parameters) of air‐light media from each pixel of the input image. In this article, an improved U‐Net architecture is proposed to enhance images and provide robust performance metrics against the existing methods. In this model, the pooling operations in generalized U‐Net architecture are replaced by discrete wavelet transform based on up and down samplings. An attention module is developed by fusing both up and down samples to identify the missing information of low‐level features in up‐samples. The proposed architecture for U‐Net tested with different datasets: See‐in‐the‐Dark (SID) dataset, Exclusively Dark Image Dataset (ExDark), Realistic Single Image Dehazing (RESIDE) dataset, and few real‐time images and achieves superior performance metrics in terms of PSNR, MSE, and SSI when compared to the other state‐of‐art methods.
Seismic data denoising (SDD) plays an important role in obtaining high-quality data for subsequent seismic imaging and inversion. However, traditional SDD methods still present several disadvantages ...such as parameter selection difficulties, high computational costs, and a strong dependence on the experience of processing personnel. Deep learning (DL)-based SDD methods can overcome these issues to a certain extent, having the advantages of high computational efficiency and negligible dependency on the experience of processing personnel. However, existing DL-based SDD methods typically adopt a specific single-network architecture, which can be further improved in terms of training efficiency and denoising quality. In this paper, we propose a two-step prediction denoising method based on the combination of two identical or different network architectures for improving the accuracy and efficiency of DL-based SDD methods. Compared with existing SDD methods, the two-step prediction denoising method is more capable of extracting feature information at multiple scales because multiple network architectures are combined in a single denoising task, thus improving the training efficiency and denoising accuracy. In this study, we implemented the two-step prediction denoising method using an improved denoising convolutional neural network (DnCNN) and a nested U-shaped fully convolutional network (U-Net++). Several synthetic numerical examples based on three different types of noise were denoised, with the results indicating that the proposed method effectively improves the accuracy and efficiency of the DL-based SDD method. Finally, a test using field data was performed to demonstrate the real-world applicability of the novel two-step prediction denoising method.
•A modified DnCNN using a different architecture is constructed.•A network-based two-step prediction method is proposed to denoise seismic data.•Examples illustrate the accuracy and feasibility of the method.
Purpose
Dual‐energy computed tomography (DECT) has shown great potential in many clinical applications. By incorporating the information from two different energy spectra, DECT provides higher ...contrast and reveals more material differences of tissues compared to conventional single‐energy CT (SECT). Recent research shows that automatic multi‐organ segmentation of DECT data can improve DECT clinical applications. However, most segmentation methods are designed for SECT, while DECT has been significantly less pronounced in research. Therefore, a novel approach is required that is able to take full advantage of the extra information provided by DECT.
Methods
In the scope of this work, we proposed four three‐dimensional (3D) fully convolutional neural network algorithms for the automatic segmentation of DECT data. We incorporated the extra energy information differently and embedded the fusion of information in each of the network architectures.
Results
Quantitative evaluation using 45 thorax/abdomen DECT datasets acquired with a clinical dual‐source CT system was investigated. The segmentation of six thoracic and abdominal organs (left and right lungs, liver, spleen, and left and right kidneys) were evaluated using a fivefold cross‐validation strategy. In all of the tests, we achieved the best average Dice coefficients of 98% for the right lung, 98% for the left lung, 96% for the liver, 92% for the spleen, 95% for the right kidney, 93% for the left kidney, respectively. The network architectures exploit dual‐energy spectra and outperform deep learning for SECT.
Conclusions
The results of the cross‐validation show that our methods are feasible and promising. Successful tests on special clinical cases reveal that our methods have high adaptability in the practical application.
PDF
