This paper tries to give a gentle introduction to deep learning in medical image processing, proceeding from theoretical foundations to applications. We first discuss general reasons for the ...popularity of deep learning, including several major breakthroughs in computer science. Next, we start reviewing the fundamental basics of the perceptron and neural networks, along with some fundamental theory that is often omitted. Doing so allows us to understand the reasons for the rise of deep learning in many application domains. Obviously medical image processing is one of these areas which has been largely affected by this rapid progress, in particular in image detection and recognition, image segmentation, image registration, and computer-aided diagnosis. There are also recent trends in physical simulation, modeling, and reconstruction that have led to astonishing results. Yet, some of these approaches neglect prior knowledge and hence bear the risk of producing implausible results. These apparent weaknesses highlight current limitations of deep ()learning. However, we also briefly discuss promising approaches that might be able to resolve these problems in the future.
Dark-field computed tomography reaches the human scale Viermetz, Manuel; Gustschin, Nikolai; Schmid, Clemens ...
Proceedings of the National Academy of Sciences - PNAS,
02/2022, Letnik:
119, Številka:
8
Journal Article
Recenzirano
Odprti dostop
X-ray computed tomography (CT) is one of the most commonly used three-dimensional medical imaging modalities today. It has been refined over several decades, with the most recent innovations ...including dual-energy and spectral photon-counting technologies. Nevertheless, it has been discovered that wave-optical contrast mechanisms-beyond the presently used X-ray attenuation-offer the potential of complementary information, particularly on otherwise unresolved tissue microstructure. One such approach is dark-field imaging, which has recently been introduced and already demonstrated significantly improved radiological benefit in small-animal models, especially for lung diseases. Until now, however, dark-field CT could not yet be translated to the human scale and has been restricted to benchtop and small-animal systems, with scan durations of several minutes or more. This is mainly because the adaption and upscaling to the mechanical complexity, speed, and size of a human CT scanner so far remained an unsolved challenge. Here, we now report the successful integration of a Talbot-Lau interferometer into a clinical CT gantry and present dark-field CT results of a human-sized anthropomorphic body phantom, reconstructed from a single rotation scan performed in 1 s. Moreover, we present our key hardware and software solutions to the previously unsolved roadblocks, which so far have kept dark-field CT from being translated from the optical bench into a rapidly rotating CT gantry, with all its associated challenges like vibrations, continuous rotation, and large field of view. This development enables clinical dark-field CT studies with human patients in the near future.
Public chest X-ray (CXR) data sets are commonly compressed to a lower bit depth to reduce their size, potentially hiding subtle diagnostic features. In contrast, radiologists apply a windowing ...operation to the uncompressed image to enhance such subtle features. While it has been shown that windowing improves classification performance on computed tomography (CT) images, the impact of such an operation on CXR classification performance remains unclear. In this study, we show that windowing strongly improves the CXR classification performance of machine learning models and propose WindowNet, a model that learns multiple optimal window settings. Our model achieved an average AUC score of 0.812 compared with the 0.759 score of a commonly used architecture without windowing capabilities on the MIMIC data set.
In industrial settings, X-ray computed tomography scans are a common tool for inspection of objects. Often the object can not be imaged using standard circular or helical trajectories because of ...constraints in space or time. Compared to medical applications the variance in size and materials is much larger. Adapting the acquisition trajectory to the object is beneficial and sometimes inevitable. There are currently no sophisticated methods for this adoption. Typically the operator places the object according to his best knowledge. We propose a detectability index based optimization algorithm which determines the scan trajectory on the basis of a CAD-model of the object. The detectability index is computed solely from simulated projections for multiple user defined features. By adapting the features the algorithm is adapted to different imaging tasks. Performance of simulated and measured data was qualitatively and quantitatively assessed.The results illustrate that our algorithm not only allows more accurate detection of features, but also delivers images with high overall quality in comparison to standard trajectory reconstructions. This work enables to reduce the number of projections and in consequence scan time by introducing an optimization algorithm to compose an object specific trajectory.
Purpose
Freehand SPECT is a 3-D tomographic imaging modality based on data acquisition with a hand-held detector that is moved freely, in contrast to conventional, fixed gamma camera systems. In this ...pilot study, the feasibility of freehand SPECT for 3-D lymphatic mapping in breast cancer was evaluated.
Methods
A total of 85 patients (age: 29–88 years) with an initial diagnosis of invasive breast cancer and no clinical evidence of nodal involvement prospectively underwent sentinel lymph node (SLN) biopsy. Preoperative lymphatic mapping (35–87 MBq
99m
Tc-Nanocoll) included tomographic imaging with a SPECT/CT device (Siemens Symbia T6) serving as reference. Initially, the freehand SPECT approach was assessed in a pilot study consisting of 50 patients. The quality of each freehand SPECT acquisition was assessed and ranked as good, intermediate or poor. In another series comprising a further 35 patients (validation study), a guidance system for the acquisition was implemented based on the results of the pilot study, ensuring acquisitions with good quality. For 3-D tomographic image reconstruction, ad hoc models and iterative reconstruction algorithms were used in all 85 patients. To allow for adequate comparison, SPECT/CT data and freehand SPECT data were registered within the same coordinate system.
Results
In the pilot study, freehand SPECT enabled mapping of 24 of 83 SLNs in 20 of 44 patients (3 dropouts, 3 patients without SLN either in SPECT/CT or in freehand SPECT). Using SPECT/CT as reference, the accuracy of freehand SPECT was 77.8% (7/9 nodes) in scans with good quality, while for intermediate and poor quality scans, the accuracy was reduced to 34.3 and 12.8%, respectively. In the validation study, quality feedback improved the results significantly and freehand SPECT enabled the mapping of at least one SLN in 87.5% of the patients (28/32 − 3 dropouts). Compared to the reference method, freehand SPECT showed a sensitivity of 83.3% (35/42 nodes). False-negative findings were related to insufficient scanning time, insufficient coverage of the axillary region, close proximity of the SLN to the injection site and low tracer uptake in the SLNs.
Conclusion
In this preliminary study, we could demonstrate that 3-D localization of SLNs is feasible using freehand SPECT technology. Prerequisites for acquisition of a good scan quality, most likely allowing precise SLN mapping, have been defined. This approach has high potential to allow image-guided biopsy and further standardization of SLN dissection, thus bringing 3-D nuclear imaging into the operating room.
Display omitted
•Accurate 3D reconstructions from a mini gamma camera at multiple poses.•Flexible nuclear imaging system to enable intra-operative SPECT-like 3D imaging.•Evaluation of number of ...required imaging positions and required imaging time.•High quality reconstructions of hot spots are possible with errors <1mm in only 15s.
Nuclear imaging modalities like PET or SPECT are in extensive use in medical diagnostics. In a move towards personalized therapy, we present a flexible nuclear tomographic imaging system to enable intra-operative SPECT-like 3D imaging. The system consists of a miniaturized gamma camera mounted on a robot arm for flexible positioning, while spatio-temporal localization is provided by an optical tracking system. To facilitate statistical tomographic reconstruction of the radiotracer distribution using a maximum likelihood approach, a precise model of the mini gamma camera is generated by measurements. The entire system is evaluated in a series of experiments using a hot spot phantom, with a focus on criteria relevant for the intra-operative workflow, namely the number of required imaging positions as well as the required imaging time. The results show that high quality reconstructed images of simple hot spot configurations with positional errors of less than one millimeter are possible within acquisition times as short as 15s.
Background
We aimed to improve the image quality (IQ) of sparse-view computed tomography (CT) images using a U-Net for lung metastasis detection and determine the best tradeoff between number of ...views, IQ, and diagnostic confidence.
Methods
CT images from 41 subjects aged 62.8 ± 10.6 years (mean ± standard deviation, 23 men), 34 with lung metastasis, 7 healthy, were retrospectively selected (2016–2018) and forward projected onto 2,048-view sinograms. Six corresponding sparse-view CT data subsets at varying levels of undersampling were reconstructed from sinograms using filtered backprojection with 16, 32, 64, 128, 256, and 512 views. A dual-frame U-Net was trained and evaluated for each subsampling level on 8,658 images from 22 diseased subjects. A representative image per scan was selected from 19 subjects (12 diseased, 7 healthy) for a single-blinded multireader study. These slices, for all levels of subsampling, with and without U-Net postprocessing, were presented to three readers. IQ and diagnostic confidence were ranked using predefined scales. Subjective nodule segmentation was evaluated using sensitivity and Dice similarity coefficient (DSC); clustered Wilcoxon signed-rank test was used.
Results
The 64-projection sparse-view images resulted in 0.89 sensitivity and 0.81 DSC, while their counterparts, postprocessed with the U-Net, had improved metrics (0.94 sensitivity and 0.85 DSC) (
p
= 0.400). Fewer views led to insufficient IQ for diagnosis. For increased views, no substantial discrepancies were noted between sparse-view and postprocessed images.
Conclusions
Projection views can be reduced from 2,048 to 64 while maintaining IQ and the confidence of the radiologists on a satisfactory level.
Relevance statement
Our reader study demonstrates the benefit of U-Net postprocessing for regular CT screenings of patients with lung metastasis to increase the IQ and diagnostic confidence while reducing the dose.
Key points
• Sparse-projection-view streak artifacts reduce the quality and usability of sparse-view CT images.
• U-Net-based postprocessing removes sparse-view artifacts while maintaining diagnostically accurate IQ.
• Postprocessed sparse-view CTs drastically increase radiologists’ confidence in diagnosing lung metastasis.
Graphical Abstract
•A new multi-stage multi-modal learning algorithm for multi-label skin lesion classification.•First stage enables fusion of clinical and dermoscopy images at both feature-level and ...decision-level.•Second stage effectively integrate the information from image-modality and patients metadata.•Our method can achieve state-of-the-art performance for multi-label skin lesion classification.
Skin disease is one of the most common diseases in the world. Deep learning-based methods have achieved excellent skin lesion recognition performance, most of which are based on only dermoscopy images. In recent works that use multi-modality data (patient’s meta-data, clinical images, and dermoscopy images), the methods adopt a one-stage fusion approach and only optimize the information fusion at the feature level. These methods do not use information fusion at the decision level and thus cannot fully use the data of all modalities. This work proposes a novel two-stage multi-modal learning algorithm (FusionM4Net) for multi-label skin diseases classification. At the first stage, we construct a FusionNet, which exploits and integrates the representation of clinical and dermoscopy images at the feature level, and then uses a Fusion Scheme 1 to conduct the information fusion at the decision level. At the second stage, to further incorporate the patient’s meta-data, we propose a Fusion Scheme 2, which integrates the multi-label predictive information from the first stage and patient’s meta-data information to train an SVM cluster. The final diagnosis is formed by the fusion of the predictions from the first and second stages. Our algorithm was evaluated on the seven-point checklist dataset, a well-established multi-modality multi-label skin disease dataset. Without using the patient’s meta-data, the proposed FusionM4Net’s first stage (FusionM4Net-FS) achieved an average accuracy of 75.7% for multi-classification tasks and 74.9% for diagnostic tasks, which is more accurate than other state-of-the-art methods. By further fusing the patient’s meta-data at FusionM4Net’s second stage (FusionM4Net-SS), the entire FusionM4Net finally boosts the average accuracy to 77.0% and the diagnostic accuracy to 78.5%, which indicates its robust and excellent classification performance on the label-imbalanced dataset. The corresponding code is available at: https://github.com/pixixiaonaogou/MLSDR.
Background
Deep learning models are being applied to more and more use cases with astonishing success stories, but how do they perform in the real world? Models are typically tested on specific ...cleaned data sets, but when deployed in the real world, the model will encounter unexpected, out‐of‐distribution (OOD) data.
Purpose
To investigate the impact of OOD radiographs on existing chest x‐ray classification models and to increase their robustness against OOD data.
Methods
The study employed the commonly used chest x‐ray classification model, CheXnet, trained on the chest x‐ray 14 data set, and tested its robustness against OOD data using three public radiography data sets: IRMA, Bone Age, and MURA, and the ImageNet data set. To detect OOD data for multi‐label classification, we proposed in‐distribution voting (IDV). The OOD detection performance is measured across data sets using the area under the receiver operating characteristic curve (AUC) analysis and compared with Mahalanobis‐based OOD detection, MaxLogit, MaxEnergy, self‐supervised OOD detection (SS OOD), and CutMix.
Results
Without additional OOD detection, the chest x‐ray classifier failed to discard any OOD images, with an AUC of 0.5. The proposed IDV approach trained on ID (chest x‐ray 14) and OOD data (IRMA and ImageNet) achieved, on average, 0.999 OOD AUC across the three data sets, surpassing all other OOD detection methods. Mahalanobis‐based OOD detection achieved an average OOD detection AUC of 0.982. IDV trained solely with a few thousand ImageNet images had an AUC 0.913, which was considerably higher than MaxLogit (0.726), MaxEnergy (0.724), SS OOD (0.476), and CutMix (0.376).
Conclusions
The performance of all tested OOD detection methods did not translate well to radiography data sets, except Mahalanobis‐based OOD detection and the proposed IDV method. Consequently, training solely on ID data led to incorrect classification of OOD images as ID, resulting in increased false positive rates. IDV substantially improved the model's ID classification performance, even when trained with data that will not occur in the intended use case or test set (ImageNet), without additional inference overhead or performance decrease in the target classification. The corresponding code is available at https://gitlab.lrz.de/IP/a‐knee‐cannot‐have‐lung‐disease.
PDF
