The proposed model for automatic clinical image caption generation combines the analysis of radiological scans with structured patient information from the textual records. It uses two language ...models, the Show-Attend-Tell and the GPT-3, to generate comprehensive and descriptive radiology records. The generated textual summary contains essential information about pathologies found, their location, along with the 2D heatmaps that localize each pathology on the scans. The model has been tested on two medical datasets, the Open-I, MIMIC-CXR, and the general-purpose MS-COCO, and the results measured with natural language assessment metrics demonstrated its efficient applicability to chest X-ray image captioning.
Compressed sensing is commonly concerned with optimizing the image quality after a partial undersampling of the measurable k-space to accelerate MRI. In this article, we propose to change the focus ...from the quality of the reconstructed image to the quality of the downstream image analysis outcome. Specifically, we propose to optimize the patterns according to how well a sought-after pathology could be detected or localized in the reconstructed images. We find the optimal undersampling patterns in k-space that maximize target value functions of interest in commonplace medical vision problems (reconstruction, segmentation, and classification) and propose a new iterative gradient sampling routine universally suitable for these tasks. We validate the proposed MRI acceleration paradigm on three classical medical datasets, demonstrating a noticeable improvement of the target metrics at the high acceleration factors (for the segmentation problem at ×16 acceleration, we report up to 12% improvement in Dice score over the other undersampling patterns).
•New paradigm for accelerating MRI, where k-space is undersampled intelligently for the benefit of downstream image analysis.•In this paradigm, the learned pattern can ruin the look of the image, while boosting the target metrics.•Our iterative gradient sampling (IGS) algorithm optimizes undersampling patterns for specific medical vision tasks and applications (cardiac, neurological, and orthopedic utility confirmed).•Noticeable improvement to target value metrics compared to the other undersampling patterns, such as equispaced or central masks.•The method proves especially instrumental at the highest acceleration factors when good image quality is impossible to obtain.
We propose a sparse computation method for optimizing the inference of neural networks in reinforcement learning (RL) tasks. Motivated by the processing abilities of the brain, this method combines ...simple neural network pruning with a delta-network algorithm to account for the input data correlations. The former mimics neuroplasticity by eliminating inefficient connections; the latter makes it possible to update neuron states only when their changes exceed a certain threshold. This combination significantly reduces the number of multiplications during the neural network inference for fast neuromorphic computing. We tested the approach in popular deep RL tasks, yielding up to a 100-fold reduction in the number of required multiplications without substantial performance loss (sometimes, the performance even improved).
Anomaly detection is the problem of recognizing abnormal inputs based on the seen examples of normal data. Despite recent advances of deep learning in recognizing image anomalies, these methods still ...prove incapable of handling complex images, such as those encountered in the medical domain. Barely visible abnormalities in chest X-rays or metastases in lymph nodes on the scans of the pathology slides resemble normal images and are very difficult to detect. To address this problem, we introduce a new powerful method of image anomaly detection. It relies on the classical autoencoder approach with a re-designed training pipeline to handle high-resolution, complex images, and a robust way of computing an image abnormality score. We revisit the very problem statement of fully unsupervised anomaly detection, where no abnormal examples are provided during the model setup. We propose to relax this unrealistic assumption by using a very small number of anomalies of confined variability merely to initiate the search of hyperparameters of the model. We evaluate our solution on two medical datasets containing radiology and digital pathology images, where the state-of-the-art anomaly detection models, originally devised for natural image benchmarks, fail to perform sufficiently well. The proposed approach suggests a new baseline for anomaly detection in medical image analysis tasks.
Parkinson's disease (PD) is the second most common neurodegenerative disorder that affects the extrapyramidal motor system. The initial clinical symptoms can appear long before the retirement age, ...affecting the patients' ability to continue their work. However, the modern healthcare lacks the apparatus to detect the early signs of the disease, with only selected experts being able to spot the onset. Moreover, even if the disease is detected on time, the current clinical routine has no means of monitoring the progression of the disease, nor the effect of the therapy. In this work, we perform a comparative study of three patient-driven monitoring approaches, including wearable sensors data, video of the routine exercises performed by the patients, and their handwriting- all fine-tuned to detect PD. Aspiring to create an inexpensive clinical workflow, we collect the sensor data using a commercial wearable wireless sensor, and the video and the handwriting data are recorded using a basic smartphone. In the analysis stage, we rely on machine and deep learning methods, using <inline-formula> <tex-math notation="LaTeX">f1 </tex-math></inline-formula>-score for gauging the performance of the patient-driven data acquisition approaches. The sensor data analysis demonstrates the best <inline-formula> <tex-math notation="LaTeX">f1 </tex-math></inline-formula>-score of 0.93. At the same time, the handwriting and remote video recordings are more comfortable and less time-consuming, offering a sensible trade-off for the patients at the later stages of PD.
From night vision and objects overwhelmed by sunlight to jammed signals and those that are purposely encrypted, detecting low-level or hidden signals is a fundamental problem in imaging. Here, we ...develop and exploit a new type of stochastic resonance, in which nonlinear coupling allows signals to grow at the expense of noise, to recover noise-hidden images propagating in a self-focusing medium. The growth rate is derived analytically by treating the signal-noise interaction as a photonic beam-plasma instability and matches experimentally measured resonances in coupling strength, noise statistics and modal content of the signal. This is the first observation of nonlinear intensity exchange between coherent and spatially incoherent light and the first demonstration of spatial coherence resonance for a dynamically evolving signal. The results suggest a general method of reconstructing images through seeded instability and confirm information limits predicted, but not yet observed, in nonlinear communications systems.
Detecting Out-of-Distribution (OoD) data is one of the greatest challenges in safe and robust deployment of machine learning algorithms in medicine. When the algorithms encounter cases that deviate ...from the distribution of the training data, they often produce incorrect and over-confident predictions. OoD detection algorithms aim to catch erroneous predictions in advance by analysing the data distribution and detecting potential instances of failure. Moreover, flagging OoD cases may support human readers in identifying incidental findings. Due to the increased interest in OoD algorithms, benchmarks for different domains have recently been established. In the medical imaging domain, for which reliable predictions are often essential, an open benchmark has been missing. We introduce the Medical-Out-Of-Distribution-Analysis-Challenge (MOOD) as an open, fair, and unbiased benchmark for OoD methods in the medical imaging domain. The analysis of the submitted algorithms shows that performance has a strong positive correlation with the perceived difficulty, and that all algorithms show a high variance for different anomalies, making it yet hard to recommend them for clinical practice. We also see a strong correlation between challenge ranking and performance on a simple toy test set, indicating that this might be a valuable addition as a proxy dataset during anomaly detection algorithm development.
Clinical examination of three-dimensional image data of compound anatomical objects, such as complex joints, remains a tedious process, demanding the time and the expertise of physicians. For ...instance, automation of the segmentation task of the TMJ (temporomandibular joint) has been hindered by its compound three-dimensional shape, multiple overlaid textures, an abundance of surrounding irregularities in the skull, and a virtually omnidirectional range of the jaw's motion-all of which extend the manual annotation process to more than an hour per patient. To address the challenge, we invent a new workflow for the 3D segmentation task: namely, we propose to segment empty spaces between all the tissues surrounding the object-the so-called negative volume segmentation. Our approach is an end-to-end pipeline that comprises a V-Net for bone segmentation, a 3D volume construction by inflation of the reconstructed bone head in all directions along the normal vector to its mesh faces. Eventually confined within the skull bones, the inflated surface occupies the entire "negative" space in the joint, effectively providing a geometrical/topological metric of the joint's health. We validate the idea on the CT scans in a 50-patient dataset, annotated by experts in maxillofacial medicine, quantitatively compare the asymmetry given the left and the right negative volumes, and automate the entire framework for clinical adoption.
We propose the first data augmentation method based on optimal transport theory, with the generated data being guaranteed to belong to the original data manifold. The proposed algorithm randomly ...samples a clique in the nearest-neighbors graph representing the data knowledge and computes the Wasserstein barycenter between the neighbours with random uniform weights. Being extremely natural-looking, many such barycenters are then produced iteratively to overpopulate the original dataset. We apply this approach to the problem of landmarks detection in unsupervised and semi-supervised scenarios in the popular tasks of face keypoints extraction, pose detection, and the segmentation of anatomical contours in medical imaging. The barycentric oversampling approach is shown to outperform state-of-the-art data augmentation methods. The code is available at https://github.com/cviaai/LAMBO/.
Abstract Epigenetic aging clocks have been widely used to validate rejuvenation effects during cellular reprogramming. However, these predictions are unverifiable because the true biological age of ...reprogrammed cells remains unknown. We present an analytical framework to consider rejuvenation predictions from the uncertainty perspective. Our analysis reveals that the DNA methylation profiles across reprogramming are poorly represented in the aging data used to train clock models, thus introducing high epistemic uncertainty in age estimations. Moreover, predictions of different published clocks are inconsistent, with some even suggesting zero or negative rejuvenation. While not questioning the possibility of age reversal, we show that the high clock uncertainty challenges the reliability of rejuvenation effects observed during in vitro reprogramming before pluripotency and throughout embryogenesis. Conversely, our method reveals a significant age increase after in vivo reprogramming. We recommend including uncertainty estimation in future aging clock models to avoid the risk of misinterpreting the results of biological age prediction.
PDF
