Image synthesis is expected to provide value for the translation of machine learning methods into clinical practice. Fundamental problems like model robustness, domain transfer, causal modelling, and ...operator training become approachable through synthetic data. Especially, heavily operator-dependant modalities like Ultrasound imaging require robust frameworks for image and video generation. So far, video generation has only been possible by providing input data that is as rich as the output data, e.g., image sequence plus conditioning in, video out. However, clinical documentation is usually scarce and only single images are reported and stored, thus retrospective patient-specific analysis or the generation of rich training data becomes impossible with current approaches. In this paper, we extend elucidated diffusion models for video modelling to generate plausible video sequences from single images and arbitrary conditioning with clinical parameters. We explore this idea within the context of echocardiograms by looking into the variation of the Left Ventricle Ejection Fraction, the most essential clinical metric gained from these examinations. We use the publicly available EchoNet-Dynamic dataset for all our experiments. Our image to sequence approach achieves an \(R^2\) score of 93%, which is 38 points higher than recently proposed sequence to sequence generation methods. Code and models will be available at: https://github.com/HReynaud/EchoDiffusion.
Two key questions in cardiac image analysis are to assess the anatomy and motion of the heart from images; and to understand how they are associated with non-imaging clinical factors such as gender, ...age and diseases. While the first question can often be addressed by image segmentation and motion tracking algorithms, our capability to model and to answer the second question is still limited. In this work, we propose a novel conditional generative model to describe the 4D spatio-temporal anatomy of the heart and its interaction with non-imaging clinical factors. The clinical factors are integrated as the conditions of the generative modelling, which allows us to investigate how these factors influence the cardiac anatomy. We evaluate the model performance in mainly two tasks, anatomical sequence completion and sequence generation. The model achieves a high performance in anatomical sequence completion, comparable to or outperforming other state-of-the-art generative models. In terms of sequence generation, given clinical conditions, the model can generate realistic synthetic 4D sequential anatomies that share similar distributions with the real data.
Quantitative cardiac magnetic resonance T1 and T2 mapping enable myocardial
tissue characterisation but the lengthy scan times restrict their widespread
clinical application. We propose a deep ...learning method that incorporates a
time dependency Latent Transformer module to model relationships between
parameterised time frames for improved reconstruction from undersampled data.
The module, implemented as a multi-resolution sequence-to-sequence transformer,
is integrated into an encoder-decoder architecture to leverage the inherent
temporal correlations in relaxation processes. The presented results for
accelerated T1 and T2 mapping show the model recovers maps with higher fidelity
by explicit incorporation of time dynamics. This work demonstrates the
importance of temporal modelling for artifact-free reconstruction in
quantitative MRI.
Medical vision-language models enable co-learning and integrating features from medical imaging and clinical text. However, these models are not easy to train and the latent representation space can ...be complex. Here we propose a novel way for pre-training and regularising medical vision-language models. The proposed method, named Medical vision-language pre-training with Frozen language models and Latent spAce Geometry optimization (M-FLAG), leverages a frozen language model for training stability and efficiency and introduces a novel orthogonality loss to harmonize the latent space geometry. We demonstrate the potential of the pre-trained model on three downstream tasks: medical image classification, segmentation, and object detection. Extensive experiments across five public datasets demonstrate that M-FLAG significantly outperforms existing medical vision-language pre-training approaches and reduces the number of parameters by 78\%. Notably, M-FLAG achieves outstanding performance on the segmentation task while using only 1\% of the RSNA dataset, even outperforming ImageNet pre-trained models that have been fine-tuned using 100\% of the data.
Data augmentation has become a de facto component of deep learning-based medical image segmentation methods. Most data augmentation techniques used in medical imaging focus on spatial and intensity ...transformations to improve the diversity of training images. They are often designed at the image level, augmenting the full image, and do not pay attention to specific abnormalities within the image. Here, we present LesionMix, a novel and simple lesion-aware data augmentation method. It performs augmentation at the lesion level, increasing the diversity of lesion shape, location, intensity and load distribution, and allowing both lesion populating and inpainting. Experiments on different modalities and different lesion datasets, including four brain MR lesion datasets and one liver CT lesion dataset, demonstrate that LesionMix achieves promising performance in lesion image segmentation, outperforming several recent Mix-based data augmentation methods. The code will be released at https://github.com/dogabasaran/lesionmix.
Understanding the intensity characteristics of brain lesions is key for defining image-based biomarkers in neurological studies and for predicting disease burden and outcome. In this work, we present ...a novel foreground-based generative method for modelling the local lesion characteristics that can both generate synthetic lesions on healthy images and synthesize subject-specific pseudo-healthy images from pathological images. Furthermore, the proposed method can be used as a data augmentation module to generate synthetic images for training brain image segmentation networks. Experiments on multiple sclerosis (MS) brain images acquired on magnetic resonance imaging (MRI) demonstrate that the proposed method can generate highly realistic pseudo-healthy and pseudo-pathological brain images. Data augmentation using the synthetic images improves the brain image segmentation performance compared to traditional data augmentation methods as well as a recent lesion-aware data augmentation technique, CarveMix. The code will be released at https://github.com/dogabasaran/lesion-synthesis.
Brain lesion segmentation plays an essential role in neurological research and diagnosis. As brain lesions can be caused by various pathological alterations, different types of brain lesions tend to ...manifest with different characteristics on different imaging modalities. Due to this complexity, brain lesion segmentation methods are often developed in a task-specific manner. A specific segmentation model is developed for a particular lesion type and imaging modality. However, the use of task-specific models requires predetermination of the lesion type and imaging modality, which complicates their deployment in real-world scenarios. In this work, we propose a universal foundation model for 3D brain lesion segmentation, which can automatically segment different types of brain lesions for input data of various imaging modalities. We formulate a novel Mixture of Modality Experts (MoME) framework with multiple expert networks attending to different imaging modalities. A hierarchical gating network combines the expert predictions and fosters expertise collaboration. Furthermore, we introduce a curriculum learning strategy during training to avoid the degeneration of each expert network and preserve their specialization. We evaluated the proposed method on nine brain lesion datasets, encompassing five imaging modalities and eight lesion types. The results show that our model outperforms state-of-the-art universal models and provides promising generalization to unseen datasets.
Cardiovascular disease, the leading cause of death globally, is an age-related disease. Understanding the morphological and functional changes of the heart during ageing is a key scientific question, ...the answer to which will help us define important risk factors of cardiovascular disease and monitor disease progression. In this work, we propose a novel conditional generative model to describe the changes of 3D anatomy of the heart during ageing. The proposed model is flexible and allows integration of multiple clinical factors (e.g. age, gender) into the generating process. We train the model on a large-scale cross-sectional dataset of cardiac anatomies and evaluate on both cross-sectional and longitudinal datasets. The model demonstrates excellent performance in predicting the longitudinal evolution of the ageing heart and modelling its data distribution. The codes are available at https://github.com/MengyunQ/AgeHeart.
This paper presents a fully automated method to segment the complex left atrial (LA) cavity, from 3D Gadolinium-enhanced magnetic resonance imaging (GE-MRI) scans. The proposed method consists of ...four steps: (1) preprocessing to convert the original GE-MRI to a probability map, (2) atlas selection to match the atlases to the target image, (3) multi-atlas registration and fusion, and (4) level-set refinement. The method was evaluated on the datasets provided by the MICCAI 2018 STACOM Challenge with 100 dataset for training. Compared to manual annotation, the proposed method achieved an average Dice overlap index of 0.88.
Design and synthesis of porous and hollow carbon spheres have attracted considerable interest in the past decade due to their superior physicochemical properties and widespread applications. However, ...it is still a big challenge to achieve controllable synthesis of hollow carbon nanospheres with center‐radial large mesopores in the shells and inner surface roughness. Herein, porous hollow carbon nanospheres (PHCNs) are successfully synthesized with tunable center‐radial mesopore channels in the shells and crater‐like inner surfaces by employing dendrimer‐like mesoporous silica nanospheres (DMSNs) as hard templates. Compared with conventional mesoporous nanospheres, DMSN templates not only result in the formation of center‐radial large mesopores in the shells, but also produce a crater‐like inner surface. PHCNs can be tuned from open center‐radial mesoporous shells to relatively closed microporous shells. After functionalization with polyethyleneimine (PEI) and poly(ethylene glycol) (PEG), PHCNs not only have negligible cytotoxicity, excellent photothermal property, and high coloading capacity of 482 µg of doxorubicin and 44 µg of siRNA per mg, but can also efficiently deliver these substances into cells, thus displaying enhanced cancer cell killing capacity by triple‐combination therapy.
Porous hollow carbon nanospheres (PHCNs) with tunable hierarchical pores in the shells, crater‐like inner surface, and high surface area and pore volume are fabricated by nanocasting using unique dendrimer‐like mesoporous silica nanospheres as hard templates. The functionalized PHCNs can codeliver drug and gene into cancer cells and achieve efficient synergistic treatment combined with photothermal therapy.
PDF