Musculoskeletal models have been widely used for detailed biomechanical analysis to characterise various functional impairments given their ability to estimate movement variables (i.e., muscle forces ...and joint moments) which cannot be readily measured in vivo. Physics-based computational neuromusculoskeletal models can interpret the dynamic interaction between neural drive to muscles, muscle dynamics, body and joint kinematics and kinetics. Still, such set of solutions suffers from slowness, especially for the complex models, hindering the utility in real-time applications. In recent years, data-driven methods have emerged as a promising alternative due to the benefits in speedy and simple implementation, but they cannot reflect the underlying neuromechanical processes. This paper proposes a physics-informed deep learning framework for musculoskeletal modelling, where physics-based domain knowledge is brought into the data-driven model as soft constraints to penalise/regularise the data-driven model. We use the synchronous muscle forces and joint kinematics prediction from surface electromyogram (sEMG) as the exemplar to illustrate the proposed framework. Convolutional neural network (CNN) is employed as the deep neural network to implement the proposed framework. Simultaneously, the physics law between muscle forces and joint kinematics is used the soft constraint. Experimental validations on two groups of data, including one benchmark dataset and one self-collected dataset from six healthy subjects, are performed. The experimental results demonstrate the effectiveness and robustness of the proposed framework.
Recent works show that generative adversarial networks (GANs) can be successfully applied to image synthesis and semi-supervised learning, where, given a small labeled database and a large unlabeled ...database, the goal is to train a powerful classifier. In this paper, we trained a retinal image synthesizer and a semi-supervised learning method for automatic glaucoma assessment using an adversarial model on a small glaucoma-labeled database and a large unlabeled database. Various studies have shown that glaucoma can be monitored by analyzing the optic disc and its surroundings, and for that reason, the images used in this paper were automatically cropped around the optic disc. The novelty of this paper is to propose a new retinal image synthesizer and a semi-supervised learning method for glaucoma assessment based on the deep convolutional GANs. In addition, and to the best of our knowledge, this system is trained on an unprecedented number of publicly available images (86926 images). This system, hence, is not only able to generate images synthetically but to provide labels automatically. Synthetic images were qualitatively evaluated using t-SNE plots of features associated with the images and their anatomical consistency was estimated by measuring the proportion of pixels corresponding to the anatomical structures around the optic disc. The resulting image synthesizer is able to generate realistic (cropped) retinal images, and subsequently, the glaucoma classifier is able to classify them into glaucomatous and normal with high accuracy (AUC = 0.9017). The obtained retinal image synthesizer and the glaucoma classifier could then be used to generate an unlimited number of cropped retinal images with glaucoma labels.
Abstract Late-onset dementia is a major health concern in the ageing population. Alzheimer's disease (AD) accounts for the largest proportion (65–70%) of dementia cases in the older population. ...Despite considerable research effort, the pathogenesis of late-onset AD remains unclear. Substantial evidence suggests that the neurodegenerative process is initiated by chronic cerebral hypoperfusion (CCH) caused by ageing and cardiovascular conditions. CCH causes reduced oxygen, glucose and other nutrient supply to the brain, with direct damage not only to the parenchymal cells, but also to the blood–brain barrier (BBB), a key mediator of cerebral homeostasis. BBB dysfunction mediates the indirect neurotoxic effects of CCH by promoting oxidative stress, inflammation, paracellular permeability, and dysregulation of nitric oxide, a key regulator of regional blood flow. As such, BBB dysfunction mediates a vicious circle in which cerebral perfusion is reduced further and the neurodegenerative process is accelerated. Endothelial interaction with pericytes and astrocytes could also play a role in the process. Reciprocal interactions between vascular dysfunction and neurodegeneration could further contribute to the development of the disease. A comprehensive overview of the complex scenario of interacting endothelium-mediated processes is currently lacking, and could prospectively contribute to the identification of adequate therapeutic interventions. This study reviews the current literature of in vitro and ex vivo studies on endothelium-mediated mechanisms underlying vascular dysfunction in AD pathogenesis, with the aim of presenting a comprehensive overview of the complex network of causative relationships. Particular emphasis is given to vicious circles which can accelerate the process of neurovascular degeneration.
UK Biobank is a population-based cohort of half a million participants aged 40-69 years recruited between 2006 and 2010. In 2014, UK Biobank started the world's largest multi-modal imaging study, ...with the aim of re-inviting 100,000 participants to undergo brain, cardiac and abdominal magnetic resonance imaging, dual-energy X-ray absorptiometry and carotid ultrasound. The combination of large-scale multi-modal imaging with extensive phenotypic and genetic data offers an unprecedented resource for scientists to conduct health-related research. This article provides an in-depth overview of the imaging enhancement, including the data collected, how it is managed and processed, and future directions.
•A new curvilinear structure segmentation network is proposed based on dual self-attention modules, which can deal with both 2D and 3D imaging modalities in an unified manner.•Two self-attention ...mechanisms are employed in the channel and spatial spaces to generate attention-aware expressive features. They can enhance the network to capture long-range dependencies and make an effective use of the multi-channel space for feature representation and normalization, enabling the network to classify the curvilinear structure from background more effectively.•Experimental results on nine datasets (six 2D datasets and three 3D datasets) demonstrate that our proposed method achieves on the whole state-of-the-art performances in detecting curvilinear structures from different biomedical imaging modalities both quantitatively and qualitatively.
Display omitted
Automated detection of curvilinear structures, e.g., blood vessels or nerve fibres, from medical and biomedical images is a crucial early step in automatic image interpretation associated to the management of many diseases. Precise measurement of the morphological changes of these curvilinear organ structures informs clinicians for understanding the mechanism, diagnosis, and treatment of e.g. cardiovascular, kidney, eye, lung, and neurological conditions. In this work, we propose a generic and unified convolution neural network for the segmentation of curvilinear structures and illustrate in several 2D/3D medical imaging modalities. We introduce a new curvilinear structure segmentation network (CS2-Net), which includes a self-attention mechanism in the encoder and decoder to learn rich hierarchical representations of curvilinear structures. Two types of attention modules - spatial attention and channel attention - are utilized to enhance the inter-class discrimination and intra-class responsiveness, to further integrate local features with their global dependencies and normalization, adaptively. Furthermore, to facilitate the segmentation of curvilinear structures in medical images, we employ a 1×3 and a 3×1 convolutional kernel to capture boundary features. Besides, we extend the 2D attention mechanism to 3D to enhance the network’s ability to aggregate depth information across different layers/slices. The proposed curvilinear structure segmentation network is thoroughly validated using both 2D and 3D images across six different imaging modalities. Experimental results across nine datasets show the proposed method generally outperforms other state-of-the-art algorithms in various metrics.
•We design a similarity-aware graph in GCN to consider disease statuses.•We propose an adaptive mechanism to estimate similarities.•We propose a calibration mechanism to fuse functional and ...structural information.•We obtain promising performance for disease deterioration prediction.
Display omitted
Graph convolution networks (GCN) have been successfully applied in disease prediction tasks as they capture interactions (i.e., edges and edge weights on the graph) between individual elements. The interactions in existing works are constructed by fusing similarity between imaging information and distance between non-imaging information, whereas disregarding the disease status of those individuals in the training set. Besides, the similarity is being evaluated by computing the correlation distance between feature vectors, which limits prediction performance, especially for predicting significant memory concern (SMC) and mild cognitive impairment (MCI). In this paper, we propose three mechanisms to improve GCN, namely similarity-aware adaptive calibrated GCN (SAC-GCN), for predicting SMC and MCI. First, we design a similarity-aware graph using different receptive fields to consider disease status. The labelled subjects on the graph are only connected with those labelled subjects with the same status. Second, we propose an adaptive mechanism to evaluate similarity. Specifically, we construct initial GCN with evaluating similarity by using traditional correlation distance, then pre-train the initial GCN by using training samples and use it to score all subjects. Then, the difference between these scores replaces correlation distance to update similarity. Last, we devise a calibration mechanism to fuse functional magnetic resonance imaging (fMRI) and diffusion tensor imaging (DTI) information into edges. The proposed method is tested on the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset. Experimental results demonstrate that our proposed method is useful to predict disease-induced deterioration and superior to other related algorithms, with a mean classification accuracy of 86.83% in our prediction tasks.
Deep subcortical lesions (DSCL) of the brain, are present in ~60% of the ageing population, and are linked to cognitive decline and depression. DSCL are associated with demyelination, blood brain ...barrier (BBB) dysfunction, and microgliosis. Microglia are the main immune cell of the brain. Under physiological conditions microglia have a ramified morphology, and react to pathology with a change to a more rounded morphology as well as showing protein expression alterations. This study builds on previous characterisations of DSCL and radiologically 'normal-appearing' white matter (NAWM) by performing a detailed characterisation of a range of microglial markers in addition to markers of vascular integrity. The Cognitive Function and Ageing Study (CFAS) provided control white matter (WM), NAWM and DSCL human post mortem tissue for immunohistochemistry using microglial markers (Iba-1, CD68 and MHCII), a vascular basement membrane marker (collagen IV) and markers of BBB integrity (fibrinogen and aquaporin 4). The immunoreactive profile of CD68 increased in a stepwise manner from control WM to NAWM to DSCL. This correlated with a shift from small, ramified cells, to larger, more rounded microglia. While there was greater Iba-1 immunoreactivity in NAWM compared to controls, in DSCL, Iba-1 levels were reduced to control levels. A prominent feature of these DSCL was a population of Iba-1-/CD68+ microglia. There were increases in collagen IV, but no change in BBB integrity. Overall the study shows significant differences in the immunoreactive profile of microglial markers. Whether this is a cause or effect of lesion development remains to be elucidated. Identifying microglia subpopulations based on their morphology and molecular markers may ultimately help decipher their function and role in neurodegeneration. Furthermore, this study demonstrates that Iba-1 is not a pan-microglial marker, and that a combination of several microglial markers is required to fully characterise the microglial phenotype.
Deep learning in medical image registration Chen, Xiang; Diaz-Pinto, Andres; Ravikumar, Nishant ...
Progress in biomedical engineering,
01/2021, Volume:
3, Issue:
1
Journal Article
Peer reviewed
Open access
Image registration is a fundamental task in multiple medical image analysis applications. With the advent of deep learning, there have been significant advances in algorithmic performance for various ...computer vision tasks in recent years, including medical image registration. The last couple of years have seen a dramatic increase in the development of deep learning-based medical image registration algorithms. Consequently, a comprehensive review of the current state-of-the-art algorithms in the field is timely, and necessary. This review is aimed at understanding the clinical applications and challenges that drove this innovation, analysing the functionality and limitations of existing approaches, and at providing insights to open challenges and as yet unmet clinical needs that could shape future research directions. To this end, the main contributions of this paper are: (a) discussion of all deep learning-based medical image registration papers published since 2013 with significant methodological and/or functional contributions to the field; (b) analysis of the development and evolution of deep learning-based image registration methods, summarising the current trends and challenges in the domain; and (c) overview of unmet clinical needs and potential directions for future research in deep learning-based medical image registration.
Multi-modality medical imaging is increasingly used for comprehensive assessment of complex diseases in either diagnostic examinations or as part of medical research trials. Different imaging ...modalities provide complementary information about living tissues. However, multi-modal examinations are not always possible due to adversary factors, such as patient discomfort, increased cost, prolonged scanning time, and scanner unavailability. In additionally, in large imaging studies, incomplete records are not uncommon owing to image artifacts, data corruption or data loss, which compromise the potential of multi-modal acquisitions. In this paper, we propose a weakly coupled and geometry co-regularized joint dictionary learning method to address the problem of cross-modality synthesis while considering the fact that collecting the large amounts of training data is often impractical. Our learning stage requires only a few registered multi-modality image pairs as training data. To employ both paired images and a large set of unpaired data, a cross-modality image matching criterion is proposed. Then, we propose a unified model by integrating such a criterion into the joint dictionary learning and the observed common feature space for associating cross-modality data for the purpose of synthesis. Furthermore, two regularization terms are added to construct robust sparse representations. Our experimental results demonstrate superior performance of the proposed model over state-of-the-art methods.
In this paper, a new technique coined two-dimensional principal component analysis (2DPCA) is developed for image representation. As opposed to PCA, 2DPCA is based on 2D image matrices rather than 1D ...vectors so the image matrix does not need to be transformed into a vector prior to feature extraction. Instead, an image covariance matrix is constructed directly using the original image matrices, and its eigenvectors are derived for image feature extraction. To test 2DPCA and evaluate its performance, a series of experiments were performed on three face image databases: ORL, AR, and Yale face databases. The recognition rate across all trials was higher using 2DPCA than PCA. The experimental results also indicated that the extraction of image features is computationally more efficient using 2DPCA than PCA.
PDF
