Current advancements in sensor technology bring new possibilities in multi- and hyperspectral imaging. Real-life use cases which can benefit from such imagery span across various domains, including ...precision agriculture, chemistry, biology, medicine, land cover applications, management of natural resources, detecting natural disasters, and more. To extract value from such highly dimensional data capturing up to hundreds of spectral bands in the electromagnetic spectrum, researchers have been developing a range of image processing and machine learning analysis pipelines to process these kind of data as efficiently as possible. To this end, multi- or hyperspectral analysis has bloomed and has become an exciting research area which can enable the faster adoption of this technology in practice, also when such algorithms are deployed in hardware-constrained and extreme execution environments; e.g., on-board imaging satellites.
•A systematic overview on integrating medical domain knowledge into deep models.•Different kinds of domain knowledge and their integrating methods are summarized.•Challenges and future directions of ...integrating domain knowledge are discussed.
Although deep learning models like CNNs have achieved great success in medical image analysis, the small size of medical datasets remains a major bottleneck in this area. To address this problem, researchers have started looking for external information beyond current available medical datasets. Traditional approaches generally leverage the information from natural images via transfer learning. More recent works utilize the domain knowledge from medical doctors, to create networks that resemble how medical doctors are trained, mimic their diagnostic patterns, or focus on the features or areas they pay particular attention to. In this survey, we summarize the current progress on integrating medical domain knowledge into deep learning models for various tasks, such as disease diagnosis, lesion, organ and abnormality detection, lesion and organ segmentation. For each task, we systematically categorize different kinds of medical domain knowledge that have been utilized and their corresponding integrating methods. We also provide current challenges and directions for future research.
Histopathology image analysis serves as the gold standard for cancer diagnosis. Efficient and precise diagnosis is quite critical for the subsequent therapeutic treatment of patients. So far, ...computer-aided diagnosis has not been widely applied in pathological field yet as currently well-addressed tasks are only the tip of the iceberg. Whole slide image (WSI) classification is a quite challenging problem. First, the scarcity of annotations heavily impedes the pace of developing effective approaches. Pixelwise delineated annotations on WSIs are time consuming and tedious, which poses difficulties in building a large-scale training dataset. In addition, a variety of heterogeneous patterns of tumor existing in high magnification field are actually the major obstacle. Furthermore, a gigapixel scale WSI cannot be directly analyzed due to the immeasurable computational cost. How to design the weakly supervised learning methods to maximize the use of available WSI-level labels that can be readily obtained in clinical practice is quite appealing. To overcome these challenges, we present a weakly supervised approach in this article for fast and effective classification on the whole slide lung cancer images. Our method first takes advantage of a patch-based fully convolutional network (FCN) to retrieve discriminative blocks and provides representative deep features with high efficiency. Then, different context-aware block selection and feature aggregation strategies are explored to generate globally holistic WSI descriptor which is ultimately fed into a random forest (RF) classifier for the image-level prediction. To the best of our knowledge, this is the first study to exploit the potential of image-level labels along with some coarse annotations for weakly supervised learning. A large-scale lung cancer WSI dataset is constructed in this article for evaluation, which validates the effectiveness and feasibility of the proposed method. Extensive experiments demonstrate the superior performance of our method that surpasses the state-of-the-art approaches by a significant margin with an accuracy of 97.3%. In addition, our method also achieves the best performance on the public lung cancer WSIs dataset from The Cancer Genome Atlas (TCGA). We highlight that a small number of coarse annotations can contribute to further accuracy improvement. We believe that weakly supervised learning methods have great potential to assist pathologists in histology image diagnosis in the near future.
•A novel self-supervised learning strategy called context restoration.•It improves the subsequent learning performance.•Its implementation is simple and straightforward.•It is useful for different ...types of subsequent tasks, including classification, detection, and segmentation.
Display omitted
Machine learning, particularly deep learning has boosted medical image analysis over the past years. Training a good model based on deep learning requires large amount of labelled data. However, it is often difficult to obtain a sufficient number of labelled images for training. In many scenarios the dataset in question consists of more unlabelled images than labelled ones. Therefore, boosting the performance of machine learning models by using unlabelled as well as labelled data is an important but challenging problem. Self-supervised learning presents one possible solution to this problem. However, existing self-supervised learning strategies applicable to medical images cannot result in significant performance improvement. Therefore, they often lead to only marginal improvements. In this paper, we propose a novel self-supervised learning strategy based on context restoration in order to better exploit unlabelled images. The context restoration strategy has three major features: 1) it learns semantic image features; 2) these image features are useful for different types of subsequent image analysis tasks; and 3) its implementation is simple. We validate the context restoration strategy in three common problems in medical imaging: classification, localization, and segmentation. For classification, we apply and test it to scan plane detection in fetal 2D ultrasound images; to localise abdominal organs in CT images; and to segment brain tumours in multi-modal MR images. In all three cases, self-supervised learning based on context restoration learns useful semantic features and lead to improved machine learning models for the above tasks.
•Outlines the setup of challenge on “Diabetic Retinopathy – Segmentation and Grading” held at ISBI-2018.•Describes the dataset used, evaluation criteria and results of top-performing participating ...solutions.•Presents various deep learning and handcrafted features based participating approaches.•Discusses the lessons learnt from analysis of the methods submitted to this challenge.
Display omitted
Diabetic Retinopathy (DR) is the most common cause of avoidable vision loss, predominantly affecting the working-age population across the globe. Screening for DR, coupled with timely consultation and treatment, is a globally trusted policy to avoid vision loss. However, implementation of DR screening programs is challenging due to the scarcity of medical professionals able to screen a growing global diabetic population at risk for DR. Computer-aided disease diagnosis in retinal image analysis could provide a sustainable approach for such large-scale screening effort. The recent scientific advances in computing capacity and machine learning approaches provide an avenue for biomedical scientists to reach this goal. Aiming to advance the state-of-the-art in automatic DR diagnosis, a grand challenge on “Diabetic Retinopathy – Segmentation and Grading” was organized in conjunction with the IEEE International Symposium on Biomedical Imaging (ISBI - 2018). In this paper, we report the set-up and results of this challenge that is primarily based on Indian Diabetic Retinopathy Image Dataset (IDRiD). There were three principal sub-challenges: lesion segmentation, disease severity grading, and localization of retinal landmarks and segmentation. These multiple tasks in this challenge allow to test the generalizability of algorithms, and this is what makes it different from existing ones. It received a positive response from the scientific community with 148 submissions from 495 registrations effectively entered in this challenge. This paper outlines the challenge, its organization, the dataset used, evaluation methods and results of top-performing participating solutions. The top-performing approaches utilized a blend of clinical information, data augmentation, and an ensemble of models. These findings have the potential to enable new developments in retinal image analysis and image-based DR screening in particular.
Robust and fast detection of anatomical structures is a prerequisite for both diagnostic and interventional medical image analysis. Current solutions for anatomy detection are typically based on ...machine learning techniques that exploit large annotated image databases in order to learn the appearance of the captured anatomy. These solutions are subject to several limitations, including the use of suboptimal feature engineering techniques and most importantly the use of computationally suboptimal search-schemes for anatomy detection. To address these issues, we propose a method that follows a new paradigm by reformulating the detection problem as a behavior learning task for an artificial agent. We couple the modeling of the anatomy appearance and the object search in a unified behavioral framework, using the capabilities of deep reinforcement learning and multi-scale image analysis. In other words, an artificial agent is trained not only to distinguish the target anatomical object from the rest of the body but also how to find the object by learning and following an optimal navigation path to the target object in the imaged volumetric space. We evaluated our approach on 1487 3D-CT volumes from 532 patients, totaling over 500,000 image slices and show that it significantly outperforms state-of-the-art solutions on detecting several anatomical structures with no failed cases from a clinical acceptance perspective, while also achieving a 20-30 percent higher detection accuracy. Most importantly, we improve the detection-speed of the reference methods by 2-3 orders of magnitude, achieving unmatched real-time performance on large 3D-CT scans.
Correlation functions are becoming one of the major tools for quantification of structural information that is usually represented as 2D or 3D images. In this paper we introduce ▪ open-source package ...developed in Julia and capable of computing all classical correlation functions based on imaging input data. Images include both binary and multi-phase representations. Our code is capable of evaluating two-point probability S2, phase cross-correlation ρij, cluster C2, lineal-path L2, surface-surface Fss, surface-void Fsv, pore-size P and chord-length p distribution functions on both CPU and GPU architectures. Where possible, we presented two types of computations: full correlation map (correlations of each point with other points on the image, that also allows obtaining ensemble averaged CF) and directional correlation functions (currently in major orthogonal and diagonal directions). Such an implementation allowed for the first time to assemble a completely free solution to evaluate correlation functions under any operating system with well documented application programming interface (API). Our package includes automatic tests against analytical solutions that are described in the paper. We measured execution times for all CPU and GPU implementations and as a rule of thumb full correlation maps on GPU are faster than other methods. However, full maps require more RAM and, thus, are limited to available RAM resources. On the other hand, directional CFs are memory efficient and can be evaluated for huge datasets – this way they are the first candidates for structural data compression of feature extraction. The package itself is available through Julia package ecosystem and on GitHub, the latter source also contains documentation and additional helpful resources such as tutorials. We believe that a single powerful computational tool such as ▪ presented in this paper will significantly facilitate the usage of correlation functions in numerous areas of structural description and research of porous materials, as well as in machine learning applications. We also present some examples as applied to ceramic, soil composite and oil-bearing rock samples based on their 3D X-ray tomography and 2D scanning electron microscope images. Finally, we conclude our paper with discussion of possible ways to further improve presented computational framework.
Program Title: CorrelationFunctions.jl
CPC Library link to program files:https://doi.org/10.17632/6gb9gfm3dw.1
Developer's repository link:https://github.com/fatimp/CorrelationFunctions.jl
Licensing provisions: MIT
Programming language: Julia
Supplementary material: Numerous Jupiter notebooks with examples are available on the GitHub page
Nature of problem: Correlation functions are invaluable universal statistical descriptors of structures used in numerous scientific fields such as astronomy, material science, rock and soil physics, hydrology and biology, to name just a handful of examples. While computational approaches are available in the literature for some functions, they are fragmented and are usually implemented in proprietary interpreted languages for CPU architecture alone.
Solution method: We contribute an open source and cross-platform solution with well documented API for computation of all classical correlation functions from both 2D and 3D images on CPU and GPU architectures. The package computes correlation functions using two approaches: computation of correlation maps and computation along predefined directions. These two approaches can be thought of as an execution time - memory trade-off, but the choice may also depend on the application. The computations are based on a) fast Fourier transform with preprocessing steps such as cluster labeling or edge detection, and b) linear scan approach to evaluate correlation functions along predefined directions. Where justified, the algorithms can be executed on both CPU and GPU which results in high execution speed on modern hardware.
End-to-End Adversarial Retinal Image Synthesis Costa, Pedro; Galdran, Adrian; Meyer, Maria Ines ...
IEEE transactions on medical imaging,
03/2018, Letnik:
37, Številka:
3
Journal Article
Odprti dostop
In medical image analysis applications, the availability of the large amounts of annotated data is becoming increasingly critical. However, annotated medical data is often scarce and costly to ...obtain. In this paper, we address the problem of synthesizing retinal color images by applying recent techniques based on adversarial learning. In this setting, a generative model is trained to maximize a loss function provided by a second model attempting to classify its output into real or synthetic. In particular, we propose to implement an adversarial autoencoder for the task of retinal vessel network synthesis. We use the generated vessel trees as an intermediate stage for the generation of color retinal images, which is accomplished with a generative adversarial network. Both models require the optimization of almost everywhere differentiable loss functions, which allows us to train them jointly. The resulting model offers an end-to-end retinal image synthesis system capable of generating as many retinal images as the user requires, with their corresponding vessel networks, by sampling from a simple probability distribution that we impose to the associated latent space. We show that the learned latent space contains a well-defined semantic structure, implying that we can perform calculations in the space of retinal images, e.g., smoothly interpolating new data points between two retinal images. Visual and quantitative results demonstrate that the synthesized images are substantially different from those in the training set, while being also anatomically consistent and displaying a reasonable visual quality.
PDF
