To realize the full potential of deep learning for medical imaging, large annotated datasets are required for training. Such datasets are difficult to acquire due to privacy issues, lack of experts ...available for annotation, underrepresentation of rare conditions, and poor standardization. The lack of annotated data has been addressed in conventional vision applications using synthetic images refined via unsupervised adversarial training to look like real images. However, this approach is difficult to extend to general medical imaging because of the complex and diverse set of features found in real human tissues. We propose a novel framework that uses a reverse flow, where adversarial training is used to make real medical images more like synthetic images, and clinically-relevant features are preserved via self-regularization. These domain-adapted synthetic-like images can then be accurately interpreted by networks trained on large datasets of synthetic medical images. We implement this approach on the notoriously difficult task of depth-estimation from monocular endoscopy which has a variety of applications in colonoscopy, robotic surgery, and invasive endoscopic procedures. We train a depth estimator on a large data set of synthetic images generated using an accurate forward model of an endoscope and an anatomically-realistic colon. Our analysis demonstrates that the structural similarity of endoscopy depth estimation in a real pig colon predicted from a network trained solely on synthetic data improved by 78.7% by using reverse domain adaptation.
•Synthetically generated endoscopy data with ground truth depth.•An efficient joint CNN-CRF-based depth estimation architecture trained from synthetic endoscopy data.•Adversarial training for ...adapting the network trained on synthetic data to real data.•State-of-the-art endoscopy depth estimation performance.•Validation using registered views of CT and endoscopy data from a porcine colon.
Display omitted
Colorectal cancer is the fourth leading cause of cancer deaths worldwide and the second leading cause in the United States. The risk of colorectal cancer can be mitigated by the identification and removal of premalignant lesions through optical colonoscopy. Unfortunately, conventional colonoscopy misses more than 20% of the polyps that should be removed, due in part to poor contrast of lesion topography. Imaging depth and tissue topography during a colonoscopy is difficult because of the size constraints of the endoscope and the deforming mucosa. Most existing methods make unrealistic assumptions which limits accuracy and sensitivity. In this paper, we present a method that avoids these restrictions, using a joint deep convolutional neural network-conditional random field (CNN-CRF) framework for monocular endoscopy depth estimation. Estimated depth is used to reconstruct the topography of the surface of the colon from a single image. We train the unary and pairwise potential functions of a CRF in a CNN on synthetic data, generated by developing an endoscope camera model and rendering over 200,000 images of an anatomically-realistic colon.We validate our approach with real endoscopy images from a porcine colon, transferred to a synthetic-like domain via adversarial training, with ground truth from registered computed tomography measurements. The CNN-CRF approach estimates depths with a relative error of 0.152 for synthetic endoscopy images and 0.242 for real endoscopy images. We show that the estimated depth maps can be used for reconstructing the topography of the mucosa from conventional colonoscopy images. This approach can easily be integrated into existing endoscopy systems and provides a foundation for improving computer-aided detection algorithms for detection, segmentation and classification of lesions.
Given the rapidly increasing use of metal oxide nanoparticles in agriculture as well as their inadvertent addition through sewage sludge application to soils, it is imperative to assess their ...possible toxic effects on soil functions that are vital for healthy crop production. In this regard, we designed a lab study to investigate the potential toxicity of one of the most produced nanoparticles, i.e. zinc oxide nanoparticles (nZnO), in a calcareous soil. Microcosms of 80 g of dry-equivalent fresh soils were incubated in mason jars for 64 days, after adding 100 or 1000 mg of biogenically produced nZnO kg
soil. Moreover, we also added rice-straw derived biochar at 1 or 5% (w: w basis) hypothesizing that the biochar would alleviate nZnO-induced toxicity given that it has been shown to adsorb and detoxify heavy metals in soils. We found that the nZnO decreased microbial biomass carbon by 27.0 to 33.5% in 100 mg nZnO kg
soil and by 39.0 to 43.3% in 1000 mg nZnO kg
soil treatments across biochar treatments in the short term i.e. 24 days after incubation. However, this decrease disappeared after 64 days of incubation and the microbial biomass in nZnO amended soils were similar to that in control soils. This shows that the toxicity of nZnO in the studied soil was ephemeral and transient which was overcome by the soil itself in a couple of months. This is also supported by the fact that the nZnO induced higher cumulative C mineralization (i.e. soil respiration) at both rates of addition. The treatment 100 mg nZnO kg
soil induced 166 to 207%, while 1000 mg nZnO kg
soil induced 136 to 171% higher cumulative C mineralization across biochar treatments by the end of the experiment. However, contrary to our hypothesis increasing the nZnO addition from 100 to 1000 mg nZnO kg
soil did not cause additional decrease in microbial biomass nor induced higher C mineralization. Moreover, the biochar did not alleviate even the ephemeral toxicity that was observed after 24d of incubation. Based on overall results, we conclude that the studied soil can function without impairment even at 1000 mg kg
concentration of nZnO in it.
Deep-learning methods for computational pathology require either manual annotation of gigapixel whole-slide images (WSIs) or large datasets of WSIs with slide-level labels and typically suffer from ...poor domain adaptation and interpretability. Here we report an interpretable weakly supervised deep-learning method for data-efficient WSI processing and learning that only requires slide-level labels. The method, which we named clustering-constrained-attention multiple-instance learning (CLAM), uses attention-based learning to identify subregions of high diagnostic value to accurately classify whole slides and instance-level clustering over the identified representative regions to constrain and refine the feature space. By applying CLAM to the subtyping of renal cell carcinoma and non-small-cell lung cancer as well as the detection of lymph node metastasis, we show that it can be used to localize well-known morphological features on WSIs without the need for spatial labels, that it overperforms standard weakly supervised classification algorithms and that it is adaptable to independent test cohorts, smartphone microscopy and varying tissue content.
Nuclei mymargin segmentation is a fundamental task for various computational pathology applications including nuclei morphology analysis, cell type classification, and cancer grading. Deep learning ...has emerged as a powerful approach to segmenting nuclei but the accuracy of convolutional neural networks (CNNs) depends on the volume and the quality of labeled histopathology data for training. In particular, conventional CNN-based approaches lack structured prediction capabilities, which are required to distinguish overlapping and clumped nuclei. Here, we present an approach to nuclei segmentation that overcomes these challenges by utilizing a conditional generative adversarial network (cGAN) trained with synthetic and real data. We generate a large dataset of H&E training images with perfect nuclei segmentation labels using an unpaired GAN framework. This synthetic data along with real histopathology data from six different organs are used to train a conditional GAN with spectral normalization and gradient penalty for nuclei segmentation. This adversarial regression framework enforces higher-order spacial-consistency when compared to conventional CNN models. We demonstrate that this nuclei segmentation approach generalizes across different organs, sites, patients and disease states, and outperforms conventional approaches, especially in isolating individual and overlapping nuclei.
•We present the first large-scale application of privacy-preserving federated learning to weakly supervised computational pathology on gigapixel whole slide images.•Validation on multi-class ...classification, binary classification and survival prediction using multi-institutional datasets on two different disease models using thousands of gigapixel whole slide images.•Multiple instance learning-inspired framework for interpretable, weakly-supervised survival prediction from histology whole slides using patient-level labels from multiˇcentric data.
Deep Learning-based computational pathology algorithms have demonstrated profound ability to excel in a wide array of tasks that range from characterization of well known morphological phenotypes to predicting non human-identifiable features from histology such as molecular alterations. However, the development of robust, adaptable and accurate deep learning-based models often rely on the collection and time-costly curation large high-quality annotated training data that should ideally come from diverse sources and patient populations to cater for the heterogeneity that exists in such datasets. Multi-centric and collaborative integration of medical data across multiple institutions can naturally help overcome this challenge and boost the model performance but is limited by privacy concerns among other difficulties that may arise in the complex data sharing process as models scale towards using hundreds of thousands of gigapixel whole slide images. In this paper, we introduce privacy-preserving federated learning for gigapixel whole slide images in computational pathology using weakly-supervised attention multiple instance learning and differential privacy. We evaluated our approach on two different diagnostic problems using thousands of histology whole slide images with only slide-level labels. Additionally, we present a weakly-supervised learning framework for survival prediction and patient stratification from whole slide images and demonstrate its effectiveness in a federated setting. Our results show that using federated learning, we can effectively develop accurate weakly-supervised deep learning models from distributed data silos without direct data sharing and its associated complexities, while also preserving differential privacy using randomized noise generation. We also make available an easy-to-use federated learning for computational pathology software package: http://github.com/mahmoodlab/HistoFL.
Sparsity exploiting image reconstruction (SER) methods have been extensively used with total variation (TV) regularization for tomographic reconstructions. Local TV methods fail to preserve texture ...details and often create additional artifacts due to over-smoothing. Nonlocal TV (NLTV) methods have been proposed as a solution to this but they either lack continuous updates due to computational constraints or limit the locality to a small region. In this letter, we propose adaptive graph-based TV. The proposed method goes beyond spatial similarity between different regions of an image being reconstructed by establishing a connection between similar regions in the entire image regardless of spatial distance. As compared to NLTV, the proposed method is computationally efficient and involves updating the graph prior during every iteration making the connection between similar regions stronger. Moreover, it promotes sparsity in the wavelet and graph gradient domains. Since TV is a special case of graph TV, the proposed method can also be seen as a generalization of SER and TV methods.
Porous graphene was photothermally induced from kraft lignin via direct laser writing. This laser-induced graphene (LIG) possessed a hierarchical structure with a three-dimensional (3D) ...interconnected network ideal for its transfer from the kraft lignin/poly(ethylene oxide) (KL/PEO) film onto polydimethylsiloxane (PDMS). The resultant LIG/PDMS composite was shown to keep the intrinsic porous structure and electrically active sites of LIG. The supercapacitors (SCs) fabricated using the LIG/PDMS composite exhibited good electrochemical performance and excellent cyclic stability. More than 90% capacitance was retained after 10 000 cycles. Moreover, due to their high flexibility, the SCs were able to endure bending deformation without significantly sacrificing their capacitance. The proposed technology for the fabrication of flexible SCs based on lignin-derived LIG demonstrated great potential to use a low-cost, renewable material for the manufacture of portable and wearable electronics.
•The lightly reinforced-SFRC slabs attained the designed moment redistribution.•The lightly reinforced-SFRC slabs showed a softening behaviour after the peak load.•The ductility of the R-SFRC slabs ...decreased with increasing moment redistribution.•The post-peak behaviour of R-SFRC flexural members was formulated mathematically.•The AS 3600:2018 predicted the capacity of R-SFRC slabs with reasonable accuracy.
The ductility and post-peak behaviour of conventionally reinforced steel fibre reinforced concrete (R-SFRC) flexural members have been found to be dependent on the volume of tensile reinforcement. Although few studies have investigated the effect of low reinforcement volume on the ductility and post-peak behaviour of simply supported and continuous members, no study was found for continuous members that were designed for moment redistribution. Further, no prior studies have investigated the ability of lightly reinforced-SFRC continuous members to redistribute moment, or if the achievement of the maximum amount of moment redistribution in design standards is possible. Because of the lack of research in this area, strict limitations are placed in standards for moment redistribution in lightly reinforced-SFRC continuous members. Thus, an experimental study was undertaken for six full-scale two-span continuous one-way slabs with the objectives of determining the moment redistribution capability of lightly reinforced-SFRC continuous members and effect of low reinforcement volume on the ductility and post-peak behaviour of R-SFRC continuous members. The nominal dosage of steel fibres in the R-SFRC slabs was 60 kg/m3 and the tensile reinforcement ratios were between 0.0021 and 0.0042 to provide for bending moment redistribution up to 30% of the linear elastic bending moments. The test results showed that the lightly reinforced-SFRC slabs achieved the designed and maximum amount of moment redistributions and had sufficient ductility, but the R-SFRC slabs showed a softening behaviour after the ultimate load whereas the reinforced concrete (RC) slabs showed a hardening behaviour. Consequently, the post-peak behaviour of R-SFRC flexural members was formulated mathematically. Finally, the rectangular stress block model of AS 3600:2018 was found to determine the capacity of the R-SFRC slabs with reasonable accuracy.
PDF
