Although artificial neural networks have shown great promise in applications including computer vision and speech recognition, there remains considerable practical and theoretical difficulty in ...optimizing their parameters. The seemingly unreasonable success of gradient descent methods in minimizing these non-convex functions remains poorly understood. In this work we offer some theoretical guarantees for networks with piecewise affine activation functions, which have in recent years become the norm. We prove three main results. First, that the network is piecewise convex as a function of the input data. Second, that the network, considered as a function of the parameters in a single layer, all others held constant, is again piecewise convex. Third, that the network as a function of all its parameters is piecewise multi-convex, a generalization of biconvexity. From here we characterize the local minima and stationary points of the training objective, showing that they minimize the objective on certain subsets of the parameter space. We then analyze the performance of two optimization algorithms on multi-convex problems: gradient descent, and a method which repeatedly solves a number of convex sub-problems. We prove necessary convergence conditions for the first algorithm and both necessary and sufficient conditions for the second, after introducing regularization to the objective. Finally, we remark on the remaining difficulty of the global optimization problem. Under the squared error objective, we show that by varying the training data, a single rectifier neuron admits local minima arbitrarily far apart, both in objective value and parameter space.
To develop a predictive model based on quantitative characteristics of geographic atrophy (GA) to estimate future potential regions of GA growth.
Progression study and predictive model.
One hundred ...eighteen spectral-domain (SD) optical coherence tomography (OCT) scans of 38 eyes in 29 patients.
Imaging features of GA quantifying its extent and location, as well as characteristics at each topographic location related to individual retinal layer thickness and reflectivity, the presence of pathologic features (like reticular pseudodrusen or loss of photoreceptors), and other known risk factors of GA growth, were extracted automatically from 118 SD OCT scans of 38 eyes from 29 patients collected over a median follow-up of 2.25 years. We developed and evaluated a model to predict the magnitude and location of GA growth at given future times using the quantitative features as predictors in 3 possible scenarios.
Potential regions of GA growth.
In descending order of out-of-bag feature importance, the most predictive SD OCT biomarkers for predicting the future regions of GA growth were thickness loss of bands 11 through 14 (5.66), reflectivity of bands 11 and 12 (5.37), thickness of reticular pseudodrusen (5.01), thickness of bands 5 through 11 (4.82), reflectivity of bands 7 through 11 (4.78), GA projection image (4.73), increased minimum retinal intensity map (4.59), and GA eccentricity (4.49). The predicted GA regions in the 3 tested scenarios resulted in a Dice index mean ± standard deviation of 0.81±0.12, 0.84±0.10, and 0.87±0.06, respectively, when compared with the observed ground truth. Considering only the regions without evidence of GA at baseline, predicted regions of future GA growth showed relatively high Dice indices of 0.72±0.18, 0.74±0.17, and 0.72±0.22, respectively. Predictions and actual values of GA growth rate and future GA involvement in the central fovea showed high correlations.
Experimental results demonstrated the potential of our predictive model to predict future regions where GA is likely to grow and to identify the most discriminant early indicator (thickness loss of bands 11 through 14) of regions susceptible to GA growth.
To develop an intratumor partitioning framework for identifying high-risk subregions from (18)F-fluorodeoxyglucose positron emission tomography (FDG-PET) and computed tomography (CT) imaging and to ...test whether tumor burden associated with the high-risk subregions is prognostic of outcomes in lung cancer.
In this institutional review board-approved retrospective study, we analyzed the pretreatment FDG-PET and CT scans of 44 lung cancer patients treated with radiation therapy. A novel, intratumor partitioning method was developed, based on a 2-stage clustering process: first at the patient level, each tumor was over-segmented into many superpixels by k-means clustering of integrated PET and CT images; next, tumor subregions were identified by merging previously defined superpixels via population-level hierarchical clustering. The volume associated with each of the subregions was evaluated using Kaplan-Meier analysis regarding its prognostic capability in predicting overall survival (OS) and out-of-field progression (OFP).
Three spatially distinct subregions were identified within each tumor that were highly robust to uncertainty in PET/CT co-registration. Among these, the volume of the most metabolically active and metabolically heterogeneous solid component of the tumor was predictive of OS and OFP on the entire cohort, with a concordance index or CI of 0.66-0.67. When restricting the analysis to patients with stage III disease (n=32), the same subregion achieved an even higher CI of 0.75 (hazard ratio 3.93, log-rank P=.002) for predicting OS, and a CI of 0.76 (hazard ratio 4.84, log-rank P=.002) for predicting OFP. In comparison, conventional imaging markers, including tumor volume, maximum standardized uptake value, and metabolic tumor volume using threshold of 50% standardized uptake value maximum, were not predictive of OS or OFP, with CI mostly below 0.60 (log-rank P>.05).
We propose a robust intratumor partitioning method to identify clinically relevant, high-risk subregions in lung cancer. We envision that this approach will be applicable to identifying useful imaging biomarkers in many cancer types.
•A deep-learning based CADx for diagnosis of embryonal and alveolar subtypes.•Diagnosis has been performed by solely by analyzing multiparametric MR images.•Created a fusion of diffusion-weighted and ...T1-weighted MR scans.•A pre-trained deep neural network has been fine-tuned based on the fused images.•Achieved 85% cross validation accuracy for classifying the two RMS subtypes.•The system can provide an efficient and reproducible diagnosis with less human interaction.
This paper presents a deep-learning-based CADx for the differential diagnosis of embryonal (ERMS) and alveolar (ARMS) subtypes of rhabdomysarcoma (RMS) solely by analyzing multiparametric MR images. We formulated an automated pipeline that creates a comprehensive representation of tumor by performing a fusion of diffusion-weighted MR scans (DWI) and gadolinium chelate-enhanced T1−weighted MR scans (MRI). Finally, we adapted transfer learning approach where a pre-trained deep convolutional neural network has been fine-tuned based on the fused images for performing classification of the two RMS subtypes. We achieved 85% cross validation prediction accuracy from the fine-tuned deep CNN model. Our system can be exploited to provide a fast, efficient and reproducible diagnosis of RMS subtypes with less human interaction. The framework offers an efficient integration between advanced image processing methods and cutting-edge deep learning techniques which can be extended to deal with other clinical domains that involve multimodal imaging for disease diagnosis.
Molecular analysis of the mutation status for EGFR and KRAS are now routine in the management of non-small cell lung cancer. Radiogenomics, the linking of medical images with the genomic properties ...of human tumors, provides exciting opportunities for non-invasive diagnostics and prognostics. We investigated whether EGFR and KRAS mutation status can be predicted using imaging data. To accomplish this, we studied 186 cases of NSCLC with preoperative thin-slice CT scans. A thoracic radiologist annotated 89 semantic image features of each patient's tumor. Next, we built a decision tree to predict the presence of EGFR and KRAS mutations. We found a statistically significant model for predicting EGFR but not for KRAS mutations. The test set area under the ROC curve for predicting EGFR mutation status was 0.89. The final decision tree used four variables: emphysema, airway abnormality, the percentage of ground glass component and the type of tumor margin. The presence of either of the first two features predicts a wild type status for EGFR while the presence of any ground glass component indicates EGFR mutations. These results show the potential of quantitative imaging to predict molecular properties in a non-invasive manner, as CT imaging is more readily available than biopsies.
Biomedical ontologies provide essential domain knowledge to drive data integration, information retrieval, data annotation, natural-language processing and decision support. BioPortal ...(http://bioportal.bioontology.org) is an open repository of biomedical ontologies that provides access via Web services and Web browsers to ontologies developed in OWL, RDF, OBO format and Protégé frames. BioPortal functionality includes the ability to browse, search and visualize ontologies. The Web interface also facilitates community-based participation in the evaluation and evolution of ontology content by providing features to add notes to ontology terms, mappings between terms and ontology reviews based on criteria such as usability, domain coverage, quality of content, and documentation and support. BioPortal also enables integrated search of biomedical data resources such as the Gene Expression Omnibus (GEO), ClinicalTrials.gov, and ArrayExpress, through the annotation and indexing of these resources with ontologies in BioPortal. Thus, BioPortal not only provides investigators, clinicians, and developers 'one-stop shopping' to programmatically access biomedical ontologies, but also provides support to integrate data from a variety of biomedical resources.
Purpose To create a radiogenomic map linking computed tomographic (CT) image features and gene expression profiles generated by RNA sequencing for patients with non-small cell lung cancer (NSCLC). ...Materials and Methods A cohort of 113 patients with NSCLC diagnosed between April 2008 and September 2014 who had preoperative CT data and tumor tissue available was studied. For each tumor, a thoracic radiologist recorded 87 semantic image features, selected to reflect radiologic characteristics of nodule shape, margin, texture, tumor environment, and overall lung characteristics. Next, total RNA was extracted from the tissue and analyzed with RNA sequencing technology. Ten highly coexpressed gene clusters, termed metagenes, were identified, validated in publicly available gene-expression cohorts, and correlated with prognosis. Next, a radiogenomics map was built that linked semantic image features to metagenes by using the t statistic and the Spearman correlation metric with multiple testing correction. Results RNA sequencing analysis resulted in 10 metagenes that capture a variety of molecular pathways, including the epidermal growth factor (EGF) pathway. A radiogenomic map was created with 32 statistically significant correlations between semantic image features and metagenes. For example, nodule attenuation and margins are associated with the late cell-cycle genes, and a metagene that represents the EGF pathway was significantly correlated with the presence of ground-glass opacity and irregular nodules or nodules with poorly defined margins. Conclusion Radiogenomic analysis of NSCLC showed multiple associations between semantic image features and metagenes that represented canonical molecular pathways, and it can result in noninvasive identification of molecular properties of NSCLC. Online supplemental material is available for this article.
Medical image biomarkers of cancer promise improvements in patient care through advances in precision medicine. Compared to genomic biomarkers, image biomarkers provide the advantages of being ...non-invasive, and characterizing a heterogeneous tumor in its entirety, as opposed to limited tissue available via biopsy. We developed a unique radiogenomic dataset from a Non-Small Cell Lung Cancer (NSCLC) cohort of 211 subjects. The dataset comprises Computed Tomography (CT), Positron Emission Tomography (PET)/CT images, semantic annotations of the tumors as observed on the medical images using a controlled vocabulary, and segmentation maps of tumors in the CT scans. Imaging data are also paired with results of gene mutation analyses, gene expression microarrays and RNA sequencing data from samples of surgically excised tumor tissue, and clinical data, including survival outcomes. This dataset was created to facilitate the discovery of the underlying relationship between tumor molecular and medical image features, as well as the development and evaluation of prognostic medical image biomarkers.
IMPORTANCE: Sensitive outcome measures for disease progression are needed for treatment trials in geographic atrophy (GA) secondary to age-related macular degeneration (AMD). OBJECTIVE: To quantify ...photoreceptor degeneration outside regions of GA in eyes with nonexudative AMD, to evaluate its association with future GA progression, and to characterize its spatio-temporal progression. DESIGN, SETTING, AND PARTICIPANTS: Monocenter cohort study (Directional Spread in Geographic Atrophy NCT02051998) and analysis of data from a normative data study at a tertiary referral center. One hundred fifty-eight eyes of 89 patients with a mean (SD) age of 77.7 (7.1) years, median area of GA of 8.87 mm2 (IQR, 4.09-15.60), and median follow-up of 1.1 years (IQR, 0.52-1.7 years), as well as 93 normal eyes from 93 participants. EXPOSURES: Longitudinal spectral-domain optical coherence tomography (SD-OCT) volume scans (121 B-scans across 30° × 25°) were segmented with a deep-learning pipeline and standardized in a pointwise manner with age-adjusted normal data (z scores). Outer nuclear layer (ONL), photoreceptor inner segment (IS), and outer segment (OS) thickness were quantified along evenly spaced contour lines surrounding GA lesions. Linear mixed models were applied to assess the association between photoreceptor-related imaging features and GA progression rates and characterize the pattern of photoreceptor degeneration over time. MAIN OUTCOMES AND MEASURES: Association of ONL thinning with follow-up time (after adjusting for age, retinal topography z score, and distance to the GA boundary). RESULTS: The study included 158 eyes of 89 patients (51 women and 38 men) with a mean (SD) age of 77.7 (7.1) years. The fully automated B-scan segmentation was accurate (dice coefficient, 0.82; 95% CI, 0.80-0.85; compared with manual markings) and revealed a marked interpatient variability in photoreceptor degeneration. The ellipsoid zone (EZ) loss-to-GA boundary distance and OS thickness were prognostic for future progression rates. Outer nuclear layer and IS thinning over time was significant even when adjusting for age and proximity to the GA boundary (estimates of −0.16 μm/y; 95% CI, −0.30 to −0.02; and −0.17 μm/y; 95% CI, −0.26 to −0.09). CONCLUSIONS AND RELEVANCE: Distinct and progressive alterations of photoreceptor laminae (exceeding GA spatially) were detectable and quantifiable. The degree of photoreceptor degeneration outside of regions of retinal pigment epithelium atrophy varied markedly between eyes and was associated with future GA progression. Macula-wide photoreceptor laminae thinning represents a potential candidate end point to monitor treatment effects beyond mere GA lesion size progression.
PDF
