The diagnosis of most cancers is made by a board-certified pathologist based on a tissue biopsy under the microscope. Recent research reveals a high discordance between individual pathologists. For ...melanoma, the literature reports on 25–26% of discordance for classifying a benign nevus versus malignant melanoma. A recent study indicated the potential of deep learning to lower these discordances. However, the performance of deep learning in classifying histopathologic melanoma images was never compared directly to human experts. The aim of this study is to perform such a first direct comparison.
A total of 695 lesions were classified by an expert histopathologist in accordance with current guidelines (350 nevi/345 melanoma). Only the haematoxylin & eosin (H&E) slides of these lesions were digitalised via a slide scanner and then randomly cropped. A total of 595 of the resulting images were used to train a convolutional neural network (CNN). The additional 100 H&E image sections were used to test the results of the CNN in comparison to 11 histopathologists. Three combined McNemar tests comparing the results of the CNNs test runs in terms of sensitivity, specificity and accuracy were predefined to test for significance (p < 0.05).
The CNN achieved a mean sensitivity/specificity/accuracy of 76%/60%/68% over 11 test runs. In comparison, the 11 pathologists achieved a mean sensitivity/specificity/accuracy of 51.8%/66.5%/59.2%. Thus, the CNN was significantly (p = 0.016) superior in classifying the cropped images.
With limited image information available, a CNN was able to outperform 11 histopathologists in the classification of histopathological melanoma images and thus shows promise to assist human melanoma diagnoses.
•A convolutional neural network (CNN) was trained with 595 histopathologic images of melanoma and nevi.•In a direct comparison, the CNN and 11 histopathologists classified a test set of 100 additional histopathologic images (1:1 melanoma/nevi).•The CNN systematically outperformed the 11 histopathologists in terms of overall accuracy, sensitivity and specificity (p = 0.016).
Several recent publications have demonstrated the use of convolutional neural networks to classify images of melanoma at par with board-certified dermatologists. However, the non-availability of a ...public human benchmark restricts the comparability of the performance of these algorithms and thereby the technical progress in this field.
An electronic questionnaire was sent to dermatologists at 12 German university hospitals. Each questionnaire comprised 100 dermoscopic and 100 clinical images (80 nevi images and 20 biopsy-verified melanoma images, each), all open-source. The questionnaire recorded factors such as the years of experience in dermatology, performed skin checks, age, sex and the rank within the university hospital or the status as resident physician. For each image, the dermatologists were asked to provide a management decision (treat/biopsy lesion or reassure the patient). Main outcome measures were sensitivity, specificity and the receiver operating characteristics (ROC).
Total 157 dermatologists assessed all 100 dermoscopic images with an overall sensitivity of 74.1%, specificity of 60.0% and an ROC of 0.67 (range = 0.538–0.769); 145 dermatologists assessed all 100 clinical images with an overall sensitivity of 89.4%, specificity of 64.4% and an ROC of 0.769 (range = 0.613–0.9). Results between test-sets were significantly different (P < 0.05) confirming the need for a standardised benchmark.
We present the first public melanoma classification benchmark for both non-dermoscopic and dermoscopic images for comparing artificial intelligence algorithms with diagnostic performance of 145 or 157 dermatologists. Melanoma Classification Benchmark should be considered as a reference standard for white-skinned Western populations in the field of binary algorithmic melanoma classification.
•This paper provides the first open access melanoma classification benchmark for both non-dermoscopic and dermoscopic images.•Algorithms can now be easily compared to the performance of dermatologists in terms of sensitivity, specificity and ROC.•The melanoma benchmark allows comparability between algorithms of different publications and provides a new reference standard.
Abstract Background Anti-programmed cell death receptor-1 (PD-1) antibodies represent an effective treatment option for metastatic melanoma as well as for other cancer entities. They act via blockade ...of the PD-1 receptor, an inhibitor of the T-cell effector mechanisms that limit immune responses against tumours. As reported for ipilimumab, the anti-PD-1 antibodies pembrolizumab and nivolumab can induce immune-related adverse events (irAEs). These side-effects affect skin, gastrointestinal tract, liver, endocrine system and other organ systems. Since life-threatening and fatal irAEs have been reported, adequate diagnosis and management are essential. Methods and findings In total, 496 patients with metastatic melanoma from 15 skin cancer centers were treated with pembrolizumab or nivolumab; 242 side-effects were described in 138 patients. In 116 of the 138 patients, side-effects affected the skin, gastrointestinal tract, liver, endocrine, and renal system. Rare side-effects included diabetes mellitus, lichen planus, and pancreas insufficiency due to pancreatitis. Conclusion Anti-PD1 antibodies can induce a plethora of irAEs. The knowledge of them will allow prompt diagnosis and improve the management resulting in decreased morbidity.
Melanoma is the most dangerous type of skin cancer but is curable if detected early. Recent publications demonstrated that artificial intelligence is capable in classifying images of benign nevi and ...melanoma with dermatologist-level precision. However, a statistically significant improvement compared with dermatologist classification has not been reported to date.
For this comparative study, 4204 biopsy-proven images of melanoma and nevi (1:1) were used for the training of a convolutional neural network (CNN). New techniques of deep learning were integrated. For the experiment, an additional 804 biopsy-proven dermoscopic images of melanoma and nevi (1:1) were randomly presented to dermatologists of nine German university hospitals, who evaluated the quality of each image and stated their recommended treatment (19,296 recommendations in total). Three McNemar's tests comparing the results of the CNN's test runs in terms of sensitivity, specificity and overall correctness were predefined as the main outcomes.
The respective sensitivity and specificity of lesion classification by the dermatologists were 67.2% (95% confidence interval CI: 62.6%–71.7%) and 62.2% (95% CI: 57.6%–66.9%). In comparison, the trained CNN achieved a higher sensitivity of 82.3% (95% CI: 78.3%–85.7%) and a higher specificity of 77.9% (95% CI: 73.8%–81.8%). The three McNemar's tests in 2 × 2 tables all reached a significance level of p < 0.001. This significance level was sustained for both subgroups.
For the first time, automated dermoscopic melanoma image classification was shown to be significantly superior to both junior and board-certified dermatologists (p < 0.001).
•Recent publications demonstrated that deep learning is capable to classify images of benign nevi and melanoma with dermatologist-level precision.•A systematic outperformance of dermatologists was not demonstrated to date.•This study shows the first systematic (p < 0.001) outperformance of board-certified dermatologists in dermoscopic melanoma image classification.
For clear cell renal cell carcinoma (ccRCC) risk-dependent diagnostic and therapeutic algorithms are routinely implemented in clinical practice. Artificial intelligence-based image analysis has the ...potential to improve outcome prediction and thereby risk stratification. Thus, we investigated whether a convolutional neural network (CNN) can extract relevant image features from a representative hematoxylin and eosin-stained slide to predict 5-year overall survival (5y-OS) in ccRCC. The CNN was trained to predict 5y-OS in a binary manner using slides from TCGA and validated using an independent in-house cohort. Multivariable logistic regression was used to combine of the CNNs prediction and clinicopathological parameters. A mean balanced accuracy of 72.0% (standard deviation SD = 7.9%), sensitivity of 72.4% (SD = 10.6%), specificity of 71.7% (SD = 11.9%) and area under receiver operating characteristics curve (AUROC) of 0.75 (SD = 0.07) was achieved on the TCGA training set (n = 254 patients / WSIs) using 10-fold cross-validation. On the external validation cohort (n = 99 patients / WSIs), mean accuracy, sensitivity, specificity and AUROC were 65.5% (95%-confidence interval CI: 62.9–68.1%), 86.2% (95%-CI: 81.8–90.5%), 44.9% (95%-CI: 40.2–49.6%), and 0.70 (95%-CI: 0.69–0.71). A multivariable model including age, tumor stage and metastasis yielded an AUROC of 0.75 on the TCGA cohort. The inclusion of the CNN-based classification (Odds ratio = 4.86, 95%-CI: 2.70–8.75, p < 0.01) raised the AUROC to 0.81. On the validation cohort, both models showed an AUROC of 0.88. In univariable Cox regression, the CNN showed a hazard ratio of 3.69 (95%-CI: 2.60–5.23, p < 0.01) on TCGA and 2.13 (95%-CI: 0.92–4.94, p = 0.08) on external validation. The results demonstrate that the CNN’s image-based prediction of survival is promising and thus this widely applicable technique should be further investigated with the aim of improving existing risk stratification in ccRCC.
Gastrointestinal cancers account for approximately 20% of all cancer diagnoses and are responsible for 22.5% of cancer deaths worldwide. Artificial intelligence–based diagnostic support systems, in ...particular convolutional neural network (CNN)–based image analysis tools, have shown great potential in medical computer vision. In this systematic review, we summarise recent studies reporting CNN-based approaches for digital biomarkers for characterization and prognostication of gastrointestinal cancer pathology.
Pubmed and Medline were screened for peer-reviewed papers dealing with CNN-based gastrointestinal cancer analyses from histological slides, published between 2015 and 2020.Seven hundred and ninety titles and abstracts were screened, and 58 full-text articles were assessed for eligibility.
Sixteen publications fulfilled our inclusion criteria dealing with tumor or precursor lesion characterization or prognostic and predictive biomarkers: 14 studies on colorectal or rectal cancer, three studies on gastric cancer and none on esophageal cancer. These studies were categorised according to their end-points: polyp characterization, tumor characterization and patient outcome. Regarding the translation into clinical practice, we identified several studies demonstrating generalization of the classifier with external tests and comparisons with pathologists, but none presenting clinical implementation.
Results of recent studies on CNN-based image analysis in gastrointestinal cancer pathology are promising, but studies were conducted in observational and retrospective settings. Large-scale trials are needed to assess performance and predict clinical usefulness. Furthermore, large-scale trials are required for approval of CNN-based prediction models as medical devices.
•Computational medical image analysis is rising in gastrointestinal cancer pathology.•Several studies deal with CNN–based identification of biomarkers on HE slides.•Results are promising but studies are at early stages.•Further studies are needed to assess performance and potential clinical usefulness.
BackgroundMyeloid-derived suppressor cells (MDSC) play a major role in the immunosuppressive melanoma microenvironment. They are generated under chronic inflammatory conditions characterized by the ...constant production of inflammatory cytokines, chemokines and growth factors, including IL-6. Recruitment of MDSC to the tumor is mediated by the interaction between chemokines and chemokine receptors, in particular C–C chemokine receptor (CCR)5. Here, we studied the mechanisms of CCR5 upregulation and increased immunosuppressive function of CCR5+ MDSC.MethodsThe immortalized myeloid suppressor cell line MSC-2, primary immature myeloid cells and in vitro differentiated MDSC were used to determine factors and molecular mechanisms regulating CCR5 expression and immunosuppressive markers at the mRNA and protein levels. The relevance of the identified pathways was validated on the RET transgenic mouse melanoma model, which was also used to target the identified pathways in vivo.ResultsIL-6 upregulated the expression of CCR5 and arginase 1 in MDSC by a STAT3-dependent mechanism. MDSC differentiated in the presence of IL-6 strongly inhibited CD8+ T cell functions compared with MDSC differentiated without IL-6. A correlation between IL-6 levels, phosphorylated STAT3 and CCR5 expression in tumor-infiltrating MDSC was demonstrated in the RET transgenic melanoma mouse model. Surprisingly, IL-6 overexpressing tumors grew significantly slower in mice accompanied by CD8+ T cell activation. Moreover, transgenic melanoma-bearing mice treated with IL-6 blocking antibodies showed significantly accelerated tumor development.ConclusionOur in vitro and ex vivo findings demonstrated that IL-6 induced CCR5 expression and a strong immunosuppressive activity of MDSC, highlighting this cytokine as a promising target for melanoma immunotherapy. However, IL-6 blocking therapy did not prove to be effective in RET transgenic melanoma-bearing mice but rather aggravated tumor progression. Further studies are needed to identify particular combination therapies, cancer entities or patient subsets to benefit from the anti-IL-6 treatment.
Abstract Background Anti-programmed cell death 1 (PD-1) antibodies represent an effective treatment option for metastatic melanoma and other cancer entities. They act via blockade of the PD-1 ...receptor, an inhibitor of the T-cell effector mechanisms that limit immune responses against tumours. As reported for ipilimumab, the anti-PD-1 antibodies pembrolizumab and nivolumab can induce immune-related adverse events (irAEs). These side-effects can involve skin, gastrointestinal tract, liver, the endocrine system and other organ systems. Since life-threatening and fatal irAEs have been reported, adequate diagnosis and management are essential. Methods and findings In total, 496 patients with metastatic melanoma from 15 skin cancer centres were treated with pembrolizumab or nivolumab. Two hundred forty two side-effects in 138 patients have been analysed. In 77 of the 138 patients side-effects affected the nervous system, respiratory tract, musculoskeletal system, heart, blood and eyes. Not yet reported side-effects such as meningo-(radiculitis), polyradiculitis, cardiac arrhythmia, asystolia, and paresis have been observed. Rare and difficult to manage side-effects such as myasthenia gravis are described in detail. Conclusion Anti-PD-1 antibodies can induce a plethora of irAEs. The knowledge of them will allow prompt diagnosis and improve the management resulting in decreased morbidity.
Abstract Background The anti-programmed cell death-1 (PD-1) inhibitors pembrolizumab and nivolumab alone or in combination with ipilimumab have shown improved objective response rates and ...progression-free survival compared to ipilimumab only in advanced melanoma patients. Anti-PD-1 therapy demonstrated nearly equal clinical efficacy in patients who had progressed after ipilimumab or were treatment-naïve. However, only limited evidence exists regarding the efficacy of ipilimumab alone or in combination with nivolumab after treatment failure to anti-PD-therapy. Patients and methods A multicenter retrospective study in advanced melanoma patients who were treated with nivolumab (1 or 3 mg/kg) and ipilimumab (1 mg or 3 mg/kg) or ipilimumab (3 mg/kg) alone after treatment failure to anti-PD-1 therapy was performed. Patient, tumour, pre- and post-treatment characteristics were analysed. Results In total, 47 patients were treated with ipilimumab (ipi-group) and 37 patients with ipilimumab and nivolumab (combination-group) after treatment failure to anti-PD-1 therapy. Overall response rates for the ipi- and the combination-group were 16% and 21%, respectively. Disease control rate was 42% for the ipi-group and 33% for the combination-group. One-year overall survival rates for the ipi- and the combination-group were 54% and 55%, respectively. Conclusions Ipilimumab should be considered as a viable treatment option for patients with failure to prior anti-PD-1 therapy, including those with progressive disease as best response to prior anti-PD-1. In contrast, the combination of ipilimumab and nivolumab appears significantly less effective in this setting compared to treatment-naïve patients.
Objective
To develop a new digital biomarker based on the analysis of primary tumour tissue by a convolutional neural network (CNN) to predict lymph node metastasis (LNM) in a cohort matched for ...already established risk factors.
Patients and Methods
Haematoxylin and eosin (H&E) stained primary tumour slides from 218 patients (102 N+; 116 N0), matched for Gleason score, tumour size, venous invasion, perineural invasion and age, who underwent radical prostatectomy were selected to train a CNN and evaluate its ability to predict LN status.
Results
With 10 models trained with the same data, a mean area under the receiver operating characteristic curve (AUROC) of 0.68 (95% confidence interval CI 0.678–0.682) and a mean balanced accuracy of 61.37% (95% CI 60.05–62.69%) was achieved. The mean sensitivity and specificity was 53.09% (95% CI 49.77–56.41%) and 69.65% (95% CI 68.21–71.1%), respectively. These results were confirmed via cross‐validation. The probability score for LNM prediction was significantly higher on image sections from N+ samples (mean SD N+ probability score 0.58 0.17 vs 0.47 0.15 N0 probability score, P = 0.002). In multivariable analysis, the probability score of the CNN (odds ratio OR 1.04 per percentage probability, 95% CI 1.02–1.08; P = 0.04) and lymphovascular invasion (OR 11.73, 95% CI 3.96–35.7; P < 0.001) proved to be independent predictors for LNM.
Conclusion
In our present study, CNN‐based image analyses showed promising results as a potential novel low‐cost method to extract relevant prognostic information directly from H&E histology to predict the LN status of patients with prostate cancer. Our ubiquitously available technique might contribute to an improved LN status prediction.
PDF
