Background
Personalized therapy planning remains a significant challenge in advanced colorectal cancer care, despite extensive research on prognostic and predictive markers. A strong correlation of ...sarcopenia or overall body composition and survival has been described. Here, we explore whether automated assessment of body composition and liver metastases from standard of care CT images can add to clinical parameters in personalized survival risk prognostication.
Methods
We retrospectively analysed clinical imaging data from 85 patients (50.6% female, mean age 58.9 SD 12.2 years) with colorectal cancer and synchronous liver metastases. Pretrained deep learning models were used to assess body composition and liver metastasis geometry from abdominal CT images before the initiation of systemic treatment. Abdominal muscle‐to‐bone ratio (MBR) was calculated by dividing abdominal muscle volume by abdominal bone volume. MBR was compared with body mass index (BMI), abdominal muscle volume, and abdominal muscle volume divided by height squared. Differences in overall survival based on body composition and liver metastasis parameters were compared using Kaplan–Meier survival curves. Results were correlated with clinical and biomarker data to develop a machine learning model for survival risk prognostication.
Results
The MBR, unlike abdominal muscle volume or BMI, was significantly associated with overall survival (HR 0.39, 95% CI: 0.19–0.80, P = 0.009). The MBR (P = 0.022), liver metastasis surface area (P = 0.01) and primary tumour sidedness (P = 0.007) were independently associated with overall survival in multivariate analysis. Body composition parameters did not correlate with KRAS mutational status or primary tumour sidedness. A prediction model based on MBR, liver metastasis surface area and primary tumour sidedness achieved a concordance index of 0.69.
Conclusions
Automated segmentation enables to extract prognostic parameters from routine imaging data for personalized survival modelling in advanced colorectal cancer patients.
BackgroundPredicting a patient‘s response to immune checkpoint inhibitors (ICIs) could help understand the benefit-risk profile of treatment, potentially improve clinical trial cohort selection, and ...may inform care pathway decisions in clinical practice. Recently, machine learning (ML)-based predictive analytics have gained momentum in this area, but models trained or evaluated on multi-center data are still rare. Therefore, it is difficult to assess the generalizability of single-center models.We present the results of an external validation of a ML framework trained on US data on a large European cohort.MethodsRandom forest classification models were built to predict overall survival (OS) greater than 100 days, one year, and three years, and to predict the occurrence of hepatitis within six weeks, 90 days, and one year after initiation of ICI treatment. For model training, we utilized routinely available real-world data from Vanderbilt University Medical Center (data cut-off December 31, 2018) of a more than 2,200 patient strong pan-cancer cohort containing patients with localized as well as metastatic tumors. Structured, routine clinical data such as age, laboratory values, medication history and condition codes were used as model features. Feature engineering involved aggregating laboratory measurements acquired over a 120-day time window, and a one-year window was applied for other data types. The hepatitis binary label was defined as 1 if any liver enzymes exceeded three times the upper limit of normal (table 1).The trained models were evaluated in an external retrospective pan-cancer cohort of the University Hospital Essen, Germany (n=4257). All input variables were extracted from a FHIR database using FHIRPACK.1 Containerized models were employed for data integration and model evaluation.ResultsOur random forest models achieved an AUC of up to 0.79 for the prediction of OS and up to 0.81 for the prediction of hepatitis in the training data. The models successfully retained at least 90% of their performance for OS and 86% for hepatitis prediction endpoints on the external evaluation cohort (tables 2 and 3).ConclusionsTo our knowledge, this is the first large-scale, external cohort evaluations of OS and hepatitis prediction ML models in ICI patients. Despite different geographic origins, our models generalized well to unseen data. In particular, short-term models showed remarkable performance retention when applied to an external cohort. Our work demonstrates the potential of ML models as valuable tools for pre-screening eligible patients for ICI clinical trials and as clinical decision support for routine patient management.Referencehttps://github.com/fhirpack/fhirpackEthics ApprovalEthics approval for using the Vanderbilt university cohort was granted by The Vanderbilt University Medical Center Health Sciences #3 Institutional Review Board, tracked as #211814. The IRB determined the study poses minimal risk to participants, and a waiver of consent was granted.The study using the external validation cohort was approved by the Ethics Committee of the Medical Faculty of the University of Duisburg-Essen (No. 21–10347-BO). The requirement for written informed consent was waived due to the retrospective design of the study.Abstract 1294 Table 1Detailed information of features and labels used in the evaluated modelsAbstract 1294 Table 2Detailed results of the external evaluation for OS modelsAbstract 1294 Table 3Detailed results of the external evaluation for hepatitis models
BackgroundMelanoma is an immune sensitive disease, as demonstrated by the activity of immune check point blockade (ICB), but many patients will either not respond or relapse. More recently, tumor ...infiltrating lymphocyte (TIL) therapy has shown promising efficacy in melanoma treatment after ICB failure, indicating the potential of cellular therapies. However, TIL treatment comes with manufacturing limitations, product heterogeneity, as well as toxicity problems, due to the transfer of a large number of phenotypically diverse T cells. To overcome said limitations, we propose a controlled adoptive cell therapy approach, where T cells are armed with synthetic agonistic receptors (SAR) that are selectively activated by bispecific antibodies (BiAb) targeting SAR and melanoma-associated antigens.MethodsHuman as well as murine SAR constructs were generated and transduced into primary T cells. The approach was validated in murine, human and patient-derived cancer models expressing the melanoma-associated target antigens tyrosinase-related protein 1 (TYRP1) and melanoma-associated chondroitin sulfate proteoglycan (MCSP) (CSPG4). SAR T cells were functionally characterized by assessing their specific stimulation and proliferation, as well as their tumor-directed cytotoxicity, in vitro and in vivo.ResultsMCSP and TYRP1 expression was conserved in samples of patients with treated as well as untreated melanoma, supporting their use as melanoma-target antigens. The presence of target cells and anti-TYRP1 × anti-SAR or anti-MCSP × anti-SAR BiAb induced conditional antigen-dependent activation, proliferation of SAR T cells and targeted tumor cell lysis in all tested models. In vivo, antitumoral activity and long-term survival was mediated by the co-administration of SAR T cells and BiAb in a syngeneic tumor model and was further validated in several xenograft models, including a patient-derived xenograft model.ConclusionThe SAR T cell-BiAb approach delivers specific and conditional T cell activation as well as targeted tumor cell lysis in melanoma models. Modularity is a key feature for targeting melanoma and is fundamental towards personalized immunotherapies encompassing cancer heterogeneity. Because antigen expression may vary in primary melanoma tissues, we propose that a dual approach targeting two tumor-associated antigens, either simultaneously or sequentially, could avoid issues of antigen heterogeneity and deliver therapeutic benefit to patients.
Nuclei detection and segmentation in hematoxylin and eosin-stained (H&E) tissue images are important clinical tasks and crucial for a wide range of applications. However, it is a challenging task due ...to nuclei variances in staining and size, overlapping boundaries, and nuclei clustering. While convolutional neural networks have been extensively used for this task, we explore the potential of Transformer-based networks in combination with large scale pre-training in this domain. Therefore, we introduce a new method for automated instance segmentation of cell nuclei in digitized tissue samples using a deep learning architecture based on Vision Transformer called CellViT. CellViT is trained and evaluated on the PanNuke dataset, which is one of the most challenging nuclei instance segmentation datasets, consisting of nearly 200,000 annotated nuclei into 5 clinically important classes in 19 tissue types. We demonstrate the superiority of large-scale in-domain and out-of-domain pre-trained Vision Transformers by leveraging the recently published Segment Anything Model and a ViT-encoder pre-trained on 104 million histological image patches — achieving state-of-the-art nuclei detection and instance segmentation performance on the PanNuke dataset with a mean panoptic quality of 0.50 and an F1-detection score of 0.83. The code is publicly available at https://github.com/TIO-IKIM/CellViT.
Display omitted
•Novel U-Net-style network for nuclei segmentation using Vision Transformers (CellViT)•Our method outperforms existing techniques and is state-of-the-art on PanNuke•First to embed pre-trained transformer-based foundation models for nuclei segmentation•We demonstrate the generalizability on the MoNuSeg dataset without finetuning
Objectives
Over the course of their treatment, patients often switch hospitals, requiring staff at the new hospital to import external imaging studies to their local database. In this study, the ...authors present MOdality Mapping and Orchestration (MOMO), a Deep Learning–based approach to automate this mapping process by combining metadata analysis and a neural network ensemble.
Methods
A set of 11,934 imaging series with existing anatomical labels was retrieved from the PACS database of the local hospital to train an ensemble of neural networks (DenseNet-161 and ResNet-152), which process radiological images and predict the type of study they belong to. We developed an algorithm that automatically extracts relevant metadata from imaging studies, regardless of their structure, and combines it with the neural network ensemble, forming a powerful classifier. A set of 843 anonymized external studies from 321 hospitals was hand-labeled to assess performance. We tested several variations of this algorithm.
Results
MOMO achieves 92.71% accuracy and 2.63% minor errors (at 99.29% predictive power) on the external study classification task, outperforming both a commercial product (82.86% accuracy, 1.36% minor errors, 96.20% predictive power) and a pure neural network ensemble (72.69% accuracy, 10.3% minor errors, 99.05% predictive power) performing the same task. We find that the highest performance is achieved by an algorithm that combines all information into one vote-based classifier.
Conclusion
Deep Learning combined with metadata matching is a promising and flexible approach for the automated classification of external DICOM studies for PACS archiving.
Key Points
• The algorithm can successfully identify 76 medical study types across seven modalities (CT, X-ray angiography, radiographs, MRI, PET (+CT/MRI), ultrasound, and mammograms).
• The algorithm outperforms a commercial product performing the same task by a significant margin (> 9% accuracy gain).
• The performance of the algorithm increases through the application of Deep Learning techniques.
The segmentation of histopathological whole slide images into tumourous and non-tumourous types of tissue is a challenging task that requires the consideration of both local and global spatial ...contexts to classify tumourous regions precisely. The identification of subtypes of tumour tissue complicates the issue as the sharpness of separation decreases and the pathologist’s reasoning is even more guided by spatial context. However, the identification of detailed tissue types is crucial for providing personalized cancer therapies. Due to the high resolution of whole slide images, existing semantic segmentation methods, restricted to isolated image sections, are incapable of processing context information beyond. To take a step towards better context comprehension, we propose a patch neighbour attention mechanism to query the neighbouring tissue context from a patch embedding memory bank and infuse context embeddings into bottleneck hidden feature maps. Our memory attention framework (MAF) mimics a pathologist’s annotation procedure — zooming out and considering surrounding tissue context. The framework can be integrated into any encoder–decoder segmentation method. We evaluate the MAF on two public breast cancer and liver cancer data sets and an internal kidney cancer data set using famous segmentation models (U-Net, DeeplabV3) and demonstrate the superiority over other context-integrating algorithms — achieving a substantial improvement of up to 17% on Dice score. The code is publicly available at https://github.com/tio-ikim/valuing-vicinity.
Display omitted
•Valuing Vicinity: A framework to consider spatial context in semantic segmentation•We add spatial context into existing histological segmentation networks•We use embedding space context instead of extra context images context•Results indicate that context is beneficial to histological cancer segmentation•The MAF can be integrated into any encoder–decoder segmentation network
Nuclei detection and segmentation in hematoxylin and eosin-stained (H&E) tissue images are important clinical tasks and crucial for a wide range of applications. However, it is a challenging task due ...to nuclei variances in staining and size, overlapping boundaries, and nuclei clustering. While convolutional neural networks have been extensively used for this task, we explore the potential of Transformer-based networks in this domain. Therefore, we introduce a new method for automated instance segmentation of cell nuclei in digitized tissue samples using a deep learning architecture based on Vision Transformer called CellViT. CellViT is trained and evaluated on the PanNuke dataset, which is one of the most challenging nuclei instance segmentation datasets, consisting of nearly 200,000 annotated Nuclei into 5 clinically important classes in 19 tissue types. We demonstrate the superiority of large-scale in-domain and out-of-domain pre-trained Vision Transformers by leveraging the recently published Segment Anything Model and a ViT-encoder pre-trained on 104 million histological image patches - achieving state-of-the-art nuclei detection and instance segmentation performance on the PanNuke dataset with a mean panoptic quality of 0.51 and an F1-detection score of 0.83. The code is publicly available at https://github.com/TIO-IKIM/CellViT
The segmentation of histopathological whole slide images into tumourous and non-tumourous types of tissue is a challenging task that requires the consideration of both local and global spatial ...contexts to classify tumourous regions precisely. The identification of subtypes of tumour tissue complicates the issue as the sharpness of separation decreases and the pathologist's reasoning is even more guided by spatial context. However, the identification of detailed types of tissue is crucial for providing personalized cancer therapies. Due to the high resolution of whole slide images, existing semantic segmentation methods, restricted to isolated image sections, are incapable of processing context information beyond. To take a step towards better context comprehension, we propose a patch neighbour attention mechanism to query the neighbouring tissue context from a patch embedding memory bank and infuse context embeddings into bottleneck hidden feature maps. Our memory attention framework (MAF) mimics a pathologist's annotation procedure -- zooming out and considering surrounding tissue context. The framework can be integrated into any encoder-decoder segmentation method. We evaluate the MAF on a public breast cancer and an internal kidney cancer data set using famous segmentation models (U-Net, DeeplabV3) and demonstrate the superiority over other context-integrating algorithms -- achieving a substantial improvement of up to \(17\%\) on Dice score. The code is publicly available at: https://github.com/tio-ikim/valuing-vicinity
