Recent advances in deep learning for medical image segmentation demonstrate expert-level accuracy. However, application of these models in clinically realistic environments can result in poor ...generalization and decreased accuracy, mainly due to the domain shift across different hospitals, scanner vendors, imaging protocols, and patient populations etc. Common transfer learning and domain adaptation techniques are proposed to address this bottleneck. However, these solutions require data (and annotations) from the target domain to retrain the model, and is therefore restrictive in practice for widespread model deployment. Ideally, we wish to have a trained (locked) model that can work uniformly well across unseen domains without further training. In this paper, we propose a deep stacked transformation approach for domain generalization. Specifically, a series of {n} stacked transformations are applied to each image during network training. The underlying assumption is that the "expected" domain shift for a specific medical imaging modality could be simulated by applying extensive data augmentation on a single source domain, and consequently, a deep model trained on the augmented "big" data (BigAug) could generalize well on unseen domains. We exploit four surprisingly effective, but previously understudied, image-based characteristics for data augmentation to overcome the domain generalization problem. We train and evaluate the BigAug model (with {n}={9} transformations) on three different 3D segmentation tasks (prostate gland, left atrial, left ventricle) covering two medical imaging modalities (MRI and ultrasound) involving eight publicly available challenge datasets. The results show that when training on relatively small dataset (n = 10~32 volumes, depending on the size of the available datasets) from a single source domain: (i) BigAug models degrade an average of 11%(Dice score change) from source to unseen domain, substantially better than conventional augmentation (degrading 39%) and CycleGAN-based domain adaptation method (degrading 25%), (ii) BigAug is better than "shallower" stacked transforms (i.e. those with fewer transforms) on unseen domains and demonstrates modest improvement to conventional augmentation on the source domain, (iii) after training with BigAug on one source domain, performance on an unseen domain is similar to training a model from scratch on that domain when using the same number of training samples. When training on large datasets (n = 465 volumes) with BigAug, (iv) application to unseen domains reaches the performance of state-of-the-art fully supervised models that are trained and tested on their source domains. These findings establish a strong benchmark for the study of domain generalization in medical imaging, and can be generalized to the design of highly robust deep segmentation models for clinical deployment.
Locoregional therapies are commonly used to treat patients with hepatocellular carcinoma. It has been noted for many years that locoregional therapies may have additional systemic effects other than ...simple tumour elimination. Immunological “side effects” have been described in response to locoregional therapies in animal studies and in patients. With the advent of immunotherapy for hepatocellular carcinoma, there is increasing interest in determining the best way to combine immunotherapy with locoregional therapies. Herein, we provide a compact summary of answered and unanswered questions in the field, including: What animal model is best suited to test combined immune-locoregional treatments? How does tumour cell death affect immune responses? What type of immune responses have been observed in patients treated with different types of locoregional therapies? What can be surmised from the results of the first study testing the combination of locoregional therapy with immune checkpoint blockade? Finally, we discuss the outlook for this rapidly growing area of research, focussing on the issues which must be overcome to bridge the gap between interventional radiology and cancer immunology.
Prostate segmentation in transrectal ultrasound (TRUS) image is an essential prerequisite for many prostate-related clinical procedures, which, however, is also a long-standing problem due to the ...challenges caused by the low image quality and shadow artifacts. In this paper, we propose a Shadow-consistent Semi-supervised Learning (SCO-SSL) method with two novel mechanisms, namely shadow augmentation (Shadow-AUG) and shadow dropout (Shadow-DROP), to tackle this challenging problem. Specifically, Shadow-AUG enriches training samples by adding simulated shadow artifacts to the images to make the network robust to the shadow patterns. Shadow-DROP enforces the segmentation network to infer the prostate boundary using the neighboring shadow-free pixels. Extensive experiments are conducted on two large clinical datasets (a public dataset containing 1,761 TRUS volumes and an in-house dataset containing 662 TRUS volumes). In the fully-supervised setting, a vanilla U-Net equipped with our Shadow-AUG&Shadow-DROP outperforms the state-of-the-arts with statistical significance. In the semi-supervised setting, even with only 20% labeled training data, our SCO-SSL method still achieves highly competitive performance, suggesting great clinical value in relieving the labor of data annotation. Source code is released at https://github.com/DIAL-RPI/SCO-SSL .
Purpose To characterize clinically important prostate cancers missed at multiparametric (MP) magnetic resonance (MR) imaging. Materials and Methods The local institutional review board approved this ...HIPAA-compliant retrospective single-center study, which included 100 consecutive patients who had undergone MP MR imaging and subsequent radical prostatectomy. A genitourinary pathologist blinded to MP MR findings outlined prostate cancers on whole-mount pathology slices. Two readers correlated mapped lesions with reports of prospectively read MP MR images. Readers were blinded to histopathology results during prospective reading. At histopathologic examination, 80 clinically unimportant lesions (<5 mm; Gleason score, 3+3) were excluded. The same two readers, who were not blinded to histopathologic findings, retrospectively reviewed cancers missed at MP MR imaging and assigned a Prostate Imaging Reporting and Data System (PI-RADS) version 2 score to better understand false-negative lesion characteristics. Descriptive statistics were used to define patient characteristics, including age, prostate-specific antigen (PSA) level, PSA density, race, digital rectal examination results, and biopsy results before MR imaging. Student t test was used to determine any demographic differences between patients with false-negative MP MR imaging findings and those with correct prospective identification of all lesions. Results Of the 162 lesions, 136 (84%) were correctly identified with MP MR imaging. Size of eight lesions was underestimated. Among the 26 (16%) lesions missed at MP MR imaging, Gleason score was 3+4 in 17 (65%), 4+3 in one (4%), 4+4 in seven (27%), and 4+5 in one (4%). Retrospective PI-RADS version 2 scores were assigned (PI-RADS 1, n = 8; PI-RADS 2, n = 7; PI-RADS 3, n = 6; and PI-RADS 4, n = 5). On a per-patient basis, MP MR imaging depicted clinically important prostate cancer in 99 of 100 patients. At least one clinically important tumor was missed in 26 (26%) patients, and lesion size was underestimated in eight (8%). Conclusion Clinically important lesions can be missed or their size can be underestimated at MP MR imaging. Of missed lesions, 58% were not seen or were characterized as benign findings at second-look analysis. Recognition of the limitations of MP MR imaging is important, and new approaches to reduce this false-negative rate are needed.
RSNA, 2017 Online supplemental material is available for this article.
Various liposomal drug carriers have been developed to overcome short plasma half-life and toxicity related side effects of chemotherapeutic agents. We developed a mathematical model to compare ...different liposome formulations of doxorubicin (DOX): conventional chemotherapy (Free-DOX), Stealth liposomes (Stealth-DOX), temperature sensitive liposomes (TSL) with intra-vascular triggered release (TSL-i), and TSL with extra-vascular triggered release (TSL-e). All formulations were administered as bolus at a dose of 9 mg/kg. For TSL, we assumed locally triggered release due to hyperthermia for 30 min. Drug concentrations were determined in systemic plasma, aggregate body tissue, cardiac tissue, tumor plasma, tumor interstitial space, and tumor cells. All compartments were assumed perfectly mixed, and represented by ordinary differential equations. Contribution of liposomal extravasation was negligible in the case of TSL-i, but was the major delivery mechanism for Stealth-DOX and for TSL-e. The dominant delivery mechanism for TSL-i was release within the tumor plasma compartment with subsequent tissue- and cell uptake of released DOX. Maximum intracellular tumor drug concentrations for Free-DOX, Stealth-DOX, TSL-i, and TSL-e were 3.4, 0.4, 100.6, and 15.9 µg/g, respectively. TSL-i and TSL-e allowed for high local tumor drug concentrations with reduced systemic exposure compared to Free-DOX. While Stealth-DOX resulted in high tumor tissue concentrations compared to Free-DOX, only a small fraction was bioavailable, resulting in little cellular uptake. Consistent with clinical data, Stealth-DOX resulted in similar tumor intracellular concentrations as Free-DOX, but with reduced systemic exposure. Optimal release time constants for maximum cellular uptake for Stealth-DOX, TSL-e, and TSL-i were 45 min, 11 min, and <3 s, respectively. Optimal release time constants were shorter for MDR cells, with ∼4 min for Stealth-DOX and for TSL-e. Tissue concentrations correlated well quantitatively with a prior in-vivo study. Mathematical models may thus allow optimization of drug delivery systems to achieve a better therapeutic index.
Cellular diversity in tumors is a key factor for therapeutic failures and lethal outcomes of solid malignancies. Here, we determined the single-cell transcriptomic landscape of liver cancer ...biospecimens from 19 patients. We found varying degrees of heterogeneity in malignant cells within and between tumors and diverse landscapes of tumor microenvironment (TME). Strikingly, tumors with higher transcriptomic diversity were associated with patient's worse overall survival. We found a link between hypoxia-dependent vascular endothelial growth factor expression in tumor diversity and TME polarization. Moreover, T cells from higher heterogeneous tumors showed lower cytolytic activities. Consistent results were found using bulk genomic and transcriptomic profiles of 765 liver tumors. Our results offer insight into the diverse ecosystem of liver cancer and its impact on patient prognosis.
Display omitted
•HCC and iCCA have a varying degree of transcriptomic diversity•Tumor transcriptomic diversity is associated with patient outcomes•Tumor-derived VEGF drives microenvironmental reprogramming•T cells derived from higher heterogeneous tumors showed lower cytolytic activities
Ma et al. perform single-cell RNA sequencing of primary liver cancers and find heterogeneity in malignant cells and in the tumor microenvironment, the degree of which negatively associates with patient prognosis. They demonstrate that VEGF expression is linked to tumor diversity and T cell dysfunction.
Immunotherapy promises unprecedented benefits to patients with cancer. However, the majority of cancer types, including high-risk neuroblastoma, remain immunologically unresponsive. High-intensity ...focused ultrasound (HIFU) is a noninvasive technique that can mechanically fractionate tumors, transforming immunologically "cold" tumors into responsive "hot" tumors.
We treated <2% of tumor volume in previously unresponsive, large, refractory murine neuroblastoma tumors with mechanical HIFU and assessed systemic immune response using flow cytometry, ELISA, and gene sequencing. In addition, we combined this treatment with αCTLA-4 and αPD-L1 to study its effect on the immune response and long-term survival.
Combining HIFU with αCTLA-4 and αPD-L1 significantly enhances antitumor response, improving survival from 0% to 62.5%. HIFU alone causes upregulation of splenic and lymph node NK cells and circulating IL2, IFNγ, and DAMPs, whereas immune regulators like CD4
Foxp3
, IL10, and VEGF-A are significantly reduced. HIFU combined with checkpoint inhibitors induced significant increases in intratumoral CD4
, CD8α
, and CD8α
CD11c
cells, CD11c
in regional lymph nodes, and decrease in circulating IL10 compared with untreated group. We also report significant abscopal effect following unilateral treatment of mice with large, established bilateral tumors using HIFU and checkpoint inhibitors compared with tumors treated with HIFU or checkpoint inhibitors alone (61.1% survival,
< 0.0001). This combination treatment significantly also induces CD4
CD44
CD62L
and CD8α
CD44
CD62L
population and is adoptively transferable, imparting immunity, slowing subsequent
tumor engraftment.
Mechanical fractionation of tumors using HIFU can effectively induce immune sensitization in a previously unresponsive murine neuroblastoma model and promises a novel yet efficacious immunoadjuvant modality to overcome therapeutic resistance.
•Prostate ultrasound volume segmentation from a new perspective by efficiently representing the prostate shape in the polar coordinate space with emphasis on the challenging base and apex regions.•A ...novel polar transform network (PTN) is proposed to implement polar transform for end-to-end deep learning based image segmentation with much fewer parameters, which achieves superior performance compared with state-of-the-art methods.•An innovative centroid perturbed test-time augmentation (CPTTA) strategy to refine the segmentation results and estimate uncertainty maps.
Display omitted
Automatic and accurate prostate ultrasound segmentation is a long-standing and challenging problem due to the severe noise and ambiguous/missing prostate boundaries. In this work, we propose a novel polar transform network (PTN) to handle this problem from a fundamentally new perspective, where the prostate is represented and segmented in the polar coordinate space rather than the original image grid space. This new representation gives a prostate volume, especially the most challenging apex and base sub-areas, much denser samples than the background and thus facilitate the learning of discriminative features for accurate prostate segmentation. Moreover, in the polar representation, the prostate surface can be efficiently parameterized using a 2D surface radius map with respect to a centroid coordinate, which allows the proposed PTN to obtain superior accuracy compared with its counterparts using convolutional neural networks while having significantly fewer (18%∼41%) trainable parameters. We also equip our PTN with a novel strategy of centroid perturbed test-time augmentation (CPTTA), which is designed to further improve the segmentation accuracy and quantitatively assess the model uncertainty at the same time. The uncertainty estimation function provides valuable feedback to clinicians when manual modifications or approvals are required for the segmentation, substantially improving the clinical significance of our work. We conduct a three-fold cross validation on a clinical dataset consisting of 315 transrectal ultrasound (TRUS) images to comprehensively evaluate the performance of the proposed method. The experimental results show that our proposed PTN with CPTTA outperforms the state-of-the-art methods with statistical significance on most of the metrics while exhibiting a much smaller model size. Source code of the proposed PTN is released at https://github.com/DIAL-RPI/PTN.
PDF
