We address the challenging problem of whole slide image (WSI) classification. WSIs have very high resolutions and usually lack localized annotations. WSI classification can be cast as a multiple ...instance learning (MIL) problem when only slide-level labels are available. We propose a MIL-based method for WSI classification and tumor detection that does not require localized annotations. Our method has three major components. First, we introduce a novel MIL aggregator that models the relations of the instances in a dual-stream architecture with trainable distance measurement. Second, since WSIs can produce large or unbalanced bags that hinder the training of MIL models, we propose to use self-supervised contrastive learning to extract good representations for MIL and alleviate the issue of prohibitive memory cost for large bags. Third, we adopt a pyramidal fusion mechanism for multiscale WSI features, and further improve the accuracy of classification and localization. Our model is evaluated on two representative WSI datasets. The classification accuracy of our model compares favorably to fully-supervised methods, with less than 2% accuracy gap across datasets. Our results also outperform all previous MIL-based methods. Additional benchmark results on standard MIL datasets further demonstrate the superior performance of our MIL aggregator on general MIL problems.
State-of-the-art light and electron microscopes are capable of acquiring large image datasets, but quantitatively evaluating the data often involves manually annotating structures of interest. This ...process is time-consuming and often a major bottleneck in the evaluation pipeline. To overcome this problem, we have introduced the Trainable Weka Segmentation (TWS), a machine learning tool that leverages a limited number of manual annotations in order to train a classifier and segment the remaining data automatically. In addition, TWS can provide unsupervised segmentation learning schemes (clustering) and can be customized to employ user-designed image features or classifiers.
TWS is distributed as open-source software as part of the Fiji image processing distribution of ImageJ at http://imagej.net/Trainable_Weka_Segmentation .
ignacio.arganda@ehu.eus.
Supplementary data are available at Bioinformatics online.
ImageJ is an image analysis program extensively used in the biological sciences and beyond. Due to its ease of use, recordable macro language, and extensible plug-in architecture, ImageJ enjoys ...contributions from non-programmers, amateur programmers, and professional developers alike. Enabling such a diversity of contributors has resulted in a large community that spans the biological and physical sciences. However, a rapidly growing user base, diverging plugin suites, and technical limitations have revealed a clear need for a concerted software engineering effort to support emerging imaging paradigms, to ensure the software's ability to handle the requirements of modern science.
We rewrote the entire ImageJ codebase, engineering a redesigned plugin mechanism intended to facilitate extensibility at every level, with the goal of creating a more powerful tool that continues to serve the existing community while addressing a wider range of scientific requirements. This next-generation ImageJ, called "ImageJ2" in places where the distinction matters, provides a host of new functionality. It separates concerns, fully decoupling the data model from the user interface. It emphasizes integration with external applications to maximize interoperability. Its robust new plugin framework allows everything from image formats, to scripting languages, to visualization to be extended by the community. The redesigned data model supports arbitrarily large, N-dimensional datasets, which are increasingly common in modern image acquisition. Despite the scope of these changes, backwards compatibility is maintained such that this new functionality can be seamlessly integrated with the classic ImageJ interface, allowing users and developers to migrate to these new methods at their own pace.
Scientific imaging benefits from open-source programs that advance new method development and deployment to a diverse audience. ImageJ has continuously evolved with this idea in mind; however, new and emerging scientific requirements have posed corresponding challenges for ImageJ's development. The described improvements provide a framework engineered for flexibility, intended to support these requirements as well as accommodate future needs. Future efforts will focus on implementing new algorithms in this framework and expanding collaborations with other popular scientific software suites.
For the past 25 years NIH Image and ImageJ software have been pioneers as open tools for the analysis of scientific images. We discuss the origins, challenges and solutions of these two programs, and ...how their history can serve to advise and inform other software projects.
Display omitted
•TrackMate is a software tool for automated, and semi-automated particle tracking.•TrackMate’s major development focus is on usability and extensibility.•TrackMate leverages Fiji to ...help provides its functionality and extensibility.•TrackMate is used for: C. elegans lineaging, NEMO assembly and clathrin dynamics.•Challenging imaging problems can robustly be analyzed using semi-automatic methods.
We present TrackMate, an open source Fiji plugin for the automated, semi-automated, and manual tracking of single-particles. It offers a versatile and modular solution that works out of the box for end users, through a simple and intuitive user interface. It is also easily scriptable and adaptable, operating equally well on 1D over time, 2D over time, 3D over time, or other single and multi-channel image variants. TrackMate provides several visualization and analysis tools that aid in assessing the relevance of results. The utility of TrackMate is further enhanced through its ability to be readily customized to meet specific tracking problems. TrackMate is an extensible platform where developers can easily write their own detection, particle linking, visualization or analysis algorithms within the TrackMate environment. This evolving framework provides researchers with the opportunity to quickly develop and optimize new algorithms based on existing TrackMate modules without the need of having to write de novo user interfaces, including visualization, analysis and exporting tools.
The current capabilities of TrackMate are presented in the context of three different biological problems. First, we perform Caenorhabditis-elegans lineage analysis to assess how light-induced damage during imaging impairs its early development. Our TrackMate-based lineage analysis indicates the lack of a cell-specific light-sensitive mechanism. Second, we investigate the recruitment of NEMO (NF-κB essential modulator) clusters in fibroblasts after stimulation by the cytokine IL-1 and show that photodamage can generate artifacts in the shape of TrackMate characterized movements that confuse motility analysis. Finally, we validate the use of TrackMate for quantitative lifetime analysis of clathrin-mediated endocytosis in plant cells.
Over the past two decades, synergistic innovations in imaging technology have resulted in a revolution in which a range of biomedical applications are now benefiting from fluorescence imaging. ...Specifically, advances in fluorophore chemistry and imaging hardware, and the identification of targetable biomarkers have now positioned intraoperative fluorescence as a highly specific real-time detection modality for surgeons in oncology. In particular, the deeper tissue penetration and limited autofluorescence of near-infrared (NIR) fluorescence imaging improves the translational potential of this modality over visible-light fluorescence imaging. Rapid developments in fluorophores with improved characteristics, detection instrumentation, and targeting strategies led to the clinical testing in the early 2010s of the first targeted NIR fluorophores for intraoperative cancer detection. The foundations for the advances that underline this technology continue to be nurtured by the multidisciplinary collaboration of chemists, biologists, engineers, and clinicians. In this Review, we highlight the latest developments in NIR fluorophores, cancer-targeting strategies, and detection instrumentation for intraoperative cancer detection, and consider the unique challenges associated with their effective application in clinical settings.
For decades, biologists have relied on software to visualize and interpret imaging data. As techniques for acquiring images increase in complexity, resulting in larger multidimensional datasets, ...imaging software must adapt. ImageJ is an open‐source image analysis software platform that has aided researchers with a variety of image analysis applications, driven mainly by engaged and collaborative user and developer communities. The close collaboration between programmers and users has resulted in adaptations to accommodate new challenges in image analysis that address the needs of ImageJ's diverse user base. ImageJ consists of many components, some relevant primarily for developers and a vast collection of user‐centric plugins. It is available in many forms, including the widely used Fiji distribution. We refer to this entire ImageJ codebase and community as the ImageJ ecosystem. Here we review the core features of this ecosystem and highlight how ImageJ has responded to imaging technology advancements with new plugins and tools in recent years. These plugins and tools have been developed to address user needs in several areas such as visualization, segmentation, and tracking of biological entities in large, complex datasets. Moreover, new capabilities for deep learning are being added to ImageJ, reflecting a shift in the bioimage analysis community towards exploiting artificial intelligence. These new tools have been facilitated by profound architectural changes to the ImageJ core brought about by the ImageJ2 project. Therefore, we also discuss the contributions of ImageJ2 to enhancing multidimensional image processing and interoperability in the ImageJ ecosystem.
Highlights
ImageJ is an open‐source image analysis software with a large, diverse user base. ImageJ has several components and distributions such that we refer to the entirety as the ImageJ ecosystem. Recent developments have adapted to the needs of users and the demands of increasingly large, complex biological datasets accompanying technological advancements in imaging. This review highlights several new tools developed in the ImageJ ecosystem to address needs for improved visualization and analysis of biological features.
Mammographically dense breast tissue is one of the greatest risk factors for developing breast carcinoma. Despite the strong clinical correlation, breast density has not been causally linked to ...tumorigenesis, largely because no animal model has existed for studying breast tissue density. Importantly, regions of high breast density are associated with increased stromal collagen. Thus, the influence of the extracellular matrix on breast carcinoma development and the underlying molecular mechanisms are not understood.
To study the effects of collagen density on mammary tumor formation and progression, we utilized a bi-transgenic tumor model with increased stromal collagen in mouse mammary tissue. Imaging of the tumors and tumor-stromal interface in live tumor tissue was performed with multiphoton laser-scanning microscopy to generate multiphoton excitation and spectrally resolved fluorescent lifetimes of endogenous fluorophores. Second harmonic generation was utilized to image stromal collagen.
Herein we demonstrate that increased stromal collagen in mouse mammary tissue significantly increases tumor formation approximately three-fold (p < 0.00001) and results in a significantly more invasive phenotype with approximately three times more lung metastasis (p < 0.05). Furthermore, the increased invasive phenotype of tumor cells that arose within collagen-dense mammary tissues remains after tumor explants are cultured within reconstituted three-dimensional collagen gels. To better understand this behavior we imaged live tumors using nonlinear optical imaging approaches to demonstrate that local invasion is facilitated by stromal collagen re-organization and that this behavior is significantly increased in collagen-dense tissues. In addition, using multiphoton fluorescence and spectral lifetime imaging we identify a metabolic signature for flavin adenine dinucleotide, with increased fluorescent intensity and lifetime, in invading metastatic cells.
This study provides the first data causally linking increased stromal collagen to mammary tumor formation and metastasis, and demonstrates that fundamental differences arise and persist in epithelial tumor cells that progressed within collagen-dense microenvironments. Furthermore, the imaging techniques and signature identified in this work may provide useful diagnostic tools to rapidly assess fresh tissue biopsies.
PDF
