Photoacoustic imaging (PAI) is a promising emerging imaging modality that enables spatially resolved imaging of optical tissue properties up to several centimeters deep in tissue, creating the ...potential for numerous exciting clinical applications. However, extraction of relevant tissue parameters from the raw data requires the solving of inverse image reconstruction problems, which have proven extremely difficult to solve. The application of deep learning methods has recently exploded in popularity, leading to impressive successes in the context of medical imaging and also finding first use in the field of PAI. Deep learning methods possess unique advantages that can facilitate the clinical translation of PAI, such as extremely fast computation times and the fact that they can be adapted to any given problem. In this review, we examine the current state of the art regarding deep learning in PAI and identify potential directions of research that will help to reach the goal of clinical applicability.
The ability of photoacoustic imaging to measure functional tissue properties, such as blood oxygenation sOFormula: see text, enables a wide variety of possible applications. sOFormula: see text can ...be computed from the ratio of oxyhemoglobin HbOFormula: see text and deoxyhemoglobin Hb, which can be distuinguished by multispectral photoacoustic imaging due to their distinct wavelength-dependent absorption. However, current methods for estimating sOFormula: see text yield inaccurate results in realistic settings, due to the unknown and wavelength-dependent influence of the light fluence on the signal. In this work, we propose learned spectral decoloring to enable blood oxygenation measurements to be inferred from multispectral photoacoustic imaging. The method computes sOFormula: see text pixel-wise, directly from initial pressure spectra Formula: see text, which represent initial pressure values at a fixed spatial location Formula: see text over all recorded wavelengths Formula: see text. The method is compared to linear unmixing approaches, as well as pOFormula: see text and blood gas analysis reference measurements. Experimental results suggest that the proposed method is able to obtain sOFormula: see text estimates from multispectral photoacoustic measurements in silico, in vitro, and in vivo.
Accurate measurement of optical absorption coefficients from photoacoustic imaging (PAI) data would enable direct mapping of molecular concentrations, providing vital clinical insight. The ill-posed ...nature of the problem of absorption coefficient recovery has prohibited PAI from achieving this goal in living systems due to the domain gap between simulation and experiment. To bridge this gap, we introduce a collection of experimentally well-characterised imaging phantoms and their digital twins. This first-of-a-kind phantom data set enables supervised training of a U-Net on experimental data for pixel-wise estimation of absorption coefficients. We show that training on simulated data results in artefacts and biases in the estimates, reinforcing the existence of a domain gap between simulation and experiment. Training on experimentally acquired data, however, yielded more accurate and robust estimates of optical absorption coefficients. We compare the results to fluence correction with a Monte Carlo model from reference optical properties of the materials, which yields a quantification error of approximately 20%. Application of the trained U-Nets to a blood flow phantom demonstrated spectral biases when training on simulated data, while application to a mouse model highlighted the ability of both learning-based approaches to recover the depth-dependent loss of signal intensity. We demonstrate that training on experimental phantoms can restore the correlation of signal amplitudes measured in depth. While the absolute quantification error remains high and further improvements are needed, our results highlight the promise of deep learning to advance quantitative PAI.
Reconstruction of photoacoustic (PA) images acquired with clinical ultrasound transducers is usually performed using the Delay and Sum (DAS) beamforming algorithm. Recently, a variant of DAS, ...referred to as Delay Multiply and Sum (DMAS) beamforming has been shown to provide increased contrast, signal-to-noise ratio (SNR) and resolution in PA imaging. The main reasons for the use of DAS beamforming in photoacoustics are its simple implementation, real-time capability, and the linearity of the beamformed image to the PA signal. This is crucial for the identification of different chromophores in multispectral PA applications. In contrast, current DMAS implementations are not responsive to the full spectrum of sound frequencies from a photoacoustic source and have not been shown to provide a reconstruction linear to the PA signal. Furthermore, due to its increased computational complexity, DMAS has not been shown yet to work in real-time. Here, we present an open-source real-time variant of the DMAS algorithm, signed DMAS (sDMAS), that ensures linearity in the original PA signal response while providing the increased image quality of DMAS. We show the applicability of sDMAS for multispectral PA applications, in vitro and in vivo. The sDMAS and reference DAS algorithms were integrated in the open-source Medical Imaging Interaction Toolkit (MITK) and are available as real-time capable implementations.
Image-based tracking of medical instruments is an integral part of surgical data science applications. Previous research has addressed the tasks of detecting, segmenting and tracking medical ...instruments based on laparoscopic video data. However, the proposed methods still tend to fail when applied to challenging images and do not generalize well to data they have not been trained on. This paper introduces the Heidelberg Colorectal (HeiCo) data set - the first publicly available data set enabling comprehensive benchmarking of medical instrument detection and segmentation algorithms with a specific emphasis on method robustness and generalization capabilities. Our data set comprises 30 laparoscopic videos and corresponding sensor data from medical devices in the operating room for three different types of laparoscopic surgery. Annotations include surgical phase labels for all video frames as well as information on instrument presence and corresponding instance-wise segmentation masks for surgical instruments (if any) in more than 10,000 individual frames. The data has successfully been used to organize international competitions within the Endoscopic Vision Challenges 2017 and 2019.
In medical applications, the accuracy and robustness of imaging methods are of crucial importance to ensure optimal patient care. While photoacoustic imaging (PAI) is an emerging modality with ...promising clinical applicability, state-of-the-art approaches to quantitative photoacoustic imaging (qPAI), which aim to solve the ill-posed inverse problem of recovering optical absorption from the measurements obtained, currently cannot comply with these high standards. This can be attributed to the fact that existing methods often rely on several simplifying a priori assumptions of the underlying physical tissue properties or cannot deal with realistic noise levels. In this manuscript, we address this issue with a new method for estimating an indicator of the uncertainty of an estimated optical property. Specifically, our method uses a deep learning model to compute error estimates for optical parameter estimations of a qPAI algorithm. Functional tissue parameters, such as blood oxygen saturation, are usually derived by averaging over entire signal intensity-based regions of interest (ROIs). Therefore, we propose to reduce the systematic error of the ROI samples by additionally discarding those pixels for which our method estimates a high error and thus a low confidence. In silico experiments show an improvement in the accuracy of optical absorption quantification when applying our method to refine the ROI, and it might thus become a valuable tool for increasing the robustness of qPAI methods.
Establishing tissue-mimicking biophotonic phantom materials that provide long-term stability are imperative to enable the comparison of biomedical imaging devices across vendors and institutions, ...support the development of internationally recognized standards, and assist the clinical translation of novel technologies. Here, a manufacturing process is presented that results in a stable, low-cost, tissue-mimicking copolymer-in-oil material for use in photoacoustic, optical, and ultrasound standardization efforts. The base material consists of mineral oil and a copolymer with defined Chemical Abstract Service (CAS) numbers. The protocol presented here yields a representative material with a speed of sound c(f) = 1,481 ± 0.4 m·s
at 5 MHz (corresponds to the speed of sound of water at 20 °C), acoustic attenuation α(f) = 6.1 ± 0.06 dB·cm
at 5 MHz, optical absorption µa(λ) = 0.05 ± 0.005 mm
at 800 nm, and optical scattering µs'(λ) = 1 ± 0.1 mm
at 800 nm. The material allows independent tuning of the acoustic and optical properties by respectively varying the polymer concentration or light scattering (titanium dioxide) and absorbing agents (oil-soluble dye). The fabrication of different phantom designs is displayed and the homogeneity of the resulting test objects is confirmed using photoacoustic imaging. Due to its facile, repeatable fabrication process and durability, as well as its biologically relevant properties, the material recipe has high promise in multimodal acoustic-optical standardization initiatives.
Photoacoustic (PA) imaging has the potential to revolutionize functional medical imaging in healthcare due to the valuable information on tissue physiology contained in multispectral photoacoustic ...measurements. Clinical translation of the technology requires conversion of the high-dimensional acquired data into clinically relevant and interpretable information. In this work, we present a deep learning-based approach to semantic segmentation of multispectral photoacoustic images to facilitate image interpretability. Manually annotated photoacoustic and ultrasound imaging data are used as reference and enable the training of a deep learning-based segmentation algorithm in a supervised manner. Based on a validation study with experimentally acquired data from 16 healthy human volunteers, we show that automatic tissue segmentation can be used to create powerful analyses and visualizations of multispectral photoacoustic images. Due to the intuitive representation of high-dimensional information, such a preprocessing algorithm could be a valuable means to facilitate the clinical translation of photoacoustic imaging.
Photoacoustic tomography (PAT) has the potential to recover morphological and functional tissue properties with high spatial resolution. However, previous attempts to solve the optical inverse ...problem with supervised machine learning were hampered by the absence of labeled reference data. While this bottleneck has been tackled by simulating training data, the domain gap between real and simulated images remains an unsolved challenge. We propose a novel approach to PAT image synthesis that involves subdividing the challenge of generating plausible simulations into two disjoint problems: (1) Probabilistic generation of realistic tissue morphology, and (2) pixel-wise assignment of corresponding optical and acoustic properties. The former is achieved with Generative Adversarial Networks (GANs) trained on semantically annotated medical imaging data. According to a validation study on a downstream task our approach yields more realistic synthetic images than the traditional model-based approach and could therefore become a fundamental step for deep learning-based quantitative PAT (qPAT).
Photoacoustic imaging (PAI), also referred to as optoacoustic imaging, has shown promise in early-stage clinical trials in a range of applications from inflammatory diseases to cancer. While the ...first PAI systems have recently received regulatory approvals, successful adoption of PAI technology into healthcare systems for clinical decision making must still overcome a range of barriers, from education and training to data acquisition and interpretation. The International Photoacoustic Standardisation Consortium (IPASC) undertook an community exercise in 2022 to identify and understand these barriers, then develop a roadmap of strategic plans to address them. Here, we outline the nature and scope of the barriers that were identified, along with short-, medium- and long-term community efforts required to overcome them, both within and beyond the IPASC group.
PDF
