Performance is a critical challenge in mobile image processing. Given a reference imaging pipeline, or even human-adjusted pairs of images, we seek to reproduce the enhancements and enable real-time ...evaluation. For this, we introduce a new neural network architecture inspired by bilateral grid processing and local affine color transforms. Using pairs of input/output images, we train a convolutional neural network to predict the coefficients of a locally-affine model in bilateral space. Our architecture learns to make local, global, and content-dependent decisions to approximate the desired image transformation. At runtime, the neural network consumes a low-resolution version of the input image, produces a set of affine transformations in bilateral space, upsamples those transformations in an edge-preserving fashion using a new slicing node, and then applies those upsampled transformations to the full-resolution image. Our algorithm processes high-resolution images on a smartphone in milliseconds, provides a real-time viewfinder at 1080p resolution, and matches the quality of state-of-the-art approximation techniques on a large class of image operators. Unlike previous work, our model is trained off-line from data and therefore does not require access to the original operator at runtime. This allows our model to learn complex, scene-dependent transformations for which no reference implementation is available, such as the photographic edits of a human retoucher.
Chaos theory has been widely used in the design of image encryption schemes. Some low-dimensional chaotic maps have been proved to be easily predictable because of their small chaotic space. On the ...other hand, high-dimensional chaotic maps have a larger chaotic space. However, their structures are too complicated, and consequently, they are not suitable for real-time image encryption. Motivated by this, we propose a new fractional one-dimensional chaotic map with a large chaotic space. The proposed map has a simple structure and a high chaotic behavior in an extensive range of its control parameters values. Several chaos theoretical tools and tests have been carried out to analyze and prove the proposed map’s high chaotic behavior. Moreover, we use the proposed map in the design of a novel real-time image encryption scheme. In this new scheme, we combine the substitution and permutation stages to simultaneously modify both of the pixels’ positions and values. The merge of these two stages and the use of the new simple one-dimensional chaotic map significantly increase the proposed scheme’s security and speed. Besides, the simulation and experimental analysis prove that the proposed scheme has high performances.
Codelivery of diagnostic probes and therapeutic molecules often suffers from intrinsic complexity and premature leakage from or degradation of the nanocarrier. Inspired by the “Y” shape of ...indocyanine green (ICG), the dye is integrated in an amphiphilic lipopeptide (RNF). The hydrophilic segment is composed of arginine‐rich dendritic peptides, while cyanine dyes are modified with two long carbon chains and employed as the hydrophobic moiety. They are linked through a disulfide linkage to improve the responsivity in the tumor microenvironment. After formulation with other lipopeptides at an optimized ratio, the theranostic system (RNS‐2) forms lipid‐based nanoparticles with slight positive zeta potential enabling efficient condensation of DNA. The RNS‐2 displays glutathione responded gene release, activatable fluorescence recovery, and up to sevenfold higher in vitro transfection than Lipofectamine 2000. Compared with a Cy3 and Cy5 labeled fluorescence resonance energy transfer indicator for gene release, the “turn‐on” indocyanine green analogs exhibit longer emission wavelength and better positive correlation with the dynamic processes of gene delivery. More importantly, the RNS‐2 system enables efficient near infrared imaging guided gene transfer in tumor‐bearing mice and thus provides more precise and accurate information on location of the cargo gene and synthesized carriers.
Inspired by the “Y” shape of indocyanine green, a viral mimicking system that integrates cyanine dye in the main part of a two‐tailed amphiphilic lipopeptide is fabricated as a real‐time imaging‐guided gene delivery vector, which exhibits reductive responded gene release, high gene transfection, and turn‐on fluorescence imaging with more precise and accurate information for carrier dissociation, gene release, and expression.
Underwater images acquired in scattering environments are generally of poor quality because of the attenuation and backscattering of light when it passes through water with scattering particles. ...Polarimetric de-scattering methods can be used to significantly enhance the imaging quality in such a so-called turbid water. However, due to the complexity of polarimetric de-scattering algorithms, it is hard to achieve real-time de-scattering output from a polarimetric camera. In this paper, in order to efficiently increase the computational efficiency, the polarimetric de-scattering algorithm is optimized and a multi-threading framework is developed that enables the algorithm to run in real-time on ordinary laptops for the same polarimetric camera. We demonstrate the imaging performance by using the underwater polarimetric de-scattering system we proposed. We analyze the algorithm under different scattering conditions and discuss its optimal parameters. We find that there is a significant increase in the number of SIFT (i.e., scale invariant feature transform) feature points extracted from the processed image compared to the original image, showing that our system has potential applications in pattern recognition, computer vision, etc.
Purpose
To perform the final quality assurance of our fluoroscopic‐based markerless tumor tracking for gated carbon‐ion pencil beam scanning (C‐PBS) radiotherapy using a rotating gantry system, we ...evaluated the geometrical accuracy and tumor tracking accuracy using a moving chest phantom with simulated respiration.
Methods
The positions of the dynamic flat panel detector (DFPD) and x‐ray tube are subject to changes due to gantry sag. To compensate for this, we generated a geometrical calibration table (gantry flex map) in 15° gantry angle steps by the bundle adjustment method. We evaluated five metrics: (a) Geometrical calibration was evaluated by calculating chest phantom positional error using 2D/3D registration software for each 5° step of the gantry angle. (b) Moving phantom displacement accuracy was measured (±10 mm in 1‐mm steps) with a laser sensor. (c) Tracking accuracy was evaluated with machine learning (ML) and multi‐template matching (MTM) algorithms, which used fluoroscopic images and digitally reconstructed radiographic (DRR) images as training data. The chest phantom was continuously moved ±10 mm in a sinusoidal path with a moving cycle of 4 s and respiration was simulated with ±5 mm expansion/contraction with a cycle of 2 s. This was performed with the gantry angle set at 0°, 45°, 120°, and 240°. (d) Four types of interlock function were evaluated: tumor velocity, DFPD image brightness variation, tracking anomaly detection, and tracking positional inconsistency in between the two corresponding rays. (e) Gate on/off latency, gating control system latency, and beam irradiation latency were measured using a laser sensor and an oscilloscope.
Results
By applying the gantry flex map, phantom positional accuracy was improved from 1.03 mm/0.33° to <0.45 mm/0.27° for all gantry angles. The moving phantom displacement error was 0.1 mm. Due to long computation time, the tracking accuracy achieved with ML was <0.49 mm (=95% confidence interval CI) for imaging rates of 15 and 7.5 fps; those at 30 fps were decreased to 1.84 mm (95% CI: 1.79 mm–1.92 mm). The tracking positional accuracy with MTM was <0.52 mm (=95% CI) for all gantry angles and imaging frame rates. The tumor velocity interlock signal delay time was 44.7 ms (=1.3 frame). DFPD image brightness interlock latency was 34 ms (=1.0 frame). The tracking positional error was improved from 2.27 ± 2.67 mm to 0.25 ± 0.24 mm by the tracking anomaly detection interlock function. Tracking positional inconsistency interlock signal was output within 5.0 ms. The gate on/off latency was <82.7 ± 7.6 ms. The gating control system latency was <3.1 ± 1.0 ms. The beam irradiation latency was <8.7 ± 1.2 ms.
Conclusions
Our markerless tracking system is now ready for clinical use. We hope to shorten the computation time needed by the ML algorithm at 30 fps in the future.
Purpose
Kilovoltage intrafraction monitoring (KIM) allows for real‐time image guidance for tracking tumor motion in six‐degrees‐of‐freedom (6DoF) on a standard linear accelerator. This study assessed ...the geometric accuracy and precision of KIM used to guide patient treatments in the TROG 15.01 multi‐institutional Stereotactic Prostate Ablative Radiotherapy with KIM trial and investigated factors affecting accuracy and precision.
Methods
Fractions from 44 patients with prostate cancer treated using KIM‐guided SBRT were analyzed across four institutions, on two different linear accelerator models and two different beam models (6 MV and 10 MV FFF). The geometric accuracy and precision of KIM was assessed from over 33 000 images (translation) and over 9000 images (rotation) by comparing the real‐time measured motion to retrospective kV/MV triangulation. Factors potentially affecting accuracy, including contrast‐to‐noise ratio (CNR) of kV images and incorrect marker segmentation, were also investigated.
Results
The geometric accuracy and precision did not depend on treatment institution, beam model or motion magnitude, but was correlated with gantry angle. The centroid geometric accuracy and precision of the KIM system for SABR prostate treatments was 0.0 ± 0.5, 0.0 ± 0.4 and 0.1 ± 0.3 mm for translation, and −0.1 ± 0.6°, −0.1 ± 1.4° and −0.1 ± 1.0° for rotation in the AP, LR and SI directions respectively. Centroid geometric error exceeded 2 mm for 0.05% of this dataset. No significant relationship was found between large geometric error and CNR or marker segmentation correlation.
Conclusions
This study demonstrated the ability of KIM to locate the prostate with accuracy below other uncertainties in radiotherapy treatments, and the feasibility for KIM to be implemented across multiple institutions.
Purpose
During image‐guided prostate biopsy, needles are targeted at tissues that are suspicious of cancer to obtain specimen for histological examination. Unfortunately, patient motion causes ...targeting errors when using an MR‐transrectal ultrasound (TRUS) fusion approach to augment the conventional biopsy procedure. This study aims to develop an automatic motion correction algorithm approaching the frame rate of an ultrasound system to be used in fusion‐based prostate biopsy systems. Two modes of operation have been investigated for the clinical implementation of the algorithm: motion compensation using a single user initiated correction performed prior to biopsy, and real‐time continuous motion compensation performed automatically as a background process.
Methods
Retrospective 2D and 3D TRUS patient images acquired prior to biopsy gun firing were registered using an intensity‐based algorithm utilizing normalized cross‐correlation and Powell's method for optimization. 2D and 3D images were downsampled and cropped to estimate the optimal amount of image information that would perform registrations quickly and accurately. The optimal search order during optimization was also analyzed to avoid local optima in the search space. Error in the algorithm was computed using target registration errors (TREs) from manually identified homologous fiducials in a clinical patient dataset. The algorithm was evaluated for real‐time performance using the two different modes of clinical implementations by way of user initiated and continuous motion compensation methods on a tissue mimicking prostate phantom.
Results
After implementation in a TRUS‐guided system with an image downsampling factor of 4, the proposed approach resulted in a mean ± std TRE and computation time of 1.6 ± 0.6 mm and 57 ± 20 ms respectively. The user initiated mode performed registrations with in‐plane, out‐of‐plane, and roll motions computation times of 108 ± 38 ms, 60 ± 23 ms, and 89 ± 27 ms, respectively, and corresponding registration errors of 0.4 ± 0.3 mm, 0.2 ± 0.4 mm, and 0.8 ± 0.5°. The continuous method performed registration significantly faster (P < 0.05) than the user initiated method, with observed computation times of 35 ± 8 ms, 43 ± 16 ms, and 27 ± 5 ms for in‐plane, out‐of‐plane, and roll motions, respectively, and corresponding registration errors of 0.2 ± 0.3 mm, 0.7 ± 0.4 mm, and 0.8 ± 1.0°.
Conclusions
The presented method encourages real‐time implementation of motion compensation algorithms in prostate biopsy with clinically acceptable registration errors. Continuous motion compensation demonstrated registration accuracy with submillimeter and subdegree error, while performing < 50 ms computation times. Image registration technique approaching the frame rate of an ultrasound system offers a key advantage to be smoothly integrated to the clinical workflow. In addition, this technique could be used further for a variety of image‐guided interventional procedures to treat and diagnose patients by improving targeting accuracy.
Purpose:
Kilovoltage intrafraction monitoring (KIM) is a real‐time image guidance method that uses widely available radiotherapy technology, i.e., a gantry‐mounted x‐ray imager. The authors report on ...the geometric and dosimetric results of the first patient treatment using KIM which occurred on September 16, 2014.
Methods:
KIM uses current and prior 2D x‐ray images to estimate the 3D target position during cancer radiotherapy treatment delivery. KIM software was written to process kilovoltage (kV) images streamed from a standard C‐arm linear accelerator with a gantry‐mounted kV x‐ray imaging system. A 120° pretreatment kV imaging arc was acquired to build the patient‐specific 2D to 3D motion correlation. The kV imager was activated during the megavoltage (MV) treatment, a dual arc VMAT prostate treatment, to estimate the 3D prostate position in real‐time. All necessary ethics, legal, and regulatory requirements were met for this clinical study. The quality assurance processes were completed and peer reviewed.
Results:
During treatment, a prostate position offset of nearly 3 mm in the posterior direction was observed with KIM. This position offset did not trigger a gating event. After the treatment, the prostate motion was independently measured using kV/MV triangulation, resulting in a mean difference of less than 0.6 mm and standard deviation of less than 0.6 mm in each direction. The accuracy of the marker segmentation was visually assessed during and after treatment and found to be performing well. During treatment, there were no interruptions due to performance of the KIM software.
Conclusions:
For the first time, KIM has been used for real‐time image guidance during cancer radiotherapy. The measured accuracy and precision were both submillimeter for the first treatment fraction. This clinical translational research milestone paves the way for the broad implementation of real‐time image guidance to facilitate the detection and correction of geometric and dosimetric errors, and resultant improved clinical outcomes, in cancer radiotherapy.
PDF
