•Average error is bellow 1 mm for deformation amplitudes 1000 times larger.•Computation time bellow 4 ms for the deformed configuration of the static problem.•The method can be applied to any linear ...or non-linear elastic model.•The method outperforms Proper Orthogonal Decomposition techniques.
Display omitted
The finite element method (FEM) is among the most commonly used numerical methods for solving engineering problems. Due to its computational cost, various ideas have been introduced to reduce computation times, such as domain decomposition, parallel computing, adaptive meshing, and model order reduction. In this paper we present U-Mesh: A data-driven method based on a U-Net architecture that approximates the non-linear relation between a contact force and the displacement field computed by a FEM algorithm. We show that deep learning, one of the latest machine learning methods based on artificial neural networks, can enhance computational mechanics through its ability to encode highly non-linear models in a compact form. Our method is applied to three benchmark examples: a cantilever beam, an L-shape and a liver model subject to moving punctual loads. A comparison between our method and proper orthogonal decomposition (POD) is done through the paper. The results show that U-Mesh can perform very fast simulations on various geometries and topologies, mesh resolutions and number of input forces with very small errors.
Introduction
Intraoperative navigation during liver resection remains difficult and requires high radiologic skills because liver anatomy is complex and patient-specific. Augmented reality (AR) ...during open liver surgery could be helpful to guide hepatectomies and optimize resection margins but faces many challenges when large parenchymal deformations take place. We aimed to experiment a new vision-based AR to assess its clinical feasibility and anatomical accuracy.
Patients and Methods
Based on preoperative CT scan 3-D segmentations, we applied a non-rigid registration method, integrating a physics-based elastic model of the liver, computed in real time using an efficient finite element method. To fit the actual deformations, the model was driven by data provided by a single RGB-D camera. Five livers were considered in this experiment. In vivo AR was performed during hepatectomy (
n
= 4), with manual handling of the livers resulting in large realistic deformations. Ex vivo experiment (
n
= 1) consisted in repeated CT scans of explanted whole organ carrying internal metallic landmarks, in fixed deformations, and allowed us to analyze our estimated deformations and quantify spatial errors.
Results
In vivo AR tests were successfully achieved in all patients with a fast and agile setup installation (< 10 min) and real-time overlay of the virtual anatomy onto the surgical field displayed on an external screen. In addition, an ex vivo quantification demonstrated a 7.9 mm root mean square error for the registration of internal landmarks.
Conclusion
These first experiments of a markerless AR provided promising results, requiring very little equipment and setup time, yet providing real-time AR with satisfactory 3D accuracy. These results must be confirmed in a larger prospective study to definitively assess the impact of such minimally invasive technology on pathological margins and oncological outcomes.
Real‐time simulation of elastic structures is essential in many applications, from computer‐guided surgical interventions to interactive design in mechanical engineering. The finite element method is ...often used as the numerical method of reference for solving the partial differential equations associated with these problems. Deep learning methods have recently shown that they could represent an alternative strategy to solve physics‐based problems. In this article, we propose a solution to simulate hyper‐elastic materials using a data‐driven approach, where a neural network is trained to learn the nonlinear relationship between boundary conditions and the resulting displacement field. We also introduce a method to guarantee the validity of the solution. In total, we present three contributions: an optimized data set generation algorithm based on modal analysis, a physics‐informed loss function, and a hybrid Newton–Raphson algorithm. The method is applied to two benchmarks: a cantilever beam and a propeller. The results show that our network architecture trained with a limited amount of data can predict the displacement field in less than a millisecond. The predictions on various geometries, topologies, mesh resolutions, and boundary conditions are accurate to a few micrometers for nonlinear deformations of several centimeters of amplitude.
Purpose:
Augmented reality can improve the outcome of hepatic surgeries, assuming an accurate liver model is available to estimate the position of internal structures. While researchers have proposed ...patient-specific liver simulations, very few have addressed the question of boundary conditions. Resulting mainly from ligaments attached to the liver, they are not visible in preoperative images, yet play a key role in the computation of the deformation.
Method:
We propose to estimate both the location and stiffness of ligaments by using a combination of a statistical atlas, numerical simulation, and Bayesian inference. Ligaments are modeled as polynomial springs connected to a liver finite element model. They are initialized using an anatomical atlas and stiffness properties taken from the literature. These characteristics are then corrected using a reduced-order unscented Kalman filter based on observations taken from the laparoscopic image stream.
Results:
Our approach is evaluated using synthetic data and phantom data. By relying on a simplified representation of the ligaments to speed up computation times, it is not estimating the true characteristics of ligaments. However, results show that our estimation of the boundary conditions still improves the accuracy of the simulation by 75% when compared to typical methods involving Dirichlet boundary conditions.
Conclusion:
By estimating patient-specific boundary conditions, using tracked liver motion from RGB-D data, our approach significantly improves the accuracy of the liver model. The method inherently handles noisy observations, a substantial feature in the context of augmented reality.
During the minimally-invasive liver surgery, only the partial surface view of the liver is usually provided to the surgeon
via
the laparoscopic camera. Therefore, it is necessary to estimate the ...actual position of the internal structures such as tumors and vessels from the pre-operative images. Nevertheless, such task can be highly challenging since during the intervention, the abdominal organs undergo important deformations due to the pneumoperitoneum, respiratory and cardiac motion and the interaction with the surgical tools. Therefore, a reliable automatic system for intra-operative guidance requires fast and reliable registration of the pre- and intra-operative data. In this paper we present a complete pipeline for the registration of pre-operative patient-specific image data to the sparse and incomplete intra-operative data. While the intra-operative data is represented by a point cloud extracted from the stereo-endoscopic images, the pre-operative data is used to reconstruct a biomechanical model which is necessary for accurate estimation of the position of the internal structures, considering the actual deformations. This model takes into account the patient-specific liver anatomy composed of parenchyma, vascularization and capsule, and is enriched with anatomical boundary conditions transferred from an atlas. The registration process employs the iterative closest point technique together with a penalty-based method. We perform a quantitative assessment based on the evaluation of the target registration error on synthetic data as well as a qualitative assessment on real patient data. We demonstrate that the proposed registration method provides good results in terms of both accuracy and robustness w.r.t. the quality of the intra-operative data.
Display omitted
•Real-time simulation of deformable structures in contact with their environment.•Implicit time integration for heterogeneous structures, haptic feedback and cutting.•Asynchronous ...preconditioner to solve the deformation and contact problem.•GPU parallelization for real-time computations.•Application to cataract surgery, hepatectomy in laparoscopy, and brain surgery.
This paper presents a numerical method for interactive (real-time) simulations, which considerably improves the accuracy of the response of heterogeneous soft-tissue models undergoing contact, cutting and other topological changes. We provide an integrated methodology able to deal both with the ill-conditioning issues associated with material heterogeneities, contact boundary conditions which are one of the main sources of inaccuracies, and cutting which is one of the most challenging issues in interactive simulations. Our approach is based on an implicit time integration of a non-linear finite element model. To enable real-time computations, we propose a new preconditioning technique, based on an asynchronous update at low frequency. The preconditioner is not only used to improve the computation of the deformation of the tissues, but also to simulate the contact response of homogeneous and heterogeneous bodies with the same accuracy. We also address the problem of cutting the heterogeneous structures and propose a method to update the preconditioner according to the topological modifications. Finally, we apply our approach to three challenging demonstrators: (i) a simulation of cataract surgery (ii) a simulation of laparoscopic hepatectomy (iii) a brain tumor surgery.
Purpose
The treatment of cardiovascular diseases requires complex and challenging navigation of a guidewire and catheter. This often leads to lengthy interventions during which the patient and ...clinician are exposed to X-ray radiation. Deep reinforcement learning approaches have shown promise in learning this task and may be the key to automating catheter navigation during robotized interventions. Yet, existing training methods show limited capabilities at generalizing to unseen vascular anatomies, requiring to be retrained each time the geometry changes.
Methods
In this paper, we propose a zero-shot learning strategy for three-dimensional autonomous endovascular navigation. Using a very small training set of branching patterns, our reinforcement learning algorithm is able to learn a control that can then be applied to unseen vascular anatomies without retraining.
Results
We demonstrate our method on 4 different vascular systems, with an average success rate of 95% at reaching random targets on these anatomies. Our strategy is also computationally efficient, allowing the training of our controller to be performed in only 2 h.
Conclusion
Our training method proved its ability to navigate unseen geometries with different characteristics, thanks to a nearly shape-invariant observation space.
•A general approach for the stochastic finite element analysis of soft tissue deformation is presented.•Sensitivity derivative Monte Carlo method to propagate uncertainty through hyperelastic models ...with stochastic parameters.•The proposed method is up to two orders of magnitude faster than the standard Monte Carlo approach.•The method is able to provide the user with statistical results on quantities of practical interest.•We applied the approach to a simple academic example and to the stochastic deformation of a brain.
We present a simple open-source semi-intrusive computational method to propagate uncertainties through hyperelastic models of soft tissues. The proposed method is up to two orders of magnitude faster than the standard Monte Carlo method. The material model of interest can be altered by adjusting few lines of (FEniCS) code. The method is able to (1) provide the user with statistical confidence intervals on quantities of practical interest, such as the displacement of a tumour or target site in an organ; (2) quantify the sensitivity of the response of the organ to the associated parameters of the material model. We exercise the approach on the determination of a confidence interval on the motion of a target in the brain. We also show that for the boundary conditions under consideration five parameters of the Ogden–Holzapfel-like model have negligible influence on the displacement of the target zone compared to the three most influential parameters. The benchmark problems and all associated data are made available as supplementary material.
PDF
