Measuring the 3D morphology of spherical cell aggregates is required in both biology and medicine. Traditional methods either use fluorescent labeling, which cause cell toxicity and are unsuitable ...for clinical treatment, or use 2D images to roughly estimate 3D morphology. To overcome these limitations, this paper presents a quantitative label-free 3D morphology measurement technique using multi-view images. This technique, for the first time, enables the morphological evaluation of a blastocyst (Day 5 embryo) from "all angles" for IVF treatment. In this technique, a spherical rotation scale invariant feature transform (SR-SIFT) is proposed to address feature distortions for the rotation matrix calculation of the multi-view images. U-Net with generalized Dice loss is used to segment individual trophectoderm (TE) cells and the inner cell mass (ICM) of the blastocyst. Based on the rotation matrices and the segmentation results, the 3D morphological parameters of the entire blastocyst were quantified. Experimental results showed that the error of rotation angle was less than 1<inline-formula><tex-math notation="LaTeX">^\circ</tex-math></inline-formula>, the Dice was 95.6% for TE segmentation and 92.3% for ICM segmentation, and the overall measurement error of clinically defined blastocyst parameters was less than 6.7%.
Proper alignment of the end-effector is a critical procedure that determines the success of micromanipulation, such as robotic cell manipulation. Presently, end-effector alignment is performed ...manually and suffers from large misalignment error and inconsistency. Manual alignment often undesirably moves the end-effector (e.g., a glass micropipette) out of the limited field of view under microscopy and risks breaking the fragile end-effector. This article presents automated end-effector alignment in robotic micromanipulation. A rotational degree of freedom was added to a standard micromanipulator with translational degrees of freedom. The kinematic model of end-effector's rotation was established, and the unknown model parameters were calibrated. To accommodate model uncertainty and parameter variations, a sliding mode controller was designed to achieve end-effector alignment. Experimental results demonstrate that the robotic alignment technique achieved an accuracy of 0.5<inline-formula><tex-math notation="LaTeX">\pm 0.3^{\circ }</tex-math></inline-formula> and a time cost of 17.9<inline-formula><tex-math notation="LaTeX">\pm</tex-math></inline-formula>7.3 s, both significantly less than manual alignment. The developed controller based on kinematic modeling and sliding mode control achieved a higher success rate and significantly less time cost for end-effector alignment than the PID controller. Standard micropipettes were used as the end-effectors for sperm immobilization and oocyte penetration, important procedures in cell surgeries. The success rate of sperm immobilization was 98% by robotic micropipette alignment, higher than the success rate of 90% by manual alignment. Oocyte deformation before penetration was 28.1<inline-formula><tex-math notation="LaTeX">\pm</tex-math></inline-formula>7.5 <inline-formula><tex-math notation="LaTeX">\mu</tex-math></inline-formula>m by robotic end-effector alignment, significantly less than the deformation of 54.5<inline-formula><tex-math notation="LaTeX">\pm</tex-math></inline-formula>13.2 <inline-formula><tex-math notation="LaTeX">\mu</tex-math></inline-formula>m by manual alignment.
Robotic manipulation of deformable objects has been a classic topic in robotics. Compared to synthetic deformable objects such as rubber balls and clothes, biological cells are highly deformable and ...more prone to damage. This article presents robotic manipulation of deformable cells for orientation control (both out-of-plane and in-plane), which is required in both clinical (e.g., in vitro fertilization) and biomedical (e.g., clone) applications. Compared to manual cell orientation control based on empirical experience, the robotic approach, based on modeling and path planning, effectively rotates a cell, while consistently maintaining minimal cell deformation to avoid cell damage. A force model is established to determine the minimal force applied by the micropipette to rotate a spherical or, more generally, ellipsoidal oocyte. The force information is translated into indentation through a contact mechanics model, and the manipulation path of the micropipette is formed by connecting the indentation positions on the oocyte. An optimal controller is designed to compensate for the variations of mechanical properties across oocytes. The polar body of an oocyte is detected by deep neural networks with robustness to shape and size differences. In experiments, the system achieved an accuracy of 97.6% in polar body detection and an accuracy of 0.7° in oocyte orientation control with maximum oocyte deformation of 2.70 μm throughout the orientation control process.
Infertility affects one in six couples worldwide, and fertility continues to deteriorate globally, partly owing to a decline in semen quality. Sperm analysis has a central role in diagnosing and ...treating male factor infertility. Many emerging techniques, such as digital holography, super-resolution microscopy and next-generation sequencing, have been developed that enable improved analysis of sperm motility, morphology and genetics to help overcome limitations in accuracy and consistency, and improve sperm selection for infertility treatment. These techniques have also improved our understanding of fundamental sperm physiology by enabling discoveries in sperm behaviour and molecular structures. Further progress in sperm analysis and integrating these techniques into laboratories and clinics requires multidisciplinary collaboration, which will increase discovery and improve clinical outcomes.
Full text
Available for:
GEOZS, IJS, IMTLJ, KISLJ, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBMB, UL, UM, UPUK, ZAGLJ
In infertility treatment, blastocyst morphological grading is commonly used in clinical practice for blastocyst evaluation and selection, but has shown limited predictive power on live birth outcomes ...of blastocysts. To improve live birth prediction, a number of artificial intelligence (AI) models have been established. Most existing AI models for blastocyst evaluation only used images for live birth prediction, and the area under the receiver operating characteristic (ROC) curve (AUC) achieved by these models has plateaued at ~0.65.
This study proposed a multimodal blastocyst evaluation method using both blastocyst images and patient couple's clinical features (e.g., maternal age, hormone profiles, endometrium thickness, and semen quality) to predict live birth outcomes of human blastocysts. To utilize the multimodal data, we developed a new AI model consisting of a convolutional neural network (CNN) to process blastocyst images and a multilayer perceptron to process patient couple's clinical features. The data set used in this study consists of 17,580 blastocysts with known live birth outcomes, blastocyst images, and patient couple's clinical features.
This study achieved an AUC of 0.77 for live birth prediction, which significantly outperforms related works in the literature. Sixteen out of 103 clinical features were identified to be predictors of live birth outcomes and helped improve live birth prediction. Among these features, maternal age, the day of blastocyst transfer, antral follicle count, retrieved oocyte number, and endometrium thickness measured before transfer are the top five features contributing to live birth prediction. Heatmaps showed that the CNN in the AI model mainly focuses on image regions of inner cell mass and trophectoderm (TE) for live birth prediction, and the contribution of TE-related features was greater in the CNN trained with the inclusion of patient couple's clinical features compared with the CNN trained with blastocyst images alone.
The results suggest that the inclusion of patient couple's clinical features along with blastocyst images increases live birth prediction accuracy.
Natural Sciences and Engineering Research Council of Canada and the Canada Research Chairs Program.
Deep learning has been increasingly investigated for assisting clinical in vitro fertilization (IVF). The first technical step in many tasks is to visually detect and locate sperm, oocytes, and ...embryos in images. For clinical deployment of such deep learning models, different clinics use different image acquisition hardware and different sample preprocessing protocols, raising the concern over whether the reported accuracy of a deep learning model by one clinic could be reproduced in another clinic. Here we aim to investigate the effect of each imaging factor on the generalizability of object detection models, using sperm analysis as a pilot example.
Ablation studies were performed using state-of-the-art models for detecting human sperm to quantitatively assess how model precision (false-positive detection) and recall (missed detection) were affected by imaging magnification, imaging mode, and sample preprocessing protocols. The results led to the hypothesis that the richness of image acquisition conditions in a training dataset deterministically affects model generalizability. The hypothesis was tested by first enriching the training dataset with a wide range of imaging conditions, then validated through internal blind tests on new samples and external multi-center clinical validations.
Ablation experiments revealed that removing subsets of data from the training dataset significantly reduced model precision. Removing raw sample images from the training dataset caused the largest drop in model precision, whereas removing 20x images caused the largest drop in model recall. by incorporating different imaging and sample preprocessing conditions into a rich training dataset, the model achieved an intraclass correlation coefficient (ICC) of 0.97 (95% CI: 0.94-0.99) for precision, and an ICC of 0.97 (95% CI: 0.93-0.99) for recall. Multi-center clinical validation showed no significant differences in model precision or recall across different clinics and applications.
The results validated the hypothesis that the richness of data in the training dataset is a key factor impacting model generalizability. These findings highlight the importance of diversity in a training dataset for model evaluation and suggest that future deep learning models in andrology and reproductive medicine should incorporate comprehensive feature sets for enhanced generalizability across clinics.
Full text
Available for:
DOBA, IZUM, KILJ, NUK, PILJ, PNG, SAZU, SIK, UILJ, UKNU, UL, UM, UPUK
Background
Automated sperm analysis has wide applications in infertility diagnosis. Existing systems are not able to measure sperm count and both motility and morphology of individual live ...spermatozoa. Morphology measurement requires invasive staining, making the spermatozoa after morphology measurement not applicable to infertility treatment.
Objective
To evaluate the reproducibility and reliability of automated measurement of individual live sperm's motility and morphology.
Materials and methods
Fresh semen samples were obtained from twenty male partners attending for fertility investigations. The system firstly measured motility for all the spermatozoa within the field of view under a low magnification (20×), then a spermatozoa of interest is selected by the user and automatically relocated by the system after switching to a high magnification (100×) for morphology measurement. Reproducibility of sperm measurements was evaluated by intraclass correlation coefficients on consecutive measurement. Reliability of motility and morphology measurement was evaluated by tracking error rate and limits of agreement, respectively, with manual measurement as benchmark.
Results
Measurement of all motility and morphology parameters had intraclass correlation coefficients higher than 0.94. Sperm motility measurement had a tracking error rate of 2.1%. Limit of agreement analysis indicated that automated measurement and manual measurement of sperm morphology were interchangeable. Automated measurement of all morphology parameters was not statistically different from manual measurement, as confirmed by the paired sample t test.
Discussion
Automated motility and morphology measurement of single sperm revealed high reproducibility and reliability. The system also achieved a high efficiency for motility and morphology measurement. In addition to the intracytoplasmic sperm injection (ICSI) samples with polyvinylpyrrolidone (PVP), the developed sperm measurement technique is also effective for analyzing semen and washed samples. The system provides a valuable tool for quantitative measurement and selection of single spermatozoa for ICSI. It can also be used for sperm motility and morphology analysis in andrology laboratories.
Full text
Available for:
BFBNIB, DOBA, FZAB, GIS, IJS, IZUM, KILJ, NLZOH, NUK, OILJ, PILJ, PNG, SAZU, SBCE, SBMB, SIK, UILJ, UKNU, UL, UM, UPUK
Denudation is a technique for removal of the cumulus cell mass from oocytes in clinical intracytoplasmic sperm injection (ICSI). Manual oocyte denudation requires long training hours and stringent ...skills, but still suffers from low yield rate and denudation efficiency due to human fatigue and skill variations across operators. To address these limitations, this paper reports a robotic system for automated oocyte denudation. In this system, several key techniques are proposed, including a vision-based contact detection method for measuring the relative z position between the micropipette tip and the dish substrate, recognition of oocytes and the surrounding cumulus cells, automated calibration algorithm for eliminating the misalignment angle, and automated control of the flow rate based on the model of oocyte dynamics during micropipette aspiration and deposition. Experiments on mouse oocytes demonstrated that the robotic denudation system achieved a high yield rate of 97.0 ± 2.8% and denudation efficiency of 95.0 ± 0.8%. Additionally, oocytes denuded by the robotic system showed comparable fertilization rate and developmental competence compared with manual denudation. Our robotic denudation system represents one step towards the automation and standardization of ICSI procedures.
Cell penetration is an indispensable step in many cell surgery tasks. Conventionally, cell penetration is achieved by passively indenting and eventually puncturing the cell membrane, during which ...undesired large cell deformation is induced. Piezo drills have been developed to penetrate cells with less deformation. However, existing piezo drills suffer from large lateral vibration or are incompatible with standard clinical setup. Furthermore, it is challenging to accurately determine the time instance of cell membrane puncturing; thus, the time delay to stop piezo pulsing causes cytoplasm stirring and cell damage. This letter reports a new robotic piezo-driven cell penetration technique, in which the piezo drill device induces small lateral vibrations and is fully compatible with standard clinical setup. Techniques based on corner-feature probabilistic data association filter and motion history images were developed to automatically detect cell membrane breakage by piezo drilling. Experiments on hamster oocytes confirmed that the system is capable of achieving a small cell deformation of 5.68 ± 2.74 μm (vs. 54.29 ± 10.21 μm by passive approach) during cell penetration. Automated detection of membrane breakage had a success rate of 95.0%, and the time delay between membrane breakage and piezo-vibration stoppage was 0.51 ± 0.27 s vs. 2.32 ± 0.98 s by manual stoppage of piezo pulsing. This reduced time delay together with smaller cell deformation led to higher oocyte post-penetration survival rate (92.5% vs. 77.5% by passive approach, n = 80 cells).
PDF
