We present a customized 3D mesh Transformer model for the pose transfer task. As the 3D pose transfer essentially is a deformation procedure dependent on the given meshes, the intuition of this work ...is to perceive the geometric inconsistency between the given meshes with the powerful self-attention mechanism. Specifically, we propose a novel geometry-contrastive Transformer that has an efficient 3D structured perceiving ability to the global geometric inconsistencies across the given meshes. Moreover, locally, a simple yet efficient central geodesic contrastive loss is further proposed to improve the regional geometric-inconsistency learning. At last, we present a latent isometric regularization module together with a novel semi-synthesized dataset for the cross-dataset 3D pose transfer task towards unknown spaces. The massive experimental results prove the efficacy of our approach by showing state-of-the-art quantitative performances on SMPL-NPT, FAUST and our new proposed dataset SMG-3D datasets, as well as promising qualitative results on MG-cloth and SMAL datasets. It's demonstrated that our method can achieve robust 3D pose transfer and be generalized to challenging meshes from unknown spaces on cross-dataset tasks. The code and dataset are made available. Code is available: https://github.com/mikecheninoulu/CGT.
Action Unit (AU) detection plays an important role for facial expression recognition. To the best of our knowledge, there is little research about AU analysis for micro-expressions. In this paper, we ...focus on AU detection in micro-expressions. Microexpression AU detection is challenging due to the small quantity of micro-expression databases, low intensity, short duration of facial muscle change, and class imbalance. In order to alleviate the problems, we propose a novel Spatio-Temporal Adaptive Pooling (STAP) network for AU detection in micro-expressions. Firstly, STAP is aggregated by a series of convolutional filters of different sizes. In this way, STAP can obtain multi-scale information on spatial and temporal domains. On the other hand, STAP contains less parameters, thus it has less computational cost and is suitable for micro-expression AU detection on very small databases. Furthermore, STAP module is designed to pool discriminative information for micro-expression AUs on spatial and temporal domains.Finally, Focal loss is employed to prevent the vast number of negatives from overwhelming the microexpression AU detector. In experiments, we firstly polish the AU annotations on three commonly used databases. We conduct intensive experiments on three micro-expression databases, and provide several baseline results on micro-expression AU detection. The results show that our proposed approach outperforms the basic Inflated inception-v1 (I3D) in terms of an average of F1- score. We also evaluate the performance of our proposed method on cross-database protocol. It demonstrates that our proposed approach is feasible for cross-database micro-expression AU detection. Importantly, the results on three micro-expression databases and cross-database protocol provide extensive baseline results for future research on micro-expression AU detection.
Texture is an important characteristic for many types of images. In recent years very discriminative and computationally efficient local texture descriptors based on local binary patterns (LBP) have ...been developed, which has led to significant progress in applying texture methods to different problems and applications. Due to this progress, the division between texture descriptors and more generic image or video descriptors has been disappearing. A large number of different variants of LBP have been developed to improve its robustness, and to increase its discriminative power and applicability to different types of problems. In this chapter, the most recent and important variants of LBP in 2D, spatiotemporal, 3D, and 4D domains are surveyed. Interesting new developments of LBP in 1D signal analysis are also considered. Finally, some future challenges for research are presented.
Remote photoplethysmography (rPPG), which aims at measuring heart activities and physiological signals from facial video without any contact, has great potential in many applications (e.g., remote ...healthcare and affective computing). Recent deep learning approaches focus on mining subtle rPPG clues using convolutional neural networks with limited spatio-temporal receptive fields, which neglect the long-range spatio-temporal perception and interaction for rPPG modeling. In this paper, we propose two end-to-end video transformer based architectures, namely PhysFormer and PhysFormer++, to adaptively aggregate both local and global spatio-temporal features for rPPG representation enhancement. As key modules in PhysFormer, the temporal difference transformers first enhance the quasi-periodic rPPG features with temporal difference guided global attention, and then refine the local spatio-temporal representation against interference. To better exploit the temporal contextual and periodic rPPG clues, we also extend the PhysFormer to the two-pathway SlowFast based PhysFormer++ with temporal difference periodic and cross-attention transformers. Furthermore, we propose the label distribution learning and a curriculum learning inspired dynamic constraint in frequency domain, which provide elaborate supervisions for PhysFormer and PhysFormer++ and alleviate overfitting. Comprehensive experiments are performed on four benchmark datasets to show our superior performance on both intra- and cross-dataset testings. Unlike most transformer networks needed pretraining from large-scale datasets, the proposed PhysFormer family can be easily trained from scratch on rPPG datasets, which makes it promising as a novel transformer baseline for the rPPG community.
We present a novel task, i.e., animating a target 3D object through the motion of a raw driving sequence. In previous works, extra auxiliary correlations between source and target meshes or ...intermedia factors are inevitable to capture the motions in the driving sequences. Instead, we introduce AniFormer, a novel Transformer-based architecture, that generates animated 3D sequences by directly taking the raw driving sequences and arbitrary same-type target meshes as inputs. Specifically, we customize the Transformer architecture for 3D animation that generates mesh sequences by integrating styles from target meshes and motions from the driving meshes. Besides, instead of the conventional single regression head in the vanilla Transformer, AniFormer generates multiple frames as outputs to preserve the sequential consistency of the generated meshes. To achieve this, we carefully design a pair of regression constraints, i.e., motion and appearance constraints, that can provide strong regularization on the generated mesh sequences. Our AniFormer achieves high-fidelity, realistic, temporally coherent animated results and outperforms compared start-of-the-art methods on benchmarks of diverse categories. Code is available: https://github.com/mikecheninoulu/AniFormer.
Abstract Background aims Mucopolysaccharidosis type IIIA (MPS IIIA) is a lysosomal storage disorder (LSD) in which an absence of sulfamidase results in incomplete degradation and subsequent ...accumulation of its substrate, heparan sulfate. Most neurodegenerative LSD remain untreatable. However, therapy options, such as gene, enzyme end cell therapy, are under investigation. Previously, we have constructed an embryonic stem (ES) cell line (NS21) that over-expresses human sulphamidase as a potential treatment for murine MPS IIIA. Methods In the present study the sulfatase-modifying factor I (SUMF1) and enhanced green fluorescence protein (eGFP) genes were co-introduced under a cytomegalovirus (CMV) promoter into NS21 cells, to enhance further sulfamidase activity and provide a marker for in vivo cell tracking, respectively. eGFP was also introduced under the control of the human elongation factor-1α (hEF-1α) promoter to compare the stability of transgene expression. Results During differentiation of ES cells into glial precursors, SUMF1 was down-regulated and was hardly detectable by day 18 of differentiation. Likewise, eGFP expression was heterogeneous and highly unstable. Use of a human EF-1α promoter resulted in more homogeneous eGFP expression, with ∼ 50% of cells eGFP positive following differentiation into glial precursors. Compared with NS21 cells, the outgrowth of eGFP-expressing cells was not as confluent when differentiated into glial precursors. Conclusions Our data suggest that SUMF1 enhances sulfamidase activity in ES cells, hEF-1α is a stronger promoter than CMV for ES cells and over-expression of eGFP may affect cell growth and contribute to unstable gene expression.
Full text
Available for:
DOBA, GEOZS, IJS, IMTLJ, IZUM, KILJ, KISLJ, NUK, OILJ, PILJ, PNG, SAZU, SBCE, SBJE, UILJ, UKNU, UL, UM, UPCLJ, UPUK
Remote measurement of physiological signals from videos is an emerging topic. The topic draws great interests, but the lack of publicly available benchmark databases and a fair validation platform ...are hindering its further development. For this concern, we organize the first challenge on Remote Physiological Signal Sensing (RePSS), in which two databases of VIPL and OBF are provided as the benchmark for kin researchers to evaluate their approaches. The 1st challenge of RePSS focuses on measuring the average heart rate from facial videos, which is the basic problem of remote physiological measurement. This paper presents an overview of the challenge, including data, protocol, analysis of results and discussion. The top ranked solutions are highlighted to provide insights for researchers, and future directions are outlined for this topic and this challenge.
Flexible-Modal Face Anti-Spoofing: A Benchmark Yu, Zitong; Liu, Ajian; Zhao, Chenxu ...
2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW),
2023-June
Conference Proceeding
Face anti-spoofing (FAS) plays a vital role in securing face recognition systems from presentation attacks. Benefitted from the maturing camera sensors, single-modal (RGB) and multi-modal (e.g., ...RGB+Depth) FAS has been applied in various scenarios with different configurations of sensors/modalities. Existing single- and multi-modal FAS methods usually separately train and deploy models for each possible modality scenario, which might be redundant and inefficient. Can we train a unified model, and flexibly deploy it under various modality scenarios? In this paper, we establish the first flexible-modal FAS benchmark with the principle 'train one for all'. To be specific, with trained multi-modal (RGB+Depth+IR) FAS models, both intra- and cross-dataset testings are conducted on four flexible-modal sub-protocols (RGB, RGB+Depth, RGB+IR, and RGB+Depth+IR). We also investigate prevalent deep models and feature fusion strategies for flexible-modal FAS. We hope this new benchmark will facilitate the future research of the multi-modal FAS. The protocols and codes are available at https://github.com/ZitongYu/Flex-Modal-FAS.
The twitcher mouse is a model of human Krabbe's disease caused by a mutation in the galacto-cerebrosidase gene. As a result of deficient catabolism of myelin, death of oligodendrocytes and ...demyelination occur widely in the central and peripheral nervous system, making it an ideal model for investigation of myelin repair strategies. Here we describe the use of mouse neural stem cells (NSCs) expressing enhanced green fluorescence protein (eGFP) for transplantation in neonatal normal and twitcher mice. Normal and twitcher mice in all age groups (20, 30, and 45 days old) showed engraftment and differentiation of injected cells. The engrafted cells were found in the ventricles and a wide range of regions in the brain parenchyma. There was no significant difference in the total number of cells engrafted and the pattern of engraftment between 30-day-old normal and twitcher mice. The average number of engrafted cells in the brain of a 30-day-old mouse was 964 +/- 281 (n = 8). Engrafted cells with the morphology of neurons, astrocytes, and oligodendrocytes were identified. Differentiation into oligodendrocytes was confirmed by immunohistochemical staining using a cell-type-specific marker. There was a higher percentage of cells engrafted in the grey matter than in the white matter (p < 0.01) in both normal and twitcher mouse brain. This study indicates that the environment of demyelination in 30-day-old twitcher mouse brain has not significantly altered the engraftment and distribution patterns of NSCs after neonatal transplantation.
PDF
