This work proposes a novel privacy-preserving neural network feature representation to suppress the sensitive information of a learned space while maintaining the utility of the data. The new ...international regulation for personal data protection forces data controllers to guarantee privacy and avoid discriminative hazards while managing sensitive data of users. In our approach, privacy and discrimination are related to each other. Instead of existing approaches aimed directly at fairness improvement, the proposed feature representation enforces the privacy of selected attributes. This way fairness is not the objective, but the result of a privacy-preserving learning method. This approach guarantees that sensitive information cannot be exploited by any agent who process the output of the model, ensuring both privacy and equality of opportunity. Our method is based on an adversarial regularizer that introduces a sensitive information removal function in the learning objective. The method is evaluated on three different primary tasks (identity, attractiveness, and smiling) and three publicly available benchmarks. In addition, we present a new face annotation dataset with balanced distribution between genders and ethnic origins. The experiments demonstrate that it is possible to improve the privacy and equality of opportunity while retaining competitive performance independently of the task.
•A new model, called Local-DNN, is proposed for the gender recognition problem.•The model is based on local features and deep neural networks.•The local contributions are combined in a voting scheme ...for the final classification.•The model obtains state-of-the-art results in two wild face image datasets.
Display omitted
Deep learning methods are able to automatically discover better representations of the data to improve the performance of the classifiers. However, in computer vision tasks, such as the gender recognition problem, sometimes it is difficult to directly learn from the entire image. In this work we propose a new model called Local Deep Neural Network (Local-DNN), which is based on two key concepts: local features and deep architectures. The model learns from small overlapping regions in the visual field using discriminative feed-forward networks with several layers. We evaluate our approach on two well-known gender benchmarks, showing that our Local-DNN outperforms other deep learning methods also evaluated and obtains state-of-the-art results in both benchmarks.
Face recognition is one of the most active research fields of computer vision and pattern recognition, with many practical and commercial applications including identification, access control, ...forensics, and human-computer interactions. However, identifying a face in a crowd raises serious questions about individual freedoms and poses ethical issues. Significant methods, algorithms, approaches, and databases have been proposed over recent years to study constrained and unconstrained face recognition. 2D approaches reached some degree of maturity and reported very high rates of recognition. This performance is achieved in controlled environments where the acquisition parameters are controlled, such as lighting, angle of view, and distance between the camera–subject. However, if the ambient conditions (e.g., lighting) or the facial appearance (e.g., pose or facial expression) change, this performance will degrade dramatically. 3D approaches were proposed as an alternative solution to the problems mentioned above. The advantage of 3D data lies in its invariance to pose and lighting conditions, which has enhanced recognition systems efficiency. 3D data, however, is somewhat sensitive to changes in facial expressions. This review presents the history of face recognition technology, the current state-of-the-art methodologies, and future directions. We specifically concentrate on the most recent databases, 2D and 3D face recognition methods. Besides, we pay particular attention to deep learning approach as it presents the actuality in this field. Open issues are examined and potential directions for research in facial recognition are proposed in order to provide the reader with a point of reference for topics that deserve consideration.
In this paper, we propose FrankenMask, a novel framework that allows swapping and rearranging face parts in semantic masks for automatic editing of shape-related facial attributes. This is a novel ...yet challenging task as substituting face parts in a semantic mask requires to account for possible spatial misalignment and the adaptation of surrounding regions. We obtain such a feature by combining a Transformer encoder to learn the spatial relationships of facial parts, with an encoder–decoder architecture, which reconstructs a complete mask from the composition of local parts. Reconstruction and attribute classification results demonstrate the effective synthesis of facial images, while showing the generation of accurate and plausible facial attributes. Code is available at https://github.com/TFonta/FrankenMask_semantic.
•A model capable of automatic face parts re-arrangement without alignment constraints.•An alternative way of disentangling face parts for semantic image synthesis.•The possibility of enriching SOTA methods for automatic mask-to-RGB face synthesis.
Face-based recognition methods usually need the image of the whole face to perform, but in some situations, only a fraction of the face is visible, for example wearing sunglasses or recently with the ...COVID pandemic we had to wear facial masks. In this work, we propose a network architecture made up of four deep learning streams that process each one a different face element, namely: mouth, nose, eyes, and eyebrows, followed by a feature merge layer. Therefore, the face is segmented into the part of interest by means of ROI masks to keep the same input size for the four network streams. The aim is to assess the capacity of different combinations of face elements in recognizing the subject. The experiments were carried out on the Masked Face Recognition Database (M2FRED) which includes videos of 46 participants. The obtained results are 96% of recognition accuracy considering the four face elements; and 92%, 87%, and 63% of accuracy for the best combination of three, two, and one face elements respectively.
•We propose a DL-based part by part analysis of the face.•We apply a multi-input CNN to recognize the faces.•We consider four different face parts: eye, eyebrows, mouth, and nose.•We conduct an ablation study to infer the relevance of each part in face recognition task.
•We propose a Siamese inference network based on contrastive learning for face completion. It helps to improve the robustness and accuracy of representation learning for complex mask patterns.•We ...propose a novel dual attention fusion module that can explore feature interdependencies in spatial and channel dimensions and blend features in missing regions and known regions naturally. Smooth contents with rich texture information can be naturally synthesized.•To keep structural information of the input intact, the dense correspondence field that binds 2D and 3D surface spaces is estimated in our network, which can preserve the expression and pose of the input.•Our proposed method achieves smooth inpainting results with rich texture and reasonable topological structural information on three standard datasets against state-of-the-art methods, and also greatly improves the performance of face verification.
Most modern face completion approaches adopt an autoencoder or its variants to restore missing regions in face images. Encoders are often utilized to learn powerful representations that play an important role in meeting the challenges of sophisticated learning tasks. Specifically, various kinds of masks are often presented in face images in the wild, forming complex patterns, especially in this hard period of COVID-19. It’s difficult for encoders to capture such powerful representations under this complex situation. To address this challenge, we propose a self-supervised Siamese inference network to improve the generalization and robustness of encoders. It can encode contextual semantics from full-resolution images and obtain more discriminative representations. To deal with geometric variations of face images, a dense correspondence field is integrated into the network. We further propose a multi-scale decoder with a novel dual attention fusion module (DAF), which can combine the restored and known regions in an adaptive manner. This multi-scale architecture is beneficial for the decoder to utilize discriminative representations learned from encoders into images. Extensive experiments clearly demonstrate that the proposed approach not only achieves more appealing results compared with state-of-the-art methods but also improves the performance of masked face recognition dramatically.
Age from Faces in the Deep Learning Revolution Carletti, Vincenzo; Greco, Antonio; Percannella, Gennaro ...
IEEE transactions on pattern analysis and machine intelligence,
09/2020, Volume:
42, Issue:
9
Journal Article
Peer reviewed
Face analysis includes a variety of specific problems as face detection, person identification, gender and ethnicity recognition, just to name the most common ones; in the last two decades, ...significant research efforts have been devoted to the challenging task of age estimation from faces, as witnessed by the high number of published papers. The explosion of the deep learning paradigm, that is determining a spectacular increasing of the performance, is in the public eye; consequently, the number of approaches based on deep learning is impressively growing and this also happened for age estimation. The exciting results obtained have been recently surveyed on almost all the specific face analysis problems; the only exception stands for age estimation, whose last survey dates back to 2010 and does not include any deep learning based approach to the problem. This paper provides an analysis of the deep methods proposed in the last six years; these are analysed from different points of view: the network architecture together with the learning procedure, the used datasets, data preprocessing and augmentation, and the exploitation of additional data coming from gender, race and face expression. The review is completed by discussing the results obtained on public datasets, so as the impact of different aspects on system performance, together with still open issues.
In this paper, we present a novel single shot face-related task analysis method, called Face-SSD, for detecting faces and for performing various face-related (classification/regression) tasks ...including smile recognition, face attribute prediction and valence-arousal estimation in the wild. Face-SSD uses a Fully Convolutional Neural Network (FCNN) to detect multiple faces of different sizes and recognise/regress one or more face-related classes. Face-SSD has two parallel branches that share the same low-level filters, one branch dealing with face detection and the other one with face analysis tasks. The outputs of both branches are spatially aligned heatmaps that are produced in parallel—therefore Face-SSD does not require that face detection, facial region extraction, size normalisation, and facial region processing are performed in subsequent steps. Our contributions are threefold: 1) Face-SSD is the first network to perform face analysis without relying on pre-processing such as face detection and registration in advance–Face-SSD is a simple and a single FCNN architecture simultaneously performing face detection and face-related task analysis—those are conventionally treated as separate consecutive tasks; 2) Face-SSD is a generalised architecture that is applicable for various face analysis tasks without modifying the network structure—this is in contrast to designing task-specific architectures; and 3) Face-SSD achieves real-time performance (21 FPS) even when detecting multiple faces and recognising multiple classes in a given image (300 × 300). Experimental results show that Face-SSD achieves state-of-the-art performance in various face analysis tasks by reaching a recognition accuracy of 95.76% for smile detection, 90.29% for attribute prediction, and Root Mean Square (RMS) error of 0.44 and 0.39 for valence and arousal estimation.
•Face-SSD does not rely on a pre-normalisation step such as face detection and cropping.•Face-SSD is a generic architecture that can be utilised for many face analysis tasks.•Face-SSD provides real-time performance for a number of face-related applications.•We evaluate and analyse the best combination of data augmentation methods for each application.•We demonstrate several example applications of face analysis using the proposed Face-SSD.
•New method for Herpes zoster early detection by using Deep Learning techniques.•Low-cost devices and process allows an effectiveness of 97% with image analysis.•Neural networks for analysis of ...images in clinical medicine.
Herpes zoster virus (HZV) or varicella-zoster virus (VZV) affects the trigeminal nerve, at the earliest possible stage will avoid the eyes injuries. In this paper, the new framework develops a new method with convolutional neural networks (CNN), the detection for the early stage of the HZV is tested with 1,000 images. It is 89.6% with low-cost image analysis, besides, the database has been analyzed with other architectures in order to validate the most appropriate algorithm. The process is pre-processing, segmentation, extraction, and classification. The VZV produces two illness: i) Varicella called chickenpox, and ii) Herpes Zoster. In order to obtain a machine learning process, it considers building blocks of convolutional layer neural network associated to a new process for early Herpes Zoster (HZ) disease detection system, structured in four stages as pre-processing, segmentation, extraction and classification. In particular, the new process includes a classification process with a comparison between the K-Nearest Neighborhood (KNN), artificial neural networks (ANN), and logistic model tree (LMT) regression for the comparison. The effectiveness during eight days is 98.1%, for early detection with minimal information. However, the training process produces 33% false positives and an average 90% true positive rate. Early HZ detection and the failures associated with electronic devices were shown and used for facial and pattern recognition associated with nerve location. With this research, the difficulties concerning to the data management and deep learning were corroborated during eight days of the illness, to better understand the process and technology that enable a successful classification.
The investigation of influences in artists’ works has been a subject of interest for art historians for many years. Therefore, computational methods can provide a new perspective for identifying ...these influences’ relationships. Indeed, several studies in computer science have proposed techniques to analyze similarities between paintings using various features. Faces are a crucial aspect of perception in art and have also been the focus of several studies in computational aesthetics. In our previous work, we proposed a method for analyzing artworks and evaluating the influence of artists. The present study improves upon the previous research by extending the analysis of influences considering second-degree influences between artists and the impact of geographic proximity, obtaining better results in terms of Recall than the previous work. In addition, we evaluated the capability of our method to detect work-to-work relationships between each pair of artworks by the artists, and we found plausible and interesting results, even though they have not yet been proven in the literature. By conducting further analysis of data extracted from the faces of works of art, the goal is to enhance the previous findings in the literature and foster further discussion and collaboration between the fields of art and computer science. The objective is not to provide a definitive answer to the question of influences but to stimulate further research in this area, pointing out new possibilities of influence and explanations about these influences.
Display omitted
•Methodology to suggest influences between artists based on the faces of paintings.•Uses groups of visual features to perform the analyses.•Proposes a way to identify influences using second-degree relationships and location.•Presents a work-by-work comparison in order to evaluate the method.
PDF
