Deep learning has recently gained popularity achieving state-of-the-art performance in tasks involving text, sound, or image processing. Due to its outstanding performance, there have been efforts to ...apply it in more challenging scenarios, for example, 3D data processing. This article surveys methods applying deep learning on 3D data and provides a classification based on how they exploit them. From the results of the examined works, we conclude that systems employing 2D views of 3D data typically surpass voxel-based (3D) deep models, which however, can perform better with more layers and severe data augmentation. Therefore, larger-scale datasets and increased resolutions are required.
In the literature, several 3D face datasets have been collected, aiming at advancing the field of 3D face analysis from different perspectives. Data collection generally follows specific research ...needs, and the existing 3D face datasets all have different characteristics that are tailored for investigating different tasks, encompassing face recognition, facial expressions and emotions analysis, 3D face reconstruction. However, the majority of these datasets are either collected with high-resolution scanners, or consumer level devices, such as the Kinect, the latter being motivated by the burdensome and costly process of collecting high-quality scans. Differently from 2D imagery, the difference in resolution in 3D data represents a non negligible problem that is under-investigated, and still prevents the successful development of methods that can work in real scenarios. In this paper, we propose a new 3D face dataset, named “Florence Multi-Resolution 3D Facial Expression” (Florence 3DMRE), which aims at bridging the gap between high- and low-resolution 3D face datasets. Its peculiarity consists in (1) including high-resolution (HR) models obtained with a HR scanner, and paired samples collected with a Kinect sensor, (2) LR and HR scans are synchronized and capture extreme and asymmetric facial deformations as used in facial rehabilitation exercises. In total, our dataset consists of 14 subjects, each performing 19 complex and asymmetric expressions. For each of them, we collected a high-resolution scan, and an RGB-D sequence. Finally, to highlight the value of the dataset and the challenges it introduces, we use the collected data to perform baseline experiments for cross-resolution 3D face recognition and reconstruction. The dataset is released for research purposes only, and complies to GDPR for data treatment. The dataset can be found at this link.
•A new 3D face dataset including synchronized pairs of high and low resolution meshes.•Asymmetric and natural facial expressions for face rehabilitation exercises.•Experimental results highlighting the open challenge of cross-resolution 3D face recognition and reconstruction.
Three dimensional (3D) printing is a technique conventionally used to manufacture prototypes. Commercial desktop 3D printers have become available which produce functional 3D printed parts. The ...MarkOne by Mark Forged manufactures printed structures reinforced with continuous Carbon, Fiberglass or Kevlar fibers. The aim of this study is to evaluate the elastic properties of the fiber reinforced 3D printed structures and predict elastic properties using an Average Stiffness (VAS) method. Samples evaluated in this study were produced by varying the volume fraction of fibers within the 3D printed structures (4.04, 8.08 and 10.1% respectively). The experimentally determined elastic modulus was found to be 1767.2, 6920.0 and 9001.2MPa for fiber volume fractions of 4.04, 8.08 and 10.1% respectively. The predicted elastic moduli were found to be 4155.7, 7380.0 and 8992.1MPa. The model results differed from experiments by 57.5, 6.2 and 0.1% for the 4.04, 8.08 and 10.1% fiber volume fractions. The predictive model allows for the elastic properties of fiber reinforced 3D printed parts. The model presented will allow for designers to predict the elastic properties of fiber reinforced 3D printed parts to be used for functional components which require specific mechanical properties.
In many robotics and VR/AR applications, 3D-videos are readily-available input sources (a sequence of depth images, or LIDAR scans). However, in many cases, the 3D-videos are processed frame-by-frame ...either through 2D convnets or 3D perception algorithms. In this work, we propose 4-dimensional convolutional neural networks for spatio-temporal perception that can directly process such 3D-videos using high-dimensional convolutions. For this, we adopt sparse tensors and propose generalized sparse convolutions that encompass all discrete convolutions. To implement the generalized sparse convolution, we create an open-source auto-differentiation library for sparse tensors that provides extensive functions for high-dimensional convolutional neural networks. We create 4D spatio-temporal convolutional neural networks using the library and validate them on various 3D semantic segmentation benchmarks and proposed 4D datasets for 3D-video perception. To overcome challenges in 4D space, we propose the hybrid kernel, a special case of the generalized sparse convolution, and trilateral-stationary conditional random fields that enforce spatio-temporal consistency in the 7D space-time-chroma space. Experimentally, we show that a convolutional neural network with only generalized 3D sparse convolutions can outperform 2D or 2D-3D hybrid methods by a large margin. Also, we show that on 3D-videos, 4D spatio-temporal convolutional neural networks are robust to noise and outperform the 3D convolutional neural network.
We present CLIP-NeRF, a multi-modal 3D object manipulation method for neural radiance fields (NeRF). By leveraging the joint language-image embedding space of the recent Contrastive Language-Image ...Pre-Training (CLIP) model, we propose a unified framework that allows manip-ulating NeRF in a user-friendly way, using either a short text prompt or an exemplar image. Specifically, to combine the novel view synthesis capability of NeRF and the controllable manipulation ability of latent representations from generative models, we introduce a disentangled conditional NeRF architecture that allows individual control over both shape and appearance. This is achieved by performing the shape conditioning via applying a learned deformation field to the positional encoding and deferring color conditioning to the volumetric rendering stage. To bridge this disentangled latent representation to the CLIP embedding, we design two code mappers that take a CLIP embedding as input and update the latent codes to reflect the targeted editing. The mappers are trained with a CLIP-based matching loss to ensure the manipulation accuracy. Furthermore, we propose an inverse optimization method that accurately projects an input image to the latent codes for manipulation to enable editing on real images. We evaluate our approach by extensive experiments on a variety of text prompts and exemplar images and also provide an intuitive interface for interactive editing.
Monolithic 3-D (M3-D) integrated circuits (ICs) provide vertical interconnects with comparable size to on-chip metal vias, and therefore, achieve ultra-high density device integration. This ...fine-grained connectivity enabled by monolithic inter-tier vias reduces the silicon area, overall wirelength, and power consumption. An open source standard cell library for design automation of large-scale transistor-level M3-D ICs is developed, thereby facilitating future research on the critical aspects of M3-D technology. The cell library is based on full-custom design of each standard cell and is fully characterized by using existing design automation tools. The proposed open source cell library is utilized to demonstrate the M3-D implementation of several benchmark circuits of various sizes ranging from 2.7-K gates to 1.6-M gates. Both power and timing characteristics of the M3-D ICs are quantified. Several versions of the cell library are developed with different number of routing tracks to better understand the issue of routing congestion in the M3-D ICs. The effect of the number of routing tracks on area, power, and delay characteristics is investigated. Finally, the primary clock tree characteristics of the M3-D ICs are discussed.
Current practice involves the use of 3D surgical planning and patient-specific solutions in multiple surgical areas of expertise. Patient-specific solutions have been endorsed for several years in ...numerous publications due to their associated benefits around accuracy, safety, and predictability of surgical outcome. The basis of 3D surgical planning is the use of high-quality medical images (e.g., CT, MRI, or PET-scans). The translation from 3D digital planning toward surgical applications was developed hand in hand with a rise in 3D printing applications of multiple biocompatible materials. These technical aspects of medical care require engineers’ or technical physicians’ expertise for optimal safe and effective implementation in daily clinical routines.The aim and scope of this Special Issue is high-tech solutions in personalized surgery, based on 3D technology and, more specifically, bone-related surgery. Full-papers or highly innovative technical notes or (systematic) reviews that relate to innovative personalized surgery are invited. This can include optimization of imaging for 3D VSP, optimization of 3D VSP workflow and its translation toward the surgical procedure, or optimization of personalized implants or devices in relation to bone surgery.
In the last decades, 3D city models appear to have been predominantly used for visualisation; however, today they are being increasingly employed in a number of domains and for a large range of tasks ...beyond visualisation. In this paper, we seek to understand and document the state of the art regarding the utilisation of 3D city models across multiple domains based on a comprehensive literature study including hundreds of research papers, technical reports and online resources. A challenge in a study such as ours is that the ways in which 3D city models are used cannot be readily listed due to fuzziness, terminological ambiguity, unclear added-value of 3D geoinformation in some instances, and absence of technical information. To address this challenge, we delineate a hierarchical terminology (spatial operations, use cases, applications), and develop a theoretical reasoning to segment and categorise the diverse uses of 3D city models. Following this framework, we provide a list of identified use cases of 3D city models (with a description of each), and their applications. Our study demonstrates that 3D city models are employed in at least 29 use cases that are a part of more than 100 applications. The classified inventory could be useful for scientists as well as stakeholders in the geospatial industry, such as companies and national mapping agencies, as it may serve as a reference document to better position their operations, design product portfolios, and to better understand the market.
A number of 3D local feature descriptors have been proposed in the literature. It is however, unclear which descriptors are more appropriate for a particular application. A good descriptor should be ...descriptive, compact, and robust to a set of nuisances. This paper compares ten popular local feature descriptors in the contexts of 3D object recognition, 3D shape retrieval, and 3D modeling. We first evaluate the descriptiveness of these descriptors on eight popular datasets which were acquired using different techniques. We then analyze their compactness using the recall of feature matching per each float value in the descriptor. We also test the robustness of the selected descriptors with respect to support radius variations, Gaussian noise, shot noise, varying mesh resolution, distance to the mesh boundary, keypoint localization error, occlusion, clutter, and dataset size. Moreover, we present the performance results of these descriptors when combined with different 3D keypoint detection methods. We finally analyze the computational efficiency for generating each descriptor.
Popular media now often present 3D printing as a widely employed technology for the production of dental prostheses. This article aims to show, based on factual information, to what extent 3D ...printing can be used in dental laboratories and dental practices at present. It attempts to present a rational evaluation of todays´ applications of 3D printing technology in the context of dental restorations. In addition, the article discusses future perspectives and examines the ongoing viability of traditional dental laboratory services and manufacturing processes. It also shows which expertise is needed for the digital additive manufacturing of dental restorations.
PDF
