State of the Art on Neural Rendering Tewari, A.; Fried, O.; Thies, J. ...
Computer graphics forum,
20/May , Volume:
39, Issue:
2
Journal Article
Peer reviewed
Open access
Efficient rendering of photo‐realistic virtual worlds is a long standing effort of computer graphics. Modern graphics techniques have succeeded in synthesizing photo‐realistic images from ...hand‐crafted scene representations. However, the automatic generation of shape, materials, lighting, and other aspects of scenes remains a challenging problem that, if solved, would make photo‐realistic computer graphics more widely accessible. Concurrently, progress in computer vision and machine learning have given rise to a new approach to image synthesis and editing, namely deep generative models. Neural rendering is a new and rapidly emerging field that combines generative machine learning techniques with physical knowledge from computer graphics, e.g., by the integration of differentiable rendering into network training. With a plethora of applications in computer graphics and vision, neural rendering is poised to become a new area in the graphics community, yet no survey of this emerging field exists. This state‐of‐the‐art report summarizes the recent trends and applications of neural rendering. We focus on approaches that combine classic computer graphics techniques with deep generative models to obtain controllable and photorealistic outputs. Starting with an overview of the underlying computer graphics and machine learning concepts, we discuss critical aspects of neural rendering approaches. Specifically, our emphasis is on the type of control, i.e., how the control is provided, which parts of the pipeline are learned, explicit vs. implicit control, generalization, and stochastic vs. deterministic synthesis. The second half of this state‐of‐the‐art report is focused on the many important use cases for the described algorithms such as novel view synthesis, semantic photo manipulation, facial and body reenactment, relighting, free‐viewpoint video, and the creation of photo‐realistic avatars for virtual and augmented reality telepresence. Finally, we conclude with a discussion of the social implications of such technology and investigate open research problems.
Synthesizing photo‐realistic images and videos is at the heart of computer graphics and has been the focus of decades of research. Traditionally, synthetic images of a scene are generated using ...rendering algorithms such as rasterization or ray tracing, which take specifically defined representations of geometry and material properties as input. Collectively, these inputs define the actual scene and what is rendered, and are referred to as the scene representation (where a scene consists of one or more objects). Example scene representations are triangle meshes with accompanied textures (e.g., created by an artist), point clouds (e.g., from a depth sensor), volumetric grids (e.g., from a CT scan), or implicit surface functions (e.g., truncated signed distance fields). The reconstruction of such a scene representation from observations using differentiable rendering losses is known as inverse graphics or inverse rendering. Neural rendering is closely related, and combines ideas from classical computer graphics and machine learning to create algorithms for synthesizing images from real‐world observations. Neural rendering is a leap forward towards the goal of synthesizing photo‐realistic image and video content. In recent years, we have seen immense progress in this field through hundreds of publications that show different ways to inject learnable components into the rendering pipeline. This state‐of‐the‐art report on advances in neural rendering focuses on methods that combine classical rendering principles with learned 3D scene representations, often now referred to as neural scene representations. A key advantage of these methods is that they are 3D‐consistent by design, enabling applications such as novel viewpoint synthesis of a captured scene. In addition to methods that handle static scenes, we cover neural scene representations for modeling non‐rigidly deforming objects and scene editing and composition. While most of these approaches are scene‐specific, we also discuss techniques that generalize across object classes and can be used for generative tasks. In addition to reviewing these state‐of‐the‐art methods, we provide an overview of fundamental concepts and definitions used in the current literature. We conclude with a discussion on open challenges and social implications.
Numerous soil gas measurements of four gaseous species with very different geochemical behaviors were performed in the Fucino Basin, an area characterized by known and inferred structural ...discontinuities. A comprehensive statistical and geostatistical treatment of these data followed in order to provide insight into the spatial influence of tectonic discontinuities and geology on deep‐seated gas migration toward the surface. The results yielded anomalies with different features, reflecting the different gas‐bearing properties of the eastern seismogenic faults related to the 1915 earthquake (Mb = 7.0) and the hidden structural features occurring in the western side of the plain. In particular, this approach demonstrates that soil gas concentration (i.e., Rn and CO2) can identify the simpler normal faults of the eastern sector of the plain. In contrast, the more pervasive fracturing and faulting, as well as the occurrence of coarser deposits, on the western side of the area, make the location of faults less clear. The results show that gases migrate preferentially through zones of brittle deformation by advective processes, as suggested by the relatively high rate of migration needed to obtain anomalies of short‐lived 222Rn in the soil pores. Furthermore, a geostatistical study of soil gas data was conducted to quantify the spatial domain of correlation and the gas‐bearing properties of faults on the basis of shallow soil gas distribution (i.e., anisotropic behavior). The results provide a clear correlation between the shape and orientation of the anomalies and the different geometry of the faults recognized in the plain.
In recent years many efforts have been made to replace dermal toxicity testing of chemicals in the animal by in vitro assays. As a member of a German research consortium, we have previously ...contributed to the validation of an in vitro test protocol for percutaneous absorption studies on the basis of reconstructed human epidermis and both human and pig skin ex vivo. Aiming to assess the barrier properties of a newly developed reconstructed skin model, this protocol has now been transferred to the Phenion Full-Thickness Skin Model (FT model). The permeation of testosterone and caffeine was quantified in parallel to that of pig skin using Franz-type diffusion cells. In addition, the permeation of benzoic acid and nicotine was studied. As expected, the FT model is more permeable than pig skin, yet its barrier properties are well in accordance with those of reconstructed human epidermis when compared to previous data. In fact, the FT model most efficiently retards testosterone as the compound of highest lipophilicity, which can be explained by an additional uptake by a reservoir formed by the dermis equivalent. Thus, the structure closely parallels human skin. In consequence, the Phenion FT model appears to be suitable for percutaneous absorption studies in hazard analysis and should be subjected to a catch-up validation study.
Summary
Scleroderma (SSc) is a rare connective tissue disease characterized by fibrosis, microvasculopathy and autoimmune features. The role of genetics is limited in SSc, as suggested by similar ...concordance rates in monozygotic and dizygotic twin pairs, while environmental factors may act through epigenetic changes, as demonstrated for specific genes. Further, sex chromosome changes have been reported in SSc and may explain the female preponderance. In the present study we compared the methylation profile of all X chromosome genes in peripheral blood mononuclear cells from monozygotic twins discordant (n = 7) and concordant (n = 1) for SSc. Methylated DNA immunoprecipitations from each discordant twin pair were hybridized to a custom‐designed array included 998 sites encompassing promoters of all X chromosome genes and randomly chosen autosomal genes. Biostatistical tools identified sites with an elevated probability to be consistently hypermethylated (n = 18) or hypomethylated (n = 25) in affected twins. Identified genes include transcription factors (ARX, HSFX1, ZBED1, ZNF41) and surface antigens (IL1RAPL2, PGRMC1), and pathway analysis suggests their involvement in cell proliferation (PGK1, SMS, UTP14A, SSR4), apoptosis (MTM1), inflammation (ARAF) and oxidative stress (ENOX2). In conclusion, we propose that X chromosome genes with different methylation profiles in monozygotic twin pairs may constitute candidates for SSc susceptibility.
With topical treatment of skin diseases, the requirement of a high and reproducible drug uptake often still is not met. Moreover, drug targeting to specific skin strata may improve the use of agents ...which are prone to cause local unwanted effects. Recent investigations have indicated that improved uptake and skin targeting may become feasible by means of nanoparticular systems such as solid lipid nanoparticles (SLN), nanostructured lipid carriers (NLC) and nanoemulsions (NE). Here we describe techniques to characterize drug loading to carrier systems and skin penetration profiles by using the lipophilic dye nile red as a model agent. Since the mode of drug association with the particle matrix may strongly influence the efficiency of skin targeting, parelectric spectroscopy (PS) was used to differentiate between matrix incorporation and attachment to the particle surface and fluorescence spectroscopy (FS) to solve dye distribution within NLC particles. Nile red was incorporated into the lipid matrix or the covering tensed shell, respectively, of SLN and NLC with all the lipids studied (Compritol, Precirol, oleic acid, Miglyol). In NLC, the dye was enriched in the liquid phase. Next, nile red concentrations were followed by image analysis of vertical sections of pigskin treated with dye-loaded nanoparticular dispersions and an oil-in-water cream for 4 and 8 h in vitro. Following the SLN dispersions, dye penetration increased about fourfold over the uptake obtained following the cream. NLC turned out less potent (<
threefold increase) and penetration appeared even reduced when applying a NE. In contrast to previous studies with glucocorticoids attached to the surface of SLN, a targeting effect was not detected here. Therefore, drug targeting appears to be more strictly related to the mode of interaction of drug and particle than penetration enhancement.
Pulmonary arterial hypertension (PAH) is a rare, debilitating and rapidly progressive disease. Although there have been important medical advances in PAH management, the search for a cure continues. ...Despite an increased understanding of the disease, data on the wider effect of PAH on patients and carers, beyond the clinical symptoms, are still limited. In order to explore this, a large-scale international survey investigated four key areas affected by PAH (physical and practical, emotional, social, and information needs) and provides new insight into patients' and carers' experiences of living with the disease. The results from the survey highlight not only the limited ability of patients to carry out everyday tasks, but also the financial impact and social isolation experienced by both patients and carers. The study confirmed that a decline in a patient's World Health Organization functional class, which indicates an increase in clinical severity of the disease, is associated with greater limitations. Results from the survey demonstrate the need for multidisciplinary PAH management and a comprehensive standard of care to assess and improve all aspects of well-being for both patients and carers. In addition, they underline the need for updated PAH guidelines that address these needs.
Essentials
Missense mutations often impair protein folding, and thus intracellular trafficking and secretion.
Cellular models of severe type I hemophilia B were challenged with chaperone‐like ...compounds.
Sodium phenylbutyrate improved intracellular trafficking and secretion of the frequent p.R294Q.
The increased coagulant activity levels (∼3%) of p.R294Q would ameliorate the bleeding phenotype.
Summary
Background
Missense mutations often impair protein folding and intracellular processing, which can be improved by small compounds with chaperone‐like activity. However, little has been done in coagulopathies, where even modest increases of functional levels could have therapeutic implications.
Objectives
To rescue the expression of factor IX (FIX) variants affected by missense mutations associated with type I hemophilia B (HB) through chaperone‐like compounds.
Methods
Expression studies of recombinant (r)FIX variants and evaluation of secreted levels (ELISA), intracellular trafficking (immunofluorescence) and activity (coagulant assays) before and after treatment of cells with chaperone‐like compounds.
Results
As a model we chose the most frequent HB mutation (p.R294Q, ~100 patients), compared with other recurrent mutations associated with severe/moderate type I HB. Immunofluorescence studies revealed retention of rFIX variants in the endoplasmic reticulum and negligible localization in the Golgi, thus indicating impaired intracellular trafficking. Consistently, and in agreement with coagulation phenotypes in patients, all missense mutations resulted in impaired secretion (< 1% wild‐type rFIX). Sodium phenylbutyrate (NaPBA) quantitatively improved trafficking to the Golgi and dose dependently promoted secretion (from 0.3 ± 0.1% to 1.5 ± 0.3%) only of the rFIX‐294Q variant. Noticeably, this variant displayed a specific coagulant activity that was higher (~2.0 fold) than that of wild‐type rFIX in all treatment conditions. Importantly, coagulant activity was concurrently increased to levels (3.0 ± 0.9%) that, if achieved in patients, would ameliorate the bleeding phenotype.
Conclusions
Altogether, our data detail molecular mechanisms underlying type I HB and candidate NaPBA as affordable ‘personalized’ therapeutics for patients affected by the highly frequent p.R294Q mutation, and with reduced access to substitutive therapy.
PDF
