•Physically based rendering.•3D hyper-realistic visualization.•Footwear model generation.•OpenGL and WebGL visualization.
3D engines for real time CAD geometry representation have been improved ...extremely in recent years, both, thanks to the evolution of the computer industry itself and the new technologies used for computer graphics, like OpenGL shaders and their implementation in the last graphic cards generation. Raytracing and radiosity rendering techniques are achieving more and more close to reality results thanks to these improvements. Many manufacturing industries are taking advantage of these technologies, like footwear industry, where 3D CAD shoe models are been rendering in real time with a huge qualitative leap in terms of efficiency, costs and close-to-reality visualization results. All the same, the performance of the computational models used for the representation of virtual materials, texture maps (like normal or roughness map), specular and diffuse reflection models, illumination or shadows among many other elements, is leading to having different visualization results depending on the 3D engine that it is been used, even though we work with the same CAD model together with its materials and representation scene. Furthermore, if we bring all these implementations to web technologies, like WebGL, we must tackle not only the visualization problem but also, data optimization and computational capacity issues due to the fact of the web browsers and Internet nature. This paper proposes a computational model for hyper-realistic representation in 3D engines that offers correct results for both, 3D engines optimized for computers and 3D engines for web environments. The proposed model adapts the different characteristics of both environments and makes possible an identical visualization of a 3D CAD shoe model whatever the environment chosen.
Physically based rendering has been widely used to generate photo-realistic images, which greatly impacts industry by providing appealing rendering, such as for entertainment and augmented reality, ...and academia by serving large scale high-fidelity synthetic training data for data hungry methods like deep learning. However, physically based rendering heavily relies on ray-tracing, which can be computational expensive in complicated environment and hard to parallelize. In this paper, we propose an end-to-end deep learning based approach to generate physically based rendering efficiently. Our system consists of two stacked neural networks, which effectively simulates the physical behavior of the rendering process and produces photo-realistic images. The first network, namely shading network, is designed to predict the optimal shading image from surface normal, depth and illumination; the second network, namely composition network, learns to combine the predicted shading image with the reflectance to generate the final result. Our approach is inspired by intrinsic image decomposition, and thus it is more physically reasonable to have shading as intermediate supervision. Extensive experiments show that our approach is robust to noise thanks to a modified perceptual loss and even outperforms the physically based rendering systems in complex scenes given a reasonable time budget.
In mixed reality (MR), augmenting virtual objects consistently with real-world illumination is one of the key factors that provide a realistic and immersive user experience. For this purpose, we ...propose a novel deep learning-based method to estimate high dynamic range (HDR) illumination from a single RGB image of a reference object. To obtain illumination of a current scene, previous approaches inserted a special camera in that scene, which may interfere with user's immersion, or they analyzed reflected radiances from a passive light probe with a specific type of materials or a known shape. The proposed method does not require any additional gadgets or strong prior cues, and aims to predict illumination from a single image of an observed object with a wide range of homogeneous materials and shapes. To effectively solve this ill-posed inverse rendering problem, three sequential deep neural networks are employed based on a physically-inspired design. These networks perform end-to-end regression to gradually decrease dependency on the material and shape. To cover various conditions, the proposed networks are trained on a large synthetic dataset generated by physically-based rendering. Finally, the reconstructed HDR illumination enables realistic image-based lighting of virtual objects in MR. Experimental results demonstrate the effectiveness of this approach compared against state-of-the-art methods. The paper also suggests some interesting MR applications in indoor and outdoor scenes.
We propose an approach for the appearance synthesis of objects with matte surfaces made of arbitrary fluorescent materials, accounting for mutual illumination. We solve the problem of rendering ...realistic scene appearances of objects placed close to each other under different conditions of uniform illumination, viewing direction, and shape, relying on standard physically based rendering and knowledge of the three‐dimensional shape and bispectral data of scene objects. The appearance synthesis model suggests that the overall appearance is decomposed into five components, each of which is expanded into a multiplication of spectral functions and shading terms. We show that only two shading terms are required, related to (a) diffuse reflection by direct illumination and (b) interreflection between two matte surfaces. The Mitsuba renderer is used to estimate the reflection components based on the underlying Monte Carlo simulation. The spectral computation of the fluorescent component is performed over a broad wavelength range, including ultraviolet and visible wavelengths. We also address a method for compensating for the difference between the simulated and real images. Experiments were performed to demonstrate the effectiveness of the proposed appearance synthesis approach. The accuracy of the proposed approach was experimentally confirmed using objects with different shapes and fluorescence in the presence of complex mutual illumination effects.
The proposed application of the HBIM methodology for digitising a productive-industrial structure is based on the integration of data from different sources. An aerial photogrammetric survey ...(Unmanned Aerial Vehicle - UAV) was considered the most appropriate technique for the case. Therefore, a Scan-to-BIM modelling was carried out, keeping in mind a subsequent texturisation of the smart objects employing the photogrammetric images obtained from the UAV survey. Currently, applying the BIM methodology to the built environment is still a challenge; indeed, three-dimensional modelling based on survey point clouds is not automatic. Any BIM software is designed for new constructions, whereas the existing Heritage is characterised by unique and distinctive shapes, where each element has a specific and variable inclination, shape and thickness; therefore, it is necessary to adapt the available tools. Creating intelligent parametric objects capable of representing the unique and singular shapes and geometries of historic architecture is a significant challenge of HBIM modelling. A workflow for the acquisition, processing and management of the survey data and the consequent modelling in a BIM environment of a disused industrial plant previously used as a tobacco factory was formalised. The aim was, therefore, to develop a model that is as close as possible to the real one and, at the same time, still keeps the informative aspects in order to promote the conservation and possible refurbishment of the cultural heritage through the use of photorealistic visualisation tools in real-time. The results confirm the proposed strategy hypotheses and seem to lead to promising future developments.DOI: https://doi.org/10.20365/disegnarecon.29.2022.15
RECREATING CULTURAL HERITAGE ENVIRONMENTS FOR VR USING PHOTOGRAMMETRY Dhanda, A.; Reina Ortiz, M.; Weigert, A. ...
International archives of the photogrammetry, remote sensing and spatial information sciences.,
01/2019, Letnik:
XLII-2/W9
Journal Article, Conference Proceeding
Recenzirano
Odprti dostop
In this paper, we propose a workflow for recreating places of cultural heritage in Virtual Reality (VR) using structure from motion (SfM) photogrammetry. The unique texture of heritage places makes ...them ideal for full photogrammetric capture. An optimized model is created from the photogrammetric data so that it is small enough to render in a real-time environment. The optimized model, combined with mesh maps (texture maps, normal maps, etc.) looks like the original high detail model. The capture of a whole space makes it possible to create a VR experience with six degrees of freedom (6DoF) that allows the user to explore the historic place. Creating these experiences can bring people to cultural heritage that is either endangered or too remote for some people to access. The workflow described in this paper will be demonstrated with the case study of Myin-pya-gu, an 11th century temple in Bagan, Myanmar.
•Progressive photon mapping simplifies parametrisation for non-expert users.•Decomposition into smaller subtasks relaxes memory limitations.•Sigma-weighted accumulation of density estimates results ...in lower noise.•Reduction of bandwidth results in reduced bias in caustics.•Iterations are executed in parallel.
Daylight redirecting components (DRCs) are characterised by complex transmissive and reflective behaviour that is difficult to predict accurately largely due to their highly directional scattering, and the caustics this produces. This paper examines the application of progressive photon mapping as a state of the art forward raytracing technique to efficiently simulate the behaviour of such DRCs, and how this approach can support architects in assessing their performance.
Progressive photon mapping is an iterative variant of static photon mapping that effects noise reduction through accumulation of results, as well as a reduction in bias inherent to all density estimation methods by reducing the associated bandwidth at a predetermined rate. This not only results in simplified parametrisation for the user, but also provides a preview of the progressively refined simulation, thus making the tool accessible to non-experts as well.
We demonstrate the effectiveness of this technique with an implementation based on the Radiancephoton mapping extension and a case study involving retroreflecting prismatic blinds as a representative DRC.
PDF
