The recent research explosion around implicit neural representations, such as NeRF, shows that there is immense potential for implicitly storing high‐quality scene and lighting information in compact ...neural networks. However, one major limitation preventing the use of NeRF in real‐time rendering applications is the prohibitive computational cost of excessive network evaluations along each view ray, requiring dozens of petaFLOPS. In this work, we bring compact neural representations closer to practical rendering of synthetic content in real‐time applications, such as games and virtual reality. We show that the number of samples required for each view ray can be significantly reduced when samples are placed around surfaces in the scene without compromising image quality. To this end, we propose a depth oracle network that predicts ray sample locations for each view ray with a single network evaluation. We show that using a classification network around logarithmically discretized and spherically warped depth values is essential to encode surface locations rather than directly estimating depth. The combination of these techniques leads to DONeRF, our compact dual network design with a depth oracle network as its first step and a locally sampled shading network for ray accumulation. With DONeRF, we reduce the inference costs by up to 48× compared to NeRF when conditioning on available ground truth depth information. Compared to concurrent acceleration methods for raymarching‐based neural representations, DONeRF does not require additional memory for explicit caching or acceleration structures, and can render interactively (20 frames per second) on a single GPU.
End‐to‐End Compressed Meshlet Rendering Mlakar, D.; Steinberger, M.; Schmalstieg, D.
Computer graphics forum,
February 2024, 2024-02-00, 20240201, Volume:
43, Issue:
1
Journal Article
Peer reviewed
Open access
In this paper, we study rendering of end‐to‐end compressed triangle meshes using modern GPU techniques, in particular, mesh shaders. Our approach allows us to keep unstructured triangle meshes in GPU ...memory in compressed form and decompress them in shader code just in time for rasterization. Typical previous approaches use a compressed mesh format only for persistent storage and streaming, but must decompress it into GPU memory before submitting it to rendering. In contrast, our approach uses an identical compressed format in both storage and GPU memory. Hence, our compression method effectively reduces the in‐memory requirements of huge triangular meshes and avoids any waiting times on streaming geometry induced by the need for a decompression stage on the CPU. End‐to‐end compression also means that scenes with more geometric detail than previously possible can be made fully resident in GPU memory. Our approach is based on a novel decomposition of meshes into meshlets, i.e. disjoint primitive groups that are compressed individually. Decompression using a mesh shader allows de facto random access on the primitive level, which is important for applications such as selective streaming and fine‐grained visibility computation. We compare our approach to multiple commonly used compressed meshlet formats in terms of required memory and rendering times. The results imply that our approach reduces the required CPU–GPU memory bandwidth, a frequent bottleneck in out‐of‐core rendering.
Our approach allows us to keep unstructured triangle meshes in GPU memory in compressed form and decompress them in the mesh shader just in time for rasterization.
Summary
A robustifying strategy for constrained linear multivariable systems is proposed. A combination of tracking model predictive control with output integral sliding mode techniques is used to ...completely reject bounded matched perturbations. It can be guaranteed that all constraints on inputs, states, and outputs are satisfied although only output information is used. Finally, real‐world experiments with an unstable plant are presented in order to demonstrate the validity and the effectiveness of the proposed approach.
Streaming rendering, e.g., rendering in the cloud and streaming via a mobile connection, suffers from increased latency and unreliable connections. High quality framerate upsampling can hide these ...issues, especially when capturing shading into an atlas and transmitting it alongside geometric information. The captured shading information must consider triangle footprints and temporal stability to ensure efficient video encoding. Previous approaches only consider either temporal stability or sample distributions, but none focuses on both. With SnakeBinning, we present an efficient triangle packing approach that adjusts sample distributions and caters for temporal coherence. Using a multi‐dimensional binning approach, we enforce tight packing among triangles while creating optimal sample distributions. Our binning is built on top of hardware supported real‐time rendering where bins are mapped to individual pixels in a virtual framebuffer. Fragment shader interlock and atomic operations enforce global ordering of triangles within each bin, and thus temporal coherence according to the primitive order is achieved. Resampling the bin distribution guarantees high occupancy among all bins and a dense atlas packing. Shading samples are directly captured into the atlas using a rasterization pass, adjusting samples for perspective effects and creating a tight packing. Comparison to previous atlas packing approaches shows that our approach is faster than previous work and achieves the best sample distributions while maintaining temporal coherence. In this way, SnakeBinning achieves the highest rendering quality under equal atlas memory requirements. At the same time, its temporal coherence ensures that we require equal or less bandwidth than previous state‐of‐the‐art. As SnakeBinning outperforms previous approach in all relevant aspects, it is the preferred choice for texture‐based streaming rendering.
Commonly used image‐space layouts of shading points, such as used in deferred shading, are strictly view‐dependent, which restricts efficient caching and temporal amortization. In contrast, ...texture‐space layouts can represent shading on all surface points and can be tailored to the needs of a particular application. However, the best grouping of shading points—which we call a shading unit—in texture space remains unclear. Choices of shading unit granularity (how many primitives or pixels per unit) and in shading unit parametrization (how to assign texture coordinates to shading points) lead to different outcomes in terms of final image quality, overshading cost, and memory consumption. Among the possible choices, shading units consisting of larger groups of scene primitives, so‐called meshlets, remain unexplored as of yet. In this paper, we introduce a taxonomy for analyzing existing texture‐space shading methods based on the group size and parametrization of shading units. Furthermore, we introduce a novel texture‐space layout strategy that operates on large shading units: the meshlet shading atlas. We experimentally demonstrate that the meshlet shading atlas outperforms previous approaches in terms of image quality, run‐time performance and temporal upsampling for a given number of fragment shader invocations. The meshlet shading atlas lends itself to work together with popular cluster‐based rendering of meshes with high geometric detail.
Tessellated Shading Streaming Hladky, J.; Seidel, H. P.; Steinberger, M.
Computer graphics forum,
July 2019, 2019-07-00, 20190701, Volume:
38, Issue:
4
Journal Article
Peer reviewed
Presenting high‐fidelity 3D content on compact portable devices with low computational power is challenging. Smartphones, tablets and head‐mounted displays (HMDs) suffer from thermal and battery‐life ...constraints and thus cannot match the render quality of desktop PCs and laptops. Streaming rendering enables to show high‐quality content but can suffer from potentially high latency. We propose an approach to efficiently capture shading samples in object space and packing them into a texture. Streaming this texture to the client, we support temporal frame up‐sampling with high fidelity, low latency and high mobility. We introduce two novel sample distribution strategies and a novel triangle representation in the shading atlas space. Since such a system requires dynamic parallelism, we propose an implementation exploiting the power of hardware‐accelerated tessellation stages. Our approach allows fast de‐coding and rendering of extrapolated views on a client device by using hardware‐accelerated interpolation between shading samples and a set of potentially visible geometry. A comparison to existing shading methods shows that our sample distributions allow better client shading quality than previous atlas streaming approaches and outperforms image‐based methods in all relevant aspects.
The humble loop shrinking property played a central role in the inception of modern topology but it has been eclipsed by more algebraic formalisms. This is particularly true in the context of ...detecting relevant non‐contractible loops on surfaces where elaborate homological and/or graph theoretical constructs are favored in algorithmic solutions. In this work, we devise a variational analogy to the loop shrinking property and show that it yields a simple, intuitive, yet powerful solution allowing a streamlined treatment of the problem of handle and tunnel loop detection. Our formalization tracks the evolution of a diffusion front randomly initiated on a single location on the surface. Capitalizing on a diffuse interface representation combined with a set of rules for concurrent front interactions, we develop a dynamic data structure for tracking the evolution on the surface encoded as a sparse matrix which serves for performing both diffusion numerics and loop detection and acts as the workhorse of our fully parallel implementation. The substantiated results suggest our approach outperforms state of the art and robustly copes with highly detailed geometric models. As a byproduct, our approach can be used to construct Reeb graphs by diffusion thus avoiding commonly encountered issues when using Morse functions.
Hydrogen is used as a carrier gas in the photovoltaics and semiconductor industry. Once scrubbed from poisonous and environmentally harmful substances, the hydrogen-rich waste gas streams are ...released into the atmosphere. This paper presents an economic analysis of different options for further use of these streams. Combustion in a gas engine with attached electricity generation is currently the most technologically proven and economically profitable solution. With the future rise of the hydrogen electric vehicle infrastructure, power generation via a polymer electrolyte membrane (PEM) fuel cell system could become economically feasible at electricity prices of 0.11 €/kWh and higher. Fuel cells are high tech and modular but still relatively expensive today. Another promising technology identified is the recycling of hydrogen via electrochemical hydrogen compression (EHC). EHC is already economically feasible today at a hydrogen price of 11 €/kg or higher. It will become even more profitable with falling membrane electrode assembly prices.
•First economic and technological assessment for second use of H2-rich waste gas.•Using waste gases for more profitable and more sustainable semiconductor production.•Fuel cells show high future savings potential in semiconductor gas abatement.•Electrochemical hydrogen compression has high future potential for gas recycling.•Electrochemical hydrogen recycling can outweigh electricity generation earnings.
Subdivision surfaces have become an invaluable asset in production environments. While progress over the last years has allowed the use of graphics hardware to meet performance demands during ...animation and rendering, high‐performance is limited to immutable mesh connectivity scenarios. Motivated by recent progress in mesh data structures, we show how the complete Catmull‐Clark subdivision scheme can be ed in the language of linear algebra. While this high‐level formulation allows for a fully parallel implementation with significant performance gains, the underlying algebraic operations require further specialization for modern parallel hardware. Integrating domain knowledge about the mesh matrix data structure, we replace costly general linear algebra operations like matrix‐matrix multiplication by specialized kernels. By further considering innate properties of Catmull‐Clark subdivision, like the quad‐only structure after refinement, we achieve an additional order of magnitude in performance and significantly reduce memory footprints. Our approach can be adapted seamlessly for different use cases, such as regular subdivision of dynamic meshes, fast evaluation for immutable topology and feature‐adaptive subdivision for efficient rendering of animated models. In this way, patchwork solutions are avoided in favor of a streamlined solution with consistent performance gains throughout the production pipeline. The versatility of the sparse matrix linear algebra ion underlying our work is further demonstrated by extension to other schemes such as and Loop subdivision.
The purpose of this study is to highlight our technique for freehand placement of juxtapedicular screws along with intraoperative computed tomography (CT) and radiographic results.
Consecutive ...patients with adult idiopathic scoliosis undergoing primary surgery by the senior author were identified. All type D (absent/slit like channel) pedicles were identified on preoperative CT. Three-dimensional visualization software was used to measure screw angulation and purchase. Radiographs were measured by a fellowship trained spine surgeon. The freehand technique was used to place all screws in a juxtapedicular fashion without any fluoroscopic, radiographic, navigational or robotic assistance.
Seventy-three juxtapedicular screws were analyzed. The most common level was T7 (9 screws) on the left and T5 (12 screws) on the right. The average medial angulation was 20.7° (range, 7.1°-36.3°), lateral vertebral body purchase was 13.4 mm (range, 0-28.9 mm), and medial vertebral body purchase was 21.1 mm (range, 8.9-31.8 mm). More than half (53.4%) of the screws had bicortical purchase. Two screws were lateral on CT scan, defined by the screw axis lateral to the lateral vertebral body cortex. No screws were medial. There was a difference in medial angulation between screws with (n = 58) and without (n = 15) lateral body purchase (22.0 ± 4.9 vs. 15.5 ± 4.5, p < 0.001). Three of 73 screws were repositioned after intraoperative CT. There were no neurovascular complications. The mean coronal cobb corrections for main thoracic and lumbar curves were 83.0% and 80.5%, respectively, at an average of 17.5 months postoperative.
Freehand juxtapedicular screw placement is a safe technique for type D pedicles in adult idiopathic scoliosis patients.
PDF
