The next generation of material representation in graphics: NVIDIA’s AI makes it possible

At Siggraph, NVIDIA presented a new approach called “Neural Appearance Models”, which uses artificial intelligence (AI) to significantly accelerate the real-time rendering of materials. This innovative approach enables a performance increase of up to 24 times compared to conventional methods.

Instead of relying on computationally intensive shading graphics, NVIDIA uses neural networks to capture every detail of an object – from fine details to complex lighting effects. This approach not only enables faster computation, but also more detailed rendering, which is especially beneficial for complex visual scenes.

Innovations and benefits

The new models offer several significant innovations:

• A scalable system for movie quality in neural materials.
• Practical training for large assets through an efficient encoder.
• Decoder with priors for normal mapping and sampling.
• Efficient execution of neural networks in real-time shaders.

The functionality of Neural Appearance Models is similar to that of traditional models. At render time, textures are checked and two MLPs (Multi-Layer Perceptrons) are evaluated to determine the BRDF (Bidirectional Reflectance Distribution Function) value and the output direction. Improvements such as built-in graphics priors and a massive resolution encoder further optimize the process and enable faster and more accurate rendering.

Thanks to the efficiency of the neural models, these textures can be scaled to different applications. NVIDIA demonstrated models of varying complexity rendered in a few milliseconds. This fast render time opens up new possibilities for real-time applications and significantly improves overall performance.

The Neural Appearance Models utilize existing machine learning frameworks such as PyTorch and TensorFlow, as well as tools such as GLSL and HLSL. In addition, hardware-accelerated matrix multiplication engines (MMA) are used. The runtime shader compiles the neural material description into optimized code using the open-source Slang shading language, which provides backends for a variety of targets such as Vulkan, Direct3D 12 and CUDA.

The performance of this new approach was demonstrated on an NVIDIA GeForce RTX 4090 GPU with hardware-accelerated ray tracing at 1920×1080 resolution. Render times were significantly reduced while maintaining the same level of detail. This efficiency enables faster and more realistic rendering of textures and materials in real time.

NVIDIA’s new approach has the potential to fundamentally change the way textures and objects are rendered in real time. Developers and content creators can create more realistic materials faster while improving the performance of their applications. This technology is expected to be used in future games and applications to provide an even more immersive and detailed experience.

Overall, NVIDIA’s new Neural Appearance Models approach represents a significant advancement in real-time rendering technology. By utilizing AI and neural networks, more efficient and accurate rendering is made possible, significantly improving both performance and visual quality.

Source: YouTube