Nvidia is proposing a hybrid approach to computer graphics that combines artificial intelligence with established rendering, graphics, and physics tools. Presented at SIGGRAPH 2026, the strategy addresses a major limitation of generative AI: models can quickly produce impressive images and videos, but creators often lack precise control over the results, tells AEC Magazine.
Rather than replacing conventional graphics technologies, Nvidia wants AI agents to orchestrate them. The company presented 21 research papers exploring techniques grounded in 3D and physics while remaining controllable by creators.
One example is ArtiFixer, an AI model designed to transform incomplete and noisy real-world 3D captures into clean, complete virtual scenes. It can repair missing or poor-quality data in Gaussian splats and predict photorealistic global illumination directly from scene geometry without ray tracing. Beyond visual effects and games, Nvidia sees potential applications in architecture, city planning, GIS, and virtual environments used to train and test robots.
Nvidia is also promoting Model Context Protocol, or MCP, as a way for AI agents to interact with established digital content creation applications without requiring numerous custom plugins. MCP can enable agents to transfer data between applications, automate repetitive processes, and provide natural-language access to tools such as Blender and Unreal Engine. In Unreal Engine, for example, assistants could reason about scenes, assets, and project states.
The company is applying the same modular philosophy to Omniverse. New Omniverse libraries expose capabilities such as RTX rendering, physics, and sensor simulation as components that AI agents can access and developers can embed in existing applications.
One integration connects Omniverse with PTC Onshape, allowing CAD designs to be converted into simulation-ready USD assets for robotics and physical AI applications.
Nvidia’s approach suggests that the future of computer graphics may not belong exclusively to generative AI. Instead, AI agents could coordinate proven graphics and simulation technologies, giving creators automation without sacrificing precision, physical accuracy, or control.