VANCOUVER, Canada (SIGGRAPH), Aug 13, 2025 – NVIDIA has announced NVIDIA Omniverse libraries and NVIDIA Cosmos world foundation models (WFMs) that accelerate the development and deployment of robotics solutions.
Powered by NVIDIA RTX PRO Servers and NVIDIA DGX Cloud, the libraries and models let developers anywhere develop physically accurate digital twins, capture and reconstruct the real world in simulation, generate synthetic data for training physical AI models and build AI agents that understand the physical world.
“Computer graphics and AI are converging to fundamentally transform robotics,” said Rev Lebaredian, vice president of Omniverse and simulation technologies at NVIDIA. “By combining AI reasoning with scalable, physically accurate simulation, we’re enabling developers to build tomorrow’s robots and autonomous vehicles that will transform trillions of dollars in industries.”
New NVIDIA Omniverse Libraries Advance Applications for World Composition
NVIDIA Omniverse software development kits (SDKs) and libraries are available for building and deploying industrial AI and robotics simulation applications.
- New Omniverse SDKs introduce data interoperability between MuJoCo (MJCF) and Universal Scene Description (OpenUSD), enabling over 250,000 MJCF robot learning developers to seamlessly simulate robots across platforms.
- New Omniverse NuRec libraries and AI models introduce Omniverse RTX ray-traced 3D Gaussian splatting, a rendering technique that lets developers capture, reconstruct and simulate the real world in 3D using sensor data.
- NVIDIA Isaac Sim 5.0 and NVIDIA Isaac Lab 2.2 open-source robot simulation and learning frameworks are available on GitHub. Isaac Sim now includes NuRec neural rendering and OpenUSD-based robot and sensor schemas that help robot developers close the simulation-to-reality gap.
Omniverse NuRec rendering is integrated in CARLA, an open-source simulator used by over 150,000 developers. Autonomous vehicle (AV) toolchain leader Foretellix is integrating NuRec, NVIDIA Omniverse Sensor RTX and Cosmos Transfer to enhance its scalable synthetic data generation with physically accurate scenarios. Voxel51’s data engine for visual and multimodal AI, FiftyOne, supports NuRec to ease data preparation for reconstructions. FiftyOne is used by customers such as Ford and Porsche.
Boston Dynamics, Figure AI, Hexagon, RAI Institute, Lightwheel and Skild AI are adopting Omniverse libraries, Isaac Sim and Isaac Lab to accelerate their AI robotics development, while Amazon Devices & Services is using them to power a manufacturing solution.
Cosmos Advances World Generation for Robotics
Cosmos WFMs, downloaded over 2 million times, let developers generate diverse data for training robots at scale using text, image and video prompts.
New models announced at SIGGRAPH deliver major advances in synthetic data generation speed, accuracy, language support and control:
- Cosmos Transfer-2, coming soon, simplifies prompting and accelerates photorealistic synthetic data generation from ground-truth 3D simulation scenes or spatial control inputs like depth, segmentation, edges and high-definition maps.
- A distilled version of Cosmos Transfer reduces the 70-step distillation process to one so developers can run the model on NVIDIA RTX PRO Servers at unprecedented speed.
Lightwheel, Moon Surgical and Skild AI are using Cosmos Transfer to accelerate physical AI training by simulating diverse conditions at scale.
Cosmos Reason Breaks Through World Understanding
Since the introduction of OpenAI’s CLIP model, vision language models (VLMs) have transformed computer-vision tasks like object and pattern recognition. However, they have not yet been able to solve multistep tasks nor handle ambiguity or novel experiences.
NVIDIA Cosmos Reason – customizable, 7-billion-parameter reasoning VLM for physical AI and robotics – lets robots and vision AI agents reason like humans, using prior knowledge, physics understanding and common sense to understand and act in the real world.
Cosmos Reason can be used for robotics and physical AI applications including:
- Data curation and annotation, which enables developers to automate high-quality curation and annotation of massive, diverse training datasets.
- Robot planning and reasoning, acting as the brain for deliberate, methodical decision-making in a robot vision language action (VLA) model. Cosmos Reason lets robots interpret environments and, given complex commands, break them down into tasks and execute them using common sense, even in unfamiliar environments.
- Video analytics AI agents built on the NVIDIA Blueprint for video search and summarization that can extract valuable insights and perform root-cause analysis on massive volumes of video data.
NVIDIA’s robotics and NVIDIA DRIVE teams are using Cosmos Reason for data curation and filtering, annotation and VLA post-training. Uber is using it to annotate and caption AV training data.
Magna is developing with Cosmos Reason as part of its City Delivery platform – a fully autonomous, low-cost solution for instant delivery – to help vehicles adapt to new cities. Cosmos Reason adds world understanding to the vehicles’ long-term trajectory planner. VAST Data, Milestone Systems and Linker Vision are adopting Cosmos Reason to automate traffic monitoring, improve safety and enhance visual inspection in cities and industrial settings.
New NVIDIA AI Infrastructure Powers Robotics Workloads Anywhere
To enable developers to take advantage of the technologies and software libraries, NVIDIA announced AI infrastructure designed for the demanding workloads.
- NVIDIA RTX PRO Blackwell Servers offer a single architecture for every robot development workload across training, synthetic data generation, robot learning and simulation.
- NVIDIA DGX Cloud, available on Microsoft Azure Marketplace, offers Omniverse developers a platform to simplify streaming OpenUSD- and NVIDIA RTX-based applications at scale from the cloud, minimizing infrastructure orchestration and management.
Accelerating the Developer Ecosystem
To help robotics and physical AI developers advance 3D and simulation technology adoption, NVIDIA also announced:
- OpenUSD Curriculum and Certification, which addresses demand for USD expertise, with support from AOUSD members Adobe, Amazon Robotics, Ansys – part of Synopsys, Autodesk, Pixar, PTC, Rockwell Automation, SideFX, Siemens, TCS and Trimble, as well as industry leaders such as Hexagon.
- Open-source collaboration with Lightwheel to integrate robot policy training and evaluation frameworks into NVIDIA Isaac Lab, featuring parallel reinforcement learning training capabilities, benchmarks and simulation-ready assets for robot manipulation and locomotion.
Source: NVIDIA
About NVIDIA
NVIDIA Corporation, based in Santa Clara, CA, is a U.S. technology company specializing in the design and production of graphics processing units (GPUs). Its hardware and software solutions support a range of applications and simulation. Operating for over 30 years, NVIDIA has seen strong financial growth, reporting $39.3 billion in revenue and $22.1 billion in net income for the fiscal quarter ending January 2025. Its headquarters are designed to promote a flat organizational structure that encourages open communication and collaboration between leadership and staff across industries. In gaming, its GPUs power high-performance visual rendering. In artificial intelligence and high-performance computing, NVIDIA provides the infrastructure needed for training and deploying large-scale models. The company also contributes to the automotive sector with systems for autonomous driving and supports robotics with tools for AI-based perception.