Daily Tech Wizard Newsletter

Stories / / By Daily Tech Wizard

Anyone can now download AI that predicts what a two-armed robot's moves will look like

Researchers can now download open-weight models that generate camera and LiDAR data for driving, and video of what a two-armed robot's actions will do.

NVIDIA added four checkpoints to its Cosmos 3 family. All four are post-trained versions of Cosmos 3 Nano with 16B trainable parameters. They are released under the OpenMDW1.1 license, which NVIDIA says permits commercial and non-commercial use under its terms.

For driving, the starting point is Cosmos3-Nano-Transfer-Auto. It generates synchronized video across up to 11 camera views, including fisheyes, and can generate LiDAR alongside the video. Text prompts guide appearance, and per-camera World Scenario Map controls specify scene structure and traffic motion. Developers can post-train it on their own camera layout, LiDAR setup and driving environment.

Cosmos3-Nano-Sim-Auto is the streaming version. It produces camera video and LiDAR in small successive chunks, using its own generated history and updated driving controls. Closed-loop use needs a simulation runtime and a World Scenario Map renderer.

For robotics, Cosmos3-Nano-Sim-Transfer takes spatial controls from a simulator plus a text prompt and renders color observations a policy can consume. Cosmos3-Nano-Sim-Bimanual predicts video from robot or camera actions, with two-arm simulation for YAM and AgiBot robots.

NVIDIA says the next step is measuring where the models faithfully support policy learning and evaluation.

Would you trust a robot trained on simulated footage?

Video: huggingface.co/blog