Details
- NVIDIA AI announces Cosmos 3, described as the world’s first fully open omnimodel with native vision reasoning, world generation, and action generation.
- The initial release includes two model variants: Cosmos 3 Super with 32B parameters and Cosmos 3 Nano with 8B parameters, targeting different performance and deployment needs.
- Cosmos 3 introduces an MoT (Mixture-of-Towers) architecture that pairs an autoregressive reasoner tower with a diffusion-based generator tower to unify world generation, physical understanding, and controlled scene generation.
- The model reportedly delivers leading results on physical AI benchmarks among open models, ranking first on Artificial Analysis, Physics-IQ, PAI-Bench, and R-Bench for world generation accuracy, as well as RoboLab and RoboArena for action policy and VANTAGE-Bench and TAR for vision.
- Beyond multimodal understanding and reasoning, Cosmos 3 is designed to simulate physical environments, predict future world states, and assist in training robots to perform specific tasks, enabling subsecond vision reasoning and large-scale synthetic data generation.
- Cosmos 3 demonstrates strong image-to-video capabilities, exemplified by generating a dashcam-style Formula 1 racing scene video, including sound, from a single input image and prompt.
- Trained on billions of multimodal samples, the model is positioned as a powerful pretrained foundation for physical AI systems, allowing developers to use less task-specific data and incur lower training costs.
- Cosmos 3 is released as fully open, including access to model weights and post-training recipes, and is available for developers to download and use via Hugging Face.
- NVIDIA provides a technical blog detailing the architecture, training data, benchmarks, and use cases for Cosmos 3 to support researchers and builders.
Impact
Cosmos 3 reinforces NVIDIA’s position at the center of the physical AI stack by combining vision, world modeling, and action generation in a single open model. By releasing weights and recipes and targeting robotics, simulation, and synthetic data use cases, NVIDIA both pressures proprietary frontier labs and strengthens the open ecosystem competing with models from OpenAI, Google, and others in multimodal and embodied AI.
