Open Source
Interactive World Simulation with Cosmos for Physical AI
arXiv cs.RO0 views
arXiv:2611.06627v1 Announce Type: cross Abstract: We present a shared framework that adapts Cosmos world foundation models for real-time interactive world simulation in Physical AI. By streaming visual observations in response to actions and scene controls, the framework enables users and embodied agents to interact with evolving virtual environments. It supports both predicting the visual consequences of actions and generating observations from physical state maintained by an external simulator. We demonstrate this approach across camera navigation and robot manipulation, controllable video transfer, and multi-view autonomous vehicle (AV) simulation. These examples provide a shared foundation for developing interactive virtual environments for Physical AI training and evaluation. Our code and model checkpoints are available at https://github.com/nvidia/cosmos and https://huggingface.co/collections/nvidia/cosmos3-examples. The project website is available at https://research.nvidia.com/labs/cosmos-lab/cosmos-simulation.
Source:https://arxiv.org/abs/2611.06627PreviousReactWorld: Structuring Interaction Responses for Embodied Video GenerationNextReinforcement Learning Enables Scaling Generalist Robot Policy Improvement
Related posts
The Dangers of Non-Self-Fixed Architecture Technical Debt and Its Impact on Time-to-Fix
0arXiv cs.SE
ViBench 2.0: A Benchmark for Long-Horizon Software Building
0arXiv cs.SE
AI-Driven SaaS Platform Integration: AI Express for Unified Generative AI Services with Pay-As-You-Go Credit Systems and Scalable Microservice Architecture
0arXiv cs.SE
Online Draft Co-Training for Speculative Decoding in Large-Scale, Long-Context RL Post-Training
0arXiv cs.DC
On solid angles and simplex shape measures
0arXiv cs.NA
Improving the performance of regularized Stokeslet simulations using Rosenbrock time integration methods
0arXiv cs.NA