Backend
JEPA Dots: Dual-Latent World Model for Language-Guided Robot Control
arXiv cs.RO0 views
arXiv:2611.24062v1 Announce Type: new Abstract: Predictive world models for language-guided robot control face a fundamental conditioning challenge: instructions should influence action selection without altering the predicted consequences of fixed actions under identical observations. To address this challenge, we introduce JEPA Dots (Dynamics-Oriented Task Separation), a compact dual-latent framework that jointly learns visual dynamics and language-guided control through separate dynamics and task representations. Language-conditioned task features guide a flow-matching policy, while masked attention restricts world prediction to instruction-independent dynamics history and supplied actions. In closed-loop evaluation on 40 trained LIBERO tasks, JEPA Dots achieves 84.9% overall success with approximately 39.1M parameters, compared with the published 78.7% of NanoVLA-S with 161.0M parameters under a different evaluation protocol. In the reference experiment, JEPA Dots also improves overall success from 76.0% for our task-ID-conditioned adaptation baseline to 84.9%, with LIBERO-Goal success increasing from 67.1% to 88.5%. Instruction-switching experiments further confirm that task representations and policy outputs respond to language changes, while dynamics representations and world predictions remain invariant under fixed observations and supplied actions.
Source:https://arxiv.org/abs/2611.24062PreviousWAM-Policy: World Action Models with On-Policy Distillation for Quadrupedal Loco-ManipulationNextSemPhys: Semantic-Physical Co-Learning for Humanoid Motion Generation via Differentiable World Models
Related posts
![Cargo UAV Autonomy and Flight AI — Ch.06: Route Planning and Airspace Integration [EN]](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F54s7rrr8fw0ipjpci49u.png)
Cargo UAV Autonomy and Flight AI — Ch.06: Route Planning and Airspace Integration [EN]
0dev.to
![Cargo UAV Autonomy and Flight AI — Ch.05: Perception and Sensor Fusion for UAV [EN]](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Faemm4fj7d911fwx0irut.png)
Cargo UAV Autonomy and Flight AI — Ch.05: Perception and Sensor Fusion for UAV [EN]
0dev.to
![Cargo UAV Autonomy and Flight AI — Ch.04: Autonomous Navigation for UAV [EN]](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F98idhn116ojz4pviet9j.png)
Cargo UAV Autonomy and Flight AI — Ch.04: Autonomous Navigation for UAV [EN]
0dev.to
What nginx Is Actually Doing: Reverse Proxies vs Forward Proxies Explained
0dev.to
![Cargo UAV Autonomy and Flight AI — Ch.03: Flight Control Software [EN]](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fminle8x2lymdhmeo5fbt.png)
Cargo UAV Autonomy and Flight AI — Ch.03: Flight Control Software [EN]
0dev.to
![Cargo UAV Autonomy and Flight AI — Ch.02: UAV System Architecture [EN]](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F2w7u1ajhaoa23mrl0dqs.png)
Cargo UAV Autonomy and Flight AI — Ch.02: UAV System Architecture [EN]
0dev.to