AI & ML
From Video Frames to Robot Meta-Frames: A Unified Latent Interface for World Action Models
arXiv cs.RO0 views
arXiv:2611.14365v1 Announce Type: cross Abstract: Pretrained video diffusion models (VDMs) offer a powerful foundation for World Action Models (WAMs), which jointly predict robot actions and future observations. Building such models requires injecting low-dimensional robot signals (e.g. proprioception, actions, and values) into a latent space shaped by natural videos, and a key design choice is the interface through which these signals enter: existing schemes treat it as a formatting detail, spending a separate latent frame on each signal or bolting on dedicated encoders and decoders. We argue that signal injection is instead a distribution-alignment problem: how well a representation matches the prior of the pretrained VDMs during denoising. Using the relative reconstruction error of a frozen VDM denoiser as a probe, we find that conventional spatial encodings sit far off this prior, whereas frequency-domain packing yields latents that the denoiser corrects far less. Motivated by this diagnostic result, we propose MetaWAM, which folds all robot signals into a single unified meta-frame: each signal is assigned to a disjoint frequency-domain channel band and lifted to the spatial latent by an inverse DCT, yielding a decodable interface aligned with the pretrained VDM prior without any added module or architectural change. A fixed band layout together with slot-aware routing lets the same meta-frame support policy, world-model, and value prediction over a shared set of robot variables. On RoboCasa, MetaWAM reaches 68.9% success while collapsing the per-signal frames into one and cutting inference latency by 39%, and attains 85.0% success on three real-world Franka tasks. Code and models are available at https://github.com/showlab/MetaWAM
Source:https://arxiv.org/abs/2611.14365PreviousHiPFuse: Real-Time LiDAR Semantic Segmentation for Robotic Perception on Edge DevicesNextUGG: Unified Generative Geometry Model
Related posts

The Interface Questions That Matter in a Disposable-Inbox Workflow
0dev.to UX
Zwei Berechtigungen, die jede App kennt – und viele Onboarding-Flows packen sie
0dev.to UX

375 reviews sits between a $14.99 Reacher novel and a 7-review LitRPG
0dev.to Data
Arkansas Layoffs November 2026: WARN Numbers Explained
0dev.to Data
District of Columbia Layoffs November 2026: WARN Numbers Explained
0dev.to Data
1M Emails/Month at 15 Providers: The Real Price Breakdown
0dev.to Data