AI & ML
What Can a VLA Learn from the Agent That Calls It? Task-Level Experience Distillation for Embodied Navigation
arXiv cs.RO0 views
arXiv:2611.16246v3 Announce Type: replace Abstract: Vision-language-action (VLA) models drive the robot directly from camera views, yet a VLA alone shows three typical failures. It walks past the goal, stops before reaching it, and rarely recovers after going off track. Pairing a VLA with a large language model (LLM) planner fixes these failures, but queries the LLM throughout every episode, adding token costs and latency. We introduce Harness-to-Executor (H2E), a task-level distillation method that pairs an LLM planner and a frozen VLA as a heterogeneous training agent, keeps only sessions that reach the goal, and then trains a VLA alone on them so no LLM is used at test time. In these sessions, the planner gives sub-instructions and moves the robot with its own tools, detours are common, and the planner decides when to stop. H2E therefore labels every motion as a VLA action for the original task, whoever made it. It masks the loss on steps moving away from the goal and assigns end labels to states near it, so the VLA learns where the task ends. H2E sets a new state of the art on REVERIE-CE and NavRAG-CE. It raises an 8B VLA's success rate by 15.6% and 11.1% respectively, surpassing a method trained on 10M samples. The distilled VLA exceeds its agent on unseen episodes with zero LLM tokens and runs 3.7 times faster.
Source:https://arxiv.org/abs/2611.16246PreviousFast Inference-Time Policy Refinement via Action-Affine World ModelsNextSALA: Semantically Aligned Latent Actions Enable Dual-Axis Scaling of Embodied Foundation Models
Related posts
Darwin-Prediction-Weather: Korean AI Model Beats ECMWF, NOAA, and 6 Other Forecast Models Across 28 Benchmark Categories
0dev.to OpenSource

Your use case is not in the benchmarks
0dev.to Programming
Gemini Notebook transforma materiais de estudo em sites interativos
0dev.to Programming

How to Identify Missing Information in Construction Drawings
0dev.to JavaScript
Agenvoy@v1.3.17
0dev.to Go

Orchestrate DuckDB analytics pipelines with Amazon MWAA
0AWS Big Data Blog