AI & ML
Bridging vision-language reasoning and learned control for legged navigation in the wild
arXiv cs.RO0 views
arXiv:2611.25215v1 Announce Type: new Abstract: Autonomous navigation in diverse real-world environments remains one of the greatest challenges in robotics. It demands both robust low-level motion control to handle diverse terrain and high-level reasoning to interpret environmental cues. We present ROAM (Robust Open-world Autonomous Mobility), a system that enables legged robots to perform kilometer-scale navigation in complex alpine and urban settings. ROAM combines the world knowledge of a Vision-Language Model (VLM), pre-trained on internet-scale data, with robust locomotion and local planning policies learned in simulation via reinforcement learning (RL). The VLM provides common-sense reasoning and semantic interpretation of the scene, assessing risks, interpreting signs, and producing semantic labels with associated navigation costs. An open-vocabulary segmentation module converts these labels into a dense, real-time risk map, which conditions two pre-trained RL policies responsible for agile locomotion and local navigation, effectively coupling high-level VLM reasoning with low-level physical control. Real-world experiments showed the system's ability to autonomously navigate a wide range of complex real-world environments. The robot completes kilometer-long hikes with minimal intervention, interprets signs to avoid danger, and evaluates its environment to identify safe areas on which to step. This work represents a step toward the reliable deployment of systems in unstructured environments. However, it also underscores the challenges that remain in VLM reliability and spatial reasoning, indicating that foundational models are a crucial component but not yet a complete solution for navigation in the wild.
Source:https://arxiv.org/abs/2611.25215PreviousVG-Steer: Visual-Geometric Policy Steering from In-Context Aligned AbstractionsNextFerrofluid Shape Control with Deep Reinforcement Learning
Related posts
More of the AI in Your Product May Run on the User's Own Device
0dev.to

So sicherst du einen MCP-Server mit codex sandbox ab
0dev.to

Perfiles de permisos de Codex: ejemplos de config.toml para ejecuciones seguras de agentes
0dev.to

GPTBot vs OAI-SearchBot: How to Configure robots.txt for AI Crawlers
0dev.to

ملفات تعريف أذونات Codex: أمثلة config.toml لتشغيل الوكلاء بأمان
0dev.to

Write a cover letter with AI and keep every fact yours: a nine-step loop
0dev.to