AI & ML
Drive the Thoughts: Runtime Monitoring of VLA Reasoning-Trajectory Consistency
arXiv cs.SE0 views
arXiv:2608.29583v2 Announce Type: replace Abstract: While machine learning models help autonomous vehicles (AVs) navigate complex environments, their black-box nature complicates validation and verification (V&V). Vision-language-action (VLA) models offer a unique opportunity: alongside trajectories, they generate an explicit chain-of-thought (CoT) explaining their decisions. We posit that this CoT can serve as a candidate specification to cross-check trajectories and expose unsafe behavior. To assess this, we curate DriveAlignBench, a dataset of 150 CoT-trajectory pairs from NVIDIA's Alpamayo 1.5 VLA, manually annotated for CoT reliability, CoT-trajectory consistency, and trajectory safety. We find that 33.3% of CoTs are unreliable, and among reliable CoTs, 26% have inconsistent trajectories. To detect these cases, we propose integrating a CoT-trajectory consistency check into a runtime monitor. This check is nontrivial because CoTs express open-vocabulary, scene-relative maneuvers, while trajectories are low-level ego-motion sequences whose semantics depend on road geometry and motion context. To bridge this gap, we develop a family of automated consistency monitors. The strongest, the lane-aware monitor, adds lane-relative evidence to the input of the judge model scoring consistency. This raises F1 across all four evaluated judges from 0.56-0.62 with raw waypoints to 0.65-0.82, showing that evidence representation improves detection across judges. Moreover, for reliable CoTs, inconsistency is associated with unsafe behavior: 27% of human-labeled inconsistent trajectories are unsafe compared to 3% of consistent ones, and the lane-aware monitor flags most of these unsafe cases. We release DriveAlignBench, the monitor implementations, and per-run results at https://github.com/less-lab-uva/drive-the-thoughts.
Source:https://arxiv.org/abs/2608.29583PreviousREFORGE: A Method for Benchmarking LLMs' Reverse Engineering Capabilities in Decompiled Binary Function NamingNextA Governance Methodology Layer for AI-Assisted Software Development: Defect Taxonomy, Controlled Ablation, and a Test of Process-Over-Capability
Related posts
Hot Switching vs Cold Switching: What Actually Kills a Coaxial RF Switch
0dev.to Testing
I replaced my budget app with a 16-page PDF
0dev.to Testing

SBOM ที่ทำแล้วไม่มีใครเปิด: เปลี่ยนมันเป็น re-scan รายวันที่จับ CVE ใหม่ได้ โดยไม่ต้อง rebuild image
0dev.to Kubernetes

Add ChatGPT-style AI chat to your Expo app in 5 minutes
0dev.to OpenSource
sheet_route: نوافذ bottom sheet في Flutter كـ routes حقيقية مع snap points و go_router
0dev.to OpenSource
Every family WhatsApp group in India has the same problem
0dev.to OpenSource