AI & ML
3DTouch: A Tactile-aware VLA Model with 3D Geometric Relations for Dexterous Manipulation
arXiv cs.RO0 views
arXiv:2611.23961v1 Announce Type: new Abstract: In contact-rich dexterous manipulation, occlusion by the fingers and interacting objects obscures local geometry, preventing visual observations from fully capturing contact locations. Visually similar scenes may also differ subtly in the relative positions of interacting objects, leading to different contact states. We propose 3DTouch, a tactile-geometric conditioning approach for vision-language-action (VLA) models. 3DTouch constructs a shared 3D visuo-tactile representation by expressing RGB-D scene points and dense five-finger tactile measurements in the robot-base frame, preserving each taxel's spatial location and force vector. Hand-object and object-object relations derived from hand landmarks and pose-aligned reconstructed surfaces explicitly describe the relative arrangement of interacting entities. Together, these representations condition the VLA action expert on spatially localized contact forces and relative surface geometry, providing explicit cues for contact adjustment and object alignment. We evaluate 3DTouch on five real-world contact-rich dexterous manipulation tasks using a UR7e robot arm equipped with an XHand dexterous hand. 3DTouch improves the average success rate by 42 percentage points over the corresponding VLA baseline and by 20 percentage points over the strongest evaluated alternative. Ablation studies reveal the complementary benefits of scene geometry and spatial tactile feedback, with further gains from shared spatial grounding and explicit geometric relations.
Source:https://arxiv.org/abs/2611.23961PreviousFacet-SLAM: Real-Time Triangle Splatting SLAMNextCoRA-VTLA: Contact-Guided Attention and Queryable Response Memory for Tactile Post-Training
Related posts
![[Phân tích kỹ thuật] Artex: Vũ khí AI Agent 'ngắm là bắn' rút ngắn thời gian tấn công mạng xuống dưới 3 giờ](https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F69aiefxt38opzs2grqoj.webp)
[Phân tích kỹ thuật] Artex: Vũ khí AI Agent 'ngắm là bắn' rút ngắn thời gian tấn công mạng xuống dưới 3 giờ
0dev.to
A homepage check is not a usability verdict: lessons from 224 garage websites
0dev.to
Designing a useful preview for an AI-generated comic
0dev.to
AI Agents: What They Are and Why They Matter
0dev.to
GPT-6.2 Astra Was Easier to Jailbreak, UK Testing Found
0dev.to
iFixit Says Apple’s HomeView Breaks Easily. Here’s Why I’m Not Worried
0CNET News