AI & ML
TouchTwin: Scaling Human Touch Data via Real-to-Sim-to-Real
arXiv cs.RO0 views
arXiv:2611.17619v1 Announce Type: new Abstract: Human demonstrations are becoming the main source of data for dexterous manipulation, and real-to-sim-to-real pipelines multiply each demonstration in simulation. These pipelines rebuild the interaction from vision, which recovers how the hand moves. Which fingers touch, when, and how hard remain unobserved, because a hand pose shows where each finger stops on the surface rather than how far it is driven to close. This gap is a force gap: on held-out recordings, a hand policy that imitates the observed pose grips with 0.31x the recorded force, and a policy trained on physics replays of vision-tracked poses grips with 1.55x. TouchTwin closes the gap with measured human touch. Each selected TouchScale recording, captured with head and wrist RGB-D cameras and an 880-taxel glove, is rebuilt in MuJoCo. Coding agents write the objects as physics-tested programs, and a per-subject tactile hand replays the interaction with the recorded touch setting the contact and the closing force of each finger. In simulation, a policy trained on touch-grounded replays grips at 0.99x the recorded force and succeeds in 20/20 trials on held-out recordings and 35/36 unseen perturbations; a contact predictor trained on them misses 13% of real contact frames, against 32% for vision-based replays. Each replay also seeds perturbed demonstrations with simulated contact and force labels, rendered as realistic video in new views and scenes.
Source:https://arxiv.org/abs/2611.17619PreviousX-Nav Technical Report: Real-World Long-Horizon Indoor-to-Outdoor Navigation on a Humanoid RobotNextOn the Utility of Pre-Trained Visual Representations for Policy Learning of Contact-Rich Insertion Tasks
Related posts
An LLM confidence score isn’t correctness: the certainty trap
0dev.to
What a host-side layer for LLM agents is, in plain words (and what it is not)
0dev.to
OpenAI's Agents API Lets Any App Rent Codex, Here's How It Works
0dev.to

94% of Singapore Small Business Owners Surveyed Say More AI Guidance Is Needed
0Meta Newsroom
Nashville, Tennessee Real Estate Dataset: 7,436 Listings, Excel File (Nov 10, 2026)
0dev.to Data
A Hundred and Eighty Six Thousand Pounds of Stock Had Left One Branch and Never Arrived at Another
0dev.to Data