AI & ML
TouchTwin: Scaling Human Touch Data via Real-to-Sim-to-Real
arXiv cs.RO0 views
arXiv:2611.17619v1 Announce Type: new Abstract: Human demonstrations are becoming the main source of data for dexterous manipulation, and real-to-sim-to-real pipelines multiply each demonstration in simulation. These pipelines rebuild the interaction from vision, which recovers how the hand moves. Which fingers touch, when, and how hard remain unobserved, because a hand pose shows where each finger stops on the surface rather than how far it is driven to close. This gap is a force gap: on held-out recordings, a hand policy that imitates the observed pose grips with 0.31x the recorded force, and a policy trained on physics replays of vision-tracked poses grips with 1.55x. TouchTwin closes the gap with measured human touch. Each selected TouchScale recording, captured with head and wrist RGB-D cameras and an 880-taxel glove, is rebuilt in MuJoCo. Coding agents write the objects as physics-tested programs, and a per-subject tactile hand replays the interaction with the recorded touch setting the contact and the closing force of each finger. In simulation, a policy trained on touch-grounded replays grips at 0.99x the recorded force and succeeds in 20/20 trials on held-out recordings and 35/36 unseen perturbations; a contact predictor trained on them misses 13% of real contact frames, against 32% for vision-based replays. Each replay also seeds perturbed demonstrations with simulated contact and force labels, rendered as realistic video in new views and scenes.
Source:https://arxiv.org/abs/2611.17619PreviousX-Nav Technical Report: Real-World Long-Horizon Indoor-to-Outdoor Navigation on a Humanoid RobotNextOn the Utility of Pre-Trained Visual Representations for Policy Learning of Contact-Rich Insertion Tasks
Related posts

Delhi-based Bharat Semi, which is developing programmable image sensors combining capture, processing, and AI capabilities on a single chip, raised a $24M seed (Jagmeet Singh/TechCrunch)
0Techmeme

Russia is bringing young foreign influencers to occupied Ukraine, encouraging them to share their highly curated experiences, to legitimize its occupation (Linda Hourani/The Kyiv Independent)
0Techmeme

Startups unveil plans to develop floating offshore data centers to bypass land and power constraints, while experts warn of challenges like saltwater corrosion (Jackie Snow/Wall Street Journal)
0Techmeme

Analysis: global manufacturing, utility, and mining companies have shed $4.2T in market value since June as the AI "picks and shovels" trade swings into reverse (Clara Murray/Financial Times)
0Techmeme
Kernel prepatch 7.4-rc3
0LWN.net
Ask HN: Startups are spamming helpdesk chats now?
0Hacker News New