The glove captures what the hand does. The model shows the hand without the glove.
Glove2Hand (arXiv:2603.20850): sensing gloves equipped with tactile sensors and IMUs record hand motions and contact forces during object manipulation. The glove is excellent for sensing (pressure distribution, joint angles, finger dynamics) but terrible for vision — it occludes the hand, changes the hand's appearance, and breaks any visual model trained on bare hands.
The system synthesizes photorealistic bare-hand videos from glove recordings. A 3D Gaussian hand model maintains temporal consistency across frames — the hand doesn't flicker or deform unrealistically. A diffusion-based restoration technique integrates the synthesized bare hand into the scene, handling the complex interactions between fingers and objects (contact deformations, partial occlusion, shadow casting).
HandSense dataset: synchronized glove-to-hand video pairs with tactile and IMU data. The synthesized videos improve downstream tasks: contact force estimation and hand tracking under occlusion both benefit from having bare-hand visual data aligned with sensor recordings.
The structural insight: the glove creates a modality gap. Sensors measure dynamics (forces, angles) but destroy visual appearance. Cameras measure appearance but can't sense forces. The synthesis bridges the gap — converting sensor-space recordings into camera-space output — without requiring the actual bare hand to be present. You get the sensing of the glove and the appearance of the bare hand. The real hand is never recorded; the synthesized hand is more useful than the real one would be, because it comes with force data.