DexMV

DexMV (Dexterous Manipulation from Videos) is a platform and dataset developed by researchers at UC San Diego that bridges computer vision and robot imitation learning for dexterous manipulation. The system provides synchronized multi-view human video demonstrations alongside simulated robot trajectories on a Shadow Hand dexterous hand. The dataset contains 50+ manipulation tasks covering diverse object interactions including grasping, in-hand manipulation, tool use, and assembly. Human demonstrations are captured from multiple camera angles, annotated with hand pose information, then used as supervision for training robotic policies on a simulated Shadow Hand in MuJoCo. DexMV demonstrated that human video demonstrations, combined with domain-randomized simulation, can effectively train dexterous manipulation policies that transfer to real hardware. The platform has influenced subsequent work in human-to-robot transfer for dexterous manipulation.