MIT researchers channel AI to turn hand gestures into robot training data

Researchers at the Massachusetts Institute of Technology (MIT) have created an ultrasound wristband that records the movement of muscles, tendons, and ligaments beneath the skin to train robotic hands. The device uses high-frequency sound waves to generate images of hand motion, which an AI algorithm decodes into precise degrees of freedom—specific ways joints bend or rotate. The human hand has 22 degrees of freedom, a complexity previously difficult for robots to replicate. In lab tests with eight volunteers, the wristband accurately mirrored hand gestures, including all 26 letters of American Sign Language, within 120 milliseconds. The system operates wirelessly, allowing remote control of robots without requiring the user and machine to be in the same space. MIT professor Xuanhe Zhao, who led the project, envisions the technology enabling robots to perform dexterous tasks like housework or surgery by learning from vast datasets of human motion. Unlike earlier systems that struggled to track even a fraction of hand movements, this AI-driven approach aims to bridge the gap between human dexterity and robotic capability. The wristband captures real-time muscle and tendon activity, translating it into robotic gestures with high fidelity. The team believes this method could eventually allow humanoid robots to learn complex tasks independently, reducing the need for direct human guidance. The development aligns with broader efforts to integrate AI with physical-world sensory data beyond digital applications.