A demo of Jorge Ortiz Valério's thesis on privacy-preserving gesture recognition

Jorge Alejandro Ortiz Valério completed his Master’s thesis, Identity-verified Gesture Recognition based on Time-of-Flight Sensors (PDF), and we put together a short video that shows the system running. Congratulations, Jorge!

Touch-free gesture control is attractive for smart buildings, but two problems keep it out of real deployments: ordinary cameras are privacy-invasive, and anyone in view can trigger a command. The thesis answers both with a single ceiling-mounted depth camera. It sees the scene only in 3D, never as a color image, so no faces and no identifiable visual information are ever captured, and it pairs the depth view with a wrist-worn IMU: by correlating the wrist’s motion with the visual motion of each tracked person, the system verifies that the gesture came from the authorized user, without enrolling anyone’s biometrics.

The demo walks through the live pipeline, from the depth image and foreground mask to per-person tracking, the IMU-vision confidence that verifies identity, and the recognized gesture, first with a single user and then with several people sharing the same space, where only the authorized wearer’s gestures are accepted.

Player not loading? Watch on YouTube ↗

The work is a joint effort between the Communication Systems Group (CSG) at the University of Zurich, our Sensing Group at ICV-HSG, and Tridonic, the Vorarlberg-based lighting-technology company that provided the Time-of-Flight camera and its initial deployment expertise. The full source code is open.