Human Robot Interaction
For a humanoid robot, the perception of the users, their locations, identities, gestures, and other communicative cues is an essential necessity for efficient and safe interaction. It allows the robot to understand what users want, and to generate an appropriate response. As a member of the SFB588 “Humanoid Robots” we are working on the visual perception capabilities of the humanoid robot Armar-III with respect to the user and his activities.
We have developed an integrated real-time person tracking system capable of localizing multiple people in the field-of-view of the robot’s stereo camera. The tracking is based on the dynamic combination of multiple cues and can cope with partial occlusion as well as with deficiencies of single features.
Once a person is localized, his or her identity is estimated by means of face recognition, and the person’s focus of interest is determined by means of headpose estimation. The user’s hands are tracked in 3D, and pointing gestures are detected based on gesture models that have been learned from hundreds of training gestures.
Recently our work on tracking, face identification and gesture recognition for the humanoid robot ARMAR-III was featured on DFG Science TV. Have a look at the videos!
Selected Publications
- R. Stiefelhagen, H. Ekenel, C. Fügen, P. Gieselmann, H. Holzapfel, F. Kraft, K. Nickel, M. Voit, A. Waibel, Enabling Multimodal Human-Robot Interaction fort the Karlsruhe Humanoid Robot. IEEE Transactions on Robotics, Special Issue on Human-Robot Interaction, Vol. 23, No.5, October 2007. (pdf)
- Kai Nickel, Rainer Stiefelhagen. Fast Audio-Visual Multi-Person Tracking for a Humanoid Stereo Camera Head. IEEE-RAS 7th International Conference on Humanoid Robots - Humanoids'07, Pittsburgh, PA, November 2007. (pdf)
- Kai Nickel, Rainer Stiefelhagen. Visual recognition of pointing gestures for human-robot interaction, Image and Vision Computing, Elsevier, Volume 25, Issue 12, 3 December 2007, Pages 1875-1884. (pdf)
- See publications page for more!