Visuelle Perzeption für Mensch-Maschine Schnittstellen
Vorlesung im WS 2007/2008

Beschreibung

Derzeitige Mensch-Maschine Schnittstellen sind immer noch weitgehend "blind" was die Wahrnehmung Ihrer Benutzer betrifft. Sie sind daher weder in der Lage, die natürlichen menschlichen Kommunikationskanäle wie Mimik, Blickrichtung, Gestik, Körpersprache etc. für die Mensch-Maschine Interaktion zu nutzen, noch um ausreichendes Wissen über Ihre Nutzer, deren Zustand und Absichten zu gewinnen. Aktuelle Forschungsarbeiten beschäftigen sich damit dies zu verbessern und bessere Mensch-Maschine Schnittstellen zu entwickeln, welche ihre Benutzer und deren Handlungen wahrnehmen, und die gewonnene Kontextinformation dazu nutzen um angemessen mit den Nutzern zu interagieren.

In dieser Vorlesung werden aktuelle Arbeiten aus dem Bereich der Bildverarbeitung vorgestellt, die sich mit der visuellen Perzeption von Personen für die Mensch-Maschine Interaktion befassen. In den einzelnen Themengebieten werden verschiedene Methoden und Algorithmen, deren Vor- und Nachteile, sowie der State of the Art diskutiert:

• Lokalisierung und Erkennung von Gesichtern
• Erkennung der Mimik (facial expressions)
• Schätzen von Kopfdrehung und Blickrichtung
• Lokalisation und Tracking von Personen
• Tracking und Modellierung von Körpermodellen ("articulated body tracking")
• Gestenerkennung
• Audio-visuelle Spracherkennung
• Multi-Kamera Umgebungen
• Tools und Bibliotheken

Die Vorlesung ist prüfbar in den Vertiefungsfächern 13 - Anthropomatik, 14 - Kognitive Systeme. Umfang: 2 SWS (1x wöchentlich).

Organisatorisches

Raum: SR -118 (Informatikgebäude)
Zeit: Dienstags, wöchentlich, 11.30-13.00 Uhr (Erster Termin: 23.10.2007.)
Dozent: Dr. Rainer Stiefelhagen

Die Vorlesung ist prüfbar im Vertiefungsfach

• 13 Anthropomatik
• 14 Kognitive Systeme

Vorlesungsbegleitendes Material

Folien zur Vorlesung:

• 23.10.2007 - Einführung [pdf]
• 06.11.2007 - Face Detection 1 [pdf]
• 13.11.2007 - Face Detection 2 [pdf]
• 19.11.2007 - Face Recognition 1 [pdf]
• 27.11.2007 - Face Recognition 2 [pdf]
• 04.12.2007 - Headpose & Focus of Attention [pdf]
• 11.12.2007 - Facial Features [pdf]
• 18.12.2007 - Facial Expression [pdf]
• 08.01.2008 - Gesture Recognition [pdf]
• 15.01.2008 - Tracking 1 [pdf]
• 22.01.2008 - Tracking 2 [pdf]
• 29.01.2008 - Activity Recognition [pdf]
• 05.02.2008 - Audio-visual Speech Recognition [pdf]

Papers:

• Face Detection 1 (06.11.07)

Phung et al., Skin Segmentation Using Color Pixel Classification: Analysis and Comparison, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 27, No. 1, January 2005. [pdf]
Stan Birchfield, An Elliptical Head Tracker, 31st Asilomar Conference on Signals, Systems, and Computers, November 1997. [pdf]
Stan Birchfield, Elliptical Head Tracking Using Intensity Gradients and Color Histograms, IEEE Conference on Computer Vision and Pattern Recognition, Santa Barbara, California, June 1998. [pdf]

• Face Detection 2 (13.11.07)

H. Rowley, S. Baluja, and T. Kanade, Neural Network-Based Face Detection, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 20, No. 1, January, 1998, pp. 23-38. [pdf]
Paul Viola and Michael Jones, Robust Real-time Object Detection, Second International Workshop on Statistical and Computational Theories of Vision, Modeling, Learning, Computing and Sampling, Vancouver, Canada, July 13, 2001. [pdf]

• Face Recognition (19.11.07)

M. Turk and A. Pentland, Eigenfaces for Recognition, Journal of Cognitive Science, pp. 71-86, 1991.[pdf]
P.N. Belhumeur, J.P. Hespanha and D.J. Kriegman, Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection, IEEE Trans. on PAMI, Vol. 19, No. 7, pp. 711-720, 1997.[pdf]
W. Zhao, R. Chellappa, P.J.J. Phillips, and A. Rosenfeld, Face Recognition: A Literature Survey, ACM Computing Survey, Vol. 35, No 4, 399-458, 2003.[pdf]
Volker Blanz, Thomas Vetter, Face Recognition Based on Fitting a 3D Morphable Model, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.25, No.9, September 2003.[pdf]
Hazim Kemal Ekenel, Rainer Stiefelhagen, Local Appearance Based Face Recognition Using Discrete Cosine Transform, 13th European Signal Processing Conference (EUSIPCO), Antalya, Turkey, September 2005.[pdf]

• Head Pose and Focus of Attention (04.12.07)

Rainer Stiefelhagen, Jie Yang, Alex Waibel, A Modelbased Gaze Tracking System, Proc. of IEEE International Joint Symposia on Intelligence and Systems, pp. 304-310, Rockville Maryland, November 1996.[pdf.gz]
Rainer Stiefelhagen, Jie Yang, Alex Waibel, Modeling Focus of Attention for Meeting Indexing based on Multiple Cues, IEEE Transactions on Neural Networks, July 2002, Vol. 13, Number 4, pp. 928-938.[pdf]
M.Katzenmaier, R.Stiefelhagen, T.Schultz, I.Rogina, A.Waibel, Identifying the Addressee in Human-Human-Robot Interactions based on Head Pose and Speech, International Conference on Multimodal Interfaces ICMI 2004, State College, PA, USA, October 2004.[pdf]

• Facial Features (11.12.07)

A. L. Yuille, D. S. Cohen, P. W. Hallinan, Feature extraction from faces using deformable templates, Computer Vision Pattern Recognition , 1989.[pdf]
T. F. Cootes, C. J. Taylor, Statistical Models of Appearance for Computer Vision, Technical Report (draft), University of Manchester.[pdf]
T. F. Cootes, G. J. Edwards, C. J. Taylor, Active Appearance Models, European Conf. on Computer Vision, Vol. 2, pp. 484-498, Springer 1998.[pdf]

• Facial Expression (18.12.07)

Y. Tian, T. Kanade, J. Cohn, Facial Expression Analysis, Handbookof face recognition, S.Z. Li & A.K. Jain, ed., Springer, Oct. 2003.[pdf]
Y. Tian, T. Kanade, J. Cohn, Recognizing Action Units for Facial Expression Analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence,Vol. 23, No. 2, pp. 97-115, Feb. 2001.[pdf]
M.S. Bartlett, B. Braathen, G. Littlewort-Ford, J. Hershey, I. Fasel, T. Marks, E. Smith, T.J. Sejnowski, J.R. Movellan, Automatic Analysis of Spontaneous Facial Behavior, Tech. Report, USCD MPLabTR 2001.08, Oct. 2001.[pdf]

• Gesture Recognition (08.01.08)

T. Starner, J. Weaver, A. Pentland: Real-Time American Sign Language Recognition Using Desk and Wearable Computer Based Video. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(12):1371--1375, 1998.[pdf]
Rabiner, L.R.: A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. Proc. IEEE, 77 (2), 257-286, 1989.[pdf]
Nickel, K., Stiefelhagen, R.: 3D-Tracking of Heads and Hands for Pointing Gesture Recognition in a Human-Robot Interaction Scenario, Sixth Int. Conf. On Face and Gesture Recognition, May 2004, Seoul, Korea.[pdf]

• Tracking 1 (15.01.08)

Kai Nickel, Tobias Gehrig, Rainer Stiefelhagen, John McDonough, A Joint Particle Filter for Audio-visual Speaker Tracking, International Conference on Multimodal Interfaces ICMI 05, Trento, Italy, October 2005.[pdf]
Stauffer, Grimson, Adaptive Background Mixture Models for Real-time Tracking. CVPR�98.[pdf]
M. Isard and A. Blake, Condensation - conditional density propagation for visual tracking, International Journal of Computer Vision 29(1), pp. 5-28, 1998.[pdf]

• Tracking 2 (22.01.08)

Mun Wai Lee, Isaac Cohen and Soon Ki Jung, Particle Filter with Analytical Inference for Human Body Tracking, Institute for Robotics and Intelligent Systems, Integrated Media Systems Center, University of South California, 2002.[pdf]
D. Focken, R. Stiefelhagen, Towards Vision-based 3-D People Tracking in a Smart Room, IEEE International Conference on Multimodal Interfaces, Pittsburgh, PA, USA, October 14-16, 2002, pp. 400-405.[pdf]
K. Nickel, R. Stiefelhagen, 3D-Tracking of Heads and Hands for Pointing Gesture Recognition in a Human-Robot Interaction Scenario, Sixth Int. Conf. On Face and Gesture Recognition, May 2004, Seoul, Korea.[pdf]

• Activity Recognition (29.01.08)

F. Bobick, J. Davis, The Recognition of Human Movement Using Temporal Templates, IEEE PAMI, Vol. 23, No. 3, March 2001.[pdf]
N. Oliver, E. Horvitz, A. Garg, Layered Representations for Human Activity Recognition, Proceedings of the 4th IEEE International Conference on Multimodal Interfaces (ICMI).[pdf]
D. Demirdjian, K. Tollmar, K. Koile, N. Checka, T. Darrell, Activity Maps for Location-Aware Computing, Proceedings of the Sixth IEEE Workshop on Applications of Computer Vision, 2002.[pdf]
C. Wojek, K. Nickel, R. Stiefelhagen, Activity Recognition and Room Level Tracking in an Office Environment , IEEE Int. Conference on Multisensor Fusion and Integration for Intelligent Systems - MFI06, September 2006.[pdf]

• Audo Visual Speech Recognition (05.02.08)

Gerasimos Potamianos, Chalapathy Neti, Juergen Luettin, Iain Matthews, Audio-Visual Automatic Speech Recognition: An Overview, Issues in Visual and Audio-Visual Speech Processing, G. Bailly, E. Vatikiotis-Bateson, and P. Perrier, Eds., MIT Press, 2004.[pdf]

Weiterführende Literatur:

• Face Recognition

R. Brunelli, T. Poggio, Face Recognition: Features versus Templates, IEEE Trans. on PAMI, Vol. 15, No. 10, pp. 1042-1052, Oct. 1993.[pdf]
Y. Moses, Y. Adini and S. Ullman, Face Recognition: The Problem of Compensating for Changes in Illumination Direction, Proceedings of European Conference on Computer Vision, pp. 286-296, 1994.[pdf]
A. Pentland, B. Moghaddam, T. Starner and M. Turk, View based and Modular Eigenspaces for Face Recognition, CVPR94, pp. 84-91.[pdf]
B. Moghaddam, T. Jebara, A. Pentland, Bayesian Face Recognition, Pattern Recognition, Vol. 33, No. 11, pp. 1771-1782, Nov. 2000.[pdf]
B. Moghaddam, Principal Manifolds and Probabilistic Subspaces for Visual Recognition, IEEE Trans. on PAMI, Vol. 24, No. 6, pp. 780-788, 2002.[pdf]
L.Wiskott, J. Fellous, N. Kruger, and C. von der Malsburg, Face recognition by elastic bunch graph matching, IEEE Trans. on PAMI, 19(7):775-780, 1997.[pdf]
M.S. Bartlett, J.R. Movellan and T.J. Sejnowski, Face Recognition by Independent Component Analysis, IEEE Trans. on Neural Networks, Vol. 13, No. 6, pp. 1450-1464, 2002.[pdf]
A. Hyvarinen, E. Oja, Independent Component Analysis: Algorithms and Applications, Neural Networks, Vol. 13, pp. 411-430, 2000. [pdf]

• Facial Features

Snakes[html]

• Gesture Recognition

V.I. Pavlovic, R. Sharma, T.S. Huang: Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1997.[pdf]
Becker, D.A.: Sensei: A Real-Time Recognition, Feedback and Training System for T�aiChi Gestures. M.I.T. Media Lab Perceptual Computing Group Technical Report No. 426, 1997.[pdf]
Cassell, J.: A Framework For Gesture Generation And Interpretation. In Cipolla, R. and Pentland, A. (eds.), Computer Vision in Human-Machine Interaction, pp. 191-215. New York: Cambridge University Press. 1998.[pdf]
Poddar, I., Sethi, Y., Ozyildiz, E. and Sharma, R.: Toward Natural Gesture/Speech HCI: A Case Study of Weather Narration. Proc. Workshops on Perceptual User Interfaces, pages 1-6, November,1998.[pdf]
Y. Xioung, F. Quek, D. McNeill: Hand Motion Gestural Oscillations and Multimodal Discourse. ICMI 2003.[pdf]

• Tracking 1

Kalman Filter Website[html]

Links