Computer Vision for Human-Computer Interaction

  • Type: Vorlesung (V)
  • Semester: WS 20/21
  • Time:

    Die Vorlesung wird online gehalten.

    Neue Zeiten (wegen Corona-Hygieneverordnung):
    Montags: 14:00-15:30
    Freitags: 10:00-11:30

     
  • Lecturer: Prof. Dr.-Ing. Rainer Stiefelhagen
    Dr.-Ing. Muhammad Saquib Sarfraz
  • SWS: 4
  • Lv-No.: 24180
Links
Notes

In this lecture current projects of the field of image processing will be presented which deal with the visual perception of persons re. human-computer interaction.
In respect of the individual topics we will discuss various methods and algorithms, their pros and cons and state of the art:
• Face detection and localisation
• Facial expression
• Assessment of head turns and viewing direction
• Person tracking and localisation
• Articulated body tracking
• Gesture recognition
• Audio-visual speech recognition
• Multi-camera environments
• Tools and libraries

The student acquires a basic understanding of computer vision topics within the context of human-computer interaction and learns how to apply them.

Language of instructionDeutsch
Description

In this lecture current projects of the field of image processing will be presented which deal with the visual perception of persons re. human-computer interaction.
In respect of the individual topics we will discuss various methods and algorithms, their pros and cons and state of the art:
• Face detection and localisation
• Facial expression
• Assessment of head turns and viewing direction
• Person tracking and localisation
• Articulated body tracking
• Gesture recognition
• Audio-visual speech recognition
• Multi-camera environments
• Tools and libraries

Bibliography

Weiterführende Literatur

Wissenschaftliche Veröffentlichungen zum Thema, werden auf der VL-Website bereitgestellt.

Workload

Besuch der Vorlesungen: ca. 40 Stunden

Vor- und Nachbereitung der Vorlesung: ca. 40 Stunden

Durchführung der Programmierprojekte: ca. 30 Stunden

Klausurvorbereitung: ca. 70 h

Summe: ca. 180 Stunden

Aim
  • Die Studierenden bekommen einen Überblick über grundlegende und aktuelle Bildverarbeitungsverfahren zur Erfassung von Menschen in Bildern und Bildfolgen sowie deren verschiedene Anwendungen im Bereich der Mensch-Maschine-Interaktion.
  • Die Studierenden verstehen grundlegende Konzepte und aktuelle Verfahren zur Erfassung von Menschen in Bildern und Bildfolgen, deren Möglichkeiten und Grenzen und kann diese anwenden

Vorlesungsplan und Materialien

Schedule updated on 22nd February 2021:

02.11.20   Introduction, Applications [pdf]

06.11.20   Pattern Recognition [pdf]

09.11.20   Face Detection I: Color, ANNs [pdf]

13.11.20   Deep Learning - Basics [pdf]

16.11.20   Hands-On Projects: Practice and questions day - all three projects [pdf]

20.11.20   Face Recognition I: Traditional Approaches: Eigenface, Fisherface, ... [pdf]

23.11.20   Face Recognition II: Features: LBP, DCT, Gabor, SIFT, Morphable Models (Pose Problem) [pdf]

27.11.20   Face Recognition III: In the Wild, Deep Learning, FR Wrap Up [pdf]

30.11.20   Facial Feature Detection / Statistical Face Models [pdf]

04.12.20   Facial Expression Recognition (FACS vs. Emotions) [pdf]

07.12.20   Hands-On Projects: Practice and questions day I

11.12.20   People Detection I - Holistic Models [pdf]

14.12.20   People Detection II - Part-based Models [pdf]

18.12.20   People Detection III - Part-based Models [pdf]

21.12.20   No Lecture - Time for Project Preparation

Weihnachten / Ferien -> No Lecture

08.01.21   No Lecture - Time for Project Preparation

11.01.21   Tracking I : Kalman, Particle Filter & Applications (AV-Tracking, Body pose) [pdf]

15.01.21   No Lecture - Time for Project Preparation

18.01.21   Tracking II: Multi-Camera Systems & Articulated Pose [pdf]

22.01.21   No Lecture - Time for Project Preparation

25.02.21   Hands-On Projects: Practice and questions day II

29.01.21   No Lecture - Time for Project Preparation

01.02.21   Hands-On Projects: Presentations

05.02.21   New: Body Pose Estimation (Kinect, Pose Machine, …) [pdf]

08.02.21   Gesture Recognition: Taxonomy, Neill, Starner (ASL), Nickel, ... [pdf]

12.02.21   Activity Analysis / Action Recognition I [pdf]

15.02.21   Activity Analysis / Action Recognition II [pdf]

19.02.21   Wrap-Up: What have we learnt [pdf]

 

Update: The written exam will take place on 26th February 2021, 9:00-10:00, in the small tent on the Forum.
Registration closed

 

 

 

Ergänzende Literatur

  • Face Detection
  • Phung et al., Skin Segmentation Using Color Pixel Classification: Analysis and Comparison, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 27, No. 1, January 2005. [pdf]
  • H. A. Rowley, S. Baluja, and T. Kanade, Neural Network-Based Face Detection, IEEE Transactions on Pattern Analysis and Machine Intelligence, January 1998. [pdf]

  • Face Recognition

  • Yaniv Taigman Ming Yang Marc’Aurelio Ranzato, DeepFace: Closing the Gap to Human-Level Performance in Face Verification, Facebook AI Research, Menlo Park, CA, USA [pdf]

  • Parkhi et al.: Deep Face Recognition, Visual Geometry Group, Department of Engineering Science, University of Oxford, 2015 [pdf]
  • Florian Schroff, Dmitry Kalenichenko, James Philbin, FaceNet: A Unified Embedding for Face Recognition and Clustering, June 2015 [pdf]
  • Facial Expression Recognition
  • Y. Tian, T. Kanade, J. Cohn; Handbook of face recognition, S.Z. Li & A.K. Jain, ed., Springer, Oct. 2003.
  • Automatic Recognition of Facial Actions in Spontaneous Expressions, M.S. Bartlett, G. Littlewort-Ford, Mark G. Frank, C. Lainscsek, I. Fasel,        T. Marks, E. Smith, T.J. Sejnowski, J.R. Movellan: Journal of Multimedia, Vol. 1(6), 2006 [pdf]
  • Combining Modality Specific Deep Neural Networks for Emotion Recognition in Video, S.E. Kahou et al. (Univ. Montreal / Y. Bengio); EmotiW Workshop 2013, Proceedings of the 15th ACM on International conference on multimodal interaction 2013 [pdf]
  • Combining Multiple Kernel Methods on Riemannian Manifold for Emotion Recognition in the Wild; Mengyi Liu, Ruiping Wang, Shaoxin Li, Shiguang Shan, Zhiwu Huang, Xilin Chen, EmotiW Workshop 2014, Proceedings of the 16th ACM on International conference on multimodal interaction 2014 [pdf]
  • Person Detection

  • Person Detection I

  • N. Dalal, B. Triggs, Histogram Of Oriented Gradients for Human Detection, CVPR 2005 [pdf].
  • N. Dalal, B. Triggs, C. Schmid, Human Detection Using Oriented Histograms of Flow and Appearance, ECCV 2006 [pdf].
  • D. Gavrila, Multi-feature Hierarchical Template Matching Using Distance Transforms, ICPR 1998 [pdf].
  • D. Gavrila, Real-Time Object Detection for Smart Vehicles, ICCV 1999 [pdf].
  • D. Gavrila (2000), Pedestrian Detection from a Moving Vehicle, ECCV 2000 [pdf].

  • Person Detection II

  • A. Mohan, C. Papageorgiu, T. Poggio, Example-Based Object Detection in Images by Componentes, PAMI 2001 [pdf].
  • K. Mikolajczyk, C. Schmid, A Performance Evaluation of Local Descriptors, PAMI 2005 [pdf].
  • E. Seemann, B. Leibe, K. Mikolajczyk, B. Schiele, An Evaluation of Local Shape-Based Features for Pedestrian Detection, BMVC 2005 [pdf].
  • B. Leibe, A. Leonardis, B. Schiele, Combined Object Categorization and Segmentation with an Implicit Shape Model, ECCV 2004 [pdf].
  • B. Leibe, A. Leonardis, B. Schiele, Robust Object Detection with Interleaved Categorization and Segmentation, IJCV [pdf].

  • Person Detection III

  • A. Angelova, A. Krizhevsky, V. Vanhoucke, A. Ogale, D. Ferguson, Real-Time Pedestrian Detection With Deep Network Cascades, BMVC 2015 [pdf]
  • N. Srivastava et al., Dropout: A Simply Way to Prevent Neural Networks from Overfitting, Journal of Machine Learning Research, 2014 [pdf]
  • J. Hosang, R. Benenson, P. Dollar, B. Schiele, What makes for effective detection proposals?, TPAMI 2015 [pdf]
  • J. Uijlings, K. can de Sande, A. Smeulders, Selective Search for Object Recognition, IJCV 2013 [pdf]
  • R. Girshick, J. Donahue, T. Darrell, J. Malik, Rich Feature Hierarchies for accurate Object Detection and Semantic Segmentation, CVPR 2014 [pdf]
  • R. Girshick, Fast R-CNN, ICCV 2015 [pdf]
  • S. Ren, K. He, R. Girshick, J. Sun, Faster R-CNN: Towards real-time Object Detection with Region Proposal Networks, NIPS 2015 [pdf]
  • W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C. Fu, A. Berg, SSD: Single Shot MultiBox Detector, ECCV 2016 [pdf]
  • R. Benenson, M. Omran, J. Hosang, B. Schiele, Ten Years of Pedestrian Detection, What Have We Learned?, ECCV 2014 [pdf]
  • J. Hosang, M.Omran, R. Benenson, B. Schiele, Taking a Deeper Look at Pedestrians, CVPR 2015 [pdf]
  • S. Zhang, R. Benenson, M. Omran, J. Hosang, B. Schiele, How Far Are We from Solving Pedestrian Detection?, CVPR 2016 [pdf]
  • L. Zhang, L. Lin, X. Liang, K. He, Is Faster R-CNN Doing Well for Pedestrian Detection?, ECCV 2016 [pdf]

Archiv

Vorlesung Computer Vision für Mensch-Maschine-Schnittstellen WS 2019/20 (keine Vorlesung)

Vorlesung Computer Vision für Mensch-Maschine-Schnittstellen WS 2018/19