Alina Roitberg

Dr.-Ing. Alina Roitberg


04.2021: I defended my PhD :).

12.2020: Two great news: our paper "Open Set Driver Activity Recognition" has received Best Student Paper Runner-up Award at IV 2020! Furthermore, we have published an overview what we achieved in the BMBF PAKoS project in a Springer book chapter.

09.2020: I am on a three months leave at Facebook, completing a PhD internship in Computer Vision for AR/VR.

06.2020: Five papers accepted at IV, ICPR, ITSC, and CVPR Workshop! One paper on open set- and one on cross-modal driver activity recognition were accepted at Intelligent Vehicles Symposium (IV). One paper about learning to obtain proper confidence estimates of activity recognition models was accepted at International Conference on Patern Recognition (ICPR). Our Intelligent Transportation Systems Conference (ITSC) paper aims at interpretable driver activity recognition models and our fifth paper accepted at CVPR Workshop presents a novel benchmark for zero-shot driver activity recognition. Preprints follow soon!

08.2019: Our paper featuring the novel Drive&Act dataset for fine-grained driver activity recognition was accepted at ICCV!

06.2019: Four papers accepted at IV, ICDAR and CVPR Workshop! One paper on forseeing future car maneuvers through driver observation is accepted at Intelligent Vehicles Symposium and one paper with a comprehensive analysis of different strategies for multimodal fusion will be published at the CVPR Workshop on Modelling of Faces and Gestures. Our paper featuring a new dataset for slide segmentation from live fotos taken during lectures, and our fourth paper on mathematical expression recognition are accepted at is accepted at ICDAR and ICDAR GREC Workshop.

03.2019: Our paper about Visual Reasoning using Graph Neural Networks was accepted at CVPR!

10.2018: Our interdisciplinary paper on pupil-based assesment of human cognitive load using Bayesian methods was presented at the ACM ICMI Modeling Cognitive Processes from Multimodal Data Workshop (ICMI-MCPMD).

08.2018:   Two papers accepted at BMVC and ECCV Workshop! One paper about identifying previously unseen classes for action recognition in open-set scenario is accepted at the BMVC and one paper on assessing the performance of zero-shot action recognition algorithms when using external datasets was accepted at the ECCV Workshop on Shortcomings in Vision and Language.

07.2018:   I participated in the International Computer Vision Summer School and presented our poster "Human Activity Recognition in Autonomous Vehicles"

Research Interests

My main research focus is deep learning-based recognition of human activities and interactions from video streams. I am especially interested in open-set, zero- and few-shot recognition as well as applying the developed models for driver monitoring in intelligent vehicles.

I am involved in the BMBF Project PAKoS (Personalisierte, adaptive kooperative Systeme für automatisierte Fahrzeuge), working on scene understanding inside autonomous cars. The goal of CV:HCI is real-time activity recognition under realistic driving conditions (e.g. varying or low illumination).


Bachelor/Master Thesis

If you are passionate about one of {machine learning, deep learning, computer vision} and want to apply what you have learned in the area of activity recognition / video comprehension in your thesis - please send me an email with a few sentences about yourself.

Supervised Theses

  • Simon Reiß, "Zero-Shot Recognition of Composite Activities in Context of Driver Observation", Master Thesis, 02.2020
  • Yilin Ji, "Machine Learning for Situation Analysis of Automated Vehicle using Small Data Sets", 07.2019, (co-supervision, thesis conducted at BOSCH).
  • Chaoxiang Ma, "Reliability of Deep Convolutional Neural Networks for Activity Recognition", Bachelor Thesis, 01.2019
  • Patrick Gebert, "Vehicle Maneuver Prediction with 3D Convolutional Neural Networks Based on Driver Observation", Master Thesis, 07.2018
  • Tim Pollert, "Fusion Methods for Multimodal Gesture Recognition with Convolutional Neural Networks", Master Thesis, 06.2018

Short CV

  • Postdoctodal Researcher - Lead of the Activity Recognition Group at CV:HCI - since April 2021
  • PhD Researcher at Computer Vision for Human-Computer Interaction Lab, Karlsruhe Institute of Technology, April 2017 -  defense in April 2021 (summa cum laude)
  • Consultant at MHP - A Porsche Company, Big Data & Analytics, 2016 - 2017
  • M.Sc. in Computer Science at Technical University Munich, 2013 - 2015
  • Erasmus studies at Chalmers University of Technology, Gothenburg, Sweden, 2012 - 2013
  • B.Sc. in Computer Science at Technical University Munich, 2009 - 2012


  • DensePASS: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation with Attention-Augmented Context Exchange. Chaoxiang Ma, Jiaming Zhang, Kailun Yang, Alina Roitberg, Kunyu Peng, Rainer Stiefelhagen. IEEE International Conference on Intelligent Transportation Systems (ITSC) [pdf] [website]
  • Let's Play for Action: Recognizing Activities of Daily Living by Learning from Life Simulation Video Games. Alina Roitberg*, David Schneider*, Aulia Djamal, Constantin Seibold, Simon Reiß, Rainer Stiefelhagen. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic, September 2021 [pdf] [website]
  • From Driver Talk To Future Action: Vehicle Maneuver Prediction by Learning from Driving Exam Dialogs. Alina Roitberg, Simon Reiß, Rainer Stiefelhagen. IEEE Intelligent Vehicles Symposium, Nagoya, Japan (virtual), July 2021 [pdf]
  • Uncertainty-sensitive Activity Recognition: a Reliability Benchmark and the CARING Models. Alina Roitberg, Monica Haurilet, Manuel Martinez and Rainer Stiefelhagen. International Conference on Pattern Recognition (ICPR),  IEEE, January 2021. [pdf]
  • Multi-Task Learning for Calorie Prediction on a Novel Large-Scale Recipe Dataset Enriched with Nutritional Information. Robin Ruede, Verena Heusser, Lukas Frank, Alina Roitberg, Monica Haurilet, Rainer Stiefelhagen. International Conference on Pattern Recognition (ICPR),  IEEE, January 2021. [pdf] [press article]
  • Personalisation and Control Transition Between Automation and Driver in Highly Automated Cars. M. Flad, P. Karg, P, A. Roitberg, M. Martin, M. Mazewitsch, C. Lange, E. Kenar, L. Ahrens, B. Flecken, L. Kalb, B. Karakaya, J. Ludwig, A. Pruksch, R. Stiefelhagen, S. Hohmann. Smart Automotive Mobility. Human–Computer Interaction Series. Springer (book chapter), 2020 [doi]
  • Open Set Driver Activity Recognition. Alina Roitberg, Chaoxiang Ma, Monica Haurilet and Rainer Stiefelhagen. Intelligent Vehicles Symposium (IV),  IEEE, October 2020, Best Student Paper 2nd Place Award. [pdf]
  • Deep Classification-driven Domain Adaptation for Cross-Modal Driver Behavior Recognition. Simon Reiß*, Alina Roitberg*, Monica Haurilet, and Rainer Stiefelhagen. Intelligent Vehicles Symposium (IV),  IEEE, October 2020. [pdf]
  • CNN-based Driver Activity Recognition: Shedding Light on Deep Spatiotemporal Representations. Alina Roitberg, Monica Haurilet, Simon Reiß and Rainer Stiefelhagen. International Conference on Intelligent Transportation Systems (ITSC),  IEEE, September 2020. [pdf]
  • Activity-aware Attributes for Zero-Shot Driver Behavior Recognition. Simon Reiß*, Alina Roitberg*, Monica Haurilet, and Rainer Stiefelhagen. In CVPR Workshop on Visual Learning with Limited Labels (VL-LL). IEEE, June 2020. [pdf]
  • Drive&Act: A Multi-modal Dataset for Fine-grained Driver Behavior Recognition in Autonomous Vehicles. Manuel Martin*, Alina Roitberg*, Monica Haurilet, Matthias Horne, Simon Reiß, Michael Voit, Rainer Stiefelhagen. In International Conference on Computer Vision (ICCV), IEEE, Seoul, South Korea, Oct. 2019 [pdf] [bib] (* denotes equal contribution)
  • WiSe - Slide Segmentation in the Wild. Monica Haurilet, Alina Roitberg, Manuel Martinez and Rainer Stiefelhagen. In International Conference for Document Analysis and Recognition, (ICDAR), IEEE, Sydney, Australia, Sep. 2019 [pdf]
  • Learning Fine-Grained Image Representations for Mathematical Expression Recognition. Sidney Bender*, Monica Haurilet*, Alina Roitberg, Rainer Stiefelhagen, ICDAR Workshop on Graphics Recognition, Sydney, Australia, Sep. 2019 [pdf] (* denotes equal contribution)
  • End-to-end Prediction of Driver Intention using 3D Convolutional Neural Networks. Patrick Gebert*, Alina Roitberg*, Monica Haurilet and Rainer Stiefelhagen. In Intelligent Vehicles Symposium (IV), IEEE, Paris, France, June 2019 (* denotes equal contribution)
  • It’s not about the Journey; It’s about the Destination: Following Soft Paths under Question-Guidance for Visual Reasoning. Monica Haurilet, Alina Roitberg, Rainer Stiefelhagen.  In Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Long Beach, USA, June 2019 [pdf] (25.2% acceptance rate)
  • Analysis of Deep Fusion Strategies for Multi-modal Gesture Recognition. Alina Roitberg*, Tim Pollert*, Monica Haurilet, Manuel Martin, Rainer Stiefelhagen. CVPR Workshop on Analysis and Modeling of Faces and Gestures (AMFG), IEEE, Long Beach, USA, June 2019 [pdf][bib](* denotes equal contribution)
  • Informed Democracy: Voting-based Novelty Detection for Action Recognition. Alina Roitberg, Ziad Al-Halah, Rainer Stiefelhagen.  In British Machine Vision Conference (BMVC), Newcastle upon Tyne, UK, 2018 [pdf][poster][bib] (29.5% acceptance rate)
  • Towards a Fair Evaluation of Zero-Shot Action Recognition using External Data. Alina Roitberg, Manuel Martinez, Monica Haurilet, Rainer Stiefelhagen.  In ECCV Workshop on Shortcomings in Vision and Language (SiVL), Springer, Munich, Germany, 2018 [pdf][poster][bib] (spotlight presentation)
  • Estimating Mental Load in Passive and Active Tasks from Pupil and Gaze Changes using Bayesian Surprise. Elena Wolf, Manuel Martinez, Alina Roitberg, Rainer Stiefelhagen and Barbara Deml. In ACM ICMI Modeling Cognitive Processes from Multimodal Data Workshop (ICMI-MCPMD), Boulder, Colorado, USA, 2018
  • Using Technology Developed for Autonomous Cars to Help Navigate Blind People. Manuel Martinez, Alina Roitberg, Daniel Koester, Boris Schauerte, Rainer Stiefelhagen. ICCV Workshop on Assistive Computer Vision and Robotics (ACVR), 2017
  • Connecting Artificial Brains to Robots in a Comprehensive Simulation Framework: The Neurorobotics Platform. Egidio Falotico, Lorenzo Vannucci, Alessandro Ambrosano, Ugo Albanese, Stefan Ulbrich, Juan Camilo Vasquez Tieck, Georg Hinkel, Jacques Kaiser, Igor Peric, Oliver Denninger, Nino Cauli, Murat Kirtay, Arne Roennau, Gudrun Klinker, Axel Von Arnim, Luc Guyot, Daniel Peppicelli, Pablo Martínez-Cañada, Eduardo Ros, Patrick Maier, Sandro Weber, Manuel Huber, David Plecher, Florian Röhrbein, Stefan Deser, Alina Roitberg, Patrick van der Smagt, Rüdiger Dillman, Paul Levi, Cecilia Laschi, Alois C Knoll, Marc-Oliver Gewaltig, Frontiers in neurorobotics 11, 2017
  • Improved skeleton estimation by means of depth data fusion from multiple depth cameras. Marco Carraro, Matteo Munaro, Alina Roitberg, Emanuele Menegatti. In International Conference on Intelligent Autonomous Systems, Springer, 2016
  • Multimodal human activity recognition for industrial manufacturing processes in robotic workcells. Alina Roitberg, Nikhil Somani, Alexander Perzylo, Markus Rickert, and Alois Knoll. In ACM International Conference on Multimodal Interaction (ICMI), Seattle, USA, 2015
  • Human Activity Recognition in the Context of Industrial Human-Robot Interaction. Alina Roitberg, Alexander Perzylo, Nikhil Somani, Manuel Giuliani, Markus Rickert, and Alois Knoll. In Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC 2014), IEEE, Siem Reap, Cambodia, 2014