I am a postdoctoral researcher and leader of the "Deep Learning for Human Activity Recognition" subgroup at the KIT Chair "Computer Vision for Human-Computer Interaction". In my research, I aim to develop reliable, interpretable and data-efficient human activity recognition algorithms to solve impactful problems in autonomous driving and robotics.

I received my PhD (summa cum laude) focused on human activity analysis with special regard to classification uncertainties and open set conditions from Karlsruhe Institute of Technology in 2021, supervised by Prof. Dr.-Ing. Rainer Stiefelhagen and Prof. Dr. Mohan M. Trivedi. during which I also completed a PhD internship at Facebook Zurich. Before coming to Karlsruhe, I received my B.Sc. and M.Sc. degrees with distinction from the Technical University of Munich (2015) and worked as a data science consultant at MHP - A Porsche Company (2016-2017). I received several scholarships and awards, including the IV Best Student Paper Runner-Up Award, the second prize of the IEEE ITSS best dissertation award and the Google Summer of Code programming scholarship.

Please visit my Google Scholar page for the complete publications list.

Research

With the accuracy of CNNs gradually reaching the ceiling, the existing gap between the published methods and their applications in practice makes us wonder about important performance aspects being overlooked. In my research, I aim to go beyond the traditional goal of high top-1 accuracy on a static carefully designed classes, and develop human activity recognition models which do not only  (1) assign the correct category, but also (2) reliably identify incorrect predictions and unknown situations and (3) trace back their causes of failure, and (4) have tools for dealing with new concepts-of-interest without costly labelling. I am especially interested in developing algorithms to solve impactful problems in challenging open-world applications, such as autonomous driving and robotics. My main research topics are:

• Human Activity Analysis [paper1] [paper2] [paper3]
• Interpretable and Uncertainty-aware Visual Recognition [paper1] [paper2] [paper3]
• Open Set and Open Domain Recognition [paper1] [paper2] [paper3]
• Data-efficient Recognition, Zero- and Few-Shot-Learning [paper1] [paper2] [paper3]
• Computer Vision for Robotics [paper1] [paper2] [paper3]
• Applications in Intelligent Vehicles [paper1] [paper2] [paper3]

Short CV

• Since April 2021: Postdoctoral Researcher - Lead of the Activity Recognition Group at CV:HCI.
• PhD at Karlsruhe Institute of Technology (summa cum laude), April 2017 - April 2021. Topic: "Uncertainty-aware Models for Deep Learning-based Human Activity Recognition and Applications in Intelligent Vehicles." Referents: Prof. Dr.-Ing. Rainer Stiefelhagen (KIT, IAR) und Prof. Dr. Mohan Trivedi (University of San Diego). The manuscript is available here. IEEE ITSS Best Dissertation Award 2021 (second prize).
• PhD internship at Facebook Zurich - Computer Vision for AR/VR, September 2020 - November 2020.
• Consultant at MHP - A Porsche Company, Big Data & Analytics, 2016 - 2017.
• M.Sc. in Computer Science at Technical University Munich, 2013 - 2015.
• Student Assistant at fortiss GmbH, developing human-robot-interfaces for industrial robotics, 2014 - 2015.
• Erasmus studies at Chalmers University of Technology, Gothenburg, Sweden, 2012 - 2013.
• B.Sc. in Computer Science at Technical University Munich, 2009 - 2012.

Scholarships & Awards

• Helmholtz Doctoral Prize for mission-oriented research in the field "Information", April 2022.
• KIT Doctoral Award, Dec 2021.
• IEEE ITSS Best Dissertation Award 2021 (second prize), Sep 2021.
• Best Student Paper First Runner Up Award, Intelligent Vehicles Symposium, Awarded Paper: "Open Set Driver Activity Recognition", Jan 2020.
• ECCV Women in Computer Vision Travel Grant awarded for the presentation "Towards Human Activity Recognition in Autonomous Vehicles", 2018.
• Deutschlandstipendium - German national scholarship for talented students, 2014.
• Google Summer of Code, a programming scholarship awarded by Google to support students working on open-source software (open-source project: Point Cloud Library), 2014.
• School-time - three awards for the second and one award for the third place in the Ukrainian Mathematical Olympiadas (7th and 8th grade).

At cv:hci I am leading the "Deep Learning for Human Activity Recognition" subgroup, where I am supervising multiple Ph.D. researchers, Master- and Bachelor students.

Ph.D. Students

• Kunyu Peng, M.Sc.
• David Schneider, M.Sc.

Current Master and Bachelor Students

• Zdravko Marinov, "Deep Learning-based Unsupervised Domain Adaptation for Activity Recognition from Synthetic Training Data", Master Thesis, ongoing.
• Calvin Tanama, "Data distillation for efficient action recognition with student-teacher networks", Bachelor Thesis, ongoing.
• Aulia Djamal, "Synthetic-to-Real Human Activity Recognition", Student Assistant.
• Shihao Xu, "Generating Action-Pose-Sequences with minimal Constraints", Master Thesis (co-supervision), ongoing.

Finished Supervised Theses

• Vincent Pfäfflin, "Curriculum Learning for Human Activity Recognition with Imbalanced Training Data", Bachelor Thesis, 08.2021
• Simon Reiß, "Zero-Shot Recognition of Composite Activities in Context of Driver Observation", Master Thesis, 02.2020
• Yilin Ji, "Machine Learning for Situation Analysis of Automated Vehicle using Small Data Sets", 07.2019, (co-supervision, thesis conducted at BOSCH).
• Chaoxiang Ma, "Reliability of Deep Convolutional Neural Networks for Activity Recognition", Bachelor Thesis, 01.2019
• Patrick Gebert, "Vehicle Maneuver Prediction with 3D Convolutional Neural Networks Based on Driver Observation", Master Thesis, 07.2018
• Tim Pollert, "Fusion Methods for Multimodal Gesture Recognition with Convolutional Neural Networks", Master Thesis, 06.2018

Interested in Bachelor/Master Thesis?

If you are passionate about one of {machine learning, deep learning, computer vision} and want to apply what you have learned in the area of activity recognition / video comprehension in your thesis - please send me an email with a few sentences about yourself.

Projects

• CC-King - Comptence Center KI-Engineering (2020 - ongoing).
• JuBot - Jung bleiben mit Robotern (Carl-Zeiss-Stiftung, 2020 - ongoing).
• SmartAge - Untersuchung intelligenter Formen von Selbstregulation und Ko-Regulation unter Realbedingungen (Carl-Zeiss-Stiftung, 2021 - ongoing).
• ESKIMO - Mit künstlicher Intelligenz die Baustelle genau im Blick (BMBF, 2020 - ongoing).
• PAKoS - Personalisierte, adaptive kooperative Systeme für automatisierte Fahrzeuge (BMBF, 2017-2021).

• Deep Learning für Computer Vision I: Basics
• Deep Learning für Computer Vision II: Advanced Topics
• Practical Course Computer Vision for Human-Computer Interaction
• Before KIT: throught my Bachelor and Master studies, I have been tutor in the in the practical course "Fundamentals of Programming" (three times) and the preparation course "Mathematics for Computer Scientists".
• Mentoring: I am happy to support computer science students at TUM as a mentor in the scope of the TUM Mentoring Program.

Latest News

04.2022: I am very honored to receive the Helmholtz Doctoral Prize in the research field "Information", a yearly prize awarded for mission-oriented research by the Helmholtz Association.

12.2021: Many great news! One paper on emotion recognition accepted at FG 2021 and two papers on scene understanding for automated driving accepted at the top autonomous vehicles journal (Transactions on Intelligent Trasportation Systems, paper 1 and paper 2.) Besides, I am extremely happy to receive this year's KIT doctoral award!

22.09.2021: I am very honored to receive the second prize of the yearly IEEE Intelligent Transportation Systems Society best PhD dissertation award at this year's ITSC conference.

09.2021: I gave an invited talk on recognizing daily living activities by learning from synthetic data at the iGibson team of Stanford Vision and Learning Lab. Check out our recent IROS paper on this topic.

04.2021: I defended my PhD titled "Uncertainty-aware Models for Deep Learning-based Human Activity Recognition and Applications in Intelligent Vehicles" with distinction (summa cum laude). I am now excited to continue my research leading the cv:hci human activity recognition group.

06.2021: Papers accepted at IROS, IV and ITSC! One paper on SyntheticReal recognition of daily living activities has been accepted at IROS. Our paper about learning to predict driver intent by learning from driving exam dialogs was accepted at IV. Our ITSC paper improves domain adaptation capabilities of semantic segmentation models when moving from conventional to 360° data.

12.2020: Two great news: our paper "Open Set Driver Activity Recognition" has received Best Student Paper Runner-up Award at IV 2020! Furthermore, we have published an overview what we achieved in the BMBF PAKoS project in a Springer book chapter.

09.2020: I am on a three months leave at Facebook, completing a PhD internship in Computer Vision for AR/VR.

• Is my Model Overconfident? Uncertainty-aware Driver Observation with Reliable and Interpretable Confidence Estimates. Alina Roitberg, Kunyu Peng, David Schneider, Marios Koulakis, Kailun Yang, Manuel Martinez, Rainer Stiefelhagen. Submitted to IEEE Transactions on Intelligent Transportation Systems, 2022 (under review) [preprint].
• ProFormer: Learning Data-efficient Representations of Body Movement with Prototype-based Feature Augmentation and Visual Transformers. Kunyu Peng, Alina Roitberg, Kailun Yang, Jiaming Zhang, Rainer Stiefelhagen. 2022 (under review) [preprint].
• A Comparative Analysis of Decision-Level Fusion for Multimodal Driver Behaviour Understanding. Alina Roitberg, Kunyu Peng, Zdravko Marinov, Constantin Seibold, David Schneider Rainer Stiefelhagen. IEEE Intelligent Vehicles Symposium (IV), IEEE, June 2022 (to appear) [pdf].
• Should I take a walk? Estimating Energy Expenditure from Video Data. Kunyu Peng*, Alina Roitberg*, Kailun Yang, Jiaming Zhang, Rainer Stiefelhagen. CVPR Workshop on Computer Vision for Physiological Measurement, IEEE, June 2022 (to appear) [pdf] [code].
• MASS: Multi-Attentional Semantic Segmentation of LiDAR Data for Dense Top-View Understanding. K. Peng, J. Fei, K. Yang, A. Roitberg, J. Zhang, F. Bieder, P. Heidenreich, C. Stiller, R. Stiefelhagen. IEEE Transactions on Intelligent Transportation Systems, 2022 [pdf] [code]
• Transfer beyond the Field of View: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation. Jiaming Zhang, Chaoxiang Ma, Kailun Yang, Alina Roitberg, Kunyu Peng, Rainer Stiefelhagen. IEEE Transactions on Intelligent Transportation Systems, 2022 [pdf] [data]
• Affect-DML: Context-Aware One-Shot Recognition of Human Affect using Deep Metric Learning. Kunyu Peng, Alina Roitberg, David Schneider, Marios Koulakis, Kailun Yang, Rainer Stiefelhagen. International Conference on Automatic Face and Gesture Recognition, IEEE, December 2021.
• Let's Play for Action: Recognizing Activities of Daily Living by Learning from Life Simulation Video Games. Alina Roitberg*, David Schneider*, Aulia Djamal, Constantin Seibold, Simon Reiß, Rainer Stiefelhagen. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic, September 2021 [pdf] [website]
• From Driver Talk To Future Action: Vehicle Maneuver Prediction by Learning from Driving Exam Dialogs. Alina Roitberg, Simon Reiß, Rainer Stiefelhagen. IEEE Intelligent Vehicles Symposium, Nagoya, Japan (virtual), July 2021 [pdf]
• DensePASS: Dense Panoramic Semantic Segmentation via Unsupervised Domain Adaptation with Attention-Augmented Context Exchange. Chaoxiang Ma, Jiaming Zhang, Kailun Yang, Alina Roitberg, Kunyu Peng, Rainer Stiefelhagen. IEEE International Conference on Intelligent Transportation Systems (ITSC), 2021 [pdf] [website]
• Uncertainty-aware Models for Deep Learning-based Human Activity Recognition and Applications in Intelligent Vehicles. Alina Roitberg. PhD Dissertation, April 2021 [pdf]
• Uncertainty-sensitive Activity Recognition: a Reliability Benchmark and the CARING Models. Alina Roitberg, Monica Haurilet, Manuel Martinez and Rainer Stiefelhagen. International Conference on Pattern Recognition (ICPR),  IEEE, January 2021, oral (<10% acceptance rate for orals). [pdf]
• Multi-Task Learning for Calorie Prediction on a Novel Large-Scale Recipe Dataset Enriched with Nutritional Information. Robin Ruede, Verena Heusser, Lukas Frank, Alina Roitberg, Monica Haurilet, Rainer Stiefelhagen. International Conference on Pattern Recognition (ICPR),  IEEE, January 2021. [pdf] [press article]
• Personalisation and Control Transition Between Automation and Driver in Highly Automated Cars. M. Flad, P. Karg, P, A. Roitberg, M. Martin, M. Mazewitsch, C. Lange, E. Kenar, L. Ahrens, B. Flecken, L. Kalb, B. Karakaya, J. Ludwig, A. Pruksch, R. Stiefelhagen, S. Hohmann. Smart Automotive Mobility. Human–Computer Interaction Series. Springer (book chapter), 2020 [doi]
• Open Set Driver Activity Recognition. Alina Roitberg, Chaoxiang Ma, Monica Haurilet and Rainer Stiefelhagen. Intelligent Vehicles Symposium (IV),  IEEE, October 2020, Best Student Paper 2nd Place Award. [pdf]
• Deep Classification-driven Domain Adaptation for Cross-Modal Driver Behavior Recognition. Simon Reiß*, Alina Roitberg*, Monica Haurilet, and Rainer Stiefelhagen. Intelligent Vehicles Symposium (IV),  IEEE, October 2020. [pdf]
• CNN-based Driver Activity Recognition: Shedding Light on Deep Spatiotemporal Representations. Alina Roitberg, Monica Haurilet, Simon Reiß and Rainer Stiefelhagen. International Conference on Intelligent Transportation Systems (ITSC),  IEEE, September 2020. [pdf]
• Activity-aware Attributes for Zero-Shot Driver Behavior Recognition. Simon Reiß*, Alina Roitberg*, Monica Haurilet, and Rainer Stiefelhagen. In CVPR Workshop on Visual Learning with Limited Labels (VL-LL). IEEE, June 2020. [pdf]
• Drive&Act: A Multi-modal Dataset for Fine-grained Driver Behavior Recognition in Autonomous Vehicles. Manuel Martin*, Alina Roitberg*, Monica Haurilet, Matthias Horne, Simon Reiß, Michael Voit, Rainer Stiefelhagen. In International Conference on Computer Vision (ICCV), IEEE, Seoul, South Korea, Oct. 2019 [pdf] [bib] (25.0% acceptance rate, * denotes equal contribution)
• WiSe - Slide Segmentation in the Wild. Monica Haurilet, Alina Roitberg, Manuel Martinez and Rainer Stiefelhagen. In International Conference for Document Analysis and Recognition, (ICDAR), IEEE, Sydney, Australia, Sep. 2019 [pdf]
• Learning Fine-Grained Image Representations for Mathematical Expression Recognition. Sidney Bender*, Monica Haurilet*, Alina Roitberg, Rainer Stiefelhagen, ICDAR Workshop on Graphics Recognition, Sydney, Australia, Sep. 2019 [pdf] (* denotes equal contribution)
• End-to-end Prediction of Driver Intention using 3D Convolutional Neural Networks. Patrick Gebert*, Alina Roitberg*, Monica Haurilet and Rainer Stiefelhagen. In Intelligent Vehicles Symposium (IV), IEEE, Paris, France, June 2019 (* denotes equal contribution)
• It’s not about the Journey; It’s about the Destination: Following Soft Paths under Question-Guidance for Visual Reasoning. Monica Haurilet, Alina Roitberg, Rainer Stiefelhagen.  In Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Long Beach, USA, June 2019 [pdf] (25.2% acceptance rate)
• Analysis of Deep Fusion Strategies for Multi-modal Gesture Recognition. Alina Roitberg*, Tim Pollert*, Monica Haurilet, Manuel Martin, Rainer Stiefelhagen. CVPR Workshop on Analysis and Modeling of Faces and Gestures (AMFG), IEEE, Long Beach, USA, June 2019 [pdf][bib](* denotes equal contribution)
• Informed Democracy: Voting-based Novelty Detection for Action Recognition. Alina Roitberg, Ziad Al-Halah, Rainer Stiefelhagen.  In British Machine Vision Conference (BMVC), Newcastle upon Tyne, UK, 2018 [pdf][poster][bib] (29.5% acceptance rate)
• Towards a Fair Evaluation of Zero-Shot Action Recognition using External Data. Alina Roitberg, Manuel Martinez, Monica Haurilet, Rainer Stiefelhagen.  In ECCV Workshop on Shortcomings in Vision and Language (SiVL), Springer, Munich, Germany, 2018 [pdf][poster][bib] (spotlight presentation)
• Estimating Mental Load in Passive and Active Tasks from Pupil and Gaze Changes using Bayesian Surprise. Elena Wolf, Manuel Martinez, Alina Roitberg, Rainer Stiefelhagen and Barbara Deml. In ACM ICMI Modeling Cognitive Processes from Multimodal Data Workshop (ICMI-MCPMD), Boulder, Colorado, USA, 2018
• Using Technology Developed for Autonomous Cars to Help Navigate Blind People. Manuel Martinez, Alina Roitberg, Daniel Koester, Boris Schauerte, Rainer Stiefelhagen. ICCV Workshop on Assistive Computer Vision and Robotics (ACVR), 2017
• Connecting Artificial Brains to Robots in a Comprehensive Simulation Framework: The Neurorobotics Platform. Egidio Falotico, Lorenzo Vannucci, Alessandro Ambrosano, Ugo Albanese, Stefan Ulbrich, Juan Camilo Vasquez Tieck, Georg Hinkel, Jacques Kaiser, Igor Peric, Oliver Denninger, Nino Cauli, Murat Kirtay, Arne Roennau, Gudrun Klinker, Axel Von Arnim, Luc Guyot, Daniel Peppicelli, Pablo Martínez-Cañada, Eduardo Ros, Patrick Maier, Sandro Weber, Manuel Huber, David Plecher, Florian Röhrbein, Stefan Deser, Alina Roitberg, Patrick van der Smagt, Rüdiger Dillman, Paul Levi, Cecilia Laschi, Alois C Knoll, Marc-Oliver Gewaltig, Frontiers in neurorobotics 11, 2017
• Improved skeleton estimation by means of depth data fusion from multiple depth cameras. Marco Carraro, Matteo Munaro, Alina Roitberg, Emanuele Menegatti. In International Conference on Intelligent Autonomous Systems, Springer, 2016
• Multimodal human activity recognition for industrial manufacturing processes in robotic workcells. Alina Roitberg, Nikhil Somani, Alexander Perzylo, Markus Rickert, and Alois Knoll. In ACM International Conference on Multimodal Interaction (ICMI), Seattle, USA, 2015
• Human Activity Recognition in the Context of Industrial Human-Robot Interaction. Alina Roitberg, Alexander Perzylo, Nikhil Somani, Manuel Giuliani, Markus Rickert, and Alois Knoll. In Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC 2014), IEEE, Siem Reap, Cambodia, 2014