Adversarial Self-Supervised Learning with Digital-Twins (Wintersemester 2021/2022)

Dozent: Prof. Dr. Holger Giese (Systemanalyse und Modellierung) , Christian Medeiros Adriano (Systemanalyse und Modellierung) , He Xu (Systemanalyse und Modellierung)

Allgemeine Information

• Semesterwochenstunden: 4
• ECTS: 6
• Benotet: Ja
• Einschreibefrist: 01.10.2021 - 22.10.2021
• Lehrform: Projektseminar
• Belegungsart: Wahlpflichtmodul
• Lehrsprache: Englisch

Studiengänge, Modulgruppen & Module

Beschreibung

The Tesla engineers manually specified 221 triggers that are activated in their fleet [1]. The  models run in a "shadow mode", which works as a digital twin [2-6] of the real system that allows to deal with disagreements between the camera and Lidar (e.g., bounding box jitter or distinct decisions from user and the model predictive control unit), which sent to the Tesla engineers. The engineers then "manually" analyze the data, apply an auto labelling mechanism and add this data to the training set. In other words, it is almost a closed AI loop, except for the engineering analysis. This corroborates the need for principled engineering methods. Even when these methods are explicit, like in Tesla example, they are evident in the involvement of the engineers to determine the prior-knowledge and analyze its consequence in ways (auto-labelling model) that informs further design decisions (inclusion of features to the training set).

Prediction models for decision-making can be efficiently (and safely) learned from data that is realistic enough. However, this would involve letting machine learning models learn in the production environment, which is costly and risky. One alternative is to generate environments that are as similar as possible as the production. This is how digital-twins play a role. That opens opportunities and further interesting challenges, like dealing with the differences between the simulated and real world environments. Among many alternatives, we will explore methods for domain randomization [7], generalization, and transfer-learning which work as adversarial environments [8].

Topics

• Theory
  • Cyber-Physical versus Cyber-Cyber Twins
  • Abstractions (leaky, coupling/decoupling)
  • Latent Models (Hidden Markov Models)
  • Theories of Out-of-Distribution Generalization
• Methodology
  • Transfer Learning (Meta-Learning, Sim2Real)
  • Adversarial and Model-Based Reinforcement Learning 
  • Robust and Safe Reinforcement Learning

 

Case studies (possible projects)

• Traffic Simulation 
• E-Commerce Platform  
• Meta-verse Applications

Literatur

1. Karpathy, A., 2021, - Closed AI Loop at Tesla for Labeling data that Trigger Corner Cases, Keynote at CVPR’21
2. Ahlgren, J., et al., 2021, Facebook’s Cyber–Cyber and Cyber–Physical Digital-Twins. 
3. Xia, K., et al., 2021, A digital twin to train deep reinforcement learning agent for smart manufacturing plants: Environment, interfaces and intelligence. Journal of Manufacturing Systems, 58, 210-230.
4. Rathore, M. M., et al., 2021, The Role of AI, Machine Learning, and Big Data in Digital Twinning: A Systematic Literature Review, Challenges, and Opportunities. IEEE Access, 9, 32030-32052.
5. Suhail, S., et al., 2021, Blockchain-based Digital Twins: Research Trends, Issues, and Future Challenges. 
6. Creating “Digital Twins” at scale, https://news.mit.edu/2021/creating-digital-twins-scale-0614 
7. Peng, X. B., et al., 2018, Sim-to-real transfer of robotic control with dynamics randomization, IEEE ICRA
8. Jiang, Y., 2021, Learning an Accurate Physics Simulator via Adversarial Reinforcement Learning 

Lern- und Lehrformen

The course is a project seminar, which has an introductory phase comprising initial short lectures. After that, the students will work in groups on jointly identified experiments applying specific solutions to given problems and finally prepare a presentation and write a report about their findings concerning the experiments.

We will organize this project seminar as a hybrid event, allowing students to participate online as well as in person.

After the registration with the Studienreferat via Moodle and before the first meeting, we will query the participants to check who prefers which format. Generally, we recommend in-person meetings for the introductory meeting and the final presentations and discussion, and an online format for intermediate lectures and project meetings, but we will try to accommodate all wishes. In case you have any questions, please contact christian.adriano(at)hpi.de

Leistungserfassung

We will grade the group's experiments (60%), reports (30%), and presentations (10%). Participation in the project seminar during meetings and other groups' presentations in the form of questions and feedback will also be required.

Termine

If you are interested in this course, please register with the Studienreferat (via Moodle poll) until 22.10. We consider your registration an expression of interest and will allow you to cancel your registration after the introductory meeting.

If you have any question on the course organization or want to register after the Studienreferat’s deadline, please contact christian.adriano(at)hpi.de

Start date: October 27 at 15:15

Room (for in-person): A 1.1

Zoom credentials (for online participants): link

Task assignments date: On November 23 and 24, we will jointly discuss the project tasks based on the topics and the students' individual interests.

Zurück