Advanced Planning for Self-Aware Systems (Wintersemester 2018/2019)

Dozent: Prof. Dr. Holger Giese (Systemanalyse und Modellierung) , Sona Ghahremani (Systemanalyse und Modellierung) , Christian Schäffer (Systemanalyse und Modellierung)

Allgemeine Information

• Semesterwochenstunden: 2
• ECTS: 3
• Benotet: Ja
• Einschreibefrist: 26.10.2018
• Lehrform: Seminar
• Belegungsart: Wahlpflichtmodul
• Lehrsprache: Deutsch

Studiengänge, Modulgruppen & Module

Beschreibung

Self-aware systems make decisions at run time that changes their behavior in response to faults, changes of the system or its environments and attacks. Therefore, having an appropriate planning approach to find an adaptation plan is critical to successful self-adaptation. For many realistic systems designing planner that is both quick and finds an optimal adaptation plan is ideal. However, for complex  real-world systems in dynamic environments designing such planners offline is hardly feasible. In this seminar we are interested in studying learning-based planning techniques for self-aware systems.

Self-aware systems are a reality in various types of technology, from auto-pilots to energy saving power grids. Self-aware systems are able to autonomously adapt to change in their environment, e.g., wind speed changes affecting airplane lift to a grid power line interruption. Self-aware systems monitor these types of events and react to compensate their impact on the system goals [1]. To do so, self-awareness employs MAPE-K feedback loops that consist of four basic activities of monitoring, analyzing, planning, and executing actions on a share Knowledge base [2]. Monitoring and Analysis observe the changes and analyze whether the the system needs to react. Planning makes the decisions to adapt the system to address the changes. Execute enacts the potential adaptation decisions on the system.

Machine learning brought many advantages to planning (decision making) of self-aware systems [3]. For complex plans, learning might be easier than having a designer craft a set of differential equations and coding them appropriately. While learning a complex function instead of coding it is shown to be feasible [4], learning can be costly and uncertainty when the environment changes. Environment can change incrementally or the system can be deployed on a new environment. For incremental changes, we will investigate online learning. For the deployment on new environment, we will investigate transfer learning. For collaborative leaning in which a collective of self-aware systems are jointly learning to adapt we investigate collaborative and collective learning. These approaches are not exclusive. They can be combined when an adaptation learned in one environment could be seamlessly transferred to other environments.

Seminar Topics

After a set of introductory lectures, the students will be offered a choice of three main focus areas :

I) Individual online learning

Online learning [5] consists of learning a model ( as an individual agent) after it was deployed on production. This is a ubiquitous reality in businesses that needs to adapt to changes in their environments, e.g., new consumers' preferences, new commuting patterns, or new knowledge about a search space. Since these environment changes cannot be foreseen at design time, online learning is required to address them. A list of challenges for online learning can be as the following:

• Non-obvious timing to detect the change and to react to it (e.g., when the adaptation can be safely deployed? i.e., when to start and to stop learning?) 
• Unknown cost to adapt to the change (e.g., how much past may be needed in the future? i.e., how much data and what type of data need to be collected for learning?) 
• Uncertain quality of the system goal that one need to pursue (e.g., what is the relation between model accuracy and learning speed?

II) Collaborative learning

Collaborative learning techniques enable the collective decision making among multiple learning systems [6,7]. In such settings, learning is distributed among more than one self-aware system ( agent). Each system, while trying to fulfill its own goals, contributes to a global goal of the whole. Some of the interesting topics in this area can be listed as:

• Types of self-aware systems than can employ collaborative learning
• Techniques and best practices for collaborative learning in self-aware systems
• Characteristics of self-aware systems that challenges collaborative learning

III) Transfer Learning

A self-aware system can learn how to adapt in order to fulfill a task in a certain environment e.g., a self-driving car that can learn to drive in a certain city such as Berlin. To operate that system in a different environment, ( i.e. driving that car in e.g., London), depending on the similarity between the two environments, still certain adaptation knowledge that has already been learned can be applicable, this is referred to as transfer learning [8, 9] . However, it is unavoidable that the system needs to re-learn parts of its adaptation knowledge to be able to adapt in the new environment. The self-driving car an use its knowledge on basics of driving, but still need to further enrich its knowledge for driving rules in London. Certain challenges we see for such technique is as listed bellow:

• How to decide which part of the knowledge is transferable between self-aware systems ?
• In case there is more than one self-aware system with matured adaptation knowledge ( converged learning), how much knowledge should be transferred from each systems? What are the criteria to decide?

PhD student Sona Ghahremani will be responsible to coordinate the research groups. In case you are interested in more details, please get in touch here or come over to room A-2.9.

Literatur

1.  S. Kounev, J. O. Kephart, A. Milenkoski, and X. Zhu, Eds., Self-Aware

   Computing Systems. Berlin Heidelberg, Germany: Springer Verlag, 2017.

2. IBM, “An Architectural Blueprint for Autonomic Computing,” 2006,

   white Paper, Fourth Edition.

3. S. Kounev, J. O. Kephart, P. Lewis, A. Diaconescu, K. Geihs,

   H. Giese, P. Inverardi, A. Zisman, K. Bellman, L. Esterle,

   N. Bencomo, H. Schmeck, H. Hoffmann, S. Götz, and

   J. Camara, The Notion of Self-Aware Computing. Berlin Heidelberg,

   Germany: Springer Verlag, 2017, pp. 3–16.

4. S. Ghahremani, C. M. Adriano, and H. Giese, “Training Prediction

   Models for Rule-based Self-adaptive Systems,” in 2018 IEEE International

   Conference on Autonomic Computing (ICAC), 2018, p. To Appear.

5. D. Kim and S. Park, “Reinforcement learning-based dynamic adaptation

   planning method for architecture-based self-managed software,” in 2009

   ICSE Workshop on Software Engineering for Adaptive and Self-Managing

   Systems, May 2009, pp. 76–85.

6. L. Panait and S. Luke, “Cooperative Multi-Agent Learning: The State of the

   Art,” Autonomous Agents and Multi-Agent Systems, vol. 11, no. 3, pp. 387–

   434, November 2005.

7. P. J. t. Hoen, K. Tuyls, L. Panait, S. Luke, and J. A. La Poutr’e, “An Overview

   of Cooperative and Competitive Multiagent Learning,” in Proceedings of the

   First International Conference on Learning and Adaption in Multi-Agent

   Systems, ser. LAMAS’05. Berlin, Heidelberg: Springer-Verlag, 2006, pp.

   1–46.

8. P. Jamshidi, M. Velez, C. Kästner, N. Siegmund, and P. Kawthekar, “Transfer

   Learning for Improving Model Predictions in Highly Configurable Software,”

   2017 IEEE/ACM 12th International Symposium on Software Engineering for

   Adaptive and Self-Managing Systems (SEAMS), pp. 31–41, 2017.

9.  S. J. Pan and Q. Yang, “A survey on transfer learning,” IEEE Transactions

   on knowledge and data engineering, vol. 22, no. 10, pp. 1345–1359, 2010.

Lern- und Lehrformen

The course is a seminar, which has an introductory phase comprised by an initial short lecture on the proposed focus areas. The lectures will cover the following topics:

• Self-aware system concepts
• Machine learning fundamentals
• Collective and collaborative learning for Multi-agent systems (MAS)
• Transfer Learning

After the introductory phase of basic concepts, the students will work (preferably in groups) on jointly identified research topics and will be assigned certain literature/techniques applying specific solutions to self-aware systems. Finally each group is required to write a report about their findings and have a presentation.
 

Leistungserfassung

For the final mark of the seminar participants, the report contributes to the 50% of the mark . The presentation of the findings contributes to the remaining 50% of the final mark. Please note that students are required to participate in the seminar during the lectures as well as the presentations of the other students in the form of questions and feedback. 

Termine

After the introductory phase with an initial short lecture, we will identify and agree on the topics for research groups and then there will not be any regular meetings. Individual meeting of the groups with the supervisor(s) will be scheduled to provide intermediate feedback. Presentations will be given on the same day (date to be determined) usually near to the end of the lecture time of the semester.

The first Lecture:  30.10.2018 at 11:00 in room A-2.1.

Zurück