Highly Scalable Systems (Sommersemester 2021)

Lecturer: Prof. Dr. Holger Giese (Systemanalyse und Modellierung) , Christian Medeiros Adriano (Systemanalyse und Modellierung) , Matthias Barkowsky (Systemanalyse und Modellierung) , Iqra Zafar (Systemanalyse und Modellierung)

General Information

• Weekly Hours: 2
• Credits: 3
• Graded: yes
• Enrolment Deadline: 18.03.2021 - 09.04.2021
• Teaching Form: Seminar
• Enrolment Type: Compulsory Elective Module
• Course Language: English

Programs, Module Groups & Modules

Description

Motivation:

The Google PageRank crawlers take a few weeks to map the entire web and a few hours to recompute the importance of pages. Although costly, the computation is still within the needs of final users of the search engine. However, one cannot assume that this crawling might keep scaling with the growth in internet users and devices. A recent industry report [Cisco 2020] estimates that  nearly 2/3 of the global population will have Internet access by 2023, i.e., 5.3 billion total Internet users (66% of global population) and up from 3.9 billion (51% of global population) in 2018.

By the year 2023 there will be :

• 29.3 billion networked devices (compared with 18.4 billion in 2018)
• 14.7 billion IoT connections (33% growth over 2-18)
• Connected home apps will have the largest share and connected cars will be the fastest growing application type.
• Connected home apps will have nearly half or 48% of IoT share by 2023 and 
• Connected car applications will grow the fastest at 30% over the forecast period (2018–2023).

We are interested in models that support the design, implementation, test, deployment, monitoring, and the evolution of highly scalable systems. Examples of highly scalable systems are IoT, swarms, social networks, air-traffic control networks, smart-city traffic networks, etc. These systems are distributed and comprise billions of heterogeneous nodes that cooperate to deliver mission-critical functions.

Ultimately, the fundamental engineering problem is to choose among the various types and models and algorithms, which combinations allow organizations to scale functions like: monitoring, prediction, and prevention of deleterious events. This is particularly challenging in large systems that are constantly evolving by adding new nodes and connections.

Goals:

• Learn the foundational knowledge of requirements and constraints that govern highly scalable systems
• Acquire broad knowledge of the current methods and models to scale up systems
• Learn how to model scalability requirements and constraints 
• Compare the different methods to improve scalability

Students will be able to choose among various domains, for instance, microservices, reinforcement learning, attacks on large networks, and self-adaptation in sparse settings.

Literature

[1] Cisco Annual Internet report, https://www.cisco.com/c/en/us/solutions/collateral/executive-perspectives/annual-internet-report/white-paper-c11-741490.html 

[2] Marz, N., and Warren, J., (2013), Big Data: Principles and best practices of scalable real-time data systems, Manning. 

[3] Hasselbring, W., and Steinacker, G., (2017), Microservice architectures for scalability, agility and reliability in e-commerce, IEEE International Conference on Software Architecture Workshops (ICSAW).

[4] Schneider, T. and Wolfsmante, A.l, (2016), Achieving Cloud Scalability with Microservices and DevOps in the Connected Car Domain, Software Engineering Workshops

[5] Avritzer, Alberto, et al., (2020), Scalability assessment of microservice architecture deployment configurations: A domain-based approach leveraging operational profiles and load tests,  Journal of Systems and Software 165: 110564.

[6] Li J, et al., (2020), Adversarial attack on large scale graph[J]. arXiv preprint arXiv:2009.03488.

[7] N. J. A. Sloane, (2020), Number of acyclic digraphs (or DAGs) with n labeled nodes, in The Online Encyclopedia of Integer Sequences, Available at oeis.org/A003024/b003024.txt 

[8] Li J, et al., (2020), Adversarial attack on community detection by hiding individuals, WWW Conference.

[9] Zügner D, et al., (2019), Adversarial attacks on graph neural networks: Perturbations and their patterns, TKDD.

[10] Vinitsky, Eugene, et al., (2020), Robust Reinforcement Learning using Adversarial Populations, arXiv preprint arXiv:2008.01825.

[11] Pagán, J. E., et al., (2011). Morsa: A scalable approach for persisting and accessing large models. In International Conference on Model Driven Engineering Languages and Systems (pp. 77-92). Springer, Berlin, Heidelberg.

Learning

There will be a mandatory introductory meeting, where basic concepts of highly scalable systems and the specific seminar topics are introduced. Students are then required to choose a topic, which they will work on during the semester individually or in teams of up to 2 students, though teamwork is encouraged. A specific assignment and corresponding materials will be provided.

The students are expected to submit a report of approx. 15 pages per student in the team with their respective assignments and present their findings in a final seminar meeting.

Throughout the seminar, a teaching assistant will offer support and supervision for each participant or team. Further meetings with all course participants may be arranged upon request, in order to answer questions of general interest and to support the exchange between teams.

Examination

We will grade the student reports (50%) and presentations (50%). Participation in the final meeting during other students' presentations in the form of questions and feedback is also mandatory.

Dates

Besides the introductory meeting and individual feedback meetings with the teaching assistants, there will be no regular meetings during the semester. Presentations will be given in one session (date to be determined) near to the end of or after the lecture period of the semester.

The introductory meeting will be held online on Thursday 15.04 at 15:15.

Regular Meetings: Thursday 15:15

Please email christian.adriano(at)hpi.de to obtain the Zoom credentials. If you are interested in the seminar but cannot attend the introduction meeting, please contact us. We will find an individual solution for you.

Announcement regarding the coronavirus regulations:

Because of the Coronavirus situation and corresponding restriction outbreak, we will organize all meetings as online meetings by default. This especially applies to the introductory meeting. If all participants agree to and the current restrictions as well as seminar room availability allow it, further meetings may also take place at HPI.

Zurück