Data integration is the merging of heterogeneous information from various data sources to a homogenous, clean dataset. Despite research and development over the past 40 years, collecting and integrating data from multiple sources remains an important and challenging task in any data-oriented or data science project. This lecture covers the basic technologies, such as distributed database architectures, techniques for virtual and materialized integration, data profiling, and data cleansing technologies. It thus combines the previous foundational lectures on information integration and data profiling to lay a foundation for handling unknown data.

Further Information:

• Lectures will be given in English.
• Please enroll yourself in our Moodle course: we will use it to coordinate the exercises.

Exercises:

• The exercises are led by Sedir Mohammed.
• It is necessary to pass the exercise to be admitted to take the exam.
• In the exercises, you will work in teams of two on four different assignments: two about data profiling (multivalued dependencies) and two about data cleaning (duplicate detection).
• We will use around five of the normal lecture slots to introduce and discuss the exercises.

The course will take place Mondays and Thursdays at 13:30 in L-E.03. Some lectures will have the form of exercises.

• Ulf Leser and Felix Naumann: Informationsintegration, dpunkt Verlag, 2006 (free pdf).
  This book is available at the UP library and also, e.g., from Amazon.de.
• Doan, Halevy, and Ives: Principles of data integration, Morgan Kaufmann, 2012.
• Ziawasch Abedjan, Lukasz Golab, Felix Naumann, Thorsten Papenbrock: Data Profiling - Synthesis Lectures on Data Management
• Özsu and Valduriez: Principles of distributed database systems, Springer, 2020.
• Stefan Conrad: Föderierte Datenbanksysteme, Springer, 1997.

Throughout the lecture, I will refer to various scientific papers, that serve as in-depth references.

Lecture grading is based 100% on the written exam (approx. 2h) after the end of the teaching period. Requirements for the exam admission are:

• "Passing" all four exercises
• At least one short presentation of an exercise solution