Error-free data sets are the foundation for the successful training of machine learning (ML) models and the use of artificial intelligence (AI). Much research has been done on classifying, describing, detecting, and cleaning data errors. The systematic generation of data errors, i.e., the pollution of data sets with known errors, is important to benchmark error detection and data cleaning algorithms. The aim of this seminar is to develop novel ideas for generating data errors that are as difficult as possible to detect.
This seminar constitutes an adversarial challenge in which teams of two students compete against each other. Each team needs to (1) first generate data errors that are as hard as possible to detect and then (2) detect difficult data errors generated by other teams. We will initially introduce the field of data quality, followed by a list of the most common types of data errors, various technologies for noise pollution, like data synthesis and perturbation, and techniques for error detection, like statistical and ML methods. Together, we will select a number of interesting data errors to focus on in this competition.
What are the goals of the seminar?
- Learn about the research area of data quality and data errors
- Read and understand scientific papers
- Develop novel ideas on how to generate and detect data errors
- Jointly discuss properties of “hard to detect” data errors (e.g., whether this correlates with the extent to which an error appears to be realistic)
- Present results in written and oral form
The data error seminar will be organized in the following four phases:
- Kickoff: We will provide clean data sets and jointly select the data error types that we will focus on for the competition.
- Research: Each team will select a (subset) of the defined data error(s), read related work, and prepare a presentation for the entire group to provide examples and describe what constitutes this kind of error and what not.
- Data error challenge: The challenge itself will be carried out in two phases:
- Phase 1: Each team will pollute the input data sets with errors that appear to be realistic and should be as difficult to detect as possible. As a result, for each input data set, (1) a polluted data set, as well as (2) a set of labels with the polluted data errors, should be generated.
- Phase 2: Each team will receive the polluted data sets from the other teams. The goal is to detect as many data errors as possible.
- Deliverable: At the end of the seminar, each team will prepare a presentation about (1) their used data error generation strategy, (2) the error detection technique used as well as (3) the percentage of errors found by the respective other group.
