• EN
Background image HPI with color gradient

Project Seminar: Reproducing Internet Measurement Research

Description

Reproducibility is a cornerstone of scientific integrity and trustworthiness, essential for advancing knowledge and enabling technology transfer. However, in computer science and specifically in networking, security and privacy research, reproducibility remains a persistent challenge. Often, only a fraction of published studies provide the necessary artifacts—such as code and datasets—to fully reproduce results, impeding the ability of independent researchers to validate and build on prior findings.

The challenges to reproducibility are multifaceted. They include legal and privacy concerns about sharing sensitive measurement data, difficulties managing datasets lacking stable ground truth, the extra workload involved in documenting experimental artifacts, and the lack of strong incentives for researchers to prioritize reproducibility over novelty in publication.

This project seminar provides master’s students with a comprehensive and practical exploration of reproducibility and replicability in the field of Internet measurement research. It teaches current research community standards and initiatives designed to improve reproducibility, including:

  • The ACM Internet Measurement Conference (IMC) 2025 Replicability Track, which promotes transparent validation of influential Internet measurement papers by encouraging artifact sharing along with explanations for cases where sharing is restricted.
  • ACM’s Artifact Review and Badging Policy, established to standardize the evaluation and recognition of shared research artifacts, offering badges that certify availability, evaluation, and reproducibility of published results.
  • Archival resources such as DatCat and CRAWDAD that provide indexed, curated repositories of measurement data to facilitate access, reuse, and verification by the community.
  • Adoption of modern tools like interactive notebooks (e.g., Jupyter), experiment tracking platforms, and research code repositories that document workflows and computational processes, thereby lowering barriers to reproducibility.
  • Educational efforts and workshops that integrate reproducibility and open science practices into early-stage scientific training, fostering a cultural shift toward transparent, ethical, and reusable research methodologies.

 

Teaching and learning methods

In this course, students will engage deeply with seminal published studies from top conferences such as Internet Measurements Conference, SIGCOMM, IEEE Security and Privacy, USENIX Security, and Privacy Enhancing Technologies Symposium undertaking replication or reproduction efforts to validate and critically analyze reported results. Through these hands-on projects, students will gain firsthand experience with the intricacies of reproducing experiments, managing data, and interpreting discrepancies.

Students will produce detailed reports and presentations that not only communicate their technical findings but also reflect on the reproducibility process and its implications for scientific integrity. Some of these methods include:

  • Reading, presenting and discussing key Internet measurement papers.
  • Artifact reviews of these papers to evaluate their reproducibility.
  • Hands-on project work replicating or reproducing results from the papers read in the class.
  • Iterative peer discussion and troubleshooting of challenges during reproduction.
  • Disseminating the reproduced results both as a report and as a presentation in the class.
  • Guided reflection on reproducibility practices, workflows, and policies.

 

Learning outcomes

By the end of the course, students will be equipped to critically evaluate published research, contribute to reproducibility efforts, and foster a culture of openness and trustworthiness in networked systems research. It will also provide empirical grounding for their careers in research and industry by equipping them with the skills to conduct open, transparent, and replicable studies—meeting the evolving expectations of the scientific community.
 

 

(Recommended) Requirements

Students should have completed an undergraduate-level course in computer networks and networking protocols. Although not mandatory, familiarity with tools commonly used for performance evaluation will be advantageous. Additionally, having programming skills and a strong interest in rigorous scientific methodology would be beneficial.

 

Course Outline and Literature

 

Grading Scheme

  • Artifact Review: 15%
  • Paper and Artifact Presentation: 15%
  • Reproducibility report: 55%
  • Reproducibility report presentation: 15%

Lecturer: Prof. Dr. Vaibhav Bajpai
Support: Dr. Vasilis Ververis, Robert Richter

Credits: 6 ECTS
Language: EN
Modules Specification: Master's Students