We are excited to announce that our paper “Table Overlap Estimation through Graph Embeddings” was accepted at the International Conference on Management of Data (SIGMOD 2025). The paper was the result of the collaboration with the University of Modena and Reggio Emilia (Italy) and the University of Verona (Italy).
Authors
Francesco Pugnaloni (Hasso Plattner Institute); Luca Zecchini (University of Modena and Reggio Emilia); Matteo Paganelli (University of Modena and Reggio Emilia); Matteo Lissandrini (University of Verona); Felix Naumann (Hasso Plattner Institute, University of Potsdam); Giovanni Simonini (University of Modena and Reggio Emilia)
Abstract
Discovering duplicate or high-overlapping tables in table collections is a crucial task for eliminating redundant information, detecting inconsistencies in the evolution of a table across its multiple versions produced over time, and identifying related tables. Candidate duplicate or related tables to support this task can be identified via the estimation of the largest table overlap. Unfortunately, current solutions for finding it present serious scalability issues for heavy workloads: Sloth, the state-of-the-art framework for its estimation, requires more than three days of machine time for computing 100k table overlaps.
In this paper, we introduce Armadillo, an approach based on graph neural networks that learns table embeddings whose cosine similarity approximates the overlap ratio between tables, i.e., the ratio between the area of their largest table overlap and the area of the smaller table in the pair. We also introduce two new annotated datasets based on GitTables and a Wikipedia table corpus containing 1.32 million table pairs overall labeled with their overlap. Evaluating the performance of Armadillo on these datasets, we observed that it is able to calculate overlaps between pairs of tables several times faster than the state-of-the-art method while maintaining a good quality in approximating the exact result.
