15th BTW conference on "Database Systems for Business, Technology, and Web" (BTW 2013)
Magdeburg, Germany
Duplicate Detection on GPUs
Benedikt Forchhammer, Thorsten Papenbrock, Thomas Stening,
Sven Viehmeier, Uwe Draisbach, Felix Naumann
Abstract. "With the ever increasing volume of data and the ability to integrate different data sources, data quality problems abound. Duplicate detection, as an integral part of data cleansing, is essential in modern information systems. We present a complete duplicate detection workflow that utilizes the capabilities of modern graphics processing units (GPUs) to increase the efficiency of finding duplicates in very large datasets. Our solution covers several well-known algorithms for pair selection, attribute-wise similarity comparison, record-wise similarity aggregation, and clustering. We redesigned these algorithms to run memory-efficiently and in parallel on the GPU. Our experiments demonstrate that the GPU-based workflow is able to outperform a CPU-based implementation on large, real-world datasets. For instance, the GPU-based algorithm deduplicates a dataset with 1.8m entities 10 times faster than a common CPU-based algorithm using comparable hardware."
Meteor: An Extensible Query Language for Big Data Analytics (Demo)
Marcus Leich, Jochen Adamek, Moritz Schubotz, Arvid Heise, Astrid Rheinländer, and Volker Markl
Abstract. Analyzing big data sets as they occur in modern business and science applications requires query languages that allow for the specification of complex data processing tasks.
Moreover, these ideally declarative query specifications have to be optimized, parallelized and scheduled for processing on massively parallel data processing platforms.
This demonstration proposal highlights Meteor, the declarative, extensible, and automatically optimizable/parallelizable query language of the Stratosphere data processing platform.
We illustrate the basic features of the language as well as its automatic optimization and parallelization by an example that demonstrates a correlation analysis of microblogging and stock trade volume data.
The demonstration will present both, the query specification using Stratosphere Meteor API, its compilation, and optimized execution on Stratosphere, leveraging the visual Stratosphere query plan, and job graph execution inspection tools.
