METL - Managing and Integrating ETL Processes

ETL-processes are defined to integrate heterogeneous data into a data warehouse. ETL management is the systematic, semi-automatic management of large sets of such processes. It includes several simple operators, such as IMPORT and SEARCH, and more complex operators, such as MATCH, MERGE, or INVERT. Links:

Bachelorprojects GrETL and Moritz

Managing and Integrating ETL Processes

ETL tools allow the definition of sometimes complex processes to extract, transform, and load heterogeneous data into a data warehouse or to perform other data migration tasks. In larger organizations many ETL processes of different data integration projects are accumulated. Such processes can encompass common sub-processes, shared data sources and targets, and same or similar operations. However, there is no common method or approach to systematically manage such ETL processes. With METL (Managing ETL) we present a next generation ETL tool that supports high-level ETL management.

People

• Felix Naumann
• Alexander Albrecht
• Karsten Draba
• Dandy Fenz
• Paul Möller
• Minh Tuan Nguyen
• Andriy Vedrych
• Bachelor project 2009/10 - ETL-Prozess-Management für BMW Financial Services [Video]

Partners

• InfoDyn AG
• BBF - Gesellschaft für Softwareentwicklung und Beratung mbH 

Publications

• Systematic ETL Management – Experiences with high-level Operators
  Alexander Albrecht und Felix Naumann
  Conference on Information Quality (ICIQ) 2013, Little Rock, AR, USA
• Schema Decryption for Large Extract-Transform-Load Systems
  Alexander Albrecht and Felix Naumann
  Conference on Conceptual Modeling (ER) 2012, Florence, Italy
• Understanding Cryptic Schemata in Large Extract-Transform-Load Systems
  Alexander Albrecht and Felix Naumann
  Technical Report 60, Hasso-Plattner-Institut für Softwaresystemtechnik an der Universität Potsdam, 2012. ISBN 978-3-86956-201-8, ISSN 1613-5652
• METL: Managing and Integrating ETL Processes
  Alexander Albrecht
  VLDB PhD Workshop 2009, Lyon, France.
• Managing ETL Processes
  Alexander Albrecht and Felix Naumann
  Workshop New Trends in Information Integration (NTII) 2008, Auckland, New Zealand

Master theses

• Summarizing Extract-Transform-Load Workflows, Minh Tuan Nguyen, 2012
• ETL Process Recommendation, Andriy Vedrych, 2010
• Merging Extract, Transform, Load Processes, Karsten Draba, 2009