Prof. Dr. Felix Naumann

Advanced Topic Modeling

Lecturer: Dr. Ralf Krestel


Topic modeling is a probabilistic approach to text mining. Large document collections are analyzed to identify latent (hidden) topics. The most popular topic model is latent Dirichlet allocation (LDA). This algorithm is based on a generative view of the writing process of documents which it tries to reverse engineer: "An author decides to write a document about the olympics in Rio. She picks the topics 'sports' and 'brazil'. Then for each word she wants to write she picks a word from one of the two topics." LDA only sees the final documents and has to estimate what these underlying topics are and how they look. There are many extensions and adaptions of the basic LDA method. We will focus on two of the four advanced topic models:

  • Labeled LDA (paper)
  • Author-topic model (paper)
  • Dynamic topic model (paper)
  • Hierarchical topic model (paper)

These models heavily rely on methods from probability theory and statistics. Prior knowledge is not required but certainly helpful. If you start sweating when seeing a math formula, this is not the right seminar for you. If you are curious and motivated to understand these models then this seminar is great. WARNING: Depending on your background you might have to do a lot of reading and thinking...

In this seminar, each student will present one topic modeling paper in a 30-minutes talk followed by 30 minutes of discussion. We will start with two groups of students implementing latent Dirichlet Allocation using

  • Collapsed Gibbs sampling (paper) and
  • Variational Bayes (paper).

Then we will split up the two groups into teams of two and each team will work on one advanced topic model (presentation+implementation). At the end of the semester, each team's implemented topic model will be used modeling a dataset of US patents. Finally, each team has to hand in a written summary report (5 pages; two column style) of their topic. Active participation in all discussions is mandatory. If all goes well, we plan to compile a scientific paper of the reports and submit it to a conference or workshop. Depending on the success of this seminar we will offer a Master's project or Master's thesis topic in the following semester based on this seminar...

This seminar is limited to 8 participants. If more apply we will pick randomly.

The grade will consist of

  • 25% Presentation of Paper
  • 25% Active Participation
  • 50% Project Report

The seminar usually takes place Tuesdays at 11:00 in D-E 9/10.


Date Topic Presenter
12.4.16Introduction Ralf Krestel
18.4.1613:00 Deadline for Registration via E-mail (15:00 Notification)
19.4.16Initial Meeting Ralf Krestel
26.4.16 Optional Meetings
3.5.16 Meeting Gibbs Group (45min) and Bayes Group (45min)
10.5.16 No Meeting
17.5.16 Two presentations Gibbs and Bayes Teams
24.5.16 Comparing Gibbs and Bayes Code, New Teams Ralf Krestel
31.5.16 Optional Meeting
7.6.16 No Meeting
14.6.16 Individual Meetings
21.6.16 Two presentations Teams
28.6.16 No Meeting
5.7.16 Comparing Code and Discussing Experiments Ralf Krestel
12.7.16 No Meeting
21.7.16 Discussing Results Teams
31.8.16 Report Deadline