Prof. Dr.-Ing. Bert Arnrich

Human Emotion and Activity Classification Using Brain Activity Sensors

Master Project (Winter Term 2021/2022)

Supervisors: Christoph Anders, Sidratul Moontaha

Graphical abstract of tasks performed during this master project

This was an interdisciplinary project spanning areas of psychology, signal processing, computer science, and medicine. Hence, a broad and diverse field of experience and interests was required.

Regarding computer science, this project covered:

  • Processing of medical data
  • Time-series data
  • Machine learning and deep learning

Regarding medicine, this project covered:

  • Human psychology e.g., affects
  • Human physiology e.g., brain waves

However, students were not expected to have knowledge in all of these fields. We expected however the interest and dedication to fill in knowledge gaps apart from the time necessary for the conduct of the work described in this Master project description.


Within the scope of this project, students conducted their experiments to classify emotions and activity. They were responsible for data recording and utilized already developed applications within the chair. As a next step, they built pre-processing pipeline designs based on literature reviews and evaluated multiple Machine learning and Deep learning models for TSC. The students learned to use an already existing toolbox in the chair for synchronizing the multimodal data sampled at different recording frequencies. The emotions that were subject to classification depended on the experimental setup of the students and were subject to proper planning to compare findings to the literature. Working in a real-world scenario with an impact later down the line posed a sufficiently tricky challenge. Eventually, future students could extend this setup to evaluate findings with medical professionals, building the foundation for exciting questions for their later master thesis.


The final project was called 'Distinction Between Work and Relaxation: A Use Case of Multimodal Wearable Sensors'. The following abstract summarizes the students work:

Mental fatigue is a problem everybody is encountering at work, at school, while driving, and during other mentally demanding tasks. Several changes in physiological signals, e.g., rising heart rate or enhanced brain activity, can indicate whether a person is mentally exhausted or relaxed. In this project, Electroencephalogram (EEG), Photoplethysmogram (PPG), skin temperature, and Galvanic skin Response (GSR) are measured by Empatica and Muse devices during studies that induce a relaxed state and mental workload/stress. The collected data is synchronized and filtered using techniques like Savitzky-Golay Filter, Butterworth bandpass filter, Adaptive Noise Cancellation filter, and finally used to compute features, using, e.g., Common Spatial Pattern. A multi-modal platform for visualizing and managing participants’ study data is built. The prediction of mental fatigue is implemented using several machine learning models. Implemented models are Support Vector Machine, Random Forest, and Deep Convolutional Neural Networks directly provided on the platform and with configurable hyper- parameters, train-test-splits, and leave-one-out cross-validation, with the best performing model being a Random Forest at an average balanced accuracy of 82.21% and the second best model being a Support Vector Machine at an average balanced accuracy of 81.23%. 


Sidratul Moontaha

Room: G-2.1.21

Phone: +49 331 5509-3481

E-Mail: sidratul.moontaha(at)hpi.de


Christoph Anders

Room: G-2.1.21

Phone: +49 331 5509-4501

E-Mail: christoph.anders@hpi.de

Bert Arnrich

Room: G-2.1.14

Phone: +49 331 5509-4850

E-Mail: bert.arnrich(at)hpi.de

Resulting Publications

  • Towards Multi-Modal Recordings in Daily Life: A Baseline Assessment of an Experimental Framework Anders, Christoph; Moontaha, Sidratul; Arnrich, Bert in IS (2022). (Vol. H) 27–30. Information Society.