Human Emotion and Activity Classification Using Brain
Activity Sensors
Motivation
Ever wondered if the development of healthcare apps and smart sensors like smartwatches or similar wearable devices can have an actual impact on the lives of people that need help and not just on your running performance? Like people that suffer from epilepsy, people with depressive mental disorders, or the grandmother that you know that slowly seems to lose her ability to tell how she feels? Well,... we all here did, and that is why we decided to work with interesting data like time-series data from wearable devices in the chair of ‘Digital Health and Connected Healthcare’!
Time-series data constitutes significantly to assess health. Multiple choices exist when classifying time-series data, depending on the underlying model for health, the aspects of well-being analyzed, and how classification is made. Due to different recommendations in the literature, difficulties in recording sufficient amounts of data, and other reasons, no overall evaluation can tell which Time Series Classification (TSC) models to use for which TSC problem. Nowadays, recording a sufficient amount of data is ensured by the availability of cool wearable devices, like the Muse S headband. The wearables we have at our chair can be providing various physiological signals, like the Heart-Rate Variability (HRV), Photoplethysmogram (PPG), Electroencephalography (EEG), and acceleration data, amongst others. While medical professionals have access to high-quality measurement devices, the use of affordable, wearable devices will drastically increase the quality of life of the grandmother we just mentioned. These devices enable data collection in the wild, freeing participants from the burden of repeated hospital stays for data acquisition.
Nevertheless, the relatively new concept of introducing multimodality, particularly the introduction of headband sensors providing both EEG and IMU sensor data, is aiding the previously existing TSC models, especially in classifying the physiological signals driven by our mood and emotions in daily life. The underlying experience of feeling, emotion, or mood is called affects in terms of psychology. Affective state classification accuracy can be improved when other contexts, i.e., a person's activity is known.
To increase the signal-to-noise ratio (SNR) of data collected in the wild, specialized algorithms for time-series analysis have been developed and can be applied as data preprocessing. These range from statistical features over filtering of the recorded data in the spectral, spatial, or temporal domain to non-linear signal analysis utilizing chaos theory.
Hence, within the scope of this master project, we would investigate three-fold TSC classification problems where emotion classification and daily activity classification build the foundation for joint emotion and activity classification. Therefore, this research-driven Master project will assist in finding or developing approaches on how to deal with specific sub-problems that arise when dealing with difficult-to-address topics---like the prediction of epileptic seizures, depressive disorders, or any mental health problems driven by emotions---using wearable devices and thereby empowering patients.
The topic at hand
In this Master project, we want to work in the field of activity and emotion classification of different types of physiological data. There are many fascinating aspects to working with wearable devices. The most fascinating aspect of this project is the evaluation of the commercial-grade wearable EEG headbands over clinical-grade EEG devices. These headbands come with an onboard IMU sensor, which can be used to further classify the daily activities of a person. We will consider the inclusion of other sensors, such that we can compare between what our heart says and what our brain says. Therefore, for better classification, we would investigate the impact of data preprocessing pipelines on activity and emotion classification and whether the same pipeline used individually can be used for both. Additionally, we will analyze how well the developed subject-dependent models generalize.
