Prof. Dr.-Ing. Bert Arnrich

Balancing Medical Human Pose Datasets using Generative Adversarial Networks

Master's Thesis

Pawel Glöckner, Supervisor: Justin Albert

Machine Learning is a valuable tool for many human motion analysis applications, such as human activity recognition (HAR), exercise recognition, or exercise rating. It also plays a vital role in medical applications such as exercise rating for rehabilitation programs. However, when collecting large medical sets, especially for a study design involving a healthy control group, most data points often belong to the class of healthy subjects. Often the access to patients can be limited, which hinders the data acquisition. In this thesis, a Machine Learning model was developed to generate new kinematic data for a specific exercise and population, therefore allowing to generate new samples for underrepresented patient classes. The here developed Generative Adversarial Network (GAN) builds upon recent work on human motion generation, mainly developed for other areas such as Augmented Reality (AR) or video game character animation. The adjusted model based on the human pose prediction GAN (HP-GAN) [1] was trained and evaluated on the KIMORE (KInematic Assessment of MOvement and Clinical Scores for Remote Monitoring of Physical REhabilitation) dataset [2]. This dataset includes five different back-pain exercises performed by stroke, Parkinson's Disease, and low-back pain patients and contains a healthy control group (with experts and non-experts). Figure 1 shows ground truth data from the KIMORE dataset for the squat exercise and our GAN's synthetically generated data. We have validated the data using a classification model that differentiates between stroke, Parkinson's Disease, and healthy subjects. Therefore, we have used the GAN to generate additional repetitions for the underrepresented stroke and Parkinson's Disease classes to balance the dataset. The classifier was trained from scratch on the original unbalanced dataset and our balanced dataset and achieved a classification accuracy of 70% and 81%, respectively.   

Fig. 1: Motion sequences of two consecutive squat repetitions. Top: Ground-truth data from the KIMORE dataset. Bottom: generated data using our GAN with 20 prior poses.


[1] E. Barsoum, J. Kender, and Z. Liu, “HP-GAN: Probabilistic 3D Human Motion Prediction via GAN,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Salt Lake City, UT, USA, Jun. 2018, pp. 1499–149909, doi: 10.1109/CVPRW.2018.00191.

[2] M. Capecci et al., “The KIMORE Dataset: KInematic Assessment of MOvement and Clinical Scores for Remote Monitoring of Physical REhabilitation,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 27, no. 7, pp. 1436–1448, Jul. 2019, doi: 10.1109/TNSRE.2019.2923060.