2024

A computer vision approach to continuously monitor fatigue during resistance training. Albert, Justin Amadeus; Arnrich, Bert in Biomedical Signal Processing and Control (2024). 89 105701.
[ Abstract ] [ BibTeX ] [ URL ] [ DOI ]
 
Monitoring fatigue during resistance training is essential to avoid injuries caused by overtraining. Fatigue can be comprehensively quantified by the external and internal load, where the external load is the work done by the athlete, and the internal load is the psychological and physiological response to the external load. This paper proposes a computer vision method to continuously monitor fatigue during resistance training by predicting external and internal parameters, namely the generated power and the rating of perceived exertion. We utilize the human pose estimation from two Microsoft Azure Kinect cameras to capture the movement of athletes while performing stationary exercises. Our method processes the obtained kinematic data, computes skeleton features to train traditional machine learning algorithms, and constructs feature maps to train convolutional neural network-based models to predict the load parameters. For evaluation, we recorded a dataset of 16 subjects who performed squat exercises on a Flywheel and rated their perceived exertion after each set. A measuring unit integrated into the Flywheel provided power readings for each repetition. The results show that our method achieves good results in predicting both parameters. Gradient Boosting Regression Trees best predicted perceived exertion with a mean absolute percentage error of 8.08% and a Spearman’s ρ=0.74. Multi-layer Perceptron performed best in predicting power with a mean absolute error of 23.13 Watts and ρ=0.79. Our findings show that our approach delivers promising external and internal load quantifications for fatigue, with great potential to provide external feedback to coaches or athletes.
@article{ALBERT2024105701, abstract = {Monitoring fatigue during resistance training is essential to avoid injuries caused by overtraining. Fatigue can be comprehensively quantified by the external and internal load, where the external load is the work done by the athlete, and the internal load is the psychological and physiological response to the external load. This paper proposes a computer vision method to continuously monitor fatigue during resistance training by predicting external and internal parameters, namely the generated power and the rating of perceived exertion. We utilize the human pose estimation from two Microsoft Azure Kinect cameras to capture the movement of athletes while performing stationary exercises. Our method processes the obtained kinematic data, computes skeleton features to train traditional machine learning algorithms, and constructs feature maps to train convolutional neural network-based models to predict the load parameters. For evaluation, we recorded a dataset of 16 subjects who performed squat exercises on a Flywheel and rated their perceived exertion after each set. A measuring unit integrated into the Flywheel provided power readings for each repetition. The results show that our method achieves good results in predicting both parameters. Gradient Boosting Regression Trees best predicted perceived exertion with a mean absolute percentage error of 8.08% and a Spearman’s ρ=0.74. Multi-layer Perceptron performed best in predicting power with a mean absolute error of 23.13 Watts and ρ=0.79. Our findings show that our approach delivers promising external and internal load quantifications for fatigue, with great potential to provide external feedback to coaches or athletes.}, author = {Albert, Justin Amadeus and Arnrich, Bert}, journal = {Biomedical Signal Processing and Control}, keywords = {bertarnrich justinalbert}, pages = 105701, title = {A computer vision approach to continuously monitor fatigue during resistance training}, volume = 89, year = 2024 }

2023

Protocol for a Randomized Crossover Trial to Evaluate the Effect of Soft Brace and Rigid Orthosis on Performance and Readiness to Return to Sport Six Months Post-ACL-Reconstruction. Jahnke, Sonja; Cruysen, Caren; Prill, Robert; Kittmann, Fabian; Pflug, Nicola; Albert, Justin Amadeus; de Camargo, Tibor; Arnrich, Bert; Królikowska, Aleksandra; Kołcz, Anna; Reichert, Paweł; Oleksy, Łukasz; Michel, Sven; Kopf, Sebastian; Wagner, Michael; Scheffler, Sven; Becker, Roland in Healthcare (2023). 11(4)
[ Abstract ] [ BibTeX ] [ URL ] [ DOI ]
 
A randomized crossover trial was designed to investigate the influence of muscle activation and strength on functional stability/control of the knee joint, to determine whether bilateral imbalances still occur six months after successful anterior cruciate ligament reconstruction (ACLR), and to analyze whether the use of orthotic devices changes the activity onset of these muscles. Furthermore, conclusions on the feedforward and feedback mechanisms are highlighted. Therefore, twenty-eight patients will take part in a modified Back in Action (BIA) test battery at an average of six months after a primary unilateral ACLR, which used an autologous ipsilateral semitendinosus tendon graft. This includes double-leg and single-leg stability tests, double-leg and single-leg countermovement jumps, double-leg and single-leg drop jumps, a speedy jump test, and a quick feet test. During the tests, gluteus medius and semitendinosus muscle activity are analyzed using surface electromyography (sEMG). Motion analysis is conducted using Microsoft Azure DK and 3D force plates. The tests are performed while wearing knee rigid orthosis, soft brace, and with no aid, in random order. Additionally, the range of hip and knee motion and hip abductor muscle strength under isometric conditions are measured. Furthermore, patient-rated outcomes will be assessed.
@article{healthcare11040513, abstract = {A randomized crossover trial was designed to investigate the influence of muscle activation and strength on functional stability/control of the knee joint, to determine whether bilateral imbalances still occur six months after successful anterior cruciate ligament reconstruction (ACLR), and to analyze whether the use of orthotic devices changes the activity onset of these muscles. Furthermore, conclusions on the feedforward and feedback mechanisms are highlighted. Therefore, twenty-eight patients will take part in a modified Back in Action (BIA) test battery at an average of six months after a primary unilateral ACLR, which used an autologous ipsilateral semitendinosus tendon graft. This includes double-leg and single-leg stability tests, double-leg and single-leg countermovement jumps, double-leg and single-leg drop jumps, a speedy jump test, and a quick feet test. During the tests, gluteus medius and semitendinosus muscle activity are analyzed using surface electromyography (sEMG). Motion analysis is conducted using Microsoft Azure DK and 3D force plates. The tests are performed while wearing knee rigid orthosis, soft brace, and with no aid, in random order. Additionally, the range of hip and knee motion and hip abductor muscle strength under isometric conditions are measured. Furthermore, patient-rated outcomes will be assessed.}, author = {Jahnke, Sonja and Cruysen, Caren and Prill, Robert and Kittmann, Fabian and Pflug, Nicola and Albert, Justin Amadeus and de Camargo, Tibor and Arnrich, Bert and Królikowska, Aleksandra and Kołcz, Anna and Reichert, Paweł and Oleksy, Łukasz and Michel, Sven and Kopf, Sebastian and Wagner, Michael and Scheffler, Sven and Becker, Roland}, journal = {Healthcare}, keywords = {sys:relevantfor:dhc bertarnrich justinalbert}, number = 4, title = {Protocol for a Randomized Crossover Trial to Evaluate the Effect of Soft Brace and Rigid Orthosis on Performance and Readiness to Return to Sport Six Months Post-ACL-Reconstruction}, volume = 11, year = 2023 }

2022

PERSIST: A Multimodal Dataset for the Prediction of Perceived Exertion during Resistance Training. Albert, Justin Amadeus; Herdick, Arne; Brahms, Clemens Markus; Granacher, Urs; Arnrich, Bert in Data (2022). 8(1)
[ Abstract ] [ BibTeX ] [ URL ] [ DOI ]
 
Measuring and adjusting the training load is essential in resistance training, as training overload can increase the risk of injuries. At the same time, too little load does not deliver the desired training effects. Usually, external load is quantified using objective measurements, such as lifted weight distributed across sets and repetitions per exercise. Internal training load is usually assessed using questionnaires or ratings of perceived exertion (RPE). A standard RPE scale is the Borg scale, which ranges from 6 (no exertion) to 20 (the highest exertion ever experienced). Researchers have investigated predicting RPE for different sports using sensor modalities and machine learning methods, such as Support Vector Regression or Random Forests. This paper presents PERSIST, a novel dataset for predicting PERceived exertion during reSIStance Training. We recorded multiple sensor modalities simultaneously, including inertial measurement units (IMU), electrocardiography (ECG), and motion capture (MoCap). The MoCap data has been synchronized to the IMU and ECG data. We also provide heart rate variability (HRV) parameters obtained from the ECG signal. Our dataset contains data from twelve young and healthy male participants with at least one year of resistance training experience. Subjects performed twelve sets of squats on a Flywheel platform with twelve repetitions per set. After each set, subjects reported their current RPE. We chose the squat exercise as it involves the largest muscle group. This paper demonstrates how to access the dataset. We further present an exploratory data analysis and show how researchers can use IMU and ECG data to predict perceived exertion.
@article{data8010009, abstract = {Measuring and adjusting the training load is essential in resistance training, as training overload can increase the risk of injuries. At the same time, too little load does not deliver the desired training effects. Usually, external load is quantified using objective measurements, such as lifted weight distributed across sets and repetitions per exercise. Internal training load is usually assessed using questionnaires or ratings of perceived exertion (RPE). A standard RPE scale is the Borg scale, which ranges from 6 (no exertion) to 20 (the highest exertion ever experienced). Researchers have investigated predicting RPE for different sports using sensor modalities and machine learning methods, such as Support Vector Regression or Random Forests. This paper presents PERSIST, a novel dataset for predicting PERceived exertion during reSIStance Training. We recorded multiple sensor modalities simultaneously, including inertial measurement units (IMU), electrocardiography (ECG), and motion capture (MoCap). The MoCap data has been synchronized to the IMU and ECG data. We also provide heart rate variability (HRV) parameters obtained from the ECG signal. Our dataset contains data from twelve young and healthy male participants with at least one year of resistance training experience. Subjects performed twelve sets of squats on a Flywheel platform with twelve repetitions per set. After each set, subjects reported their current RPE. We chose the squat exercise as it involves the largest muscle group. This paper demonstrates how to access the dataset. We further present an exploratory data analysis and show how researchers can use IMU and ECG data to predict perceived exertion.}, author = {Albert, Justin Amadeus and Herdick, Arne and Brahms, Clemens Markus and Granacher, Urs and Arnrich, Bert}, journal = {Data}, keywords = {arneherdick bertarnrich justinalbert}, number = 1, title = {PERSIST: A Multimodal Dataset for the Prediction of Perceived Exertion during Resistance Training}, volume = 8, year = 2022 }

Unsupervised Activity Recognition Using Trajectory Heatmaps from Inertial Measurement Unit Data. Konak., Orhan; Wegner., Pit; Albert., Justin; Arnrich., Bert (2022). 304–312.
[ BibTeX ] [ URL ] [ DOI ]
 
@conference{icaart22, author = {Konak., Orhan and Wegner., Pit and Albert., Justin and Arnrich., Bert}, booktitle = {Proceedings of the 14th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,}, keywords = {bertarnrich justinalbert orhankonak}, organization = {INSTICC}, pages = {304-312}, publisher = {SciTePress}, title = {Unsupervised Activity Recognition Using Trajectory Heatmaps from Inertial Measurement Unit Data}, year = 2022 }

2021

Using Machine Learning to Predict Perceived Exertion During Resistance Training With Wearable Heart Rate and Movement Sensors. Albert, Justin; Herdick, Arne; Brahms, Clemens Markus; Granacher, Urs; Arnrich, Bert (2021).
[ Abstract ] [ BibTeX ] [ URL ] [ DOI ]
 
The quantification of subjective exertion during training is an important measurement as it has the potential to avoid injuries resulting from overtraining. In this paper, we present a method to predict the subjective exertion during resistance training using Inertial Measurement Units (IMU) and electrocardiographical data. The participants’ subjective exertion was assessed using a Rating of Perceived Exertion (RPE) scale. We obtained data from 16 participants performing squats on a flywheel training machine while being equipped with six IMU sensors and an electrocardiography (ECG) sensor. Data was analyzed using multiple regressors, such as Support Vector Regression, Random Forests, and Gradient Boosting Regression Trees, to predict the personal exertion level on the processed IMU and ECG data. The best learning model achieved a mean absolute percentage error of 7.71% with a Pearson correlation coefficient of 0.85 and a R 2 of 0.48. Additionally, we investigated the impact of supplementing the IMU data features with ECG-derived heart rate variability (HRV) parameters in the training stage. Our results indicate that the HRV parameters derived from ECG significantly improve prediction results, with the training impulse (TRIMP) parameter acting as the most informative feature for predicting perceived exertion.
@inproceedings{albert2021using, abstract = {The quantification of subjective exertion during training is an important measurement as it has the potential to avoid injuries resulting from overtraining. In this paper, we present a method to predict the subjective exertion during resistance training using Inertial Measurement Units (IMU) and electrocardiographical data. The participants’ subjective exertion was assessed using a Rating of Perceived Exertion (RPE) scale. We obtained data from 16 participants performing squats on a flywheel training machine while being equipped with six IMU sensors and an electrocardiography (ECG) sensor. Data was analyzed using multiple regressors, such as Support Vector Regression, Random Forests, and Gradient Boosting Regression Trees, to predict the personal exertion level on the processed IMU and ECG data. The best learning model achieved a mean absolute percentage error of 7.71% with a Pearson correlation coefficient of 0.85 and a R 2 of 0.48. Additionally, we investigated the impact of supplementing the IMU data features with ECG-derived heart rate variability (HRV) parameters in the training stage. Our results indicate that the HRV parameters derived from ECG significantly improve prediction results, with the training impulse (TRIMP) parameter acting as the most informative feature for predicting perceived exertion.}, address = {Los Alamitos, CA, USA}, author = {Albert, Justin and Herdick, Arne and Brahms, Clemens Markus and Granacher, Urs and Arnrich, Bert}, booktitle = {2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)}, keywords = {arneherdick bertarnrich justinalbert}, month = {dec}, publisher = {IEEE Computer Society}, title = {Using Machine Learning to Predict Perceived Exertion During Resistance Training With Wearable Heart Rate and Movement Sensors}, year = 2021 }

Data Augmentation of Kinematic Time-Series From Rehabilitation Exercises Using GANs. Albert, Justin; Glöckner, Pawel; Pfitzner, Bjarne; Arnrich, Bert (2021). 1–6.
[ Abstract ] [ BibTeX ] [ URL ] [ DOI ]
 
Machine learning, especially deep learning, offers great potential for medical applications. However, deep learning algorithms need a vast amount of training data. Especially in the medical domain, it is challenging to collect larger datasets. Access to patients can be limited, and data recording is mainly bound to laboratory settings requiring expertise from medical professionals. When involving a healthy control group, datasets are often unbalanced, with most data belonging to the control group. This paper proposes a data augmentation method to generate pose data of repetitive rehabilitation exercises trained on a specific population, e.g., a specific neurological disease. Our method is based on a generative adversarial network (GAN) that uses convolutional and long short-term memory (LSTM) layers. We evaluated our method using a dataset that contains rehabilitation exercises from stroke and Parkinson’s disease patients and a healthy control group. We demonstrated that a classifier trained using our augmentation method could distinguish between healthy, stroke, and Parkinson’s disease patients with an accuracy of 81%. In contrast, the same classifier achieved only 75% when using a standard resampling technique.
@inproceedings{9524284, abstract = {Machine learning, especially deep learning, offers great potential for medical applications. However, deep learning algorithms need a vast amount of training data. Especially in the medical domain, it is challenging to collect larger datasets. Access to patients can be limited, and data recording is mainly bound to laboratory settings requiring expertise from medical professionals. When involving a healthy control group, datasets are often unbalanced, with most data belonging to the control group. This paper proposes a data augmentation method to generate pose data of repetitive rehabilitation exercises trained on a specific population, e.g., a specific neurological disease. Our method is based on a generative adversarial network (GAN) that uses convolutional and long short-term memory (LSTM) layers. We evaluated our method using a dataset that contains rehabilitation exercises from stroke and Parkinson’s disease patients and a healthy control group. We demonstrated that a classifier trained using our augmentation method could distinguish between healthy, stroke, and Parkinson’s disease patients with an accuracy of 81%. In contrast, the same classifier achieved only 75% when using a standard resampling technique.}, author = {Albert, Justin and Glöckner, Pawel and Pfitzner, Bjarne and Arnrich, Bert}, booktitle = {2021 IEEE International Conference on Omni-Layer Intelligent Systems (COINS)}, keywords = {sys:relevantfor:dhc bertarnrich bjarnepfitzner justinalbert}, pages = {1-6}, title = {Data Augmentation of Kinematic Time-Series From Rehabilitation Exercises Using GANs}, year = 2021 }

2020

Will You Be My Quarantine: A Computer Vision and Inertial Sensor Based Home Exercise System. Albert, Justin; Zhou, Lin; Gloeckner, Pawel; Trautmann, Justin; Ihde, Lisa; Eilers, Justus; Kamal, Mohammed; Arnrich, Bert (2020). (Vol. 14)
[ Abstract ] [ BibTeX ] [ URL ] [ DOI ]
 
The quarantine situation inflicted by the COVID-19 pandemic has left many people around the world isolated at home. Despite the large variety of mobile device-based self exercise tools for training plans, activity recognition or repetition counts, it remains challenging for an inexperienced person to perform fitness workouts or learn a new sport with the correct movements at home. As a proof of concept, a home exercise system has been developed in this contribution. The system takes computer vision and inertial sensor data recorded for the same type of exercise as two independent inputs, and processes the data from both sources into the same representations on the levels of raw inertial measurement unit (IMU) data and 3D movement trajectories. Moreover, a Key Performance Indicator (KPI) dashboard was developed for data import and visualization. The usability of the system was investigated with an example use case where the learner equipped with IMUs performed a kick movement and was able to compare it to that from a coach in the video.
@inproceedings{albert2020quarantine, abstract = {The quarantine situation inflicted by the COVID-19 pandemic has left many people around the world isolated at home. Despite the large variety of mobile device-based self exercise tools for training plans, activity recognition or repetition counts, it remains challenging for an inexperienced person to perform fitness workouts or learn a new sport with the correct movements at home. As a proof of concept, a home exercise system has been developed in this contribution. The system takes computer vision and inertial sensor data recorded for the same type of exercise as two independent inputs, and processes the data from both sources into the same representations on the levels of raw inertial measurement unit (IMU) data and 3D movement trajectories. Moreover, a Key Performance Indicator (KPI) dashboard was developed for data import and visualization. The usability of the system was investigated with an example use case where the learner equipped with IMUs performed a kick movement and was able to compare it to that from a coach in the video.}, author = {Albert, Justin and Zhou, Lin and Gloeckner, Pawel and Trautmann, Justin and Ihde, Lisa and Eilers, Justus and Kamal, Mohammed and Arnrich, Bert}, booktitle = {14th EAI International Conference on Pervasive Computing Technologies for Healthcare - Demos and Posters}, journal = {Pervasive Health}, keywords = {bertarnrich justinalbert linzhou}, month = 8, publisher = {EAI}, title = {Will You Be My Quarantine: A Computer Vision and Inertial Sensor Based Home Exercise System}, volume = 14, year = 2020 }

Evaluation of the Pose Tracking Performance of the Azure Kinect and Kinect v2 for Gait Analysis in Comparison with a Gold Standard: A Pilot Study. Albert, Justin; Owolabi, Victor; Gebel, Arnd; Brahms, Markus Clemens; Granacher, Urs; Arnrich, Bert in MDPI Sensors (2020). 20(18)
[ Abstract ] [ BibTeX ] [ URL ] [ DOI ]
 
Gait analysis is an important tool for the early detection of neurological diseases and for the assessment of risk of falling in elderly people. The availability of low-cost camera hardware on the market today and recent advances in Machine Learning enable a wide range of clinical and health-related applications, such as patient monitoring or exercise recognition at home. In this study, we evaluated the motion tracking performance of the latest generation of the Microsoft Kinect camera, Azure Kinect, compared to its predecessor Kinect v2 in terms of treadmill walking usinggold standard Vicon multi-camera motion capturing system and the 39 marker Plug-in Gait model. Five young and healthy subjects walked on a treadmill at three different velocities while data were recorded simultaneously with all three camera systems. An easy-to-administer camera calibration method developed here was used to spatially align the 3D skeleton data from both Kinect cameras and the Vicon system. With this calibration, the spatial agreement of joint positions between the two Kinect cameras and the reference system was evaluated. In addition, we compared the accuracy of certain spatio-temporal gait parameters, i.e., step length, step time, step width, and stride time calculated from the Kinect data, with the gold standard system. Our results showed that the improved hardware and the motion tracking algorithm of the Azure Kinect camera led to a significantly higher accuracy of the spatial gait parameters than the predecessor Kinect v2, while no significant differences were found between the temporal parameters. Furthermore, we explain in detail how this experimental setup could be used to continuously monitor the progress during gait rehabilitation in older people.
@article{albert2020evaluation, abstract = {Gait analysis is an important tool for the early detection of neurological diseases and for the assessment of risk of falling in elderly people. The availability of low-cost camera hardware on the market today and recent advances in Machine Learning enable a wide range of clinical and health-related applications, such as patient monitoring or exercise recognition at home. In this study, we evaluated the motion tracking performance of the latest generation of the Microsoft Kinect camera, Azure Kinect, compared to its predecessor Kinect v2 in terms of treadmill walking usinggold standard Vicon multi-camera motion capturing system and the 39 marker Plug-in Gait model. Five young and healthy subjects walked on a treadmill at three different velocities while data were recorded simultaneously with all three camera systems. An easy-to-administer camera calibration method developed here was used to spatially align the 3D skeleton data from both Kinect cameras and the Vicon system. With this calibration, the spatial agreement of joint positions between the two Kinect cameras and the reference system was evaluated. In addition, we compared the accuracy of certain spatio-temporal gait parameters, i.e., step length, step time, step width, and stride time calculated from the Kinect data, with the gold standard system. Our results showed that the improved hardware and the motion tracking algorithm of the Azure Kinect camera led to a significantly higher accuracy of the spatial gait parameters than the predecessor Kinect v2, while no significant differences were found between the temporal parameters. Furthermore, we explain in detail how this experimental setup could be used to continuously monitor the progress during gait rehabilitation in older people.}, author = {Albert, Justin and Owolabi, Victor and Gebel, Arnd and Brahms, Markus Clemens and Granacher, Urs and Arnrich, Bert}, journal = {MDPI Sensors}, keywords = {bertarnrich justinalbert}, title = {Evaluation of the Pose Tracking Performance of the Azure Kinect and Kinect v2 for Gait Analysis in Comparison with a Gold Standard: A Pilot Study}, volume = {20(18)}, year = 2020 }

2017

Geometric Algebra Computing for Heterogeneous Systems. Hildenbrand, D.; Albert, Justin; Charrier, P.; Steinmetz, C. in Advances in Applied Clifford Algebras (2017). 27 599–620.
[ BibTeX ] [ URL ] [ DOI ]
 
@article{Hildenbrand2017GeometricAC, author = {Hildenbrand, D. and Albert, Justin and Charrier, P. and Steinmetz, C.}, journal = {Advances in Applied Clifford Algebras}, keywords = {justinalbert prior-hpi}, pages = {599-620}, title = {Geometric Algebra Computing for Heterogeneous Systems}, volume = 27, year = 2017 }