Hasso-Plattner-InstitutSDG am HPI
Hasso-Plattner-InstitutDSG am HPI
Login
  • de
 

Neuroergonomics in Software Engineering

Chair for Digital Health - Connected Healthcare
Hasso Plattner Institute

Office: Campus III Building G2, Room G-2.1.11
Tel.: 
Email: Fabian.Stolp(at)student.hpi.de
Links: Homepage

Supervisor: Prof. Dr. Bert Arnrich

Starting Date: 01.12.2021

 

Cases of occupational burnout, boreout, and other mental health problems associated with the workplace are on the rise and have received increasing attention in recent years. Occupational health and safety are the goals of ergonomics [1]. The field deals with the ”theoretical and fundamental understanding of human behavior and performance in purposeful interacting socio-technical systems, and the application of that understanding to design of interactions in the context of real settings” [2]. While the discipline of ergonomics has been around for about 70 years [2], a relatively recent branch shows the potential to provide more insight into the development and prevention of the aforementioned mental health problems. This field focuses on ”understand[ing] the neural bases of such functions as seeing, attending, remembering, deciding, and planning in relation to technologies and settings in the real world” [3] and is respectively called Neuroergonomics. Software engineering is no exception to other occupations. Overly challenging as well as overly easy tasks can lead to negative consequences for the mental health of software engineers [4].
While there are several other aspects of neuroergonomics, I focus on mental workload. Mental workload can be seen as a variable ”reflecting the interaction of mental demands imposed on operators by tasks they attend to” [5]. Its health and performance implications and novel possibilities of measuring it make it an especially interesting topic to study.
Overall, researching cognition and the human brain, in particular, has a long history. From inferring that the brain is the seat of sensation and thought to recent efforts of simulating the brain on different levels, knowledge was progressively collected [6]. The importance which is assigned to this research area today becomes apparent when considering that in large parts of the world heavily funded, major initiatives exist such as the Human Brain Project¹, a European Commission Future and Emerging Technologies Flagship, or the BRAIN Initiative², a U.S. Grand Challenge.
In the context of neuroergonomics and mental workload, noninvasive neuroimaging techniques play an important role. Several such techniques have been developed in the past and allowed to study the reaction of the human brain to stimuli. Three of the most popular ones are Functional Magnetic Resonance Imaging (fMRI), Functional Near-Infrared Spectroscopy (fNIRS), and Electroencephalography (EEG). EEG measures the electrical activity of the brain with the help of electrodes that are positioned at certain locations of the scalp. Out of the three techniques, I focus on EEG in the context of my research, due to it having several advantages in comparison to the other two that are, for example, listed by Kosti et al. in [7]. The most important aspect, I want to focus on here, is that recently, more and more Direct-to-Consumer (DTC) devices capable of measuring EEG data conveniently are entering the market. [7]
In addition to their availability, developments like brain-computer interfaces could lead to a growing acceptance and use of DTC EEG devices in the future and increasingly larger amounts of data being collected by such devices. Possible advantages of these devices in comparison to those that are used for medical gold standard EEG measurements are an easier use and shorter preparation times, or the possibility to obtain EEG measurements more often, over longer periods, or in more diverse and realistic environments. Challenges exist for example regarding the quality of the data provided by DTC devices. Although often marketed for use in realistic environments, DTC EEG devices conduct measurements that are quite susceptible to movements [7]. Therefore, such realistic environments should rather require little movement. This is the case for some of the major tasks conducted in software engineering such as code comprehension.
There has already been research conducted regarding the mental workload of software engineers using EEG. In some very recent publications, researchers started to evaluate the application of DTC EEG devices for assessing mental workload in general [10], and also mental workload in the software engineering sector [7, 11]. In both contexts, it was found that it was feasible to measure the mental workload with the help of the respective wearables. The authors saw limitations mainly in the sensitivity of the devices to motion, data quality, and the need for further studies in this area.
An often-used task, which was also used in the before-mentioned studies, in the context of software engineering for which mental workload is assessed is program comprehension, i.e. letting test persons interpret code. Although some experiments have already been performed regarding this task there remains a wide variety of programming paradigms and patterns of which the influence on brain activity remains unclear.
Besides the health aspect of mental workload and related positive aspects regarding performance [1, 12], further implications have been researched and could be further investigated. Crk et al. studied the connection of mental workload to the expertise of software programmers using EEG [13]. Furthermore, Medeiros et al. drew a link between mental workload, error rate, and code quality [11]. Also especially interesting in this context seems the evaluation of the relationship between measurable mental workload and established code complexity metrics [14]. Moreover, there is research regarding biofeedback augmented software engineering including a diverse set of goals, among others, the reduction of bugs found in code by detecting relationships between the sensor measurements of different body sensors and software error making and discovery [15]. Overall, using further sensor modalities to enrich the results of possible studies is an interesting aspect on its own. Examples are heart rate variability and electrodermal activity which are influenced by the autonomic nervous system and allow indirect inference of mental workload, or eye-tracking to detect certain lines of code that are especially challenging according to other sensor measurements [11].
On a computational level, neural signal and time series data processing and analysis are required to work on the proposed topic and the application of machine learning algorithms seems promising. For example, Fritz et al. investigated predicting task difficulty using classification models trained with sensor data from psycho-physiological sensors and feedback from software engineers [16].
From the creation of experiments, considering the characteristics of novel wearable sensors, to data processing and analysis to the creation of models for the prediction of mental workload, error making, or task complexity there are many areas in the field of neuroergonomics that require further investigation and pose interesting challenges. I concentrate on the assessment of mental workload in software engineering with the help of DTC EEG devices to systematically approach those challenges. Such efforts are supported by the software engineering field playing an important role and the DTC EEG devices present at the Hasso Plattner Institute. Furthermore, there exist potential points of connection to neurodesign research³ at the institute.
Overall, I would like to contribute to the efforts of understanding mental workload in software engineering to work towards reducing the risks for occupational mental health problems such as bore- and burnout, and towards the creation and utilization of respective technological and software designs and solutions.

 

[1] Jan Dul and W Patrick Neumann. “Ergonomics contributions to company strategies”. In: Applied ergonomics 40.4 (2009), pp. 745–752.
[2] John R Wilson. “Fundamentals of ergonomics in theory and practice”. In: Applied ergonomics 31.6 (2000), pp. 557–567.
[3] Raja Parasuraman and Glenn F Wilson. “Putting the brain to work: Neuroergonomics past, present, and future”. In: Human factors 50.3 (2008), pp. 468–474. 
[4] Ramyashilpa D Nayak. “Anxiety and mental health of software professionals and mechanical professionals”. In: International Journal of Humanities and Social Science Invention 3.2 (2014), pp. 52–56. 
[5] Brad Cain. “A review of the mental workload literature”. (2007). 
[6] Xue Fan and Henry Markram. “A brief history of simulation neuroscience”. In: Frontiers in neuroinformatics 13 (2019), p. 32. 
[7] Makrina Viola Kosti et al. “Towards an affordable brain computer interface for the assessment of programmers’ mental workload”. In: International Journal of Human-Computer Studies 115 (2018), pp. 52–66. 
[8] Marcello Ienca, Pim Haselager, and Ezekiel J Emanuel. “Brain leaks and consumer neurotechnology”. In: Nature biotechnology 36.9 (2018), pp. 805–810. 
[9] Marcello Ienca, Pim Haselager, and Ezekiel J Emanuel. “Author Correction: Brain leaks and consumer neurotechnology”. In: Nature biotechnology 37.7 (2019), pp. 819–819. 
[10] Ekaterina Kutafina et al. “Tracking of Mental Workload with a Mobile EEG Sensor”. In: Sensors 21.15 (2021), p. 5205. 
[11] J ́ulio Medeiros et al. “Can EEG Be Adopted as a Neuroscience Reference for Assessing Software Programmers’ Cognitive Load?” In: Sensors 21.7 (2021), p. 2338. 
[12] Daniel Graziotin, Xiaofeng Wang, and Pekka Abrahamsson. “Happy software developers solve problems better: psychological measurements in empirical software engineering”. In: PeerJ 2 (2014), e289. 
[13] Igor Crk, Timothy Kluthe, and Andreas Stefik. “Understanding programming expertise: an empirical study of phasic brain wave changes”. In: ACM Transactions on Computer-Human Interaction (TOCHI) 23.1 (2015), pp. 1–29. 
[14] Norman Peitek et al. “Program comprehension and code complexity metrics: An fmri study”. In: 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE). IEEE. 2021, pp. 524–536. 
[15] Ricardo Couceiro et al. “Biofeedback augmented software engineering: monitoring of programmers’ mental effort”. In: 2019 IEEE/ACM 41st International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER). IEEE. 2019, pp. 37–40. 
[16] Thomas Fritz et al. “Using psycho-physiological measures to assess task difficulty in software development”. In: Proceedings of the 36th international conference on software engineering. 2014, pp. 402–413.

Publications

Behrens, F., Bischoff, S., Ladenburger, P., Rückin, J., Seidel, L., Stolp, F., ... & Hunger, M. (2018, April). Metaexp: Interactive explanation and exploration of large knowledge graphs. In Companion Proceedings of the The Web Conference 2018 (pp. 199-202).