Brain data is frequently visualized through colourful fMRI images or EEG plots. However, many aspects of brain functioning cannot be intuitively comprehended or analyzed by gazing at static images only. Brain activity involves dynamics and rhythms that visual depictions can only convey in a limited manner. By utilizing 3D sonification models tailored for headphones and multi-speaker setups in a room, brain data can be presented in an auditory manner, providing distinctive experiences of the dynamics in ever-changing neural activities.
The objective of a project by Luca Hilbrich, Lukas Hartmann, Philipp Steigerwald, and Tim Strauch is to present EEG data via a 3D sound installation. Specifically, the project aims to present rhythms and topographies of brain activity in an intuitively understandable way, even for individuals without extensive prior training in neuroscience. To control audio parameters, the team monitors power in different EEG frequency bands over time, such as alpha, low beta, and high beta. The distribution of audio channels corresponds to EEG channel locations on the skull: EEG activity recorded at frontal positions triggers speakers at the front of the room, while EEG activity from lateral/medial positions on the skull is represented by speakers in the middle of the room. EEG signals at occipital positions, from the back of the head, are conveyed through speakers at the back of the room.
You can try out the web application HERE.
Sonification involves the use of non-speech audio to convey information or represent data perceptually. It offers advantages in terms of high temporal, spatial, amplitude, and frequency resolution, making it a promising alternative or complement to visualization techniques. Sonification has been applied in various contexts, including healthcare , the detection of gravitational waves , auditory altimeters , and weather sonification .
One significant consideration in sonification projects is the approach to synthesis: How can musical elements like rhythm or pitch be employed to enable listeners to intuitively grasp the data being presented? The design of sonification approaches is highly dependent on their intended applications. For instance, when the objective is to monitor EEG alpha-activity in a participant engaged in a creative task, the acoustic signal for alpha activity should stand out within the overall auditory experience.