Translated by an automated translation plugin.
For people with hearing loss, it is important that speech is clearly articulated so that the content can be understood. In addition to articulation, the speaker’s pitch is also a key factor. However, speech intelligibility also depends on external conditions such as room acoustics, noise, background sounds or conversations; these can make hearing more difficult. In contrast, hearing aids, learning to read lips or using an electronic device to follow speech simultaneously can make hearing and understanding easier. The frequency range in which the most important speech-acoustic features are found lies between approximately 1 and 4 kilohertz. Nathalie Giroud notes that ‘after repeated requests from the person with hearing loss, conversation partners often start to speak more loudly, even though this does not help the person concerned’.
Research findings show that the comprehension of spoken language does not depend solely on hearing. Equally important are auditory and linguistic processing in the brain, as well as cognitive processes such as attention and working memory. Hearing loss and a reduced ability to perceive subtle acoustic differences are associated with changes in grey matter in cortical regions of the brain. By monitoring brain activity in real time, it is possible to observe how neural changes are linked to difficulties with speech intelligibility and language comprehension. “This finding goes beyond the mere effects of hearing sensitivity and has important implications for the psychosocial health, quality of life and communication of those affected,” emphasises Nathalie Giroud. Her conclusion is therefore: “Simply amplifying sound with hearing aids is often not enough to overcome the challenges of speech comprehension faced by older people or those with hearing loss.”
Spoken language produces sound waves that cause the hair cells in the inner ear to vibrate; these vibrations are then converted into electrical signals. The signals travel via the auditory nerve to the brain, where frequencies, volumes, timbres and spatial origin are analysed and interpreted. Speech processing relies on complex brain networks. There is no single area of the brain that processes speech. Different areas of the brain work together and constantly exchange information. This interaction is individual, which is why two people with similar hearing loss may understand speech to varying degrees.
In people with hearing loss, the signals from the inner ear are transmitted to the brain not only at a lower volume, but also in a more distorted and incomplete form . The ear must fill in the gaps in the information contained in the signal, extract speech from the external conditions mentioned earlier, and complete words or sentences using context and prior knowledge. The brain must therefore exert greater auditory effort and cognitive processing capacity to compensate for what is missing. This results in other cognitive processes being neglected.
Nathalie Giroud is interested in where this compensatory neuroplasticity – the reorganisation of brain processing – begins; how and where the brain recognises environmental stimuli and how it can respond to them; and how it can express thoughts and feelings and communicate them.
Compensating for missing auditory information through increased cognitive processing, attention and alternative neural strategies often works only to a limited extent and consumes energy. Those affected can often follow conversations, but are exhausted afterwards. Whilst compensation improves speech comprehension, it does not fully replace the missing auditory information. Nathalie Giroud refers to studies showing that, as speech signals weaken, additional brain networks are activated, particularly prefrontal and attention-related regions. She and her team are investigating which networks are active and how they interact. The words ‘sofa’ and ‘chair’, for example, are represented by different patterns of activity in networks of nerve cells. Functional magnetic resonance imaging (fMRI) can be used to indirectly measure activity in brain regions via changes in blood flow. Machine learning is often used to make predictions, such as which categories of terms or objects are currently being processed.
‘We are investigating whether the measured patterns for “sofa” and “chair” are similar,’ explains Nathalie Giroud. Electrophysiological methods (EEG, MEG) allow electrical or magnetic activity to be measured with high temporal resolution. Differences can already be detected immediately after a word is presented. These differences may be large, moderate or only slight. From this, a semantic map of the brain can be created.
Nathalie Giroud and her team are also attempting to understand sentence structure and meaning. Unlike words, syllables are acoustically marked because they are spoken measurably louder, longer or higher in pitch, and are therefore more easily audible. People with a high working memory capacity achieve better speech comprehension scores than people with a comparable audiogram. This suggests that cognitive compensation can account for a significant proportion of the differences in performance.
Phantom noise exists because it has been imprinted on the brain’s neural circuits. A perception arises without a corresponding external stimulus. In the case of hearing loss, fewer acoustic signals reach the brain. The auditory system attempts to compensate for this by increasing its internal amplification. Synapses are junctions between nerve cells where electrical signals are converted into chemical signals and then back into electrical signals. If an imbalance arises between excitatory and inhibitory neurotransmitters, this can result in overactive neural networks. These are perceived as sounds such as tones, whistling or hissing.
Nathalie Giroud points out that tinnitus cannot always be explained by a lack of auditory information. Stress, tiredness or tension can also exacerbate tinnitus, even though nothing has changed in the ear itself. “Research into tinnitus helps us understand how the brain constructs acoustic reality – a process that also underlies speech comprehension,” says Nathalie Giroud. Viewing chronic tinnitus as a disorder within the brain opens up fundamentally different therapeutic approaches.
‘Constant noise is harmful to our ears and therefore impairs our ability to understand speech.’
Prof. Nathalie Giroud, University of ZurichTargeted auditory and perceptual training helps to prevent or mitigate unfavourable adaptations caused by sensory overload in the brain. These overcompensatory mechanisms of the brain, which can take many different forms, are systematically addressed in needs-based auditory training. Nathalie Giroud explains: ‘In our practical training sessions, we use virtual reality headsets and a dialect-based speech model for the avatar, which acts as a conversation partner for the person undergoing training. Using this tool, we simulate situations such as those in a restaurant. To measure progress, we alter the environmental setting or play back similar voices.’
When participants have to process a lot of different information at the same time, over-activation can occur. The brain should be focusing on the content, but is distracted by background noise. The adaptive algorithms built into the programme help to find the optimal configuration so that the brain can once again better distinguish between content and ambient noise.
There are also auditory cognitive training programmes that take a holistic approach. Together with Pro Audito Switzerland, Giroud’s team has developed a lip-reading training programme. Lip-reading is a visual process designed to support hearing. Those affected can follow and make use of mouth movements because the visual process is much quicker to grasp than the auditory one. Differences between the letters M and N or F and S are easy to see but difficult to understand because they are high-frequency sounds. The training programme using virtual reality glasses therefore incorporates these visual aspects. People with hearing loss and an audiovisual deficit thus also make progress with tasks they have not practised. The ability to apply what has been learnt to new situations is an indicator of successful training, which can also be carried out at home.
Combining listening and reading enables people to understand spoken language better. There are many listening situations in which this combination is very helpful, such as in online meetings, video clips, films or television programmes. Despite these technical advances, the sheer volume of information can once again become overwhelming for people with hearing impairments. In such cases, it is advisable to focus on what is being heard and to ignore lip-reading or the text. It is therefore crucial to identify the personal challenges faced by people with hearing impairments.
For neuroscientists conducting research, developments aimed at improving hearing ability at the brain level are of great interest. However, before these findings can be translated into practical applications, complex regulatory requirements must be met. The Swiss National Science Foundation (SNSF ) and Innosuisse, both government funding organisations, offer the BRIDGE programme, which supports researchers in rapidly transforming their findings into products or services.
“We are delighted with this programme. It allows us to further develop and scale up our test systems, which are still at an early stage, in collaboration with partners, so that they can be made available for use by audiologists and medical professionals alike,” emphasises Nathalie Giroud.
In their research projects, Nathalie Giroud and her team have repeatedly found that the voices of people with hearing loss can provide insights into their state of health. If speech recordings are available that have been made over a longer period of time, a system could in future detect individual deviations from a person’s usual voice. Such changes are often more meaningful than comparisons between those affected and the general population.
In a SATW interview on hearing ability, hearing care professional Christoph Schwob made the same point regarding the early administration of an audiogram in young people. The voice therefore offers itself as a potential early warning signal. Computer-assisted analyses can evaluate countless characteristics in a voice simultaneously. AI-supported voice analyses will help to identify people with unusual voices or changes in their voice at an early stage. Much like a blood pressure monitor, which provides the first indications of a possible condition. Studies show that changes in the voice in Parkinson’s disease can be detected years before motor symptoms appear. This also applies to Alzheimer’s disease, where speech and vocal patterns change even in the early stages.
Several start-ups have chosen Parkinson’s disease as their first area of application. In Parkinson’s disease, the fine motor control of the larynx, tongue and respiratory muscles is affected before tremors become apparent. These companies are pursuing the idea that changes in voice, language, breathing and speech patterns can be used as digital biomarkers (voice stability, intonation, articulation, speech rate and rhythm) for diseases. The bottleneck is clinical validation, as proof of benefit can only be provided through validated studies. The path for these new products into everyday medical practice is bumpy and protracted. A good example of successful implementation in this area is the German company EVOCAL Health. The company has developed a device-independent platform that enables the collection, monitoring and analysis of patients’ voice data to identify a wide range of characteristics in the human voice that are directly linked to medical conditions.
| Role | Title + Name |
|---|---|
| Text by | Kaspar Eigenmann, Daniel Gygax |
| Expertise | Nathalie Giroud |
