In a packed room, individuals using hearing aids often find it impossible to concentrate on the sounds or voices they wish or need to hear. Advances in auditory neurotechnology could potentially improve sounds and voices while minimizing noise instantly.
Hearing aids boost all sounds "indiscriminately," according to a team of researchers developing a system that could enable brain-controlled hearing aids.
"Current hearing aids excel at amplifying sounds and voices, but they fall short in addressing the classic 'cocktail party problem'—determining which voice is important to the listener," stated Vishal Choudhari, lead author of their study published in *Nature Neuroscience*.
Focusing on a single voice in a crowded space can demand significant effort. "Hearing isn't just about whether words are understood correctly," Choudhari told DW. "Two individuals might both grasp what's being said, but one may require much more mental energy to keep up with the conversation. That can become draining over time."
Consequently, many people stop using hearing aids precisely when they need them most—in restaurants, cafeterias, parties, or bustling social settings.
To address this, Choudhari and his colleagues are working on creating smart technology that can identify what a hearing aid user is paying attention to. Their goal is to amplify that specific sound or voice while lowering the volume of all other sounds, voices, or background noise.
To achieve this, they developed a system that reads brain waves and, with the help of artificial intelligence, determines what a listener is focusing on.
"Many hearing aids use beamforming, which boosts sounds from a specific direction, typically in front of the user. But real conversations are fluid," Choudhari explained. "People move their heads, shift their focus, or even listen to someone without looking directly at them."
**Better hearing — In Good Shape**
To view this video please enable JavaScript, and consider upgrading to a web browser that supports HTML5 video
**Brain-controlled hearing from theory to practice**
Under the guidance of Nima Mesgarani, a professor and principal investigator at Columbia's Zuckerman Institute, Choudhari and his team created real-time machine learning algorithms capable of analyzing brain waves and identifying the conversations that four normal-hearing test subjects were following.
The researchers termed this a closed-loop auditory attention decoding (AAD) system. They aimed to determine if AAD could be precise and swift enough to selectively amplify one speaker's voice while reducing background voices.
Although the concept sounds promising, it is far from being ready for widespread use. Currently, the approach requires electrodes attached to the brain in a clinical environment.
In the study, the four test patients were already undergoing brain monitoring for epilepsy, meaning they had intracranial electrodes in place, which was convenient for the researchers.
Participants were exposed to recordings of two competing sound sources from small speakers placed on the left and right. The recordings included people of various gender combinations discussing food, travel, and exercise, with their words mixed with what the researchers called "multi-talker babble" and pedestrian noise.
Mesgarani and colleagues had previously discovered in 2012 that brain waves rise and fall based on a person's focus, showing peaks and troughs that align with the sounds and silences in a conversation.
In the new study, "the volume of the competing conversations was adjusted dynamically in real time based on the decoded brain signals," Choudhari said. "The conversation being attended to became louder, while the competing conversation grew quieter."
The system performed well, according to the researchers, whether participants were instructed to listen to a specific conversation and then asked to shift their attention, or when they freely chose a conversation.
**Challenges and promises ahead for brain-controlled hearing**
Other experts in the field acknowledge the progress made by Choudhari and Mesgarani's work.
Volker Hohmann, a professor of Auditory Signal Processing at the University of Oldenburg's Cluster of Excellence Hearing4all, highlighted "the impressive accuracy with which auditory attention can be decoded from brain signals, especially when intracranial electrodes are used."
However, in an email to DW, Hohmann noted that the system is not yet practical for everyday use—a point the researchers also acknowledged. The acoustic conditions were "completely fixed, with the listener not moving... Acoustic communication in daily life is far more dynamic," Hohmann said.
**The science of good hearing**
To view this video please enable JavaScript, and consider upgrading to a web browser that supports HTML5 video
"The issue is that if one source is amplified, the other becomes harder to hear, making it difficult to switch attention to the quieter source and decode the change," said Bernhard Seeber, a professor of Audio Information Processing at the Technical University of Munich, in an email to DW.
Seeber credited the US team with demonstrating that the system could respond in real time when a listener shifts attention, but he noted that further research is needed "to achieve reliable real-time attention decoding from skin-electrode signals"—meaning a less invasive method for monitoring brain signals than intracranial electrodes.
That is also where Choudhari sees potential for the system and, ultimately, its integration into smart, wearable technology: "Imagine smart glasses or earbuds that use your brain signals to know which conversation you're listening to, can summarize key information, or even assist with memory and note-taking in noisy environments," he said.
*Vishal Choudhari conceived the research as a PhD candidate under Nima Mesgarani at Columbia University's Zuckerman Mind Brain Behavior Institute. Choudhari is now a Founding Research Scientist at an AI company in Seattle, US.*
Edited by: Richard Connor