Three research teams surgically placed electrodes onto a human brain recently, so AI could turn patients brain signals into computer-generated speech.
Scientists constructed words and sentences from brain signals by using artificial intelligence. Some of those sentences were gibberish, but encouragingly some made sense. This has never been achieved before.
Dr. Stephen Hawking was unable to speak. He communicated by means of a trigger on his cheek, which he activated by tensing. This limited his ability to show tone or emotion or his ability for sarcasm would certainly have been hindered.
A brain-computer interface could change all of that. AI would allow for speech to be recreated directly from brain signals, this creating a more human-like voice. This significant breakthrough for people like the late Dr. Hawking or for anyone who has lost their voice through stroke or illness.
So How Did They Do It?
Scientists monitored certain parts of the brain as people read aloud, silently mouthed speech or listened to recordings. Using Artificial Intelligence, they were able to reconstruct speech that was actually understandable.
Nima Mesgarani, a computer scientist at Columbia University commented: “We are trying to work out the pattern of neurons that turn on and off at different time points and infer the speech sound.”
Every person is different so computer models must be tailored to each individual. These models need very precise data to be trained properly. Unfortunately, data can only be gathered by opening a person’s skull. I can’t imagine a long line of volunteers.
How Is The Data Collected?
There is a very small window that researchers can use (between 20 and 30 minutes). The recordings are obviously very invasive and therefore, only patients undergoing surgery can partake.
Patients undergoing a brain tumour removal would be ideal candidates. Surgeons must monitor electrical readouts from the exposed brain in order to avoid key speech and motor areas.
An epileptic patient undergoing surgery would be a good candidate because during surgery they are implanted with electrodes to locate the origin of their seizures. Both of these scenarios leave researchers a small window to collect data.
Now for the technical bit…
The valuable data is fed through neural networks, which processes complex patterns by passing information through layers of computational “nodes.” The networks learn by adjusting connections between nodes. In the experiments, networks were exposed to recordings of speech that a person produced or heard and data on simultaneous brain activity.
In total there were three amazing research teams and their patients behind this breakthrough.
Let’s find out about them…
The first team led by Nima Mesgarani extracted data from five people with epilepsy. Researchers analysed recordings from the auditory cortex. This particular part of the brain is active during both speech and listening. Patients simply listened to recordings of people naming digits and then the results were analysed.
Excitingly, spoken numbers were reconstructed by the computer. By using just neural data, the AI was able to speak numbers which a group of listeners identified with 75% accuracy.
Another team, led by computer scientist Tanja Schultz extracted data from six patients undergoing brain tumour surgery. Electrodes were placed on the brain’s speech-planning and motor areas and then each patient spoke simple words aloud.
A trained network mapped the electrode readouts to the audio recordings and from there, AI reconstructed words. However, this experiment was not as successful as the first with only 40% of the words understandable.
A research team out of San Francisco astonishingly reconstructed entire sentences from brain activity. Three epilepsy patients read simple sentences aloud and scientists then extracted data from the speech planning and motor section of the brain to analyze the results.
An online test was created to determine the success of the experiment. There were 166 volunteers who listened to the reconstructed sentences. Out of ten selections, volunteers chose which one they heard. The results were promising with some of the sentences correctly identified 80% of the time.
However, “What we’re really waiting for is how these methods are going to do when the patients can’t speak,” says Stephanie Riès, a neuroscientist at San Diego State University in California.
This is just one of many exciting applications of AI in medicine. While this research is only beginning, it has again pushed the boundaries of what was thought possible.
Get the Alldus Advantage
Alldus International covers your business consulting, staffing and networking needs across the United States and Europe. Alldus can help you discover how Data Science and AI can transform your company with an unrivalled network of C-Suite executives and Senior AI Professionals.
Become a member of the Alldus community and enjoy and enjoy our AI Meetups. Once a month our community gathers to hear from some of the leading experts in the world of Data Science and AI. Our speakers come from all over the world including Dublin, Boston, New York and Berlin.
We also have the AI Mentors podcast where our experts will provide mentoring to Alldus members and of course, the AI in Action podcast where each week we have guests from all over the world talking us through their education, career and more.
Become an Alldus member and get the Alldus advantage by signing up to our newsletter below.