Silent speech with ultrasound

https://alephneuro.com/blog/silent-speech

by chrwn • 15 days ago

109 35 15 days ago

hdjrudni• 15 days ago

The two biggest hardware challenges are reducing the size and weight of the ultrasound probe and replacing ultrasound gel with a more practical coupling material, such as hydrogel. We think both are solvable, making it possible for the probe to eventually become a lightweight wearable or adhesive patch.

Not sure I'd want to put an adhesive patch on my neck every morning so I can silently talk to an LLM in the cubicle farm. I hope this is not our future.

Very cool tech though and surprisingly good results for so little training.

I think time might be better spent improving a lip reading model (no adhesive required), assuming we're unable to read brainwaves directly.

delichon• 12 days ago

Keeping a camera focused on the lips could get awkward. A hands free throat patch may be more ergonomic.

vadimsc8• 11 days ago

It won’t necessarily be adhesive. It could be attached mechanically - we’re still exploring the form factor.

The idea is that you won’t need cubicles anymore. Your hands will always be free because you won’t need a keyboard or mouse. You’ll be able to control the digital world using only your tongue.

The future is bright: you’ll talk to your Jarvis while driving through the Norwegian fjords.

ubedan• 14 days ago

Wonderful tech, and video example. I think there may also be a special forces application, but I don't know enough about how well their current solution works.

In the office, a non-contact video solution (lip reading) is likely to be far more popular, but a lot depends on which is more accurate.

readthenotes1• 12 days ago

I wonder if, like with lip reading, they switch from American English to a different language that's not so peculiar they would have and much less error rate

m463• 12 days ago

reminds me of the handwriting recognition language Graffiti from palm/handspring days.

It ended up altering my handwriting even after I stopped using it.

nine_k2• 12 days ago

I suppose Spanish or German would be easier to recognize than English, due to the more distinct sounds, but Chinese would be much harder, due to the tones.

vadimsc8• 11 days ago

Tongue movement detection cannot track only six phonemes: - b - p - m - f - v - h

That’s a remarkably small number compared with other methods. English could easily be optimized to achieve a sub-5% WER with a very small model.

nine_k2• 12 days ago

That could be immensely helpful for people who cannot speak due to vocal chord problems.

It could also be the ultimate, always-on remote control for everything around, with a near-zero error rate.

vadimsc8• 11 days ago

Exactly - a universal input device.

It could help not only people with vocal cord problems, but also essentially anyone who is paralyzed from the neck down.

gblargg• 12 days ago

I remember a story decades ago about "subvocal" speech, similar to this: https://spacenews.com/nasa-develops-system-to-computerize-si...

ehead• 12 days ago

These guys are on a roll!

vadimsc8• 11 days ago

eviluncle• 12 days ago

Interesting. This seems similar to (though using different methods/techniques) what q.ai (that was sold to Apple for $2b) is developing.

See this hn thread about it: https://news.ycombinator.com/item?id=46816228

willwade• 12 days ago

Sony presented a simpler solution to this at CHI https://dl.acm.org/doi/10.1145/3772318.3791397 - check video in that. Also it seems every Cornell student is working on this. Just search on YouTube Cornell silent speech

paufernandez• 12 days ago

I had thought of this, so nice to see it done!

stanislavzza• 12 days ago

Straight out of Life Artificial...

mike_hock• 12 days ago

I find it annoying to try and talk completely silently. I'm always whispering at least a bit. I can't imagine doing this for the entirety of a call.

vadimsc8• 11 days ago

When I was collecting data, I spoke silently for more than two hours.

After the session, I tried to use Wispr Flow but forgot that I was still speaking silently, so it didn’t work.

When I finally said it out loud, it felt like such a hassle that I essentially sold myself on the idea of silent speech.

thomascountz2• 11 days ago

The use of ultrasound is interesting! The first step (identifying speech) is called subvocalization recognition, and early research (Chuck Jorgensen, et. al (2023) from NASA) use EMG[1].

[1] https://scispace.com/pdf/sub-auditory-speech-recognition-bas...

thomascountz2• 11 days ago

s/2023/2003/

npilk• 11 days ago

I'm very eager for subvocalization tech to take off - this, AlterEgo, etc.

I have never liked talking aloud to Siri or similar, but I could see using "voice" as an interface for so much more if I could speak silently to my device.

vadimsc8• 11 days ago

Voice is still a very underexplored modality. Right now, it’s used mostly for dictation, but it could also be used for editing.

The funny part is that the tongue itself could act as a cursor or perform gestures, which sets this approach apart from other silent-speech methods.

dschwede• 11 days ago

Really impressive. Seems like this could help people with health problems like ALS or Parkinson's, if you can discern clear intent from their tongue muscles. Or just help people who are intubated in the hospital.

smath• 11 days ago

Very cool! While this is a different type of inverse problem, it reminds me of a radiolab story [0] about a device to help blind people “see” using electrical impulses on a metal strip under the tongue.

[0] https://radiolab.org/podcast/seeing-tongues

albert_e• 11 days ago

This sounds promising -- would love to see this become more practical.

With wearable devices becoming more common - I am anticipating a wave of "sensors" that can be as simple as small band-aid patches that wirelessly send data to your smart device. Those sensors could also open up human-coputer-interface innovations like these.

---

In similar space ...

There was a post about a thought-to-text project from MIT no less --

"AlterEgo"

10 months ago: https://news.ycombinator.com/item?id=45174125 8 years ago: https://news.ycombinator.com/item?id=16780357

When I saw the demo posted last year it left me with an uneasy feeling -- gut feel said it was more marketing than a real working technology demo. Nothing seems to have come out of that lab since then -- strengthemning my suspicions.

fatuna• 11 days ago

I remember reading about something like this tech in one of Asimov's Robot novels. Thought about trying to build something like this, but don't really have the skills. Awesome they did build this! I guess if they make it easy to train on your speech (just combine it with speech to text), it's a product waiting to happen!

vadimsc8• 11 days ago

Thank you for the warm words!

Model already generalizes on out-of-distribution people, so you actually don't need to train a model for each person

singpolyma32• 11 days ago

I'm having trouble imagining how one would use this. How do you speak without making sounds?

jimbo_joe• 11 days ago

You move your mouth and tongue as if you are gonna make the sounds, just don't exhale the air. They have a demonstration video in the blog post. As somebody else have already pointed out, it's a great tool for interacting with always-on camera devices like smart assistants, makes it a bit less weird.

singpolyma32• 11 days ago

Oh I see. So no sound but not with mouth closed

vadimsc8• 11 days ago

One cool fact: with silent speech, you can “speak” not only while exhaling, but also while inhaling

didntknowyou• 11 days ago

there is a perception that ultrasound is not as dangerous as xrays (radiation etc). As we find more use-cases for it we need to be aware that the damage caused by blasting organs with energy waves is not negligible.

mikeodds• 11 days ago

is this true for medical use cases where it’s primarily for imaging? i.e. echocardiogram or pregnancy scans

laxpri• 11 days ago

I am bullish in this tech , but there is something missing to become this to private like headphones .another thing is it still has to do some physical movement which is not going to too much fast like direct brain to text .

I am researching in this field right now, MY ideas are if there is minimum amount of muscle movement will be the best .

Silent speech with ultrasound

Related Stories