Scientists Use GPT AI to Passively Read People's Thoughts in Breakthrough

https://www.vice.com/en/article/4a3w3g/scientists-use-gpt-ai-to-passively-read-peoples-thoughts-in-breakthrough

109 104 3 years ago

phkahler• 3 years ago

Black Mirror here we come

mdp20213• 3 years ago

...a consequence of "Philomena Cunk". (Not precisely (and in a way the opposite - at least Cunk investigates), but to stay in the metaphor.)

gjvc• 3 years ago

"Philomena Cunk" is a fictional character in a mockumentary. She's not investigating anything.

stuartjohnson12• 3 years ago

oh no

dmm• 3 years ago

Perhaps we can finally know what the dead salmon is thinking?

https://archive.is/1WTnq

airstrike• 3 years ago

:-O :-) :-(

techwiz137• 3 years ago

This is coming one way or another and is pretty frightening. Mass surveillance is one thing, but also personal interrogation. You can extract the deepest darkest secrets of any one individual.

The thin line of privacy will disintegrate into nothing.

bandyaboot• 3 years ago

Your point is well-taken. But also worth considering is that people with, say, ALS might have quite a different perspective on this.

nivenkos• 3 years ago

Or even arthritis.

f6v3• 3 years ago

Everything has two sides. Chemotherapy discovery was aided by chemical weapons.

brightstep• 3 years ago

That’s not true. We don’t need to do this. Absent intervention, sure these technologies are likely to be introduced and invade our lives. But we are humans, not computers, and despite the decisions of the last several decades, we are not required to allow market forces and the profit motive algorithm to make all decisions in our society. We can make whatever rules and decisions we want. It won’t come without a fight, but this horrifying development seems a pretty good place to start.

freedomben2• 3 years ago

What government is going to turn down these capabilities? Even if they banned "private sector" development, you can bet they're still gonna go down this road.

And frankly, that might be for the best. Imagine whatever country is most antagonistic toward your own, and now imagine their military fully develops this and yours doesn't. How safe/secure are you feeling?

13years4• 3 years ago

All roads of AI are converging on the end of privacy. A free society does not exist in the absence of privacy.

When you lose privacy you lose ownership over your own thoughts. You lose agency over the direction of your own life.

Humanity suffers by either scenario, either by our own agency in control of power we are not prepared to manage or we will be managed by power that we can not control.

Many have their focus looking forward to the unknown concerns of AGI; however, these other issues are already nearly imminent and are far less present in discourse around AI.

The impacts on socialization and society in general from AI performing exactly as we have requested it may be one of the greatest threats. I've written quite a bit on that aspect, FYI - https://dakara.substack.com/p/artificial-intelligence-ai-end...

stickfigure5• 3 years ago

You have to physically climb into an multimillion dollar, 10 ton, liquid-helium-cooled machine and spend hours training it on your brain. We're a long way from the end of private thought, nor is there a roadmap to it.

If someone is working on a handheld device that can perform MRI-level scans from across the room, I would be worried about the privacy implications of that technology, not AI.

13years4• 3 years ago

That is missing the point. It is the trajectory. Machines that can analyze all orthogonal data to understand your behavior is not limited to a single dimension of a MRI machine.

Facial expression analysis, all of your online conversations, the existence of your entire life is being monitored and recorded. There will be enough data to perceive beyond the veil into the thoughts of the mind.

cma• 3 years ago

Liquid helium wouldn't be a barrier to being used on key individuals (say leaders of opposition parties in some large authoritarian-ish country). The training would be if it requires cooperation.

bookofjoe• 3 years ago

Wearable MRI helmet makes brain scans quicker, quieter and crisper

https://healthimaging.com/topics/medical-imaging/magnetic-re...

Auxetics-Inspired Tunable Metamaterials for Magnetic Resonance Imaging

https://onlinelibrary.wiley.com/doi/10.1002/adma.202109032

waboremo• 3 years ago

You have to lug around a giant bulky laptop and you can't even call people on it! We're a long way from mobile computing.

Anyways, even given current limitations, I'd likely side with the very researchers working on such. They explicitly highlight privacy reasons as a serious concern. Going out of their way to highlight misuse through bypassing requirements (cooperation of subject) and intentionally misinterpreting for nefarious reasons.

mock-possum• 3 years ago

You have to physically climb into an multimillion dollar, 10 ton, liquid-helium-cooled machine and spend hours training it on your brain

the article suggests it could be accomplished with a neurosurgical implant as well - and which, honestly, if I could transcribe my thoughts to review later, I'd love to try that out. In which case, the question of security moves to the matter of accessing the implant.

anigbrowl2• 3 years ago

You do now. But if you are arrested, you can be compelled to do that, and in a few years EEG skullcaps will probably be sufficient.

Honestly, it baffles me that so many people on a site devoted to technology evaluate long-term trends based on current capabilities. Capacity is going to continue to double every ~2 years. If we can't make transistors 2x smaller, we'll find a different architecture to make transistor arrays 2x larger, or [something].

Technical progress compounds. It's often lumpy rather than linear, but it's going to keep imposing an accelerating effect wherever people see profit in applying it.

Gigachad• 3 years ago

Just wait until we get the musk nuralink chip which will eventually be mandated to prevent terrorism and misinformation.

mdp20213• 3 years ago

A free society

It is not just that. The mandate to privacy is anterior (and ancient), in terms of "do not expose yourself to the judgement of the ignorant".

deet• 3 years ago

Loss of agency is a deeply profound concern that is not discussed enough.

So much of human life is tied to the reward of the sense of accomplishment, and AI threatens that in a meaningful way. From large things, like seeing a piece of software you built ship and get used, or seeing your child graduate from college after years of hard work raising them, to the small things, like even eating a delicious meal you cooked yourself.

When machines are not only offering to solve problems and perform actions for us, but are reading are thoughts and doing everything for us before we even make an attempt, I worry that sense of accomplishment will become ever less frequent.

And while we can compensate for machines taking jobs through, eg, universal basic income or the like, how we can solve for machines replacing struggle, perseverance, and accomplishment is still very much TBD.

williamcotton• 3 years ago

Yeah but can I get one of those lawn mowing robots so I can instead feel accomplished by my guitar playing?

13years4• 3 years ago

Indeed, this is one of the major topics I focused on in the referenced article.

"AI brings into question who is really the creator. Am I the creator if I simply role the dice until I win the lottery? When I regenerate a new response over and over until finally getting the image or output I most desire? When the output is the final product, is AI still just a tool? Or are we simply observers foolishly convincing ourselves of the value of our contribution?"

2OEH8eoCRo0• 3 years ago

Privacy is gray and there has never been absolute privacy. For example, if I go somewhere in public then people see me. I should not expect my location to be private.

JohnFen13• 3 years ago

if I go somewhere in public then people see me. I should not expect my location to be private.

If that's the case, then we have almost no meaningful privacy. Fortunately, that doesn't have to be the case.

If I go out in public, people can see me, sure. But the odds of any of those people knowing who I am are miniscule. To them, I'm just another body in the bulk of bodies that populate their landscape. My privacy is retained.

It's when cameras are everywhere, recording everyone they see, when we carry devices that report our locations to others, etc., and that data is correlated with other data, that privacy is compromised.

piloto_ciego• 3 years ago

Only if we let it. But we will need to explore options other than “continue with the status quo” before that comes along.

kaesar142• 3 years ago

Stunning how scientists who work on this stuff delude themselves into thinking it'll do more good than harm.

ravenstine• 3 years ago

Many of us here on HN do the exact same thing.

AlexandrB• 3 years ago

When the work you're doing is several layers of abstraction removed from its worst applications it's easy to rationalize. For the authoritarian leader, this is a huge advantage of specialization and "replaceable cog in the machine" style of job standardization - no one person is building enough of the "evil" thing to feel responsible for the result, and most of the workers are replaceable enough that the "if I don't do it, they'll just get someone else who will" rationalization is probably correct.

f6v3• 3 years ago

So the solution is to not work on anything that can be weaponized?

throwaway71271• 3 years ago

anything can be weaponized, john wick kills people with a pencil :)

JohnFen13• 3 years ago

Everything can (and will) be weaponized, so the only realistic way to approach it is as a cost/benefit analysis. If something can bring more good than harm, excellent. If something can bring more harm than good, then maybe rethink things.

kaesar142• 3 years ago

The upside to this technology is allowing a few people to communicate who have neurological damage. The downside is every government on earth being able to read your mind. Sounds like a good trade!

drdeca• 3 years ago

To not work on anything where the risks from its potential weaponization outweigh the likely benefits of beneficial uses?

mberlove• 3 years ago

From the study: "...we tested whether successful decoding requires subject cooperation and found that subject cooperation is required both to train and to apply the decoder."

Reading the discussion in the paper, it looks like if you lie "out loud" in your head about what you're thinking, you'll fool the interpretation. For now.

Of course that can change -- will almost certainly change as the technology improves. But it's an important caveat at the moment and the researchers clearly paid attention to this aspect.

coding1232• 3 years ago

It does not require realizing you are cooperating, if you're just playing a VR game in some future version of Quest that captures brain waves while you're navigating a labyrinth.

JohnFen13• 3 years ago

But it can't read what you aren't sounding out in your mind. It can't determine truthfulness, and it can't find memories.

If you're playing a VR game, you probably aren't using your inner voice to articulate your secrets. Although if you are of great importance and interest to people with sufficient resources, they might be able to trick you into doing so in a game world.

famouswaffles• 3 years ago

There's also a ways to go with the accuracy. On the other hand though, they trained with an extremely small GPT model (GPT one i believe) so...bitter lesson incoming ?

jebarker• 3 years ago

The need for cooperation suggests that what's being learned here is the correlation between brain activity and produced language. Language is not all (maybe most?) of thinking though [1]. So it may not be a certainty that technology improvements allow decoding of subconscious thought processes. This feels related to the possible fundamental limitations of LLMs trained on language too.

[1] https://nautil.us/the-kekul-problem-236574/

niemandhier• 3 years ago

That sounds as if what actually was learned were the neural patterns associated with speaking. AFAIK its thinking out loud in you head causes activation in the motor neurons one would use for speaking.

Things like this have been tried before to help lock-in patents, unfortunately things like ALS seem to correlate with a weakening of those patterns.

JKCalhoun• 3 years ago

Anyone else's AI Bingo card have mind reading? Mine sure as hell doesn't. I think I'm done playing.

realce• 3 years ago

The WEF featured this at Davos this year: https://www.youtube.com/watch?v=4QvtT_lgl_Q

Funny - with all this mind reading, youtube won't autocomplete the query "WEF mind reading" for me.

PrimeMcFly• 3 years ago

Honestly terrifying. The potential for abuse is just to great (maybe not at the moment, but how long until the tech is perfected?).

Imagine, for example, being gay in a state that has harsh penalties for being so.

Or Theocracies that are not friendly to minority religions.

And that's just the more obvious examples.

f6v3• 3 years ago

Truth be told, it’s possible to get information out of people without any of this. Simply by torture, like in Abu Ghraib.

bugglebeetle• 3 years ago

Torture is notoriously unreliable for retrieving accurate information. Abu Ghraib was just a reflection of America’s unthinking sadism on the global stage, not comparable to a mind reading machine.

edgyquant2• 3 years ago

Way less efficient though

jstarfish2• 3 years ago

Mind reading and thought crime are equal-opportunity problems.

Godwin's Law really needs to be updated to account for the gay persecution complex. Any state that corrupt would have a cheaper and easier time just tickling you until you confess.

hospitalJail• 3 years ago

You also need to be in an MRI....

Not exactly sure what the 'its just a fad' crowd thinks about this + the Hollywood strikes.

landryraccoon• 3 years ago

This technology has tremendous potential for good. It’s not all nefarious.

People who are paralyzed or suffering from neurodegenerative diseases might have a chance of regaining independence. New treatments for mental health diseases could be a benefit as well.

passion__desire• 3 years ago

It would be nice to have throat muscle movement to text transcription for general people.

brainbag• 3 years ago

I've usually seen this referred to as subvocalization, and if anyone is working on this or has in the past, I'd love to chat.

ranprieur• 3 years ago

Wow, I thought this was a hundred years away.

So far it only works with words. I'm wondering how soon it could work with pictures, so you could imagine a willow tree and the machine shows a willow tree on a screen. A powerful daydreamer could make movies without a camera.

usaar333• 3 years ago

There's been research into this for over the past decade. Here's a 2017 report: https://www.cmu.edu/dietrich/news/news-stories/2017/june/bra...

Of course if you want to get a lot of press, use the term "GPT"

mxmlnkn• 3 years ago

From my perspective the reconstruction of lingual thoughts is the new one. There already were reconstruction of thought images several years ago and recently using diffusion models: https://www.biorxiv.org/content/10.1101/2022.11.18.517004v2

outlace• 3 years ago

It is important to keep in mind that the subjects had to get their brains scanned for 16 hours to train the machine learning algorithm to subsequently "decode their thoughts." So this cannot just be wielded on any unsuspecting person.

Moreover, I'm skeptical this can be significantly improved as fMRI is quite a blunt instrument when it comes to assessing brain activity; it is a delayed and course-grained view of relatively large volumes of aggregated brain tissue.

lisper• 3 years ago

this cannot just be wielded on any unsuspecting person

...yet.

Alifatisk• 3 years ago

The Transformer model is really this powerful?

cloudking• 3 years ago

Think about it as a transformer of bits rather than text. Any data can be transformed really.

mdp20213• 3 years ago

For "«powerful»" we could assume that the reference is to identifying the relevant patterns output by the fMRI, while the ANN transformers seem to be more related to the translation into «reconstruct[ed] continuous language from [the] cortical semantic representations»¹. I.e. they seem to be useful to give the interpretation a "form", a way to express it. What you read must be matched with something - transformers are "good at language".

¹https://www.nature.com/articles/s41593-023-01304-9.epdf

DesiLurker2• 3 years ago

"There is more to it than meets the eye."

-- transformer

jvanderbot• 3 years ago

Alarm bells aside, this was fascinating:

The decoder produced more accurate results during the tests with audio recordings, compared to the imagined speech, but it was still able to glean some basic details of unspoken thoughts from the brain activity. For instance, when a subject envisioned the sentence “went on a dirt road through a field of wheat and over a stream and by some log buildings,” the decoder produced text that said “he had to walk across a bridge to the other side and a very large building in the distance.”

It sounds like it's reading a persons snapshot of the story in a visually-descriptive language. That's really neat! I can imagine this is the beginning of telepathy.

squarefoot• 3 years ago

In a world of consumers who would salute as the next cool thing to have a device like a button that interfaces with brainwaves and connects online with devices and social media from the user's forehead, all running on closed non auditable platforms, what could possibly go wrong?

seu• 3 years ago

Why is it that in every other development, these days, one reads a quote like "researchers think this would be used for <insert thing that benefits humanity, but will remain underdeveloped because nobody wants to invest enough money in it> but they warn that it could also be used for <insert very negative possible use, in which many with the resources and power will gladly invest if given the possibility and lack of regulation or control of any type>" and yet, they continue developing it because it might "benefit humanity"? Where the hell are you going like this?!

ftxbro• 3 years ago

They continue developing it for a career where they afford healthcare and education and stave off misery for a generation or two for themselves and their family.

bheadmaster• 3 years ago

Exactly, global poverty and inflation makes it easier for money-havers to control what population at large puts effort towards. We're all slaves, just on a much bigger scale.

adverbly• 3 years ago

Interesting.

Hard not to think about potential applications when it comes to legal proceedings and court evidence.

porkbeer3• 3 years ago

Polygraphs are already inadmissable, thos tech will be the same.

JohnFen13• 3 years ago

True, but yet they're still used with certain job applications, with criminal investigations (the results just aren't used directly as evidence), etc.

Most people believe that polygraphs have some amount of validity, and that's a problem that means they continue to get used and continue to do harm.

DesiLurker2• 3 years ago

Okay but how about using this in conjunction with polygraph for some high stakes case where you'd benefit even if you can have slight confirmation of truth. then interrogator (for lack of better word) can structure questions such that you lean the conversation towards truth.

In fact now that I think about it, even 'pleading the 5th' can be worked around using this type of setup.

Nihilartikel• 3 years ago

This is probably the missing link for useful implanted brain-machine-interfaces too.

Previous decoding of neural signals was barely able to meet us half way, that is the subject also adapted to get the desired results and maybe the receiver learned a bit too.

The receiver just got a major upgrade..

soulofmischief• 3 years ago

It's useful for prosthetics but I'm concerned about a new era of harmful legal pseudoscience in vein of the lie detector.

kaba0• 3 years ago

Tangentially related, I was always intrigued whether it is possible to “record” someone’s inner monologue, when they actively move their vocal cords/folds, but not produce any sound. I think that you might get enough data to reconstruct sentences given that know which language to find words in.

porkbeer3• 3 years ago

The used this concept using throat mics a few years ago, but cant find the link.

minchov• 3 years ago

Dylan’s closing to his 1965 song, It’s alright Ma…, suddenly feels a bit too real:

“And if my thought-dreams could be seen They’d probably put my head in a guillotine But it’s alright, Ma, it’s life, and life only”

That song, btw, was somewhat prescient of today’s Intrusive Advertising Industrial Complex:

“Advertising signs they con You into thinking you’re the one That can do what’s never been done That can win what’s never been won Meantime life outside goes on All around you”

soupjacket• 3 years ago

Privacy concerns aside - I'm unclear on the specific approach used in this study and how it differs from previous work and would love some input as to whether my interpretation is correct.

From my reading, the tl;dr is that they:

- Build a subject-specific model to predict fMRI activations when presenting a subject with words

(I don't believe this is novel in and of itself. I know it has been done with ECoG - [1] off the top of my head but I am fairly certain there are others. so maybe using fMRI is the main advance here?)

- Use GPT to generate candidate sentences, and see which candidates most match the true activations.

(This method of narrowing the solution space seems new.)

The improvement on within-subject accuracy over between subject leads me to believe that this is an actual benefit, but I'm struggling to determine how they quantify the improvements over and above "GPT is good at predicting human language".

I may be misunderstanding the approach altogether however, so take this with a grain of salt.

[1] https://www.ucsf.edu/news/2021/07/420946/neuroprosthesis-res...

jstarfish2• 3 years ago

You don't need MRI machines and experimental technology to read people's thoughts. You just need access to their search (or chat) history. It's not protected by HIPAA, ethical standards or attorney/client privilege. And it's always just ambiguous enough for you to paint whatever picture you need to in order to hang them.

Out of concern for data leakage to third-parties, my employer gave employees access to an internal-only Azure instance of GPT3.5.

The next thing they did was enable logging of all queries. Security collects the data, but forwards it to the analytics group for reasons even I don't know.

Tread lightly with WorkplaceGPT if you don't want your employer to "read your mind" and make future RIF decisions based on your AI-obfuscated incompetence.

Scientists Use GPT AI to Passively Read People's Thoughts in Breakthrough

Related Stories