Bypassing Gmail's spam filters with ChatGPT

https://neelc.org/posts/chatgpt-gmail-spam/

by neelc • 3 years ago

207 120 3 years ago

jeffbee2• 3 years ago

People are overestimating the importance of the message body in spam classification. The stuff that appears in your spam label on gmail is what google considered marginal, almost ham. The vast, vast majority of what they think is spam is rejected with temporary failure codes at SMTP time and never gets delivered with any label. IP reputation and other related metadata features are the key features in spam classification, and repeatedly sending different messages is not a valid test of whether the body looks spammy or not.

topicseed• 3 years ago

and other related metadata features

Such as? Actually curious and doing a lot of sales myself, I'm interested!

jeffbee2• 3 years ago

Every email operator considers their classification features to be trade secrets. The closest you will get to advice from Google on this topic is https://support.google.com/mail/answer/81126?hl=en

But anyway it sounds like you intend to send spam. I recommend doing literally anything else.

DwnVoteHoneyPot• 3 years ago

Yeah, I’m starting to hear and understand that more myself. There was a extremely long twitter thread (i think from former Reddit CEO) that said the key to content moderation is moderating bad behaviour, not bad content.

walrus013• 3 years ago

I concur on this for the most part because I would say that my custom postfix + spamassassin + opendkim setup, on my self run MX, correctly classifies 75%+ of the spam or outright rejects it for SMTP transfer just based on:

a) invalid rdns of other mx

b) invalid spf

c) invalid DKIM / no DKIM signature

d) failed RBL list check - I subscribe to and feed it a few different common sense SMTP RBLs

Rejecting as spam things in the above category before it even looks at the content.

Adding a high score for invalid rdns, spf or dkim before something generally similar to spamassassin or a more advanced message subject line/body analyzing system begins classifying things help.

And then additional score is added of course for text spam content in message subject line and body.

BenjiWiebe2• 3 years ago

Which RBLs do you use/suggest? My main anti spam solution is currently just Thunderbird's adaptive spam filter, but that doesn't help when I access it with a different app or via web.

astrange4• 3 years ago

One signal for spam is just the same email going to many different people who don't like receiving it, but "the same email" doesn't have to mean it has exactly the same text.

mfi• 3 years ago

I’m no AI or machine learning expert so I don’t know how it works. But I am also worried that spammers could use ChatGPT to get around Gmail and Outlook’s spam filters.

This will not only increase the spam-problem, but will most likely be used to scale and do targeted phishing attack as well. I wrote an extensive article[0] where I analyzed this. And to no surprise, GPT-3 can be used to generate dynamic phishing campaigns on the fly in multiple languages, classify email responses, improve email thread hijacking attacks etc.

[0] https://www.xorlab.com/en/blog/why-ai-powered-phishing-will-...

shadowgovt• 3 years ago

On the other hand, ChatGPT becomes a very convenient tool for generating example data to train the ML component of the spam filter on.

aitball• 3 years ago

something.. something... arms race

HyperSane• 3 years ago

Maybe the Amish were right after all.

amelius• 3 years ago

There is a nonzero probability that a new technology will destroy the world. So at some point, the Amish will be right.

thyrox• 3 years ago

All this can be done since GPT3 API has been available.

I see a lot of people thinking chatgpt is something new capable of such stuff but GPT3 is far less restrictive and has been able to do all this for almost an year now.

puffybuf• 3 years ago

But you must pay to use these APIs. They did give me $20 free trial though. You could make a bunch of accounts and abuse the free trial I guess. It must be cost effective for the scammers.

You can change up the prompt to change the writing style so spam filters will have trouble catching this new world of spam.

dhruval• 3 years ago

ChatGPT also throttles the number of requests an hour and has various measures to prevent bots (though not that hard to bypass, easier to pay for API)

jokoon• 3 years ago

only way to make email work is to let the user choose what domain they trust, and maybe to have a feature where the user must whitelist addresses.

Avamander• 3 years ago

In addition to further emphasis on "trusted senders" in the form of contact books, we'll also hopefully see a rise in identity-validated S/MIME. Though I get the feeling it'll hurt really bad before either gets deployed to a sufficient extent.

fIREpOK• 3 years ago

Then everyone would choose "gmail.com" and Google will get its true monopoly?

killingtime74• 3 years ago

Give google half a chance and they will make "Gmail" domain only opt-out, won't even ask you to choose

deno• 3 years ago

I can’t find it but Gmail actually had some sort of whitepaper or something once about Gmail-originated SPAM and how it became a huge problem which was partially the reason they started doing phone verification.

The parent’s comment is valid. Any modern email peer is doing domain based reputation which is possible thanks to SPF and DKIM, and if you don’t have those configured you’ll have a bad time. Then it’s the job of the domain owner or email operator (postmaster) to make sure you’re not blasting out SPAM and respond to abuse feedback. If you think about it, this is the only sane way for email to function without preauthentication.

The only major outlier to this is Outlook, which is still doing IP based reputation. And of course a long tail of small server operators that rely on legacy SPAM lists from decade ago and reject only legitimate emails and pass through plenty of Viagra ads.

marginalia_nu• 3 years ago

Anecdotally, almost every email I see in my gmail inbox is advertisement of some sort. There's newsletters I never signed up for, special offers from companies I've never had dealings with, it never ends.

Some of it hasn't even been sent to me as an email, but shows up in the inbox as though it was an email.

Granted, there's fewer scam emails than in my non-gmail inbox, but man is there a lot of spam.

4g• 3 years ago

Using these tools to mitigate spam is as likely a scenario as this, imagine that every spam mail receives a masterfully crafted response showing utter fascination and interest in SEO, or helping out a Nigerian prince. Every phone call to an unregistered number is answered by an artificial, frail, and forgetful lady that is trying her best to register gift cards.

When reporting an e-mail as spam it will not only block the address but waste the spammers time, rendering the actions unprofitable.

wolverine8762• 3 years ago

So the two AIs will be talking to each other, trying to suss out if the other is fake (a sort of Turing test), trying to con the other to keep talking or to really buy in?

beachy• 3 years ago

Sounds absurd all right. But what's to prevent this from being the future of the internet?

codetrotter• 3 years ago

Reminds me of Adventure Time Season 4, episode 10: Goliad

https://adventuretime.fandom.com/wiki/Goliad_(episode)

(Spoilers follow)

Princess Bubblegum is concerned over her mortality, and reveals her designated successor to Finn and Jake. However, things get out of control when the heir turns against Princess Bubblegum.
[…] The heir is Goliad, a pink Sphinx with a mound on her forehead […]
[…] Goliad reveals that her mound concealed a third eye, and proceeds to psychically control Finn and the obstacles in the course for a perfect completion. She explains that with her in control, everything would be perfect. […]
[…]
[…] Finn and Jake confront Goliad but are beaten by her psychic powers. Goliad tries to read Finn's mind, and Finn narrowly avoids revealing the plan by interrupting his memories with nonsense. The new creature; another sphinx with an eagle's head, white feathers, and golden hair; rescues Finn and battles Goliad. Goliad tries to convince her brother, named Stormo, that they should work together, but Stormo refuses by screeching. They engage in a psychic showdown, but with their powers matched the two creatures are eternally locked in a mental stalemate.

gernb• 3 years ago

How about 2 AI developing a language to be able to speak to each other

https://www.imdb.com/title/tt0064177/

Waterluvian3• 3 years ago

Oh my god a future where you are never ever quite sure if your online circle of friends are human or not.

jimkleiber• 3 years ago

I wonder if that inspires new platforms or types of tech that verify someone was typing it in vs pasting. But then do bots get better at typing it into the input boxes? Ugh.

InCityDreams• 3 years ago

And you're confident right now?

kibwen• 3 years ago

At the same time, it's not necessarily pleasant to consider the prospect of an internet where 99.9% of traffic is generated by AI-powered spambots engaged in adversarial games with AI-powered anti-spambots.

sharkweek• 3 years ago

But it does make for an interesting idea as a plot device in speculative fiction!

ThrowawayTestr• 3 years ago

Honestly the most likely source of computers becoming self-aware.

honkler• 3 years ago

dead internet theory: https://dailyfreepress.com/2021/10/21/the-dead-internet-theo...

atorodius• 3 years ago

I thought we are already nearly there? I remember reading 90%+ of emails are spam and this was a while back

cortesoft• 3 years ago

Why is that bad? As long as it is invisible to humans and doesn't cause bottlenecks, who cares?

shagie4• 3 years ago

If you haven't read Accelerando, I'll suggest it - https://www.antipope.org/charlie/blog-static/fiction/acceler...

yieldcrv2• 3 years ago

Cyberpunk 2077 features a world after this has happened. Not because it happened, its just after. If there is a reference to an older internet it features its disuse due to this.

nbardy• 3 years ago

I think the opportunity cost of better employee will make it hard for that to scale.

If you’re that good at scaling systems like that someone with legit traffic would be able to pay more than the spam companies.

Tsiklon• 3 years ago

Part of this is what I imagine is behind the “Blackwall” in the Cyberpunk universe.

epicureanideal• 3 years ago

There are still solutions in that case. Email addresses only allocated to verified humans, which only accept email from the same addresses or specifically allowed outside email addresses.

Alternatively, email addresses that are necessarily paid for that have maximum sending limits or extra costs above those limits, so those addresses are not profitable for spam bots to use.

I could imagine some kind of “super priority email” that costs a dollar or something, that gets prioritized and is very unlikely to be marked spam.

More ideas: Maybe the amount paid per email sent changes depending on the percent of previous emails marked “accept emails from this sender”. Maybe some percentage of the e-mail fee is given to the recipient of the email.

Note that if a highly spam resistant email account is adopted by everyone the amount of spam sent may fall significantly. Gmail doesn’t count because it still isn’t as good as a system where sending millions of spam emails is simply too expensive.

sangnoir• 3 years ago

Every day we get closer to the Slug from Accelerando (by @cstross): a self-owned sentient corporation/419 scam

5e92cb50239222b• 3 years ago

If as a (hypothetical) Nigerian prince spammer I get a "masterfully crafted response" from a mark, it would be obvious right away that I am talking to a bot. The kind of people who respond to such bait would hardly be able to write anything like that.

Kamq• 3 years ago

A "masterfully crafted" response to a nigerian prince spammer would probably sound a lot like a person who can barely write, possibly with what sounds like the beginning of dementia setting in.

jahewson• 3 years ago

Fun idea but impersonating someone else, especially your customer, sounds like a way to land in hot water. Also LLMs are not exactly cheap.

mimimi31• 3 years ago

imagine that every spam mail receives a masterfully crafted response showing utter fascination and interest in SEO

That would necessitate reliably detecting the emails as spam in the first place though. False positives in particular could be devastating. Imagine a chat bot coming up or going along with business proposals in your name for example.

crummy3• 3 years ago

You could do it with humans - every time you click "mark as spam" it doesn't just trash the email, it begins a long and drawn-out chatGPT conversation with the spammer, stringing them along.

walrus013• 3 years ago

Sure, turn loose these tools to answer the actual spammers/UCE. But:

Speaking as an ISP, if somebody turns loose what is clearly an AUTOMATED tool shitting up the contents of my abuse@ispname.com inbox with reports from some software script, I can guarantee you it goes to /dev/null

At some point we will just block their MX at the SMTP transfer point and call it a day.

98% of that already is abusive DMCA rights holders who are ignoring our federally designated DMCA-agent address for copyright violation complaints. With their automated 3rd party things complaining about people torrenting Yellowstone or whatever.

Actual reports that are clearly written by a human saying "hey it looks like this /32 of an IP address is compromised as some sort of botnet" will get a thousand times more attention. Or the very rare cases where we have a network-engineering emergency escalation and somebody calls me on the phone.

Anything generated by chatGPT or similar will be clearly obvious enough that it matches a similar pattern and comes from an automated script.

awb• 3 years ago

James Vitech did this, but manually and with humorous results:

https://youtu.be/4o5hSxvN_-s

https://youtu.be/IUjpoauJcKo

miketery• 3 years ago

I think for next few years that will be cost prohibitive for 95% of Americans.

rglullis• 3 years ago

Already a reality for phone spammers/scammers: https://jollyrogertelephone.com/

just_boost_it• 3 years ago

There's lots of legitimate email traffic that would find itself stuck in here. I could see business questions being answered and those answers actioned on. Or legitimate sales prospecting resulting in actual orders being placed. If you choose to let a tool do your communication for you by impersonating you to the extent that another person would reasonably expect that they're talking to you, then I'm not sure you can just say "lol, that just was my spam bot" as a way of getting out of it.

themitigating2• 3 years ago

It wastes my time and spammers will create new addresses automatically

EGreg• 3 years ago

Sure, but most spam will go the other way. When you're in a forum and you express an opinion that doesn't fit what the bot swarm is designed to police, they will gang up on you and argue against you from various angles, and you'll think your opinion is just super unpopular. If you are a famous person they'll do reputational attacks, after amassing karma points and followers across the swarm. Then they'll move on to combating the remaining human-powered outlets like nytimes.com, becoming the next Vox or Vice, but totally AI-powered. Their bots will spread their articles over the other ones. And finally, having driven humans underground in the dark forest of social media, they'll basically dominate the Internet.

The real sea change will come, though, when bot swarms control capital. For example, amassed for doing spam tasks. That capital can be deployed in a variety of ways, but the point is that corporations and networks will prefer bots because they generate more social capital, in coordinated ways, and also amass more financial capital than humans.

If you think this is far fetched, IT ALREADY HAPPENED on wall street and hedge funds. Bots have replaced people in trading and control the capital. Humans who trade among the bots get fleeced and don't even know it.

astrange4• 3 years ago

Letting bots make all the trading decisions is actually a good way to die due to adverse selection.

https://fortune.com/2022/06/02/zillow-6-billion-home-flippin...

Humans who trade among the bots get fleeced and don't even know it.

Anyone who trades gets fleeced unless they know what price impact is. Don't trade, invest.

ilyt2• 3 years ago

Using these tools to mitigate spam is as likely a scenario as this, imagine that every spam mail receives a masterfully crafted response showing utter fascination and interest in SEO, or helping out a Nigerian prince. Every phone call to an unregistered number is answered by an artificial, frail, and forgetful lady that is trying her best to register gift cards.

Right. And who the fuck would pay for GPU time for that ?

arcticbull2• 3 years ago

Apparently phone scammers hit Americans for $40B in 2022. At this point the economic incentives are significant enough that a federal budget line item may be in order.

[1] https://www.cnbc.com/2022/11/05/how-phone-scammers-tricked-a...

arcticbull2• 3 years ago

Every phone call to an unregistered number is answered by an artificial, frail, and forgetful lady that is trying her best to register gift cards.

Kitboga has actually been playing with this. [1] Apparently his bot was successful enough to get some bank accounts from scammers that he reported.

[1] https://www.youtube.com/watch?v=maP2DwgdBts

sipos• 3 years ago

A lot of spam is just a link to a page for someone to download malware or enter card details. It is already relatively easy to get these taken down if you care enough to, but a waste of time as the senders have moved on by then. The idea that spammers cannot make something that people tricked by their scam will use, but that insulates them from time wasters, is ridiculous.

wolverine8762• 3 years ago

Think what can be done with an AI trained on all the data that's been collected about you.

b1n• 3 years ago

Spear-fishing now scales.

astrange4• 3 years ago

Not all that much, considering how when you buy a blender on Amazon it recommends a second blender.

thaumasiotes• 3 years ago

I assume most blender purchases on Amazon are for "will it blend?" YouTube channels. ;D

Tagbert2• 3 years ago

AFAIK all it takes to bypass Gmails spam filters is to resend the same scammy email that was Flagged but using a different email address. I get the same kind of scammy emails, flag them as spam, and then a couple days later, Gmail lets the same email in though it is coming from a different 123134r12345124@blahblah.com address.

orlp• 3 years ago

In my experience the new strategy is to just send an email with no body, no title, optionally with an image of a conventionally attractive lady, in the hope people respond after which they're automatically whitelisted (as there's now a 'conversation').

Tagbert2• 3 years ago

I get a lot of those. Never download the image so I don’t know if there is an attractive lady or not. I do always mark them as phishing spam for gmail but it just keeps delivering more. I lost count of the free Yeti mugs I’ve missed out on.

ceejayoz• 3 years ago

There also seems to be a *@salesforce.com exception built in.

steveBK123• 3 years ago

Interpretations I've seen of ChatGPT type tools, and playing around with it, I would sum up as "reducing the marginal cost of creating BS to $0". Great for content farms, spam, disinfo/propaganda campaigns.

Stuff where it doesn't have to be correct, have a high hit rate, or even be edited. Just need to produce plausible enough sounding, human-like content.

natch2• 3 years ago

I don’t think ChatGPT is really needed here.

Gmail’s spam filters are Google’s weakest tech. At least I don’t know of anything worse.

teruakohatu• 3 years ago

Gmail’s spam filters are Google’s weakest tech. At least I don’t know of anything worse.

I get a fair amount obvious spam coming through the filter, but the issue with any sort of classification is the tradeoff between False Positives and False Negatives.

The occasional False Negative causes a lot less damage (2 seconds to delete or report as spam) verses the damage of a False Positive (not seeing an important email for two weeks or ever).

natch2• 3 years ago

I get a ton of false positives. The tech is truly lame.

narrator• 3 years ago

As I have said before, the future will have two kinds of AI everywhere.

Their AI to get you to buy something, do something, believe something, or in a warzone to kill you and Your AI to protect you from Their AI. Reality may even become so dangerous and illusory that humans lose a lot of their agency to Your AI.

sorry_outta_gas• 3 years ago

This is about all its' good for, it won't even write sexy fan fiction for me

astrange4• 3 years ago

OpenAI Playground will do that. Whether it's good quality I can't say. The RLHF models are too boring and the older ones are very hard to control.

Crystalin• 3 years ago

well, you don't need ChatGPT for that. I receive daily scam in my gmail with 99% the same content each time (and i mark it spam each time)... the worst part ? the server serving it is google (but the email attached to it is always a different obscure email)

londons_explore2• 3 years ago

I don't think message content is weighted very heavily in modern spam filters...

Things like IP reputation, sender reputation, and various SPF-like headers are far more important.

tnzk• 3 years ago

It makes sense in itself, but then why do you think the second email went through in this case, considering no factors other than content didn't change?

londons_explore2• 3 years ago

well this is the 2nd email from the same server, and the first was viewed by the user for 20+ seconds and not deleted, so probably not spam...

sirsinsalot• 3 years ago

If we thought the signal vs noise problem was bad, wait until 90% of data is banal AI drivel drowning out any semblance of authenticity.

Boring, brown, homogeneous noise

teaearlgraycold• 3 years ago

We’ll be forced back to the outside world

cosmojg• 3 years ago

Is it finally time for Hashcash[1]/PennyPost[2]?

[1] https://en.wikipedia.org/wiki/Hashcash

[2] https://pennypost.sourceforge.net/PennyPost

graypegg• 3 years ago

I think the future model of email will be spam-by-default, where you whitelist senders as you need them.

Already doing that with hey email [0] where you screen all incoming senders.

[0] https://hey.com

okasaki• 3 years ago

If you want to spam gmail you can just share documents with people or add events to their calendar. Apparently there's no way to stop this.

_glass• 3 years ago

On the other hand it's easy to detect ChatGPT, at least for now https://huggingface.co/openai-detector

sprremix• 3 years ago

which detects the text from the article as 99% real...

mk_stjames• 3 years ago

I must be in the extreme minority upon reading comments on email spam, as I almost never receive a spam message in my main gmail inbox that is truly spam and not something I'm getting an email for because I lazily opted-in to a mailing list or was put on one due to association with a product. Actual, oldschool spam (Nigerian princes, claim your prize etc) literally never gets to my inbox. Maybe I'm just not a popular person. Or maybe I've kept my email private enough that it doesn't get scraped for such things? I don't understand the discrepancy between my experience and so many others here.

butz• 3 years ago

Whoever is doing this, please stop. Thank you.

low_tech_love• 3 years ago

Considering how much crap I get in my Gmail inbox, doesn’t seem like it’s a hard task. Seriously, my Gmail account is borderline useless nowadays.

Bypassing Gmail's spam filters with ChatGPT

Related Stories