How to get gmail.com banned (2011)

http://mailinator.blogspot.com/2011/05/how-to-get-gmailcom-banned-not-that-i.html

by makmanalp • 11 years ago

327 60 11 years ago

mailinator.blogspot.com

zer00eyz• 11 years ago

"Thousands of people use Mailinator everyday, so clearly, its a useful tool that many sites accept"

How many of you would have an outright revolt on your hands from your QA/QE folks if you banned mailinator? I think everyplace I worked would experience this same issue if we did this.

nkassis• 11 years ago

Could use + in the first part of email such as: youremail+blahblah@example.com to create throwaways. most sites consider those to be different email address then youremail@example.com for account purposes but email service, who respect the rfc, will threat them as the same.

cdubzzz• 11 years ago

I've always found this to be such an odd option. If the concern is people spamming you or selling your email address, how does this really help? Anyone intentionally doing nefarious things can just add a simple regex to strip the garbage.

paulmd2• 11 years ago

Many sites won't accept email addresses with + in them, because many devs have extremely wrongheaded ideas about validation.

I used to have a first.m.last@university.edu address and that one was touch-and-go as well due to the fact that the mailbox had two .'s in it. I actually had to file a support request to get Amazon Student to accept it, even. Nobody from a university with that scheme ever registered before?

For the record, the gold standard for email validation is "send a confirmation link and see if they click it". Don't try and get fancy.

One other trick is that Gmail ignores .'s in addresses entirely. first.last@gmail.com is the same as firstlast or f.irstlast.

redthrowaway• 11 years ago

This is how we do it, with a predefined string after the first plus. Makes it easy to prune test accounts from the members db.

matt_kantor• 11 years ago

Could use + in the first part of email [...] email service, who respect the rfc, will threat them as the same.

I'm not aware of any RFC that says that mail sent to a+foo@example.com should go to the same mailbox as mail sent to a+bar@example.com (nor am I aware of any RFC that forbids this). I thought that GMail made up that feature and other vendors followed suit since users find it handy.

dragonwriter• 11 years ago

Could use + in the first part of email such as: youremail+blahblah@example.com to create throwaways. most sites consider those to be different email address then youremail@example.com for account purposes but email service, who respect the rfc, will threat them as the same.

There is no RFC that requires this behavior. Subaddressing within the local part is recognized as a common practice (e.g., in RFC 5233), but nothing requires a system to support subaddressing, or requires a system that does to support a particular separator character or character sequence (e.g., "+") for subaddressing. Email systems are free to implement or not implement subaddressing, and to use any character sequence they want as the separator.

brobinson• 11 years ago

This is great, but it seems like he got rid of the separate page now (the link in the article 404s) and the text is just inline again.

zinxq• 11 years ago

Wow. Do my daily HN scan for the day and find an article you wrote ~4 years at #1.

I hadn't read that in many years, and what fun to do a re-read.

Thanks Internet - don't stop being you.

dice• 11 years ago

I was sad to see that the link to the domain generator was broken. The new one on the home page is a div that's generated server-side.

I hope you don't mind that I wrote a quick one-liner to see if you're still detecting bots...

    @bobmail.info
    @zippymail.info
    @thisisnotmyrealemail.com
    @spamhereplease.com
    @safetymail.info
    @suremail.info
    @mailinator2.com
    @spamherelots.com
    @mailinator2.com
    @spamhereplease.com
    @spamherelots.com
    @spamherelots.com
    @mailinator.net
    @mailinator.net
    @mailinator2.com
    @mailinator.net
    @mailinator.net
    @mailinator2.com
    @mailinator.net
    ...

Yup :)

I didn't see any "evil" insertions, though...

nathanm412• 11 years ago

I tried something similar and got a similar pattern. After a few changes, I just started seeing mailinator and mailinator2 over and over.

33W• 11 years ago

I saw the same, but just by refreshing a bunch of times.

deckar01• 11 years ago

It looks like an exponential backoff except that it starts to favor the generic domains instead of refusing connections.

simi_2• 11 years ago

Here's a list of disposable email domains if you'd really like to block them: https://github.com/lavab/disposable

I would go one step further and look for {spam_words} in "username+{text}@{googledomain}.com", where spam_words can be "junk", "spam", etc. This is like a very narrow edge case, but still might catch something. Again, if you're into that kind of thing; I'm quite skeptical that it brings any value.

octo_t• 11 years ago

until you have a the german guy, Joseph Unker, with junker89@gmail and your validation prevents them signing up :)

pbhjpbhj• 11 years ago

Or someone from Scunthorpe (cf the "Scunthorpe Problem").

aidenn02• 11 years ago

Or Dick van Dyke.

simi_2• 11 years ago

That's not at all what I meant. Gmail redirects emails to "youremail+whateveryouwanthere@gmail.com" to "youremail@gmail.com". Some people filter emails this way, henceforth my suggestion to check for this edge case.

http://gmailblog.blogspot.de/2008/03/2-hidden-ways-to-get-mo...

PS: The downvoters are downright silly on this website lately.

scoj• 11 years ago

That was a fun read.

It took me a bit to get my head around the use cases. It's sometimes amazing how many different ways you can twist a simple (complex really) thing like email into a product/idea.

w8rbt• 11 years ago

Great story! Before deciding to use blacklists or lockouts (on anything), know that it can and will be used against you.

serve_yay• 11 years ago

So much fun to read. Great post, thank you.

CWuestefeld• 11 years ago

A few years back we came into work one morning to find that some bot was scanning our site so hard that it seemed the lights nearly dimmed. Some detective work suggests that it was a service performed on behalf of a competitor, to get our price list (bear in mind that our catalog has a few hundred thousand products).

We were really annoyed that rather than just ask us, they had launched what amounted to a DDOS attack. So we thought about how we might exact vengeance...

After a few hours we figured out a pattern to the rogue requests that allowed us to filter them, despite their efforts at stealth (like, they cycle through a list of various user agent strings to make it look like there are multiple different users). We toyed with the idea of, rather than outright banning them, making our pages sensitive to their presence, so that when we detected them, we'd display a false price, defeating their whole operation.

We finally just decided to take the high road, temporarily banning any rogue IP addresses we detected (we couldn't make it permanent because many of the requests came from the Amazon cloud, from which we also receive some legitimate requests)

EDIT: you wouldn't think that requests for a few hundred thousand products would amount to a DDOS, but the bot was rather poorly written and grossly inefficient in the way it walked through the list.

adrianpike• 11 years ago

I built a system called caltrops that did almost exactly that. As a given session's requests grew more and more suspicious, their data would skew from reality further and further. A real user on the line would notice immediately (and the more real-looking the user interactions, the more it would reduce suspicion), but competitors scraping our data would get pretty deliciously bunk data.

motocycle• 11 years ago

to deal with similar problems, kickstarter built a pretty useful tool called rack-attack https://github.com/kickstarter/rack-attack

mangeletti• 11 years ago

This is a most excellent idea!

Btw, did you actually return incorrect price data, or did you just insert random bytes, etc.?

tegansnyder• 11 years ago

I'm not sure his method would prevent a headless scraper like CasperJS or PhantomJS from doing the dirty work, but nice technique nonetheless.

geofft2• 11 years ago

At least at the time of writing, if you had enough foresight and engineering time to set something like that up, you had enough foresight and engineering time to not make your system treat email addresses as meaningful identities.

nirvdrum2• 11 years ago

Perhaps I'm missing something, but an extremely high percentage of the sites I have accounts on use my email address for authentication. Those that don't often suffer from username squatting. Maybe most sites are just doing it wrong, but what's the prevailing alternative?

jessaustin2• 11 years ago

This reminds me of the sites that discouraged hotlinking by examining Referer and then sending Goatse.

kpcyrd2• 11 years ago

http://ascii.textfiles.com/archives/1011

jessaustin2• 11 years ago

I'm glad I was brave enough to click. b^)

8ig8• 11 years ago

One way to get around a domain blacklist is to point your own domain to Mailinator. Heck, since last year you can even get your own private Mailinator...

http://mailinator.blogspot.com/2014/10/mailinator-launches-p...

codexon• 11 years ago

Mailinator seems to have added some other anti-scraping detection.

Unfortunately it does not work very well as I was not scraping mailinator, but still somehow got IP banned. Fortunately my ip has changed. But they definitely have some strange and overzealous method now.

fapjacks• 11 years ago

I've used Mailinator for years and it's always interesting to read what this dude has to say.

thetruthseeker1• 11 years ago

I love mailinator!

However tricking site scrappers may not work perfectly if the site scrappers maintained a list of websites in their "whitelist". Say if I am scrapping mailinator.com for domain names, if I see gmail.com or yahoo.com, I might just not put them in my database because they are in my whitelist.

belovedeagle2• 11 years ago

Most of the comments here about '+' parts are rendered completely irrelevant by single-user domains. Not to mention "one email ~= one person" schemes.

botbot• 11 years ago

Why not encode the domain strings into an image?

OCR requires a lot more programming effort compared to a text-based content scraper

eru• 11 years ago

It's unfair to blind people.

tempestn• 11 years ago

FTA: "Could I make it harder to scrape? Well, I could, but wouldn't really slow anyone down much."

I think that's the basic idea. He could spend his time making it harder to scrape, like the bar across the steering wheel. Some people would be deterred, others wouldn't, and time would be wasted all around.

How to get gmail.com banned (2011)

Related Stories