Complexities of e-mail validation logic

https://www.netmeister.org/blog/email.html

by Tomte • 5 years ago

287 385 5 years ago

arkitaip3• 5 years ago

Most of these are just overcomplicating validation. What really matters is account verification, i.e. sending an email to the specified email address in order to verify its authenticity before sending any kind of email (transactional, marketing) to the account.

At this point, not doing email verification should be considered a dark pattern because it causes so much trouble when people's email addresses are used without their permission.

delecti• 5 years ago

And "permission" isn't even the only issue. Months of Doordash account emails were lost to the ether because I made a typo (gmail.lcom) in my personal email, and it was basically impossible to change the email on an account (their SMS verification seems broken). It does explain why I never got order confirmations though, that had seemed odd.

jerf• 5 years ago

It seems to be a common antipattern for somewhat smaller sites to make the email address the primary key on the account too, and then it's virtually impossible to ever change it after that. As you scale up it becomes impossible to ignore the fact that people change email addresses sometimes, but I've lost track of the number of smaller sites that assume it's a safe primary key.

anttisalmela2• 5 years ago

It doesn't really hurt if some more exotic email addresses are not accepted, no one can really use them anyway.

asddubs• 5 years ago

in fact it probably catches more mistakes than deliberate weird email addresses to be stricter than the standard mandates

jiofih2• 5 years ago

Well, you can’t use them because of attitude like yours. You’re not only annoying those of us who want to use plus signs or international domains, but ensuring the “exotic” half of the world that doesn’t use the Latin alphabet is kinda left out.

anttisalmela2• 5 years ago

Non-ascii alphabets need a lot more support than just accepting them. But I wouldn't really consider plus signs exotic anyway.

abrowne• 5 years ago

If you are consistent. A couple times I've successfully signed up for something with a username+string@gmail.com address but then have been unable to unsubscribe because my address is "invalid".

NullPrefix• 5 years ago

I kinda recall a lawsuit many years ago about a unsubscribe confirmation email.

buro92• 5 years ago

I have a tld that was recently created (2014) and I still cannot use it in an email address reliably.

The domain in question being david.kitchen, so an email may be email@david.kitchen

The issue I encounter more than any other is trivial: Most sites still have a tld validation that only accepts domains that end in net|com|org and some other small list of accepted suffixes such as co.uk

The list of TLDs is constantly expanding https://newgtlds.icann.org/en/program-status/sunrise-claims-... so even `[a-z0-9.-]+@[a-z0-9.-]+\.[a-z0-9]+` would be better than what I see in the wild.

vidarh2• 5 years ago

I was involved in setting up .name back in 2001. We spent ages contacting people with validation rules based on the old set of TLDs. Given that was the first expansion of the gTLD space in a long time, it wasn't so unreasonable then. But it's just astounding that it's still and issue 20 years later.

duped• 5 years ago

sometimes you get regressions too. Kaiser Permanente invalidated my email address earlier this year.

znpy• 5 years ago

and the regex you provide doesn't even account for unicode..

buro92• 5 years ago

The regex came from the article, and lack of unicode is addressed in the article.

tmk11082• 5 years ago

I managed to buy firstname.dev a while ago and this was one of my fears of using it as my email address. I ended up switching to a .com one just to avoid any issues. I certainly don't want government services emails not to work just because maybe they didn't account for .dev TLD

toyg• 5 years ago

yeah, I routinely use .email and .cloud and it's so annoying when the occasional site goes "THAT IS NOT AN EMAIL ADDRESS !!111!1 YOU HAXXXOR!".

sparrc2• 5 years ago

I have the same issue. I use sparr.email and it fails validation on a few critical websites, namely my online utilities account (seattle public utilities) and payroll processor (ADP).

schwinn140• 5 years ago

Link is just hanging.

Also, anyone notice the OP posts the same couple of posts constantly?

u801e• 5 years ago

I provide my email address with the +companyname suffix on the local part as a way to filter my email into various folders based on the To header contents.

Unfortunately, many websites are configured to reject email addresses that contain a plus character. I've also encountered websites in the past that did accept the + character when creating the account where the email address serves as the user name, but then could not log in because their log in form rejected the + character in the user name.

vidarh2• 5 years ago

If you use Gmail here's a fallback option: Gmail ignores "." in the local part. So foo.bar is the same as f.ooba.r to Gmail. Obviously quite limited and more hassle to keep track of.

caymanjim2• 5 years ago

One of my primary pet peeves with Gmail. It leads to a lot more junk mail arriving in my inbox. My real Gmail address is 'first.m.last', and almost all the spam I get is addressed to 'firstmlast'. Gmail is great at filtering out spam so that I don't see most of it, but if not for their unconventional filtering of recipients, I'd get even less. I also get a lot of email from idiots who don't know their own address and provide mine instead, and literally all of that would bounce without their . handling.

grey-area• 5 years ago

This pattern is often abused by spambots trying to avoid dupe detection, so using it excessively may lead to your login being treated as spam.

moojd• 5 years ago

I was unable to provide my email address for a retail rewards program last week because the input field for the domain was a dropdown in their POS. Not the TLD, the entire part of the email after '@'!

StavrosK2• 5 years ago

Jeez, wow. How many domains were in that box?

jacobkg• 5 years ago

Yes this is terrible. On the other hand, if your goal is to prevent people from signing up using disposable domains, the blacklist approach (which I have tried before) is a never ending game of whack a mole.

Sounds like this was in person at store though which is extra weird because seems unlikely that scammers would be trying to sign up en masse at a physical location (unlike if the form is connected to the internet)

skhr0680• 5 years ago

Until about a decade ago, this was extremely common in Japan. RIP mobile email, another victim of smartphones in general and the iPhone in particular

theshrike79• 5 years ago

Fastmail allows for companyname@youraccount.fastmail.com -style addresses. Even for your own domains.

Much more reliable than the + -thing, which breaks in the weirdest of places.

aidenn0• 5 years ago

I've been using fastmail for years and didn't know that. Thanks!

theandrewbailey• 5 years ago

I use Fastmail with my own domain name and unlimited email inboxes, so I use companyname@mydomain.com to sort incoming mail.

jetpackjoe• 5 years ago

I do the same thing and believe it or not I’ve seen websites reject emails with their own name in the email.

fullstop2• 5 years ago

Ages ago, back in myspace days, their system would permit + when creating an account, but could not handle this in their forgot password / password reset system. I never was able to delete my account because of this.

inopinatus• 5 years ago

Sony's SEN used to have an account creation page that would permit +, but subsequent sign-in interpreted it as a URL-encoded whitespace. No login for you

torstenvl• 5 years ago

All social media accounts are delectable on a long enough timescale.

innocenat• 5 years ago

I find that a lot of website don't allow + sign precisely because of Gmail usage.

ThalesX• 5 years ago

I used this wonderful trick to sign up for my government issued eID (it was something else but works for explaining). What they decided to do is to simply remove the + and don’t let me know about it.

my_email+service@foo.bar thus became my_emailservice@foo.bar

I tried logging in, resetting passwords, nothing worked. I had to go to the authorities and make a written request to allow them to interrogate the database by the equivalent of my social security number, and that’s when we realized they just stripped the +.

SAI_Peregrinus3• 5 years ago

I got sick of companies rejecting email with "+", and bought a domain to use for email (among other reasons). Now I've got a wildcard entry in DNS, so any valid local part gets routed to my inbox. So instead of "username+company@example.com" I can do "company@example.com".

richardwhiuk• 5 years ago

That causes weird behaviour in places, where they assume the bit before the @ is a "username".

MivLives• 5 years ago

What provider do you use for email? That does sound nice.

ElFitz2• 5 years ago

I ended up giving up on that after one too many websites rejecting my custom domain (which I’m the only one using) on signup. These lazy / ignorant colleagues are annoying -_-‘

axaxs• 5 years ago

Can you explain the DNS part? AFAIK the sender just looks for MX on the domain itself, regardless of local part.

ryandrake• 5 years ago

I just set up my mail server to use - rather than +, and don’t encounter this problem.

caymanjim2• 5 years ago

I got sick of + not being accepted and switched to using - for all my aliases, which works everywhere I've tried. It's annoying, but practical (assuming you run your own mail server, or have the ability to manage it client-side).

8ytecoder3• 5 years ago

Plenty of hosted solutions support wildcard - including GSuite and Fastmail.

jbgreer• 5 years ago

Ditto, with the same hassles mentioned by you and others, such that I'm actively looking at email services that handle this sort of thing better using approaches such as mentioned below - domain@mydomain style registration addresses.

agustif• 5 years ago

You can have unlimited handles with fastmail if you're looking for that

rpadovani• 5 years ago

I use a catch-all to have a <website>@<mydomain>.com login for every website.

Samsung doesn't accept emails with "samsung" as prefix, so I have samsun@mydomain.com for them. I have no idea what's the logic behind.

kccqzy• 5 years ago

I use reverse DNS notation for the local part. So that would be "com.samsung@mydomain.com" in my case.

burke• 5 years ago

I’ve never seen much point in trying to do better than .+@.+, unless you’re going to pull out the (gargantuan!) authoritative version for some reason.

kevinmchugh• 5 years ago

Implementing the authoritative version is a waste since you'll also need to keep an up-to-date list of TLDs, and more importantly, you might have a typo in the input that gives a valid-but-incorrect email.

After doing your simple regex, the best move is to just send a verification email and wait for the user to click the link, if you really need to be sure.

wffurr• 5 years ago

Yeah, do simple validation, and then just send an email. Even a validated email can still be non-deliverable if there’sa typo in the domain or the first portion.

dkersten• 5 years ago

Or the user typos their address and it goes to the void or to someone else. Even a valid address can’t be assumed correct. I get a ton of emails to my old gmail that aren’t meant for me because some people are too dumb to get their email addresses correctly (even for someone’s covid vaccination appointment confirmation and details recently...) or just make a mistake.

michaelt• 5 years ago

Too many sites refuse to let me register as

  "><script>alert("XSS");</script>@example.com

The oppression must end!

SAI_Peregrinus3• 5 years ago

I think the mismatched quotes actually do make that one invalid.

mro_name4• 5 years ago

shouldn't is read Bobby Tables rather than a mere XSS?

Avamander6• 5 years ago

Honestly if that'd ever work, the website has bigger problems anyways.

kristaps• 5 years ago

Yep, these arcane rules are maybe relevant to the 5 or so people writing mailservers, but not to web developers.

user3939382• 5 years ago

TFA won’t load for me, but I’d like to make a short PSA: RFC 5322.

Lookin’ at you, Walgreens.

Wronnay• 5 years ago

https://web.archive.org/web/20210408080002/https://www.netme...

ericcholis• 5 years ago

In addition to a very simple regex, you can do some light verification on DNS and SMTP

- nslookup –type=mx email.com

- pick the highest priority MX server

- telnet mx1.email.com 25

- validate SMTP handshake

- Start a connection: EHLO email.com

- mail from:<sender@youremail.com>

- rcpt to:<recipient@email.com>

Obviously, this might be outside the capabilities of some hosts or users. There's a bunch of services that expose this workflow for you as an api. (https://trumail.io/, for example)

teh_klev4• 5 years ago

See point 10 in the article:

"The domain name does not need to resolve"

Also the mail server may be temporarily offline or unreachable.

jusssi• 5 years ago

Non-resolving or offline mail servers count to your bounce rate if you have a 3rd party service handling your outgoing mail. So for that purpose, it is an invalid address in the sense that you should avoid sending anything to it.

icedchai2• 5 years ago

Ok, so the article is wrong. For an email to be valid right now, yes, the domain part has to resolve. If you're accepting any email address that might be an email in the future, then they are correct, but for 99.9% of use cases: yes, the domain has to resolve.

jeffbee• 5 years ago

Close but you also need to fallback to AAAA or A lookup of the domain when the MX record doesn't exist. Also, do you really want transient unavailability to stop your signup flow? The whole point of the way mailers are written is the mail gets delivered even in the face of transient unavailability.

cphoover• 5 years ago

If your validation function works for 99.99% of your user's email addresses and it's a big unnecessary lift to get that other .01% your logic is not wrong.

tgv• 5 years ago

I don't think I've seen a bang path since 1990. The claim "Your E-Mail Validation Logic is Wrong" is just pedantry.

strken• 5 years ago

What you've seen since the 80s ended is unfortunately only a subset of all the horrible edge cases your users will run into.

icedchai2• 5 years ago

A little later for me. I last used bang paths in 1994, when I had a UUCP feed.

LinAGKar• 5 years ago

Just get rid of that pointless filtering altogether.

chrismorgan5• 5 years ago

I think the best syntax validation technique for email addresses now is found in the HTML spec: https://html.spec.whatwg.org/multipage/input.html#valid-e-ma.... As they say, this is a wilful violation of RFC 5322, because that’s simultaneously too strict, too vague and too lax to be useful. They give a grammar, and the following regular expression implementing it:

  /^[a-zA-Z0-9.!#$%&'*+\/=?^_`{|}~-]+@[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?(?:\.[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)*$/

Remember that the web is a platform that lives and breathes this stuff. A lot of thought went into this grammar for valid email addresses. This is a good way of filtering out obviously bad stuff while allowing all realistic and sane inputs.

One part of all this that I’m not aware of the situation around is “8. You can put emojis in the local part.” The HTML spec’s validator is all ASCII. It does remind you to punycode the domain labels, but makes no mention of internationalised local parts, and I’ve never learned about non-ASCII local parts or how well they’re supported. I gather they may require the sender to be capable as well as the receiver, whereas internationalised domain names were made compatible with all systems via punycode.

yjftsjthsd-h• 5 years ago

while allowing all realistic and sane inputs.

Isn't that a way of saying "while disallowing perfectly valid options"?

jcranmer4• 5 years ago

What's disallowed are a) IP literal addresses and b) localparts that require quoting. These email addresses are highly likely to break many processing steps anyways; I've only ever seen category b in sendmail configs (it can be useful for internal email rerouting purposes).

There's a distinction to be drawn between the requirements of the actual MTA/MUA/MSA layers and user applications built on top of them. For the latter, considering emails to be invalid if they contain IP literals or quoted localparts is going to be more helpful than harmful (there's less scope for vulnerabilities in doing so). It's just like assuming email addresses are case insensitive: it's inappropriate if you're an MTA, but for everybody else, go ahead and assume they are.

biztos4• 5 years ago

That regular expression fails to validate a bunch of the examples from the article. And also single-word addresses, which are pretty useful if you want to route email locally.

So what makes it the best?

[Edit: it also assumes you've already parsed out the "real" address from the rest of the text field, which to me makes it a half-validator at most.]

crazygringo3• 5 years ago

Yes, that's the explicit point of it.

But it would seem to be the best for general-purpose web use, e.g. signing up for a newsletter with an e-mail address that's pretty much guaranteed not to break anything.

Instead of being conservative in output, it's intentionally being conservative in input.

chrismorgan5• 5 years ago

Single-word addresses? I presume you main domain names with only one domain label (like “localhost”). Read the grammar again, they’re supported.

it also assumes you've already parsed out the "real" address from the rest of the text field, which to me makes it a half-validator at most

I’m confused. The explicit purpose of this stuff is to validate an email address. Not to extract an email address from a freeform text field, which I think is what you’re talking about. Deciding how to do that is a whole ’nother can of worms.

toomanybeersies5• 5 years ago

And also single-word addresses, which are pretty useful if you want to route email locally

Unless you're developing an app for an intranet, that's not a concern for most people.

judge20203• 5 years ago

Hope you're not in PHP, Perl, or Ruby!

http://emailregex.com/

jcranmer4• 5 years ago

That's a pretty bad page, given that it gives regexes that match very different things for different languages, without a) explanation what the differences are, and b) any rationale for why you may or may not want to choose between the different versions, let alone c) why different languages "deserve" different versions of the regex.

This is already a field where there is a lot of misinformation flying around, and a page that merely regurgitates all of that misinformation without the perspicacity to realize that its purported information is internally incoherent is not helpful.

eli5• 5 years ago

How sure are you that the 61 character limit won’t change in some future DNS improvements? People used to think TLDs would only ever be up to 3 characters long.

More importantly, what problem is this even trying to solve? Someone accidentally typing a 300 character domain? If they are intentionally feeding you gibberish they’ll just give you more realistic looking gibberish.

chrismorgan5• 5 years ago

You seem unreasonably hostile.

I’m absolutely certain that the 63-character limit for domain labels is never going to change, because it’s hardcoded in enormous amounts of software and hardware, and there’s no even vaguely compelling reason to even attempt to change it. But if such a thing did change, then you’d just add this to the extremely long list of things that needed to be updated.

People who thought TLDs would only ever be up to three characters long were simply wrong from the very start because they didn’t understand what they were dealing with. (As a simple example, .arpa was there from the start.) Understand that this wasn’t a matter of anything changing, it was that some people misunderstood and thought that a convention they observed was in fact a rule.

The problem this sort of validation is solving is weeding out things that are definitely not going to work, as soon as possible, because it’s good to point out problems to users as soon as possible, rather than having something silently fail or only notifying the user about it much later. Syntactic validation isn’t the be-all and end-all of accepting email addresses, but it’s definitely still worthwhile, even though you should generally do other validation based on DNS lookups and/or sending actual emails as well.

crazygringo3• 5 years ago

Huh. Interesting this doesn't support international email [1] addresses, e.g. квіточка@пошта.укр or Dörte@Sörensen.example.com.

Seeing as the web has long supported Unicode, where are e-mail addresses currently at in that evolution?

Are full Unicode e-mail addresses something that is decently supported today, or still largely theoretical? Is this regex sufficient? What kind of e-mail addresses do people in China most commonly use, for instance?

[1] https://en.wikipedia.org/wiki/International_email

Avamander6• 5 years ago

where are e-mail addresses currently at in that evolution?

Baby shoes because of anglosphere programmers that can't fathom people wanting to use their own alphabets and thus forget to support it.

chrismorgan5• 5 years ago

Internationalised domain names are supported: as I mentioned, the spec explicitly reminds you to do punycode.

For the local part, though, it does look like browsers have fallen down, though I’m not particularly familiar with the situation there. Testing it in Firefox to confirm, ascii@υνικοδε validates, but υνικοδε@ascii doesn’t. https://github.com/whatwg/html/issues/4562 seems to be where progress is made from time to time. As usual, it’s not as simple as we might hope.

toomanybeersies5• 5 years ago

I've always just used

  /^.+@.+\..+$/

That is, "Some characters, an @, some more characters, and a period"

I couldn't care less if users want to enter undeliverable email addresses, they won't get emails. All that regex is intended to achieve is ensuring that the user hasn't accidentally filled the wrong field (e.g. tried entering their phone number) or mistyped a punctuation mark (foo#bar.com, foo@bar,com)

Strictly speaking, it won't match some valid email addresses, such as IPV6 domains. But if I receive a support ticket complaining that we don't accept email addresses with IPv6 address domain, I'll reply advising that the customer should purchase a domain name or sign up to one of many free email services.

StavrosK2• 5 years ago

I gave a presentation on this topic in FOSDEM a few years ago:

https://www.youtube.com/watch?v=xxX81WmXjPg

(Loudness warning)

_wldu• 5 years ago

Perfect is the enemy of good.

If it is a string that has an @ sign, a dot and is at least six characters long, it's probably a valid email address.

a@b.cc

No need to go further than this. It's not worth the time.

criddell3• 5 years ago

You don't need the dot. Item 12 in the list is "You can have dotless domain names."

Biganon• 5 years ago

Do you know many people who own a TLD?

harg2• 5 years ago

indeed, the point the parent comment was making is that effective email validation need not perfectly implement the RFC.

dotless domains are going to be so rare in practice (unless your project has some niche use-case) that you can probably ignore them and call them invalid for the sake of simplicity.

biztos4• 5 years ago

This is a great run-down of the trouble with e-mail addresses.

I worked in e-mail security for quite a while. "Write an e-mail address parser" was my go-to technical interview question.

It was pretty easy to see if the candidate had ever given any real thought to e-mail (most had not); and you could also pick up a lot of signals about engineering style, for instance if they started with a regex (fewer did than I expected). And it was trivial to adjust the difficulty: if someone thought the question was easy and had a fast solution, you could just throw them a test-case like the ones in this article.

(Note: the actual title is "Your E-Mail Validation Logic is Wrong" -- and it's only about addresses, the author isn't implying that e-mail systems can't validate messages nor for that matter addresses.)

LambdaComplex• 5 years ago

What if the answer they gave was "This is a very hard problem that honestly isn't worth solving, just check against a .@. regex and call it a day?"

mrunkel• 5 years ago

So only a@b?

I think you’re missing some stuff in your regex.

CydeWeys2• 5 years ago

"Can you write me a parser that has a <1% false negative and <1% false positive rate on real email addresses?"

A similar enough issue happens in coding interviews anyway. Sometimes the interviewee is aware of a library that essentially solves the problem for you. In those cases I give them some credit for knowing of it and then ask them to implement it anyway, as if the library didn't exist (because there are a large number of problems out there for which a solution doesn't yet exist, and when hiring a SWE you need to find someone who can write new solutions from scratch for those situations; whether a given toy interview problem is such a situation doesn't matter for the purpose of evaluating said skills).

biztos4• 5 years ago

That would demonstrate an understanding of email -- it is a very hard problem -- but probably also an unpleasant attitude you might not want in a co-worker. Whether the problem is worth solving is very often not your call as an engineer.

duxup2• 5 years ago

I'd sort of raise the issue that "Writing an email parser from scratch is a bad idea due to the sheer complexity involved. If you're looking for serious email address validation there may be better options out there that have dealt with this complexity rather than start from the ground up."

Not to say I wouldn't try just for the sake of working through it as an example / 'where would you start' discussion.

But if we're pretending this is a real world task I'd probably discuss how this is an endless / possibly ultimately futile time sink and there might be better options than starting at point A ;)

LogicX• 5 years ago

Nice article.

My only technical nit would be the statement “if there was an MX record”.

Many systems will fall back to an A record to attempt delivery in absence of a MX record.

high_byte• 5 years ago

"How to Hack Things with These 13 Simple Tricks"

duckfang• 5 years ago

Email validation :

Accept email from user.

Send email to that address with a link to verify.

Go/no-go test if link is clicked.

(If you're doing some fever garbage or otherwise trying to parse it, you're doing it wrong.)

swiley• 5 years ago

This doesn't work with hotmail where sometimes a robot will click the link but refuse to deliver the mail.

hutrdvnj• 5 years ago

There is no point in many cases. Even if you can verify that the email address is syntactically valid, you'll still need to check that it was not mistyped, and that it actually goes to the person you think it does. The only way to do that is to send them an email and have them click a link to verify.

However, if you still want to validate an email address then use a library. All popular programming languages have email validation libraries. Yes, it's an extra dependency if it's not included in the std lib or the framework you use, but email validation is wrong in 99% of the cases, if you wrote it yourself.

yawaramin2• 5 years ago

Or use the browser. HTML form validation has <input type="email"> which checks that the entry is a valid email address.

mikl2• 5 years ago

Not quite, browsers have slightly different implementations here, we recently had to stop using this because we had customers using IDN e-mails like hansi@lübeck.de and this would fail validation in some browsers (but not others).

ds• 5 years ago

If your email is <RFC>fan 69™@root I am not going to let you signup. Sending emails cost money and bouncing emails affects your sender reputation. Also, for every user out there using <RFC>fan 69™@root as their email address, there is going to be thousands of people accidently entering their email address incorrectly and not getting a alert about it. Yes you could do fancy shit like checking mx records and whatnot, but come on- Im not going to maintain/build that infrastructure for the one out of a million people who are trying to use that address.

Developer time is precious at a startup and supporting <RFC>fan 69™@root while still denying b ob@gmailcom is very, very far down the list of things to do.

In summary: I don't suggest doing 'perfect' email validation to RFC spec. You will save money/devtime and make more of your users happy by not doing it.

josephcsible2• 5 years ago

This logic is why so many Web sites today won't let you use a plus sign in email addresses, which ruins a really nice Gmail feature.

tyoma• 5 years ago

The best are sites that let you sign up with a ‘+’ but not log in. Zappos used to be the most prominent example.

raffijacobs• 5 years ago

Can't you use "." Anywhere in your email to use the same multiple times in Gmail?

toxik• 5 years ago

OTOH, it being a standardized thing, a spammer would absolutely just strip that plus part off. Better do it secretly like a catch-all.

kyrra• 5 years ago

As people said, true spammers know to just strip off the "+" in the email address. This is actually a fun reason to set up your own domain and set up email forwarding for *@example.com to go to your main gmail or whatever account, then the "username" part of the email I just set to the domain of the account I'm signing up for. So I'll use amazon@example.com when signing up at Amazon (or whatever site).

j_wtf_all_taken• 5 years ago

I don't really think that's the same. Forgetting the "+" in the validation regular expression is something else than refusing to implement all kinds of extra checks to support very weird and very unused things.

ridaj• 5 years ago

Why do so many people responding to this seem to assume the plus sign is to fool spammers? Of course it's not useful for antispam. It's mostly meant to make it easier to trace where a (legit) email comes from, for example to set up filters. https://gmail.googleblog.com/2008/03/2-hidden-ways-to-get-mo...

alkonaut2• 5 years ago

Sites likely prefer your canonical/standard email address over any plus version. It would be easy to trim anything after the plus too I guess and just email you at your normal address

gumby• 5 years ago

I configured my mail server to use _ as a sub mailbox identifier to stop creeps who block +. I assume they are doing it to make sure their precious spam shows up in my inbox.

mderazon2• 5 years ago

Spammers aside, I'm interested to know what strategy different saas companies do in regards to users creating an account with + alias - Do you let users create multiple accounts with the same email but different + alias ? Or do you recognize that it's an alias and say that the account already exists ?

Not all email providers support the + notion so you'd have to run domain lookup on some hard coded list

jjav• 5 years ago

This logic is why so many Web sites today won't let you use a plus sign in email addresses, which ruins a really nice Gmail feature.

Contrary to popular belief, it is not a gmail feature.

I first heard of the + as destination filtering in the very early 90s at CMU where it was broadly used. Every single email address I've had since then has support the same (and notably, apart from a test account, I've never used gmail much, so that's not including gmail).

jimktrains2• 5 years ago

The plus is a standard feature of email, not a Google specific thing like ignoring dots.

ipaddr• 5 years ago

gmail != email. Breaking gmail features helps ensure a level playing field.

welder4• 5 years ago

Yes, in practice I've found the exact same thing. Either use an email validation service or be more restrictive than the RFC. [1] Also prompting "Did you mean bob@gmail.com?" when the user types "bob@gmaail.com" helps a lot with human input errors. [2]

[1] https://www.mailgun.com/email-validation/

[2] https://www.npmjs.com/package/mailcheck

ivraatiems• 5 years ago

Except that as someone with an email at a .co domain, I get really irritated when it asks me "do you mean [mydomain].com?"

I always have to tell people, in real life, "it's .co, not .com," just in case - humans do this too.

WindyLakeReturn• 5 years ago

It depends upon where you are validating email input at.

For the initial email input, your logic works fine. Once it is applied downstream in a process, it begins to get messy. Someone might do an incorrect email validation that happens to block emails that you have already accepted or which you are importing from a valid source. Someone has already given the example of a login field not allowing them to use the email they signed up with. If such upgrades occur later in a projects life cycle, not only might you have to spend developer's time, you may also have a production outage.

Personally, I suggest using some, even if imperfect, validation when gathering the email initially (for the reasons you point out) and then not validating that information any further.

paulmd2• 5 years ago

I actually run into this all the time with passwords using a password manager. Lots of places will accept the creation of a password that's long/complex/etc but then when you actually try to log in with it it won't accept a long password, won't accept certain characters, will silently truncate it and throw an invalid password error, etc.

Sometimes disabling Javascript will fix it, sometimes not. I occasionally have resort to using "I forgot my password" until I figure out what the actual underlying requirements of the passwords are.

novok3• 5 years ago

I've run into this with labcorp. Their desktop webapp takes subdomain emails, but their mobile iOS health webpage login thinks a subdomain email is invalid and disables the login button. They also don't let you change your account email so you can never really fix this issue properly.

scotu• 5 years ago

I found websites not allowing perfectly valid tlds, so maybe they could be starting not using .com in their regex. (.email)

the_arun• 5 years ago

Instead of every developer implementing validation logic, shouldn't we have validation libraries to take care of this?

3np• 5 years ago

How about not doing any pre-validation (save for whitespace stripping) and have a validation e-mail (which you should require anyway) take care of any typos?

With precious dev time, you can do better by doing less.

edoceo• 5 years ago

You risk sending a junk message tho, which affects your sender-spam score with other providers.

I just make folks email me first.

markonen• 5 years ago

You absolutely should check the MX records, though. It’s easy and catches tons of typos. I was floored by the difference when I implemented this as pre-check before a Stripe Checkout form.

jcranmer4• 5 years ago

The basic rule of thumb I use this: are you implementing email at the MTA level (needing to build/parse RFC 5321 commands or RFC 5322 blobs directly), or are you using email closer to a "universal internet ID" purpose (i.e., application perspective)?

If you are in the former category, then yes, follow the spec to the letter. If you're in the latter, then screw the precise guidelines of the spec and reject emails that are very unlikely to be valid: no quoted localparts, no IP address literals. In addition, go ahead and say that email is case-insensitive (more precisely, case-preserving).

The hard part is if you're writing an email client, because you're basically forced to have your hands in both pies.

jcelerier• 5 years ago

Sending emails cost money and bouncing emails affects your sender reputation.

that works as long as <RFC>fan 69™@root does not write articles for ZDNet

vorpalhex2• 5 years ago

My email address is valid and has been valid for a really long time.. but about 5% of ecommerce shops refuse to accept it.. so they don't get my money.

Don't get clever, just follow the spec.

lisper3• 5 years ago

about 5% of ecommerce shops refuse to accept it

That's surprising to me because there is nothing particularly weird about your email address. What exactly do they complain about?

jchw3• 5 years ago

If you can show that "just follow the spec" ends up opening up more opportunity than it closes off, then you can convince people. However, when gmail, outlook, etc. do not allow these zany e-mail addresses, you're going to have a hella hard time convincing me of this unless you are in the 1% of spenders.

skeeter2020• 5 years ago

> Don't get clever, just follow the spec.

I'd suggest being clever is wasting countless hours to handle your edge case. Or writing your own email validation in the first place.

yupper324• 5 years ago

If 5% of ecommerce shops refuse to accept it, it's likely you being clever.

My email is refused by 0% of ecommerce shops... because I just have a normal email.

Don't be clever, pick a better email.

alkonaut2• 5 years ago

Your money is likely a minuscule part of the revenue and supporting your email would likely cost more. This was the point, that it is probably clever to choose a validation that covers 99.99% of customer emails rather than cover the whole spec.

forty• 5 years ago

100% agree. This is especially true if the address mail is going to be displayed somewhere for example, it's generally a good idea to limit email address to a sunset of what the RFC allows.

To adapt from a famous quote: "all email validation logics are wrong, but some of them are useful" ;)

unoti• 5 years ago

Totally agree with this. Trying to be perfect is a good road to paralysis and not getting things done. Software is like people: it's ok to not be perfect, especially if they're always trying hard to be better and doing good things for society.

paulmd2• 5 years ago

This sounds great but what you think is "common" probably isn't.

When I was validating myself for Amazon Prime Student, I literally had Amazon refuse to accept my student email in the form first.m.last@myschool.edu because there were two '.'s in the mailbox portion. I had to send an email to support and it was eventually dutifully fixed.

And that's not an uncommon format for, you know, school emails. And that's an Amazon engineer who should have known.

I imagine there's developers who think "domain.tld" is the only thing valid to put in the domain portion, and that's going to fail with "domain.co.uk", or uncommon TLDs, or other perfectly valid constructs. And sure "it's only x% of the users" but it's a pain in the ass if you're that user. You need to be reasonably permissive.

(but on the other hand "myname@..." is not valid either, and that will fail and cost you money as well... hence leading us back to 'just follow the spec')

incrudible• 5 years ago

This is a false dichotomy. Supporting dots in the email address is trivial, following the spec is not.

Furthermore, for a user, it is trivial to get another email address if the one they have causes issues, so it is not really an accessibility issue either.

throwaway09223• 5 years ago

How do you reconcile your concern for the cost of sending emails with your unwillingness to do super basic validation like checking an MX record?

nawgz• 5 years ago

From where I sit, both of those concerns sit on the same side of fence. GP argues against extensive developer time spent on validating edge-case emails, and says they do so in no small part to avoid having emails bounce etc., as doing MX or other validation to follow-up on these edge-case emails validity within your service does nothing to imply others have put in this same costly and nearly superfluous support, likely leading to more emails bouncing and accordingly degrading the trust in their business as a sender

toomanybeersies5• 5 years ago

I also came to the same conclusion some years ago. Or more specifically, my manager brought me around after I tried arguing that it was worth the time to make sure that users could use an IPv6 address as their domain (the lack of periods after the @ would cause `user@2001:0db8:85a3:0000:0000:8a2e:0370:7334` to fail validation)

He made a very convincing argument that while an IP address is technically a valid domain, but how many legitimate users were seriously using an IP address as their email domain? (zero)

bombcar2• 5 years ago

You’ll love this SSL cert:

https://[2606:4700:4700::1111]/

goto112• 5 years ago

But why restrict the syntax arbitrarily in the first place? It is not going to catch the common typos anyway. Most typos will just result in a wrong but still syntactically valid email address.

harryf• 5 years ago

I’ve always wondered if it’s possible to have a valid email address which is also an SQL injection attack, XSS or similar ?

eli5• 5 years ago

There's also incredibly low stakes in allowing a technically-invalid email address to pass validation. Just use a very permissive pattern (e.g. contains an '@') and be done with it.

No matter what you will constantly be getting addresses that conform to the spec but cannot actually receive mail.

novok3• 5 years ago

I've run into some places where a subdomain email is not ok, which has been pretty annoying. All email validators should be able to at least take first.last+company@subdomain.example.com

macksd• 5 years ago

Especially when using a country TLD, suffixes like .co.za are appended to the name of the actual ISP or email provider.

NikolaNovak• 5 years ago

I get your point, but it ends up pretty arbitrary who picks up what part of spec to implement / which part of spec they deem "common sense".

e.g. It drives me BONKERS how many systems absolutely reject my single-letter email (~"N@domain.com"), which I created specifically to make it easy and safe to type on mobile devices etc. Others will reject the "+" sign, or underscore, or dot/period, or (brilliantly) two periods or underscors, etc etc etc :=/

hwbehrens• 5 years ago

My email address ends with the .cc TLD, and the number of websites which say "Did you mean to type .ca?" and then refuse to let me continue without changing it drives me similarly batty.

Strom• 5 years ago

There are also blacklists for names you can use. My real e-mail is admin@myname.com but Facebook doesn't allow me to use that e-mail, warning me that only personal e-mails are allowed. Paradoxically I ended up using my work e-mail to get around the restriction.

ivan888• 5 years ago

I don’t even have anything weird going on with mine other than using my own domain with a less common 2 letter country code TLD, but it gets me rejected from signing up to at least a few services per year

mobjack• 5 years ago

Customer complaints usually determine the spec.

If enough people wrote in about not accepting one letter email addresses, then they would likely update the validation.

But if customer service tells users to use another email address in that scenario and the customer does that, then it might not be worth the effort to fix it.

AgentME• 5 years ago

Agreed. Anyone that decides to come up with a "common sense" subset of emails to allow is more likely to accidentally block some perfectly normal addresses than actually accomplish something useful. I remember I once made this point in an HN thread, and I got a very dismissive and overconfident reply by someone who posted a regex that turned out to block my own email address (which had a number immediately before the @ symbol).

Writing code that doesn't need to exist that has a possible failure mode of not letting someone sign up at all is just a bad decision. If you're going to write that code, either go through the effort of getting it completely right or soften the failure mode. If you really think the user is somehow mistyping their email with special characters or an unusual TLD, then you could show them a non-blocking warning message.

skissane• 5 years ago

Maybe what we need is a narrower standard that subsets RFC5322 to remove (or at least deprecate) rarely used address syntaxes, such as double-quoted local parts.

(Actually RFC5322 already deprecates some syntaxes. For example, "John Hacker"."Ph.D., Esq."@example.com is a deprecated syntax (obs-local-part), because it contains multiple quoted-string components separated by dots.)

or (brilliantly) two periods or underscors

Two or more consecutive periods is actually disallowed by RFC5322 (unless quoted). foo.bar.baz@example.com is a valid address, foo..bar@example.com is not. ("foo..bar"@example.com is however)

megous4• 5 years ago

Yes, you should reject it, because the address is invalid. You can't have unquoted <> in the local part. You can't have spaces there either.

Your other reasons for breaking interop between systems/languages are just whimsical and invalid. :)

bradknowles2• 5 years ago

This falls into the category of the classic “Stupid shit that most programmers believe”.

Fundamentally, the problem is that if you’re trying to validate an e-mail address as being correct and you’re not sending an actual e-mail message to that address, then you’re doing it wrong.

We learned this lesson back in 1995, people.

bradknowles2• 5 years ago

As the Sr. Internet Mail Administrator for AOL in 1995, we quickly learned that the only real way to validate an e-mail address was to send mail to it. Even that wasn’t guaranteed, but anything less was doomed to unexpected failure and misery.

And this wasn’t a new lesson then. But at least we were smart enough to listen to the people who had learned that lesson before us.

It is now over 25+years later, and I’m sad to see that many people seem to be bound and determined to force themselves to re-learn that lesson the hard way.

fencepost• 5 years ago

Hah, even beyond the question of address format variations there are also commercial services that do some level of email address validation - and one of them regards my business email address as invalid (firstname@companyname.com) - adding another letter works, adding punctuation works, it's just my specific first name.

Unfortunately it's done via black box on their server(s), so it's not like I can even dig through the code and figure out what's going wrong.

bcrl• 5 years ago

I think the best email address I ever knew of was n@ai . Unfortunately, the .ai TLD eventually decided it wasn't a good idea to have an MX recorded resolving on the TLD.

arkitaip3• 5 years ago

It's cute but doomed to fail because it goes against everything most people understand about email addresses.

mLuby• 5 years ago

Validation errors are common, but warnings are not.

I'd like to see more of "Patterns like [what you entered] are uncommon—are you sure?" instead of "Patterns like [what you entered] are not allowed—change it to proceed."

richeyryan• 5 years ago

I recently implemented this using the great Mailcheck library. So if someone types "gnail.com" or "gmail.con" it detects it and we can show "Did you mean gmail.com?". If someone ignores the suggestion, fair enough. If someone purposely wants to give us a junk email, fair enough. At least we're not frustrating them needlessly.

https://github.com/mailcheck/mailcheck

zzo38computer3• 5 years ago

Mostly, yes. However, some things should probably still be prohibited, such as:

- An email address ending with ".invalid", unless invalid email addresses are supposed to be allowed (which in some cases is useful, but you can then disable sending email to such an address, using it only for identification). (I do use such an email address for identification on NNTP.)

- Email addresses without at least one at sign.

- Email addresses containing control characters (at least ASCII control characters).

- If the domain name does not resolve or resolves to a loopback address or LAN address (except for some specialized cases where such a thing is desirable). The same is true for literal IP addresses; if it is a loopback or LAN address then it should be disallowed, but otherwise it can be allowed.

mooreds• 5 years ago

Hah, just went over the email address validation logic in our app last week because a client asked.

Turns out we do minimal validation (make sure there is a local and a domain, that there are not two periods next to each other, and a few other things) but what we really rely on is deliverability.

In other words, if your email needs to be verified, we'll try to send an email to the address you provide. If the link is clicked (or the code entered), that's good enough for us.

Applications using our service (we're an auth provider) can decide for themselves if they need email address validation. It's a boolean flag on the user object. If they do, they can use the functionality we provide to ensure it.

csours• 5 years ago

This feels like a discussion for backend implementations/email forwarders, not for email signups... but hey while this has some attention - For god's sake, put a button that says "This ain't me", at least for important stuff.

I'm sorry, but I just can't bring in Clyde's truck for the oil change, cause Clyde ain't me!

I also cannot attend Cassidy's parent teacher conference, apologies, I am not in Ohio.

cratermoon4• 5 years ago

This feels like a discussion for backend implementations/email forwarders, not for email signups...

And yet I've worked multiple places where product people asked for "simple email validation" on user signup. If they insist, I ask them to provide some actual test cases that they care about. Sometimes the product folks can be convinced to drop the validation requirement if they can be shown that anyone who can't sign up because their email address doesn't validate will simply move on and not sign up.

In the case where your product is B2B and all the employees of your customers are users (say an HR product), then the first time a VIP at an important customer complains, that's usually enough to convince your stakeholders to disable the email validation.

Yaina• 5 years ago

What this article really showed me that this RFC is actually pretty harmful.

Supporting all of the rules outlined in the spec is probably a huge burden for maintainers of mail clients and servers. Obviously some parts of the spec are going to be omitted. It's hard to blame them for it, but the same person that rightfully skipped over implementing the routing thingy might've also wrongfully assumed there won't be a Japanese character in the address. And that's what's so bad.

You might introduce more issues in your system, by taking the full spec into consideration for your validation, instead of using the whatwg regex someone posted here.

nradov• 5 years ago

Well if there are problems with the RFC then you should work with the IETF to correct those. They have an open standards development process.

awestroke• 5 years ago

Another option is to just ignore the RFC

k__• 5 years ago

My main email, which is only 8 characters long, got denied sometimes based on length.

Doesn't happen very often, though.

cratermoon4• 5 years ago

I've decided that the best way to validate email address is to not validate them, but require that any signup be finalized by the individual following a link emailed to them.

This allows a person to use any damn thing they want as their email address, provided it works and they can get the email.

welder4• 5 years ago

If sending emails is 100% free, but you still have to worry about your sender reputation. [1] Sending a large amount of mail to invalid emails will start getting your emails put in people's spam folders. That's the reason email validation services exist, to prevent sending to invalid emails. [2]

Also, humans make mistakes. You should detect spelling errors and typos then suggest corrections. [3]

[1] https://www.mailjet.com/blog/news/3-factors-that-impact-your...]

[2] https://www.mailgun.com/email-validation/

[3] https://www.npmjs.com/package/mailcheck

mro_name4• 5 years ago

Even if 0% free you'll have to do the opt-in anyway, or how on earth will you figure out if the recipient wants your email?

It's hard to be smart with something like names.

eli5• 5 years ago

Mickey@mouse.com is a perfectly valid address but it isn’t my address. If that matters for your application you need to spend the capital to send an email. No way around it.

jdhawk• 5 years ago

So you use another 3rd party validation service, paying $300-500/million addresses.

deckard12• 5 years ago

This is really just a problem for spammers going out and either buying mailing lists that haven't been validated or scraping the web for email addresses. In the case of the spammer, they would probably care a lot more about their bounce rate than their false negative rate (i.e. valid addresses that fail some sort of validation regex). In fact, they would probably tune their validation to actually throw away addresses that didn't look correct just to be safe.

Obviously, this is a different scenario than your bank not accepting your valid (per RFC) email address. Which is why any sort of blanket advice is pretty dumb. Not that I care to aid spammers...

The other scenario might be a site that puts up a "paywall" type thing, where you are forced to enter an email address to gain quick access to something, but doesn't want to bother you with going and verifying an email (e.g. instant discounts, downloading a PDF, etc.). Or in-person email address collection when you buy something in a store. It's never a good idea to collect email addresses of people that have no desire to subscribe to your marketing.

Avamander6• 5 years ago

Oh don't worry about this at all because spammers are going to sign up with legitimate e-mail addresses that are going to get your reputation lowered. Very common tactic and you won't be saved by some dumb regex that would just probably hurt a few real users.

cratermoon4• 5 years ago

Yet those tools are full of bugs as well as culturally blind and prudish assumptions. Hint: try signing up with the address null@null.com

- https://blog.jgc.org/2010/06/your-last-name-contains-invalid...

- https://haacked.com/archive/2007/08/21/i-knew-how-to-validat...

- https://fosdem.org/2018/schedule/event/email_address_quiz/

serial_dev2• 5 years ago

This is also my preferred approach.

If I can send you an email and you can verify that you have access to that email, your email is "valid enough" for me.

Then, the validation is basically "is there an @ and after a dot in there?". I find that after that, every hour spent on improving the validation will just cause more emails falsely flagged as invalid, more support requests from the people who couldn't sign up with valid emails, it's code we need to maintain, anytime edits the validation logic risks breaking sign ups completely.

So with more "improvements" to the validation, you just cause more problems. Then why do it?

I hear the reputation arguments, but in practice, it never happened to any of the organizations I worked for.

What happens though very often is naive engineers trying to solve problems the business doesn't have with knowledge they lack...

mro_name4• 5 years ago

naive engineers trying to solve problems the business doesn't have with knowledge they lack.

premature implementation is the source of most evil. :-)

Arubis• 5 years ago

100% agreed here. Accept a text field; maybe validate that it has an @ in it and a . after the @.

Send that address a confirmation email. Now you've got consensual opt-in and you've somewhat protected yourself from adding a wrong address to your recurring mailing list.

Prevent abuse with long (seconds) delays between submissions from the client. If the user thinks they did it right, they're waiting on their email inbox anyway; if they immediately realize they made a typo, it'll take 2-3s to fix.

The RFCs were written when manually (not from cron) sending email to another user on your local system as a thing that actually happened. I'm certain you actively want to avoid that now.

eli5• 5 years ago

Yup I’ve been working in email marketing for a long time and this is what I do if I need a regex. I remember when .mobi TLD came out and people with those address had a terrible time signing up for things because a bunch of developers got too cute and assumed a TLD could only be 2 or 3 characters. You want to be really lax in what you validate.

manmal• 5 years ago

My cheap-o approach to this is: Check there’s an @, and that there is a dot afterwards. This excludes local domains obviously, but I don’t want those anyway.

xyst2• 5 years ago

g-mail smtp is returning this error:

"Verify that you have addressed this message correctly. Check your SMTP server settings in Mail preferences and verify any advanced settings with your system administrator.

The server response was: The recipient address <'*+-/=?^_`{|}~#$@[ipv6:2001:470:30:84:e276:63ff:fe72:3900]> is not a valid RFC-5321 address. <...> - gsmtp"

Not even Google engineers can get it right. We are doomed.

aeharding• 5 years ago

I just want my .dev email address to not be rejected.

pmontra2• 5 years ago

I just check that the string contains at least an @ character. That ensures that we're not rejecting people with uncommon patterns in their email address and takes very little time to design, develop and test.

In a project we're doing something fancier: we check the result of sending mail and store it in the database record for the account (Mandrill notifies us on a webhook.) Then we might take actions for bouncing addresses. The actual impact on the project has been zero so far.

dokem• 5 years ago

Sorry no, your email address is wrong. Make a new one.

slifin• 5 years ago

There's a lot of busy work in computing in the name of preventing mistakes

At a certain level of complexity it's easier to just let mistakes happen and provide correction tools if & when required

I remember getting our IPs blacklisted trying to programmatically ask email servers if the email address we were provided was real

JoyfulPanda• 5 years ago

There is is awesome talk about E-Mail by Ricardo Signes:

https://www.youtube.com/watch?v=JENdgiAPD6c

The first 5 minutes are perl specific, but the rest is email and just hilarious.

mro_name4• 5 years ago

Actually email validation is simple: do the opt-in.

If confirmed, it's valid.

mfbx9da4• 5 years ago

I once had a crack at building a sensible email validation library.

* Validate the string contains "@" and a "." to the right of it.

* Validate common typos

* Validate disposable emails

* Validate MX records

* Validate SMTP server and mailbox

https://github.com/mfbx9da4/deep-email-validator

I don't have the time to keep it maintained but it works for the most part!

devfatigue• 5 years ago

What if we flipped email validation around and made the users email a one time code to validate their email?

danrl• 5 years ago

I have a very short email address in the format a@b.tld and for my special friends that don't know how to validate correctly I have created abc@ur-email-validation-is-broken.b.tld

I need to use the latter ~5% of the time. Most often I take my business to someone else for the sake of principle.

jacobobryant2• 5 years ago

I just outsource this to Mailgun. User signs up, I send them a confirmation email, account doesn't get created till they click the link. If the email address is invalid, Mailgun returns an error and I show a page that says "We couldn't send an email to <address>. If you're sure that's a valid address, please try again." (Also use recaptcha for bot detection).

wyldfire• 5 years ago

The local part is case-sensitive.

This seems more like a bug than a feature. Maybe in 1983 the average email user knew what DNS was and could be expected to know one part of the email address would be case sensitive and the other not.

But email RFCs are probably like any other RFCs out there and specify existing behavior for the sake of interoperability.

deckard12• 5 years ago

yeah, and that's a hill I'm willing to die on.

Imagine the average non-technical person talking to some customer service agent on the phone and having to figure out if her email is JANEWATSON@gmail.com, JaneWatson@gmail.com, janewatson@gmail.com, or Janewatson@gmail.com. Could you imagine the horror and complete security nightmare of multiple people running around using the same gmail address with different case. Those Jane email addresses above would be four different people. We'd be receiving mail intended for other people all day long.

eldenbishop• 5 years ago

It's not even about crazy emails. My wife works for an Aerospace company so her work email was blahblah@blah.aero and a huge number of websites still don't recognize that as a valid top level domain.

slavik81• 5 years ago

I created myusername@hotmail.com in 1999, then immediately lost the password. I couldn't recover my account or use the same name again, so I created myusername_@hotmail.com.

In the twenty-two years that have followed, the only website that has had a problem with my email address is Chapters Indigo, which explicitly rejects it as invalid.

For email validation, keeping it simple is best.

BenjiWiebe2• 5 years ago

Why not accept absolutely anything in the email address field, and just require an emailed link to be clicked before marking the email as validated?

rblatz• 5 years ago

Because it causes conversion drop off.

zzo38computer3• 5 years ago

I cannot send a message to the email address they provide, but not because of anything wrong with the email address itself, but because that email address is version 6 internet, and I have version 4 internet.

eli5• 5 years ago

No the problem is developers confusing validation with verification. You can’t validate your way to a correct address and it’s wrong to try.

If your goal is to catch typos you’re better off with very lax validation plus a library that suggests corrections like “gmail.com” for “gnail.com” (both of which are of course technically valid domains)

permo-w• 5 years ago

I think the author is stretching the words "valid" and "invalid" past their limits here for the sake of hooking you into the article. Yeah, in some countries it's a valid social practice to spit on the floor in public, but in most, it's not.

let's say I'm a dev at google, and I'm writing some aspect of gmail. Is !"£$%@gmail.com a valid email? No. So the word valid is clearly not being used correctly here.

At the core of smtp these emails are allowed, but in practice they almost never are, and so the opposite is true. All the cases he described are, in practice, invalid.

crystaln• 5 years ago

So <any chars>@<any chars> seems beyond good enough. There is no benefit to validating beyond that for almost all cases.

quercusa• 5 years ago

These days, I think 80% of email validation is just catching 'gmial.com'

mk81• 5 years ago

Clicked thinking this might be useful information; was mostly disappointed.

sparrc2• 5 years ago

I have my own custom email domain (sparr.email) that fails validation surprisingly often.

saurik• 5 years ago

This is all a massive misunderstanding. An email address is the local name and the host; a host can't contain an @, so the only thing you frankly need do is split on "last @" and demand the user not escape anything. As for validation, go ahead and try to resolve the domain to make sure it works (and, if you want to verify the local part, do an online check with their server).

If this squicks you for some reason--as maybe that format is non-obvious with respect to the lack of a need to escape @--give the user two boxes with a hardcoded @ between them and have them type the two parts separately: pre-parsed input need not ever be escaped, as you aren't going to parse it at all; no need to implement " dequoting.

All of these escaping rules are then to support embedding this identifier into SMTP. The rules for embedding the same identifier into MIME are different... and even more complex! In MIME they support random stuff like "comments" in the middle of the string... is that part of the email address identifier? No.

An email address simply is not defined by the format you use to send it as part of an SMTP command, nor is it defined by the format you use to send it as part of a MIME message header :/. Into is an identifier that exists separately from either of those two (different) protocols and one would expect any number of ways to escape that content.

To demonstrate how ridiculous this all is, imagine someone comes up with a JSON protocol for mail submission and then documents how email addresses now should use \u encoding and escape quotation marks... does that mean users should type that into your app? No.

Hell: your email address form is taking an email address and then sending it over HTTP... the escaping rules for HTML form fields are different still, yet no one is asking users to type HTML-escaped strings into other applications, right?

The core thing wrong then with your email validation is that you are simply validating the wrong thing: unless you are developing an SMTP server, the rules for how to escape and parse escaped email addresses in RFC5321 are irrelevant; and, likewise, unless you are developing a MIME parser, the rules for how to escape and parse escaped email addresses in RFC5322 are also irrelevant.

The only thing that matters from either of these specifications is the underlying basic rule for what semantically can exist in a hostname and a localpart, and RFC5321 is extremely lax: you can use any "ASCII graphic or space", and so excludes only ASCII control characters and 8-bit characters... and then, as mentioned, another RFC removes the 7-bit limitation and opens up the world of Unicode.

(To push on it even further: it isn't even clear to me that one should consider the ASCII control character limitation to be fundamental to the email address identifier or a weird limitation of the current version of SMTP; and since none of those email addresses are going to work, I think one may as well just consider the local part to be any string of Unicode code points.)

Think about this: it is up to your SMTP library to correctly escape the email address you give it for SMTP, and here's the fun part: if you give it a pre-escaped email address, then clearly it is going to have to double escape it, right? So, semantically, these extended discussions of quoted strings and character limitations are always just so ridiculous :/... you absolutely should not be dealing in SMTP-escaped addresses or asking your user to understand SMTP (and the same goes for MIME).

(BTW, if you want some "real hell", one of these two protocols--I forgot which... I presume SMTP--seriously supports an empty local part. If that doesn't tell you everything you need to know about un-opinionated these RFCs are with respect to "anything goes" then I don't know what will ;P.)

roachpepe• 5 years ago

Not really an email issue so sorry if maybe somewhat off topic but on the subject of validation I can't not bring this up - this, remember the guy whose name was Null? https://www.wired.com/2015/11/null/

sneak2• 5 years ago

This is a bunch of weird edge cases that nobody uses in real life except maybe the plus trick.

American Express and Walgreens don't let you set a whatever@whatever.email address because they check for a TLD known at the time of their app's validation code, or something.

novok3• 5 years ago

I use a amex@whatever.com & walgreens@whatever.com email for both amex and walgreens?

I've run into 2, old-ish institutions that didn't quite work with my whatever@whatever.com and had to modify it slightly for them.

sneak2• 5 years ago

.email is a new-ish TLD.

flerchin• 5 years ago

The one I see over and over is failing to trim the email before doing validation. This is especially egregious at account creation where you want no friction. Users enter their email with their smartphone, and it may append a space at the end. More than once, I've had a relative call me trying to figure out why $website wouldn't accept their email as valid.

bilater• 5 years ago

Unpopular opinion: Just ignore these edge cases and focus on the 99.99% of the sane population that doesn't get off on having a weird email address.

More generally: If an edge case exists and has nothing to do with accessibility (it was caused by a user having a different workflow like needing a screen reader or being in a less developed part of the world with slow internet) then you should dismiss them and not make your code/life unnecessarily complicated.

jillesvangurp• 5 years ago

The only relevant email validation is verifying a user can click on a link or enter some code sent to them to the address specified by them. Without verifying ownership, the email address is worthless as an identifier so making it conform to some syntax is not that relevant and you should not use unverified email addresses as identifiers (identity theft is a thing).

Obsessing about regular expressions for these addresses is generally a waste of time except for maybe preventing a lot of failed attempts to send stuff to a clearly invalid email address. A simple string contains '@' is probably good enough for that. Worst case the email address does not work and you discard the entered information after some reasonable time frame. The user has the option to try again and do a better job of typing their email address.

tehwebguy• 5 years ago

Can't seem to find it anymore but wasn't there a post about how the letter "d" by itself was a valid email address at one point?

radicalriddler• 5 years ago

I have two things.

The amount of times I've tried to sign up with my protonmail account to a service and it doesn't pass validation simply because it's a protonmail account (not a gmail, outlook, hotmail or aol apparently). makes me wish everyone did follow the RFC. I actually emailed a service one time, and they responded that it's due to protonmail usually being associated with shady stuff wtf.

The second. I had to implement an email validator at one of my previous jobs, and fell down the RFC rabbit hole. Not only did I have to follow the RFC as per my bosses request, but I also had make sure that Amazon SES allowed it. Came out of the office wanting to just walk out onto the road. The weird things that not only email servers allow, but also, what do email clients allow.

lilyball• 5 years ago

I’m of the opinion that all you should validate is that there is an @, the text to the right side (of the final @) is a well-formed dotted DNS domain, and that there exists at least one (non-whitespace) character to the left of the (final) @.

Yes, I can craft garbage emails that pass this quite easily, but who cares? If I’m crafting fake emails I can make valid ones too. This rule ensures I typed the @ and the dot in my domain (we really don’t need to support dotless email domains and it’s better to catch “foo@gmailcom”) and it won’t reject all the weird random emails people might have.

jedberg• 5 years ago

There is exactly one right way to validate an email address: Send an email to it.

It makes a lot of sense to have a good regex that validates it on the frontend and even tells the user, "hey, this doesn't look valid, are you sure?" but don't ever reject someone for failing your regex.

At the end of the day, you should only validate by having them click on a link in their email. Then you know the whole thing works.

If you're worried about someone hurting your reputation with a bunch of bad email addresses, you can certainly mitigate that by rate limiting your signup API or even deprioritizing sending confirmation emails to email addresses that don't look valid. But you still should eventually send an email to make sure it's valid and not reject it until it bounces back.

mikl2• 5 years ago

People love to quote the RFCs, digging up examples of weird things that are technically allowed in e-mail addresses, but not widely used or supported.

Frankly, I don’t give a damn about supporting some l33t h4x0r wanting to be clever about his e-mail address.

So no, my logic is not wrong. It just doesn’t care about all the weird edge cases.

wildmanx• 5 years ago

I'm always amazed by those "you're doing Xyz validation wrong" posts. This is 2021. There should be libs out there that do this perfectly and we should be using those. Nobody should be writing this kind of thing from scratch, especially not somebody with "precious developer time at a startup". For fun, sure, why not. But doing this from scratch for production software is nonsense. Parsing email adresses, URLs, IPs, OS path names, even parsing common file formats, stack frames or, god forbid, hacking your own crypto. Don't.

Complexities of e-mail validation logic

Related Stories