SpamChannel: Spoofing emails from 2M domains and virtually becoming Satan [pdf]

https://media.defcon.org/DEF%20CON%2031/DEF%20CON%2031%20presentations/byt3bl33d3r%20-%20SpamChannel%20Spoofing%20Emails%20From%202%20Million%20Domains%20and%20Virtually%20Becoming%20Satan.pdf

by mfru • 3 years ago

341 104 3 years ago

media.defcon.org

crtasm• 3 years ago

Video of the talk: https://www.youtube.com/watch?v=NwnT15q_PS8

or here (video format didn't work in firefox for me but VLC plays it): https://media.defcon.org/DEF%20CON%2031/DEF%20CON%2031%20vid...

Brybry• 3 years ago

I think these might be the demos that were unable to play during the talk:

https://www.youtube.com/watch?v=61PIOBp30vA

https://www.youtube.com/watch?v=eODw4t4WaCw

jeffbee3• 3 years ago

TL;DR They found an open relay included in many SPF records.

3np• 3 years ago

Where out of those 2M domains with SPF records opening themselves up, <1k had set up DKIM/DMARC records and somehow even those with DKIM configured and neglected still pass as authentic in Gmail.

jeffbee3• 3 years ago

That part did not make any sense to me. I do not see how this person determined that these domains are not using DKIM. There is no standard DKIM domain name, so what did they check?

ttul15• 3 years ago

Except it's not an open relay. MailChannels would not exist on the internet if it didn't aggressively control spam and phishing. The DEFCON talk did not prove the existence of a gaping hole; it showed something that has existed since the dawn of internet email: the sender domain cannot be adequately authenticated unless you use message signatures such as S/MIME and DKIM. And even with DKIM, DKIM replay attacks allow widespread abuse.

jeffbee3• 3 years ago

Sorry, no. Any service that will issue emails masquerading as 3rd-party domains, which does not use an effective authentication mechanism, is an open relay by definition.

elif• 3 years ago

That's a terribly incomplete understanding of what's going on here.

It is incumbent upon them to validate that their API users own the domain they intend to send from. All the reputable senders do this. Allowing unchecked user-defined from field is the definition of an open relay.

weird-eye-issue8• 3 years ago

You don't understand the situation at all

Manouchehri4• 3 years ago

Ironically it's not possible to use S/MIME with MailChannels here because you don't allow sending raw MIME content.

weird-eye-issue8• 3 years ago

Worse than that. The platform itself didn't even attempt to verify domain ownership so you could send as anyone.

charles_f• 3 years ago

The level of complacency of mailchannels when they were contacted about the issue is just unbelievable. Those guys are responsible for a sizeable portion of the corporate email traffic, and don't seem to give a shit about security or spam (the cancer of the very thing they sell) until publicly shamed about it.

Entertaining presentation

Manouchehri4• 3 years ago

Eh, it's not just their security that has complacency. I contacted their support and explained how to fix their incorrect handling of text/x-amp-html content (they insist on putting text/html first, which breaks Apple Mail when sending AMP emails), and this is the eventual response I got after wasting time trying to help:

```

I do understand what you want, but we're not going to make that change at this time.

This probably isn't the answer you wanted, but I'm afraid we can't be any more help with this issue. I'm marking this ticket solved. Please open a new ticket if you need anything else.

```

sambazi• 3 years ago

sure, otoh they build big money faucet upon "don't worry, we'll take care of it" which you've gotta respect

teddyh4• 3 years ago

Display a banner on any email that doesn’t have a DKIM signature but comes from a domain that’s implemented DKIM (I’ve never seen this, I’m assuming possible?)

IIUC, it’s mostly not possible, because there is no sure way to know if a domain has “implemented DKIM”.

You can, in theory, do a DNS query for “_domainkey.example.com” and see if you get NXDOMAIN or NOERROR; the latter usually¹ means that there are subdomains present, which in turn would imply that there are some DKIM keys present in DNS (although you can’t know what the selector names are). But you don’t know if these keys are active, or if they are meant to be activated later. Or there might be multiple authenticated senders for a domain name, and some might use DKIM signing, but others might not.

1. Not all DNS servers obey the standards correctly, so the NXDOMAIN/NOERROR distinction only mostly works.

mike_d• 3 years ago

It is possible for providers who see a statically significant potion of mail traffic to see all or almost all DKIM selectors.

I think what the line you quote is trying to say is reject mail from MailChannels that isn't DKIM signed.

teddyh4• 3 years ago

It is possible for providers who see a statically significant potion of mail traffic to see all or almost all DKIM selectors.

Yes, but you can’t know that every authenticated sender is using DKIM at all. If the mail you just recieved from a new sender (even if it is from a domain which you have recieved DKIM signed mail from before) passes SPF but is not signed using DKIM, they are quite probably from a valid, albeit rarely used, mail sender who simply does not use DKIM.

I think what the line you quote is trying to say is reject mail from MailChannels that isn't DKIM signed.

No, they can’t be saying that, since elsewhere they say that only 105 out of 2 million MailChannel domains are using DKIM. Therefore, they can’t be reasonably suggesting to block all mail from all but 105 of these domains.

promiseofbeans• 3 years ago

We use cloudflare workers + mailchannels in production. Yikes.

We were already sorking on migrating away from cf workers to real servers, and now also we'll also have to move away from mailchannels I guess.

Security risks aren't worth the convenience.

ttul15• 3 years ago

Note that since June 2023, it is impossible to send email from Workers through MailChannels unless you publish a _mailchannels record in your DNS.

inetknght3• 3 years ago

Ahh yes, another "standard". https://xkcd.com/927/

Thorrez2• 3 years ago

Source? The talk slides seem to contradict that:

● CF workers will be required to set their “Domain Lockdown” record in order to send emails after a specific cutoff date (TBA). You’re still able to spoof all the things until then!

pevey• 3 years ago

This doesn’t seem like a real fix for the issue. This brings the number of spoofable sites way down from all MC users to just those who send mail through CF. But it is still a big vulnerability.

For any domain that is using mailchannels through cloudflare, you can see what region they are using. And you can continue to spoof them. You just have to do it from the same region.

And this is terrible positioning from Cloudflare’s POV. Why would anyone send email through a CF worker since it requires advertising through a public record (DNS) that is by design accessible/scrapable by bots that you are using an insecure service. It’s like asking people to spoof you.

I don’t get why CF doesn’t do something more sensible, like limit sender addresses to domains that are already set up in the cloudflare account where the worker was created. Basically every other provider does this.

vinay_ys3• 3 years ago

This seems like a hack. why doesn't mail channels do a proper authentication of the sender?

vinay_ys3• 3 years ago

To be clear, this issue is really with mail channels not authenticating the sender as the verified owner of the sending domain. CF workers are aren't the issue here.

mc10• 3 years ago

I wish there was some way to evolve the DMARC spec such that only DKIM and not SPF is used for validation. Sadly things like Google Calendar invites still fail DKIM…

ttul15• 3 years ago

The next release of DMARC will likely include an option to exclude SPF from DMARC verification. Google's team is pushing for this on the IETF DMARC mailing list:

https://mailarchive.ietf.org/arch/msg/dmarc/PDktxOYkB28k6ukL...

We think this is a very good idea as it will allow domain owners to use DMARC but specify that they prefer DKIM to be the only mechanism that truly authenticates traffic as their own.

The industry has many workarounds for the weaknesses of SPF and DMARC, such as using SPF macros to dynamically shape authentication based on criteria one only knows at the time of SPF resolution, but none of these workarounds is better than just saying, "DKIM only for my domain, please."

kevincox• 3 years ago

I would love this. Right now I am running a service that sends email. For cost reasons I try to do this directly. It works surprisingly well even from cloud-provider IP addresses. Most providers quickly learn to trust my domain. But there are some big players that have an outright block on public cloud ranges (notably Microsoft and Apple). I end up needing to use a relay for these, but I would prefer not to give the relay a DKIM key or allow it to munge my messages.

This mostly works today, for example SES allows the origin to sign messages (although as of a few months ago I started hitting an SES bug where they modify a header field that they said they wouldn't, breaking the signature) and there are a few other providers that allow this capability as well.

However I basically still need to mark them as trusted in the SPF record otherwise the spam score goes up. This effectively allows them to spoof messages from my domain. I would love to close off this loophole.

It also isn't great with sending from public cloud VM instances either since I have to update the SPF record with the changing IPs and DNS caching can cause false fails (if a new instance sends mail before the cache is refreshed) and false passes (if the old IP gets reused for a different customer before the cache expires).

Yeah, I know. I should just rent a public IP. But that raises costs and would require me to raise my prices. Especially at the current small scale.

bhaney4• 3 years ago

The effect that ARC headers have on spam scores is interesting. As someone who runs his own private mail server, I can seemingly improve the deliverability of my emails by just adding a set of useless ARC headers to my own emails?

movedx2• 3 years ago

I want to say no, purely based on the idea that if that were the case all my inboxes - spreads across several providers - would be overwhelmed with spam? Surely the (organised, educated) spammers are all over this sort of thing and would be using that to break through?

bhaney4• 3 years ago

That's what I would assume too, but one of the slides very specifically claims:

The presence of an arc=pass generally guarantees a better spam score. Confirmed with ProtonMail. Seems to be the case with Gmail and Outlook as well.

And it doesn't seem like there's anything that would prevent me from adding this to my own emails. I don't think it would turn blatant spam into non-spam, but it seems like it could help something that's already teetering on the line.

ttul15• 3 years ago

Nobody in the email world considers ARC to be a bulletproof way to bypass spam filtering at a major receiver. The DEFCON presenter was ill-equipped to make that determination.

bhaney4• 3 years ago

bulletproof way to bypass spam filtering

Yeah that's definitely not what I meant or what the presenter seems to be implying. If it helps spam scores even a little bit, that's very interesting and potentially worth implementing in private hosts that often get dinged by larger hosts just for not being well-known. It doesn't need to get anywhere near turning actual spam into non-spam.

salawat• 3 years ago

As someone whose recently facechecked getting my domains email ready, including the absolute frustration of dealing with an ISP who has decreed that "those who want to send email from their own metal shalt have a business account", shit like this makes my blood boil.

I bust my ass to be a responsible member of the Net and do my damnedest to ensure that my systems are up to snuff, up to and including ripping through/tearing apart the current SotA... and yet assholes like that not only exist, but flippantly run as an open relay. With almost half of the Net paying them to do so. Are you frigging kidding me?!

hoerzu• 3 years ago

Does the issue still exist if you register an $80 SMTP outbox and spoof the address?

Thorrez2• 3 years ago

The slides say yes:

● Anyone is still able to sign up on MCs website and for 80$ spoof all their customers via their SMTP relay. You’re fully trusting in their mitigating controls.

ttul15• 3 years ago

Thought experiment: Would an email sending service that sends email for 42% of the domains globally that send through a transactional service [1] possibly exist on today's internet if it _wasn't_ incredibly good at blocking spam?

The DEFCON presenter makes the assertion that you can set up a spamming operation for $80/mo, but has he tried?

[1] https://trends.builtwith.com/mx/transactional-email/traffic/...

iamjason892• 3 years ago

looks like this was even ack'd in May 2022: https://news.ycombinator.com/item?id=30533032

hackideiomat2• 3 years ago

From CEO, apparently:

We have extensive spam and phishing detection capabilities and can handle the abuse.

Ah yes :D

johnklos• 3 years ago

Due to laziness / ignorance / some combination of both, many domains that use Amazon can be trivially spoofed by anyone else who can send via Amazon. Last I checked, that was true of outlook.com, too. I think there was a recent article about it...

These companies are so hungry for money, they forget that their products need anything else at all after they get to the "charge customer money for service" part. Companies have been told about issues like these for ages now, yet they all act so surprised when the same ancient problem pops up, time and time again.

ahoka• 3 years ago

And if you want to setup an honest mail server as Random Joe, you will struggle to not get blocked by spam filters while these too-big-to-fail mail senders can get away with being on open relay.

azca• 3 years ago

Ken is a good friend in the industry and always has the best interest of email security at heart. This may have been an architectural oversight but they are not wrong that SPF is surely a cause for concern, as is misconfigured DNS based trust and recertifications via arc (which was supposed to solve a problem for forwarding scenarios).

The centralization of email services to a handful of providers basically has led to multihoming of millions of domains that open SPF auth to the same handful. Any integrations by them or changes to existing stack can cause issues to pop up, because delegation of sending rights isn't strictly auth controlled. The same also happens with dkim delegation to saas providers who share backend keys across other customers of theirs and if their API is open to experiment (or an account gets popped) then the customer domains are possibly at risk.

Email is hard to do right. No auth no entry should be the default. But majority of domain owners aren't very good at figuring out how to secure things, or have business/product interests that are a priority, specially when delegated and authorized to third party senders on their behalf.

baobabKoodaa3• 3 years ago

Ken is a good friend in the industry and always has the best interest of email security at heart.

Based on the information in OP we know as a fact this is false. Please stop spreading misinformation.

kijin2• 3 years ago

MailChannels main customers are web hosting providers who don’t own the domains they send emails from!

That's the lamest excuse I've heard in a while.

Web hosting providers generally don't "own" the domains they host, but they absolutely know which domains they host. Routing domains to customer accounts/directories is the whole point of web hosting!

All they need is some sort of cPanel integration to report the list of domains and tie each domain to a randomly generated key. All of this can (and should) be automated without bothering the end users in any way.

ttul15• 3 years ago

What about mailing lists? How about mail forwarding?

kijin2• 3 years ago

The hosting companies have full access to any forwarding configuration set up by their customers, too. It all goes through their control panel anyway.

knodi• 3 years ago

Well this explains all the email2text spam with spoofed emails lately.

notpushkin• 3 years ago

Is this why I'm getting bounces from emails I didn't send with broken DKIM signatures for the past couple of weeks?

(Edit: I'm not using MailChannels, though.)

LeonM3• 3 years ago

SPF is broken in many more ways than described in this presentation. I work as an email hardening / deliverability support engineer and our advice is always to focus on DKIM + DMARC, rather than SPF. You still need SPF for legacy reasons, but it should never be relied on for deliverability or anti-impersonation.

Slide 54 says that DKIM + DMARC does not help against this attack, but that is not completely true.

If (and only if) you have set up DKIM for all your delegated senders, then (and only then) can you safely enable a DMARC p=reject policy. Once you have reached that level, you can start opting out of SPF for third party senders, by using the '?' (neutral) modifier in SPF.

So this:

    v=spf1 include:relay.mailchannels.net ~all

Becomes this:

    v=spf1 ?include:relay.mailchannels.net ~all

This gives emails from MailChannels a neutral SPF stance with DMARC capable receivers, causing them to use DKIM instead. Old legacy email services should still accept neutral results as well.

Granted, it is not a perfect solution, but email will never be 100% reliable, or secure anyway.

lucb1e3• 3 years ago

What's wrong with SPF then?

Since source IP spoofing is rather tricky¹ with TCP, I've always wondered what advantage DKIM has over SPF. I've always just configured SPF to allow $myIP only: good luck sending spam in the name of my domain, you'd have to compromise my ISP or registrar first, at which point you can also get TLS certs for my domain for example.

Even for larger organisations which need to allowlist multiple sending systems, how would one spoof being one of the legitimate SPF sender in a situation where it is not also possible to spoof the DKIM records?

That is, besides allowlisting a range of IPs that anyone can publicly use (as in the submitted talk), which is just dumb.

¹ although not impossible, spoofing an entire mail exchange requires terabytes of traffic for one email of a few bytes, iff starttls is not enforced in which case it becomes impossible

toast0• 3 years ago

It's easy to have too wide of a range specified for SPF. You can't properly DKIM sign a mail unless you have a valid key (or you control DNS). As a medium/large corp, you might blanket allow mail from your whole IP space, but you're probably not pushing DKIM keys to all your hosts.

fiirhok12• 3 years ago

A lot of normal email setups, e.g. forwarding, break SPF. As log as the message isn't modified, DKIM will work in these setups.

LeonM3• 3 years ago

What's wrong with SPF then?

- SPF breaks with forwarding. This is by far the biggest problem.

You typically end up with very large IP blocks being included in your policy (for example: MS365 includes a total of half a million IPv4 addresses, and billions of IPv6 addresses). All of which are now allowed to send on behalf of your domain. You'll have to trust (or hope) that the third party properly prevents other tenants from sending email on behalf of your domain. This is the problem with MailChannels as explained in the PDF.

- SPF can be very tricky to get right, one simple mistake and you either allow way to many IPs, or break your entire email domain. Even most seasoned IT administrators don't fully understand the entire RFC with all its errata and amendments. Now imagine some less technical user trying to set up SPF.

- You can only have one SPF record. If you want to add some external service, you'll need to merge your existing SPF record with the example given by your vendor. This can be highly confusing to many.

- The SPF DNS record is placed at the domain root (apex), which as shown in other comment here can be problematic for large orgs. This also prevents using a CNAME for SPF.

- SPF limits to 10 DNS lookups, which has proven far to low for modern use. The rules on how to calculate the number of required lookups is also ambiguous, so some receivers may accept your email, some may reject based on lookup limits.

- SPF contains many rarely used features (such as macros) that may or may not be implemented by the receiver, so you can't really rely on it.

- IP spoofing is an issue, but as you noted, this is not always a practical attack.

DKIM is in any regard more robust, more secure, and more important: easier to set up for the domain administrator. Just copy/paste the DKIM record provided by the third party email service. The third party vendor can (and most do) easily verify that the DKIM record is set up correctly. This is about as easy as it gets for DNS.

Even for larger organisations which need to allowlist multiple sending systems, how would one spoof being one of the legitimate SPF sender in a situation where it is not also possible to spoof the DKIM records?

That is the problem with the (current) DMARC specification, you need SPF or DKIM to pass DMARC. Emphasis on or. So no matter how well you manage your DKIM keys, if someone can spoof (or has access to) one IP in your SPF policy, then the email is delivered. This is also explained in the talk.

Also, with DKIM you don't 'spoof' the record, you'd have to MITM the receiver when they fetch the DKIM DNS record, this is very hard to pull of in practice.

remram• 3 years ago

Since about September 1st, email forwarding from my Gmail account to another Gmail account fails with a DMARC-related bounce depending on the original sender's configuration. Email security has become too hard for email providers to get right.

donutshop• 3 years ago

But... What about BIMI

tamimio• 3 years ago

I think this is primarily a problem of the domain owners/sysadmins, all my domains have SPF, DKIM, and DMARC and I also keep them monitored for any blacklist/alerts, if you are not aware of these, you should hire someone to audit it, email protocol is broken by design and all of these are duct tape solutions to prevent OP attempts. This whole document gives me nostalgia back in the late 90s early 2000s how there were tools you install to spoof emails!

XCSme• 3 years ago

Can anyone confirm whether that this short article I wrote a while ago is "good enough" for avoiding spoofing?

https://www.uxwizz.com/blog/stop-others-use-your-domain-emai...

SpamChannel: Spoofing emails from 2M domains and virtually becoming Satan [pdf]

Related Stories