Mining my mailbox for top email service providers

https://obem.be/2020/02/18/mining-my-mailbox-for-top-email-service-providers.html

by kehers • 6 years ago

154 106 6 years ago

bfishadow• 6 years ago

That’s really nice analysis. Mailchimp is the biggest one as I guessed. Don’t know if you count the spam folder since some marketing emails could be automatically sent there.

frereubu• 6 years ago

Not quite sure how you got that from the graphs. Top three:

Sendgrid: 30.2%

SES: 16.8%

Mailchimp: 16.8%

dane-pgp• 6 years ago

Seeing Amazon's SES with such a large share is no surprise, but it is a little disheartening knowing about this limitation:

"Amazon SES only supports open tracking over HTTP domains."

https://docs.aws.amazon.com/ses/latest/DeveloperGuide/config...

Yes, big webmail providers fetch the tracking pixels for you and serve them to your browser over HTTPS, but not every mail provider does that, sometimes for good reason, leading to problems like this:

https://protonmail.com/support/knowledge-base/connection-not...

amirathi• 6 years ago

Deliverability is a big marketing/selling point for all these providers & important selection criteria for their customers. So it'd be very interesting to see what % of emails for each provider landed in Inbox vs Promotions/Spam/Junk folder.

michaelbuckbee• 6 years ago

As someone currently dealing with this on multiple fronts: it's complicated. ESP's block out different ranges for different customers, dedicated vs shared IPs, etc. really hard to generalize.

npsomaratna2• 6 years ago

Shameless plug: we had the same issue with our deliverability rates. We ended up creating a tool to automate the process of checking spam % vs. inbox % for various email subject/body combinations: https://vetter.monsoonyeti.com/

oefrha• 6 years ago

So it'd be very interesting to see what % of emails for each provider landed in Inbox vs Promotions/Spam/Junk folder.

Are those percentages really a good measure of deliverability? I mean, a significant portion of “one million dollars for you” scams in my Gmail spam folder are from @gmail.com addresses; that doesn’t say anything about gmail.com to gmail.com deliverability.

thaumaturgy• 6 years ago

Gmail is somewhat a special case because email services are kinda forced to accept inbound Gmail, even though it's quite a large source of scams and phishing attempts today. But for others, they'll still readily get blacklisted for a little while here and there.

I was thinking about spam as I read the post too, since this is a problem I get to deal with almost daily. Marketo has a terrible signal-to-noise ratio and it can often end up blackholed on my network with nary a user noticing. Sendgrid is my next least-favorite email-as-a-service, and in fact I happened to have this article open in one tab and in another tab I had this morning's automated reports from some of my spam software showing that a pile of inbound messages from Sendgrid had tried to land at a list of nonexistent mailboxes -- someone was rolling through "andy@", "barbara@", "vijay@", "kenneth@", etc.

I have had very little trouble with Mailgun in this regard, so far, but that may only be because the spammers haven't fully adopted them yet.

As a sysadmin on the receiving side, I really loathe these services, they make my job significantly harder and more time consuming than it used to be.

kakwa_2• 6 years ago

Well, it's the first barrier.

If you mess-up really bad you will have bad deliverability across the board.

The mistakes can be numerous:

* bad setup like broken DKIM

* bad sending parameters like opening too many connections by IP to a given provider

* bad content format like too many large images

* bad actual content

* bad recipient list management with lists containing lots of typos, none existent mailboxes, spam traps, etc

* miss-managing unsubscribes

* bad bounce handling

If this happens, you can have some if not all of your IPs blacklisted, maybe even your domain. In worst cases, a whole IPv4 range is blacklisted.

Then a second aspect to take into account is the open rate and the click through rate (people actually clicking a link in a marketing email), this one can vary depending on the company activities, and also the quality of their marketing content, but is generally in the range of a few percents at most. And then, you also have to take into account the actual impact in term of sells which will be even lower.

Deliverability is import, it's not rocket science, but it's a lot of small items various people at various stages (tech, creatives, marketers, etc) must be cautious about which makes it actually quite hard.

bognition• 6 years ago

Very interesting thanks for sharing. initially I was surprised that HubSpot didn’t make the list, until I realized that there is a nice intersection here with what you observed and the target businesses of the Email services.

For example MailChimp Targets small B2C companies while Hubspot targets small B2B business.

I’m guessing the op isn’t a small B2B business hence the lack of hubspot in the list.

basch• 6 years ago

Same with Pardot missing.

kehers4• 6 years ago

Yes, for the mailbox I ran it on. I'll like to see someone do this on a "business" mailbox

TacticalTable• 6 years ago

It's interesting that this is missing a lot of major enterprise marketing email providers.

The author clearly isn't subscribing to things like Best Buy's weekly deal email, or Target cart reminders.

runlevel12• 6 years ago

A few years back, one of our (Twilio SendGrid's) developer evangelists created an open source Chrome extension to show the icon of the ESP who sent a message in Gmail.

The last big Gmail UI update broke it, but most of the ESP identification strategies probably still work: https://github.com/nquinlan/Email-Intelligence/blob/master/c...

kehers4• 6 years ago

This is nice. Shows a different approach to getting the ESP from the mail header.

pingec• 6 years ago

This is a brilliant idea, if only it were updated for the latest gmail UI

paulmendoza• 6 years ago

OMG, this sounds like a massive security risk. That nice developer could easily siphon off all the email to his backend server with this little extension. I install almost no extensions because they are too risky.

ragebol• 6 years ago

The author clearly isn't subscribing to things like Best Buy's weekly deal email, or Target cart reminders.

I don't think that will be interesting for the author given where he's based. (I first assumed Belgium due to the domain's TLD, but it's actually Nigeria)

dubcanada• 6 years ago

It's missing basically every major email marketing provider.

* Salesforce * IBM * Contact Contact * Campaign Monitor * HubSpot * ActiveCampaign

etc etc

kakwa_2• 6 years ago

It can be quite hard to detect. A lot of this services will not have an easily recognizable marker.

It can be for various reasons:

* They can use these services themselves

* They can support full customization of domains, including for the "message-id" and the "received" headers.

I'm working as SRE in a fairly large marketing provider (not sure in term of %, but we provide services for quite a few large brands sending several millions emails per day), and we do this full customization by default. There are other way to recognize us (other headers in the email or the pattern for the tracked url for example), but the one described in the article will not work.

It would also be interesting to be able to detect which MTA is used by these service providers. Is it an in-house one or an off the shelf one? The two off the shelf ones I know for mass sending are Momentum and PowerMTA which are both owned by Sparkpost but they might be others.

tjoff2• 6 years ago

The author clearly isn't subscribing to things like Best Buy's weekly deal email, or Target cart reminders.

Who on earth, that knows how to unsubscribe, would ever subscribe to spam?

Marsymars• 6 years ago

The author clearly isn't subscribing to things like Best Buy's weekly deal email, or Target cart reminders.

I'm subscribed to some marketing emails, but I delete them after reading them. I save transactional emails.

thih92• 6 years ago

Nitpicking: in the final chart with merged providers, the 'mandrillapp.com' section and 'mailchimp' section are still shown as separate.

Is this intentional? I assumed the opposite, given that:

Mandrillapp.com is the transactional email service for Mailchimp. It used to be a standalone service until it became deeply integrated into Mailchimp.

kehers4• 6 years ago

Yes it's intentionally as Mandrill is still used solely for transactional emails and Mailchimp for marketing. Same reason I didn't merge Mailgun and Mailjet.

thih92• 6 years ago

Thanks for the explanation and for the article. That was a fun read!

simias• 6 years ago

I must admit that I was a bit shocked when I realized that the author had used javascript for this task. At first I assumed that he was scripting some webmail interface but then I saw that he was using 'emailjs-imap-client'.

I understand that people use what they're familiar with and there seems to be a vibrant ecosystem for JS outside of the browser but it'll never not be weird seeing it take over what would once have been written in Perl or Python, especially since for a long time all I heard about Javascript was "well yeah it sucks but it's not like I have a choice if I want to script in the browser".

Software is eating the world, and the web is eating software.

volgar1x2• 6 years ago

who cares, it's just a 100-line long script that probably won't be maintained or used anymore

jaynetics• 6 years ago

It would be more readable in almost any other widespread language, and I'm saying that as someone who does a lot of JS.

kerkeslager6• 6 years ago

I'm not sure I could count the number of 100-line long throwaway scripts that I wrote that are still being used in production systems before I realized how dangerous that idea is and stopped treating any code as throwaway code. If I show my code to another human, there's a chance it will end up in production.

The Pragmatic Programmer says to throw away prototypes, but in practice it always seems easier to click the merge button than to rewrite working code.

andrewzah• 6 years ago

Temporary code is permanent code in many, many cases.

quonn• 6 years ago

Modern JavaScript/TypeScript is fine. Especially when compared to Perl, of all languages.

AnIdiotOnTheNet2• 6 years ago

I disagree. JavaScript is one of those languages who's default failure mode seems to be to pretend everything worked and just produce nonsensical results.

lonelappde2• 6 years ago

Modern Perl is fine too

cat199• 6 years ago

unless you want any sort of 'system' interfacing like process execution control, etc.

sashavingardt2• 6 years ago

As as a hardcore Perler and JS eng of 20 years - yes, Perl or Python would've been the best tool for the job. Perl in particular due to its regex readability.

clSTophEjUdRanu• 6 years ago

Isn't JavaScript much faster than Python these days?

speedgoose2• 6 years ago

Was Python faster than javascript a long time ago? I'm not super old and javascript has always been way faster in my mind, but it's fine. Python has many great libraries for doing heavy computations very quickly.

speedgoose2• 6 years ago

Javascript is quite neat today. I play a lot with map/reduce and async/await and it's great.

isatty• 6 years ago

It always annoys me when some general purpose utility or script is written in JavaScript because I refuse to install node on my computer.

4cao• 6 years ago

What would have been a shell script in the 1980s, a Perl script in the 1990s, a Python script in the 2000s, is now a JS bundled with an outdated Chromium build aka an Electron app.

pedrogpimenta• 6 years ago

I'm curious, why do you refuse to install node on your computer?

JosephRedfern• 6 years ago

I started (but quickly got bored of) writing a Python version here: https://gist.github.com/JosephRedfern/79314fdf1875166e084897... if anyone wants to pick it up.

dredmorbius• 6 years ago

This website presents as a 23-chacter-wide skinny column in mobile vew.

airstrike• 6 years ago

You'd think someone who picks JavaScript as their go-to language would pick a better layout but alas...

_Anima_• 6 years ago

On my mobile/brave browser, it was even less than that. Opened on issue on his Github :-)

lettergram• 6 years ago

There’s some really interesting things you can pull out of your email. Having done some analysis myself it’s actually fairly easy to export data from Gmail to do an analysis.

This is from a few years ago, but I bet it’ll still do the trick:

https://austingwalters.com/analyzing-email-data/

Everything is in python:

https://github.com/lettergram/Email_Analysis

Regardless, I encourage everyone to do an analysis of their emails. Cut out the junk, set reminders on when to check, etc. it’s both enlightening and potentially horrifying to discover trends.

saagarjha• 6 years ago

Semi-related: emails can often give away more information than you might intend. At one of my previous employers, it was quite easy to get a general idea of how internal development was progressing “from the outside” by keeping track of who was testing which version of a prerelease OS via proxy of the information it would leave in the emails they sent.

technion• 6 years ago

I've identified companies as connected to each other by enumerating domains on an azure tenancy.

hellofunk• 6 years ago

Wow those pie charts are almost impossible to read! I've never seen such a small font in a chart that readers are expected to understand easily.

hombre_fatal• 6 years ago

[Nitpick] Instead of this:

    ;(async function () {
      try {
        // All your code
      } catch (e) {
        console.log(e)
      }
    })()

Just do this:

    ;(async function () {
      // All your code
    })().catch(console.error)

Overuse of try/catch is probably the #1 mistake I see when people use async/await.

throwanem2• 6 years ago

The point of async/await is to enable asynchronous code to participate in the same patterns synchronous code does, instead of being coerced into a promise-style interface. Using try/catch is part of that, and it's good practice here. If you're not going to take advantage of it, why bother with async/await at all, especially when using it at top level requires a workaround anyway?

mrweasel2• 6 years ago

Either way, I find the syntax of both to be extremely ugly. Arguably this may simply be me not getting JavaScript.

Starting a line with ; is just plain weird and the last bit })() seems like German, where all the stuff that you technically not need to understand the sentence is at the end.

Maybe someone can explain with JavaScript developers love anonymous functions so much. Why not just name your function and make the whole thing a bit more readable.

akamaozu2• 6 years ago

Not all JavaScript devs.

Love the language, hate the popularity of anonymous functions and non-descriptive variable names.

My production code as a sample. http://designbymobi.us/wp-content/uploads/2020/02/rabbitmq-c...

tln• 6 years ago

I don't think you can assume console.error is bound. Also it's good to avoid the possibility of blindly passing through unintended extra arguments (not an issue with catch, but is with many methods such as forEach etc).

    catch(err => console.error(err))

ksahin2• 6 years ago

I know the results are biased given it's a sample from OP inbox but I'm surprised not so see https://postmarkapp.com/ here. I thought they were pretty popular.

opan• 6 years ago

I've never heard of them before this comment. Maybe they're popular only in certain regions.

jamestomasino• 6 years ago

They're just for transactional emails, not for marketing ones. They wouldn't show very much based on how he's measuring.

ksahin2• 6 years ago

It seems this post is about transactional emails, not marketing ones. CF the first paragraph!

wjossey• 6 years ago

I had actually just tweeted about my woes dealing with email going to spam for no good reason from my GSuite account.

https://twitter.com/dustywes/status/1228733404826333185?s=21

I’m planning on doing a more thorough write-up on my solution (it’s convoluted), but my tl;dr is that email seems woefully broken at the moment. Hope my experience can be helpful for others dealing with the same issue.

londons_explore• 6 years ago

I'm wondering the legality of this...

The emails in your mailbox aren't yours to do with as you please - they're all still owned by their original sender, who has given you an implicit license to read them, and store them for later re-reading.

You don't really have rights to datamine, aggregate, train AI on, post stats about, etc. the data in those emails. Its a violation of the rights of the original sender to do it without permission.

layoutIfNeeded3• 6 years ago

So if I send an email to a gmail address, it is illegal for Google to datamine it? Even though I’d love this to be true, somehow I doubt it really is the case.

scambier• 6 years ago

Are you serious? Unless those emails contain work protected by copyright, they're not "owned" by their senders. And I doubt that metadata falls in this category.

icedchai• 6 years ago

I disagree. They relinquished all rights when they hit send.

roseway4• 6 years ago

The methodology used excludes a number of services using SparkPost. For e.g. the excerpt below of an Intercom header. It's a challenging problem to solve given the complexity of email delivery and appreciate this was a fun project to take on.

Return-Path: <bounces+XXXX@XXXX.intercom-mail.com> Received: from mta-216-35.sparkpostmail.com (mta-216-35.sparkpostmail.com. [147.253.216.35]) by mx.google.com with ESMTPS id t13si7000210pgg.534.2020.02.03.08.37.30 for <XXXXXX> (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 03 Feb 2020 08:37:30 -0800 (PST)

Mining my mailbox for top email service providers

Related Stories