Mailing lists are resistant to censorship

https://sourcehut.org/blog/2020-10-29-how-mailing-lists-prevent-censorship/

by ddevault • 6 years ago

205 98 6 years ago

kome• 6 years ago

So true.

A lot of activism in my city happens on facebook, and it's simply terrible.

Perhaps emails are a bit more difficult to use, but it is important skill to master, especially in context where you have to self organize and discuss. It's also very democratic, because opening an email account is faster.

And in terms of "real" privacy, emails are better, just because they are decentralized.

sjmulder2• 6 years ago

Re. activism, this is a talk on how Extinction Rebellion have set up their infrastructure:

https://www.youtube.com/watch?v=I_O3zj3p52A (technical talk starts around 22:30)

It isn't quite decentralised but fairly resilient. The more mature local chapters can replicate much of this if need be and in any case the use of technology is quite fluid.

blfr• 6 years ago

Extinction Rebellion enjoys either outright support or warm neutrality from most governments and corporations. Their resilience to censorship won't be really tested.

This is not the situation youtube-dl will find itself in now.

asciident• 6 years ago

I think this is what tech people don't get about non-techies using technology. Tech people talk about edge cases, about past stories where Facebook shut down a group or posts, theoretical guarantees about "real" privacy, and decentralized control being more idealistic.

Real people don't need stuff to work perfectly, because they'll come up with a workaround by moving platforms or changing how they do things. Nothing works perfectly for them anyways. The platforms they use just need to work 95% of the time, and be good enough. Real people are aware that what they write can be read by others, and don't care about the difference between whether it's encrypted in transit, or on the server, or to their recipient.

For real use, it's all about the content, not the platform.

fredthomsen• 6 years ago

Agree with this and this is why everytime I see people suggesting some project should be moved from being hosted on a mailing list to github, I roll my eyes

hugh4life• 6 years ago

Well, if you're worried about censorship(and lets be real, 99.9% of what people use mailing lists for won't be affected), don't use Mailchimp.

"We’ve updated our language to further clarify our Rules (Section 17 of STOU), which state that Mailchimp does not allow the distribution of Content that is, in our sole discretion, materially false, inaccurate, or misleading in a way that could deceive or confuse others about important events, topics, or circumstances."

iongoatb• 6 years ago

Thanks for letting me know that I should move all of my clients off of Mailchimp. I don't trust Mailchimp's "sole discretion" and interpretation of what is "inaccurate or misleading". What is true and factual is constantly in flux and subject to intepretation and debate. For example, an intelligent Democrat or Republican person, in many cases, have completely different views of what is "true" and "factual". I could apply this to many non-political examples as well.

I'm going to look into moving to one of the many of the cheaper, more trustworthy competitors if what you have commented is true.

jasode• 6 years ago

Ok, let's go beyond general principles about mailing lists and be more concrete in thinking and see if a truly censorship proof dev & distribute architecture is possible.

Let's work backwards from this desired command line:

  youtube-dl -U

That command automatically downloads latest version. (Important because Youtube, Vimeo, etc changes constantly and breaks old youtube-dl versions.)

We have 2 main areas to implement:

1) the code artifacts: a source code repo (and binary releases for download) ; for Linux kernel, I think this is "www.kernel.org"[1]

2) the workflow: the issues / bug reports / pull requests ; for Linux kernel collaboration, a popular mailing list is "majordomo@vger.kernel.org"[2]

If youtube-dl switches from github to mailing lists, how would the "-U" command work? Would somebody need to setup a SMTP server (or at minimum, a SMTP email account) dedicated to youtube-dl? Would youtube-dl then use SMTP protocol to scan the mailing list for which server has the latest code artifacts?

Does making youtube-dl act as a pseudo email-client make it RIAA proof? Are SMTP servers beyond the legal reach of RIAA?

EDIT to reply: This article is about the developer workflow, not about what the end-user software should do.

I understand that but I think it's productive to go beyond that and think out loud what a realistic implementation could be to help end users. Besides, many end users are also the code contributors.

[1] https://www.kernel.org/

[2] http://vger.kernel.org/vger-lists.html#linux-kernel

ddevault15• 6 years ago

This article is about the developer workflow, not about what the end-user software should do.

Out_of_Characte2• 6 years ago

Developing free from censorship is useless if you can't ever reach the user due to censorship

mr_toad• 6 years ago

You can use git as an ‘end user’. You just pull or clone without ever contributing.

ocdtrekkie2• 6 years ago

youtube-dl -U would be challenging without a download server, but, the issue here is collaboration. Many people have copies of the youtube-dl repo, and many people can build the youtube-dl binary if provided the up-to-date changes. But to develop youtube-dl, they need to be able to continue to collaborate, which is currently at risk due to the DMCA takedown.

If they can continue to share patches over distributed mediums like a mailing list, they can continue to collaborate, and share improvements to the youtube-dl code that many already have copies of.

Possibly not a fantastic long-term solution, but certainly worth attempting whilst fighting it out with the RIAA.

Another important point is that many projects use GitHub as their general communication about issues, support, roadmap, etc. When GitHub suspends a repo, all of it goes away. Whereas Sourcehut is selling the point that your collaboration tools would survive even if they were required to remove your code.

scottlocklin• 6 years ago

Just a reminder: git was developed as, and can be used as a decentralized source control system. You don't need github at all. Github came 3 years after git was invented.

pfortuny2• 6 years ago

I think that your concern about the SMTP server is the serious issue. In the end, (for the RIAA) it should be indifferent whether the source code can be accessed using git/http/smtp, whatever protocol.

So, in any case, it looks like the idea is not good to fight that issue (right now no one prevents anyone from sending his copy of the master of youtube-dl and letting other people reproduce it/develop it). The use of a mailing list is just an improvement over this but it should be perfectly possible (for the RIAA) to take it down as well.

toomuchtodo• 6 years ago

The RIAA can't reach into everyone's mailbox to get the code (patches, etc) out. If everyone's commits are signed, you can rebuild the repo at anytime from anyone's mailbox contents. It's SMTP/IMAP/JMAP DHT/Bittorrent. Automating git pulls (cron or similar) accomplishes that same thing, but with more effort on everyone's part.

LinuxBender• 6 years ago

Email is usable for code commits. Some people update git over smtp today. For artifacts, I envision Usenet or distributed rsync on Tor.

    youtube-dl -U

    Fetching signed update from usenet over TLS...
    Validating GPG signiture...
    Installing...
    Updated to youtube-dl-20201029

People could subscribe to artifact updates over email, but the client won't know how to get it. You would need a plugin that allows mapping to a local mail dir or a plugin that has an imap library to fetch updates and validate GPG signature. Or maybe use a plugin that talks to Tor relays? This is all hypothetical of course. I am not factoring in legal consideration and risk to the developers.

pokoleo2• 6 years ago

You could:

* use the blockchain to distribute a magnet link for the latest revision, and

* use the torrent protocol to download the latest revision

centimeter• 6 years ago

This is one of the few legitimate uses for blockchain tech (besides money) that isn’t just an ICO scam. Bitcoin solved zooko’s triangle! We should make use of that.

strogonoff• 6 years ago

Off the top of my head, an auto-updater in a legally hostile environment could work through a hard-coded a network of mirrors updated in every new version, torrent protocol, etc. It’s a feature, even if complex it can be addressed with development effort, and it only really becomes impossible if development is not happening.

sofetch• 6 years ago

git-ssb has interesting properties.

https://github.com/noffle/git-ssb-intro

0df8dkdf• 6 years ago

git over i2p seems a good option as well to go outside RIAA proof.

Santosh83• 6 years ago

Why hasn't the community tried a GitLab (or equivalent) over Tor? This ought to enable both collaboration with social features as well as anonymity from outside entities.

LockAndLol• 6 years ago

That exists already https://i2pgit.org/

I guess most people don't use it yet. The problem with Gitlab is also that it doesn't federate. All the project metadata stays on Gitlab. One can import from Github, but then you're stuck on Gitlab.

Maybe tools like fossil[0] or git-bug[1] will pick up steam, but I have a feeling the current events will simply pass and next week we'll be talking about something different. In the meantime, some people will switch or consider alternatives and everytime this happens again, the number of those people will grow. Progress is slow.

[0] https://fossil-scm.org/

[1] https://github.com/MichaelMure/git-bug

dublinben• 6 years ago

When this was discussed earlier, someone shared a link to a repository hosted on Darktea, an instance of Gitea provided as an onion service.[0]

[0] http://it7otdanqu7ktntxzm427cba6i53w6wlanlh23v5i3siqmos47pzh...

marketingPro• 6 years ago

I disagree a bit here. Spam folders has turned big tech into gatekeepers.

I run a data driven self help website and there is no paywall or advertising on the website. Everyone clearly signs up for emails. I still get above 0.5% spam+bounce ratio which means I need to ask Amazon for forgiveness everytime I email. (4x per year)

Yes I purge the lists, don't use clickbait, etc...

If 0.5% can get you into spam folders, this is an easy attack vector to silence someone.

My bigger question is how Walmart, Joann Fabrics, and HBO get into my mailbox. I'm sure they get "marked as spam" by more than 0.5%

sumtechguy• 6 years ago

I keep telling the spam filters to let some stuff in and they still bounce it in there. Though to be fair they get most of the stuff. About 2 years ago I went from maybe 100 or so a month to well over 1500 a week now. Got on some list...

ddevault15• 6 years ago

There's a BIG difference between "data driven self-help" emails from a self-proclaimed "marketing pro" and on-topic development discussion and patchsets.

syntheticnature2• 6 years ago

Yes, based on my experience with discussion email lists, the data-driven emails have a better chance of getting through.

Meanwhile I'm grepping through mailing list server logs and going "your provider accepted it without error, don't know why it's not hitting your inbox."

forgotmypw17• 6 years ago

Back over a decade ago, I managed to get Yahoo! interested in me as a potential employee...

I would have never known if I hadn,t looked through my Spam on Gmail.

I didn,t get the job, but I got a nice aep to San Jose and back.

It was from a yahoo-inc.com address, probably not used for spam much.

Since then, never trusted it... I comb through all my spam every couple of days. Takes only couple of minutes to skim it 20 at a time.

Would recommend, technique has paid off.

im3w1l• 6 years ago

Some emails don't even get to the spam folder. Well idk if gmail works like that, but I know some systems do.

marcinzm2• 6 years ago

The article I feel confuses collaboration between an existing group of people and open public collaboration. The former can setup private repos on github or any other of fifty approaches. The later however just means you have a different set of single points of failure (mailing list, mailing list archives, domain for mailing list, SMTP server, etc.). Mailing lists may allow the latter to fall back to the former a bit more easily but it's far from a clean transition. So they're at best a very partial band aid for the underlying issues so provide little value versus building an actual solution (ie: over Tor, etc.).

dkdk8283• 6 years ago

Isn’t the point that if you’re subscribed to a list and receive a message the mail list has no technical ability to recall the message?

marcinzm2• 6 years ago

But anyone new who is trying to join won't have the messages unless you rebuild a new public archive/website/etc. At which point you can also presumably do daily backups of gitlab data to do basically the same thing.

protomyth• 6 years ago

Well, until Google decides your e-mail is spam or harmful.

option2• 6 years ago

once Gmail moved to spam our official job offer letter to intern. They missed offer acceptance deadline because of that. We extended the deadline but then COVID hit

gpanders2• 6 years ago

This actually happened to me when I was an intern (many years ago). My official job offer went to spam and I didn't see it until weeks later. Fortunately I happened to check the spam folder and saw it. I now make it a habit to regularly check the spam folder in Gmail and I often find non-junk mail in there.

zeusflight2• 6 years ago

Some universities recommend avoiding gmail accounts for application process precisely for this reason (I can't find the reference). My personal experience is that open source spam filters (spamassasin/rspamd) are far better at classifying spam than gmail's filter.

However, there is another reason why no one should use gmail for anything serious. There is always a risk that you may get shut out of your account without recourse, taking down access to other services along with it.

imtringued• 6 years ago

Well, there is the obvious problem of obtaining a copy of the archive. There is also a usability problem. I despise email archives.

Theoretical responding author:
Yes, but there is a good reason for that.

There is an example linked [0] in the first blog post and I honestly don't understand how to read it. A lot of information is being drowned out by the patch file. The diff is in a separate tab on Github. The response hierarchy is inverted.

Theoretical responding author:
You'll get used to it in no time.

The idea of responses being a quote inside the text they are responding to is absurd. I honestly can't comprehend it. Usually people quote the text they are responding to, not the opposite. Why do things have to be so complicated? Github doesn't even have a hierarchy. It's just an ordered list of messages.

[0] https://lists.sr.ht/~philmd/qemu/patches/5556

ibn_khaldun2• 6 years ago

I lack the experience or acumen to give educated input on the topic at hand. But I am an advocate for any practices that are simple and resistant to Big Colorful Corporations and their like.

The problem that I see with society in general is that many of us have a strange attachment to these services (GMail, Facebook, Github, Youtube, Evernote, whatever) to the extent of outright dependency and diminished wherewithal to try something different, even if it may be to our advantage in the long run.

I sympathize with people, such as Mr. DeVault to an extent, who are determined to invite people to alternatives that are not entirely foreign. In the case of his well documented campaign to re-emphasize mailing lists with git, it appears to be a return to its original format. It is also worth mentioning his efforts in educating people how to use this traditional method.

If a person has no intention of ever publishing something that threatens the status quos and bylaws of Big Colorful Corporations and and their pals within governments and elsewhere, then what difference does it make?

Be it on a git server, a website, or a major publication, whatever it may be, as society begins to slither towards regression and oppression on various fronts, the most skilled people, will be those of the most sound moral character and belief who have the acumen to navigate a world that is actively plotting against them.

These sort of efforts are not necessarily irrelevant to a greater cause, when put in the proper perspective.

darau1• 6 years ago

A few questions I have:

- what happens after you send the email?

- Is there somewhere I can pull that up on sourcehub?

- Does feedback come to the email attached to my git user?

- if the maintainer decides to apply my patch, does it still show up as my patch, or theirs?

ddevault15• 6 years ago

- It gets, uh, sent. If you send it to a mailing list, the mailing list will forward it to the subscribers.

- Yes, there are online archives available for each mailing list.

- Yes.

- Both. It shows up with you as the author and them as the committer.

zzo38computer• 6 years ago

Another idea is to use NNTP with email subscriptions (do any NNTP server software have email subscriptions?) and propagations to other NNTP servers as well as SMTP. It is possible to cancel a article, but some servers (and clients) may ignore the cancel. You can still distribute the messages in many other ways too (does not have to be limited to internet), including but not limited to email and NNTP. Maybe also IPFS.

Another consideration is Fossil. Fossil artifacts are identified by a SHA1 hash or SHA3-256 hash, so you can make a magnet URI involving it. SHA1 hashes are listed in the Wikipedia article for magnet URIs; SHA3-256 isn't, although it would still be possible to do (perhaps "urn:sha3-256"). Fossil also includes a HTTP server (or can run on an existing HTTP(S) server), which the artifacts can be downloaded from. Some Fossil artifacts are structural artifacts; if it is a manifest of a check-in then its data will list the name and hash of each file. (I do not know how Git and Mercurial work; I do not know about how the similar things would work with Git/Mercurial, if any.)

sedachv• 6 years ago

do any NNTP server software have email subscriptions?

You do not really need to build that into an NNTP server; you can just have something like rss2email for NNTP running locally. There is a way to do it with CNews though: https://tldp.org/HOWTO/Usenet-News-HOWTO/x714.html

Mailman can forward list messages to newsgroups, which is something you do want to do on the listserv server side: https://docs.mailman3.org/projects/mailman/en/latest/src/mai...

Also check out DFeed: https://github.com/CyberShadow/DFeed

qwerty4561272• 6 years ago

You don't really need a mailing list, anything as obsolete as the e-mail format and protocols it relies on to achieve this. All you need is an offline-first system with mandatory redundant caching and no unattended cache flushing. Ideally it should also intend clients to distribute seen messages between each-other. It can also store each message hashes in a blockchain.

Legacy e-mail should be gone for good, with its 7-bit headers (it takes 24 bits to represent a 8-bit symbol in "quoted-printable"), codepage hell (most of the Americans can hardly understand, meanwhile e-mail software still doesn't default to UTF-8), overquoting tradition (most of the people quote all the messages of the thread below every message and the software encourages this), pointless subject field encouraged for every message (which usually remains intact when the subject changes, and often is of little informativeness from the very start), incompatible HTML formatting, etc.

Out_of_Characte2• 6 years ago

"offline-first system with mandatory redundant caching and no unattended cache flushing." Something like email "Ideally it should also intend clients to distribute seen messages between each-other." Maybe a mailing list "overquoting tradition" Like I'm doing on this UTF-8 enabled page with compatible HTML. Yes, quoting can be redundant but on email it preserves sometimes valuable context in a redundant way.

Aren't you reinventing email exept its compatibility to every device on the world? Compatibility and availability are sometimes more usefull than the tax of legacy systems.

qwerty4561272• 6 years ago

Aren't you reinventing email exept its compatibility to every device on the world?

No, in fact I'm just dreaming about the return of FidoNet, which already had none of these problems back in the 1990-ies ;-) Sure, there was no UTF-8 yet, but a standardized set of 8-bit ASCII codepages was already not bad though - you could always tell the codepage given the language/country and you didn't have to fit a symbol into 7 bits.

And this is an illustration of FidoNet-style concise quoting: quote just an atomic idea, answer it, then quote the next one:

...

This way.

Whoever needs to see the whole original message you are answering - it's up there in the thread tree, just a button away. And it is meant to be automatically pre-downloaded for you.

Meanwhile, almost every single e-mail message I get just includes the whole thread quoted under its actual body (duplication level = insane). And it looks and works very weird when people actually need to respond to particular parts of my message paragraph-by-paragraph.

matheusmoreira2• 6 years ago

Mailing lists are a great idea but I don't know how to use them. Last time I tried I ended up making some embarrassing mistakes. I tried to look up a guide but only found the manuals for the mailing list software.

Does anyone know of a good guide to mailing lists?

ddevault15• 6 years ago

I have written a guide for emailing patches:

https://git-send-email.io

And also for configuring your client for plaintext, which is strongly preferred by most mailing lists:

https://useplaintext.email

If you or anyone else reading this is ever anxious about making mistakes in their early posts on a mailing list, feel free to send the email you'd like to write to me first - sir@cmpwn.com - and I'll look it over and make any suggestions if necessary.

matheusmoreira2• 6 years ago

I've read both, they're amazing resources. Thank you for creating these sites.

What I don't understand is how the mailing list itself works. When I want to start a thread, I send an email to the list. When I want to reply to someone, do I email the list or the person? How do I make sure the emails are threaded properly? During my first mailing list interaction I ended up emailing another developer directly somehow.

The git email client seems to have support for message IDs. They seem to work just like post identifiers in forums. Mail clients like mutt let users actually edit the email's headers and include metadata like that. I'm not sure if gmail does it behind the scenes.

Also, gmail has moved on to newer authentication mechanisms. Is there a simple plain text email client with native support for this? Mutt apparently requires an external program which is seemingly unmaintained.

Jaruzel3• 6 years ago

What I'd like to see is a way to turn on plaintext email replies only for certain recipients. I know you can't do that in Outlook it's a global setting or nothing - can any other (non-cloud) email client do it?

option2• 6 years ago

that depends on who operates your email.

Twitter censors direct messages, why can’t Gmail censor who can email whom?

ibn_khaldun2• 6 years ago

Abandon Gmail.

zeusflight2• 6 years ago

Don't let anyone else control your email. Use a personal domain and offline backups (eg: mbsync) for email. That should give you enough portability to dictate your own terms for the service. This might sound like an overkill - but email IDs are too important these days to let anyone else control it.

Groxx• 6 years ago

... I'm not convinced.

You can continue an existing thread with known individuals without the mailing list, true. And that undeniably has value, and can be used to re-form connections after loss of the mailing list, and that's great. It both adds value to ongoing use of emails and adds value beyond its lifespan (compared to not ever existing).

But the list is how many people contribute. The list is how general discussions occur. The list is how new contributors join and contribute, as they lack other connections. Without the list, which can be censored just like github, a massive connection-increasing piece of the community is still gone. I feel pretty confident arguing that that is the feature that separates git from github, and email from mailing lists, and it's no more resistant to censorship than any other centralized system.

ddevault15• 6 years ago

A good practice when firing off an email to a list like this is to do a git blame on nearby code, identify the maintainers, and come up with a list of people to Cc directly on the email.

If you can get the git repo, you don't need the list to contribute. It helps, for sure, but you can work without it pretty easily.

And as I said in the article, I don't think it would be necessary to turn off the list in the event of a DMCA request, at least not on SourceHut. On GitHub the git repo is the single source of truth, so when a git repo is pulled, so too go the issues, pull requests, and so on. Not so on SourceHut.

zaarn5• 6 years ago

Why would these things remain on sourcehut if they get a DMCA filing? If you do not take the offending material offline, you open yourself to a lawsuit and if you do, new contributors have no way of accessing this data.

Open issues and similar will likely also get lost unless restores, unless you count "exists in someone's inbox/archive" as "the issues and pull requests continue to exist". Because in my opinion, that's barely better than having no backup as all relational data in regards to these things is lost.

pabs3• 6 years ago

I'm reminded of this article "Free Software Needs Free Tools":

https://mako.cc/writing/hill-free_tools.html

tchock23• 6 years ago

Are they also resistant to propaganda?

aminozuur2• 6 years ago

Resistance to prograganda comes best from your brain's natural firewall (scepticism and critical thinking).

zaarn5• 6 years ago

That's not a firewall, the brain just denies outside information in conflict with internal biases unless it's formed in a way that changes internal biases subtly, which will very quietly switch your bias. There is no firewall in the brain, not even scepticism and critical thinking.

aminozuur2• 6 years ago

MailChimp banned Stefan Molyneux.

Mailing lists are resistant to censorship

Related Stories