Streams: Mail 3.0 concept

by scapbi • 11 years ago

109 60 11 years ago

AlexeyMK• 11 years ago

Neat concept. One challenge is imagining the transition from email to stream - how can I start using streams internally with my company gradually, but still use email when needed? Not another inbox...

tinco3• 11 years ago

I think you'd configure your client to pull e-mail as a stream or a collection of streams (with filters). You can then in your client fuse or split the streams into the rest of your streams. It could be a pretty fluent transition in my imagination.

tinco3• 11 years ago

Yes, this could work. I like the idea of streams being an url with a path that might be a unique hash for you. The url's domain controller decides who is authorized to post on their streams. You decide which domains to trust.

Most social services we use today could be streams. Facebook, Twitter, even WhatApp, Instagram (I guess not snapchat). If they'd all implement some common stream spec. Unfortunately they won't, Twitter has demonstrated that they don't like alternative client implementations.

If it could replace e-mail is another big question. Nikita hilights the amount of cruft that's around e-mails, but the cruft is social protocol, and we humans love social protocol. Or perhaps that's just old fashion speaking.

The posting is quite old already (Oct 2014), so this isn't some big hype building and it doesn't look like anyone is working on a big implementation.

rakoo3• 11 years ago

When you read the post, you know that XMPP has all the technical parts to fill the role here. There are some people who build things on top of XMPP, but I haven't seen something as advanced as buddycloud (http://buddycloud.com/) here. They have a concept of "channels" (https://xmpp.org/extensions/inbox/buddycloud-channels.html) which can have multiple types with multiple participants, each having a definite role. All of the use cases in the article are (theoretically) possible with buddycloud.

Technically, the standard XMPP PubSub protocol with its PEP companion could get you very far already, unfortunately there's not many clients that handle them today.

anonbanker2• 11 years ago

jitsi does. who uses anything else for xmpp nowadays?

cromwellian• 11 years ago

To me what is important about email as infrastructure is SMTP and the principles of delay tolerant networking and federation. SMTPs store and forward reliability model is a bedrock of communication on the Internet, when all else fails email won't or it'll die trying.

Anything vying to replace it needs to capture these features. I won't trade reliability for real time. There are other real time channels I can use.

tinco3• 11 years ago

If the e-mail is simply stored persistently until it's fetched succesfully (over HTTP), isn't that delay resistant networking? Are persisting proxies really important?

zx2c4• 11 years ago

Check out DJB's smtp replacement, InternetMail2000:

http://cr.yp.to/im2000.html

mondoshawan• 11 years ago

That seems less a replacement and more notes on a random idea with lots of questions. Some of his ideas in there are questionable, too.

Seems like he's increasing the bandwidth usage for an MX in a big way.

zouhair• 11 years ago

Isn't not being real time the main great thing about email?

frenchman99• 11 years ago

I dislike how half of this blog post is just ranting about email. For example, I don't get this: "electronic messages are not real-time, they’re hard to manage in huge amounts, they work best for person-to-person dialogs"

These are "limitations" according to the author. But IMO they're the reason I like email:

- I like to be able to reply when I want, without the sender having the expectation of everything happening in real time, I have other things to do than answering real time to everything.

- Anything is hard to manage in huge amounts, which is why we hide less important stuff. Email has folders where you can put stuff in. You can archive emails. Or maybe this management issue is technical, as in "storing petabytes of emails is a difficult task" ? In that case, I have no real answer.

- I actually like the concept of mailing lists: they work where I already am (ie my mailbox) and I can stay up to date just like in a forum. And person-to-person dialogs do indeed work very well with email. I don't see that as a limitation.

---

Also, about this: "Stream is a group of threads. Simple as that."

That does not sound simple. In fact, just from that sentence, I didn't understand at all what it meant. I had to read on to get the idea. But I do see the point of "streams", and I agree that filters are usually complex to setup and maintain.

---

The part that I don't understand is this one: "With proper user/access management Streams could replace not just email, chats and group chats but also forums, blogs, mail lists, RSS and most of web 2.0 communities."

It sounds like what you need is RSS everywhere. RSS has a pull model too. You can setup ACLs on an RSS feed if you have some kind of authentication in place. It can replace mail lists, except that you'd still need to be able to push messages to it, which I don't see anything about in the blog post.

---

Finally, regarding this: "This fixes ambiguity around do-not-reply@ addresses"

I get the point of ambiguity and I have replied to no-reply addresses before myself, with an error message coming back. But I think ambiguity could be fixed with a header here, instead of throwing away the whole email infrastructure.

Imagine a "X-No-Reply: true" header. That way, email clients could disallow replying to such an email address. This has the advantage of being backwards compatible with everything that exists.

---

Overall, I do think that you can try something new, trying to "replace" email. But realistically, Google has tried several times and is struggling. Email has solid tooling and knowledge built around it, and I think trying to push for its self-improvement is worth a look.

Slack is sometimes said to be an "email-killer", but the fact is it replaces only one use of email. At my last company, we used Slack for real time stuff, notifications, quick questions, but not for everything. And that's great. Because email sucks for real time stuff right now (although initiatives like JMAP may make things better).

Email seems to carry the notion of importance, which is not true of chat messages I think. The reasoning behind this is, I think, an incoming email stands alone, whereas a chat message is quickly surrounded by other messages.

Also, emails have a clear receiver, the receiving email address. This means that if you get a message to "devs@project.org" and you are a dev, you are de-facto concerned by this email, which is a reason to read it thouroughly. This is not true of chat messages which do not have a "receiver" but are instead posted to the more liberal notion of "channel".

glesica• 11 years ago

Email seems to carry the notion of importance, which is not true of chat messages I think. The reasoning behind this is, I think, an incoming email stands alone, whereas a chat message is quickly surrounded by other messages.

This is critical. Understandably, "chat" user interfaces don't make individual messages stand out by themselves. This means that it is easy to miss a message, or even a conversation. It also means that you can't delete everything but the important stuff you need to reply to, which is trivial in email clients.

jakobegger• 11 years ago

This sounds extremely exciting. But there's one question: how do I tell someone to subscribe to a stream? Do I need to email them a link to my stream? The killer feature of email is that it allows anyone to contact anyone else.

Also, issue tracking is something that could be replaced with a stream...

zrail• 11 years ago

Sounds like Usenet re-invented, where stream == newsgroup.

cm21872• 11 years ago

To me the problems with email have more to do with spam and privacy than threads and real time.

We shouldn't have a unique address but rather give a unique random token to anyone asking for our email address which could be severed (or bound to receive from a single address, ie that counterparty can not transmit that token to anyone as my server wouldn't receive mail from another address).

We should have systematic guarantees that the sender is the mailbox it pretends it is.

Encryption in smtp should be mandatory.

We should have an easy to use content encryption (pgp style).

Ideally you would like to decorrelate the address from the service provider (lots of people are prisoners of gmail, hotmail or their ISP).

I am sure we can make the chistmas list much longer but to me these are the major issues with the current system.

HappyTypist2• 11 years ago

This is easily solved by an email service that makes creating aliases take 5 seconds, and offer a very large maximum number of aliases.

Address exhaustion is a real issue though, so you will need to acquire many email-worthy domains for this service.

cm21872• 11 years ago

Well, most of the items in my list exist in one form or another, but all are optional and/or not widely used.

As you mention we can use aliases for unique emails (which is what I do, but I had to create a software on the side of my mail server to manage them, as my mail server never intended to use aliases that way, and for instance makes the list of aliases visible to everyone on the server, and does not allow to assign a keyword to an alias).

It is possible to have guarantees on senders by using spf and dkim, but not many server actually use them and you currently can't make it a hard requirement for your incoming traffic, so pretty much useless.

Smtp encryption has taken a step back with TLS, as with STARTTLS we are now forced to use optional encryption that can be downgraded by a MITM, whereas before one could elect to only have an SSL port open.

Content encryption is possible using PGP but clunky and almost no one uses it.

And people can buy their own domains if they know it is possible and how to configure a DNS server, needless to say not many people do.

What I suggest is that we would need a protocol that does all that by default.

amelius• 11 years ago

It’s hard to believe people still use email. It has so many flaws yet still remains die-hard to replace.

Well, at least it doesn't live inside a silo, like most other messaging protocols.

FreakyT• 11 years ago

This concept seems quite reminiscent of Google Wave, if anyone remembers that:

http://www.renepedersen.com/wp-content/uploads/2010/01/googl...

aroman• 11 years ago

...the author spends a good amount of time in the article explicitly comparing and contrasting his idea to Google Wave.

goblin892• 11 years ago

To me this seems so different from email that it rather invites a comparison with RSS feeds or Slack-like services.

What if it was a no-brainer to publish to and subscribe to feeds, there was some kind of feed exchange and access control built-in, and everyone was using them to communicate? What if Slack was open and decentralized, and everyone was using it?

chimeracoder• 11 years ago

What if Slack was open and decentralized, and everyone was using it?

We tried that; it was called IRC. As far as I can tell, Slack is still little more than a hosted IRC service with its own web and mobile clients. Virtually none of the functionality they provide couldn't be replicated this way; it's more a matter of branding than anything else that they eschew the label "IRC".

Slack even provides an IRC gateway so you can access it using IRC clients. I don't know if their web and mobile clients are using the IRC protocol, but from what I can tell, they might as well be, which is the whole point.

rakoo3• 11 years ago

Virtually none of the functionality they provide couldn't be replicated this way

And as always, there is a huge gap between "X is just A+B+C" and "Here is an actual X". Slack may have not done anything on the transport side, but they did make the client better. AFAIK the only IRC client with full text search on the whole history, with support for files and images, is irccloud.com. That's not a lot. I wouldn't say IRC does everything Slack does.

troymc• 11 years ago

"was called IRC"

IRC ain't dead yet. For example, there are IRC channels for many of the projects within Mozilla:

https://wiki.mozilla.org/IRC

hobarrera• 11 years ago

What we need though, is something federated. Where everyone can interact with anybody, just like email (or like XMPP did).

IRC does not have this. I've to connect to your server, or you to mine. This doesn't scale as an email/IM replacement; it only works for rooms with a certain usage/target.

The huge problem with any replacement (aside from federation) is the network effect. No matter how good your solution is, you need critical mass for it to pick up, and no huge company is going to invest into getting users into a federated network which they won't control. There's no profit for the company.

Demiurge• 11 years ago

It’s hard to believe people still use email.

It's hard to believe people don't understand that 'shortcomings' are sometimes 'advantages'. Email is perfect for what it is, it's a textual, non-realtime medium of communication. You take the time to compose a long, coherent message using written language. Mail has existed for a long time, and hopefully it will never go away.

It's great if you want to invent something more instant, more automated, more visual, but it's not going to be an improvement on email, it will be something else that fills some niche.

HappyTypist2• 11 years ago

Absolutely. All these attempts at reinventing email into a new instant messaging service are fundamentally flawed, even if they only intend to use email as bootstrapping. I /don't/ want enter to send the message -- I want enter to create a new line. I /don't/ want you to hide the reply field for noreply@ emails, because sometimes I use draft responses as short memos to a specific email.

seiji3• 11 years ago

So, basically you're arguing never change anything because https://xkcd.com/1172/

jasonjei• 11 years ago

Agreed. SMTP is such a lasting protocol that it is unlikely e-mail will be replaced in the long term. I certainly think improvements can happen on the server side, but the protocol has good failover, and for the most part, the reliability is just as good if not better than physical mail delivery.

Yes, the protocol seems dated, but it is well-understood and implemented by thousands of vendors. It's a miracle that we were able to agree on a protocol, and on top of that, agree on a protocol that is relatively decentralized (although it does depend on DNS if you prefer not to use an IP to email). It isn't like Twitter or FB where a single vendor controls the platform.

rakoo3• 11 years ago

It's a miracle that we were able to agree on a protocol

It wasn't that miraculous, actually. Remember that it was invented before HTTP even existed. It was developed at a time where the only contender was Usenet. Internet was not yet used for any commercial purposes; there was no concept of platform. Internet, and thus email, was the medium between universities that we know of. Sendmail was included by default in each UNIX from that moment on; when you connected to the internet, you could already send emails.

The conditions were very different, it doesn't make a lot of sense to compare it to what we have today.

falcolas• 11 years ago

OK, silly question time. What does this concept of streams have which is different from a mailing list?

mailto:(code-review|off-topic|blog)@tonsky.me?subject=[un]subscribe

Sounds more like he is suggesting improvements to mailing list interfaces than email itself.

MetaCosm• 11 years ago

That is exactly what I thought. "Stream" is the subject line. You mail to a group on some sort of listserve, which gives you a long-term history and a URL to give new people.

This exists in rather decent form in Google Groups already -- there is Google Groups for Business (with a Google Apps account) that gives you a great web interface as well as email interface all just for your company.

fiatjaf2• 11 years ago

It is possible to make Streams work based on existing email, by saving email into a Streams server whenever the receiver opt-in to get that email (well, it is filters again) as a Stream.

Also, let people publish Streams as Streams and non-Streams users can subscribe to those Streams as email.

fiatjaf2• 11 years ago

The only problem is that if email was replaced by Streams every form of sender-initiated contact would be gone from the internet.

joshka• 11 years ago

Would that be a bad thing though? What's the use case for needing it? Any sender initiated contact assumes that the receiver is in acceptance of your need to contact them. This is the failing property of email that allows for and implies that SPAM will always exist. I buy something from Amazon. I'm initiating the contact, so I provide the means for replying / providing an invoice etc. (currently an email address etc.)

ytjohn• 11 years ago

I read this and this that it's a lot closer to what RSS intends to solve, especially the read-only streams. Two-way streams is mainly providing a method to post back into that feed. It's a little more real time (rss feeds are typically cached and refreshed), and a real-time implementation would e more of an collection of entries that are pushed to the server presenting the stream and ultimately to the client.

What I think With that in mind, I feel this concept is very close to a lot of the p2p or decentralized content distribution systems being tested out these days, like aether[1] or matrix.org[2].

The backend implementation that matrix.org uses is rather ideal. It's Federated JSON over HTTP(S). The clients out today are heavily focused on chat, but it is data agnostic. You could have client that writes rich media content and a client that displays this as a blog. Going back to the streams concept, you could have a client subscribed to multiple streams (rooms in matrix vernacular) and as people post their messages - rich media or simple "how are you" comments, it would get federated out to each home server, which can present a web client interface like in tonsky's mock-ups.

For the concept of amazon providing sensitive order information as a stream/matrix room, there would have to be an add-on concept built into the client of some sort of security token. I'm not talking about a losable gpg keypair, but something amazon generates and provides to you (like they do with their AWS services already) Amazon would then create a personalized stream for you, using that security token to encrypt. Since amazon provides the content and the key, they can replace your key at any time.

[1] http://getaether.net/ [2]: http://www.matrix.org/

jp10558• 11 years ago

The problem I have with all of this is it seems like the XKCD where for some reason, the latest person / thing seems to somehow not think to google if there's anything like what they're doing, so we have yet another program / protocol / community that 5 people use and then dies. EVERYONE uses E-mail.

And now with Aether - has no one heard of freenet and i2p and Tor? No, we'll re-do all the sort of starting work on anonymity and security and get to re-make for the thousandth time the mistakes where it really isn't that secure or anonymous rather than building on anything that has at least some of the actual battle testing and bug squashing already done.

What is Aether but Frost with less testing? Hell, what is it but TOR access to Usenet? Maybe I don't understand what's so special that it can't use any of the long worked on infrastructure level work that's gone into the above projects for a decade or so, and it actually does make sense to start over from day one - but if so, they certainly don't explain it.

And I'm not talking about UI - I'm talking about the anonymity equivalent of deciding TCP isn't good enough so I'm going to re-invent IPX or something.

anonbanker2• 11 years ago

is there anyone doing an Aether-equivalent over I2P or Freenet yet?

mgkimsal• 11 years ago

"Transferred to the internet, email inherited these limitations: electronic messages are not real-time, they’re hard to manage in huge amounts..."

Anything is hard to manage in huge amounts, and anything real time is harder to manage even in small amounts.

harel• 11 years ago

We go through this a few times a year. "Email is broken, we must replace it", only to realise a few short seconds later that email is not broken, and does not need replacing. Its long lived for a reason - its simple, its textual, it delegates all the fancy stuff to the inbox provider, e.g., videos in emails are usually a link the email provider, say Gmail, shows as a video.

If I had 10 pence for every "email is broken" post, I could have had a free pint down the pub.

robinhood• 11 years ago

I don't understand why people want to get rid of emails. It's the best asynchronous way of communication.

thawkins• 11 years ago

Is this not just folksomomies for taging email, it will suffer the same problem that tags do, consitancy of the tags, how will it know that kitties, cats, felines and pussys are all the same stream. How will people know what to name the content in the stream. How is this different to adding [CATS] to the front of the subject line, and having the software partition on that.

erikpukinskis• 11 years ago

Every now and then someone decides we need to replace email, or the web, or Craigslist, and they think the reason it hasn't happened yet is that no one has really thought about the details.

It's not.

The reason those things haven't been dethroned is they are he lowest common denominator. You can't dethrone them by making something better. You can only dethrone them by making something that is both better and more common.

By definition the lowest common denominator is crap. By definition it is some legacy junk that was already on every computer 10 years ago. By definition it's a hundred competing standards from multiple companies because how else will a technology work on every device ever?

You can try to replace it with another tech, but it's going to have he same kind of characteristics, and it's arc of adoption is going to look like email's. Which means maybe 10 years from now it starts to look like a real contender. And by then how much crap has been jury-rigged into the spec?

It's the long tail. It's janky, but it's a common standard that does one job like a freaking champ. The long tail will never be engineered away. It ebbs and flows. Only in the last 10 years is snail mail losing its status as a reliable way to reach people.

markbnj• 11 years ago

As others have said, the shortcomings of SMTP can look a lot like advantages, at least in terms of its simplicity and durability. When a protocol has survived and continued in use for so long there are usually deep-seated reasons why it is a fit for certain use cases. I don't want to go off on a chat vs. mail rant, but the following ...

> You operate on a level of the entire messages flow. Forums, chats and web 2.0 networks are trying to make conversations feel as lightweight and barrier-free as possible.

... caught my eye. I manage a team of younger developers for whom real time chat is simply the way they grew up interacting. Everyone is signed into hangouts and "operating on a level of the entire messages flow" (sic) all day long. A large part of the interaction is off topic, vague, and ineffective. Here's an example from yesterday...

Dev 1: the repo url is wrong, I'm getting a 404.

Dev 1: wait, might be a typo.

Dev 1: my bad.

Everyone on the team received this message and presumably stopped their current mental task at least long enough to read it, as I did. In the end it had no impact whatsoever. The developer clearly should have thought through his problem before notifying the entire team that he had one. But this is how chat is used.

I've tried very often to get them to be more deliberate in chat communications, without any success. I have also tried to encourage complicated topics to move quickly to voice hangouts, with even less success. Streams of real time chat messages remains the primary way the group interacts, to the point where I have backed off my efforts and come to the conclusion that I'm likely the one who has to change.

That said, there will always be a need for deliberate communications within organizations. Email grew up as an analog for letters and memos, which served as deliberate, archived, time-stamped records of an important communication. They were considered, usually minimal (because they took effort to create), and got to the point. They were events, not streams of ephemera. I'm not sure that this constant buzz of chats does a good job of replacing that. So for that reason the sentence "You operate on a level of the entire messages flow" doesn't make me smile.

m_mueller• 11 years ago

I think we should look at the proposed idea as two-fold:

1) giving E-Mails more inherent structure that is managed centrally (streams) instead of by each user (filters, folders).

2) making E-Mail more synchronous instead of asynchronous.

As far as I understand, you're arguing against (2). I agree with everything you wrote, however I still find (1) a very good idea. It is also an idea that IMO could be implemented entirely without changing the protocol - it's just a question of where you put this additional information. Streams could also simply be represented as a group address in the CC field that is managed differently by their client/server architecture, but otherwise fully downwards compatible.

jv22222• 11 years ago

I hate to sound synical but Ive both invented this ui and seen others invent it a few times over. Even so I do commend the effort and thought process that went in to the post.

Edit: To see a fully functional version check out http://plugg.io

g8oz• 11 years ago

Offtopic: Thank you for ezSQL

jrochkind1• 11 years ago

When you start new thread, you don’t just start it by filling To: and Subject:, you also have to put it to some stream.

And right there, is why it won't catch on.

owenwil• 11 years ago

All I saw here was what Google tried to do with Wave

miguelrochefort• 11 years ago

Hopefully that person doesn't think he's the first to think about any of this.

grinich• 11 years ago

This is an awesome article, and has some similar ideas to what we are working on at Nylas. https://nylas.com/

I don't mean to steal this thread, but if anyone is interested in building the future of email (including this) I'd love to talk with you. mg@nylas.com

amagumori• 11 years ago

e-mail at its best retains something unique to the written letter, a tendency towards long, frank, thoughtful compositions that are sent with a lot of aforethought and intention. i think if we fully move towards communication regimes where the importance of individual messages is lessened (a chat / forum thread type thing) we might eventually lose this style of writing and the enjoyment of communicating this way.

Streams: Mail 3.0 concept

Related Stories