TMTP a Internet protocol combining elements of email and the web

by modinfo • 4 years ago

77 39 4 years ago

The demo is not really usable on mobile. Glad to see efforts to build new protocols, I think that’s going to be the only way to fix the web.

networkimprov10• 4 years ago

(Author here.) The client UI is designed for desktop. A mobile frontend is in the works. Tentatively planning to use Ionic Framework with Vue.

mxuribe• 4 years ago

Firstly, i love the recursive nature of the implementaion: "mnm is not mail"! Also, quite a fascinating approach; hoping it works out! Me being a fanboy of the matrix protocol, I've often wondered if there is a valid use-case for matrix to enable safe, secure communications between site user/customer, and site manager(s). Not just chat, but a sort of customer support mechanism, but also a safe, private mechanism for site managers and/or orgs to do reasanble outreach to user/customers. And, no, it need not be chatbots, though i suppose there would be a desire by some orgs for an implementation like that at the end of the day. My hope would be of course for a better chatbot than current standards. Obviously, there would need to be the delicate balance of ease of use, privacy for user/customer, ideally decentralization (for empowering liberty/freedoms), integrity of content conveyed (by way of non-tampering/security of content), etc.

In any case, good luck to the folks behind this effort; the more internet protocols (and not centralizied systems/orgs) that can help people, the better!

networkimprov10• 4 years ago

(Author here.) Thanks for your interest & support!

Yes, the default concept for an Internet messaging service is "everyone can reach anyone". This isn't helpful for online services and businesses who need to reach their members/customers directly & securely, and with a rich content model (e.g. survey/form in message, and structured result in reply).

indymike2• 4 years ago

Interesting. The protocol TMT, seems to have the idea of users having multiple nodes (which are referred to in comments as devices). I suspect if I were to use this in a product, I'd use nodes for application instances instead, so if a user had multiple apps (i.e. consumer payment app and merchant dashboard app) on the same device I could keep them straight. Seems promising.

networkimprov10• 4 years ago

(Author here.) TMTP clients maintain concurrent (possibly intermittent) connections to any number of TMTP services, and each service gets its own inbox. So you don't need a separate app instance per service.

Inboxes for a set of related services could conceivably be merged, tho I haven't done any work on that yet.

indymike2• 4 years ago

Thanks for clarifying.

duncan-donuts• 4 years ago

I’m struggling to understand what this actually does. I poked around looking for information on the actual protocol, read through the site, and I don’t understand what this solves. The mechanics of the protocol aren’t very well defined, and the site says a lot of things without giving much information. TMTP isn’t an idea I would invest in and it’s nice that people are working on new protocols.

vxNsr• 4 years ago

I don’t quite understand how this works. They say it’s a new protocol but it’s still using a browser which would imply it’s layered on top of UDP or TCP.

Building a new protocol today is kinda like inventing a new engine that doesn’t require gas, it’s gonna need a giant motivator to be adopted and trillions of dollars.

The days of grassroots bootstrapping are over, no one has any reason to adopt your thing because what we have now is Good Enough(tm). Your new thing needs to be revolutionary AND have government backing across multiple governments and administrations. There is no other way, consider the electric car vs hydrogen.

cestith3• 4 years ago

Most Internet application protocols run on top of UDP or TCP. They mean a new application protocol, like SMTP, HTTP, HTTPS, L2TP, POP3, IMAP4, SNMP, LDAP, SSH, FTP, RDP, LPD, XMPP, Kerberos, remote syslog, etc etc etc. If you're on any flavor of Unix, take a look at /etc/services and you'll see there are lots of protocols in the world.

icedchai3• 4 years ago

The protocols you list are decades old. There hasn't been a new mainstream protocol in a very long time. "Innovation" kind of stopped when we started tunneling everything over HTTP(S).

RajT88• 4 years ago

What sites use this?

Seems like Hilton is mostly what comes up when I try searching for "sites supporting tmtp"

squarefoot• 4 years ago

About time someone thought of replacing email protocols with something new but without adding the bloat that seems so inevitable today. I would also think of integrating functions to replace also NNTP (News, USENET), so that it would cover all needs between private one to one and public many to many communications.

myself248• 4 years ago

Compare also NNCP: http://www.nncpgo.org/

networkimprov10• 4 years ago

(Author here.) I imagine that existing web-based bulletin boards would simply copy messages to TMTP users if requested.

Conversely, TMTP groups/lists could be copied to a read-only website.

Forge36• 4 years ago

Previous discussion 208 comments

https://news.ycombinator.com/item?id=25804869

cestith3• 4 years ago

Oh, look, we've come full circle to private bulletin boards with a custom client.

eis5• 4 years ago

Seems like a poorly thought out protocol.

  1. Inconsistences like the value for "op" being sometimes integer but then sometimes string. Why is this an integer anyways? We are already in inefficient JSON land and a lot of the rest is strings.
  2. No mentioning of ordering or guarantees in the draft spec. Bad recommendation to use time for sequential message ids.
  3. Says E2EE not in it right now but planned. Says E2EE not default because sites want to scan and archive messages but then the page says email is bad because it doesn't have E2EE and nodes can scan messages?
  4. The usage of the word "node" is confusing and inconsistent. Sometimes it's described as a device, sometimes it's a password. "nodeid", "node", "newnode"...
  5. Seems like one can spoof the alias for each message? What's the reason for this? The page specifically mentions phishing being an issue with email but wouldn't this help phishers? There is no way that I can see if a message came from the site itself or a random user on it.

Then a quick look at the code. First noticed the unusual (for Go) coding convention like "tFoo" structs, "eErrFoo" errors. Calling a mutex "door", weird "//:" comments. I haven't tested it but I'm fairly sure the convoluted way of storing user info in flat files has path traversal issues. Seems like there's a cache for alias -> uid but anyone can pick any alias so they would collide?

Just from a cursory glance. Also doesn't seem like it has moved much in a couple years.

networkimprov10• 4 years ago

(Author here.) Development has been paused for ~1y as I've been at work creating a vehicle to take this forward.

The protocol specifics (e.g. identifiers & types) aren't polished, because I focused on features in working code. Thanks for the feedback!

You can't have alias collisions; it checks whether the string you propose is acceptable & available :)

A sender's alias is (or will be) checked against their recorded aliases.

The reason why lack of E2EE in email is a serious problem is that email traverses at least two "federated" services, which are typically third parties with whom you have no intrinsic need for a relationship. A TMTP message passes through a single service which you explicitly joined.

I don't see how your feedback justifies "poorly thought out"; it's merely an unpolished preview.

georgia_peach• 4 years ago

Wrong place to share. The harmless people here will nitpick you to death as the adult version of sucking-up to teacher & showing how smart they are. The dangerous ones will silently steal the better parts of your idea & run with them, but with better funding & connections.

nominusllc• 4 years ago

I don't see how your feedback justifies "poorly thought out"; it's merely an unpolished preview.

I don't think they meant this with sincerity. It's unpolished, which is accurate, but there's promise there, and I think their frustration was the last 20%, which is often overlooked- the cohesion of marketing babble and featureset.

karmakaze2• 4 years ago

TMTP messages pass through a single service which you explicitly joined.

This seems like a product rather than an internet draft protocol.

GekkePrutser• 4 years ago

Really you shouldn't introduce a new service in 2022 without E2EE by default IMO. Not if you want it to be useful going forward :)

eis5• 4 years ago

Cheers for the reply.

  > The protocol specifics (e.g. identifiers & types) aren't polished, because I focused on features in working code. Thanks for the feedback!

Got it. IMHO it's better to start with a clean protocol spec before implementing features. At least for the very basic stuff. Of course for an MVP I wouldn't start with a spec but then again this seems past the MVP stage seeing as it started 2017 and wants to be a protocol and not just product.

  > You can't have alias collisions; it checks whether the string you propose is acceptable & available :)

You can't use a volatile cache to check for collisions though. Unless I'm missing something and the "cache" is prefilled with all existing aliases somewhere and not volatile but then it wouldn't be a cache? Even if it did, it's asking for trouble.

  > A sender's alias is (or will be) checked against their recorded aliases.

What about the following: 1. user A adds alias Foo 2. Server restarts 3. user B adds alias Foo 3. random other user sends message to alias Foo.

My point is a cache should just be used to speed up certain operations but has to be handled with care (cache invalidation etc). Better to skip this as it's just premature optimization and can just cause trouble.

In fact all this work to juggle JSON in flat files will cause more headache than it's worth. Just use something like SQLite and you'll save yourself a lot of time and issues. It'll make the implementation much easier especially to adapt. But it'll pay dividends for a long time (migrations etc). You'll run into filesystem limitations quite quickly anyways with these flat files.

  > The reason why lack of E2EE in email is a serious problem is that email traverses at least two "federated" services, which are typically third parties with whom you have no intrinsic need for a relationship. A TMTP message passes through a single service which you explicitly joined.

But what prevents a "gmnm" here? Companies don't want to host heir own email servers and they might not want to run their own mnm servers. Nothing here guarantees to the user that he's directly communicating with the other party. And all data is also stored unencrypted.

As GekkePrutser mentioned in a sibling comment, it's 2022 and any new messaging protocol without E2EE will be dead on arrival. I'm maybe exagerating but it'll have a tough time. Especially if it justifies itself partially because email doesn't have E2EE.

  > I don't see how your feedback justifies "poorly thought out"; it's merely an unpolished preview.

Maybe so. I just saw it mentioning problems with mature protocols like SMTP and Matrix and I agree with some of that criticism but the solutions are not in TMTP as far as I can see. What about synchronization between multiple client devices? What about permissions in groups? What about identity verifications? What about mobile clients which don't want to open dozens persistent connections? There is a long list of topics to cover when it comes to messaging protocols.

To create a new protocol isn't an easy task. A protocol should be precise and robust. To create a product? Much easier.

BTW no one forces people to federate when using Matrix. Or one could use something like the Signal protocol. Sure they are more complex than TMTP but they have to be because they solve a ton of non-trivial issues.

PS: Seems like the demo doesn't actually use the protocol at all? It looks like it just loads all static data from https://mnmnotmail.org/w/data-demo.js and basically is a static page.

vidarh2• 4 years ago

It seems to be conflating a communications protocol, an application level list management protocol, and parts of a message format.

It'd be far more likely to gain support if they unbundled this and made it work as an upgrade over e-mail, which would seem to make a lot of the spec redundant.

networkimprov10• 4 years ago

(Author here.) The message format definition is included in the protocol doc only as a convenience. Ultimately it would be two standards.

Trying to "upgrade" the email ecosystem is an impossible goal, and the requirements I'm working to can't be shoehorned into the email model.

vidarh2• 4 years ago

I don't see anything in your current page that can't be layered into the e-mail model, and my prediction is that this will never get serious traction for that reason, because anything people want from it will just be integrated into e-mail if it starts to get attention.

Consider that SMTP was "shoehorned into" the email model, and so was HTML, and attachments. My first e-mail exchange involved UUCP rather than SMTP. For many years my e-mail clients did not understand this newfangled HTML thing. E-mail is extremely malleable because e-mail is extremely simple.

Add a "To: " header to a text file and you have a basic e-mail. Copy the file to another machine and you have a mail transport (that's roughly how UUCP worked - by implementing basic file transfer between hosts, and layering distribution of that mail on top with a program that'd read a file of concatenated e-mail messages and distribute them to mailboxes).

E-mail is at its core a simple key-value header format, a message body and an address. Everything else is auxiliary. Transport layers can coexist: UUCP coexisted with SMTP for many years. If you invent a new one you just need to decide if you care about having a gateway to let "outsiders" contact you. Message formats can evolve: As long as you provide the basic MIME headers, you're in full control of the format of the message past the first set of headers. You choose how much interop you want by whether or not you use multipart/alternative to provide version readable by "old style" clients.

These features are why e-mail in its current form is still around.

What you gain* from this is interop and an opportunity to have your system evangelise itself as you use it.

To address your page more specifically:

+ choose the sites where you participate + select which members of a site can message you

If you enforce DMARC (see below), you can do this in a mail server simply with an option to whitelist domains or users or both.

+ always know from which site a message originated

This is supported in e-mail today via DMARC. The reason you still receive e-mail for which it's not the case is lack of universal deployment. You can solve this issue simply by configuring a mail server to reject all e-mail that fails DMARC.

+ can block anyone with whom you’ve made contact
+ may leave a site and never see traffic from it again

If you enforce DMARC, you can certainly block sites at the server level or client level or both, or just block individuals.

The upside of doing this with DMARC is that people with it correctly set up will only notice anything different if they try to send you unsolicited messages and get an error message back.

+ message formatting & layout via Markdown (aka CommonMark)

Solved with client support - just needs the appropriate MIME type set and a ideally a multipart/alternative to deal with any clients that fail to fall back on plain text rendering.

+ hyperlinks to messages and other threads

Solved with a URN spec and client support.

+ hashtags and private tags + slide deck layouts + data-driven charts & graphs

Solved with message formats and MIME types.

+ forms/surveys whose results are collected into tables

Needs server side support if you want something standardised, but works fine with links to the web today with appropriate client support and HTML, and if you want to provide an upgradeable path it's easily done with client support and a MIME type + requiring multipart alternative w/ a link to a web version for clients without support.

So to summarise how I would have done this if I wanted to meet the goals stated on your page:

* Pick a mail server that supports JMAP for the client.

* Configure it to reject all mail that fails DMARC with a suitable message explaining why. Now you've created pressure to add DMARC for those who don't.

* Add a JMAP profile for the client to tell the server if it wants whitelisting or blacklisting, and which users and domains it wants to whitelist and blacklist. I'll tell you right away that most users will reject a whitelisting option, so consider greylisting or quarantine as options.

* Optionally define a MIME type to carry subscription/follow/unfollow requests to remote sites, and a DNS record to hold public keys to sign requests.

* Define your desired message format and pick a MIME type.

* Specify the rendering of a plain text version of the message formats for delivery via multipart/alternative, which includes a footer that specifies that this is a possibly imprecise rendering and links to an info page.

Using JMAP means people would be able to connect with any JMAP capable client and "just" lose out on functionality not supported by that client. Using a mail server means most well configured mail servers can already talk to you if whitelisted/not blacklisted. Using multipart/alternative that way means clients can "upgrade" gradually and possibly after users see a link to an info page, and the footer in the plain text parts (or other rendered alternative) would help evangelise the system.

networkimprov10• 4 years ago

Hi, author here, AMA!

networkimprov10• 4 years ago

FWIW, most of the roadmap for this project is not described on the website nor defined in the protocol.

This is a deep topic, and "messaging" is not the only goal, nor even the principal one :)

sbuk• 4 years ago

Not sure about the “features” at the end. Specifically Markdown support. This is a client-side issue. If MUAs parsed Markdown, then there is no reason that markdown couldn’t replace html in the relevant part of a MIME multipart message. The SMTP server only handles ASCII characters; it really doesn’t care how that is presented. It also afford the possibly of plaintext as a fall back. Arguably, you could have all three.

TMTP a Internet protocol combining elements of email and the web

Related Stories