Locutus: A platform for building decentralized apps, from the creator of Freenet

by sanity31415 • 4 years ago

146 59 4 years ago

I really would love to see decentralized apps, but I don't see how to get around the "back end storage" problem. If you have a ton of data (e.g., video) how and where does this get stored in a decentralized system? It seems impossible to compete with the fast and reliable infrastructure of the big folks (e.g. Amazon, Google, MS, ...).

sanity3141518• 4 years ago

Locutus is more like a cache than long-term storage, so data can fall out of the network if nobody is requesting it, but information publishers can serve as a backup for data stored in the network, replacing it if it falls out.

I think making the publisher responsible for long-term retention and the decentralized network responsible for distribution is the best option here.

Other tools like FileCoin take the alternate approach of trying to do long-term storage in a decentralized way.

wmf• 4 years ago

A bunch of projects are working on storage: Filecoin, Sia, Storj, etc.

trappist• 4 years ago

Not just working on it. We use Storj in production and it's dramatically cheaper than S3.

rklaehn3• 4 years ago

There are plenty of options for long term storage of large assets in a decentralised way. Also, decentralised does not have to mean that you don't use traditional cloud providers.

What is missing is a really polished way to do decentralized apps, so that you don't have to think about the decentralized aspect all the time, but just use it because it is the best option. But it is coming.

Full disclosure: hacking on something to make writing decentralized apps easier: https://github.com/cloudpeers/tlfs

indigochill• 4 years ago

so that you don't have to think about the decentralized aspect all the time

Depending how strict you want to be about this, torrents and Secure Scuttlebutt are pretty good in this respect. I especially love Secure Scuttlebutt because it simply makes the client the store of which messages it's seen which also means there is no authoritative view of the whole network but that's by design.

But probably the even better example here is email. It's the OG federated protocol and nobody thinks of it that way. Probably because casual users have consolidated around a handful of convenient clients (gmail, hotmail, whatever) and not thought too hard about how emails get exchanged from one corporate mail server to another.

BlueTemplar2• 4 years ago

I've been quite surprised by the reliability of Popcorn Time (relatively bad, but torrents have not been designed with this in mind, have they?), and especially PeerTube (how is that even possible ?!).

jjeaff• 4 years ago

Isn't decentralized sharing with resiliency exactly what torrents we're created for?

Ambolia• 4 years ago

This is a problem, but this is a problem with centralized websites too. How many links to broken blogs, broken youtube videos or to broken tweets have you ever encountered? Heck, most people don't even link to tweets anymore, they just post screenshots. The current internet is terrible at archiving information.

cryptome12345• 4 years ago

Why not starting paying them for their services on a decentralized network like BTC pays for computer power?

I can see a decentralized design for paid hosting like that.

sanity3141518• 4 years ago

I'm the creator of Locutus - happy to answer any questions.

rklaehn3• 4 years ago

It is really hard to see how this is intended to work from the README. Where is a good example of the programming model? This one? https://github.com/freenet/locutus/blob/main/contracts/test_...

Since this is written in rust: is there also a version of freenet in rust in the works?

sanity3141518• 4 years ago

We're definitely still working on how we explain it.

That is an example of the contract API - you can find a better explanation of it here: https://github.com/freenet/locutus/issues/9

We're not re-implementing Freenet in Rust, this is a separate project with related but different goals - however it does borrow a number of ideas from Freenet (like using a small-world network as a DHT).

texaslonghorn5• 4 years ago

Given that Locutus was meant to be a centralized human intermediary for the Borg, is the name of the platform intended to be ironic? If so, I like it!

sanity3141518• 4 years ago

:) "Locutus" also means "speaker" in Latin.

Not sure we're going to stick with the name "Locutus" when we launch, so for now it's just a working title for the project.

the_arun• 4 years ago

Define - Decentralized. Does it mean my data can stay in anyone’s computer safely? does it use p2p or such for communication? What are the sizing limitations? Where can read more?

sanity3141518• 4 years ago

Yes, it's P2P. Your data will be distributed around the network on the computers of other peers, with data integrity guaranteed through cryptography.

With respect to reading more, currently there isn't too much beyond the Github project, although you can learn more about my previous work here: https://en.wikipedia.org/wiki/Freenet

traverseda2• 4 years ago

A lot of these projects seem to utilize append-only data structures. It looks like this one doesn't?

For example let's say I set of a chat room, and someone spams multiple GBs of data into a channel, can my moderators (in principle) remove those messages without essentially just publishing a new message saying "please ignore all this spam"?

How do you deal with "network splits", that is a smart contract that could have two equally valid states?

You say you use contracts as the key in a key-value store, can you have the same contract with two different data sets? Like two different chat rooms that work the same way but which have different moderators and different message histories?

sanity3141518• 4 years ago

A lot of these projects seem to utilize append-only data structures. It looks like this one doesn't?

Correct, in Locutus contract state is mutable, with the contract determining whether changes are permitted and how they are applied to the state.

For example let's say I set of a chat room, and someone spams multiple GBs of data into a channel, can my moderators (in principle) remove those messages without essentially just publishing a new message saying "please ignore all this spam"?

Yes (although such spamming shouldn't be possible in the first place thanks to the reputation system).

How do you deal with "network splits", that is a smart contract that could have two equally valid states?

It should be very rare, but contract states that get out of sync can always be "merged" to produce a new state, peers will do this automatically as soon as they figure out the mismatch. The small world protocol that determines network topology should make network splits very unlikely.

You say you use contracts as the key in a key-value store, can you have the same contract with two different data sets? Like two different chat rooms that work the same way but which have different moderators and different message histories?

Yes, a contract consists of web assembly code to specify the contract functionality, and also "parameters" which are arbitrary data that "configures" the contract (chat room name, owner, etc).

ilaksh• 4 years ago

It seems like there is a lot of overlap with some leading edge cryptocurrencies (such as Algorand, which also has Turing complete contracts that control state). Does Locutus have a facility for something like payments?

Also, although web assembly can theoretically be highly optimized by compiling to native code, there are hard limits on scalability especially since many things need to be executed in a specific order. So you have some mechanism for sharding, or are all computations repeated by 100% of the network? Also is there a mechanism for sharding data?

Anyway, I believe that the future of cryptocurrencies, money, and new types of browsers or even normal web applications will all skew heavily towards web assembly.

sanity3141518• 4 years ago

Does Locutus have a facility for something like payments?

It should be possible to build a cryptocurrency on Locutus, using the arbiter mechanism to prevent double-spending.

So you have some mechanism for sharding, or are all computations repeated by 100% of the network? Also is there a mechanism for sharding data?

Computations are not replicated across the network. There is a mechanism that serves a similar purpose to sharding but is more "organic".

Peers are assigned locations in the network when they join (a number between 0.0 and 1.0). Contracts also have locations determined from a hash of the contract code itself, and peers store contracts that are close to their location. The more people using a contract the more widely cached it will be, as a form of auto-scaling.

Locutus uses a small-world network to allow contracts to be found very quickly in an entirely decentralized way.

cropcirclbureau• 4 years ago

Including big overlap with Holochain (which also has WASM and Rust but maybe a bit too ambitions), AD4M protocl over Holochain and maybe Gun. Always glad to see more effort in this area though.

tough• 4 years ago

Can I use this to build a internet video archive a la netflix? I don't trust the copyright owners to do so forever.

sanity3141518• 4 years ago

It's more designed for distribution than bulk storage, but yes - this should be possible.

EGreg3• 4 years ago

How to best get in touch with you privately? There is a lot of interesting research we had to do with regard to implementing byzantine consensus and securing large amounts of value using this same kind of underlying framework. We use Kademlia DHT for swarming and then rely on some mathematical papers from 2007-2015 to estimate the probability of taking over a section of the network and preventing further progress. You’ll need to periodically shuffle consensus groups and a few other things, in order to recover from sabotage. I had to speak to Leslie Lamport, the people at SAFE network, Ripple etc. about aspects of the system.

As it is right now, are you planning to have byzantine consensus for these contracts? Or just try to do verifiable computation?

sanity3141518• 4 years ago

State changes propagate virally through peers storing any given contract. If a peer doesn't get an update for whatever reason there are a variety of mechanisms that get it back in sync ASAP, so it will be eventually consistent.

The contract ensures that the state is valid both before and after any change and every peer who sees the state will ensure it is valid, so it is verifiable computation in that sense.

Happy to chat - you can pm me on reddit or twitter, username "sanity" in both cases.

hitovst• 4 years ago

Is Locutus a fork of SAFE Network?

sanity3141518• 4 years ago

No, it's unrelated.

It's most closely related to https://freenetproject.org/ - which was a previous project of mine.

emacsen• 4 years ago

Did you know that Arne and I have been working on a more universal secure base storage spec (ie everyone should stop reinventing the base blocks that underlie a secure CAS)?

Just curious. :)

sanity3141518• 4 years ago

I wasn't aware, I spoke to Arne a few weeks ago but I don't think he brought it up.

aa-jv• 4 years ago

Will we ever see an implementation that doesn't require a web browser?

sanity3141518• 4 years ago

It will have a websocket-based API so can work with any client software that can talk to that, no browser required.

logifail2• 4 years ago

Decentralized email [can be built on Locutus]

I don't understand this particular one.

I've hosted my own email for the last 20+ years, every single component of the stack is open-source, all you need is your favourite Linux distro, a bit of patience and a [virtual] server with a static IP.

Isn't email decentralized the moment you stop using Gmail or Office365 - or, $deity forbid, your ISP's webmail...?

sanity3141518• 4 years ago

In theory[1] you can run a mail server on your home computer, but even then you must depend on the centralized domain name system to host your MX record, so I would say it's more "federated" than decentralized.

[1] In practice, spam mitigation measures will make it very difficult to get other mail servers to trust your home IP address.

mountainriver• 4 years ago

Yeah but the whole promise of putting dns on the blockchain hasn’t really ones out either

logifail2• 4 years ago

even then you must depend on the centralized domain name system to host your MX record

I've been running my own DNS servers for 20+ years too.

Don't we typically describe the DNS system as "hierarchical and decentralized"[0]?

[0] https://en.wikipedia.org/wiki/Domain_Name_System

zeckalpha• 4 years ago

You are reliant on the receiver’s mail infrastructure. There have been past proposals to flip this around, such as https://en.m.wikipedia.org/wiki/Internet_Mail_2000

endisneigh2• 4 years ago

6. Chat with us We're in #locutus on Discord.
7. Stay up to date Follow FreenetOrg on Twitter.

lol

sanity3141518• 4 years ago

We'll happily use Locutus for those things once it's ready ;)

ocdtrekkie• 4 years ago

Not using IRC/Matrix though is a weird choice, libera.chat is a pretty "standard" choice for decentralized open source projects.

romseb• 4 years ago

I think it's legitimate to first use well-known platforms where people are already registered, especially in such an early development phase.

This has not much to do with dogfooding.

endisneigh2• 4 years ago

You can use GitHub discussions and GitHub now has a feed you can subscribe to, lol.

Literally no need to use discord or Twitter. For a project that complains about centralized entities using more (GitHub) than necessary is so dumb imo. You need to make an account to use Twitter and Discord. Seems like a big step backward.

adg001• 4 years ago

It is called network effect, which is a significant driver. It incentives entities to join networks where their target audience lives. This does not stop those entities to educate the said audience about topics which are in tension with the network hosting the conversation (e.g., decentralised vs centralised systems).

devmunchies2• 4 years ago

each key is a cryptographic contract implemented in Web Assembly

Hope more language start targeting WASI so the entire industry can get behind a single runtime.

For those unfamiliar, this would replace things like containers and allow more portability. It’s the dream of “write once, run anywhere” accept is language agnostic.

I’d imagine there would be a k8s-like infra tool for managing instances of WASM processes.

dezzeus• 4 years ago

I don’t want to be harsh, but to me it sounds pretty much like the value proposition of the JRE (Java Runtime Environment) et similia…

devmunchies2• 4 years ago

yeah, which is why I have “write once, run anywhere” in quotes. The difference being that this is language agnostic and run by standards bodies. its just a compilation target.

We already are trying to do write once, run anywhere with containerization, except that containers use lots of memory and space because you need to virtualize an entire OS.

Whereas WASM is just the bytecode for my program, and I can give that to a WASM interpreter or a compiler to compile to machine code.

EGreg3• 4 years ago

Nice! That’s what we’re building at Intercoin and we are calling it Intercloud

Locutus: A platform for building decentralized apps, from the creator of Freenet

Related Stories