Ian Clarke explains the next generation of Freenet [video] (2023)

https://www.youtube.com/watch?v=yBtyNIqZios

by janandonly • 2 years ago

86 58 2 years ago

gustavus• 2 years ago

I love freenet as we continue to see the growth of authoritative bodies attempt to control the Internet I think it will become more and more important.

The other thing I love about freenet is it feels exactly like using the Internet in the late 90's in almost every way. Same speeds, same simple pages, same crazy whackos spouting conspiracy theories about how JFK shot first, it really is a time capsule that I love.

immibis• 2 years ago

The second freenet has absolutely no relation to the first one. Only the name was stolen.

sanity23• 2 years ago

No relation except that it's created by the same person tackling the same problem (in a more general way) and sharing many underlying concepts like small world networking. The FAQ covers the main differences: https://freenet.org/faq#faq-3

sanity23• 2 years ago

I didn't post this but I'm the guy in the video dressed like Tom Scott. This talk was 8 months ago but everything is still relevant. We're very close to an initial release which I'll (of course) announce on HN. In the meantime I'm happy to answer any questions. You can learn more about Freenet at https://freenet.org/.

onlytime• 2 years ago

Could you change the name? Since there hasn't been a release yet, it would break nothing, and there's already a project (that you also started, but that is used by more people, since it is released software) that was named Freenet, that would probably like its name back.

I like this new project though. It seems cool.

duskwuff• 2 years ago

Seconding this.

Reusing the name of a longstanding software project for a similar but distinct project with wildly different security guarantees is hazardous; it means that existing documentation which directs people to use "Freenet" in a specific way may expose them to unexpected risks. Please reconsider.

sanity23• 2 years ago

Thanks re: liking the project.

I'm not going to change the name again. I carefully weighed up the pros and cons over the course of a year - debating the issue with those that disagreed. Eventually I made a call as the architect of Freenet. It's not without risk, but risks are sometimes necessary.

People are entitled to disagree but I'm not going to relitigate it at this point.

fabrice_d• 2 years ago

Very nice to see progress on this "modern" version!

I have one question for you: how would you compare Freenet to Veilid (https://veilid.com/)?

sanity23• 2 years ago

It's always tricky to do head-to-head comparisons as there is so much detail with these projects.

Freenet is designed to be a complete drop-in replacement for the world wide web, handling the discovery, distribution, and execution of an ecosystem of decentralized software. It's a platform. Similar to how installing a browser gives you access to the entire web, installing Freenet gives you access to all decentralized services built on it.

Freenet's most unique architectural feature is that it's a global key-value store where keys are cryptographic contracts that control what values are permissible under the key and how these values can change. This is the key (pun intended) idea that makes it so general purpose.

My impression with Veilid is that it's more a set of tools and libraries that can be incorporated into software - but it isn't a platform in its own right that can allow software to be discovered and distributed.

Think of it like the difference between buying a car (freenet) or a crank shaft that must be integrated with other components to be useful.

0x1ceb00da• 2 years ago

I read the info on the main page and don't see how it's different from web3 and some other crypto related technologies.

sanity23• 2 years ago

Crypto is mostly about building global decentralized ledgers, think of Freenet as a much more general global decentralized computer. I suggest watching the video if you can.

greyface-• 2 years ago

Using hash-of-validation-WASM as a state key is a really cool idea and I'm excited to see how it goes operationally.

Using (blinded) proof of donation as an anti-abuse signal is a clever band-aid but seems fraught in the long term. Is there a plan to switch off of it to something less centralized, or is it staying this way until it causes problems?

sanity23• 2 years ago

Using hash-of-validation-WASM as a state key is a really cool idea and I'm excited to see how it goes operationally.

Thank you :)

Using (blinded) proof of donation as an anti-abuse signal is a clever band-aid but seems fraught in the long term.

Agreed.

Is there a plan to switch off of it to something less centralized, or is it staying this way until it causes problems?

These blind trust tokens are a proof-of-concept because while it is anonymous and has the benefit of funding the project, it is centralized which is far from ideal.

Proof-of-work could be used as a decentralized mechanism for mitigating abuse but I don't like deliberately burning energy. I quite like a concept called "proof-of-trust" that I describe here [1], but the design needs to be fleshed out.

[1] https://freenet.org/blog/799/proof-of-trust-a-wealth-unbiase...

petermcneeley• 2 years ago

The primary non-technical challenge to all these distributed storage system is the storage of illegal data. How does this protocol deal with this issue?

sanity23• 2 years ago

Freenet side-steps this problem because it is more of a communication medium than a storage medium. It also isn't optimized for the distribution of large files that are the most likely to pose copyright issues. It's not a substitute for BitTorrent.

generalizations• 2 years ago

From the FAQ about the differences: https://freenet.org/faq#faq-3

While the previous version was designed with a focus on anonymity, the current version does not offer built-in anonymity but allows for a choice of anonymizing systems to be layered on top.

I wonder why this choice was made. To me it's concerning...seemed like one of the big draws of freenet was the anonymous nature of it, and it seems like this would cause some fragmentation if there's multiple options to choose from.

giantg2• 2 years ago

I didn't think the original was truly anonymous without other systems being used anyways. Probably better to just allow integration with other systems since the methods of staying anonymous will change over time.

sambazi3• 2 years ago

will be interesting to know if this is a bait and switch or simply a concession to the majority of users that just want free stuff

sanity23• 2 years ago

Freenet is free as in free speech, not free beer. The goal is to provide a platform that allows people to build sophisticated decentralized systems to replace today's centralized systems. The original Freenet architecture wasn't flexible enough to do this.

sanity23• 2 years ago

I wonder why this choice was made.

Because it's a better architecture. There are numerous different approaches to anonymity, each with different pros and cons (mixnets, dining cryptographers, etc). It's better to have a system that can offer a choice of anonymity mechanisms without locking users into one approach.

The new Freenet is a platform for building interoperable decentralized systems. Some of these systems can provide anonymity, and can be integrated into other systems if the creators of those systems choose to do so.

csande17• 2 years ago

This sounds a lot like IPFS's strategy regarding anonymity, and AFAIK it resulted in basically no one being able to use IPFS anonymously. When anonymity requires building your own separate sub-community and maintaining your own custom plugin/patchset/configuration for the client, it's really hard to achieve that because the default non-anonymous project sucks all the oxygen out of the room.

Do you at least plan to provide some level of baseline consideration for privacy/anonymity in the main project, like making sure core protocols/libraries don't expose fingerprinting surfaces or leak information?

janzer• 2 years ago

FYI, the original freenet can now apparently be found under the name hyphanet[1].

Although it seems there has been no activity since the rename happened almost a year ago?

1. https://www.hyphanet.org/freenet-renamed-to-hyphanet.html

omoikane• 2 years ago

The original Freenet (with some focus on anonymity) is rebranded Hyphanet, and the new version (which does not offer built-in anonymity) is now the official Freenet. What a mess.

sambazi3• 2 years ago

does not inspire confidence indeed

iforgotpassword2• 2 years ago

And haven't there already been at least two reasonably popular but incompatible versions before? I think 0.5 was still used for quite a while when 0.7 was around. They offered similar services and anonymity concepts though.

sanity23• 2 years ago

The new version will offer a choice of approaches to anonymity. I wouldn't call it a mess but there is 25 years of history to the project which does inevitably lead to some complexity. I summarize the naming situation here [1]

[1] https://www.reddit.com/r/Freenet/comments/1d05zw9/questions_...

ArneBab5• 2 years ago

Check the prepared release news for the changes since last year: https://github.com/hyphanet/fred/releases/tag/build01498

Currently waiting for the new signing infrastructure for the Windows installer before turning the release to final.

Freenet 0.7.5 build 1498

This release resolves the last blocker for Freenet / Hyphanet 0.8 by providing an official Debian package. Additionally it optimizes the networking and data transfer core and provides many improvements for website authors and user experience.

Starting with this release, Freenet / Hyphanet has an official Debian package built automatically via github actions. This was the most important [high-impact-task][] and the last release blocker of version 0.8 in our [Roadmap][]. Big thanks go to DC!

With this finally realized, the next step is to get in contact with the many privacy focussed distributions which build on Debian to make `hyphanet-fred` available where it is most important. Once this is done, tools which build on Hyphanet — like FMS, but also jSite and tools from pyFreenet — can be packaged to work out of the box, using Hyphanet as an ordinary background service. That’s a step towards Hyphanet as decentralized, privacy-preserving communication backend for other applications.

Another step towards this is accepting the Schema hypha[net] to simplify writing browser extensions that forward hypha:-links to Hyphanet.

The networking layer was optimized significantly. Searching packet types is often stopped early and common or cheaper checks are done before less common or time-consuming checks. This gives significant reductions of CPU load, especially for very fast nodes.

Juiceman fixed a bug limiting MTU to 1280 where not needed.

And recently failed and data not found cooldown times were reduced to 5 minutes and 3 minutes, reducing one of the big annoyances when accessing a site quickly after upload.

On the data transfer layer, healing was optimized. After 1495 strongly increased the amount of healing to keep large files available for longer, 1498 specializes healing to keys close to the node location. This reduces healing per file, but improves privacy, because healing inserts are then more similar to forwarding — they mostly send data close to the nodes location — and it reduces the network load of healing, because the specialized healing inserts need fewer hops to reach the optimal storage location in the network.

In addition to these changes deep down, there are a number of directly visible improvements.

The plugins KeepAlive and Sharesite are updated (the latter now uses the new Night Zen Garden style). The UPnP2 plugin is now visible in simple mode. It can replace UPnP and should work better. On the flipside the Library plugin is moved to advanced plugins, because it does not work reliably enough.

The plugin list is easier to navigate by removing the defunct option to download plugins from the clearnet and by adding better styling. Downloading from the clearnet was an unnecessary privacy risk since we’ve been bundling essential plugins with the installer for a few years now.

The noderef for friend-to-friend connections is shown in simple mode again, because it is robust enough with the changes in recent years. This should remove a barrier to adding direct connections and enabling fully confidential messages between friends.

There are new configuration options to allow connecting via local services. That’s a step towards making it easy to add a second layer of security, for example confining connections to a local network. Thanks goes to s7r for these changes!

When bandwidth detection fails, the upload bandwidth now defaults to 160KiB/s. Also the NLM config is now disabled statically. This was a testing setup which could still be active in old nodes, but it would break connectivity nowadays.

The default bookmarks include the Opennet SeedNodes statistics, the generate media site to create decentralized streaming sites, and the high-impact-tasks. The bookmarks are also re-ordered to be a better match for newcomers. Starting category: first steps, clean spider, Index of Indexes. For the software category ordered by ease of use from fproxy.

For website authors, more CSS elements, selectors and combinators (`:checked`, `word-wrap: anywhere`, `focus-within`, `^=`, `$=`, `=`, `>`, `+`, `~`) and additional HTML elements (`summary`, `details`, `<meta name="Viewport"...>`) are available. This strongly expands the possibilities of websites authors in Hyphanet, because Javascript or webassembly are no viable options in an environment where a privacy breach could put people at risk. We’ve seen with Java applets, that untrusted code will always break out of its containment. The CSS improvements in contrast provide a safe way to enable limited interactivity.

Streaming support via m3u lists was improved to allow accessing segments of up to 200MiB.

And using `-1` as version in a USK now properly finds version `0`, if this is the only existing version.

There were a number of Java 21 fixes, including all our tests (thanks to Bombe!), and improvement to the github actions (thanks to AHOHNMYC).

In addition to that there was a lot of polish. Bert Massop and Veniamin Fernandes replaced our homegrown CurrentTimeUTC with modern Java options. Alex fixed the pronoun used in strings. Bombe added getters for all direct field access in the node. Hiina reduced logging level of store warnings so no unneeded backtraces are created for node with large stores and Juiceman updated code to use more modern structures.

Time-dependence of compressor selection was removed. This caused non-determinism for inserts and could cause keys to be non-reproducible on systems with faster or slower network.

And finally the new [exe signing workflow][] we built to fulfill the requirements of SignPath, our new windows installer signing provider for the upcoming releases, runs the [verify-build script][] on every release to ensure that the jar we release has actually been built from the sources. This provides a second safety net, in addition to anonymous users running the script and posting the results (thanks to all who did this — please keep it up, otherwise people have to fully trust github). The release is not yet byte-by-byte reproducible, because the jar MANIFEST defines among other info the exact java version used to compile it, and the java version available differs by distribution and time, so it would get harder over time to verify the build.

A special thanks goes to Bombe for many careful reviews!

[high-impact-task]: https://github.com/hyphanet/wiki/wiki/High-Impact-tasks [Roadmap]: https://github.com/hyphanet/wiki/wiki/Roadmap [exe signing workflow]: https://github.com/hyphanet/sign-windows-installer [verify-build script]: https://github.com/hyphanet/scripts/blob/master/verify-build

usernamistaken• 2 years ago

Often a lot of the chat is just people who read the headline but this discussion seems to be just people who read the word 'freenet' and stopped right there.

The architecture is really interesting and pretty simple for a distributed system. It's nicely presented in the video too.

The examples offered in the docs are very specific to builders of a globally distributed system but some examples of how you might go about building some useful day to day applications would be handy.

sanity23• 2 years ago

Thank you.

I agree about needing more practical examples.

We are working on this, we do have a simple email app [1] and recently completed a more "hello world" type example freenet-ping [2].

Right now our focus is on getting the network up and running (hopefully we're days away - just tracking down some final bugs), but some good practical example apps is high on our todo list.

[1] https://github.com/freenet/freenet-core/tree/main/apps/freen...

[2] https://github.com/freenet/freenet-core/tree/main/apps/freen...

Ian Clarke explains the next generation of Freenet [video] (2023)

Related Stories