Decentralizing Distribution

https://f-droid.org/en/2022/02/05/decentralizing-distribution.html

by marcodiego • 4 years ago

119 41 4 years ago

redsolver• 4 years ago

I am working on a very similar project (decentralized app distribution on Android) called SkyDroid (https://skydroid.app/), with the difference that it is based on Sia Skynet (a Filecoin/IPFS competitor) and uses the DNS system for global app discovery - so for example the SkyDroid app itself is available on the skydroid.app domain inside of it. I'm curious about how they will try to solve the discovery issue, because if they just have one global decentralized pool of apps, it will be very hard to ensure that no malicious apps get in. But if they keep it a central repository of apps trusted by F-Droid by default, there's not really much decentralization going on. Most developers would still publish their apps in the main repo directly.

deepstack• 4 years ago

it will be very hard to ensure that no malicious apps get in

Well isn't the whole idea of F-Droid is that the app can be build from the source that is available and the build checksum will match the source from anyone? Sure there can be malicious code, but code for that malicious code will also be available publicly.

eighthave3• 4 years ago

We're taking a hybrid approach right now, where we incorporate IPFS support into our working stack. Then we can transition more and more to IPFS as it proves mature enough to replace the more centralized methods.

awinter-py2• 4 years ago

interesting that it's 'veracity and distribution', not just distribution. makes sense that you need to hash the file, but I wonder about various higher-level ways to decentralize CI while maintaining a cheaply checkable 'affirmation'

for example, can you cheaply verify repeatable builds? (on a phone even?). can you do intense linting for privacy violations like phone-home, then publish a 'lint payload' that's cheaply verifiable against the built binary?

feels like if f-droid leads here, commercial software will have to adopt their tools to maintain their position of trust

(also huge fan of f-droid project, I donate, I think it gets more important every year)

mhitza2• 4 years ago

Somehow related take.

When I was still interested in blockchains back in 2018 I was thinking that instead of proof of work, wouldn't it be nice to have a distributed CI where you run your builds & tests on end-user machines, it computes a hash of the resulting artifact. Which is validated by multiple nodes that run the same job for validation (then split the PoW fees among themselves).

It's not an easy problem to solve of course, and didn't go too deep into what TPM (Trusted Platform Module) can do, but I think you would need an isolation unit from the kernel/OS to allow running untrusted code, and prevent tampering. There's definitely a fine line between such a functionality and spyware that would need serious safeguards.

yjftsjthsd-h3• 4 years ago

Just make it fully isolated with WASM or whatever, and then port the necessary compilers to cross-compile back to whatever native platform you want to use the outputs binaries on. Simple! (Not easy, but conceptually simple and probably doable.) The big problem that I can't see an obvious solution for is proving that a given output corresponds to the work requested. I guess you can just run it on enough machines that you hope they're not collaborating?

yjftsjthsd-h3• 4 years ago

I'm not sure if this is exactly what you want, but https://f-droid.org/en/docs/Reproducible_Builds/ is a thing

awinter-py2• 4 years ago

ah, thanks for link. From this it seems like verification builds require the same amount of work as the original -- you can't save work by delegating to untrusted CI.

yjftsjthsd-h3• 4 years ago

Expand outreach and assistance to software developers, media organization, tech companies and more to promote adoption of the F-Droid platform for their software, content and devices (example: Mozilla could easily run a Mozilla app store that includes all the Mozilla channels: releases, nightlies, etc. They could also include a curated collection of reproducibly built apps that Mozilla approves of. Someone who trusts Mozilla can then easily choose to only have access to the Mozilla-curated app store)

That would be fantastic!

franga2000• 4 years ago

This would also finally remove the excuse the Signal devs have for not including the app in F-Droid. Third-party repos aren't very discoverable and the official repos don't let the devs use their own keys.

eighthave3• 4 years ago

Yes, but f-droid.org actually does let devs use their own keys via the reproducible builds process (https://f-droid.org/docs/Build_Metadata_Reference/#Binaries). The only blocker for getting Signal into f-droid.org is that it includes proprietary Google libs.

throwaway9843932• 4 years ago

FWIW, Linux distributions are decentralized without any modern tech. It is all secured via signed checksum files you can download from hundreds of geographically-distributed independent sources. There is no dependence on any single protocol or technology, because it's just copied files. Anything that can copy files can spread the distribution. This makes them future-proof, backwards-compatible, and flexible enough to work in any situation. You can cache files, proxy connections, use any storage mechanism, any transfer method. The packaging tools support multiple transport protocols. Works offline and online. You can install a Linux distribution today the same way you did 30 years ago, from the same sources, even though the underlying servers and software have been replaced.

giantrobot2• 4 years ago

No. Clearly old things are bad. Only new things are good. Everything but be ridiculously complicated and extremely fragile. Reliability is bad! /s

The Linux distro model is awesome. It would be more awesome with reproducible builds but it's still awesome as-is. Not only can Linux distros easily distribute their software but even mix and match packages between different mediums and have completely consistent systems with the same level of security. Even better is anyone can freely set up a distro mirror and anyone using it can trust it the same as they trust an "official" mirror.

pabs32• 4 years ago

It would still be good to have Linux distros on IPFS too though.

boramalper2• 4 years ago

It is all secured via signed checksum files you can download from hundreds of geographically-distributed independent sources.

It's not permissionless though, or put differently, discoverability is the problem. You can set up your own mirror, and through cryptographic protocols the authenticity of the data you are distributing can be verified all well, but you need the distro maintainers to manually add you to their list of announced mirrors before any traffic hits your servers.

The point of BitTorrent, IPFS, Dat, etc is that they create a network of participants that does not need anyone to add them (nor, consequently, their permission) to start contributing. This also reduces the administrative burden of course, and provides load-balancing as well since the content is now distributed not in file-units but block-units (~ 16 KiB).

throwaway9843932• 4 years ago

Seems like a bad idea.

First, BitTorret is unreliable and slow, whereas a single transport server is very fast and reliable, and mirroring makes it both globally redundant and verifiable.

Second, protocols that build in all features (discovery, transport, verification) are less flexible and more centralized than decoupling into different protocols.

Third, when random participants can contribute anything, you get a bunch of malware (look at every app store). Linux distributions' app repositories don't have any malware.

Fourth, you don't need a distro maintainer to do anything at all if you want someone to download your app from your own repo. You just serve your public key and put your files online somewhere. Anyone can download the key, add the repo, and install the packages. You can even put them on GitHub, GitLab, etc.

skyfaller2• 4 years ago

Dammit, yet another project integrating/promoting cryptocurrency (Filecoin in this case). I like and use F-Droid, and I'm really unhappy about how difficult it's becoming to avoid web3 crap turning everything into a financial instrument to fuel speculation, energy use and e-waste.

BLKNSLVR• 4 years ago

Don't let the cloud of BS surrounding web3 (and cryptocurrency as a whole) obscure your view of the potential gold that lies at its core.

Your comment reminds me of Letterman ridiculing Bill Gates' description of (very) early internet potential by saying "have you ever heard of the radio?".

Edit:

(the following was added after a couple of upvotes were received, so the following isn't necessarily "agreed with" by anyone)

Uneducated opinion: The financial instrumentalisation of some of these web3 things is required for these two sides of the same coin (pun unintended):

1. A barrier to entry against those who would seek to sabotage the system / network / <thing>

2. Incentive to participate and donate resources to the system / network / <thing>

The speculation side of this financial instrumentalisation is, to a greater or lesser degree, unintended consequences (potentially inflammatory example: Bitcoin was intended as an uncensorable way to exchange value, ie. a currency, and has now turned into a store of value to 'hodl' long-term rather than regularly exchange for goods and services). I put this down to human nature rather than design.

sinkholed2• 4 years ago

I looked up the Letterman video expecting to see someone confidently dismissive about the Internet.

Letterman came across as fine: https://m.youtube.com/watch?v=gipL_CEw-fk

The radio quote is taken out of context.

Letterman doesn't come across as well as Bowie though: https://m.youtube.com/watch?v=8tCC9yxUIdw

giantrobot2• 4 years ago

I put this down to human nature rather than design.

The design of Bitcoin plays a larger part in "hodl" than human nature. The cost of mining growth with growth of the network pretty quickly eliminated Bitcoin's utility as a value exchange.

It didn't take long for Bitcoin transactions to get more expensive than traditional payment methods. There's no sense in paying fees to transact in Bitcoin if they're higher than traditional payment. Bitcoin's "value" has also always been highly variable thanks in part to the fees so there's no good way to post prices in Bitcoin for vendors either. Shit the value can fluctuate wildly while your transaction is processing! Your agreed upon "price" is meaningless.

rpdillon2• 4 years ago

I generally agree with this sentiment, but IPFS and Filecoin I view as quite different than the rest. I think the F-Droid team does as well:

Since the “decentralized web” includes all sorts of things, it is important to also mention things that we will not be doing with this grant. We are not looking at NFTs, blockchain, DAOs, smart contracts or related aspects of “Web3”.

Filecoin is acceptable to me because it seems to be a currency tied to a real asset that folks need: drive space. I'm not an expert on this, but Wikipedia's entry mentions this aspect:

It is made by Protocol Labs and builds on top of InterPlanetary File System, allowing users to rent unused hard drive space.

I don't think it's an arms race for energy usage and e-waste the way e.g. Bitcoin is, but if I'm getting this wrong, I'd be interested to hear about it from folks who have better context!

wmf2• 4 years ago

Filecoin's deflationary "number go up" economics distract from its use for storage. It's true that Filecoin incentivizes miners to provide storage space, but the vast majority of that space is unused so it appears that the incentives are out of balance. In other words, Filecoin's "users" are just hodling Filecoin to get rich; they're not storing data. It's too soon to tell whether this situation is temporary or not.

Also, Filecoin has fairly high non-storage hardware requirements so miners aren't just providing storage; they have to spend significant energy and capex to prove that they are providing storage.

hanniabu• 4 years ago

I'm really unhappy about how difficult it's becoming to avoid web3 crap turning everything into a financial instrument to fuel speculation

Almost like it's not all speculation and it actually provides a utility....

varnaud• 4 years ago

What utility? In the Funded work section of the article, I don't see anything that web3 is making more useful or that could not be done without it.

I hope they simply cash out the fund they received and implement none of the IPFS crap.

On April 1, 2021, the date of the archive.org article mentioned in this f-droid article, FIL jumped literally to its all time high. The 50,000 FIL archive.org received was then worth $7,965,000. Today, the same 50,000 FIL are worth $919,500.

It's almost like it's all about speculation and the actual utility is making early investor rich.

mishmosh• 4 years ago

Mosh from Protocol Labs here. Specifically on energy use, the Filecoin Green initiative is working to rigorously map and authenticate energy consumption on the network, and become the first carbon-neutral (and eventually carbon-negative) blockchain.

Filecoin uses about the same energy as a comparably sized data center. Specifically, Filecoin provides about 0.77% of all data center storage capacity across the world and consumes about 0.62% of total data center electricity use.

We're also ramping up our ability to help build new solar generation, including a $38M fund to build new solar projects in the US, to produce at least as much renewable energy for the grid as the network consumes.

More on this here: - https://www.coindesk.com/tech/2021/12/02/filecoin-might-have... - https://github.com/redransil/filecoin-energy-estimation/blob... - https://filecoin.energy/methodology

lucb1e3• 4 years ago

to produce at least as much renewable energy for the grid as the network consumes

So redirecting funds from investors/people and resources (production capacity) that could have gone to make everything renewable, to make your blockchain use renewable energy? Why not just not use a PoW blockchain but instead use DHT, IPFS, and other technologies for decentralization?

skyfaller2• 4 years ago

The biggest alarm bell going off for me when reading these articles is "Notably, the system could also be used by bitcoin to help clean up the largest cryptocurrency’s dirty image, according to Filecoin Green’s creator, Alan Ransil."

This is why I lump all cryptocurrency together, even when some projects consider environmental concerns: you're willing to cover for, greenwash and promote the worst actors. There is no scenario where intentionally wasting energy helps us solve the climate crisis.

That is in part because of one flaw in your reasoning: believing that simply switching to renewable energy will solve the problem. Concepts like Jevon's paradox and induced demand really need to be general knowledge at this point... when technology becomes more energy-efficient, people just use more of it, and total energy use doesn't decrease. https://solar.lowtechmagazine.com/2018/01/bedazzled-by-energ... So long as total energy use keeps rising, adding renewable energy sources won't replace fossil fuels, fossil fuels will continue being used at the same (or growing) rate: https://solar.lowtechmagazine.com/2009/11/renewable-energy-i... And even if you succeeded in moving entirely to renewable energy, that would not be a license to use as much energy as you please, because installing renewable energy still causes emissions and there are hard limits to e.g. how many solar panels we can install each year and still remain within our carbon budget: https://solar.lowtechmagazine.com/2015/04/how-sustainable-is...

Relatedly, when you say "Filecoin uses about the same energy as a comparably sized data center", you're assuming that ordinary data centers are compatible with our continued existence as a civilization. The current internet may not be sustainable, we may already have too many energy-intensive data centers: https://solar.lowtechmagazine.com/2015/10/can-the-internet-r... The time has come for frugal computing, and Filecoin incentivizing people to put more data storage online seems incompatible with that: https://wimvanderbauwhede.github.io/articles/frugal-computin...

Finally, insofar as you're pursuing "carbon-neutral" or "carbon-negative", I'm skeptical of carbon offsets in general. It's the same financialization of everything which is a core problem underlying all cryptocurrency. Gresham's law should also be common knowledge at this point: carbon offsets are a market for lemons where the most meaningless offset will be the cheapest (because it requires doing no work) and real offsets will be sidelined or driven out of business. https://pluralistic.net/2020/12/12/fairy-use-tale/#greenwash... If you don't have a solar panel sticking out of your Filecoin server directly powering it, I don't trust it, I assume it's some sort of shell game. Relatedly, how will I be able to hold you accountable, if every project I use ends up using Filecoin or something like it, so I can't avoid it? At some point, consumer choice will be as imaginary as it is when opposing multinational conglomerates: https://pluralistic.net/2021/03/24/greenwashing/

Let me be clear, this is not an attack on your intentions, I'm sure that you as a person have ethics and there are ethical people working with you who do not wish to destroy the world.

Several months ago, a local low-waste shop was using carbon offsets on their shipping, and I noticed they were using the exact fake Nature Conservancy offset exposed here: https://www.bloomberg.com/features/2020-nature-conservancy-c... I mentioned its uselessness to them, and they thanked me for the info, and continued to use it. I think they mean well, but running their business (which I otherwise like and support) is higher priority for them than navigating the details of carbon offsets, or researching all of their limitations. So they send money to a scam, because that appearance of sustainability makes more business sense for them than the real thing.

"It is difficult to get a man to understand something, when his salary depends on his not understanding it." And so I forgive you in advance for not understanding this, but I won't forget. I think your project has bad effects, no matter how well-meaning you as an individual are.

JeremyNT• 4 years ago

Yeah, sad to see this nonsense from a project I have a lot of respect for. They could just use bittorrent, if distribution were the issue.

eighthave3• 4 years ago

I was also initially skeptical, but after looking into IPFS, I think the core ideas are real and useful. IPFS is inspired by bittorrent, and addresses some of the issues with bittorrent that keep torrenting a very manual process. Debian did experiments with bittorrent mirrors, they worked poorly. I think IPFS is worth trying since it could actually work.

sinkholed2• 4 years ago

Full disclaimer: I have no cryptocurrency assets, nor interest.

Since the “decentralized web” includes all sorts of things, it is important to also mention things that we will not be doing with this grant. We are not looking at NFTs, blockchain, DAOs, smart contracts or related aspects of “Web3”.

This absolutely is Web3, and Filecoin is on a blockchain. This is a great use, though I understand why they wish to distance themselves (Web3 is popular amongst political opponents of the chattering class).

lazzlazzlazz• 4 years ago

It's funny to see a web3 project insisting it isn't one; developers begrudgingly accepting that centralizing a platform creates an infrastructural and economic risk.

Great to see!

jancsika• 4 years ago

Filecoin is built on top of IPFS, which is “a distributed system for storing and accessing files”.
We are not looking at NFTs, blockchain, DAOs, smart contracts or related aspects of “Web3”.

Wow, blockchain is so nearly synonymous with "complicated digital pump and dump scam" that the author has to disavow the very technology they started with in their description. So Filecoin is blockchain, but not, you know, blockchain.

pabs32• 4 years ago

I note that F-Droid are looking for Android and Python contractors, probably related to this IPFS/Filecoin work:

https://www.fossjobs.net/job/10840/seeking-androidpython-con... https://guardianproject.info/contact/android-python-contract...

Decentralizing Distribution

Related Stories