Ask HN: Why don't more software projects use BitTorrent for downloads?

https://news.ycombinator.com/item?id=30436367

by nerdponx • 4 years ago

90 91 4 years ago

news.ycombinator.com

It seems like it would be great for open source projects with limited funding to use BitTorrent for downloads. However I only see a handful of projects using this model, e.g. LibreOffice and GIMP. Whereas a lot of Linux-based OSes just host their disk images over HTTP, which I imagine could incur substantial hosting costs over time.

Why don't more software projects do this? Are there any big downsides to distributing torrents as opposed to traditional FTP/HTTP downloads?

linkdd• 4 years ago

When distributing over torrent, you need at least one seeder with a good connection. You obviously. So you still need to host a server with your package on it, and configure a torrent client to seed it.

I know a lot of HTTP/FTP storage space providers, or I can just put my binary on Github (Github Pages, or Release asset), or host it on a CDN like netlify. I think there is more solutions than real numbers between 0 and 1.

But I don't know a single "torrent seedbox" provider. Setting it up yourself is not complicated, but since it's DIY, this alone explains why it's not more wide-spread.

charcircuit• 4 years ago

You don't need a seedbox. You just make the torrent file reference the normal HTTP download.

pabs37• 4 years ago

There is a way that you can add a URL to a torrent file as a seed location, then you always have a backup seed for when there are no torrent seeds.

https://en.wikipedia.org/wiki/BitTorrent_%28protocol%29#Web_...

kelseyfrog• 4 years ago

Can't you use S3 to seed a torrent?

https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjec...

miked85• 4 years ago

They seem to have deprecated support [1]

[1] https://news.ycombinator.com/item?id=27524549

banana_giraffe3• 4 years ago

As mentioned, S3's torrent server is no longer a thing.

When it worked, it also was never that great. The WebSeed AWS ran was slow, the tracker was always problematic as well.

PaulHoule2• 4 years ago

It's not really worth it for small downloads. I looked at one of my projects and it is an 880kb download.

If you are talking a gigabyte or more there is a big advantage in terms of (1) a completely reliable download, and (2) being able to split up the download into multiple streams. At home I have two ADSL lines and a load balancer and with bittorrent I get double-speed downloads.

stevenwoo• 4 years ago

Why don't large companies use it for updates? I have a lot of trouble downloading the 10+ gigabyte IOS downloads on DSL for some reason but torrenting works fine for larger files and stopping and resuming does not break torrenting unlike Apple's update process.

PaulHoule2• 4 years ago

I don't know how they do it but when I download from Steam there are multiple streams and I get about 200% of the throughput of a single ADSL.

jaybeavers2• 4 years ago

Taxes killed p2p...

https://youtu.be/5-sF7N1bfv0

jazzyjackson• 4 years ago

iirc Windows does have a way that computers on a shared network can share the update with each other.

https://support.microsoft.com/en-us/windows/windows-update-d...

retrac• 4 years ago

Some have. Blizzard used a variant of BitTorrent about 10 - 15 years ago to distribute the multi-gigabyte patches for their online games. Peak demand was very high with hundreds of thousands of users all on the same day.

It was built into the game installer / launcher. I think they've switched away from that since. I remember it causing some consternation at the time given some people pay by the byte for network service and felt it was unfair.

llui85• 4 years ago

Musescore is planning to use P2P sharing to download sample libraries (>1gb) and updates in version 4 (coming in the next few months).

Terry_Roll2• 4 years ago

Its redundancy. Just because the download is small, doesnt mean your webserver providing the downloads doesnt go down.

toast0• 4 years ago

Sure, but if the webserver providing the downloads goes down, usually so does the website providing the torrent link.

ralusek• 4 years ago

The best downside that I've heard, at least for doing this automatically, is that not all users have cheap/available upload bandwidth. As long as you're able to opt in as a seeder, it's great.

garbagecoder• 4 years ago

Until everyone has synchronous internet it won’t need to be a thing. I’ve pointed out many times that between BitTorrent, tor, and bitcoin we’ve already done alleged web3, which is what we used to call the Dark Web.

unforeseen9991• 4 years ago

Plus a whole number of other distributed projects over the years. Everything web3 is just hyped ideas that are already out there, in some cases for 10+ years.

betwixthewires• 4 years ago

That's a good point, you're right. I hadn't thought of it this way. What we want already exists, we just have to use it.

wmf• 4 years ago

There's a ton of free HTTP mirroring available for open source; BitTorrent really has no advantage. Bandwidth is so cheap in general that BitTorrent is not needed.

cjbprime• 4 years ago

For the least friction, you could make the download page use Webtorrent, which is a BitTorrent bridge using WebRTC datachannels.

Froedlich• 4 years ago

A lot of ISPs nuke anything that looks like a torrent and send angry messages to their customers.

amir734jj• 4 years ago

Can confirm this.

copperx• 4 years ago

Country? Provider? Type of download?

LeoPanthera2• 4 years ago

Is that actually true? Because I thought it was "content" owners hunting for torrents containing their stuff and then sending DMCA complaints (or local equivalent) to the ISPs of everyone in that particular swarm.

Are there really ISPs that just blanket block all torrents? I've never used one that does.

Shared404• 4 years ago

There's some local to me that don't necessarily block torrents, but will get upset.

Mostly just means I set up a VPN through Linode or whatever.

banana_giraffe3• 4 years ago

And some corporate environments might be ok with non-business uses of Internet data, but uncomfortable talks will be had if they detect BitTorrent.

malermeister• 4 years ago

Blizzard did this for a while for their WoW updates [0]. Not sure if they still do though.

[0] https://wowpedia.fandom.com/wiki/Blizzard_Downloader

pabs37• 4 years ago

Unfortunately web browsers don't support it, so it requires an external client to download.

TheFreim• 4 years ago

The Brave browser has a BitTorrent client built in, though I've never used it. Not a major player in the browser market, though.

chc4• 4 years ago

There is WebTorrent, which uses WebRTC. instant.io uses it for file sharing, for example.

pabs37• 4 years ago

I got the impression that WebTorrent users can't download from BitTorrent users and vice versa, is that true?

belltaco• 4 years ago

That and blocked on many firewalls, especially corporate ones.

Also, Opera used to support torrent downloads.

1over137• 4 years ago

Unfortunately web browsers don't support it, so it requires an external client to download.

I guess I'm old, but it used to be common to - gasp! - install multiple applications. Now, the browser is basically everything.

pabs37• 4 years ago

To be clear, I'm old too, vastly prefer applications rather than web browsers.

smt88• 4 years ago

There's no reason to be condescending. Adding download friction for $0 of benefit for a developer makes no sense. I have had a BitTorrent client installed for more than 20 years, and I use it perhaps once a year these days (and only to download old video games that I can't get anywhere else).

saboot• 4 years ago

It's been so long since I used a torrent that I don't know what program I'd use. I last used mutorrent but read they were putting crypto mining code in their client.

mobilio• 4 years ago

Brave support that native.

Try it.

pabs37• 4 years ago

A lot of Linux based OSes also have BitTorrent downloads.

badrabbit• 4 years ago

How is it better than HTTPS+CDNs both performance wise and implementation/maintainance complexity wise?

gbraad• 4 years ago

Blocked or otherwise impeded in some places: eg, unable to request peers from a tracker.

HellsMaddy• 4 years ago

GitHub/GitLab releases makes it easy, free, and has virtually no overhead to distribute software to millions of users globally. Setting up torrents and making sure they stay seeded costs time and money, and torrents are less convenient for most users.

nhoughto• 4 years ago

Yep came to say the same, because a lot of software is already hosted in a lower friction way by GitHub and similar. So unless you have a need/want to host it yourself and maybe benefit from the optimizations then why bother?

dagmx• 4 years ago

Those aren't necessarily mutually exclusive though. The Git hosts could also provide their releases by seeding them as an option.

sodality2• 4 years ago

You can add the web URL seed to the torrent, pretty cool feature.

newaccount74• 4 years ago

We've had a few users complain that they had issues downloading large files from Github -- either S3 was blocked alltogether (China) or just very slow (Australia). That was a couple of years ago, don't know what the situation is like today.

I think offering BT as an alternative would be a good idea, especially for downloads > 100MB.

d--b• 4 years ago

Raspberry pi OS has both.

Regarding hosting, I am not exactly sure that it changes anything. You still have to seed the torrent, so the file needs to be hosted somewhere. What is reduced is the bandwidth, but you rarely pay for bandwidth.

I do like the torrent option, but it’s a little more work for the person who downloads.

psuter• 4 years ago

Facebook reportedly was using Bittorrent to distribute their own application across their servers. I remember outreach presentations where they mentioned this over 10 years ago, and this article seems to have some corroborating details: https://arstechnica.com/information-technology/2012/04/exclu...

soperj2• 4 years ago

This is something I've wondered specifically for Musescore 4. They're releasing their Orchestral library as a separate 7.5gb download, but instead of using bit-torrent, they're making you download a download manager. Seems so bizarre.

bombcar• 4 years ago

Could the download manager be a torrent client in disguise?

soperj2• 4 years ago

Why would they do that though?

cuteboy19• 4 years ago

If it's not p2p then there is no point to bittorrent

LeoPanthera2• 4 years ago

Windows 10 claims it distributes updates in a peer-to-peer manner. Or at least it used to. Does that still happen?

waste_monk• 4 years ago

Delivery Optimisation? Yes, that's still a thing.

It doesn't have to be internet based, you can turn it down to just peers on your local network, or disable it entirely (if you have WSUS infrastructure and no need for it).

u801e• 4 years ago

Maybe not for individual software projects, but I would like to see something like this for installing and updating packages via the OS's package manager. Given the fact that package repositories have mirror servers, it should definitely be possible.

perihelions2• 4 years ago

Sounds like something that would have significant security impact, wouldn't it?

Jtsummers• 4 years ago

BT is resistant to content spoofing. So unless the users download arbitrary .torrent files from untrusted sources, it will be hard to inject malware this way. You'd need to find hash collisions for the contents of the torrent. It would have some privacy implications, as you end up announcing "I'm using this OS!" to your peers (well, it's a high probability that you are at least).

banana_giraffe3• 4 years ago

I suspect a big part of the reason is "more work for very little gain" since most users will just download over HTTP given the choice.

I'd love to see some stats from somewhere like Internet Archive that offers both, it'd be great to be proven wrong, but I suspect a number basically 0% after rounding of their data is downloaded via peer-to-peer BitTorrent traffic.

shiado• 4 years ago

Many ISPs give customers much better download speeds than upload speeds. I remember personally having problems using bittorrent and seeding too much destroying the network for all using it.

WatchDog• 4 years ago

There are so many free CDNs and mirrors, especially for hosting open source. Also most open source projects aren't particularly bandwidth hungry, why bother?

gorgoiler• 4 years ago

Spotify used to use BitTorrent, or something closely related.

Back in the early days, colleagues in my office would be falsely accused by IT of file sharing, when the only crime they were really committing was listening to Jack Johnson.

crtasm• 4 years ago

If that was the very early days they were quite possibly sharing unlicensed tracks: https://torrentfreak.com/spotifys-beta-used-pirate-mp3-files...

jldugger2• 4 years ago

I'm pretty sure OSS gets a free ride on mirrors, because of peering agreements: if traffic between networks is equal, nobody has a case that they're the stronger party on negotiations, so no money changes hands. If there's an asymmetry, theres' an argument that one side has more demand or need for peering and use that as leverage to demand payment. So smaller ISPs that want to balance out a lopsided consumer traffic, will set up mirrors to protect themselves from this.

Since OSS is free to redistribute, a loose federation of mirrors has set up to better match ISPs who want to send more bytes with projects that need to send a lot of bytes.

Are there any big downsides to distributing torrents as opposed to traditional FTP/HTTP

Analytics is the big reason Mozilla shut down their community FTP mirrors, and bit torrent doesn't really solve that portion.

pugworthy• 4 years ago

Valve hired Bram Cohen back in 2004, and AFAIK it is/was core to Steam's distribution system.

bbulkow• 4 years ago

Bittorrent clients for windows and mac are generally malware infested pits of adware. Wherever i try to be virtuous and use one, i have to clean whatever i installed. I am sure i am not the only one.

Why not build your own install using the bittorrent libraries? The security model is more complicated, for one. If i was an architect and proposed that, i would have to understand the attack vectors and present why they are than the well known solution. And taking control of the software update channel is a massive risk.

Second, i have less control of the user experience. With http, i can pay for the right amount of bandwidth, or time on a cdn. If i implement bittorrent, i still have to have buy capacity, i just have a more complicated model for how much to buy.

Suggested updates can be spread over time - and need to be, for canary purposes. I think Android often pushes an app update over 4 or 5 days, by default? Steady state infra capacity, or even better, low priority which and be interrupted, is cheap.

Given the complexity and business risk (people can't download our software! Our binary got hijacked! Two code paths to test?) And the inexpensive nature of mirrors, and the competence of cdns, there are rarely causes where it would make sense.

Mandatum• 4 years ago

Deluge, qBittorrent, Transmission, oh my!

AlchemistCamp• 4 years ago

Blizzard has used BT for binary downloads for years. None of your fears came to pass.

alophawen• 4 years ago

Linux distros/archive.org and other legit users of BitTorrent tend to use HTTP seeds [1], so you can still calculate for bandwidth costs.

Commercial software like World of Warcraft used BitTorrent in their updater for years [2].

1: https://www.bittorrent.org/beps/bep_0019.html

2: https://wowpedia.fandom.com/wiki/Blizzard_Downloader

kjeetgill• 4 years ago

A few companies I've worked at have used torrents to roll out new service binaries (multi-gb tarballs of Java jars) and periodic index deployments using torrents.

The biggest advantage is just diffusing a thundering heard without increasing latency considerably.

Shout-out to tTorrent.

lerigner• 4 years ago

Twitter using BitTorrent on the Backend (2010):

The project uses BitTorrent technology and is called Murder

https://www.bittorrent.com/blog/2010/02/09/twitter-using-bit...

fxtentacle• 4 years ago

A) If you don't use the cloud, bandwidth is so dirt cheap that nobody cares.

B) Some ISPs will throttle you heavily if you upload too much.

C) Some users still have to pay fees for upload traffic.

ksec• 4 years ago

Without Browser Support BT Requires additional client.

which I imagine could incur substantial hosting costs over time.

And bandwidth is extremely cheap. ( Look at Cloudflare ) Lots of sponsorship and help for Open Source project or if you want to host it on Github for free. Bandwidth is also getting cheaper. May be there is a floor somewhere, but we dont seems to have that in sight.

wink2• 4 years ago

Because I dislike it so much as a user that I never got into it and never would have thought to propose it.

In Germany, before Fibre (and lol who has that), upload speeds for DSL and Cable have usually been only 10-20% of download speeds, and I know a lot of people who have stayed away from it for that reason alone. Yes, you can limit it, but it kinda kills some of the whole thing. Also the hassle with port forwarding etc.pp. I think I didn't have a client installed on any machine for the last 10 years.

Pooge2• 4 years ago

upload speeds for DSL and Cable have usually been only 10-20% of download speeds

Why or how is that an issue? Anybody that contributes to the seeding, however small, matters. Because files are cut into a lot of pieces, there is no competition for the "faster upload speed".

Also the hassle with port forwarding

That, I can understand, but it's fairly easy to open a port. And if your router supports it you can enable UPnP.

I am the exact opposite of you: as I user, I absolutely love it and it has to be one of my favorite piece of technology.

wink2• 4 years ago

Why or how is that an issue? Anybody that contributes to the seeding, however small, matters. Because files are cut into a lot of pieces, there is no competition for the "faster upload speed".

If it's a localized thing, for example a game client in your language, and everyone has shitty upload speed, the critical mass would need to be 10x as big so that not everyone has a shitty download rate. Maybe a few single cases soured me on the experience.

port forwarding.

it's easy if you do 'port X to machine Y' but like in the above example, try 3 computers in the network who need the same 20GB download. it sucks. (esp if it's not /a file/ you can then locally distribute, but it's a game company's launcher)

Ask HN: Why don't more software projects use BitTorrent for downloads?

Related Stories