My contribution to XKCD’s #949

http://www.joachim-breitner.de/blog/archives/632-My-contribution-to-XKCDs-949.html

by lelf • 13 years ago

29 55 13 years ago

Looks quite neat :).

billpg• 13 years ago

Ever tried talking a non-techie friend/relative through the process putting a port-forward on a NAT router? Not fun at all.

aroch2• 13 years ago

I'd say UPnP but that's a whole new can of worms (a much bigger one at that).

simfoo• 13 years ago

Even worse since more and more ISPs switch to carrier grade NAT without also switching to dual stack IPv6.

slig• 13 years ago

I believe he's connecting to a SSH server, and the server is proxying the request back to a local HTTP server. So there's no need to mess with port-forwards.

epaga• 13 years ago

"You can already build such a system yourself quite trivially by getting an FTP account, mounting it locally with curlftpfs, and then using SVN or CVS on the mounted filesystem."

https://news.ycombinator.com/item?id=9224 :)

tOkeshu5• 13 years ago

It's a really geeky approach. The 949 problem concerns a lot of persons who does not even know what a server is.

shailesh• 13 years ago

https://news.ycombinator.com/item?id=8863

laumars4• 13 years ago

FTP is just a horrible and massively outdated protocol which we really shouldn't be advocating for the modern internet[1]. Particularly when everything you've suggested can be much more cleanly done using existing tools on Linux / UNIX / OS X: scp, rsync, git, sshfs (some of which are often pre-installed on the *nix image)

[1] http://mywiki.wooledge.org/FtpMustDie

nomeata5• 13 years ago

Note that all these require transferring the file to the server first, before a link can be shared with anyone, and especially cleaning up later. With my tool, the link is instantly available, and as soon as I stop the program, nothing is left anywhere, especially not on the server.

biesnecker• 13 years ago

Didn't Dropbox's share links more or less solve this problem? Maybe not for 100% of the population, but I've managed to walk my mom through the process, and isn't that generally the success metric for things like this?

galaktor• 13 years ago

I recently had to sign up with Dropbox just to download a file somebody was sharing with me.

The way UbuntuOne (and maybe other providers) just give you a link anybody can use is much better imho.

slazaro• 13 years ago

I'm pretty sure they did it wrong then. You can share a link to any file inside of your dropbox, and anyone can download it regardless of whether they have an account.

tOkeshu5• 13 years ago

Dropbox has a storage limit, requires an account and is non free software.

The usage proposed here is a "no-strings" attached approach. Ideally you want something you can trust, hence free software, and that does not store anything.

verroq2• 13 years ago

You can always uses a p2p data transfer with WebRTC. All you need is a browser.

jerf2• 13 years ago

OK, I've got a browser window open right here. Now what?

(Serious question.)

ddorian43• 13 years ago

http://lmgtfy.com/?q=webrtc+transfer+files

RamiK• 13 years ago

https://rtccopy.com/

reconbot• 13 years ago

This still requires a server for orchestration.

qznc• 13 years ago

WebRTC is actually a C++ library [0]. One could implement a "share-file" command-line tool, which uses this library to share files. Other would have to use another tool for "recv-shared-file", which probably takes a hex-id and spawns a local web server or some other gui.

[0] http://www.webrtc.org/reference/architecture

tOkeshu5• 13 years ago

It is way more complicated than that. WebRTC is all great and shiny but there is strong obstacles to build correct data transfer applications on top of that:

- You still need a server to do the "signaling" part

- NAT are everywhere and will fail your p2p connections sometimes (14% of WebRTC calls according to Google, back in may) [1]

- Ultimately there is a problem with JavaScript API that does not allow you to download big files seamlessly

The signaling part can be solved in many ways but the server approach is the easiest and probably the most efficient/reliable one. For more informations I recommend checking an article on the infrastructure needed for WebRTC applications [2]

The NAT problem is solved by ICE via STUN and TURN [3]. TURN is the ultimate fallback, it is a media relay. TURN servers are bandwidth expensive and most of them require credentials in a way that does not fit WebRTC properly.

Finally, there is the browsers problem. None of them provide a proper and standard way to give you the ability to download a stream of data coming from JavaScript (although after checking the issue, sharefest may have solved this problem, but I don't know if it works properly with firefox). There the File API in Chrome but it is not supported in Firefox. There is the blob URI but you need the whole file to create this object. This mean that your application will slow down your browser in the case of a big file sharing [4]. Ideally I wanted to see the stream API be implemented but it seems it's not going to happen, so we need something else [5].

In conclusion, no, it's not as easy as you think it is. To have a proper file sharing application you will need to address ALL of these problems. Moreover, it's good to note that these problem are not specific to WebRTC applications. Indeed, most of these are problems we have with the Internet in general. WebRTC is just an API on top of existing solutions.

I hope this answer was not too negative but informative enough :)

[1] http://webrtcstats.com/first-webrtc-statistics/

[2] http://www.html5rocks.com/en/tutorials/webrtc/infrastructure...

[3] https://en.wikipedia.org/wiki/Interactive_Connectivity_Estab...

[4] https://github.com/Peer5/ShareFest/issues/8

[5] https://bugzilla.mozilla.org/show_bug.cgi?id=891286

EDIT: formating/typos

verroq2• 13 years ago

Having used webRTC with the TURN server REST API (multimedia conferencing app), it's not that bad.

STUN will get though the NAT most of the times. With the latest browsers, Firefox to Chromium is a bit dodgy, but there are workarounds. But Chromium to Chromium is perfectly fine.

puzzlingcaptcha• 13 years ago

On Windows, I run a local torrent tracker (built into uTorrent: bt.enable_tracker true) and share the magnet links.

The disadvantages are that I need to create a torrent file with my ip in announce field for each file I want to share and that the recipient must have torrent software (all of my friends incidentally do).

Perhaps I could simplify the torrent creation using the correct announce field with a nifty right click - 'Send to' script. edit: Looks like it would be possible with a combination of https://github.com/bittorrent/btc , http://www.robertnitsch.de/projects/py3createtorrent and a batch file.

The advantages are built-in checksumming, no intermediate upload server, interoperability with various torrent software, ability to stop and resume the transfer at any time and very good transfer rates (for some reason I've had terrible performance with those browser-to-browser services like e.g. sharefest)

enricopulatzo• 13 years ago

Dropbox solves the problem of sharing a file I have, but on the receiving side of the equation, dbinbox (http://dbinbox.com/) is probably the easiest solution for non-technical folks.

vacri2• 13 years ago

The problem isn't 'sending a file', it's 'peer to peer communication', which is hard without an intermediary and/or shennanigans due to ubiquitous NAT. I image in the everyone-has-an-IP universe of ip6, this will no longer be an issue.

tlunter• 13 years ago

Private networks will always have a place for security and safety of all systems which will still make this a problem.

mjolk2• 13 years ago

ipv6 will still have this issue, because _no fucking way_ will i put a directly-accessible-from-anywhere computer on a network.

the "how do i serve a directory" problem is _so_ solved:

`python -m SimpleHTTPServer`

or:

dropbox (it's not hard to install, as much as xkcd whinges about it) google drive email

This is a solved problem.

falcolas• 13 years ago

directly-accessible-from-anywhere computer on a network

followed by

python -m SimpleHTTPServer

is a bit of a contradiction.

Dropbox has some pretty serious limitations for transferring files; you're limited on file size, you have to upload to their servers, you're subject to their definition of objectionable content, and sharing a particular file too much (for their definition of too much) will cause them to take down the file.

tOkeshu5• 13 years ago

This is a solved problem.

There is strong technical obstacles but yes the problem is more or less solved in different ways.

The point of Randall Munroe here is to emphasis the gap between the fact the Internet made possible a lot of incredible things but sharing a simple file is still a hassle. And IT IS, creating an account is not what you can call a no-hassle process (I would probably say, do not forget about Term Of Services, but I understand no one read them nowadays).

vacri2• 13 years ago

I was being a bit idealistic, but in theory that's the story.

Yeah, xkcd handwaved away a couple of other solutions too: use a chat client to transfer a file? "no-one uses this obsolete one!"; use a file upload site? "I'm scared of porn popups and they don't work properly anyway"

freehunter• 13 years ago

Quick and easy file sharing is hard to do not only from a technical standpoint, but also from a legal standpoint. It only takes one DMCA lawsuit to shut down a file sharing startup. The likelihood of a lawsuit increases as the file sharing becomes easier, quicker, and more anonymous (as the chances that someone will use it for illegal purposes increases the easier and quicker and more anonymous it becomes).

You have to set some restrictions on who can share what with how many people and how often. Even torrent trackers that do not store any files other than the tracker become targets. I really would love it if there was a way to type "scp /home/freehunter/your-granddaughter-laughing.mpg //grandma/home/grandma/videos" with UPnP to get around grandma's dial-up NAT. Unfortunately, the closest we have is Dropbox (requires an account) or BitTorrent (requires dodging law enforcement and porn popups).

lbatista• 13 years ago

it really works: http://www.bittorrent.com/sync

rtpg• 13 years ago

I ran into a "trying to share files" thing recently, and I ran into the "pipe things over netcat" solution (http://superuser.com/questions/98089/sending-file-via-netcat). For techies it's pretty fun.

With things like google docs, I've found that file sharing has become a lot less necessary. We're sort of moving into a very file-less space, and I'm kinda fine with it.

onosendai• 13 years ago

As far as quick 'n dirty solutions go, if you're not overly concerned with security, you can always use "python -m SimpleHTTPServer" to serve all the files on the current directory over HTTP.

I also recall that KDE had a panel applet that pretty much did the same thing on the 3.x days (not sure it's been ported to 4.x)

tOkeshu5• 13 years ago

Do not forget that this solution is inherently mono-threaded and does not benefits from a proper concurrency model like node.js for example (which is also mono-threaded by "design" but provide non blocking IO).

So for big files with multiple downloaders, it can be sometimes not as good as you want it to be.

secstate2• 13 years ago

But at that point you're hosting a file server, not just sending a 25mb file to a friend. Out of scope.

secstate2• 13 years ago

A neat solution indeed, but for anyone capable enough to setup what you've concocted, why not just scp the files on a lowendbox.com VPS. You also benefit from an intermediary with a decent up and down connection for the person you're sending to.

KJasper• 13 years ago

WeTransfer (wetransfer.com) is pretty popular in the Netherlands. It's easy to use, only requires an e-mailadress and allows you to send files up to 2GB and as an added bonus it's quite pretty. I really love it's simplicity.

huskyr• 13 years ago

WeTransfer solves this problem, without needing an user account. I don't know why people wouldn't use it.

laumars4• 13 years ago

So he's built a file sharing tool which requires an SSH server to work? So why not just use sshfs, scp or rsync?

I don't mean to stamp on someone's creativity, but SSH already has a whole bunch of tools for transferring files - all of which already work pretty well.

nomeata5• 13 years ago

Because with file-share, the file is never stored on the ssh-server; it is transferred the instant the other sides starts the download. The SSH-Server is required to share the file at a publicly available address (which my own computer is likely not). So no need to clean up files after the download, for example, or even to have any free disk space.

laumars4• 13 years ago

If you need an SSH server to begin with, then there's still some setting up you need to do before hand. The pain of having to delete a few files afterwards is really nothing compared to renting a Linux box, setting up SSH accounts with fail2ban and/or SSH keys; and so on.

So file-share doesn't even work as a pain free solution like you suggest it does.

As I said before, I don't like to stamp on another's creativity. I'm just saying that there are better tools for doing this which already come pre-installed with most Linux distros, UNIX's and OS X.

skizm• 13 years ago

cough onewaybits.com cough (shameless self promotion)

Supports up to 50mb files for now.

BerislavLopac• 13 years ago

Wow, thanks for pointing me to https://filetea.me

hengheng• 13 years ago

Munroe did not mention Skype. Tons of people have accounts there, and it transmits files like ICQ did back in the day.

brechin2• 13 years ago

I've got a CLI that lets you upload a file through FileTea. I'll clean up the source and post it soon.

See https://github.com/brechin/FileTeaSend Tested with Python 2.6. Only external requirement is requests.

nomeata5• 13 years ago

Great! Guess I’ll ditch my tool sooner than I thought...

brechin2• 13 years ago

Glad I could help!

nomeata5• 13 years ago

Follow up post, summing up alternatives (especially the FileTea command line client): https://www.joachim-breitner.de/blog/archives/634-My-contrib...

My contribution to XKCD’s #949

Related Stories