Stop Using FTP

http://stevenf.com/archive/dont-use-ftp.php

by bdotdub • 18 years ago

65 65 18 years ago

tx4• 18 years ago

Well, I'd gladly switch to sftp, but dumb-ass OSX and Windows don't understand it out of the box and most users are unaware of more powerful (and easier to use) alternative OS, which leaves us stuck with this commercialized mediocrity.

Why wouldn't we also stop using Apple Finder, FAT, IE6, non-UTF encodings and bunch of other legacy crap? Ohh-ho-ho... so many topics to blog about...

crescendo2• 18 years ago

OS X does support sftp out of the box. Also, it can act as an sftp host if you enable Remote Login (ssh).

tlrobinson• 18 years ago

From the command line, yes, but not the GUI. When I do command-K in the Finder and enter a sftp://whatever I get this message:

URLs should begin with afp://, at://, file://, ftp://, http://, https://, nfs://, smb://, cifs://, or vnc://

It's easy enough to get a nice SFTP client though (I prefer Transmit. It even supports Amazon S3)

aasarava• 18 years ago

For Windows, you can get WinSCP pretty easily. It supports SFTP and, obviously, SCP.

kirse• 18 years ago

Good and simple program. I used to use WinFTP_LE but this one is where it's at for free Windows FTP tools.

froo2• 18 years ago

I'd say that the reason why people aren't switching so readily is actually a function of the human condition moreso than any technical reason.

Looking at other industries, like health care, people are more content in spending effort/money on "miracle cures" than worrying about taking steps towards preventing these problems from happening.

The exact same thing happens in say, the Automotive industry - a large percentage of consumers dont take active measures towards ensuring their vehicle is in good running order, and are more content with dealing with the consequences.

The same thing seems to happen when talking about computer security. I dont think it's because people dont care about computer security, they're just more ready to accept the consequences rather than taking preventative steps.

It's a real damn shame too.

rw• 18 years ago

"people are more content in spending effort/money on 'miracle cures'"

... or businesses are just more interested in researching products that have obvious profitability. Let's say you spread the gospel of eating healthy foods. Where is your income coming from?

tx4• 18 years ago

What? It's MUCH simplier than than: I can't send a Windows/OSX user an email with an SFTP version of this line:

ftp://server.com/report.pdf

It's that simple.

bprater• 18 years ago

I'm not sure what out of the box means. Good clients like Coda and YummyFTP support SFTP.

SwellJoe11• 18 years ago

Note that "SFTP" has no single meaning. Depending on the client, it may mean "FTP encrypted with SSL" or "FTP with encryption extensions of one of a couple of types invented independently by FTP server authors over the years" or "SSH File Transfer Protocol" (which is actually the real meaning of the abbreviation, and was in use for several years before secure FTP began to be common).

Historically, people tried to convince folks to call it FTPS (for FTP over SSL, which is the most popular form of secured FTP), but that seems to have never taken off. Unfortunate, since SFTP means too many incompatible things.

It's stabilized somewhat in recent years, and usually clients will do the right thing--and the most common meaning of SFTP has come to be "FTP encrypted with SSL" (which is actually one of the weaker forms of security for FTP--but it's got some semblance of "standardization", and works in lots of clients and servers, so that makes it superior). And, of course, there is no RFC defining secure FTP, so it's a de facto "standard" which can lead to buggy implementations and a lot of finger pointing--server says client is wrong, client says server is wrong, user says "I give up" and uses FTP, instead.

nuclear_eclipse3• 18 years ago

On that note, from a Linux system, what does `sftp` use on the command line? I had always assumed that SFTP was always and only SSH transfers, just like using `scp`. Is this wrong of me to think this way?

jrsims2• 18 years ago

'man sftp'

SwellJoe11• 18 years ago

I had always assumed that SFTP was always and only SSH transfers, just like using `scp`. Is this wrong of me to think this way?

No, this is absolutely correct. But don't be surprised if someone misinterprets you.

jimbokun• 18 years ago

"I had always assumed that SFTP was always and only SSH transfers, just like using `scp`."

Which brings up what I was wondering, what use cases do SFTP/FTPS/FSTP etc. satisfy that a combination of scp and rsync do not? Maybe better for GUI client and file browser interfaces?

attack2• 18 years ago

Filezilla on windows assumes sftp to be SSH transfers. So I think you've got that backwards.

SwellJoe11• 18 years ago

Filezilla on windows assumes sftp to be SSH transfers. So I think you've got that backwards.

What, exactly, do you think I have backwards? I just tried to explain how wildly diverse the meanings are, and how people use the terms. I don't care what your favorite FTP client means when it uses the term SFTP (bully for you, that you happen to use one that uses the term correctly)...I'm telling you that there is little agreement in the wild about what the term means, and many people use it incorrectly to refer to FTPS (and other kinds of secured FTP).

nailer4• 18 years ago

I disagree. SFTP means SFTP. FTP over SSL is a different thing.

Since you haven't provided references, I will - look at Adobe Dreamweaver or other web based clients. SFTP is assumed to mean SFTP as implemented by an SSH server.

Traditional FTP uses two ports and it's a bitch to run via SSL - as a result, it's extremely rare.

PS. Moderated the article down for "This is going to sound a little weird at first considering what I do for a living, but I want you to stop using FTP."

It's 2008. You should have stopped using FTP in 1998.

SwellJoe11• 18 years ago

You disagree with what? I thought I was quite clear that the appropriate meaning of SFTP is SSH File Transfer Protocol and has been for years. I certainly didn't think I was arguing that people ought to use it to mean anything else.

Many users (and a few clients and servers) do use the term incorrectly. I don't like it, and clearly you don't either...I don't see what you're disagreeing with.

And, of course FTP over SSL is not extremely rare...not even close to rare. All of the major FTP servers, and many FTP clients, support it quite easily, and many, many end users insist on it. ProFTPd, possibly the most popular FTP server, can be configured to use it with just a few configuration directives.

I'm not encouraging anyone to use the terms incorrectly or to use the FTPS protocol instead of SFTP. In fact, I've been recommending SSH FTP for years (one could probably dig up mailing lists posts that include me scolding people for using FTP from ten years ago).

I guess I've not been very clear. I apologize for misleading anyone into thinking I intended to praise the misuse of terms. I just wanted to point out that people use the terms incorrectly extremely often, and it's wise to be explicit about the protocol you mean.

aaronblohowiak• 18 years ago

this is a matter of right tool for the job. for important stuff, everything should happen over a secure tunnel (ssl, ssh port tunneling or just using scp.) ftp is good for public file download/drop box where you want to easily support resume.

emmett2• 18 years ago

Almost all his arguments apply equally to HTTP.

anthonyrubin• 18 years ago

- Unless tunneled over a secure socket, FTP is 100% insecure.

This is true for HTTP.

- The spec defines no way of setting the modification dates/times of files.

The HTTP RFC defines the Last-Modified header.

- The spec defines no reliable way of determining the string encoding used for filenames.

The HTTP RFC defines several headers related to encoding.

- FTP was designed to be used interactively by a human sitting at a terminal, not by a GUI application working on the human's behalf. The spec doesn't even define the output format that should be used for directory listings.

This problem doesn't translate exactly to HTTP as it isn't used in the same manner. Directory listings are typically sent as HTML and a user clicks a link which all browsers are able to handle trivially. WebDAV adds features similar to FTP, but it is much more standardized.

- The spec defines no way of dealing with file metadata, such as Unix permissions, owners, and groups.

There are HTTP error codes which indicate lack of permission, but there are no RFC-defined headers for these things. I believe WebDAV does define a method to deal with them.

- FTP requires a minimum of two socket connections to transfer a file: the control connection, which is established first, and then data connections which are created and destroyed every time you transfer a single file, or request a directory listing.

HTTP has no such requirement. Keep-alive prevents sockets from being closed and the same connection can be used to get a directory listing and then transfer a file. There are no separate data and control connections.

- FTP is not friendly with firewalls. Because it constantly needs to establish new connections, this has led us to "passive mode" which might as well be black magic as far as most people are concerned.

This is FTP's biggest flaw and this problem absolutely does not exist with HTTP. Once it is encrypted FTP is impossible to handle in a reasonable manner with a firewall.

In active mode FTP creates one connection to a well-known port for the control channel (21). For the data connection the server attempts to establish a connection from a well-known port (20) to a high-numbered port on the client. This obviously doesn't work well with a firewall in front of the client.

Passive mode modifies this arrangement so that the data channel comes from the client to the server. The problem is the ports used. On both ends the port will be a high-numbered. This requires allowing connections from any high-numbered port (withing a large range) to any high-numbered port. Some firewalls understand the FTP protocol and are able to see the port that is to be used; the server gives the client this port with a command in the control connection. The firewall can then allow that specific connection. Once the control connection is encrypted this is impossible.

Now then. How do his arguments apply equally to HTTP?

emmett2• 18 years ago

"Almost all" might have been hyperbole, but "many" is certainly true.

On the other hand, HTTP is clearly far superior to FTP, especially if you include extensions like WebDAV and the power of custom headers.

I was just pointing out, insecure formats that don't define how to do listings or set mod times or deal with Unix permissions can be quite good. FTP sucks mostly because of the ports/multiple connection issue, as you pointed out.

paul• 18 years ago

Not really. FTP is an awful protocol, and as he mentions it doesn't really even have a spec. HTTP works quite well and has proven to be surprisingly flexible.

gaius3• 18 years ago

You know, back in the day, there was an idea called "opportunistic encryption". How it worked was, you would put a FreeBSD box running S/WAN at the edge of your network, and if it detected a similar box at the other end, they would seamlessly encrypt your stream (and if not, just let it straight through). Now, if this idea had gotten the traction it deserved, we could all happily be using telnet, FTP, whatever, rather than everyone having to be aware of encryption, your SA would just sort it once, for everyone, and they'd keep on using familiar tools.

notauser• 18 years ago

Opportunistic encryption is generally a good idea, but as it has no feedback method you can't really be certain if it is working or not. The seamlessness works both ways.

gaius3• 18 years ago

Right, but the current situation is silly. For example right now if I want to connect into work, I fire up the Cisco VPN client, then I ssh from my Mac to whatever Unix box I want to work on there. I'm not sure that "double encryption" really buys us anything here. Unless someone's actually in our datacentre tapping the wire coming out of the VPN host, in which case we've bigger problems...

deathbyzen• 18 years ago

I'd definitely leave age out of the argument that FTP is bad seeing as how plenty of people use old programming languages, old OSes, and old software for the simple fact that it "just works."

kylec2• 18 years ago

I'd like to use SFTP or scp for everything, but I've discovered that it's considerably slower than regular FTP. I wish that there was a way of encrypting the login but sending everything else via plain FTP for speed.

SwellJoe11• 18 years ago

You can do that with WebDAV and Digest authentication. I don't actually know that it's comparable in speed, though.

I'd also question the speed argument...for large files, SSH can perform compression and make the transfer faster than FTP.

kylec2• 18 years ago

Thanks for the compression tip! My extremely informal testing produced the following results taken by copying a random binary file I had to another computer:

SSH: 2.0MB/s

SSH w/ compression: 2.5MB/s

FTP: 2.8MB/s

While SSH does not approach the speed of FTP it does come close, perhaps close enough to finally ditch FTP altogether in most situations.

pon• 18 years ago

I think using encryption limits the size of the files to about 2 GB in current versions. It is too little for DVD images.

newt0311• 18 years ago

In what product? I have frequently transferred files larger than 2GB on SFTP as well as storing them with encryption on my computer and there is no 2 GB limit.

SwellJoe11• 18 years ago

Simply not true.

The openssh family of tools have had LFS support for many years. I've used scp several times to copy huge files...and our products have scp as a filesystem backup target, which means it deals with archives that are dozens or hundreds of GBs in size on a daily basis.

kmt• 18 years ago

sftp is not ideal; looks like the author is not aware that ssh (which is what sftp uses) is rather slow on WAN due to buffer issues:

http://www.psc.edu/networking/projects/hpn-ssh/

http://www.psc.edu/networking/projects/hpn-ssh/papers/i2mm-h...

tptacek6• 18 years ago

SFTP is its own debacle. It's a handy alternative to "scp" when you don't want to open a second connection to grovel through "find" on the target system, but it's even less firewall-friendly than FTP: you can't enable users to grab files with SFTP without also enabling them to SSH to remote hosts.

Most enterprises don't allow outbound SSH through their firewalls, in part because SSH is its own slippery slope: if you're allowing SSH, you're allowing people to build VPN tunnels.

The real-world operational problem that FTP solves is upload. There is no download application that HTTP doesn't solve better than FTP. If Apache shipped with a mod_upload with a reasonable configuration system and a simple, workable web interface for specifying what to put where, FTP could be laid to rest.

LogicHoleFlaw• 18 years ago

I have seen enterprise systems where sftp access is granted to third parties by restricting the login shell severely to allow only sftp uploads. Is there a problem with that approach? SOX / PCI is making the 'secure upload problem' more pressing daily.

tptacek6• 18 years ago

Yes, for instance: X11 and port forwarding.

The bigger problem is, it's just hard to know what else you're allowing once you let someone "only" access SFTP via SSH.

Most of the BigCo's I work with use "Secure File Transfer" software to solve this problem, which are just glorified upload CGI scripts under SSL. That's the best solution, I think.

Stop Using FTP

Related Stories