Shutting down public FTP services

Re:Shutting down public FTP services by jhbadger on Tuesday April 25, 2017 @08:31PM (#14199525)

It really isn't as convenient if you have to download lots of files at one time though. FTP has mget. That's probably why FTP lives on for scientific data (NCBI, ENSEMBL, etc). Yes, you could use some tool like wget or curl to spider through a bunch of http links, but that's more work.

[ Parent | on HN ]

Re:Shutting down public FTP services by justinsaccount on Tuesday April 25, 2017 @10:02PM (#14199923)

> FTP has mget

Not quite, ftp CLIENTS have mget. The ftp protocol has absolutely no awareness of mget. In fact, ftp is terrible at downloading more than one file at a time because it has no concept of pipelining and keepalive, both things that http supports.

With a nice multi protocol client like lftp, http directory indexes work just like an ftp server:

  $ lftp http://http.debian.net/debian/
  cd: received redirection to `http://cdn-fastly.deb.debian.org/debian/'
  cd ok, cwd=/debian
  lftp cdn-fastly.deb.debian.org:/debian> ls
  drwxr-xr-x  --  /
  -rw-r--r--         1.0K  2017-01-14 10:44  README
  -rw-r--r--         1.3K  2010-06-26 09:52  README.CD-manufacture
  -rw-r--r--         2.5K  2017-01-14 10:44  README.html
  -rw-r--r--          291  2017-03-04 20:08  README.mirrors.html
  -rw-r--r--           86  2017-03-04 20:08  README.mirrors.txt
  ..[snip]..
  lftp cdn-fastly.deb.debian.org:/debian> mget README*
  5315 bytes transferred
  Total 5 files transferred
  lftp cdn-fastly.deb.debian.org:/debian>

[ Parent | on HN ]

Re:Shutting down public FTP services by anc84 on Wednesday April 26, 2017 @05:22AM (#14201635)

For anyone else wondering how to recursively download with this:

    lftp cdn-fastly.deb.debian.org:/debian> mirror doc
    Total: 2 directories, 43 files, 0 symlinks                             
    New: 43 files, 0 symlinks
    1031755 bytes transferred in 1 second (678.7 KiB/s)
    lftp cdn-fastly.deb.debian.org:/debian>

Warning, -R means reverse (upload!), not recursive. ;)

[ Parent | on HN ]

Re:Shutting down public FTP services by Spiritus on Wednesday April 26, 2017 @02:25AM (#14200895)

Wow, I had no idea lftp had that feature. That's super cool.

[ Parent | on HN ]

Re:Shutting down public FTP services by justinsaccount on Wednesday April 26, 2017 @09:18AM (#14202674)

lftp has a ton of features. background jobs, tab completion, caching of directory contents, multiple connections, parallel fetching of a SINGLE file using multiple connections.

Yes, it looks like '/usr/bin/ftp' from 1970, but it's far far far more advanced than that.

[ Parent | on HN ]

Re:Shutting down public FTP services by Symbiote on Wednesday April 26, 2017 @02:08AM (#14200828)

It would make more sense to offer rsync (unauthenticated), to ensure integrity of what's transferred.

  rsync -r rsync://... ./

will retrieve everything in a directory.

[ Parent | on HN ]

Re:Shutting down public FTP services by mgbmtl on Tuesday April 25, 2017 @09:36PM (#14199827)

I suppose you can sometimes do `mget x.csv` on FTP (if the client supports it?), but with wget you can do:

wget -r -A 'x.csv' https://example.org/

(where 'x' is an asterisk, but HN's formatting eats it)

More work, in the sense that it's more command line options to remember, I agree, but otherwise it's easier to integrate in scripts and much more flexible than mget.

(I don't miss FTP for the sysadmin side of maintaining those servers.)

[ Parent | on HN ]

Re:Shutting down public FTP services by _JamesA_ on Tuesday April 25, 2017 @10:14PM (#14199963)

Download managers such as Down Them All! [1] for Firefox are more convenient (and useful) than mget in an ftp client.

[1]: http://www.downthemall.net/

[ Parent | on HN ]