News for hackers. Stuff that matters.
Re:Shutting down FTP services by geofft on Saturday January 28, 2017 @05:45PM (#13509981)
Why is this design better than the HTTPS design, where you make an (integrity-protected) request for a file and you get a (framed, integrity-protected) response for that file over the same TCP connection?

I understand the argument for peer-to-peer gaming, but I don't see why for file transfer, FTP's design is better.

The big difference I see is that a separate TCP connection has separate flow control and avoids head-of-line blocking for multiple simultaneous transfers, but for static file transfers, the only cause of head-of-line blocking should be actual congestion, which would impact every connection simultaneously, right? (That is, it's better for the Internet to route all traffic between two hosts, where neither side is CPU-bound, over a single TCP connection so that TCP can do its thing.)

And certainly active mode (server connects back to client) doesn't seem like it has any benefits to congestion or anything else: once a connection is established, it doesn't matter who established it.

Re:Shutting down FTP services by zrm on Saturday January 28, 2017 @07:59PM (#13510905)
Imagine you're not bandwidth-limited and you want to transfer multiple files. One is on Disk A, one is on Disk B, one is on NFS, etc. Sending them all at once exploits parallelism, and the transfer on the faster disk doesn't get stuck behind the transfer on the slower disk.

One file per connection also makes the implementation simpler. If you want to use one TCP connection for multiple transfers then you need some kind of a header and logic to determine start and end of file, and new control messages get delayed in queue behind already-buffered data. If you have a data connection for exactly one file then you just call sendfile() and then close().

And transferring a file with active mode FTP looks like this:

  Client -> Server: I want to transfer a file, connect to 1.2.3.4:12345.
  Server -> Client: TCP SYN to 1.2.3.4:12345
Passive mode requires another half trip latency:

  Client -> Server: I want to transfer a file, passive mode.
  Server -> Client: OK, connect to 4.3.2.1:54321.
  Client -> Server: TCP SYN to 4.3.2.1:54321
A different protocol could avoid the extra latency by having the server always use the same port, but that makes the implementation more complicated again. The server could get a data connection from the client before it gets the associated control message. Now it has to deal with matching them up, another timeout in case the control message never arrives, etc.

It's possible to run FTP over TLS. Hardly anybody does that because it makes the NAT problem worse, because then the NAT device can't even snoop the control connection and map the port. FTP programs could fix this by mapping the data port using something like NAT-PMP or PCP, but I'm not aware of any that currently do.

Re:Shutting down FTP services by comex on Saturday January 28, 2017 @10:18PM (#13511752)
Unfortunately, these days crypto really is a mandatory feature, and sendfile doesn't work with TLS anyway. (IMO it should - secure communication should be an OS-level service rather than requiring each application to figure it out, just like the OS provides TCP rather than requiring each application to implement reliable connections - but that's another story.)
Re:Shutting down FTP services by geofft on Saturday January 28, 2017 @10:26PM (#13511804)
Hmmm. sendfile is great because it lets the kernel arrange for DMA straight from disk to the network card, in addition to bypassing context switches and copies to and from userspace buffers. If the kernel gains an SSL stack (the encrypted version of the file is going to be different for each connection), you can't DMA and you'll need to make copies into kernelspace buffers to do the encryption on. Is skipping the context switch enough to make a meaningful performance different?
Re:Shutting down FTP services by feld on Saturday January 28, 2017 @11:07PM (#13512015)
Yes. And I believe Solaris was the first to discover this. Here's a paper Netflix put out covering their first attempt. Measurable gains, hopefully more coming in the future.

https://openconnect.netflix.com/publications/asiabsd_2015_tl...

edit: check this post first

https://news.ycombinator.com/item?id=12037748

Re:Shutting down FTP services by zrm on Saturday January 28, 2017 @11:31PM (#13512133)
If you do it that way you could add hardware encryption support to the network card and be back to using DMA.
Re:Shutting down FTP services by geofft on Sunday January 29, 2017 @06:45PM (#13517558)
Oh, that's fascinating and awesome! Thanks for the links.
Re:Shutting down FTP services by lomnakkus on Sunday January 29, 2017 @05:32AM (#13513516)
> Is skipping the context switch enough to make a meaningful performance different?

Only in (very) high-bandwidth cases and (maybe!) for low-power cases. For most things it's just added complexity for no benefit.