Re:ZeroMQ: Modern & Fast Networking Stack
by sandGorgon on Sunday September 05, 2010 @12:08AM
(#1663971)
what would be the best way of sending large files server-to-server ? this is a question that has been puzzling me.
Is there like a library or a protocol that is tuned towards transferring multi-gb files ?
Are you intending to have many of these transfers happening in parallel, or are you sending a single file and hoping to saturate your link? Do you have full control of both ends, or only the server? Are you streaming or is the whole file always transferred? Do you need encryption?
For simultaneous large files with no special needs, my instinct would be to just choose any lightweight HTTP server that supports sendfile(). Basically, pass it off to the kernel, and let it handle it. But maybe there are more subtle keys to good large file performance: http://www.psc.edu/networking/projects/tcptune/
For cases where you are controlling both ends, you might find useful suggestions here: http://serverfault.com/questions/111813/what-is-the-best-way... In particular, Tsunami and uftp look interesting.
It's also wholly unsuitable as a floor wax or a dessert topping. :)
Does your criticism also apply to its intended use case? At a glance, it doesn't strike me as an unreasonable design choice, but I haven't thought about it deeply.