News for hackers. Stuff that matters.
Re:ZeroMQ: Modern & Fast Networking Stack by nkurz on Saturday September 04, 2010 @03:00PM (#1663180)
If you are streaming a large file over TCP, ZeroMQ, with its emphasis on small messages sent and received apparently atomically, would be a stunningly poor choice.

It's also wholly unsuitable as a floor wax or a dessert topping. :)

Does your criticism also apply to its intended use case? At a glance, it doesn't strike me as an unreasonable design choice, but I haven't thought about it deeply.

Re:ZeroMQ: Modern & Fast Networking Stack by sandGorgon on Sunday September 05, 2010 @12:08AM (#1663971)
what would be the best way of sending large files server-to-server ? this is a question that has been puzzling me.

Is there like a library or a protocol that is tuned towards transferring multi-gb files ?

Re:ZeroMQ: Modern & Fast Networking Stack by nkurz on Sunday September 05, 2010 @03:46AM (#1664196)
Are you intending to have many of these transfers happening in parallel, or are you sending a single file and hoping to saturate your link? Do you have full control of both ends, or only the server? Are you streaming or is the whole file always transferred? Do you need encryption?

For simultaneous large files with no special needs, my instinct would be to just choose any lightweight HTTP server that supports sendfile(). Basically, pass it off to the kernel, and let it handle it. But maybe there are more subtle keys to good large file performance: http://www.psc.edu/networking/projects/tcptune/

For cases where you are controlling both ends, you might find useful suggestions here: http://serverfault.com/questions/111813/what-is-the-best-way... In particular, Tsunami and uftp look interesting.