Ask HN: What to use for fast uploads of GB-sized files?

Posted by yread on Sunday February 25, 2018 @05:29AM
from the ask-hn dept.

I'm working on a SaaS that will allow users to upload large files (I expect 100MB - 5GB). These files are hard to compress. My users will probably be sitting on very good lines - a gigabit link shared for an institution is typical. I probably don't want to saturate the whole line.

I want to offer fast upload from the browser, so no sFTP/rsync/...

Does anyone have experience in this? I've found resumable.js and tus.io but I couldn't find a good comparison between the two. Then there are some cloud services but with my file sizes I would have to get a second mortgage to pay for those.

( 4 comments | 5 points | on HN )

Re:Ask HN: What to use for fast uploads of GB-sized files? by dgarud on Monday February 26, 2018 @09:02AM (#16465268)

Try asking the network admin of your clients network to allocate a max limit - e.g. in a 4 GBPS line, your application will get max 2 GBPS, so the files keep uploading but your users wont feel their net is down when someone uses your app.

[ Parent | on HN ]

Re:Ask HN: What to use for fast uploads of GB-sized files? by JoachimSchipper on Monday February 26, 2018 @02:32PM (#16468230)

No experience, but note that Amazon - and presumably other cloud providers - can allow users to upload directly to the cloud: https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingHTTPPOS.... S3 is not that cheap, but I'd expect uploading to the nearest Amazon datacenter to be decently fast.

[ Parent | on HN ]

Re:Ask HN: What to use for fast uploads of GB-sized files? by WhiteOwlLion on Tuesday February 27, 2018 @12:53PM (#16475977)

If the sending side can saturate your line, then you want to set a max. Some providers offer bursting where you can exceed your max for a short period of time. You can inquire about that. I know some people drop the connection and re-initiate so they can use burst bandwidth for the entire upload.

[ Parent | on HN ]

Re:Ask HN: What to use for fast uploads of GB-sized files? by floatboth on Sunday February 25, 2018 @05:54AM (#16458333)

tus is a well defined protocol with a couple reference implementations of the server part. resumable just says "This should be a fairly simple task" and a small description of the "protocol"…

[ Parent | on HN ]