News for hackers. Stuff that matters.
Re:Show HN: B2blaze – A Backblaze B2 library for Python by dzek69 on Wednesday June 27, 2018 @02:05AM (#17406078)
Hah, I hadn't heard about Blackbaze in a while and I was even thinking about creating Ask HN asking if anyone was maybe using Blackbaze for a longer while and can say something about them (speed, data reliability). Now I'll take my chance:

Had you used Blackbaze B2? How was your experience?

Re:Show HN: B2blaze – A Backblaze B2 library for Python by manigandham on Wednesday June 27, 2018 @02:22AM (#17406134)
It's very cheap and effective for archival storage without having to deal with time/cost issues when you actually need to retrieve something. I use it for all my media so I can store terabytes and download in minutes for viewing.

Bandwidth is limited since they aren't connected like the major clouds, but it's workable if you don't need gigabit speeds. Single API key for permissions and lacks all the other features like events, object lifecycle, etc. Basic reporting but shows bucket size in real-time which is nice.

API can be annoying because it requires a request to "start" an upload (to get the address of where to upload), then doing the actual upload itself, but this can be automated away. Only single region for now (with multiple datacenters that aren't visible to you) so no global replication for extra durability or locality.

They have a partnership with https://www.packet.net (cloud bare metal) for free interconnect between their servers and B2 so you can do processing on your data without the public internet bottleneck and fees. Allows for an interesting data lake/warehouse option.

Use Cyberduck for a decent GUI client. If you just need personal computer backup, then use their actual backup offering which is unlimited storage and has auto-uploading background app.

Re:Show HN: B2blaze – A Backblaze B2 library for Python by lithiumfrost on Wednesday June 27, 2018 @02:40AM (#17406196)
Perhaps. Personally I use Arq with B2 for a back end, and this usually costs me less than $1/mo (their regular backup starts at $5). In addition, using B2 with something like duplicity is the better approach for backing up Linux or NAS boxes, where their official client is not supported.
Re:Show HN: B2blaze – A Backblaze B2 library for Python by maxyme on Wednesday June 27, 2018 @02:24AM (#17406140)
I've used B2 for some internal backup handling (several 100s of GB but millions of files) and have largely found it inexpensive and performant.

A few considerations: Their web UI can not handle large amounts of files (support said after a few million the file browser will not work). Sometimes when making a large number of deletions at once the API may serve 500 errors and the web UI give Java Servlet errors (this only happened a few times and resolved itself in a hour or two). As another user noted the per file/fragment upload speed isn't fantastic but I could max out my gigabit fiber with many concurrent downloads/uploads. The API has no concept of folders, only file path strings (which is mildly annoying to work with). Lastly I think all the data is currently housed in one geographic area but they are working on a DC in Phoenix.

Overall a pretty smooth experience but I was mostly using it for cold data.

Re:Show HN: B2blaze – A Backblaze B2 library for Python by voltagex_ on Wednesday June 27, 2018 @03:28AM (#17406363)
Have you reported this stuff to support? I found their support was great, even when dealing with me noodling around in C# and breaking things.
Re:Show HN: B2blaze – A Backblaze B2 library for Python by chrisper on Wednesday June 27, 2018 @02:17AM (#17406112)
I have used B2 for a couple of months. From Europe it is way too slow. They only have like 2 DC, which are both on the west coast I believe.

As I am using this for backups only, I went back to Google Drive where I can max out my Gbit upload.

Re:Show HN: B2blaze – A Backblaze B2 library for Python by tribaal on Wednesday June 27, 2018 @03:05AM (#17406280)
If you are connecting from Europe, maybe you could consider exoscale.com 's S3 storage offering?

It's 100% european (with Swiss and German regions), and priced pretty aggressively. https://www.exoscale.com/object-storage/

Disclaimer: I now work for exoscale, but was a happy user before that.

Re:Show HN: B2blaze – A Backblaze B2 library for Python by sschueller on Wednesday June 27, 2018 @03:15AM (#17406323)
Cool, you guys should send out mailings to your customers when new offerings come online. I asked about s3 at the end of last year and was told it is currently unavailable for new customers. I haven't thought to check if it was available until now and may have gone to a more expensive competitor.
Re:Show HN: B2blaze – A Backblaze B2 library for Python by haywirez on Wednesday June 27, 2018 @08:30AM (#17407580)
Thanks for this - contacted now. If you hear about any discounts for small startups, please let me know :)
Re:Show HN: B2blaze – A Backblaze B2 library for Python by icefo on Wednesday June 27, 2018 @04:00AM (#17406503)
I can max out the upload of my residential internet connection (about 1 megabyte/sec) and that's enough for me.

I use use it to back up my Nas where all my other computers are backed up. I set it up with duplicacy-cli, rate limited the upload to 700KByte/sec (internet stay usable that way) and the script that launches duplicacy check that it is not already running.

Since I never upload more than 60Gb per day on average to my Nas I don't have any issues

Re:Show HN: B2blaze – A Backblaze B2 library for Python by chrisper on Wednesday June 27, 2018 @04:51AM (#17406696)
Yes, I think I maxed out at 30 MB/s.
Re:Show HN: B2blaze – A Backblaze B2 library for Python by brianwski on Wednesday June 27, 2018 @01:06PM (#17409868)
Disclaimer: I work at Backblaze.

> From Europe it is way too slow.

We are opening a European datacenter in 2018, so stay tuned!

For now, we recommend you use multiple threads and you should be able to saturate any network connection, including yours in Europe. However, we do realize not all programmers or applications are capable of using threads and it would be more convenient to have lower latencies, thus the European datacenter in 2018. :-)

Re:Show HN: B2blaze – A Backblaze B2 library for Python by GordonS on Wednesday June 27, 2018 @02:59AM (#17406258)
I tried it from the UK a couple of years ago, and had the same experience. I have ~1TB of data to backup, and it was going to take months to upload vs a few days for Azure or AWS hosted storage.
Re:Show HN: B2blaze – A Backblaze B2 library for Python by glenneroo on Wednesday June 27, 2018 @08:06AM (#17407476)
Did you try enabling multi-threaded support? I was able to upload ~12 TB using 12-16 threads in a few months (and I'm based in EU as well).
Re:Show HN: B2blaze – A Backblaze B2 library for Python by GordonS on Friday June 29, 2018 @03:49AM (#17423066)
It was a while ago, but I seem to recall I had enabled it. Things might have changed since then, so I guess I'll give it another go at some point.
Re:Show HN: B2blaze – A Backblaze B2 library for Python by lloeki on Wednesday June 27, 2018 @04:36AM (#17406648)
> From Europe it is way too slow.

That's why we're still using AWS S3 currently in spite of the price, and consider moving to Digital Ocean's Spaces: B2 location is a non-starter.

Re:Show HN: B2blaze – A Backblaze B2 library for Python by brianwski on Wednesday June 27, 2018 @01:07PM (#17409877)
Disclaimer: I work at Backblaze.

> B2 location is a non-starter.

We are opening a European datacenter in 2018, so stay tuned!

Re:Show HN: B2blaze – A Backblaze B2 library for Python by lloeki on Thursday June 28, 2018 @04:49AM (#17415148)
Interesting! Hopefully it's within the EU and not in the UK (some of our customers are picky for legal reasons)
Re:Show HN: B2blaze – A Backblaze B2 library for Python by jonatron on Wednesday June 27, 2018 @02:51AM (#17406237)
It'd be interesting to find out if it's acceptably fast when proxying via a cloud provider.
Re:Show HN: B2blaze – A Backblaze B2 library for Python by gsibble on Wednesday June 27, 2018 @03:25AM (#17406350)
Here is the CTO of Backblaze addressing speed questions earlier today: https://www.reddit.com/r/Python/comments/8u40xa/i_wrote_a_li...
Re:Show HN: B2blaze – A Backblaze B2 library for Python by voltagex_ on Wednesday June 27, 2018 @03:43AM (#17406429)
Maybe I misunderstand his point, but isn't transferring data from Australia to the US or reverse always going to be slower due to the speed of light? What he's said doesn't negate the fact that he's only got one POP.
Re:Show HN: B2blaze – A Backblaze B2 library for Python by monort on Wednesday June 27, 2018 @04:16AM (#17406561)
Speed of light affects latency, not throughput. TCP works bad with large delays, that's why it's recommended to use several TCP connections to saturate the link.
Re:Show HN: B2blaze – A Backblaze B2 library for Python by voltagex_ on Wednesday June 27, 2018 @05:26AM (#17406818)
Yeah, I can never remember how TCP window size + latency affects transfer speed but you're right.
Re:Show HN: B2blaze – A Backblaze B2 library for Python by gsibble on Wednesday June 27, 2018 @03:54AM (#17406478)
I'm not sure how many POPs they have so I do agree with that. I'm not shilling for B2 and have no affiliation with them. I've had good performance them but am US based.
Re:Show HN: B2blaze – A Backblaze B2 library for Python by glenneroo on Wednesday June 27, 2018 @08:17AM (#17407518)
My individual connections to B2 are slow (200-500 KB/s) coming from Europe but that's why the multi-threaded option makes B2 very usable.
Re:Show HN: B2blaze – A Backblaze B2 library for Python by voltagex_ on Wednesday June 27, 2018 @03:27AM (#17406359)
Doesn't that destroy the value of B2 in that you're going to pay Amazon or whoever's egress costs?
Re:Show HN: B2blaze – A Backblaze B2 library for Python by gsibble on Wednesday June 27, 2018 @04:23AM (#17406603)
Not if you're only egressing to B2 once to save and then egressing many times from B2 to serve.
Re:Show HN: B2blaze – A Backblaze B2 library for Python by namibj on Wednesday June 27, 2018 @10:59AM (#17408640)
In that case, consider packet, egress to B2 is free. They claim a good network, but I didn't test (yet).
Re:Show HN: B2blaze – A Backblaze B2 library for Python by voltagex_ on Wednesday June 27, 2018 @08:37PM (#17413143)
You use the SJC1 option in packet.net to get the free ingress/egress. It's slowish single threaded but 100-200 megabit is easy with 4 threads.
Re:Show HN: B2blaze – A Backblaze B2 library for Python by yani on Wednesday June 27, 2018 @03:01AM (#17406265)
Only good if you are using it from the states.
Re:Show HN: B2blaze – A Backblaze B2 library for Python by trigoman on Wednesday June 27, 2018 @02:14AM (#17406103)
I’ve been using BackBlaze B2 for about 6 months and it’s been very stable!

The Java SDK is OK. So I plan on eventually having my application talk to Minio which will allow me to use the S3 API.

Re:Show HN: B2blaze – A Backblaze B2 library for Python by gsibble on Wednesday June 27, 2018 @03:23AM (#17406341)
The idea behind this was to avoid Minio at least when using Python with a very simple to use SDK.
Re:Show HN: B2blaze – A Backblaze B2 library for Python by haywirez on Wednesday June 27, 2018 @08:03AM (#17407460)
I'm trying to use it as an S3 replacement for audio content delivery - seems slow and laggy unfortunately. Uploads also fail frequently enough. I don't know if CDNs would make a big difference. (Europe)

Edit: lack of webhooks or something similar for doing follow-up after successful uploads is also irritating.