News for hackers. Stuff that matters.
Re:Backblaze B2 Drops Download Pricing by 60% by Veratyr on Tuesday April 18, 2017 @01:25PM (#14140825)
I'd be interested to know, have you looked into reducing/eliminating bandwidth prices further for clients close to your network? It seems like you're with Unwired, which has heavy inbound traffic and peers at a number of facilities in the Bay Area. It'd be really interesting to know if you could, for example, offer free traffic to Linode, DigitalOcean or Vultr, all of which have offerings in the area and could help balance out Unwired's traffic flow (I think that's a thing ISPs care about).

The big thing missing from B2 for me now, relative to the other big players, is the ability to process my data once it's stored with you. If I store my images in S3 for example, I can use EC2 to thumbnail them without paying anything for retrieval. At the moment I can't do that with B2 and it seems unlikely to make sense for you to build your own compute cloud but perhaps you can integrate with others.

Re:Backblaze B2 Drops Download Pricing by 60% by budmang on Tuesday April 18, 2017 @01:35PM (#14140945)
I would love to do this. Cross-connects aren't free, but they are certainly less expensive than going out across the Internet. And enabling compute/storage use cases by (possibly) partnering with other providers would be excellent. I'll reach out to a few of those compute providers and explore.

Can you tell me more about how you use/would use cloud storage? (with or without compute)

Re:Backblaze B2 Drops Download Pricing by 60% by _mgu7 on Tuesday April 18, 2017 @01:56PM (#14141164)
Currently I don't really use cloud storage, it's cheaper for me to colocate a 2U stuffed with disks and buy transit.

Primarily though I store media. I have (for a home user) large amounts of high resolution (lossless audio, RAW images, high bitrate (even ProRes) video) media that I like to keep for archival and transcoding but don't need for playback. I want to store the original media somewhere safe and cheap but have the ability to transcode it to a format suitable for playback/streaming on demand.

Actually it just occurred to me that it might be worth you talking to the folks at Plex (https://plex.tv) about this too, as they're working on their Cloud product and need more places for their users to store media.

Re:Backblaze B2 Drops Download Pricing by 60% by jjeaff on Tuesday April 18, 2017 @02:59PM (#14141804)
If I was a storage company, I would consider Plex a hot potato and would stay far, far away. There is a reason Plex isn't building out their own storage and the reason isn't cost. It's because they know that 99% of all the media their customers handle is pirated content. At some point, Plex will become large enough to get on the radar of the MPAA and at that point, I assume they will go after the cloud storage providers.
Re:Backblaze B2 Drops Download Pricing by 60% by cookiecaper on Tuesday April 18, 2017 @04:20PM (#14142508)
Yeah, I'm not sure about that. Plex has nothing to do with how users are getting the content, it just offers playback and cataloging after they've already retrieved it.

I'd agree if they allowed something like library sharing between users, but I don't see them as a likely target just for allowing playback of consumer media.

I use a (local) Plex installation for home videos, downloaded clips, and various other non-infringing things. It's the most convenient way to share the library out to a Chromecast/phone/Roku.

Re:Backblaze B2 Drops Download Pricing by 60% by jjeaff on Tuesday April 18, 2017 @09:57PM (#14144868)
Ya, I don't think Plex would be necessarily at fault, although they could be susceptible to DMCA takedown notices. But a storage provider could get into deep trouble if they know tons of pirated content was being uploaded. Kim Dotcom claims MEGA had no idea if the content being uploaded was infringing.
Re:Backblaze B2 Drops Download Pricing by 60% by cookiecaper on Wednesday April 19, 2017 @12:15PM (#14148883)
Megaupload (which I assume you mean because afaik MEGA has not been in legal trouble) was a distribution platform; the purpose was to upload something and give the link to others. If it was a personal file locker that couldn't link out or be shared, as Plex intends to be, it probably would not have had issues.
Re:Backblaze B2 Drops Download Pricing by 60% by Veratyr on Tuesday April 18, 2017 @03:06PM (#14141876)
I feel like it'd be the opposite. DMCA protects the storage provider as long as they're unaware that they're storing infringing content and Plex limits sharing enough that any infringement is unlikely to reach the MPAA/RIAA.
Re:Backblaze B2 Drops Download Pricing by 60% by jjeaff on Tuesday April 18, 2017 @09:55PM (#14144862)
I think inking a deal with Plex would be pretty obvious. I'm not sure where the line is for knowing that you are storing infringing content, but MEGA claims they didn't know what users were uploading.
Re:Backblaze B2 Drops Download Pricing by 60% by Veratyr on Wednesday April 19, 2017 @12:54PM (#14149266)
Plex itself is a US company and were it "obvious" that its existing Cloud product was infringing on copyrights, it would have already been the target of a lawsuit.
Re:Backblaze B2 Drops Download Pricing by 60% by budmang on Tuesday April 18, 2017 @02:42PM (#14141617)
Got it. Transcoding & making image thumbnails are two common use cases we hear. Thanks.
Re:Backblaze B2 Drops Download Pricing by 60% by derekp7 on Tuesday April 18, 2017 @02:53PM (#14141744)
I can give you my example -- I have an open source backup tool called Snebu which uses an sqlite DB to store the backup catalog (file names, file hash, metadata etc), whereas the individual files are stored in a vault area (currently just a disk volume, with files named after the sha1 hash). Backup data is sent to it by piping in tar format to the main process, which updates the catalog DB and deposits files into the storage vault.

I'd like to add cloud capabilities, but for that I'd have to run the backend process somewhere that can easily update the DB, and easily read/write the data in the vault, along with syncing the full DB to the vault location after a backup.

My other option is to run everything locally, and compute a delta file between versions of the DB and send that delta over to the vault (i.e., do D2D2C backups).

Edit: Another possible requirement -- I'd like to pack multiple smaller files into an archive file, and deposit that into a bucket. But as files age out of the archive, I'd like to be able to move active ones out of it and into a new one. Right now, I'd have to re-create the still-valid backup data out of on-hand information, upload the new archive, then send a command to delete the old one.

Re:Backblaze B2 Drops Download Pricing by 60% by loxias on Tuesday April 18, 2017 @03:40PM (#14142186)
Speaking for myself, I'm excited about using B2 as a replacement for more expensive local disk in not-heavy-iops-role machines. It's not the most exotic use, but if you look at every vm provider (linode, digital ocean, even amazon) the prices of local disk are huge in comparison to the prices of cloud storage like B2 & S3.

By combining them I can have a cheap box (Linode for instance) with a local write cache, with B2 providing a terabyte of raw storage, and get a slow but effective VM with a 1TB "local disk" for under $20 month. Which is great.

Re:Backblaze B2 Drops Download Pricing by 60% by Veratyr on Tuesday April 18, 2017 @03:51PM (#14142278)
Unless you're going to be doing pretty much entirely sequential reads of entire files, I don't think this is likely to work well for you. B2 (and other cloud storage) isn't just priced by bandwidth but by "transactions" (requests).
Re:Backblaze B2 Drops Download Pricing by 60% by loxias on Tuesday April 18, 2017 @04:04PM (#14142373)
It's true, there is a price for some API requests, but it's actually quite low if you're using the cloud storage to populate a local cache! Backblaze categorizes download as a "class B" transaction with 2500 free per day and $0.004/10k. I fully admit it's not a suitable solution for many workloads, but I've been playing with a VM using B2 as slow additional disk (as an additional volume, not suggesting putting / or /usr..) and am yet to exceed the free tier.
Re:Backblaze B2 Drops Download Pricing by 60% by budmang on Tuesday April 18, 2017 @06:51PM (#14143751)
So are you actually running a VM image somewhere and using B2 as a virtual disk drive from it?
Re:Backblaze B2 Drops Download Pricing by 60% by loxias on Tuesday April 18, 2017 @02:56PM (#14141771)
I would love this! Great idea. So far, B2 beats S3 on nearly every metric, with the exception that with S3 I can process data more cheaply with EC2.

A crosspromotion deal between Linode and Backblaze would be fantastic.

Re:Backblaze B2 Drops Download Pricing by 60% by infogulch on Tuesday April 18, 2017 @01:38PM (#14140972)
So you're asking for cheaper egress to specific cloud provider peering partners that offer compute. ?
Re:Backblaze B2 Drops Download Pricing by 60% by Veratyr on Tuesday April 18, 2017 @03:32PM (#14142110)
It doesn't need to be a specific provider, those were just examples. The idea is to have free traffic between Backblaze and someone that provides compute, so you can transform the data without fetching all of it. I suggested those in particular because I believe they're all in the SF bay area and should be present at one of Unwired's existing peering points.

The reason I can't see traffic just being straight free for everyone is that for Backblaze to make traffic free, it needs to make arrangements with the providers to reduce costs (for example by connecting directly rather than through the internet).

Re:Backblaze B2 Drops Download Pricing by 60% by cookiecaper on Tuesday April 18, 2017 @01:28PM (#14140859)
B2 is a competitor of long-term storage solutions like Glacier or TarSnap, not virtualized storage like S3. It's meant to be almost 100% write-only. Basically, you pull your backups down in case of disaster.
Re:Backblaze B2 Drops Download Pricing by 60% by budmang on Tuesday April 18, 2017 @01:37PM (#14140963)
Backblaze has a backup service for laptops/desktops, but our B2 service is actually more like S3 - the data is available without delays or penalties for access. (Unlike Glacier.)
Re:Backblaze B2 Drops Download Pricing by 60% by cookiecaper on Tuesday April 18, 2017 @01:48PM (#14141086)
Yeah, it's similar to Google Nearline or TarSnap in that unlike Glacier, the data can be read out instantly instead of waiting for Amazon to restore it off of a tape or whatever they do. The "penalty for access" is the download cost.

However, my understanding is that B2 was intended for long-term storage and not everyday cloud storage. Is this understanding incorrect? Backblaze does take the position that B2 should be considered as a competitor for S3?

Up to this point, my experience with B2 has been that they don't want to be the authoritative storage source for the data. I seem to recall reading that B2 should not be used as primary storage (though that was in beta days). In support discussions, a representative expressed surprise when it was implied that I may not have local copies of files that already exist on B2 (I do indeed have local copies, but it was beside the point) before confirming that yes, I would need to redownload and reupload to complete a rename.

Re:Backblaze B2 Drops Download Pricing by 60% by brianwski on Tuesday April 18, 2017 @02:37PM (#14141567)
Brian from Backblaze here.

> my understanding is that B2 was intended for long-term storage...

Long term storage is a great use of B2, but it was never "intended" as anything other than the ability to reliably store blobs of data and retrieve them really fast whenever and how ever often you like. I suppose there are different use cases that work well for B2 and others that don't work well, but that is up for customers to decide, not Backblaze.

> ... they [Backblaze?] don't want to be the authoritative storage source for the data

Who told you that? Backblaze wants people to use B2 for whatever they want. Backblaze makes an AMAZINGLY GOOD authoritative storage source.

> I would need to redownload and reupload to complete a rename

Right now the B2 service does not include a "rename" function. If that would be useful to customers, we can add it? We collect customer requests and add them in the order that we have time and what gets the most customer votes.

Re:Backblaze B2 Drops Download Pricing by 60% by cookiecaper on Tuesday April 18, 2017 @03:04PM (#14141849)
Thanks for clarifying.

>Long term storage is a great use of B2, but it was never "intended" as anything other than the ability to reliably store blobs of data and retrieve them really fast whenever and how ever often you like. I suppose there are different use cases that work well for B2 and others that don't work well, but that is up for customers to decide, not Backblaze.

I guess that the pricing structure of free uploads and expensive downloads is the main thing that caused it to be compared to Glacier and other long-term storage solutions like Nearline. Reducing the download cost is an attempt to get more people to see B2 as an S3 replacement, right?

>Who told you that? Backblaze wants people to use B2 for whatever they want. Backblaze makes an AMAZINGLY GOOD authoritative storage source.

Like I said, I must be recalling discussion from the time the beta was announced, and that such restrictions were shed post-beta. Did Backblaze ever expand into its promised second datacenter?

>Right now the B2 service does not include a "rename" function. If that would be useful to customers, we can add it? We collect customer requests and add them in the order that we have time and what gets the most customer votes.

Yes, this made me stop using B2 for all practical intents and purposes. It's such an easy thing that it doesn't seem like there's any reason it shouldn't be supported. Between the web UI's lack of pagination and the inability to rename a file to comply with virtual folders, the web interface is not usable.

Re:Backblaze B2 Drops Download Pricing by 60% by brianwski on Tuesday April 18, 2017 @06:04PM (#14143406)
> Reducing the download cost is an attempt to get more people to see B2 as an S3 replacement, right?

A price drop MIGHT open new markets to us. What those markets are are kind of an internal debate at Backblaze. What we agree on is that we priced outbound bandwidth incorrectly when we launched B2. We know this for two reasons:

A) The price Backblaze pays bandwidth is WAAAAAY lower than 5 cents per GByte.

B) Almost nobody is choosing to use Backblaze as a CDN or website server or a large source of outbound data.

One new market might be making B2 more attractive as a CDN or website server. It is my understanding that CDN customers use the CDN because of two things: 1) cheap download prices and 2) fault tolerance and reliability and distribution all over the world.

Most CDNs charge about 2 cents/GByte or maybe a little more.

If B2 outbound bandwidth prices drop below what CDNs charge, then SOME of their customers might consider B2 as one part of their solution.

In my humble opinion we are still overpriced, but the reason we're dropping prices slowly is because we're slightly terrified of becoming too attractive too fast and not being able to handle the onslaught as the world abandons all the other CDNs and goes with B2 since we are so much cheaper. So we're going slowly at first while we ramp up, and then we can drop prices much more in the future as we gain comfort with serving up gigantic amounts of cached data.

> Did Backblaze ever expand into its promised second datacenter?

In the process of. Servers are sitting in the new datacenter powered up and talking with each other (and even have customer data on them), but we do not yet let customers choose which datacenter their data lands in (or choose to duplicate their data between).

Re:Backblaze B2 Drops Download Pricing by 60% by extra88 on Tuesday April 18, 2017 @04:46PM (#14142752)
> Reducing the download cost is an attempt to get more people to see B2 as an S3 replacement, right?

B2's download cost was already cheaper than AWS for everyone but the largest customers but I expect a little guy has to try harder.

I think something that has contributed to seeing B2 as mainly for backup, aside from it being a service of a backup company, is greater emphasis on durability rather than availability. And looking at web pages now, B2's availability SLA is 99.9% [0] while S3's is 99.99% [1] (S3 Infrequent Access is 99.9%).

[0] https://www.backblaze.com/b2/cloud-storage.html

[1] https://aws.amazon.com/s3/