News for hackers. Stuff that matters.
Re:Using Backblaze B2 and Cloudflare Workers for free image ... by kaivi on Sunday August 25, 2019 @09:24AM (#20792814)
You do need access to an index/DB of all files in a bucket in order to delete them in parallel. Otherwise you're stuck paginating with the B2 API.
Re:Using Backblaze B2 and Cloudflare Workers for free image ... by hinkley on Sunday August 25, 2019 @01:52PM (#20794381)
You need a DB of all of the dead entries that need to be deleted, and that’s a fine thing to have.

There are lots of problem spaces where deletion is expensive and so is time shifted not to align with peak system load. Some sort of reaper goes around tidying up as it can.

But I think by far my favorite variant is amortizing deletes across creates. Every call to create a new record pays the cost of deleting N records (if N are available). This keeps you from exhausting your resource, but also keeps read operations fast. And the average and minimum create time is more representative of the actual costs incurred.

Variants of this show up in real-time systems.

Re:Using Backblaze B2 and Cloudflare Workers for free image ... by kaivi on Sunday August 25, 2019 @03:00PM (#20794794)
My case was really simple. I was done with my ML pipeline and nuked the database, but pics in B2 remained with no quick way to get rid of them and/or to stop the recurring credit card charges.

IMO an "Empty" button should have been implemented by Backblaze.

Re:Using Backblaze B2 and Cloudflare Workers for free image ... by ekiara on Tuesday August 27, 2019 @04:02AM (#20807260)
Would this technique have been faster?

A single pass: paginating through all entries in the bucket without deletion, just to build up your index of files. And then using that index to delete objects in parallel.

Re:Using Backblaze B2 and Cloudflare Workers for free image ... by PopeDotNinja on Sunday August 25, 2019 @10:57AM (#20793332)
I believe S3 is the same way.
Re:Using Backblaze B2 and Cloudflare Workers for free image ... by kaivi on Sunday August 25, 2019 @12:34PM (#20793864)
S3 has an "Empty bucket" button, unlike B2.
Re:Using Backblaze B2 and Cloudflare Workers for free image ... by brianwski on Monday August 26, 2019 @07:13AM (#20798835)
Disclaimer: I work at Backblaze.

> no way to empty a bucket.

Backblaze currently recommends you do this by writing a “Lifecycle rule” to hide/delete all files in the bucket, then let Backblaze empty the bucket for you on the server side in 24 hours: https://www.backblaze.com/b2/docs/lifecycle_rules.html