Redid + Sidekiq was a default for a long time in the Rails world as well, but it’s an unnecessary complication (and expense) for most use cases. Just use your existing DB until you need to seriously scale up, and then look at a message queue.

I’ve used Redis for leaderboards and random matchmaking though, stuff which is doable in postgres but is seriously write-heavy and a bit of a faff. Gives you exactly the sort of goodies you need on top of a K/V store without being difficult to set up.

As for caching - it’s nice to use as an engineer for sure, but pretty pricey. It wouldn’t be my default choice any more.

Rails is attempting to solve this with Solid Queue, which was inspired heavily by GoodJob, both of which use Postgresql (and more in the case of Solid Queue). Both seem to be fairly capable of "serious scale", at least being equivalent to Sidekiq.

A place I work at which I can’t name uses GoodJob at FAANG scale. And it works perfectly. The small startups and lower scale places still reach for sidekiq because they seem to think “it’s faster,” but it ends up being a nightmare for many people because when they do start reaching some scale, their queues and infra are so jacked up that they continually have sidekiq “emergencies.” GoodJob (and SolidQueue) for the win.

I like the sidekiq guy and wish him the best, but for me, the ubiquitous Redis dependency on my Rails apps is forever gone. Unless I actually need a KV store, but even for that, I can get away with PG and not know the difference.

Unfortunately there are still some CTOs out there that haven’t updated their knowledge are are still partying like it’s 2015.

I’m curious, are you running GoodJob in the same database as the application? For smaller scale stuff this is super convenient, but I wonder if it will become a problem at higher loads.

Using Redis exclusively remotely never made much sense to me. I get it as a secondary use case (gather stats from a server that’s running Redis, from another machine or something) but if it’s not acting as (effectively) structured, shared memory on a local machine with helpful coordination features, I don’t really get it. It excels at that, but all this Redis as a Service stuff where it’s never on the same machine as any of the processes accessing it don’t make sense to me.

Like you have to push those kinds of use cases if you’re trying to build a business around it, because a process that runs on your server with your other stuff isn’t a SaaS and everyone wants to sell SaaS, but it’s far enough outside its ideal niche that I don’t understand why it got popular to use that way.

To your last: yep, especially having in mind that Redis is ephemeral. I've had much more success with SQLite + a bunch of stricter validators (as SQLite itself is sadly pretty loose), and more performance too.