Your server has 128 gibibytes of RAM, and has for five years now. In fact, you can buy off-the-shelf servers with a tebibyte of RAM now. As long as you have some way to recover from a power failure, if there's something you could do with Postgres without sharding fifteen years ago, you can do it in RAM on a single server now. But, if it was disk-bound, you can do it about a million times faster than you could do it then.

It's reasonable to argue that shelling out the US$800 for a server with 128 gibibytes of RAM (or renting one from a cloud provider for US$150 a month) amounts to buying nifty toys for the engineering department. But if you're paying each engineer US$200k per year in salary and a similar amount in benefits, that's US$190 an hour, so each engineer's workday buys you almost four such servers. Your RAM has a minimum transaction time of about 100 ns, while an SSD is more like 10000 ns. If you have 100+ GiB of data to query, the alternative to buying nifty toys is paying the engineering department to do optimization work that wouldn't be necessary if your data was stored in memory that was 100x faster.

The rough equivalency here is that adding one more engineer to your team costs as much as adding 16 dedicated Xeon servers at Hetzner with 128 GiB of RAM each.

Sometimes the best answer is still a relational database, of course! It's often easier to write your queries in relational terms than by looping over hash tables, though probably not if you're using an ORM, and Postgres is delighted to cache all your data in RAM.

At some point, you may grow beyond being able to run your production environment on a single server, at which point splitting things into tiers is worthwhile. (In some cases you start at that point, for whatever reason, but a lot of web services can serve a million users on a single server. It sure sounds like Fast's service would have worked fine on a single server.)

The point of a RDBMS has never been to save on RAM, raw files have undesirable properties for lots of data management scenarios. You can have a very simple schema if you like, e.g. a RDBMS makes for a damn good key-value store.

The most boring tech imaginable would be a file. Perhaps containing JSON or similar structured data that can be handled using commonly available libraries. Which is exactly what they used for a long time.

I've done something similar before and it saved countless hours wasted on configuring databases, writing queries, fighting impedance mismatches. Postgres brings a lot of work. With a file you just need a couple of lines to write your memory out to a file and a couple more to read it back. Simple and effective.

Until it stops scaling, but at that point you've established that your business is successful enough that it warrants the effort of doing something shiny. Postgres has a place in this world, but no need to put the cart before the horse.

Perhaps one problem with the current backlash against over-complicated infrastructure is that for some of us, doing everything in one process on one machine using a local embedded database is its own form of playing with nifty toys.

yeah, I guess, I'm not going to get too specific here but I do think it would be nifty if I didn't go through a couple 10 minute periods every day where I have to basically shut everything down run a number of scripts, then restart individual processes that are problematic, to be able to work again.

Note that this basically works 99% of the time now and is part of a learning process that had the whole thing taking 2 - 5 hours every day for several weeks, down to 2 hours a day for several weeks after that, and now down to the nearly inescapable 20 - 40 minutes a day I have now.

Yeah, not doing that would be my own form of playing with nifty toys.

Wow. I would find it super annoying to be interrupted twice a day to do a 10min task. Can’t you try to narrow down the underlying bug? Or somehow automate the restart?