News for hackers. Stuff that matters.
Re:Death by a Thousand Microservices by crooked-v on Tuesday September 12, 2023 @02:08AM (#37477396)
"A while" is underselling it. As long as you have people who are half-decent with SQL, "just put Postgres on a big db server" will get you to 50 million row tables before you have to start thinking about even hiring a real DBA.
Re:Death by a Thousand Microservices by jedberg on Tuesday September 12, 2023 @02:26AM (#37477497)
True. "While" for some large value of time. And if you set up the auto-vacuumer correctly from the start, you can go even further!
Re:Death by a Thousand Microservices by turtles3 on Tuesday September 12, 2023 @03:13AM (#37477736)
Unpopular opinion: if you're skimping on ops/DBA resources (as you may need to do in a startup), then MySQL is a better default. By all means use postgres if your use case demands it, but personally I find the ops story for MySQL takes less engineering overhead.
Re:Death by a Thousand Microservices by lenkite on Tuesday September 12, 2023 @03:35AM (#37477881)
I agree with this "unpopular opinion". Worked with both MySQL and postgres based mid-scale apps of several thousands of users. Postgres is so deeply lauded here at HN yet requires two more magnitudes of operations work to keep it up and running. Vacuuming sucks hard.
Re:Death by a Thousand Microservices by avereveard on Tuesday September 12, 2023 @04:56AM (#37478379)
That's not wrong, but postgres allows for cramming a lot of functionality in the db, and it's fast at following storage trend, i.e. with pgvector
Re:Death by a Thousand Microservices by turtles3 on Tuesday September 12, 2023 @05:27AM (#37478594)
That is a very specific use case, and might only be a small subset of your actual data. If you don't have these specific requirements (eg. CRUD apps), you can save yourself a lot of unnecessary headaches by defaulting to MySQL.

My main point is attempting to counter the narrative popular on HN that postgres should be an automatic default. For sure there are many aspects in which postgres is superior, I absolutely do not debate that, especially when it comes to developer experience. But there is much more to it than that when it comes to delivering business value. That's where ops and DBA concerns start to matter, and IMO MySQL is so far ahead in this regard that it outweighs all the other hideous warts of working with it, when you consider the bigger picture of the business as a whole.

Re:Death by a Thousand Microservices by savrajsingh on Tuesday September 12, 2023 @07:33AM (#37479492)
Yes! Vanilla MySQL and a good ORM (peewee, anyone?) gets you extremely far.
Re:Death by a Thousand Microservices by PedroBatista on Tuesday September 12, 2023 @09:45AM (#37481171)
Yes, and most successful companies who started in the last ~20 years started ( and many continue ) with a monolith and a MySQL database.

Only the mega-cap ones started to pursue other options mostly due to their type of business and bucket-loads of "free" VC money with explicit orders to burn it and get "unicorn" status - which involves hiring thousands of developers in record time and the whole thing turns into a zoo. Which is an organizational problem, mostly not a tech one.

Other than the ones we pretend are the whole Universe, there are thousands and thousands of medium to big companies with billions of revenue who started their product with a monolith and a MySQL database and many still do just that.

Re:Death by a Thousand Microservices by baq on Tuesday September 12, 2023 @02:55AM (#37477638)
50M is something that can easily be done with just a dev who understands that sql is more than select, insert and update with the manual, google and chatgpt.

You can get really damn far with a fat postgres box.

Re:Death by a Thousand Microservices by dalbasal on Tuesday September 12, 2023 @03:28AM (#37477839)
The problem with these discussions is that you can always get a lot out of any given architecture, structure, db approach or whatnot. Can. There's just a lot of daylight between "can" and "likely will."

Ultimately, everything has its limitations and tradeoffs. If we respect them, it's generally a smooth ride. Problem is that we rarely do... within a company (startup or otherwise) under real conditions. There's also a dynamic where we build until the point where something stops us. Tech debt, complexity, over-engineering, under-engineering, feature bloat or antagonism between early decisions and current goals.

There's a self-regulating aspect to this. If architecture is spot on, perfect for the task at hand we can move faster to reach the point where it no longer is.

Re:Death by a Thousand Microservices by szundi on Tuesday September 12, 2023 @03:38AM (#37477896)
The real tragedy is this clever people usually talking about their usecases and those present the requirements. There is no average system out there.
Re:Death by a Thousand Microservices by cnity on Tuesday September 12, 2023 @03:57AM (#37478017)
"There is no average system" is such a good insight.
Re:Death by a Thousand Microservices by baq on Tuesday September 12, 2023 @03:43AM (#37477928)
You're right of course.

My point is that postgres is, compared to almost everything else, easy to get from "can" to "likely will" with just somebody with a brain, a manual and google. In absolute terms it of course depends, but the point is relative.

Re:Death by a Thousand Microservices by dalbasal on Tuesday September 12, 2023 @04:33AM (#37478237)
A brain, a manual and google is usually a pretty good way generally to make solid decisions, respect the limits of your chosen stack.

What happens when there is >1 brain involved... or when brainless, manualless decisions eventually get made... or two pivots from now...

I'm not disagreeing with your approach. I agree with it, especially as starting point. I'm cautioning that resilience against complexity isn't about how easy it is to make good decisions when you understand the spec, read the manual and calmly proceed. Complexity and fragility accumulate when one or all of these are absent. How easily you can (and thus inevitably will) make a mess... not how easily you can keep it clean.

IRL situations with regular rdbs, a very common trend seems to be long term drift between schema and spec. The flexibility and approachability of postgres often enables a lot of kludge eventually.

Data stores have this dichotomy between "look how easy" and "is limiting factor" that speaks to difficulties we don't know how to articulate or isolate.

Re:Death by a Thousand Microservices by MichaelZuo on Tuesday September 12, 2023 @09:31AM (#37480922)
Of course if there are lots of bozos in the startup then even getting to and maintaining 50 million rows is going to be very difficult.

But if you have a small group of folks that are at least as competent as the folks at WhatsApp pre-acquistion, then there really shouldn't be any doubt whatsoever.

Re:Death by a Thousand Microservices by dalbasal on Tuesday September 12, 2023 @10:09AM (#37481549)
> a small group of folks that are at least as competent as the folks at WhatsApp pre-acquistion

That's a success case... beware survivorship bias.

> if there are lots of bozos in the startup

No arguing that quality engineers are fundamental to quality engineering. That said... by this standard, there's no point in having this entire discussion. Every good db/store out there is good. They all work very well if used as they should be, with due respect to tradeoffs. Yet, almost everyone has db problems. Almost every one of these problems occur well within the technical limits of postgres or whatnot.

"It shouldn't be a problem" when it usually is irl is tunnel vision. There is an empirical reality disagreeing with you. Walking into it with "this shouldn't be a problem unless everyone is a moron" is bad strategy. If you can't think of reasons why architecture can and will become a problem, then just assume that you (or some of you, some of the time) are morons, and try to make it moron proof.

Re:Death by a Thousand Microservices by MichaelZuo on Thursday September 14, 2023 @11:05AM (#37510002)
> That said... by this standard, there's no point in having this entire discussion.

Yes, I agree just repeating tautologies is unlikely to be meaningful.

[dead]
Re:Death by a Thousand Microservices by renegade-otter on Wednesday September 13, 2023 @09:49AM (#37496707)
Ignorance of database basics is something I allude to in the article, but it's a larger topic that I might vent about later.
Re:Death by a Thousand Microservices by pizza234 on Tuesday September 12, 2023 @04:05AM (#37478059)
I've been on both the sysadmin, development, and hiring sides, and with data models at scales of 50M+ records, devs who "understand that sql is more than select [...]" are rare in my experience, as they're a cross between db admins and developers.

Administrating (in particular, query planning and production operations) databases with tables sizes with magnitude of 10M and more records is challenging, and requires a skill set that is very different from pure development.

One won't get "really damn far with a fat postgres box", unless they're doing very simple SELECTs, which is not the case with modern web apps.

Re:Death by a Thousand Microservices by lelanthran on Tuesday September 12, 2023 @04:22AM (#37478168)
> Administrating (in particular, query planning and production operations) databases with tables sizes with magnitude of 10M and more records is challenging, and requires a skill set that is very different from pure development.

Is it more or less challenging than the alternatives? Is it less challenging enough to add a new tech to your stack, add the required knowledge to the team, etc?

I mean, knowing "enough-to-perform-CRUD" SQL is table stakes for developing on the back-end, but knowing $CURRENT-FLAVOUR-NOSQL (of which there are multiple products, all with such substantial differences that there is no knowledge transfer between using them) isn't, so there's going to be ramp-up time for every dev, and then every dev that is added to the team.

I'm not disputing your argument, I'm just pointing out that, sometimes, it's easier and faster to upskill your PostgreSQL developer to "scale the DB" than it is to teach them how to properly use, maintain, architect and code for DynamoDB and others.

Re:Death by a Thousand Microservices by dgroshev on Tuesday September 12, 2023 @08:00AM (#37479745)
It's not just $CURRENT-FLAVOUR-NOSQL. It's also doing custom transactions and locking on top and/or thinking in terms of eventual consistency. It's so, so much more complex than just SELECT FOR UPDATE/BEGIN TRANSACTION.
Re:Death by a Thousand Microservices by baq on Tuesday September 12, 2023 @04:33AM (#37478238)
> are rare in my experience, as they're a cross between db admins and developers.

That's fine. You only ( ;) ) need one on the team.

Re:Death by a Thousand Microservices by jiggawatts on Tuesday September 12, 2023 @03:58AM (#37478025)
It’s so cute that you think 50M rows is big. Your phone can handle many times that, and update it tens of thousands of times per second.
Re:Death by a Thousand Microservices by dagw on Tuesday September 12, 2023 @04:16AM (#37478133)
The problem isn't storing or inserting 50M rows, it querying 50M rows in non trivial ways. And the difference in performance between doing that 'right' and 'wrong' is orders of magnitude.
Re:Death by a Thousand Microservices by omegabravo on Tuesday September 12, 2023 @04:39AM (#37478275)
doing it on your phone only has one person querying it. Scale that to several thousand and it might appear different.
Re:Death by a Thousand Microservices by jiggawatts on Tuesday September 12, 2023 @05:41AM (#37478704)
I've scaled 300M rows to 1M users... in 2005... on boxes with spinning rust in them.

It amazes me that my comment (while flippant) got voted down. It's true.

Your phone can quite literally update a 50M table, say, 10K times per second.

If you think that you need more transactions than that per second (nearly a billion per day), then congratulations: you're Facebook.

Re:Death by a Thousand Microservices by sethammons on Tuesday September 12, 2023 @06:58AM (#37479210)
I believe you are downvoted due to the "how cute" part, not the scale part. You could have delivered useful info without a put down
Re:Death by a Thousand Microservices by quyse on Tuesday September 12, 2023 @07:13AM (#37479337)
It's not that good usually, e.g. PostgreSQL writes data in pages (8 KB by default), and changing 10K random rows in the 50M rows table can be quite close to the worst case of 1 changed page per changed row, so 8x of your estimate. Also need to multiply x2 to account for WAL writes. Also indexes. It's not hard to hit a throughput limit, especially with HDDs or networked storage. Although local SSDs are crazy fast indeed.
Re:Death by a Thousand Microservices by jiggawatts on Tuesday September 12, 2023 @07:34AM (#37479497)
Agreed: 80MB/s for the random 8K page updates. However, transaction logs in modern databases are committed to disk in batches, and each log entry is smaller than a page size. So a nice round number would be 100 MB/s for both.[1]

For comparison, that's about 1 gigabit per second in the era of 200 Gbps networking becoming common. That's a small fraction of SSD write throughput of any modern device, mobile or not. Nobody in their right mind would use HDD storage if scaling was in any way a concern.

[1] Indexes add some overhead to this, obviously, but tend to be smaller than the underlying tables.

Re:Death by a Thousand Microservices by peteradio on Tuesday September 12, 2023 @08:43AM (#37480283)
> spinning rust

That's your reason right there, you used rust! It's both performant and secure!

Re:Death by a Thousand Microservices by forgetfreeman on Tuesday September 12, 2023 @08:27AM (#37480058)
Eh, intelligent table design should knock most of that out. If you've got an 8 page query implementing a naïve solution to the knapsack problem (I've seen this in the wild) several mistakes have been made.
Re:Death by a Thousand Microservices by n0us on Tuesday September 12, 2023 @09:33AM (#37480961)
> As long as you have people who are half-decent with SQL

:/