Re:Kafka is Fast – I'll use Postgres
by jghn on Wednesday October 29, 2025 @08:27PM
(#45754936)
What I've found to be even more common than resume driven development has been people believing that they either have or will have "huge scale". But the problem is that their goal posts are off by a few orders of magnitude and they will never, ever have the sort of scale required for these types of tools.
[ Parent
| on HN ]
The problem is when discussing techniques everyone uses the same terms but no one actually defines them.
I had this very same argument today. It was claimed that a once per year data mapping process of unstructured data that we sell via our product - would not scale. The best part is if we somehow had ten of these to do it would still be something that would take less than a year. Currently it takes a single person a few weeks and makes millions of dollars. This is the sort of fiddly work that you can find an Ontologist for and they’re happy to do it for the pay.
I’m unsure what is unattractive about this but I guess anything can be a reason to spend a year playing with LLMs these days.
I’ve had the same problem with compliance work (lightly regulated market) and suddenly the scaling complaints go away when the renewals stop happening.
I think because so many blogs, resources, textbooks etc focus on scale, developers are biased into thinking that they need to build for scale.
Which is wrong a lot of the time! You need to build what is needed. If only 10 people use your project, the design will be entirely different than if 10 million people use it
LLM solves this by meeting devs in the middle: the vibe coded DB schema, coupled with agentically-made application code, makes even 20,000 records a "huge scale".
This is so accurate. I’ve looked in wonder at how someone is maxing out an r6i.32xlarge MySQL DB, when I have ran 4x the workload on an r6i.12xlarge.
Schema design and query design will make or break your app’s ability to scale without skyrocketing the bill, it’s as simple as that.
Re:Kafka is Fast – I'll use Postgres
by notpaulgraham on Thursday October 30, 2025 @11:05AM
(#45760856)
any blogs/books you'd recommend on schema & query design? it honestly surprises me that these coding-focused models can't look at a schema; look at how data is being queried; reason about the use case for the data; and help prioritize solving for the most likely bottlenecks to scaling the underlying data services.
https://learn.microsoft.com/en-us/sql/relational-databases/s...
This one is a classic for MSSQL, most of it is applicable on postgres.
I like http://www.redbook.io/