News for hackers. Stuff that matters.

A comment on Big data is dead (2023). Parent comment

Re:Big data is dead (2023) by newaccount74 on Monday May 27, 2024 @07:09AM (#40489656)
The biggest problem with planning for scale is that engineers often have no idea what problems they will actually run into when they scale and they build useless shit that slows them down and doesn't help later at all.

I've come to the conclusion that the only strategy that works reliably is to build something that solves problems you have NOW rather than trying to predict the future.

[ Parent | on HN ]
Re:Big data is dead (2023) by fuzzy2 on Monday May 27, 2024 @07:19AM (#40489723)
Exactly this. Not only would they not know the tech challenges, they also wouldn’t know the business/domain challenges.
[ Parent | on HN ]
Re:Big data is dead (2023) by vegetablepotpie on Monday May 27, 2024 @03:00PM (#40493579)
The flip side of that is that you end up with spaghetti code that is expensive to add features to and is expensive to clean up when you are successful. Then people in the business implement workarounds to handle special cases that are undocumented and hidden.
[ Parent | on HN ]
Re:Big data is dead (2023) by citizen_friend on Monday May 27, 2024 @04:57PM (#40494629)
No it doesn't. Simple and targeted solutions are not bad code. For example, start with a single postgres instance on a single machine, rather than Hadoop clusters and Kubernetes. Once that is maxed out, you will have time and money to solve bigger problems.
[ Parent | on HN ]