News for hackers. Stuff that matters.
Re:smallpond - A lightweight data processing framework built... by lvl155 on Sunday March 02, 2025 @02:39PM (#43234164)
Looking forward to next few years when we can finally abstract away all the back-end techs.
Re:smallpond - A lightweight data processing framework built... by BobbyJo on Sunday March 02, 2025 @02:48PM (#43234282)
We ain't even solved garbage collection yet, and you think "back end systems" are going to abstracted away in the next few years?
Re:smallpond - A lightweight data processing framework built... by purplerabbit on Sunday March 02, 2025 @03:44PM (#43234861)
Maybe they just mean for the type of projects they care about
Re:smallpond - A lightweight data processing framework built... by BobbyJo on Sunday March 02, 2025 @05:28PM (#43235935)
Can't you already just use FaaS and managed persistence?
Re:smallpond - A lightweight data processing framework built... by tarruda on Sunday March 02, 2025 @02:59PM (#43234395)
> We ain't even solved garbage collection yet

Can you elaborate on that?

Re:smallpond - A lightweight data processing framework built... by BobbyJo on Sunday March 02, 2025 @03:06PM (#43234476)
People still write in languages that force you to manage your own memory.

Once performance starts to matter (either due to scale or time requirements) abstractions always have tradeoffs you can't accept.

Re:smallpond - A lightweight data processing framework built... by pyrolistical on Sunday March 02, 2025 @03:48PM (#43234896)
So then, how can garbage collection ever be solved if it’s a trade-off
Re:smallpond - A lightweight data processing framework built... by BobbyJo on Sunday March 02, 2025 @05:30PM (#43235948)
And how can backends be abstracted away if there is a trade off?

As long as compute is a meaningful percentage of spend, the trade off will matter.

Re:smallpond - A lightweight data processing framework built... by pyrolistical on Sunday March 02, 2025 @05:43PM (#43236046)
Right. So what does it look like for garbage collection to be solved? You’re saying it’s not ever possible
Re:smallpond - A lightweight data processing framework built... by BobbyJo on Sunday March 02, 2025 @07:05PM (#43236732)
I am saying it's not possible for the foreseeable future, yes. The same way backends becoming an abstraction most developers don't need to worry about is also not going to happen in the foreseeable future.
Re:smallpond - A lightweight data processing framework built... by threeseed on Sunday March 02, 2025 @04:43PM (#43235481)
We've had this for at least a decade now.

If you use a cloud provider there are managed solutions for data engineering pipelines.

Re:smallpond - A lightweight data processing framework built... by m2f2 on Monday March 03, 2025 @12:46AM (#43238667)
Sure, and it's not cheap.
Re:smallpond - A lightweight data processing framework built... by agilob on Monday March 03, 2025 @03:28AM (#43239584)
AI isn't going to be any cheaper
Re:smallpond - A lightweight data processing framework built... by lvl155 on Thursday March 06, 2025 @08:32AM (#43279989)
We did? That’s news to me.