Big data is dead (2023)

Re:Big data is dead (2023) by ImPostingOnHN on Monday May 27, 2024 @08:35AM (#40490179)

I remember replacing a CLI program built like Lego blocks. It was 90-100 LEGO blocks, written over the course of decades, in: Cobol; Fortran; C; Java; Bash; and Perl, and the Legos "connected" with environmental variables. Nobody wanted to touch it lest they break it. Sometimes it's possible to do things too smartly. Apache Spark runs locally (and via CLI).

[ Parent | on HN ]

Re:Big data is dead (2023) by pdimitar on Monday May 27, 2024 @08:52AM (#40490289)

No no, I didn't mean that at all. I meant a script using well-known CLI programs.

Obviously organically grown Frankenstein programs are a huge liability, I think every reasonable techie agrees on that.

[ Parent | on HN ]

Re:Big data is dead (2023) by actionfromafar on Monday May 27, 2024 @09:20AM (#40490466)

Well your little CLI-query is suddenly in production and then... it easily escalates.

[ Parent | on HN ]

Re:Big data is dead (2023) by pdimitar on Monday May 27, 2024 @09:31AM (#40490545)

I already said I never managed a data lake and simply got stuff when it was needed but if you need to criticize then by all means, go wild.

[ Parent | on HN ]