News for hackers. Stuff that matters.
Re:The Data Engineering Handbook by fifilura on Wednesday November 20, 2024 @02:26AM (#42191534)
I think it should be worth it. For the reason that you are closer to the product you are building, you help defining what to build.

As opposed to programming which is more like plumbing work.

Re:The Data Engineering Handbook by gregw2 on Wednesday November 20, 2024 @05:47AM (#42192652)
Believe me, there's a lot of plumbing moving stuff from point A to B and dealing with poop ("dirty data" is the industry euphemism) in the data engineering and data analyst space.

In my more analytic moments I try to convince myself that data engineering and analysis is like chemical refining, creating useful byproducts out of raw liquids, but in my cynical moments, the plumbing metaphors for it are just so much more evocative.

Re:The Data Engineering Handbook by fifilura on Wednesday November 20, 2024 @08:48AM (#42193905)
Still, somehow, I think it is where it all started in the 1950s. "I have all these numbers, I need a machine to help me do something useful".

And then after this came a huge industry with programmers.

For me it is more like back-to-basics.