News for hackers. Stuff that matters.
Re:Engineers Shouldn’t Write ETL by zimablue on Sunday November 04, 2018 @08:18PM (#18379093)
I do this job and I'd say it's only enjoyable for now because I have complete freedom and I'm still learning. Splitting up the parts: Design architecture Wire up pipelines (once architecture is decided, this tends to declarative, connected is choosing schemas). Data science

I'd definitely not want to be stuck doing stage 2 forever, would prefer 3. I think you're saying that you enjoyed a job which was some 1 and some 2. I'm sure there's someone out there who wants to wire up pipelines with no engineering and no analysis all day but I'd imagine it's a rare breed.

Edit: I think the important distinction is team/company size. Doing a bit of everything as a 1 man team is challenging, if you have a team where devops/engineering/reports/tools have been chosen/built/standardized by specialist and you really are just wiring pipelines up, I think that would be tough. On the other hand being in a small team condemns you to always be doing the same fractions of work because there's noone to hand off to.

There are no comments on this comment yet.