Engineers Shouldn’t Write ETL

Re:Engineers Shouldn’t Write ETL by jboggan on Friday March 18, 2016 @12:04PM (#11312679)

I thought it nailed a lot of dynamics at my last gig as well. Of course, I joined for the challenges of scaling and wrote an ETL framework in Rails (that was a mixed bag but very instructive) and then got bored after I realized how small our data really was. Then I left for Google and all my data went up by two prefixes.

I do love reading an article that supports my contention that ETL is the Charlie Work [0] of software engineering.

0 - http://www.avclub.com/tvclub/its-always-sunny-philadelphia-c...

[ Parent | on HN ]

Re:Engineers Shouldn’t Write ETL by mikestew on Friday March 18, 2016 @12:20PM (#11312778)

I do love reading an article that supports my contention that ETL is the Charlie Work [0] of software engineering.

Though I've never formally done work with the "ETL" label, what I've seen of it reminds me of the work I used to do for client years ago where I'd take some CSV file (or what have you) and turn it into something that their FoxBase-based accounting system could use. It was boring grunt work (or shall we say, "Charlie work"), but I billed it at my usual rate and it paid the mortgage. I would never, ever wish to make a career of it, however. (And if my assessment of what someone knee-deep in ETL does all day is completely off base, I apologize.)

[ Parent | on HN ]

Re:Engineers Shouldn’t Write ETL by jboggan on Friday March 18, 2016 @09:09PM (#11316651)

The thing is, I kinda identify with Charlie. I rather enjoy getting it done and it's the kind of thing that keeps the ship sailing smoothly. Sometimes you can derive a strange appreciation from thwacking rats in the basement, so to speak.

[ Parent | on HN ]

Re:Engineers Shouldn’t Write ETL by kafkaesq on Friday March 18, 2016 @12:11PM (#11312715)

It's all Charlie Work for the VCs, "founders", and (occasional) early hires who take in the lion's share of profits and prestige for the long hours most of us pour into our jobs -- day after day, year after year.

BTW, that may sound polemic, but it's not -- that's really how a lot of business types, academic researchers, and others think of nearly all programming-related work.

[ Parent | on HN ]

Re:Engineers Shouldn’t Write ETL by spyrosg on Friday March 18, 2016 @12:44PM (#11312990)

Not that polemic indeed, but only if we're willing to stop gazing our own navels. Most of us here aren't hired with nice paycheks because we're smart and creative, but because we're smart, creative, and yet without the sense to find better work than Charlie Work.

Anyway, this is a shame. I thought "data science" still had nice R&D flavored jobs. Colour me disillusioned.

[ Parent | on HN ]