The Dawn of the Data Intelligence Engineer

Re:The Dawn of the Data Intelligence Engineer by zodwick on Friday November 13, 2026 @07:10AM (#50482381)

While in general I agree with most of what's said, some of these feel like "Horseless Carriages".

Like for example

>It doesn't know how your company counts revenue, or which of several similar-looking tables is the real one (it is revenue_cpx_new_revised_v2_use_this_one )

This assumption mostly only holds for non-clean data ( which is mostly a result of humans being lazy and backlog being clogged up ). As AI starts building more of the pipelines upstream - this becomes a non-issue.

Even for existing pipelines, I'd expect fixes to move upstream: when an agent encounters ambiguity at query time, the useful output isn't an edit of the context,but a PR that fixes it.

[ Parent | on HN ]

Re:The Dawn of the Data Intelligence Engineer by aleda145 on Friday November 13, 2026 @07:35AM (#50482639)

You can't adequately clean some data though. That your psp had a hickup and only processed 50% of orders on Christmas Eve can't be modeled away. That's where you'd need a context layer to explain why it happened

[ Parent | on HN ]

Re:The Dawn of the Data Intelligence Engineer by zodwick on Friday November 13, 2026 @10:56AM (#50485390)

Yeah fair point tbh. But even that doesn't require an human in the loop no ?

[ Parent | on HN ]

Re:The Dawn of the Data Intelligence Engineer by aleda145 on Friday November 13, 2026 @11:21AM (#50485760)

I guess as long as it gets into the context layer it's all good! I'm excited, but skeptical, for those company OS agents that will be able to figure this out

[ Parent | on HN ]

Re:The Dawn of the Data Intelligence Engineer by zodwick on Friday November 13, 2026 @11:43AM (#50486073)

We're all figuring out as we go i guess :)

[ Parent | on HN ]