News for hackers. Stuff that matters.
Re:There is no Data Engineering roadmap by camgunz on Friday July 14, 2023 @05:52AM (#36721735)
We have a DBT-style system that shunts "bad" data to "rejected_<model>" tables where we can generate reports. We also have tons of SQL-based tests to prevent inaccurate data from getting through the pipeline. So far so good, but it took us a little while to get there.

There are no comments on this comment yet.