This needs to be done by more people, more often. I've worked with more than one (often SQL, but it doesn't matter¹) database whereby someone will claim that X is always true. It X is something that can be put into a DB query, then you should do that, and run the query. Every time. In my experience, if it isn't enforced by constraints in the database software, then it isn't true.
¹The nice thing about SQL is that you can but constraints on the data, and use the database to keep various invariants invariant.
Often, you'll find people making assumptions about the data based on the business logic, not the constraints in the database. Such thinking is flawed: you cannot reason about what you think the data is, you must reason from what the data actually is.
Worse, you'll find people making assumptions about what they think the business logic is.
I can't comment too much on it publicly but well, it looks like it was worth it. I definitely agree that there is a split between statisticians, and "data engineers" in both personality and interests, and you should have both on your "data science"/"big data"/whatever they call analysts these days team. It is however bloody hard to convince upstairs to pay a lot of money for the work! (as with anything "without deliverables", really)
After twenty years of financial operations, including a decade in the back-office of a top hedge fund, I eventually accepted that I'm a bit backwards for my desire not to jump straight to a pivot table when encountering a new data set. Like a farmer reaching down to touch the soil, my first step is exploring the rows and columns with little tests here and there to find weaknesses within the information at hand rather than paving over them with instantly flawed reporting.
Anyway, for all the big data scientists out there too sexy to clean up their own data, I'm looking for work and don't mind pushing a broom. Check my profile for more info.