nabroleonx

it is understandable to a certain degree and it is entirely dependent on your company policy. however, with dbslice browser UI, you can audit every column and make sure nothing falls through the crack and get a signed-off config. once you do that you can just use that yaml file to do as many extractions as you need

the compliance profile + UI will be on the next release

this is the main reason i started doing the compliance profiles for. you can choose a compliance like hippa and dbslice auto-applies masking rules, scans the output for residual PII, and generates an audit manifest your data custodian can review.

if you want to see what is being masked before anything runs, i also have a browser UI where you can review every table and column. see which fields each compliance profile covers, adjust mappings as much as you want(select what columns to anonymize and how), and export the config

both will be out in the next release.

that is a valid point. dbslice finds cycles in the fk graph and usually resolves them by nulling a nullable fk for insert order, then patching it back with deferred updates after inserts. if a cycle has no nullable fks, postgres output can still work when deferred fk checking is enabled and the cycle constraints are deferrable, otherwise it fails fast with a clear error.

traversal automatically pulls in parent records so you don’t end up with dangling references, and a validator (enabled by default) can double-check the slice before output. for complex joins, you can opt into subqueries in seed where clauses.

it covers a lot of messy cases, but i won’t claim it’s fully solved yet. there’s no automatic discovery of relationships that only exist in app code (beyond heuristic hints), and real production schemas will still surface new edge cases. it’s still early-stage, so the more people test it on messy production-like datasets, the faster i can iron those out.

i would also love to hear what you think of the implementation if you check out the code.