Love it, I had a similar but probably worse idea a few years back while woodworking. I agree the issue with essentially all CAD and sketchup etc is it forces you to work with primitives that are different from your real world ones. Extruding shapes as you said, instead of actually milling lumber and working in common known dimensions.

I also suspect there’s a fairly finite number of sane join primitives, you can have rabbets, dados, dominos, dovetails, pocket screws, etc. These could be experimented with digitally against an existing design. And your tool availability could determine what you can do.

And of course cut order and assembly is just a DAG and can be generated from geometric constraints of the actual design.