You can mostly do that, but: forget about system wide debugging, desktop shell changes, easy security sandboxes, being able to just run new AI software (they're often assuming cuda or CPU), building obscure software without patching makefiles. And the get the general feeling of "whatever I want to change, I'm fighting against the system".