> basically, if you cd Projects/foo, that should be it's own user

Agreed. I went further and turned that into its own isolated virtual machine. The credentials problem is really annoying though. AI agents need the access in order to be useful.