Sure. It'd drastically complicate the build process and CI for little benefit compared to other approaches (e.g. sparse checkout), but you could even in principle create a repo for each code directory and stitch them all together.