You can, but it's probably not worth the effort. Most of that's in the form of lazy-loaded byte code. Other than some lightweight stubs for the entry points, Emacs doesn't load those pieces until you use them.