I usually keep a separate repo for PSD/AI files (completely neglected to consider them as binary files).

As for images, icons, fonts and similar, I just have a build script that copies/generates them, if it's needed.

I guess I've always been a little bit "obsessed" about the tidiness of my repositories.