To give another example - diff is a pretty common action to see changes between two specific versions of the code. Now if I'd treat both code and non-code assets as equivalent (because they are human produced, etc.), then I'd expect the tool which manages them to also show me diff between versions of binary files. But simple byte-by-byte diff* would be pointless, exactly because non-text files need to be decoded before they can be used. So if I'd treat them as equivalent, I'd expect graphical binary assets to show me the picture with differences between two versions; I'd expect video binary file to show me which time points are different in the recording, etc. etc.
* Well, yes, not exactly byte-by-byte, since it needs to be encoding-aware. But still, much simpler than parsing any random binary file format to show reasonable diff.
[EDIT:] Perhaps we were both unclear! By "code", I mean stuff that can be read and edited by normal humans, which by some automatic mechanism can be used to control computers. E.g., if you had a log of photoshop actions that could be replayed to produce a given image, that would be code, and it would be suitable for storage in git. In general, the image itself ought to be elsewhere.