Git works great for binary files. However, it struggles with large repositories (beyond 5 GB). So large files are a problem, and binary files are often large. But the binariness itself is no problem.

(5 GB is github's "recommended" limit. You can push beyond it, but things will get sluggish and it might not work everywhere.)

git-lfs is clunky and awkward, but may still be the best solution for large files.