You could, but since git isn't designed for handling large binary files the performance will be poor. That's why there are large file support plugins like (the aptly named) Git LFS[0] and git-annex[1].

[0] https://git-lfs.github.com

[1] https://git-annex.branchable.com/