You can store and diff them, but not merge them.

It's basically the same as using Word's own history features.

Wow, this is going to be very helpful for me. I had no idea. Thanks! So if I install git on Windows and do `git diff MyWordDoc.docx` in the command prompt it'll open up Word?

> You can also use the Git attributes functionality to effectively diff binary files. You do this by telling Git how to convert your binary data to a text format that can be compared via the normal diff.

First, you’ll use this technique to solve one of the most annoying problems known to humanity: version-controlling Microsoft Word documents. Everyone knows that Word is the most horrific editor around, but oddly, everyone still uses it.

...Now Git knows that if it tries to do a diff between two snapshots, and any of the files end in .docx, it should run those files through the “word” filter, which is defined as the docx2txt program. This effectively makes nice text-based versions of your Word files before attempting to diff them.

https://git-scm.com/book/en/v2/Customizing-Git-Git-Attribute...

I am using Tortoise. Not sure about the command line.