Yup, you nailed it. cmake for handling the builds, and the C++11 stdlib covers every platform-specific thing I needed. The only platform-specific thing I needed to do was point the CMakeLists.txt at the correct libusb (libusb.so, libusb.dylib, libusb.dll). Absolutely beautiful.

Edit: to elaborate a little more about cmake... You can use cmake to generate both Makefiles and Visual Studio build files.

On Windows:

mkdir build; cd build; cmake .. -G "Visual Studio 12 2013"; cmake --build .

On OS X or Linux:

mkdir build; cd build; cmake ..; make