But neither GCC nor Clang support Microsoft's debug symbols as far as I'm aware. If you're writing libraries for Win32, your users are likely using Visual Studio and they're going to get unhelpful stack traces without those debug symbol files.

Mixing MinGW and MSVC object files is indeed a problem as neither toolchain understands the other's debugging information.

If you can get away with it, just don't support MSVC. If that's not an option, your best bet is probably restricting yourself to the common subset of C and C++...