If you write your library in Perl, guess what, only people writing Perl will use it. If you write it in Java, well, maybe the Scala folk will wrap it up, or the Jython folk, but you won't get any people writing in Ruby to use it. If you write it in C++, you can provide efficient bindings to any of those languages.
For an example, look at FFmpeg or ImageMagick. They are libraries that just _won't die_, no matter how hard people have tried. My god, ImageMagick was written in 1987, and it now has Haskell bindings[0]! The only way to get that sort of longevity (and old code is good code, after all [1]) is to write in language that will outlast the ups and downs of the language or framework du jour, and C and C++ have proven to have that staying power. Yeah, they're terrible; I hate them. But there is a reason to use them.
[0] http://hackage.haskell.org/package/imagemagick
[1] http://www.joelonsoftware.com/articles/fog0000000069.html
- Another layer to debug (FFI)
- Possible loss of performance when crossing the FFI barrier
- Loss of productivity, because that cool API isn't available and so one has to write it
- Writing drivers, daemons/services, OS plugins, ...
One of the reasons I moved from Turbo Pascal to C++, when I started coding on Windows 3.0, was being fed up to write binding libraries by myself. Although I really enjoyed Turbo Pascal back then.
Which happens to be C++, until OS vendors support another one on their SDKs.
So until Ada, Rust, D, Swift, .NET Native or something else get such a position, C++ it is.