The only one that comes to mind is https://research.mozilla.org/2014/08/26/javascript-servos-on... , which is Rust (pre-1.0) and JavaScript, but that somewhat doesn't count because Rust isn't really a GC'd language. (This article was written about a year after the special syntax for garbage-collected references was removed and Gc<T> became just a thing in the standard library, and about a month before that type was also removed because it wasn't a very good GC.)
It seems to me that if you make your GC a purely refcounting system this is easier: you just need to specify how to incref and decref objects, and put a well-known function in the vtable for destroying and deallocating the object. I'd guess that's how things like GObject or Qt bindings in Python or similar languages work.
I personally think ownership is a biggie. If I pass in a pointer, is it expected to outlive the call?
And buffer sizes, of course.
The C ABI and type system omits a lot of stuff that could be useful. Some points off the top of my head:
* Vector types
* Multiple return values
* Functions with bound environments
* Coroutines
* Dynamically-sized callee-allocated structures (most notably lists)
* Distinguishing between "byte array" and "string" types
* Vtables
* Something that lets GC'd languages cooperate more nicely. I'm not sure how exactly to specify this, but this is the sort of thing we could have if we stopped insisting that everything must be in terms of C ABI.