As a Java developer, I'm very happy with the compromise that UTF-16 strikes. For day-to-day unicode use, UTF-16 covers all the bases. In the rare cases I need to step outside of UTF-16 to the higher planes, it would work transparently for me up to the point where I start slicing strings naively.
To be honest, I've never actually had to use anything outside of the Basic Multilingual Plane. That's not from lack of breadth in my day-to-day job either - I wrote the first implementation of our web crawler/search engine at DotSpots and had to deal with fetching and indexing pages in languages from most of the BMP (mainly English, Russian and Chinese).
Anyways, I strongly disagree with your assertion that Java was "generally considered" to have "made a mistake" by using UTF-16. It's made my life far easier and the memory costs aren't an issue on today's machines. We don't live in an ASCII world most of the time and storing strings internally as UTF-8 completely ignores this fact.
[edit]
Just so it's clear: UTF-8 only wins when you are representing pure ASCII text. For everything else, it either breaks even or loses. For most Chinese text, UTF-8 is 50% larger:
For characters <= U+0080 (ie: ABC), you win by one byte.
For characters > U+0080 but <= U+07FF (ie: Ȁɐ), you break even at two bytes.
Everything higher than U+07FF in the BMP, < U+10000 (ie: 丂且⬄☃), you lose (by one byte).
For characters >= U+10000 you break even again at four bytes.
If you really want to optimize for ASCII text in Java, there's always byte[] and you're free to wrap a CharSequence around it.
In general practice, however, this is a non-issue. Characters outside of the Basic Multilingual Plane are not in common use, especially on the web. It's not a perfect programming practice, but it's very pragmatic.
Making everyone pay the development tax of variable-sized characters for any sort of multi-lingual code just means that more code will be written incorrectly.
This is a red herring. Because of combining characters, it is rarely valid to slice between Unicode code points, regardless of encoding. Even in the BMP, a semantic symbol can be composed of multiple code points.