The first such choice was to support a strictly static subset of Java - one that eliminates reflection - so that the compiler can prove exactly which lines of code will be used in the application, and aggressively eliminate those that are not. The compiler does this by building a graph rooted at the entry point and following method calls. For example, if your GWT application uses the ArrayList class, but it never uses its remove() method, then the code for remove() is not included in the resulting JavaScript output. They refer to this as the "pay as you go" design philosophy.
Other minification techniques are used in addition to this, but the most impact comes from choosing a strict static subset of the source language.
The next choice that the creators of GWT made was to make the compiler aware of a special method GWT.runAsync(). When the compiler encounters this method in the aforementioned graph, it knows that it can break the javascript into pieces, so that code on the far side of that call is loaded lazily when it is encountered, as opposed to in the initial bundle. This "developer guided code splitting" saves the user from having to wait for a 5 megabyte download when they only need to wait for 1 megabyte, etc.
There's no reason that Emscripten cannot use these techniques, other than an unwillingness to make the hard choice of sacrificing some expressiveness in the source language.
Tell that to Gmail. They load at least 2MB of code. New Twitter loads 1MB. Facebook used to load that much, but it appears they've cut it down a lot recently.
It's certainly prohibitive for webpages and lightweight page-oriented web applications, but not single-page web apps. I would expect this to be used more for the latter.
This is definitely cool, but I think something like Google's Native Client is needed to make C/C++ work on the web. Maybe Emscripten could be used as a fallback for browsers without NaCl support.
Currently that is basically the situation, yes (but it depends on the benchmark - some run much faster). However, improvements to Emscripten can probably get that up to 25% of native speed. As LLVM and JS optimizers like Closure get better that will improve even more.
And JS engines are getting faster too. Things like type inference will work particularly well on code like this, since the code is basically statically typed (but not explicitly).
So, in the long term I believe this approach can yield code that runs at very decent speeds. Plus, when comparing to NaCl it will have two main advantages: (1) it doesn't require a plugin and runs everywhere, even on iPads, and (2) it doesn't have the sandboxing overhead of NaCl (NaCl code itself is fast, but communication with outside code on the web page is slow).
A more general caching system would be better. For example, "Cache by digital signature", as some people above mention.
As the size of their executables are already competitive with their C/C++ counterparts this is not easily fixed. CPython's "Javscript" runtime is 2.8 megabytes whilst CPython's C runtime is 2.2 megabytes, less than a 30% increase. With compression the JS size decreases, but not by enough.
A 2 megabyte JS file is likely prohibitively large. Even if common JS "executables" were hosted by Google, as is currently done for jQuery and other libraries, the problem is only mildly alleviated.
Emscripten is an amazing technical feat but I fear due to this it will see little usage. Their main focus currently is improving performance, but is anyone looking into decreasing the size or improving the handling of these "executables"?