The JVM is going to be the preferred server target at Google over C# and .NET by a wide wide margin. Which probably rules out Xamarin.
Add to that the fact that native Android is already Java (source). And GWT is a mature technology that is Google maintained. Seems like the logical route to take.
Otherwise, yes, they could have written the shared logic in C#, or C++, or a few other languages, all of which are portable between the platforms they want.
There might also be history, I believe gmail has been using Java/GWT, so it was a natural decision for them to continue using Java?
It is a little surprising on the performance side, though. I would expect C++ to be faster when compiled/transpiled to all three relevant platforms (iOS, Android, Web), compared to Java similarly compiled/transpiled. But maybe most of their time is spent rendering anyhow, and little in the shared code.
Simple fact is, Google is a Java oriented company. Inbox uses lots of components and shared code across the company that are based on Java. To use C# or C++ to do it would rewriting a lot of code that already exists.
JS performance is not the gating factor for the Web client in most cases, it's rendering and startup (code size). We run expensive calculations off the main thread in a Web Worker anyway.
The translation of Java to Objective-C could have been to C++ as well. I started prototyping something a while ago that does such. Objective-C makes more sense, because for iOS, the front end developers get to consume 'idiomatic' code that looks like they had hand-written it and it meshes well with other tools, and would work with Swift too.