If you setup the compiler with trimming enabled[0] it gets significantly smaller. You can also lazy load assemblies by route[1] to further restrict the upfront cost.
Of course, this is not acceptable for the average web page by any means. This is really intended for behind the login type applications where you load up the initial runtime once and its cached heavily for the rest of the applications lifecycle by the user. This is really targeted at true applications-in-the-browser type situations.
Blazor server side works too, though everything then has to run via a SignalR connection and can be a bit more flaky at scale.
Runtime performance however, I actually don't find it to be a bottleneck. The apps I've built with Blazor are pretty fast. I haven't worked with it in 9 months though.
[0]: https://docs.microsoft.com/en-us/aspnet/core/blazor/host-and...
[1]: https://docs.microsoft.com/en-us/aspnet/core/blazor/webassem...
The benefits claimed were you don't need a modern browser, i think the thin shell + signalR combo works gracefully back as far as IE9 or something silly, also you don't need much processing power on the client because the signalR pipe is just a conduit for pre-rendered HTML generated by a blazor component running server side.
The down side is that for every client, there's a websocket (or long polling connection) for every client to the server.