Not entirely true anymore. If you select eg "Quantum Reference (Windows, browser, x64)" and look at the Breakdown list, you'll see a couple of browser benchmarks that look at responsiveness and rendering. Speedometer and Unity WebGL, for example. But yeah, it's mainly all about JS.

Since the Quantum effort started, the JS team has prioritized perf work higher, so quite a bit of perf-related improvements have been made in Spidermonkey for 57 as well. (And Quantum DOM needed some rearchitecting to re-multithread the engine in a limited way.)

(Source: I work on Spidermonkey.)