I believed the same thing, right up until I picked up a HomePod, which quickly cured me of it.

Does it shake your chest? Not quite. Can it literally shake the windows? I mean, make actual windows produce an audible buzzing noise?

Yes, it emphatically can. I would still choose a larger cone for a bass guitar, though; it's right there in the name.