That is actually a nice idea (and I’ve seen designers’ personal pages that do do that), but would probably look too artsy on today’s web. Gestures on mobile are also quite married to the vertical direction, so unless you’re willing to switch directions depending on screen size (way too jarring) that’s going to be somewhat broken as well.

Mouse wheels. That‘s the issue. They mostly scroll only vertically. Horizontal scrolling frequently is a massive pain for everyone involved.