Nope, it's also a scarce resource. And similarly to water, excessive use is (usually still) bad for the planet.

But internet connectivity is somehow inherently limitless and deliverable for free?

Internet connectivity is all about maximum capacity. Everything past that is just sunk costs.

It doesn't cost anything more to use it at maximum capacity than it does to just leave it on standby. (The end points MIGHT use a TINY bit more power, but we're talking on the order of cents per month for terrabytes/month of traffic; it's a rounding error.)

The electrical analogy is more closely comparable, but I've found water is a simpler concept for most to comprehend.