CC-By would require attributing on any page you use it on, right?

Their license simply says to include the license if you re-distribute the font.

Aren’t you re-distributing the font by using it on a webpage? Well, assuming you self-host or provide a third-party link.

The distinction is more between an EULA vs one of the common open source licenses.