> If you open your distro package manager and look at the dependencies for an end-user app, it will not list header-only libraries

Well, it will list them in build dependencies (e.g. build-depends in Debian).