This is not really a use-case for Prometheus, as you already suggested by the lacking capabilities. APM suites have tooling tailored for this, e.g. New Relic or Sentry (but using these for this purpose only is overkill, otherwise expensive and vendorlocked), or you could use purpose-built js libraries. Alternatively you could instrument your frontend app and leverage tracing to get a better understanding of changes in performance and bottlenecks, which is recommended regardless of web vitals but from reliability perspective.

https://www.debugbear.com/ can track those for you and alert when there are changes. It keeps the full (Google Lighthouse) breakdown in the history.