It's very easy to load React only on the page that needs it. Use dynamic ES module import, or link the standalone builds only on the pages that use it.

Alternative like Preact is very small and can be easily (pre)cached on the client side.