If anything, smaller sites have more to gain from HTTP/2 and HTTP/3 than the likes of Google. For example:
- Both HTTP/2 and HTTP/3 seek to reduce the number of round trips, mitigating latency between the user and the server. Now, from Google's perspective, the "server" is the nearest load balancer in a globally distributed network, which is probably geographically close to wherever the user is. Thus, users with good Internet connections typically have low enough latency for the improvements not to matter much. But Google still cares about latency because of users with poor internet connections – such as anyone on a cell network in a spotty coverage area. Well, poor connections affect all sites equally. But small sites tend to not be fully distributed; they probably only have a single origin server for application logic, and perhaps a single server period, if they're not using a CDN. That means a fixed geographic location, which will have higher latency to users farther away even if they have a good connection – thus more benefit from latency mitigation.
- QUIC can send stream data in the first packet sent to the server, without having to go through a SYN/ACK handshake first. TCP Fast Open lets plain old TCP do the same thing – but only when connecting to a server you've seen in the recent past (and retrieved an authentication tag from). Thus, QUIC is faster when connecting to a server for the first time – which affects smaller sites a lot more than Google.
End users complain all the time about latency. And that includes the latency to your small website hosted on a single server hundreds of milliseconds from your visitor... certainly more than it includes google's websites.
What you really mean is that small website operators generally don't care that their visitors are irritated by how slow their website is... and just brush it off and ignore it because they have no solution to the problem.
Maybe you should consider h2 as being for the benefit of visitors across the internet, and a benefit for those who care about performance.
It says it all that even though h2 is not required, small website have adopted it across the globe... now at 1/3rd of all websites, and growing.