I have experienced the same, though I’ve done less visual comparisons. Comparing % quality like-for-like between webp and avif, on average avif files are slightly larger than webp counterparts.

Things didn’t materially improve when using high compression (slow) encoding settings. Could have been a bad encoder (libavif iirc)— there might be an explanation but on the face of it I saw worse performance than webp.