Every single encoder, regardless or hardware or software will output different output quality. Video Encoding is a lossy encoding, not lossless like ZIP or RAR. That is the same with Audio that is why people test different audio encoder. It seems more people knows this about Audio than Video. At least on HN.
GPU encoding quality has always been lower quality than Software. Primary because they trade off quality for speed. It is good enough once you hit certain bitrate, but at low bitrate where absolutely encoding efficiency is required Hardware Encoding just dont compare to software. Even with Software you could have multiple encoder with different results. It is not like H.266 / HEVC only has x266 as encoder, example Netflix uses BEAMR. And Broadcasting TV station uses other encoder that better suits their needs.
And it is one reason why I dislike all these AV1 discussion, whenever people said it is slow they said use SVT-AV1, well yes SVT-AV1 is faster but dont produce the best AV1 quality encode. So what is the point. It is like every time these discussions about AV1 AOM supporters will just move the goal post.
If you want to retain visual quality, you always have an option to use higher bit rate.
What I should say is, please show me an AV1 encoder that can beat x264 in quality at a source-transparent encode of a 4K or 1080p Blu-ray film, if given ~3GB per hour of content and the encode had to run in at least 3x the runtime. I’d start using it! It may be there as of recently, it’s been a year or two since I looked.
This is a usecase where AV1 encoders and even x265 have had trouble.
If you only use h.266 and you're willing to wait extreme amounts of time to encode, then that's valid, but understand that you're an outlier. Most people don't have that much time to spend on encoding.
Does it mean just that nVidia and AMD engineers done a poor job when implementing those encoders?
Compare x264 veryfast and veryslow presets. There is a quality difference at the same bitrate.
Additionally, GPU encoders don't have as many psychovisual options as CPU encoders as they would need to be included in the hardware and adding extra options to CPU encoders is much faster, easier and cheaper.
You could build a non-realtime GPU encoder, but there is not much point.
FWIW, as someone who is a complete layman on this topic but has spent much of the past 6 months converting age-old videos to h265...
i have a pi5 where i queue up old videos into a RAM disk, it converts them to h265, and moves them back to the machine which uploaded them. It can take up to 2 days to compress any given video (depending mostly on the input format - old AVIs tend to compress in a few hours), but the space savings is _much_ higher than what you're reporting. The average size of my h265-encodes is anywhere from 1/3rd to 1/2 of the original, and 1/4th of the original is not terribly uncommon. That said: the input formats vary wildly, with possibly only a minority fraction being h264.
i've tried the same thing using a pi4 but it needs roughly 4-5x as long as the pi5. An 8-core intel-based laptop can do it in some 25% of the time of the pi5 but pulls a lot more electricity while doing so. So my otherwise idle pi5 gets the assignment.
My Firefly collection (14 tv episodes plus the movie) dropped from 8gb to 2.1gb. My Bob Ross collection (141 videos) went from 38gb to 7.4gb. 128 Futurama episodes/movies was cut from roughly 50gb to 21gb.
i was once asked why i didn't choose AV1 and the answer is simple: when i googled along the lines of "how to get the best video compression with ffmpeg" the most common answer was h265. Whether that's correct or not, i can't say, but it works for me and i've freed up the better part of a terabyte of space with it.
I do most of my encoding on an i9-13900k. I've also noticed that h.265 seems to be quite a bit more likely to trigger the smt related crashes in ffmpeg, which makes it a pain when I've queued up a lot of things to encode over night.