News for hackers. Stuff that matters.
Re:FFmpeg by Example by jazzyjackson on Tuesday January 14, 2025 @04:49PM (#42704383)
This reminds me I need to publish my write up on how I've been converting digitized home video tapes into clips using scene detection, but in case anyone is googling for it, here's a gist I landed on that does a good job of it [0] but sometimes it's fooled by e.g. camera flashes or camera shake so I need to give it a start and end file and have ffmpeg concatenate them back together [1]

Weird thing is I got better performance without "-c:v h264_videotoolbox" on latest Mac update, maybe some performance regression in Sequoia? I don't know. The equivalent flag for my windows machine with Nvidia GPU is "-c:v h264_nvenc" . I wonder why ffmpeg doesn't just auto detect this? I get about 8x performance boost from this. Probably the one time I actually earned my salary at work was when we were about to pay out the nose for more cloud servers with GPU to process video when I noticed the version of ffmpeg that came installed on the machines was compiled without GPU acceleration !

[0] https://gist.githubusercontent.com/nielsbom/c86c504fa5fd61ae...

[1] https://gist.githubusercontent.com/jazzyjackson/bf9282df0a40...

Re:FFmpeg by Example by dekhn on Tuesday January 14, 2025 @07:03PM (#42705676)
I used ffmpeg for empty scene detection- I have a camera pointed at the flight path for SFO, and stripped out all the frames that didn't have motion in them. You end up with a continuous movie of planes passing through, with none of the boring bits.
Re:FFmpeg by Example by hnuser123456 on Tuesday January 14, 2025 @08:14PM (#42706173)
Then can you merge all the clips starting when motion starts and see hundreds of planes fly across at once?
Re:FFmpeg by Example by dekhn on Tuesday January 14, 2025 @08:32PM (#42706281)
Interesting. Yes, I assume that's possible although I'm not sure how you handle the background- I guess you find an empty frame, and subtract that from every image with a plane.

One of the advantages of working with image data is that movies are really just 3d data and as long as all the movies you work with are the same size, if you have enough ram, or use dask, you could basically do this in a couple lines of numpy.

Re:FFmpeg by Example by hnuser123456 on Tuesday January 14, 2025 @09:34PM (#42706663)
You could do an add operation if the background is dark enough, or yeah, take a local sample of the background to subtract from each clip, and choose a specific background...
Re:FFmpeg by Example by rahimnathwani on Tuesday January 14, 2025 @05:18PM (#42704723)

  -c:v h264_nvenc
This is useful for batch encoding, when you're encoding a lot of different videos at once, because you can get better encoding throughput.

But in my limited experiments a while back, I found the output quality to be slightly worse than with libx264. I don't know if there's a way around it, but I'm not the only one who had that experience.

Re:FFmpeg by Example by icelancer on Tuesday January 14, 2025 @06:32PM (#42705412)
So counterintuitive that nvenc confers worse quality than QSV/x264 variants, but it is both in theory and in my testing as well.

But for multiple streams or speed requirements, nvenc is the only way to fly.

Re:FFmpeg by Example by Gormo on Thursday January 16, 2025 @01:06PM (#42728666)
Why's that counterintuitive? It makes intuitive sense to me that an approach optimized for throughput would make trade-offs that are less optimized for quality.
Re:FFmpeg by Example by icelancer on Wednesday January 29, 2025 @02:20AM (#42862462)
Oh it's just odd/funny to me that a graphics processing unit is not very good at processing graphics (in single cases) compared to a central processing unit.
Re:FFmpeg by Example by xnx on Tuesday January 14, 2025 @05:39PM (#42704928)
Co-signing. Encode time was faster with nvenc, but quality was noticeably worse even to my untrained eye.
Re:FFmpeg by Example by jazzyjackson on Tuesday January 14, 2025 @05:42PM (#42704950)
Fascinating, it didn't occur to me quality could take a hit, I thought the flag merely meant "perform h264 encoding over here"

Edit: relevant docs from ffmpeg, they back up your perception, and now I'm left to wonder how much I want to learn about profiles in order to cut up these videos. I suppose I'll run an overnight job to reencode them from Avi to h264 at high quality, and make sure the scene detect script is only doing copys, not reencoding, since that's the part I'm doing interactively, there's no real reason I should be sitting at the computer while its transcoding.

Hardware encoders typically generate output of significantly lower quality than good software encoders like x264, but are generally faster and do not use much CPU resource. (That is, they require a higher bitrate to make output with the same perceptual quality, or they make output with a lower perceptual quality at the same bitrate.)

[0] https://trac.ffmpeg.org/wiki/HWAccelIntro

Re:FFmpeg by Example by throwaway4PP on Tuesday January 14, 2025 @07:36PM (#42705932)
It’s crude, but you can overcome the degraded quality imparted by gpu accelerated encoding by specifying a higher video bitrate.

Find a complex short scene in your cpu encoded video, extract it, ffprobe it to get average video bitrate, and take the same clip in raw and try gpu accelerated encoding at +20% bitrate. From there, iterate.

For a friend’s use-case that I helped with, +30% video bitrate bump overcame the degraded vquality.

Edit: strangely enough, if memory serves, after the correcting +30% was applied the actual ffprobe bitrates between the videos were very similar, maybe a 10% or less difference. Someone smarter than me can work that logic out.

Re:FFmpeg by Example by norgie on Tuesday January 14, 2025 @06:29PM (#42705389)
Cutting videos with copy can result in some weird playback issues if the cuts aren't on keyframes.

For manually cutting up videos, I use LosslessCut, which I think uses ffmpeg under the hood and is really helpful for finding and cutting on keyframes.

Re:FFmpeg by Example by pertymcpert on Tuesday January 14, 2025 @07:35PM (#42705925)
Yes. The reason is that there's many ways to encode a video, you can choose to spend a lot of compute in order to get the best quality, or the best size efficiency. Fixed function hardware encoders by nature have much less freedom in how they encode, and additionally are expected to be faster, or even real time.
Re:FFmpeg by Example by ziml77 on Tuesday January 14, 2025 @09:09PM (#42706513)
IIRC they have improved the hardware encoder over the generations of cards, but yes NVENC has worse quality than libx264. NVENC is really meant for running the compression in real-time with minimal performance impact to the system. Basically for recording/streaming games.
Re:FFmpeg by Example by jack_pp on Tuesday January 14, 2025 @11:32PM (#42707331)
> Probably the one time I actually earned my salary at work was when we were about to pay out the nose for more cloud servers with GPU to process video when I noticed the version of ffmpeg that came installed on the machines was compiled without GPU acceleration !

Issue with cloud CPU's is that they don't come with any of the consumer grade CPU built-in hardware video encoders so you'll have to go with the GPU machines that cost so much more. To be honest I haven't tried using HW accel in the cloud to have a proper price comparison, are you saying you did it and it was worth it?

Re:FFmpeg by Example by radicality on Wednesday January 15, 2025 @12:16AM (#42707579)
Are the hardware encoders even good? I thought that unless you need something realtime, it's always better to spend the cpu cycles on a better encode with th software encoder. Or have things changed ?
Re:FFmpeg by Example by jack_pp on Wednesday January 15, 2025 @12:45AM (#42707723)
I know they used to be worse, haven't tested the newest ones
Re:FFmpeg by Example by jmb99 on Wednesday January 15, 2025 @03:41AM (#42708747)
They still suck compared to software encoders. This is true for both H.264 and H.265 on AMD, Nvidia, and Intel GPUs. They’re “good enough” for live streaming, or for things like Plex transcoding, or where you care only about encoding speed and have a large bandwidth budget. They’re better than they used to be, but not worth using for anything you really care about.
Re:FFmpeg by Example by zos_kia on Wednesday January 15, 2025 @05:23AM (#42709355)
That's my experience too. I transcode a lot of video for a personal project and hardware acceleration isn't much faster. I figure that's because on CPU I can max out my 12 cores.

The file size is also problematic I've had hardware encodes twice as large as the same video encoded with CPU.

Re:FFmpeg by Example by siscia on Wednesday January 15, 2025 @05:50AM (#42709521)
Would you, or anyone else, be interested in ffmpeg in the cloud?

Connect credit card, open a web UI, send the command, the files, and eventually get the output?

Re:FFmpeg by Example by zos_kia on Wednesday January 15, 2025 @06:27AM (#42709740)
I would SO love it ! I regularly take a look at the existing offerings, and there's a few options for "transcode video as API". However it's pretty costly, i regularly have batches of videos that would set me back 30 to 80 bucks if i were to transcode them in the cloud. I don't think it can be done at any price point i'd be happy with for this kind of personal project - especially considering that the alternative is just to max out my CPU for a day or two.
Re:FFmpeg by Example by jack_pp on Thursday January 16, 2025 @09:11PM (#42733327)
Well it wouldn't be hard at all to make a POC for yourself. You could make an open source project to automate it all. I suggest using hetzner (cloud) because of the price.

You just need to use the hetzner API's to put all your video on a shared drive, write a simple job runner in whatever language you like or even simpler you could write your commands in a text file on the shared drive. Write a simple script to mount the shared drive, look for the job file on machine startup; then have your machine delete itself via hetzner API. Email yourself before that. There, you have your weekend project.

Re:FFmpeg by Example by jazzyjackson on Wednesday January 15, 2025 @02:37PM (#42715874)
Thanks for that datapoint, I was a little bummed to see ffmpeg not using any of my Macs GPUs, but the CPUs ain’t no slouch so I’ll just go with software encoding on Mac
Re:FFmpeg by Example by jazzyjackson on Wednesday January 15, 2025 @02:35PM (#42715861)
We were a quick and dirty R&D team that had to do a lot of video processing quickly, we were not very cost sensitive and didn’t have anything other than AWS to work with, so I can’t speak to whether it was worth it :)
Re:FFmpeg by Example by Gormo on Thursday January 16, 2025 @01:04PM (#42728636)
> I wonder why ffmpeg doesn't just auto detect this?

Hardware encoding is often less configurable and involves greater trade-offs than using sophisticated software codecs, and don't produce exactly equivalent results even with equivalent parameters. On top of that, systems often have multiple hardware APIs to choose from that often different features.

FFMpeg is a complex command-line tool intended for users who are willing to learn its intricacies, so I'm not sure it makes sense for it to set defaults based on assumptions.

Re:FFmpeg by Example by Trixter on Friday January 17, 2025 @12:45PM (#42741043)
In your snippets, you don't appear to be deinterlacing. If your pre-digitized clips are already deinterlaced, that's fine, but if they're not, you're encoding interlaced material as progressive, and mangling the quality. Try adding a bwdif filter so that your 30i content gets encoded as 60p (which will look more like the original videotapes).