Linux with “memory folios”: a 7% performance boost when compiling the kernel

Re:Linux with “memory folios”: a 7% performance boost when c... by IntelMiner on Tuesday June 15, 2021 @07:05AM (#27513569)

Not a scientific codebase, but ffmpeg seems to benefit massively from optimization

I use it to encode my DVD and Blu Ray TV/movies to HEVC. Doing so reduces the filesize on DVD's by roughly 80% and Blu Rays by 60%

The downside is the encoding is an incredibly CPU intensive process. Hardware encoders like Nvidia's Nvenc or Intel's QuicSync look absolutely terrible and are a non-starter for archival storage

On a stock Fedora XFCE install, I would get roughly 0.5 FPS for a 1080p Blu Ray file (29.97 FPS at 1920x1080)

A Gentoo Linux installation with a global -O3 -march=native as well as LTO, PGO and Graphite enabled globally boosts it from 0.5 to roughly 1.3 FPS. Still slower than realtime, but an absolutely massive improvement

[ Parent | on HN ]

Re:Linux with “memory folios”: a 7% performance boost when c... by astrange on Tuesday June 15, 2021 @01:39PM (#27518764)

I suspect -march=native is the only thing doing any work here. The other optimizations are as likely to find compiler bugs as they are to improve things, once you get off well-tested paths.

[ Parent | on HN ]

Re:Linux with “memory folios”: a 7% performance boost when c... by bayindirh on Tuesday June 15, 2021 @02:22PM (#27519433)

I think it's the opposite. A good annotated code which can use SIMD instructions accelerates tremendously with -O3. -march=native -mtune=native generally improves things if you can saturate the cores with instructions, which can be measured by running perf record and looking to IPC & instruction retirement numbers.

When I was using Eigen on my code, the biggest performance boost came from -O3. -march and -mtune did minimal improvements on the systems which I've ran benchmarks on.

[ Parent | on HN ]

Re:Linux with “memory folios”: a 7% performance boost when c... by astrange on Tuesday June 15, 2021 @02:57PM (#27519903)

ffmpeg and associated projects do not rely on autovectorization, which pretty much only works on naive scientific code. They write their SIMD in assembly and don't need to care about compiler settings.

[ Parent | on HN ]

Re:Linux with “memory folios”: a 7% performance boost when c... by bayindirh on Tuesday June 15, 2021 @03:33PM (#27520381)

If you're compiling hand tuned assembly, -march and -mtune probably will have more effect when compared to compiling and optimizing C/C++ code.

OTOH, I'd like to underline that heavily optimized scientific code and libraries are neither naive (in terms of algorithmic complexity/implementation) nor straightforward :D

E.g.: This is how Eigen configures its internal vectorization parameters: https://gitlab.com/libeigen/eigen/-/blob/master/Eigen/src/Co...

[ Parent | on HN ]

Re:Linux with “memory folios”: a 7% performance boost when c... by ta988 on Tuesday June 15, 2021 @09:43AM (#27515423)

That's an amazing compression ratio, how is the quality?

[ Parent | on HN ]

Re:Linux with “memory folios”: a 7% performance boost when c... by IntelMiner on Tuesday June 15, 2021 @10:50AM (#27516341)

ffmpeg recommends a crf of 23 to preserve 1:1 for H.264 -> h.265

I do a probably overkill CRF of 20 just to be safe. But everything looks absolutely perfect, even blown up on my 65 inch (1080p) TV

[ Parent | on HN ]

Re:Linux with “memory folios”: a 7% performance boost when c... by bayindirh on Tuesday June 15, 2021 @11:12AM (#27516663)

Can you share your complete parameters, so I can do the same thing for my DVD archives & other stuff?

No hard feelings if you don't want to though :)

[ Parent | on HN ]

Re:Linux with “memory folios”: a 7% performance boost when c... by IntelMiner on Tuesday June 15, 2021 @10:46PM (#27524319)

Sure! This is the script I wrote for it. I use MP4 containers so I can import them into MAGIX (formally Sony) Vegas but it will iterate over an entire folder of MakeMKV files and convert them https://dpaste.com/DBPA9C59M

[ Parent | on HN ]

Re:Linux with “memory folios”: a 7% performance boost when c... by bayindirh on Wednesday June 16, 2021 @02:09AM (#27525401)

Thank you. It's greatly appreciated. :)

[ Parent | on HN ]