One wart of unique_ptr (and other smart pointers) is that it cannot be passed in a register when used as a function parameter, at least with the System V ABI used on Linux.
Also, the caller is responsible for destruction and there is no way to specify that a function always "consumes" a unique_ptr so the compiler cannot eliminate the destructor code: https://godbolt.org/z/sz79GoETv
Of course if the compiler can inline the call or at least controls both and can clone the function with a custom calling convention then that doesn't have to be a problem. But it still sucks that even something as seemingly simple as a tiny wrapper around a pointer does come with a cost.
As a C programmer, I try to avoid tracking ownership in separate struct member fields. I try to make central data structures that keep care of the tracking. Cleaning up shouldn't happen pointer-by-pointer. Usually a much bigger context has a shared lifetime, so there is no point in splitting stuff up in individually tracked "objects". Instead you just track a bigger block of memory.
It's like the classic argument against using exceptions: compared with the traditional C method of completely ignoring error conditions and not checking status, they're much slower.