The linux kernel forces your application to include the vDSO shared library (by pre-loading it). You can ignore it and talk to the kernel directly, but you cannot get the same performance as you could when using that shared library.
On some architectures, atomics are implemented in the kernel by the vDSO shared library (__kernel_cmpxchg), which is supplied to user via libatomic. You can ignore it, but then you cannot interoperate other code (any which uses libatomic.so) without introducing data-races into the code which were not present in the shared-library version, since they may attempt to execute their atomics differently and thus incorrectly.
There is no platform ABI mandate nor Linux kernel requirement for userspace applications to be dynamically linked to anything. If you can talk to the kernel, you can do anything those libraries can do.