Copy "hällö" from Mac to Linux over samba -> works fine.
scp or rsync that same file again from Mac to Linux (without special conversion options) -> you will have two "hällö" there.
The first one is recognized by Samba, the second one is invisible.
On HFS+, because it normalizes, if you try to pass either form to the filesystem, it will treat them the same. But on filesystems that don't normalize (which is most of them) they'll be treated as distinct files. As a result, depending on the input you provide and whether the tools in question do any normalization, you could end up with two files that look identical (e.g. your two "hällö" files) but are different unicode sequences under the hood.
And none of this really has anything to do with UTF-8. UTF-8 is just a byte encoding scheme that can represent all valid unicode sequences.
But this doesn't make it wrong. You can reproduce this exact behavior with other systems too simply by changing the local filename to be decomposed. The only Mac-specific part here is that HFS+ will automatically decompose the filename.
Umlauts on HFS+ are still Umlauts anywhere else. The only real oddity of HFS+ (beyond the fact that it does normalization at all) is that it's not using NFD, it's using a variant of NFD based on an old version of Unicode (it has to be this way because the normalization tables must be immutable or there's compatibility issues when reading drives written to from different versions of the OS). So if you take a filename and convert it to NFD, it may not be the exact same byte sequence that you get if you plug that filename into HFS+ (but in most cases it will be). But whatever byte sequence you get from HFS+ is still going to be a valid Unicode sequence.