Aren't iptables just an emulation layer on top of netfilter?

Yes, AFAIU (not an expert), iptables and nftables are two command line tools and abstractions (chains vs. tables) for interacting with the same underlying netfilter API.

I believe at one time they were two separate subsystems, but they got merged in 4.x or 5.x

I run 6.3 and the incriminated files were not compiled in my kernel thanks to CONFIG_NF_TABLES=n during make config.

Probably depends on the distro. Iptables is a wrapper around nftables in most distros, but probably not all.

You can check with: iptables -V

If it says (nf_tables), you are using the compatibility layer from the iptables-nft package.

It works quite well. Apps like Docker that inserts rules using the legacy iptables syntax are oblivious to the fact that they are actually inserting nftables rules.

It also provides an easy migration path. Insert your old rules using your iptables script then list them in the new syntax using nft list ruleset.

The problem is that it works so well that it seems most users just stayed with the iptables syntax and did not bother migrating at all.

IMO, the problem is that the people who created nftables (and the "ip" tool) couldn't create a user interface that anyone but themselves would like to use. Linux traffic shaping functionality suffers from the same "obscure word soup" interface.

I agree for the "ip" tool (from iproute2).. I got used to it but I still prefer the ifconfig output. It is somehow consistant and you can get used to it.

I somehow got accustomed to the nftables rules format. It is in fact objectively much better than the iptables format in many ways. The native JSON, easy bulk submit to the kernel, built-in sets and maps (the source of the currently discussed CVE though). It really does fix a lot of what was wrong with iptables.

But iptables was probably not broken enough for most users to warrant re-learning everything.

Now, the traffic shaping tool, oof.. I still cannot grok any of it. I've been happy with the fireqos script so far to abstract everything out of the tc syntax.

For modern distros, the nft package includes an alternative binary that takes the place of /sbin/iptables and translates the input to an nft compatible format. As far as the kernel is concerned, iptables is still iptables. Old iptables can be accessed by calling the iptables-legacy binary which will auto load the old iptables ko.