DHCPv4 and DHCPv6 are different (they even run on different ports) and probably they have more experience with monitoring and operating DHCPv4.

Honestly I think the fundamental mistake of IPv6 was trying to fix everything else along the way. Certainly there are things that can be fixed in DHCPv4. But there isn't a way to put off that operational work until you're ready to get good at it; there is no way to just get more addresses and not also sign up to run a new version of DHCP, to decide if you even want DHCPv6 or you want RAs and SLAAC, to give up ARP, etc. It's a big lesson in the second system effect.

What was the problem with ARP anyway? Why Neighbor Discovery is better?

As I understand it, ARP is abstraction-breaking and hard-codes that IP is running on top of Ethernet or something Ethernet-compatible - you're sending Ethernet packets (with a non-IP EtherType) that reference IP and an IP address, initially over Ethernet broadcast. NDP runs over ICMPv6, which is an IP protocol, and the way it works is that it's sent to well-known IPv6 multicast addresses, ff02::1 for all nodes or ff02::2 for all routers. If you're a node and/or a router, you join those multicast groups. So there's no assumption of Ethernet; that just depends on some link layer with a working multicast. (And broadcast is a perfectly legal implementation of multicast, although if your layer 2 protocol makes it easy to do a better implementation, great.)

Also, there's some multicast fanciness so you join a multicast group based on some hash of your IP address so that NDP packets don't even go to all machines, but just some hash bucket that contains that IP address.

(I think these are good reasons to make a successor to ARP, but I don't think these are good reasons to require that you use the successor protocol if you don't want to NAT, especially given the extent to which deployed networking gear does ARP snooping and IGMP snooping - and does not expect a pile of multicast groups.)

That's what I thought too, I really tried a lot of configuration client side, on different OS (linux, bsd...), but none was working reliably, even the provider preconfigured linux image. It might be client that are all broken or something else, to be honest I didn't investigate it further, as I don't need IPv6, I just wanted to be "ready" for the future.