The De Beers cartel was able to avoid anti trust scrutiny because the best reserves are outside (Africa/Canada/Russia) of the most lucrative markets (US/Europe/Asia).

Corporations control only a small faction of housing supply.

Diamond prices are famously crashing now due to artificial lab made diamonds (increased supply)

Insanely stupid restrictions on residential construction make it easier.

They can only do that (own them all) with government assistance. See De Beers diamonds, who leveraged the legal system to create a monopoly.

The solution is to build more housing and get the government out of the picture.