I see this Custom Colour thingie at the top-right corner of the Wallpaper section, above a "Show on all Spaces" checkbox and left of a fairly big representation of the current desktop background.

After a bit of tinkering: this Custom Colour element is replaced with something else depending on the current background mode. If you selected a wallpaper image, it shows the name of the wallpaper. If you select a predefined colour, it shows the name of the color. When adding a custom colour, it will show an interactive element which allows to change the color in place, and that shows the buggy behaviour for me.

Ok, this at least explains why other people don't stumble over this as an obvious bug, I assumed it would be obvious, because the first thing I always do on a new Mac is to customize the background color by right-clicking the desktop, and since that moment I have that buggy Custom Colour element sitting there.

Not a great UX either way though.

PS: ...and now after adding a new custom color via the to bottom row of predefined colors, the bug in the 'Custom Colour' widget is gone and nobody will believe me it was ever there. Great :D

PPS: nope, it's coming back after going through the 'desktop => right-click => change wallpaper...' route again, phew.