For two monitors on one video card, things are pretty okay on Ubuntu and derivatives. More than that, especially 3+ with non-uniform geometries, and you will have fun. If you have multiple video cards, you will have even more fun.
Setup was pretty simple using the binary nVidia drivers on Kubuntu, 0 config file wrangling.
Now, having those three monitors in non-uniform geometries is something I wouldn't even consider tackling
[EDIT]
Kubuntu 12.04
All monitors in a single X Session and automatic application window sizing works as you would expect in KDE
(i.e. Maximising a window maximises to the current monitor only and being able to drag to right side of one monitor causes the window to use up half of the current monitor)
I ran Gentoo for years, and my experience was similar to yours. I could get things to work but it took a lot of frustrating trial and error, and then periodically a version upgrade would break everything and I'd get to start over. About a year and a half ago I finally got sick of it and installed Kubuntu.
Kubuntu 12.04 + 2 monitors + AMD proprietary drivers = you get to choose between accelerated OpenGL and multiple monitor support. You can't have both at the same time. (At least I couldn't, with the video card and driver versions I had 6 months ago. I spent many hours trying, then remembered I have more money than time and bought a $90 Nvidia card.)
Kubuntu 12.04 + 2 monitors + Nvidia proprietary drivers = easy. The GUI worked, first try, no problems. The only goofy thing was that my left monitor showed up on the right side, giving me a choice between physically moving the monitors or changing the default positions in the GUI. I changed the defaults in the GUI.
Also, on my former work machine, Kubuntu 11.04 and 12.04 + 2 Nvidia cards + 3 monitors = easy. (Some cow-orkers had 2 Nvidia cards and 4 monitors, also easy.)
I haven't tried newer versions of Kubuntu on a multi-monitor box, so it's possible (but not likely) they've gone backwards.
You could have switched the cables. Just saying :)
I did use xrandr to setup the virtual screen so that I could define the exact position of one screen relative to the other, which enabled objects to continue in an horizontal line across monitors.