I'm probably misunderstanding something but on my system (Ubuntu 14.04 LTS) the X server runs as root & not as my user.
Ubuntu does not enable it by default, as it mixes poorly with some drivers: https://wiki.ubuntu.com/X/Rootless
AMD only added KMS to their proprietary driver last year, and Nvidia this year (and IIRC only in a beta driver so far); and systemd makes the permission handling a lot easier. So Ubuntu will transition to it eventually, but didn't have all puzzle pieces until too recently for even 16.04 LTS.
It's a standard feature of Xorg nowadays, but it's only a feature of vanilla Xorg for a few years.