I was mainly worried I'd be spending time downloading noveau/radeon drivers and associated packages on a server with no attached GPUs. I've been leaning towards languages that compile fat binaries (and running with docker regardless), so this is why I'm a little wary of Ubuntu bringing too much to the table. Also, it's been a long time since I ran Ubuntu on a personal machine, I am still a little worried about the risky the dist-upgrade process can be.
Basically all I feel I need is ufw, docker, ssh and was worried that Ubuntu brings too much along for the ride.
[0]: https://vadosware.io/post/trying-and-failing-to-get-lxd-runn...
People also get overly bent out of shape for having Amazon search integration with the desktop at one point, which I did not like, but it had a clear, functional way to disable that function.
At least, those are my theories.