I haven't really run into notable downsides. Installation was somewhat of a pain since I had to resize the partition (all of the prebuilt images assume small hardware), and all of the default packages are as minimal as possible - like busybox for userspace/shell. Upgrading on X86 is I guess possible (I haven't tried yet), but most of OpenWRT's upgrading instructions/support are geared towards reflashing firmware, not updating packages on ext4.

But, there are a bunch of available packages - coreutils, zsh/bash, etc., that can be installed as desired. In general OpenWRT is pretty set-it/forget-it for me.