I had a similar experience and ended up doing the same. OPNsense does include a userspace implementation of Wireguard, but is obviously inferior to what Linux already has. It ended up being that, how poorly the initial kernel space implementation was handled for FreeBSD and that I wanted Cake for QoS, which made me move back to OpenWRT.

I do want to ask though, has there been any downsides to using OpenWRT on x86?

Late reply but -

I haven't really run into notable downsides. Installation was somewhat of a pain since I had to resize the partition (all of the prebuilt images assume small hardware), and all of the default packages are as minimal as possible - like busybox for userspace/shell. Upgrading on X86 is I guess possible (I haven't tried yet), but most of OpenWRT's upgrading instructions/support are geared towards reflashing firmware, not updating packages on ext4.

But, there are a bunch of available packages - coreutils, zsh/bash, etc., that can be installed as desired. In general OpenWRT is pretty set-it/forget-it for me.

Makes sense! I'm using wireguard on opnsense (but it's a newish feature) and so far it's been a great experience.