Other than when I restart the mac it sometimes connects, my android devices connects no issues, and bootcamp/VM do so too without problem, yes it could be windows.
I also used to have very slow transmission speeds between the two devices until I changed the mac to 2.4G (from 5G) (PC is wired)
Are you sure it's the Mac? In my experience, things suddenly start working on the network once you remove Windows from the equation.