The light elements that you prefer for fusion, have a lower absortion rate for neutrons than the heavier elements, which you don't prefer to absorb neutrons. In other words, to get usable fusion, you need so much neutrons that a lot of material in/out of the reactor will get too radioactive.