A second factor is the effect of a lifting body on stability, and the cost of compensating for it. Even in the case of nominally non-lifting fuselages, they tend to produce lift at high angles of attack, and that tends to be destabilizing, producing an increasing pitch-up moment as the angle of attack increases (ordinary straight wings do too, but the effect is proportionately less, as the wing chord is considerably less than the fuselage length.) This has to be corrected with measures (typically a larger stablizer) that also create drag. Having a fuselage designed to produce lift exacerbates this problem, commensurably with the amount of lift being generated.
Modern sailplane design is a case study in the persuit of aerodynamic efficiency, at least at low speeds, and lifting bodies have not proved to be useful. The fact is that, once you are flying, you do not need more lift; what you want is less drag.
https://en.wikipedia.org/wiki/Lifting_body