It's not even close to two orders of magnitude, and usually significantly less than one order of magnitude. For chicken and fish the feed conversion ratio (FCR) can be less than 2. See https://en.wikipedia.org/wiki/Feed_conversion_ratio#Conversi... (FCR can be 1 or even <1, though the difference there presumably comes from water weight--input feed is dehydrated while outputs are measured by weight, which includes some amount of water. Whether FCR includes only dressed meat depends on the data source, but for chickens and fish it's mostly irrelevant because everything ends up being processed into food, and by weight the carcasses are mostly protein. If we're pessimistic about published FCR numbers I think we can ignore the water weight difference.)

Also, farmed animals aren't very picky about the look of their food. In some aspects industrial meat production permits us to make more efficient use of agricultural land in terms of providing calories and especially proteins for human consumption.

What's hidden in the FCR number is the energy required to process the feed stock to get those high FCRs. But a human consuming "whole foods" also needs more of that input for the same amount of useable calories. Flours and similarly processed foodstuffs require more energy than that required for the inputs to livestock. So in terms of energy use we're basically just looking at the fixed overhead of the animal conversion as captured by FCR, possibly offset by some efficiencies that would be absent in direct-to-human consumption.

TL;DR--modern industrial agriculture is crazy efficient. Beef isn't as horribly bad as we think (notwithstanding the methane problem), but consuming pork and, especially, chicken and farmed fish shouldn't be very guilt inducing, ceteris paribus.