If a one way trip takes X amount of fuel and supplies (let's assume it is the same for an outgoing one way trip and an incoming one way trip, for simplicity), then a two way trip has to carry an extra X outgoing to use for the return trip.

Since X is going to be many times the mass of the ship and crew and non-consumables, having an extra X as cargo massively ups the fuel requirements for the outgoing trip.