Getting a sample from the surface basically means you need to do a deceleration burn to get into orbit, and then the reverse after grabbing the sample (which itself must be accelerated to orbital speed). No small thing.
Entering orbit almost always requires an orbital insertion burn, and in the case of an Earth-Mars Hohmann transfer definitely does.
Your point stands though: returning from a fly-by (including a free-return) requires much less fuel than returning from orbit, and even that's much easier than returning from the surface.
For fun, take a look at rotovators (AKA "momentum exchange tethers") that could make orbital capture/departure—and even landing/launch—"free" on bodies with minimal atmosphere.
The difference in difficulty (mainly, fuel requirements) between a flyby and an orbit is a crucial distinction often overlooked.
This page describes the process: https://medium.com/spaceinmylifetime/how-spacex-will-refuel-...