Yes, but when I exchange bitcoin off chain, I'm giving up a lot of the benefits of using bitcoin in the first place.

Lighting in particular solves transaction speed by introducing additional potential security problems that need to be accounted for, particular closed channel type attacks.