>I'd actually expect performance to improve since the body isn't using resources for digestion

This is exactly what I've noticed as well. Even a few hours after eating my runs feel sluggish. All my PRs are from running first thing in the morning, before any food.