Can't you just tell them what you're doing..?

"I like to wander around campus a little during lunch to recharge. Thanks for asking though, maybe dinner?"