The body temporarily shuts down multiple processes like digestion during exercise to increase peak performance. However, it still needs food etc so those calories are still burned during downtime. Active people can consume 2-3x as many calories in a day as inactive people do. At the extreme end some elite athletes need 8,000+ calories per day diets simply to maintain weight.