It's a question of what your baseline is. It seems OP may have had (say) a 3500 calorie diet as a baseline, then started exercising, perhaps burning an extra 500 calories but now consuming 4000 calories to feel normal. In your case, it sounds like you had a lower baseline--let's say 2500 calories. You may have consumed as much as 2000 additional calories, but you ran it all off--your comparatively lower calorie count was what felt normal to you, and what you made a habit of aiming for.