You probably subconsciously downregulate your physical activity when you reduce calories, due to lower energy levels. If you wear a fitness tracker and equate number of steps to high calories you should see a difference even with slight reductions in caloric intake. Even the amount of fidgeting you do during the day can change your base caloric expenditure.