People often switch from more processed grains to whole grains specifically to increase their fiber intake and reduce the processable calories they're ingesting. I don't think it's nitpicking at all, this is the most basic stuff you need to think about when doing dieting - CICO is naive at best.