It's more that your enamel is weakened temporarily by acids (which get produced by bacteria in your mouth as a side effect of their consuming the sugar). It is true that you shouldn't brush immediately after a meal -- waiting 30 minutes to an hour is a good practice.