It depends on how fondly you look back at that time and if you have academic inclinations/had strong academic inclinations before you went to college (especially if your family has them).

Plus I'd imagine it depends a lot on the intensity of the college studies. I'd imagine folks going to MIT would remember more stuff than people at a random community college.

Besides what the other folks are saying regarding jogging your memory.