A huge factor is what your used to. If you regularly consume sweet things you get used to that flavor and stuff tastes bland without it.

On the other hand, if you eat a rather bland diet you start noticing many starches taste sweet as your saliva breaks them down.