If you have the main meal in the middle of the day then it is called "dinner". The snack you have at the end of the afternoon is "tea".
If you have the main meal at the end of the afternoon it is "dinner". The snack you eat in the middle of the day is "lunch".
The difference probably depends on if you go home for a meal at mid-day or not. The later is probably more traditional as people used to work or study much closer to home. And there was somebody around during the day to prepare a meal. But nowadays going home for a main meal in the middle of the day is unusual so we all just have a small snack.
On the other hand, the after-work evening meal is usually less heavy, and is rendered as "supper" in English. And, again, I find that Americans usually assume that it takes place much later than it normally does, and is also much lighter than it normally is (basically a snack rather than a proper meal), if I use that word.
So clearly there is some implicit time-of-day mapping there.
And sometimes if you're really lucky you'll meet a Southerner who'll freak out about it, "Ha! You said 'dinner' but you meant 'lunch'!! Ha! Ha! When do you eat your lunch?! Ha Ha!!"
I had an old boss that did this regularly. The dick.