In the system I'm familiar with, votes are counted twice. The first count is done on site, by the same people who staffed the polling station. It's important that the votes are counted and the results are published before those people leave and before the ballots are moved.

Then there is a second count that produces the official results. That count is done at a higher level (county-equivalent for most elections and city/municipality for local elections), largely to ensure a more uniform treatment for various edge cases.