"Balances" are stored on every block. If you threw away the beginning of the chain, you could keep generating blocks, but you couldn't spend any transaction outputs that are stored in the thrown away parts of the chain.

Unless I'm missing something something pretty big about how Bitcoin's blockchain works.