> A wallet with $100 in it gets $5 in and then sends out $5. How do you decide if this was the $5 that went in? Is it first in first out?

Even assuming you mean "address" rather than "wallet", you can decide by looking at which UTXO was consumed. This is a drawback of Bitcoin.