That's not the risk. The risk is introducing harmful bacteria into a bottle and then having it reproduce for some days before someone drinks it. Since you're not storing milk in the glass it's not an issue if it isn't completely sanitized.