I also am not sure (and never thought about it until just now) as to where the forfeited core charge money "goes" for consumers who buy a new phone (paying the core deposit) and then don't return a core phone within the required amount of time or if they buy a brand A phone and return an old brand B phone.
But I suspect that the overall model could work pretty well with some of the details carefully thought out. (AFAIK, in the lead-acid battery case, I can't just go to a parts store and force them to buy my core for $20, but if I am buying a new battery, they are forced to take my old core instead of charging me the $20 core charge [or whatever it is nowadays].)
Aka, you buy an Apple phone at your carrier, you turn in your Samsung phone, and Samsung gets sent the bill for the phone's collection and recycling. This would work even if someone just wanted to drop an old phone off at the store: Samsung's still responsible for footing the bill, so stores have no reason not to collect.
If you agree to that premise, now the store has to have a way to figure out whether a given phone has a core charge refundable from the manufacturer, whether the phone is genuine, etc in order to not have the core credit they paid out rejected by the manufacturer.