Keep them within easy reach, like the driver door pocket.

Changing glasses while driving on a high speed lane is dangerous regardless of where you put your spare glasses

Considering what people do whole driving, I doubt this will add any measurable risk.

On the contrary, if you are a perfect driver and only reach for you glasses, that adds a small risk. But this is a complex system, driving and multitasking, so added complexity surely compounds, maybe even exponentially. If you are texting, then talking to your kids, then your brother next to you starts blasting loud music on the radio, adding another task like changing glasses increases the risk of an accident a lot, because less and less you are concentrated on the traffict.