Filling your tank with fossil fuel is 5 minutes wall clock but also 5 minutes where you can't do anything else. If you need to pee or eat that's a few more minutes.
Filling your battery might be longer 10-15 minutes, but (at least for Tesla superchargers) about 20 seconds of your time, for the remaining 10-15 minutes you're free to do whatever you want.
But in any case, 10 minutes is fine no matter what. If you are willing to spend 4x as much for fuel in order to save less than 10 minutes per week when you stop for fuel, then the problem isn't the charge time.