If 80% of cars of a house need to be charged in the morning because everybody uses it to go to work, and they use it during the day where else are they gonna charge it than at home and when else are they gonna charge it than after work?
Charging a EV takes longer than pumping gas which is why collisions are more likely. The whole thing might work if the charging stations auto-negotiate some compromise, and leave some cars uncharged or charge some cars less, but waking up and have your car not fully charged will ot be a rare occurance based on the physics of our existing infrastructure alone. You cannot draw more than 43 kW at once in the very typical example building (in fact it might be much less).
Also: everything upstream of those houses (cables, fuses, transformer stations, ...) assumes a certain load factor, running every single house near full load at once is not something that might be feasible, without digging open the street, exchanging that cable, transformer etc.
In Germany this is not the case. Also: A typical residential building over here has 3× 230 V / 63 A which results in ~43 kW. Medium charging power mentioned by Tesla is 11 kW (16 A), so if we wanna leave some headroom for the residents you can charge 2-3 Teslas maximum there at once.
Of course you could dig out the electrical cable leading to the house, add another house connection etc. but that would also mean all the electrics of the whole house have to be brought up to current standards etc. Also at some point the main cable under the road will be the limiting factor, and this has to go as well.
I think EVs have an advantage over hydrogen now that they are not in totally widespread use, because their true infrastructure cost will only appear once everybody wants to charge them at once, and this is certainly not the case right now.