> I don't know anybody who learned math or physics outside of a college classroom.

In fact, I will go one further. Almost all self-taught physicists I have met are kooks - bonkers crazy conspiracy theorists.