I grew up in a college town and I can highly recommend. I grew up with kids of professors and because of that (and, of course, my own parents) had a great academic environment. Schools in college towns often punch above their weight.

Another benefit is access to cultural benefits that wouldn’t normally be available, like great libraries and concerts and orchestras (often free). Check out Pullman, WA as another example.