I turned down a couple of better name universities for a lesser known university (for graduate school) because my advisor there was top of his field. At that point, the university doesn't really matter. You look at the quality of work of your advisor and how well funded he/she is.