NumPy/SciPy's install page tells you to use Anaconda (or equivalent) for Windows, and build from repositories for Linux.

http://www.scipy.org/install.html

Granted, it does not say "and by the way, when you do that you get tons of other great things like pandas, matplotlib, etc". But they are very clear about not trying to do this yourself.