fwiw I've been running homebrew out of my ~ folder without any problems, because I'm not a fan of that default either. But yeah, use what works :)
I really like that aspect of homebrew. I don't let anything else use /usr/local, and it means I can install things without using sudo, so no risk they are messing something up elsewhere in system directories.
But you risk the installation messing with your data and home directory. MacPorts uses sudo because it drops privileges to a specific 'macports' user for the build process and applies sandboxing. Therefore, the execution of Makefile rules and other build scripts cannot harm your data.
Apple has a habit of not updating the libraries in Mac OS X until absolutely necessitated by a security vulnerability.
https://trac.macports.org/wiki/FAQ#ownlibs
One other reason I use MacPorts is because I didn't like the idea of HB changing permissions of /usr/local:
http://superuser.com/a/371976
HB may have changed this in the last couple years but I haven't bothered looking over the fence: MP has been near bulletproof for me, and has provided nearly everything relevant that I've needed.