Really, I do. "Oversee" can be used in a sentence as a verb without the connotation. But when I see "oversee.net" -- the noun -- I think "overseer", and "overseer" is not a word that I can hear without thinking "slavery".
I don't get the slavery connection from "overseer", either. That's still a common term, and not closely tied to slavery. It just means someone who oversees, a manager/supervisor/foreman. Dictionaries (the ones I checked at least) don't even mention slavery, though I did notice a lot of slavery-related "related searches" on dictionary.com.
Obviously what you see is what you see, so you're not wrong, but I think the connection isn't something most people would see.
Edit: Seriously, do a search on the term "oversee". It's a common term and it's related to slavery the same way "driver" is, which is to say basically not at all.
http://www.google.com/search?q=oversee