>if you had a site where 3rd parties uploaded copyright materials for download you would be held responsible

Only if you had knowledge of the infringing material, or if your site overtly encouraged its upload. Otherwise, as long as you comply with the DMCA you would be eligible for safe harbor. (I think.)