That said, I really need to write some code to extract the document metadata from the PDF files and display it on page; right now, the only thing that search engines (and the site itself) have to go off is the filename, which is far from optimal.
In fact, the same file could be uploaded multiple times with different metadata. There's room here for experimentation, e.g. publishing content hashes and linking "duplicates" that have different metadata.
If PDF metadata can be published in a structured format, it should be then be possible for Calibre or Docear / Zotero to import the PDF + JSON metadata directly into the document database.
Perhaps crowdsourcing metadata might be an idea - but that involves quite a bit more complexity, implementation-wise.
Too idealistic?
Google would do the rest.
[1]: http://searchenginewatch.com/article/2067225/Google-Does-PDF...