It's not about hand annotating; it's about exposing databases in a well defined way and using standard methods to describe structured data. Look at what dbpedia is able to do with wikipedia data.