The failures of the semantic Web are pretty much the same as the failures of the Web of evil, i.e. the internet: 1. You cannot make people tell the truth. 2. You cannot always determine when someone is not telling the truth. 3. You cannot always make people do things the right way. 4. You cannot always determine when someone is not doing things the right way.
So, you are correct. The true creed of each hard-core technologist is: "Everything would work great if only everyone always did everything my way."
The Semantic Web hasn't "failed" and it's not something that end users need to see, know about, or care about directly. It's those technologists that use Semantic Web tech and data to build applications for the end users.
Freebase as a prominent example, was pointless for an average person.
Likewise Github is pointless to an average person. Because the average person isn't who it's meant for.
Freebase as a prominent example, was pointless for an average person. There was no reason for it to exist in regards to doing something for millions of people.
Wikipedia, Quora, Stack Exchange, etc. are what people want to consume. Until the semantic Web leads to a dramatic improvement on those types of end user products, it's not going to matter.