One of the problems with replying to every piece of feedback is that a link on Google's search page--even a link that many people don't notice--gets a huge volume of feedback. And the resulting feedback can be pretty unhelpful ("I lost my cat. I think you should build a search for lost cats.") So we don't have the resources to reply to all of those. Personally, I would like if we offered a lower-volume feedback method with a higher hit rate of good feedback. Hacker News is acting as the prequalifier in this instance, but I find that people annoyed enough to blog often provide good feedback too.

Overall I agree with you, but even if we had that channel, many times the feedback would be: "Yup, that's a bad set of search results. We'll try to come up with a way to make it better, but it might be a while. Getting an algorithm to do better on this search will be hard."