Oh, sure, but it's as if someone told me "what you're doing now isn't going to work, so we don't want to work with you on it". It makes me think that maybe I need to be doing something else, but all the thousands of users who use and love the service count more than six people who talked to me about it for ten minutes.

Honestly I think you're giving YC too much credit. Sure, they're a great service, but jesus christ, they aren't the promised land.

Half of the YC startups will probably fail. You have users who love your service, so you're already WAY ahead of the game.