Got it. It's OK to be honest that things aren't growing without feeling like the site "sucks".

Have you tried having a 15 minute Skype conversation with your 10 most passionate users? They can tell you loads about what they love/hate. You'll get some good clues about how to find more people like them and convince them to share it with others.