I've found that while this technically true, if you're building a classifier of any importance, you're going to need to monitor it and grade the efficacy of any changes, which will require labeled data anyways.

This kind of approach works for one-shot "let's just see if it works" type stuff, but anything that I would stand behind needs at least some level of labeled data and curated dataset.