I don't think "cult" means "small group" in this context, but "strongly followed".

At least, that is how I always interpreted stuff like "apocalypse now is a cult movie".