I also think a more important qualifier is the primary focus being development of a proprietary product or service, as opposed to e.g. consulting or providing infrastructure. (Not sure exactly how to word it properly to make the distinction clear, but I think you get the gist)

In my book, a VC money is not a prerequisite for being a startup (although that tends to be how things go these days)