Are you trying to say it is better to have A/B testing without statistical significant as compared no A/B testing at all?

I'd disagree with you completely and since you are taking this stance, I am actually worried now about recommending the plugin. Not caring about statistical significance is equivalent to taking decisions at random. You may better flip a coin, why bother A/B testing? You are letting your users take decisions not knowing if those are real or just random.

Keep in mind that this is just step one of many we'd like to take with this plugin. It's not that we don't care about statistical significance - it just didn't make it into this version, so we'll work to get that in there soon.

We will also be adding more in the FAQ tab that talks about these sorts of issues where users could possibly make decisions too soon instead of letting the tests gather more data.

I am absolutely not suggesting you withhold tools and knowledge, but that you should give them more tools tools that fully inform them. When you show "Improvement: 25%" after say, 20 page views - that is simply inaccurate.

Fair enough - it is definitely something we are going to add very quickly.