So, while it might be nice to a/b test everything - sometimes you just want to start out on the right foot and then test from there.
Hmmm.... I'd caveat that advice a bit. I'd have to be pretty sure that the company I'm copying is facing the same sort of problems I am. Especially since, after N instances of being wrong, I've learned that I'm often quite bad at figuring out whether company Foo is facing the same problem as company Bar!
Especially when you have limited resources and solving one problem means that you're not solving another.
I've seen far too many instances of people coping what Amazon or Facebook do and it not helping in any way because - well - they're not Amazon or Facebook.
Personally I'd try and spend a more time talking to / observing actual users if I don't have the numbers for more quantitative techniques. Something, sadly, most companies seem to do just about as much as they a/b test :-/ That'll give me some strong hints on what the most significant problems are.
Deploying a demo requires a certain amount of effort.
We're bootstrapping so I'm careful with our efforts.
I'm only asking around to figure out if for most people that did it, the effort was worth it.
If you have any traffic at all, the habit of A/B testing will quickly pay for itself.
I see your point though and do agree that once the demo is in place, A/B testing will make sense.
Yes, I know, you're hoping someone else will know the answer. But if you don't develop the habit of testing, you won't be in the habit of testing, and you won't be testing things that are unique to your business.
This is what patio11 keeps on complaining about. When there is a discussion of A/B testing, everyone says how great it is. But over 90% of you simply aren't doing it. And in a thread like this you can see that people really aren't doing it.