To follow up on this - with A/B price testing how does this work when you are lowering and raising your prices all the time - what's the general approach - to admit 'mistakes' or refund the difference or just manage on an adhoc per issue raised basis ?
A recurring service that is purchased, where you priced things too high and saw prices drop off, and then quickly moved it back - telling the customers who paid too much, and who would likely get pissed and leave when they find out you were unlucky represent an opportunity to market yourself - saying "Hey, we over did it a bit - we're crediting what you already paid forward at our new reduced rate, and keep charging you at that rate going forward, we appreciate your business!". If anything, that's going to make your customer who was willing to pay double what you are now charging him happy, and spread the word. Who doesn't like to find out they just got a 50% refund?
You don't lower and raise your prices all the time, you do it once to determine the sweet spot and then you stick with that for a much longer time.
For me it was exactly as described: 2, evaluate, 2, evaluate, *2 evaluate -> go back one step.
At each step we had at most a few hundred signups processed before we made the decision.
The fact that it took as long as it did is only because I was skeptical, I should have realized much earlier that it can't cost too much to try, after all your signup rate would have to halve before you start to lose money.
It seemed to be the most fair way to deal with it.
This is easier to see with one of those apps that keep track of apps that lowered their price, some of them keep appearing on that list.
The concern I could see with this would be that you raise your price for a few days - you get a few takers but not as many as you hoped, so you lower the price back down to the original levels. I'd be a little concerned that those who subscribed at the higher levels would not be happy paying a higher recurring monthly fee when they could now subscribe to the product at a cheaper rate.
Would you / did you just manage those users back down to the original payment levels for the future months payments ? Or leave them at the higher payment levels and manage on an adhoc basis should they raise issue with pricing levels.