I know more than a bit about statistics and A/B testing. My first reaction is that asking for more information from a limited sample set does not actually give you the power to do more powerful statistics. Claiming otherwise is a bad sign.

Secondly a lot of what A/B testing gets at is subconscious reaction, not conscious reactions. Consciously when we're asked a question and are paying attention, we are poor judges of how we're going to react to, say, a larger button to click on. But when we're actually using the site, we notice and hit the larger button.

And thirdly, if you're actually going to seek out a small sample of people for detailed information capture, do it in person. I can't stress this enough. If you think up questions in a vacuum it is hard to ask the right questions. That is part of what makes A/B testing so frustrating. But if you have a conversation and are observant, you'll quickly generate good ideas.