News for hackers. Stuff that matters.
Re:A/B Testing vs MAB algorithms - It's complicated by ckluis on Monday September 17, 2012 @11:51AM (#4533110)
At the end of the day, I think A/B testing is utilized more just because it is easier to understand & implement. I would love to utilize something like unbounce for MAB testing, but I haven't seen good MAB frameworks that are ready for the average marketer to use.

It would be nice to have a MAB framework that utilizes MVT so that you could preload 100s of alternates (c2a, headline, copy, images, placement) and see let it go to town.

Re:A/B Testing vs MAB algorithms - It's complicated by btilly on Monday September 17, 2012 @01:15PM (#4533602)
You make good points. However the ultra-simple MAB for home pages is a solvable problem. Noel at Myna has been responsive to me. If you give him feedback on how he can target that market segment, particularly if you're willing to be a paying customer, he'll likely be responsive.

MVT is inherently more complex. I guarantee you that there are MAB based systems that do MVT on problems of insane complexity. I pointed at one system at Yahoo that likely is an example. And the potential win isn't small. Yahoo claims a 300% increase in clicks on their home page module since implementing it.

But I haven't noticed anything like that publicly available. (I have seen companies that do it behind the scenes - but the one that first comes to mind hides that detail and tries to be priced on a pay for performance model.) Of course a company that already does MAB could build a custom solution. Or you can try to build your own if you think there is money in it. My personal life does not currently permit me the time commitment that a startup would require. But if you or someone else wants to build it, I'd be willing to consult on how to make it work.

Re:A/B Testing vs MAB algorithms - It's complicated by ckluis on Monday September 17, 2012 @03:43PM (#4534460)
My first piece of advice is that screenshots & videos are good.

Picture is 1,000 words. A video is 30,000 words a second.

I dream of a MAB/MVT combo solution. I have a project coming up that I might try this out on.

Re:A/B Testing vs MAB algorithms - It's complicated by noelwelsh on Monday September 17, 2012 @05:08PM (#4534905)
Usability for non-programmers is certainly not a strong point of Myna at the moment. Right now the best place to look is the demos: http://myna.github.com/myna-js/ The redirection demo is an example of how you can do something like Unbounce.

In terms of the MVT style stuff you discuss, we're in very early stages of looking at it. Not sure when it'll arrive -- depends on how much interest we get.

Re:A/B Testing vs MAB algorithms - It's complicated by larrydag on Monday September 17, 2012 @12:05PM (#4533196)
Good fair analysis of both implementations. I really agree with the article writer that the best implementation is the one that is right for their business.
Re:A/B Testing vs MAB algorithms - It's complicated by btilly on Monday September 17, 2012 @01:38PM (#4533732)
My hope is that I helped people figure out which one is right for their business. :-)