News for hackers. Stuff that matters.

nickdpi

Re:An A/B Testing Story on Tuesday September 20, 2016 @04:10PM (#12542916)
We shot a second trailer video on The A/B Testing Manual’s site that was the exact same pitch, only with my dog (64lb border collie) in my lap.

Seriously contemplated A/B testing them as a joke, but erred on the side of sanity.

Re:An A/B Testing Story on Tuesday September 20, 2016 @03:20PM (#12542440)
Hi, I'm the one who's putting together The A/B Testing Manual with Patrick. Gonna answer a few questions (thanks for asking 'em!):

1. On the difficulty of A/B testing: It is heavily dependent on a lot of different factors in your organization. Developers' ability to quickly generate variant pages is a huge issue: lots of places simply can't make changes to revenue-generating pages. Lots of places deal with politics that keep them from rationally analyzing A/B tests. And c-level support can ram through a lot, of course.

I know that we're all from planet Vulcan and are very hyper-logical about testing, but most orgs don't work that way. 98% of my work involves doing therapy on people who are used to making design decisions by internal debate. Shifting that culture is tremendously difficult in large organizations where lots of smart people have lots of strong opinions.

Put another way: yes, it's easy to run an A/B test. It is hard to ship the revenue-generating design decisions that come from A/B testing.

2. On where to start: I try to research my test ideas so I know that I'm testing the right things, in the right places. Messaging tends to have much higher impact than specific design elements, so you want to make sure you're communicating to people effectively.

With total blank-slate clients, I get your GA install in order (nobody ever has this), and run heat & scroll maps on all the key pages in your funnel. I also email a survey of your existing customers to see if there are any surprising patterns. And with my bigger clients, I even run usability tests (on usertesting.com) and customer interviews (recruiting through ethn.io) to understand their motivations for purchasing.

None of this has anything to do with actual A/B tests – but it has everything to do with making sure you're not stabbing in the dark when you do test.

Finally, agreed with @gk1 on big, drastic changes. Harder to implement (see point about dev time above) but much more likely to bear fruit – and force the org to really question how they're speaking to customers.

3. It's a new article. The A/B Testing Manual was launched yesterday at abtestingmanual.com. Videos will be ready in a month or two, tops. Launch discount expires this Friday.

Great questions. Keep 'em coming!

Re:The A/B Testing Manual on Monday September 19, 2016 @09:39AM (#12531324)
Thanks for the feedback!
Re:The A/B Testing Manual on Monday September 19, 2016 @08:50AM (#12530994)
Hi, I’m making this with @patio11. He’s a regular here, so I figured I’d post it and spend some of the day answering questions.
Hey, creator here.

Obviously this depends heavily on the system, but I hope to address both the logistic/technical and support side of the migration in our final month together. You should come away with at least the overall strategy for how to execute on this.

Re:Startup Stagnation on Friday August 24, 2012 @07:03PM (#4430165)
Worth noting that this is an excerpt of a (much) longer essay in Distance: http://distance.cc

Some of the critiques here are totally fair, but folks should read the whole thing before forming a final opinion.