I've been using website testing tools for quite a number of years, and I've literally never heard anyone refer to GWO's testing setup as easier. GWO was an implementation nightmare.

Every test had a distinct set of JavaScript tags that had to be individually implemented for every single test. You had to place the scripts on each of the pages, and then verify that all 3+ of the script snippets are in the correct place (and if your test failed validation, you couldn't launch it). Making changes on the fly was difficult. If you misconfigured your goal, your data was forever lost (and you couldn't know if you misconfigured your goal until the data came through, and that wasn't in real time).

Contrasted with Optimizely, for every project, there's a single global JavaScript snippet that goes everywhere. No need to change snippets for every test. You can configure your tests from one place. And, for page-view level goals, you can add goals after an experiment starts, and they'll retroactively apply experiment data.

For a simple test, you can implement an Optimizely experiment in less than 5 minutes and feel assured that everything is working immediately, since it has real time data. If you need more power or complexity, it's there, but if you don't need it, it doesn't get in the way.

(Disclosure: I've been using Optimizely for nearly 4 years and recently spoke at their user conference.)

Sorry to hear that! Our goal is to make Optimizely so easy to use it just works. Clearly we've failed in this case. I'd love your feedback on what we could have done better to improve your experience.

FWIW, two weeks ago we launched our Optimizely Academy to help folks like you go from zero to hero: https://learn.optimizely.com/hc/en-us

The thing is that, when you need an academy or getting people certified, your product is too complicated to use.

You are generally right. The vast majority of our customers don't need help and hence our homepage is basically just a text box to enter a URL and get started.

That said, there are other folks who do need a little extra help so we built them an academy.

On the contrary, it might just mean that their product is expanding beyond its original tech-centric customer base and is now reaching a much larger but less tech-savvy demographic as well.

That's a very naive view of the role of an academy. Most of the best ones go far beyond product training and into best practices and skill development.

It's not the role of HootSuite's product to teach you the theory & best practice of being a great social media manager. It's not the role of HubSpot's product to teach you how to be a great marketer. But that's precisely what HootSuite and HubSpot's academies do, and why those certifications actually mean something.

You can even see in Optimizely first iteration of their academy, they have wisely split into separate learning tracks: strategy, configuration, product, and results. https://learn.optimizely.com/hc/en-us/sections/200386128-Beg...

Finally even on the product side, that doesn't make sense one you get to high value products. On one end of the spectrum are consumer apps – and I agree with you there. E.g. If you needed an academy to learn how to use the Twitter iphone app, something is seriously wrong. On the other end of the spectrum are products bought by businesses for $100k+. These products generate a ton of value, when implemented properly, and touch many processes – they are complex and they need to be. As an executive, if I'm buying software for $250K you will be absolutely sure I'm paying $5K for all my admins to get certified on the product ASAP. You can't wait 6 months for them to learn it when there's thousands of dollars of value on the line every day tied to their efficiency. B2B products lie at different points along this spectrum and academies generate great ROI at almost every level when done properly.

Car analogy time:

This isn't GM saying "we have an academy so you can learn to drive a GM" - this is GM saying "we offer driving lessons". Not everybody is born knowing how to drive like a boss.

I hear sentiments like this a lot, but I don't agree. Many companies have training on how to use Google Calendar/Drive properly. That doesn't mean these products are too complicated.

>Many companies have training on how to use Google Calendar/Drive properly

Is this really a thing? I would be embarrassed if I couldn't figure out a site as simple as Google Calendar/Drive. How should I take advantage of this?

Disagree. Academies can also help customers get more out of your product, or use it better or more effectively.

I find at times like this, it's good to offer an alternative perspective as well, as public criticism of a product can be easy to take personally.

For what it's worth, I found the product incredibly intuitive and very easy to setup. (This was 2 years ago, and the product was young, so I don't know what new features, read complexity, have been introduced to the product.)

Yeah, I've had no trouble dipping my toes in, trying things out, and then expanding beyond the simpler stuff to more advanced things. In a couple months my company has achieved some awesome results already, and we've been able to put together little fun promos like an Easter Egg Hunt that would have been much more complicated to do without (outsourced) back end support if we didn't have a tool like Optimizely to let us play around with our website.

The simpler alternative is VWO: http://visualwebsiteoptimizer.com/

Sadly, I don't think paraschopra posts here on HN much anymore, though.

Actually, the newer VWO is much simpler and intuitive: https://vwo.com/early-access

Note: I'm one of the engineers who works on VWO

How much did you guys pay for the 3 letter domain :)

We paid $30k

New VWO UI looks amazing. Great work on it.

What an amazing username/domain.

I do (sometimes).

Google Website Optimizer is still around. It's called Content Experiments now and it's under Google Analytics.

Optimizely is cool because it lets you run arbitrary javascript for tests without modifying the code base in most cases.

No multivariate testing and have to use 2 different pages now. Really miss GWO, very easy multi-variate testing, simple, FREE. Not real time was the only downside.

You can have Experiments running on client side (JS) without redirect to different versions of the page, but setup process is... strange to say the least - https://developers.google.com/analytics/solutions/experiment...

Still, after setting it up it worked just fine for us.

Also, I have yet to figure out how to run multiple experiments on the same page (rather than multivariate single experiments). Usually frowned on, but when you're testing a new search box site-wide and different CTA buttons, they're discrete tests.

The initial learning curve is a bit steep -- but once you are over it, usage is quite smooth. If you are using a lot of AJAX and dynamic stuff it's another story.

Haven't tried Optimizely, but visual website optimizer is very easy to use.

Hey Pud - we were avid GWO users and became converts to Optimizely and/or Visual Website Optimizer where more appropriate.

Happy to walk you through a 5-min real-world tutorial.

Drop me a line @ profile email if you like.