Let's start with the good stuff: it's very, very engaging, and if I tell an applicant to go through codecademy, it's pretty likely that they'll get it done. In other words, they're very good at conversion optimization. They give the user very straightforward, bite-sized goals, and people end up doing a lot of work.
However, people that pass through this system end up with a very consistent and warped skillset. It's not because they're still newbs -- I run into lots of newbs that didn't go through codecademy, and these guys are unusual.
1) They're very familiar with syntax, and can fluently type code for common tasks, but have difficulty solving problems with these tools. For instance, it's very common that I find a codecademy graduate that can easily iterate through an array and initialize/modify variables, but not sum the elements in an array.
2) They don't model the program's execution in their head. This is a really profound and strange fact, with lots of weird corollaries. For instance, they don't have ways of dealing with errors when syntax-based guess-and-check fails. The notion of code executing line-by-line is surprising to them. This is a pernicious problem that requires unlearning.
It's very easy to trace these unusual skill profiles back to the mechanism that drives codecademy's success in the first place. I don't know what I would do in their place, but I have a few ideas of things that I'd like to see in the space:
--A curated project-based curriculum that takes followers along a similarly-well thought-out path.
--A wconsole-based codecademy that makes heavy use of the debugger.
I just did the Evernote API lesson now, and I made a note, and that's cool and all. But it taught me essentially zero about ruby or APIs or Evernote's API, and if I were to try to estimate a programming task as "Well, I used the api to make a note in about 10 minutes so obviously this other thing that sounds similar must be simple too" I would end up doing more harm than good.
I really feel like opportunities to take a page-long tangent explaining concepts at every step should have been in the lesson. Even just links to wikipedia articles on programming concepts, something to provide the vocabulary - I think it would be hard for me to really learn more fundamentals based on this lesson because I wouldn't even know what terms to google.
It gives a nicely presented introductory content, but these days the stuff you learn off CodeAcademy is the equivalent of an office worker mastering Microsoft Office VB scripting.
Which is to say, it's an absolute necessity to stay competitive considering the majority of modern office work, and in turn people, are replaceable by an Excel spreadsheet script.
This is the case especially when you go to somewhere like Google where outside the engineering roles it's just a bunch of highly accredited people being overpaid and overstimulated creating and babysitting automated processes. That culture will slowly creep over to every other industry at the inverse rate of a that industry's keeping around a fax machine.
The gameification is only good for entertainment purposes, considering every credible higher education institution don't recognize online classes as credible crediting.
So how can CodeAcademy get beyond being a fad?
Distribution and then vertical integration. They probably already know that. Reach as many kids as soon and as early as possible then start going deeper with the content, the more highly specific the knowledge the more you can charge for it.
They're really in the disruption of tech publishers, like O'Reilly or Safari or Apress. When they have a full library charging for a monthly subscription will keep them going for quite awhile.
In my case, as a non-technical co-founder, Codecademy lessons help me gain a minimum practical understanding of the Web in a more time-efficient way, ultimately allowing me to better communicate with my more technical co-founders. I just wonder if Codecademy will ever be able to become more than that.