Mathematica still has some parts that are nice though. Especially interactive plots (which you can do in IPython but not as simply) and some of the neat metaprogramming stuff. A big thing is that Mathematica (the language) is homoiconic, which I think works really well for their interface.

But I prefer IPython as well. Mathematica is awesome, but Wolfram is pretty heavy handed with the proprietary stuff.

Edit: one thing that I'd really like to have is some of Mathematica's tools for managing scope, like Module and similar. You can kinda do this, but it's clunky. And on top of that, some way to limit declarations to a section (group of cells). It's kinda awkward to have multiple separate sections in an IPython notebook because your declarations start to overlap. Unless someone here knows how already?