People actually do other actions on their phone after unlocking it.

If somebody swipes on their homescreen, browse the web, etc, the trail would not be just the unlock pattern.

The exploit you're talking about may work if you get hold of the phone right after the user unlocks it since the trail only has the pattern.

True enough, there is other "noise" on the phone, in the form of point-like finger prints, and even other trails. But you're imagining a blank phone, where you have to try and discern one trail from another. Now turn the phone on, and the unlock background appears. Which trails intersect all of the dots of the unlock background?

It's much easier that you imagine. I've been using my phone as I normally do throughout the day, and I can see the unlock pattern clearly on the phone.

That's why you need a unlock pattern that intersects / overlaps itself. (Not foolproof, just makes that attack harder.)

It also doesn't work nearly as well if you are OCD about wiping your screen often, not that that makes the pattern lock any better but it isn't quite that trivial to defeat

Doesn't the trail get cleaned off when you put it in your pocket?

No, it takes pretty vigorous scrubbing, at least on mine.