Right, it would seem that they should only be moving away in one direction, and eventually we'd even see a difference between the forward and backward direction. However, I've heard people claim that eventually, all directions go toward the singularity?

That's once you're inside the black hole, and not entirely true; "past" points outwards. I'll admit it's close enough to true.

Which is not to that's how it would look. Photons are also falling, so you'd still be able to see the external universe even if the apparent direction it's in isn't traversable. (The photons are coming from the past. See above...)

>"past" points outwards.

I don't think that grasps space-time as one thing and why black holes are so mental. The event horizon is where the past no longer points outwards, that path is broken. There's no direction in space or time with which to point your space engine to escape. No time you could travel to were that is restored.