Some walls have vicious down currents, but if you just swim away from the wall a few meters the current will decline and you can establish positive buoyancy.

You're right, but I wasn't willing to take that risk.

One of the divers in my group was already in a mild state of panic and I didn't want to chance them letting go of the wall and descending another 10m.

I had a plan to deal with the current, but oxygen toxicity is another thing entirely.

Oxygen toxicity isn't really a factor at that depth; you'd run out of air on a single tank before experiencing any toxicity symptoms. The immediate concerns would be narcosis and hypercapnia.