In general, muggers are small fry criminals. You want to carry on you enough cash for them to be happy, call it a day and go to spend it on meth/booze - e.g. without hurting you - but not enough to give them funny ideas and attempt an amateur kidnap on you.

I've been told the sweet spot is in the ballpark of $20 - $50 USD. But of course this has to be adjusted by location.