We use Julia in our group and I really like it: the language design is pretty neat and it offers a very good performance. I don't know of any libraries targeted specifically at agent-based modelling, though. (But in the end, as an experienced programmer you don't really need that - plain object-orientation is enough.)

I would argue that for certain kinds of problems, agent-based modeling is easier to think about and work with than object-oriented programming. It's not a matter of your programming experience, but rather what you're trying to model and how.

I've heard great things about Julia, but haven't ever used it. Yet another thing that's on my list...