IRC has major issues, actually - notably, it's very tricky to parse correctly (and very easy to think you're parsing it correctly when you aren't), and doesn't support more than one space-containing value per 'line'. It's also implemented differently by every single server.
You will need some kind of serialization format for a reliable message protocol. JSON is just the simplest possible format that can reasonably represent common data, and works in 95% of the usecases.
I'm fond of Avro. It has both a compact binary format and a human-readable plaintext format that's nearly identical to JSON (the only difference being that order of fields is significant in Avro but insignificant in JSON), with tools to convert between them.
I personally found BSON* (or maybe protobuf) a better format for a wire protocol. I really miss support for a date type and binary data in JSON (for my own applications).
msgpack[1] or LEON[2] might be more useful alternatives, in that case, depending on your interoperability requirements.
[1] http://msgpack.org/ [2] https://github.com/raypulver/LEON
IRC and MSNP (the early versions, before they started XML-ifying everything) are what I'd consider a good format protocol for the application - a mostly line-based, textual one you can use pretty easily without any client more complex than netcat. They are also extensible enough even without XML.
(Personally, I think it is absurd that a protocol needs to wrap a single text message, the most common form of message that would be encountered, in several times more bytes than the message itself.)