XMPP is simply a publisher/subscriber architecture. So you can just have a "global" publisher that any user subscribes to when they get on the service (registered user or not) for public viewing. The protocol isn't limited to just private chat.

So then XMPP doesn't solve this problem as you need a service which saves and publishes the message for public viewing.

I'm in agreement that I wouldn't necessarily choose XMPP for Twitter but under different reasons. There is literally nothing holding back the XMPP protocol from saving and publishing messages to the public viewing. XMPP is the eXtensible Messaging and Presence Protocol and suppose to be extremely modular so I really don't think what you're describing is impossible.