I think + for append is fine, it certainly seems to make sense to me that you get "helloworld" from "hello"+"world".
I think the confusing thing is the type coercion. Does it make sense that "2"+3 is "23" but 2+"3" is not 5?
(3*4).toString() + "=" + 3.toString() + 4.toString())
would not be as surprising that the two sides are different.
And let's be honest, you definitely don't want to 'fix' this by tweaking precedence to include types. That way madness lies. ;)
Definite shout to ubernostrum's commment though - perhaps the confusing thing is that string concatenation is +, but a world of explicit Appends seems horrid, and that ship has very much sailed.
Just be (maybe) glad java doesn't allow generalised operator overloading ;)