(If anything, that should probably be "errors", quotes included, because what people usually mean by the term is "easily-forseeable occurrence that I couldn't be bothered to write the code for".)
Exceptions are ideal for things that require aborting and unwinding to a point - usually a loop, like a server request handler or UI event dispatcher - high up on the stack. If the general behaviour is not to abort and unwind, it's not a good fit for an exception.
It's also why exception handlers should be very rare. There shouldn't be any more than a half-dozen in most programs. Error codes that need checking and aborting by hand at every function call boundary are a poor fit for replacing exceptions, because they optimize for the case where the error can be handled, rather than the usual case for exceptions, where the error cannot be handled.
From what I understand, the Go language team wanted to write robust servers, thus they wanted to "optimize for the case where the error can be handled."
But were programmers to consult the history of their compiler's output (representative summary: "ERROR ... ERROR ... FAILED ... WARNING ... ERROR"), it would be obvious that in fact errors are highly likely, even if you know what you're doing. So what makes the error case different from all the other cases?
You're quite right that exceptions promote giving up control to a higher level. And that's the right thing to do for exceptional situations. The ultimate higher level is the programmer who's going to have to fix the bug, or the admin who needs to fix the configuration problem - because they are the two main classes of errors for which exceptions are great. (The third class is a logical abort, and depending on the language and how good its idioms support exceptions, this may or may not be good practice.)
You get rich information about the error, including a call stack, the ongoing operation gets cancelled as soon as it occurs rather than muddling onwards and causing more damage. If you can foresee the error such that you can write an error handler, using the method that throws exceptions is probably not the right one - unless it's unavoidable owing to race conditions, or impoverished libraries.
It's very similar to exceptions, except you have to consciously choose to do it. That's a virtue to me.
It is very different. With a try/catch block, you can see the normal logic flow of your code in one place, separated from the error handling logic. If you wrap each and every line that can have an exception with it's own try/catch block, then yes, that is pretty much the same as handling explicit error codes. Which is why I consider that poor style in languages that have exceptions.
Separating error handling code apart from the logic flow of normal operation is simply not possible with error codes as function return values.